New Functions

I added 14 functions to the limited C API:

• PyDict_GetItemRef()
• PyDict_GetItemStringRef()
• PyImport_AddModuleRef()
• PyLong_AsInt()
• PyMem_RawCalloc()
• PyMem_RawFree()
• PyMem_RawMalloc()
• PyMem_RawRealloc()
• PySys_Audit()
• PySys_AuditTuple()
• PyType_GetFullyQualifiedName()
• PyType_GetModuleName()
• PyWeakref_GetRef()
• Py_IsFinalizing()

It makes code using these functions compatible with the limited C API.

Py_TRACE_REFS

I modified the special debug build Py_TRACE_REFS. Instead of adding two members to PyObject to create a double linked list of all objects, I added an hash table to track all objects.

Since the PyObject structure is no longer modified, this special debug build is now ABI compatible with the release build! Moreover, it also becomes compatible with the limited C API!

Argument Clinic

I modified Argument Clinic (AC) to generate C code compatible with the limited C API.

First, I moved private functions used by Argument Clinic to the internal C API and modified Argument Clinic to generate #include to get these functions. Then I modified Argument Clinic to use only the limited C API and to not generate these #include.

At the beginning, only some converteres were supported and only the slower METH_VARARGS calling convention was supported.

Now, more and more converters and formats are supported, and the regular efficient METH_FASTCALL calling convention is used.

/*[clinic input]
grp.getgrgid

    id: object

Return the group database entry for the given numeric group ID.

args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf);
if (!args) {
    goto exit;
}
id = args[0];
return_value = grp_getgrgid_impl(module, id);

if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:getgrgid", _keywords,
    &id))
    goto exit;
return_value = grp_getgrgid_impl(module, id);

PEP 737: Format Type Name

One issue that I had with Argument Clinic was to format an error message with the limited C API. I cannot use the private _PyArg_BadArgument() function, nor access to PyTypeObject.tp_name (opaque structure in the limited C API) to format a type name. While the limited C API provides PyType_GetName() and PyType_GetQualName(), it's still different than how Python formats type names in error messages.

I proposed different APIs but there was no agreement. So I decided to write PEP 737 "C API to format a type fully qualified name".

After four months of discussions, the Steering Council decided to accept it in Python 3.13.

Changes:

• Add PyType_GetFullyQualifiedName() function.
• Add PyType_GetModuleName() function.
• Add %T, %#T, %N and %#N formats to PyUnicode_FromFormat().

I also proposed adding a new type.__fully_qualified_name__ attribute, and a few methods to format a the fully qualified name of type in Python. But the Steering Council was not convinced and asked me to remove these Python changes until someone comes with a strong use case for this attribute and methods.

In 2018, I made a first attempt: I made a similar change, but I had to revert it. I created a discussion on the python-dev mailing list, but we failed to reach a consensus.

In 2011, I already asked to stop the cargo cult of truncating type names, but I didn't implement my idea by proactively stop truncating type names.

_PyArg_BadArgument("getpwnam", "argument", "str", arg);

PyErr_Format(PyExc_TypeError,
             "getpwnam() argument must be str, not %T",
             arg);

Add _testlimitedcapi extension

In Python 3.12, C API tests are splitted in two categories:

• _testcapi: public C API
• _testinternalcapi: internal C API (Py_BUILD_CORE)

I added a third _testlimitedcapi extension to test the limited C API (Py_LIMITED_API). I moved tests using the limited C C API from _testcapi to _testlimitedcapi.

The difference between _testcapi and _testlimitedcapi is that the _testlimitedcapi extension is built with the Py_LIMITED_API macro defined, and so can only access the internal C API.

Convert stdlib extensions to the limited C API

At August 2023, I proposed to: Use the limited C API for some of our stdlib C extensions.

In March 2024, there are now 16 C extensions built with the limited C API:

• _ctypes_test
• _multiprocessing.posixshmem
• _scproxy
• _stat
• _statistics
• _testimportmultiple
• _testlimitedcapi
• _uuid
• errno
• fcntl
• grp
• md5
• pwd
• resource
• termios
• winsound

Other stdlib C extensions use the internal C API for various reasons or are using functions which are missing in the limited C API. Remaining issues should be analyzed on a case by case basis.

This work shows that non-trivial C extensions can be written using only the limited C API version 3.13.

What's Next?