Building Extension Modules¶
Setuptools can build C/C++ extension modules. The keyword argument
ext_modules
of setup()
should be a list of instances of the
setuptools.Extension
class.
For example, let’s consider a simple project with only one extension module:
<project_folder>
├── pyproject.toml
└── foo.c
and all project metadata configuration in the pyproject.toml
file:
# pyproject.toml
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "mylib-foo" # as it would appear on PyPI
version = "0.42"
To instruct setuptools to compile the foo.c
file into the extension module
mylib.foo
, we need to add a setup.py
file similar to the following:
from setuptools import Extension, setup
setup(
ext_modules=[
Extension(
name="mylib.foo", # as it would be imported
# may include packages/namespaces separated by `.`
sources=["foo.c"], # all sources are compiled into a single binary file
),
]
)
See also
You can find more information on the Python docs about C/C++ extensions. Alternatively, you might also be interested in learn about Cython.
If you plan to distribute a package that uses extensions across multiple platforms, cibuildwheel can also be helpful.
Important
All files used to compile your extension need to be available on the system when building the package, so please make sure to include some documentation on how developers interested in building your package from source can obtain operating system level dependencies (e.g. compilers and external binary libraries/artifacts).
You will also need to make sure that all auxiliary files that are contained inside your project (e.g. C headers authored by you or your team) are configured to be included in your sdist. Please have a look on our section on Controlling files in the distribution.
Compiler and linker options¶
The command build_ext
builds C/C++ extension modules. It creates
a command line for running the compiler and linker by combining
compiler and linker options from various sources:
the
sysconfig
variablesCC
,CXX
,CCSHARED
,LDSHARED
, andCFLAGS
,the environment variables
CC
,CPP
,CXX
,LDSHARED
andCFLAGS
,CPPFLAGS
,LDFLAGS
,the
Extension
attributesinclude_dirs
,library_dirs
,extra_compile_args
,extra_link_args
,runtime_library_dirs
.
Specifically, if the environment variables CC
, CPP
, CXX
, and LDSHARED
are set, they will be used instead of the sysconfig
variables of the same names.
The compiler options appear in the command line in the following order:
first, the options provided by the
sysconfig
variableCFLAGS
,then, the options provided by the environment variables
CFLAGS
andCPPFLAGS
,then, the options provided by the
sysconfig
variableCCSHARED
,then, a
-I
option for each element ofExtension.include_dirs
,finally, the options provided by
Extension.extra_compile_args
.
The linker options appear in the command line in the following order:
first, the options provided by environment variables and
sysconfig
variables,then, a
-L
option for each element ofExtension.library_dirs
,then, a linker-specific option like
-Wl,-rpath
for each element ofExtension.runtime_library_dirs
,finally, the options provided by
Extension.extra_link_args
.
The resulting command line is then processed by the compiler and linker.
According to the GCC manual sections on directory options and
environment variables, the C/C++ compiler searches for files named in
#include <file>
directives in the following order:
first, in directories given by
-I
options (in left-to-right order),then, in directories given by the environment variable
CPATH
(in left-to-right order),then, in directories given by
-isystem
options (in left-to-right order),then, in directories given by the environment variable
C_INCLUDE_PATH
(for C) andCPLUS_INCLUDE_PATH
(for C++),then, in standard system directories,
finally, in directories given by
-idirafter
options (in left-to-right order).
The linker searches for libraries in the following order:
first, in directories given by
-L
options (in left-to-right order),then, in directories given by the environment variable
LIBRARY_PATH
(in left-to-right order).
Distributing Extensions compiled with Cython¶
When your Cython extension modules are declared using the
setuptools.Extension
class, setuptools
will detect at build time
whether Cython is installed or not.
If Cython is present, then setuptools
will use it to build the .pyx
files.
Otherwise, setuptools
will try to find and compile the equivalent .c
files
(instead of .pyx
). These files can be generated using the
cython command line tool.
You can ensure that Cython is always automatically installed into the build
environment by including it as a build dependency in
your pyproject.toml
:
[build-system]
requires = [..., "cython"]
Alternatively, you can include the .c
code that is pre-compiled by Cython
into your source distribution, alongside the original .pyx
files (this
might save a few seconds when building from an sdist
).
To improve version compatibility, you probably also want to include current
.c
files in your revision control system, and rebuild them whenever
you check changes in for the .pyx
source files.
This will ensure that people tracking your project will be able to build it
without installing Cython, and that there will be no variation due to small
differences in the generate C files.
Please checkout our docs on Controlling files in the distribution for
more information.