Authors: | Russell Keith-Magee (2023-11) |
---|
This document provides a quick overview of some iOS specific features in the Python distribution.
These instructions are only needed if you're planning to compile Python for iOS yourself. Most users should not need to do this. If you're looking to experiment with writing an iOS app in Python, tools such as BeeWare's Briefcase and Kivy's Buildozer will provide a much more approachable user experience.
Building for iOS requires the use of Apple's Xcode tooling. It is strongly recommended that you use the most recent stable release of Xcode. This will require the use of the most (or second-most) recently released macOS version, as Apple does not maintain Xcode for older macOS versions. The Xcode Command Line Tools are not sufficient for iOS development; you need a full Xcode install.
If you want to run your code on the iOS simulator, you'll also need to install an iOS Simulator Platform. You should be prompted to select an iOS Simulator Platform when you first run Xcode. Alternatively, you can add an iOS Simulator Platform by selecting an open the Platforms tab of the Xcode Settings panel.
--enable-framework[=DIR]
This argument specifies the location where the Python.framework will be installed. If
DIR
is not specified, the framework will be installed into a subdirectory of theiOS/Frameworks
folder.This argument must be provided when configuring iOS builds. iOS does not support non-framework builds.
--with-framework-name=NAME
Specify the name for the Python framework; defaults to
Python
.Use this option with care!
Unless you know what you're doing, changing the name of the Python framework on iOS is not advised. If you use this option, you won't be able to run the
make testios
target without making significant manual alterations, and you won't be able to use any binary packages unless you compile them yourself using your own framework name.
iOS apps can be deployed on physical devices, and on the iOS simulator. Although
the API used on these devices is identical, the ABI is different - you need to
link against different libraries for an iOS device build (iphoneos
) or an
iOS simulator build (iphonesimulator
).
Apple uses the XCframework
format to allow specifying a single dependency
that supports multiple ABIs. An XCframework
is a wrapper around multiple
ABI-specific frameworks that share a common API.
iOS can also support different CPU architectures within each ABI. At present, there is only a single supported architecture on physical devices - ARM64. However, the simulator supports 2 architectures - ARM64 (for running on Apple Silicon machines), and x86_64 (for running on older Intel-based machines).
To support multiple CPU architectures on a single platform, Apple uses a "fat binary" format - a single physical file that contains support for multiple architectures. It is possible to compile and use a "thin" single architecture version of a binary for testing purposes; however, the "thin" binary will not be portable to machines using other architectures.
The Python build system will create a Python.framework
that supports a
single ABI with a single architecture. Unlike macOS, iOS does not allow a
framework to contain non-library content, so the iOS build will produce a
bin
and lib
folder in the same output folder as Python.framework
.
The lib
folder will be needed at runtime to support the Python library.
If you want to use Python in a real iOS project, you need to produce multiple
Python.framework
builds, one for each ABI and architecture. iOS builds of
Python must be constructed as framework builds. To support this, you must
provide the --enable-framework
flag when configuring the build. The build
also requires the use of cross-compilation. The minimal commands for building
Python for the ARM64 iOS simulator will look something like:
$ export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" $ ./configure \ --enable-framework \ --host=arm64-apple-ios-simulator \ --build=arm64-apple-darwin \ --with-build-python=/path/to/python.exe $ make $ make install
In this invocation:
iOS/Resources/bin
has been added to the path, providing some shims for the compilers and linkers needed by the build. Xcode requires the use ofxcrun
to invoke compiler tooling. However, ifxcrun
is pre-evaluated and the result passed toconfigure
, these results can embed user- and version-specific paths into the sysconfig data, which limits the portability of the compiled Python. Alternatively, ifxcrun
is used as the compiler, it requires that compiler variables likeCC
include spaces, which can cause significant problems with many C configuration systems which assume thatCC
will be a single executable.To work around this problem, the
iOS/Resources/bin
folder contains some wrapper scripts that present as simple compilers and linkers, but wrap underlying calls toxcrun
. This allows configure to use aCC
definition without spaces, and without user- or version-specific paths, while retaining the ability to adapt to the local Xcode install. These scripts are included in thebin
directory of an iOS install.These scripts will, by default, use the currently active Xcode installation. If you want to use a different Xcode installation, you can use
xcode-select
to set a new default Xcode globally, or you can use theDEVELOPER_DIR
environment variable to specify an Xcode install. The scripts will use the defaultiphoneos
/iphonesimulator
SDK version for the select Xcode install; if you want to use a different SDK, you can set theIOS_SDK_VERSION
environment variable. (e.g, settingIOS_SDK_VERSION=17.1
would cause the scripts to use theiphoneos17.1
andiphonesimulator17.1
SDKs, regardless of the Xcode default.)The path has also been cleared of any user customizations. A common source of bugs is for tools like Homebrew to accidentally leak macOS binaries into an iOS build. Resetting the path to a known "bare bones" value is the easiest way to avoid these problems.
--host
is the architecture and ABI that you want to build, in GNU compiler triple format. This will be one of:arm64-apple-ios
for ARM64 iOS devices.arm64-apple-ios-simulator
for the iOS simulator running on Apple Silicon devices.x86_64-apple-ios-simulator
for the iOS simulator running on Intel devices.
--build
is the GNU compiler triple for the machine that will be running the compiler. This is one of:arm64-apple-darwin
for Apple Silicon devices.x86_64-apple-darwin
for Intel devices.
/path/to/python.exe
is the path to a Python binary on the machine that will be running the compiler. This is needed because the Python compilation process involves running some Python code. On a normal desktop build of Python, you can compile a python interpreter and then use that interpreter to run Python code. However, the binaries produced for iOS won't run on macOS, so you need to provide an external Python interpreter. This interpreter must be the same version as the Python that is being compiled. To be completely safe, this should be the exact same commit hash. However, the longer a Python release has been stable, the more likely it is that this constraint can be relaxed - the same micro version will often be sufficient.The
install
target for iOS builds is slightly different to other platforms. On most platforms,make install
will install the build into the final runtime location. This won't be the case for iOS, as the final runtime location will be on a physical device.However, you still need to run the
install
target for iOS builds, as it performs some final framework assembly steps. The location specified with--enable-framework
will be the location wheremake install
will assemble the complete iOS framework. This completed framework can then be copied and relocated as required.
For a full CPython build, you also need to specify the paths to iOS builds of
the binary libraries that CPython depends on (XZ, BZip2, LibFFI and OpenSSL).
This can be done by defining the LIBLZMA_CFLAGS
, LIBLZMA_LIBS
,
BZIP2_CFLAGS
, BZIP2_LIBS
, LIBFFI_CFLAGS
, and LIBFFI_LIBS
environment variables, and the --with-openssl
configure option. Versions of
these libraries pre-compiled for iOS can be found in this repository. LibFFI is
especially important, as many parts of the standard library (including the
platform
, sysconfig
and webbrowser
modules) require the use of the
ctypes
module at runtime.
By default, Python will be compiled with an iOS deployment target (i.e., the
minimum supported iOS version) of 13.0. To specify a different deployment
target, provide the version number as part of the --host
argument - for
example, --host=arm64-apple-ios15.4-simulator
would compile an ARM64
simulator build with a deployment target of 15.4.
Once you've built a Python.framework
for each ABI and and architecture, you
must produce a "fat" framework for each ABI that contains all the architectures
for that ABI.
The iphoneos
build only needs to support a single architecture, so it can be
used without modification.
If you only want to support a single simulator architecture, (e.g., only support
ARM64 simulators), you can use a single architecture Python.framework
build.
However, if you want to create Python.xcframework
that supports all
architectures, you'll need to merge the iphonesimulator
builds for ARM64 and
x86_64 into a single "fat" framework.
The "fat" framework can be constructed by performing a directory merge of the
content of the two "thin" Python.framework
directories, plus the bin
and
lib
folders for each thin framework. When performing this merge:
The pure Python standard library content is identical for each architecture, except for a handful of platform-specific files (such as the
sysconfig
module). Ensure that the "fat" framework has the union of all standard library files.Any binary files in the standard library, plus the main
libPython3.X.dylib
, can be merged using thelipo
tool, provide by Xcode:$ lipo -create -output module.dylib path/to/x86_64/module.dylib path/to/arm64/module.dylib
The header files will be identical on both architectures, except for
pyconfig.h
. Copy all the headers from one platform (say, arm64), renamepyconfig.h
topyconfig-arm64.h
, and copy thepyconfig.h
for the other architecture into the merged header folder aspyconfig-x86_64.h
. Then copy theiOS/Resources/pyconfig.h
file from the CPython sources into the merged headers folder. This will allow the two Python architectures to share a commonpyconfig.h
header file.
At this point, you should have 2 Python.framework folders - one for iphoneos
,
and one for iphonesimulator
that is a merge of x86+64 and ARM64 content.
Now that we have 2 (potentially fat) ABI-specific frameworks, we can merge those
frameworks into a single XCframework
.
The initial skeleton of an XCframework
is built using:
xcodebuild -create-xcframework -output Python.xcframework -framework path/to/iphoneos/Python.framework -framework path/to/iphonesimulator/Python.framework
Then, copy the bin
and lib
folders into the architecture-specific slices of
the XCframework:
cp path/to/iphoneos/bin Python.xcframework/ios-arm64 cp path/to/iphoneos/lib Python.xcframework/ios-arm64 cp path/to/iphonesimulator/bin Python.xcframework/ios-arm64_x86_64-simulator cp path/to/iphonesimulator/lib Python.xcframework/ios-arm64_x86_64-simulator
Note that the name of the architecture-specific slice for the simulator will depend on the CPU architecture(s) that you build.
You now have a Python.xcframework that can be used in a project.
The iOS/testbed
folder that contains an Xcode project that is able to run
the iOS test suite. This project converts the Python test suite into a single
test case in Xcode's XCTest framework. The single XCTest passes if the test
suite passes.
To run the test suite, configure a Python build for an iOS simulator (i.e.,
--host=arm64-apple-ios-simulator
or --host=x86_64-apple-ios-simulator
), specifying a framework build (i.e. --enable-framework
). Ensure that your
PATH
has been configured to include the iOS/Resources/bin
folder and
exclude any non-iOS tools, then run:
$ make all $ make install $ make testios
This will:
- Build an iOS framework for your chosen architecture;
- Finalize the single-platform framework;
- Make a clean copy of the testbed project;
- Install the Python iOS framework into the copy of the testbed project; and
- Run the test suite on an "iPhone SE (3rd generation)" simulator.
While the test suite is running, Xcode does not display any console output.
After showing some Xcode build commands, the console output will print Testing
started
, and then appear to stop. It will remain in this state until the test
suite completes. On a 2022 M1 MacBook Pro, the test suite takes approximately 12
minutes to run; a couple of extra minutes is required to boot and prepare the
iOS simulator.
On success, the test suite will exit and report successful completion of the test suite. No output of the Python test suite will be displayed.
On failure, the output of the Python test suite will be displayed. This will show the details of the tests that failed.
The easiest way to diagnose a single test failure is to open the testbed project in Xcode and run the tests from there using the "Product > Test" menu item.
To test in Xcode, you must ensure the testbed project has a copy of a compiled
framework. If you've configured your build with the default install location of
iOS/Frameworks
, you can copy from that location into the test project. To
test on an ARM64 simulator, run:
$ rm -rf iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator/* $ cp -r iOS/Frameworks/arm64-iphonesimulator/* iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator
To test on an x86-64 simulator, run:
$ rm -rf iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator/* $ cp -r iOS/Frameworks/x86_64-iphonesimulator/* iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator
To test on a physical device:
$ rm -rf iOS/testbed/Python.xcframework/ios-arm64/* $ cp -r iOS/Frameworks/arm64-iphoneos/* iOS/testbed/Python.xcframework/ios-arm64
Alternatively, you can configure your build to install directly into the testbed project. For a simulator, use:
--enable-framework=$(pwd)/iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator
For a physical device, use:
--enable-framework=$(pwd)/iOS/testbed/Python.xcframework/ios-arm64
To test on an iOS device, the app needs to be signed with known developer credentials. To obtain these credentials, you must have an iOS Developer account, and your Xcode install will need to be logged into your account (see the Accounts tab of the Preferences dialog).
Once the project is open, and you're signed into your Apple Developer account, select the root node of the project tree (labeled "iOSTestbed"), then the "Signing & Capabilities" tab in the details page. Select a development team (this will likely be your own name), and plug in a physical device to your macOS machine with a USB cable. You should then be able to select your physical device from the list of targets in the pulldown in the Xcode titlebar.
As the test suite is being executed on an iOS simulator, it is not possible to
pass in command line arguments to configure test suite operation. To work around
this limitation, the arguments that would normally be passed as command line
arguments are configured as a static string at the start of the XCTest method
- (void)testPython
in iOSTestbedTests.m
. To pass an argument to the test
suite, add a a string to the argv
definition. These arguments will be passed
to the test suite as if they had been passed to python -m test
at the
command line.
By default, Xcode will inserts an automatic breakpoint whenever a signal is raised. The Python test suite raises many of these signals as part of normal operation; unless you are trying to diagnose an issue with signals, the automatic breakpoints can be inconvenient. However, they can be disabled by creating a symbolic breakpoint that is triggered at the start of the test run.
Select "Debug > Breakpoints > Create Symbolic Breakpoint" from the Xcode menu, and populate the new brewpoint with the following details:
- Name: IgnoreSignals
- Symbol: UIApplicationMain
- Action: Add debugger commands for:
-
process handle SIGINT -n true -p true -s false
-process handle SIGUSR1 -n true -p true -s false
-process handle SIGUSR2 -n true -p true -s false
-process handle SIGXFSZ -n true -p true -s false
- Check the "Automatically continue after evaluating" box.
All other details can be left blank. When the process executes the
UIApplicationMain
entry point, the breakpoint will trigger, run the debugger
commands to disable the automatic breakpoints, and automatically resume.