Swift on Android: Building toolchain

One day in December 2018 I decided to repeat steps written in Android.md readme-file from official Apple Swift Git repository. Final goal was to experiment with Swift programming language on Android device I have.

At that point of time Swift runtime on Linux was able to utilise Dispatch and Foundation frameworks. There were also various examples (like one from Readdle) how to build Swift toolchain (including Dispatch and Foundation frameworks) for Android.

After first attempts of building Swift compiler, Dispatch and Foundation frameworks it became clear that official Android.md readme-file is not up to date. Other examples were either outdated (outdated NDK version or Swift version) or literally wasn’t working due execution errors in Shell scripts (like missed dependency or file).

That’s why I decided to make Automated workflow to build Swift toolchain which is easy to maintain and keep up to date. To avoid Shell programming I decided to use Ruby programming language for general purpose scripting.

After about three weeks of research the initial version of Automated workflow was made. It called “Swift-Everywhere”. It can build Swift version capable to generate Arm and Intel 32/64 bit binaries.

Using Pre-built Swift toolchain

Since Release 1.0.14 it is possible to download and use pre-build toolchain. It supports aarch64, arm, x86 and x86_64 architectures.

Since Swift toolchain depends of Android NDK it needs to be configured before first usage. How to do it described in Readme.md file included into downloadable archive. In short — you need to make symbolic link which points to Android NDK. Below you can find file structure of the Swift toolchain:

$TOOLCHAIN/
          /ndk                (Symbolic link to Android NDK)
          /bin                (Swift compiler)
          /lib/swift/android  (Pre-built Shared Objects)
          /share              (Licenses, etc.)
          /VERSION
          /Readme.md

Inside bin directory there are several versions of swift compiler. Also there are several versions of helper copy-libs scripts which can be used in order to copy shared objects to desired location (i.e. jniLibs folder). As you can see below the difference only in target tripple suffix which defines for which target platform swift should generate the code.

$TOOLCHAIN/bin/
              /copy-libs-aarch64-linux-android
              /copy-libs-arm-linux-androideabi
              /copy-libs-i686-linux-android
              /copy-libs-x86_64-linux-android
              /swiftc-aarch64-linux-android
              /swiftc-arm-linux-androideabi
              /swiftc-i686-linux-android
              /swiftc-x86_64-linux-android

To compile Swift source code and deploy to Android Device do following:

1. Compile:

$TOOLCHAIN/bin/swiftc-i686-linux-android -emit-executable -o [destination]/hello main.swift

1. Copy pre-built dependencies:

$TOOLCHAIN/bin/copy-libs-i686-linux-android [destination]

1. Copy dependencies and executable to Android Device (or Simulator):

adb push [destination]/hello, [destination]/libswiftGlibc.so, …

1. Run executable on Android Device:

adb shell LD_LIBRARY_PATH=/data/local/tmp /data/local/tmp/hello

See Samples projects for details.

Building Swift toolchain (including Dispatch and Foundation frameworks) for Android

The process is automated. All you need to do is to launch command make. It will show available build targets similar to shown below:

$ make

To Build Toolchain with One-Action:

$ make bootstrap

To Build Toolchain Step-by-Step:

1. Checkout sources:
   $ make checkout

2. Build toolchain:
   $ make build

3. Install toolchain:
   $ make install

4. Archive toolchain:
   $ make archive

Building certain component (i.e. llvm, icu, xml, ssl, curl, swift, dispatch, foundation):

To build only certain component:
   $ make build:llvm

To clean only certain component:
   $ make clean:llvm

At the moment in order to build Swift toolchain (including Dispatch and Foundation frameworks) and sample projects you need to do following steps:

1. Get sources of components such as LLVM, Swift, ICU, libXML, CURL, etc.
2. Install Android Studio from Google website and enable NDK Tool.
3. Build LLVM Compiler Infrastructure.
4. Build Dependencies such as ICU, libXML, CURL, etc.
5. Build Swift Compiler Infrastructure.
6. Build Dispatch and Foundation frameworks.

All steps above automated. Once build is done you can Build sample projects included into separate repository and deploy them to Android device. Detailed steps described in Readme.md files included into repository with sample projects.

How Automated workflow works

As you see workflow for building Swift toolchain is automated. But you may interesting what inside and how it works.

Dependencies and Build steps

At first we can build Foundation dependencies such as ICU, libXML, libCURL, OpenSSL. Then we can build Swift compiler, Dispatch and Foundation. Swift compiler build will also trigger LLVM and CMark builds.

Here is a structure of dependencies:

• Swift compiler → LLVM, CMark.
• LLVM → Compiler-RT, Clang.
• Dispatch → LLVM, Swift compiler.
• Foundation → LLVM, Swift compiler, Dispatch, ICU, libXML, libCURL
• libCURL → OpenSSL

Patches and Fixes

Not everything “Out of the Box” yet. As you can see Git repository contains patches for several components used during build process. Swift development is in active phase and often you can find code like below:

// File: FileManager.swift
func mountPoints(_ statBufs: UnsafePointer<statfs>,
                 _ fsCount: Int) -> [URL] {
   ...
   #error("Requires a platform-specific implementation")

   ...
}

Surprise! Type FileManager still has non implemented function mountPoints for Android. Of cause later someone (maybe you) will implement function body. But without applying patch Foundation framework will not be compilable.

ICU Patches

Patches needed to avoid conflicts and unexpected behaviour of Swift runtime. ICU library needs to be altered with “swift” suffix. So, that ICU library name will be libicui18nswift.so instead of libicui18n.so which is shipped with Android.

Dispatch Patches

First patch contains CMAKE_SYSTEM_PROCESSOR fix addressed wrong architecture determination. (TODO) This can be easily fixed and merged to Swift codebase.

Second patch removes -Werror compiler flag. Without this patch compiler will fail with error due warning. (TODO) Source code needs modification to avoid warnings. After updating sources this patch can be removed.

Foundation Patches

First patch contains CMAKE_SYSTEM_PROCESSOR fix addressed wrong architecture determination. (TODO) This can be easily fixed and merged to Swift codebase.

Second series of patches addressed compiler defines similar to DEPLOYMENT_TARGET_ANDROID, ICU_LIBRARY, os(Android) or compile options like -fcf-runtime-abi=swift.

FileManager.swift file contain most of the patches. Certain functions not yet implemented. So, it is a pure workaround in order to get build Foundation. (TODO) Appropriate bug submitted to Swift developers.

Fix for missed __CFConstantStringClassReference symbol is most interesting. Without it you can build Swift toolchain, but demo project will fail to execute upon dynamic linking due missed symbol.

-Xlinker --defsym -Xlinker '__CFConstantStringClassReference=$$s10Foundation19_NSCFConstantStringCN'

Note that Swift symbol in example below is $s10Foundation19_NSCFConstantStringCN. But we need to use double dollar symbol $$ to escape.

Future plans

Primary goal is to keep up to date Automated workflow. Actually it is easy to do. It requires changing SHA commits in checkout configuration, rebuilding Swift toolchain and releasing new version of Automated workflow on GitHub.

Next goal is to fix shell scripts from Swift repository. So that after following steps from Android.md readme-file you should be able to get Swift toolchain for Android. This require to promote needed changes to Swift sources and get rid of patches. After that updating shell scripts and Readme files from Swift sources will be a trivial task.

Ideally we need to achieve automated builds for Swift toolchain for Android on official Swift CI system.

If you can help with patches, then go ahead and fork Swift Git repository, implement changes and make pull request.

Thank you!

Updates

You can find Releases, Discussions and Issues in GitHub Repository.

Featured image Photo by Victor Benard on Unsplash