Distributing a macOS software is not an easy task. It requires taking the binary, signing it, packaging it, signing the package and notarizing it. These tasks are simple when using a machine running macOS. However, when trying to build a CI/CD pipeline, you often come across a Linux OS based device (such as a Jenkins slave).
The problem is the tools needed for these tasks are only available on macOS. In this article we will show an automated workaround we developed to solve this problem.
For signing:
For notarization:
The task will be accomplished in two stages:
The first: Enrolling in the Apple Developer Program
To get the certificates, you’ll need an enrolled Apple Developer ID. You can enroll either as an individual or as an organization. Visit the following link to read more about this process: https://developer.apple.com/support/app-account/
The second: Downloading Developer Certificates
After your Apple ID has been enrolled, head over to: https://appstoreconnect.apple.com/get_started
Make sure you sign in with the Account Holder Account! Without it you won’t be able to download the certificates.
Once signed in, select View account:
This will lead you to the main Account dashboard. Select Program resources.
From here we will retrieve:
To get the certificates:
According to Apple:
The Apple Worldwide Developer Relations Intermediate Certificate has a version that’s set to expire on February 7, 2023 and a renewed version that’s set to expire on February 20, 2030. The renewed version is used to sign new software signing certificates issued after January 28, 2021. ”
To read more: https://developer.apple.com/support/expiration/
Now that we have the certificates, it’s time to get the notarization stuff.
Head back over to Program resources in the main Account dashboard and select Users and Access under App Store Connect (shown above).
Select the Keys tab and on the side menu Key Type select App Store Connect API.
Click the blue + button beside the Active title.
Enter a Name and Access for the key and click Generate
Now, copy the Issuer ID & KEY ID, download the API Key and save all three of them for later.
Last and final thing you’ll need to do in this step is convert the Mac Developer Installer certificate & Mac Developer Application certificate to a p12 format.
Follow this process twice for each certificate:
openssl genrsa -out distribution.key 2048openssl x509 -inform der -in distribution.cer -out distribution.pemopenssl x509 -in AppleWWDRCA.cer -inform DER -out AppleWWDRCA.pem -outform PEMIn our example, the package we're building consists of two files:
We will store it under a directory called pkgroot
root@ip-10-1-0-1:/home/ubuntu# ll pkgroot/
total 16
drwxr-xr-x 4 ubuntu ubuntu 4096 Nov 17 08:06 ./
drwxr-x--- 11 ubuntu ubuntu 4096 Nov 21 14:02 ../
drwxr-xr-x 3 ubuntu ubuntu 4096 Nov 16 08:22 Library/
drwxr-xr-x 3 ubuntu ubuntu 4096 Nov 16 08:22 opt/
We use several tools inthe process of building and signing the package, you can install them by running the following commands:
apple-codesign
wget <https://github.com/indygreg/apple-platform-rs/releases/download/apple-codesign/0.20.0/apple-codesign-0.20.0-x86_64-unknown-linux-musl.tar.gz> && \\
tar -xvf apple-codesign-0.20.0-x86_64-unknown-linux-musl.tar.gz -C /usr/local && \\
ln /usr/local/apple-codesign-0.20.0-x86_64-unknown-linux-musl/rcodesign /usr/local/bin
bomutils and cpio
apt-get install make g++ libxml2-dev cpio -y
git clone <https://github.com/hogliux/bomutils.git> && \\
cd bomutils/ && sed -i 's/data/dataa/g' src/lsbom.cpp && sed -i 's/data/dataa/g' src/mkbom.cpp && \\
make && make install
xar
wget <https://github.com/downloads/mackyle/xar/xar-1.6.1.tar.gz> && tar -xvzf xar-1.6.1.tar.gz && \\
cd xar-1.6.1/ && echo " deb [trusted=yes] <http://security.ubuntu.com/ubuntu/> bionic-security main" >> /etc/apt/sources.list && \\
apt-get update && apt-get install -y libbz2-dev libssl1.0-de && ./configure && make && make install
Next, we will need to sign the binary file before we can create the package.
Signing binaries / packages requires using both the Developer ID Application and the Apple Worldwide Developer Relations Certification Authority certificates.
Note: in our example, we have our certificate password protected
In our case, the first one is called cert.p12 and the second certroot.cer
However, the rcodesign tool requires the two to be merged into one certificate with two different certificate types, so, we will use the following commands to create a pem file and a der file
openssl pkcs12 -in cert.p12 -password pass:<PASSWORD> -certfile certroot.cer -nodes | openssl rsa -out key.pem
openssl pkcs12 -in cert.p12 -password pass:<PASSWORD> -certfile certroot.cer -nodes | openssl x509 -outform der -out cert.der
Now, we can sign the binary by running the command
rcodesign sign \\
--pem-source key.pem --der-source cert.der \\
--code-signature-flags runtime \\
./pkgroot/opt/coolprogram/binfileWe now have a signed binary file that can run on a mac without errors.
When using macOS, we can use apple tools to create the package, it creates several files describing the content of the package, installation instructions and other files. Unfortunately, we don’t have this luxury when using linux so we will need to build the package content ourselves.
A package consist of two files:
However, the base.pkg is also a package which consists of the following files
Let’s create a folder that will contain all the files needed to make the package, call it flat.
In order to create the Bom file (this file list the files in the installation – their path, octal mode, UID/GID, size and CRC32) we will use the bomutil we installed earlier.
mkbom -u 0 -g 80 ./pkgroot/ .flat/base.pkg/Bom
In our example, we have two scripts included – one for pre-installation and one for post-installation. for reference, this is our pre-install script:
#!/bin/bash
sudo launchctl unload /Library/LaunchDaemons/com.coolprogram.binfile.plist
exit 0
We must add execution permissions to the scripts and to the binfile as well:
chmod +x ./scripts/preinstall
chmod +x ./scripts/postinstall
chmod +x ./pkgroot/opt/coolprogram/binfile")
Next, we will create the Scripts and the Payload files:
cd ./scripts && find . | cpio -o --format odc --owner 0:80 | gzip -c ) > ./flat/base.pkg/Scripts
cd ./pkgroot && find . | cpio -o --format odc --owner 0:80 | gzip -c ) > ./flat/base.pkg/Payload
The next couple of files require some information, run these commands to get the files count and files size of your pkgroot folder:
# Files count
find ./pkgroot/ | wc -l
# Files size
du ./build/pkgroot/ | awk '{print $1}'
Change the placeholders in these files with the output you got (in our example, the package version is 1.0.0, you can change it as you like) and save them:
Distribution in the flat folder and PackageInfo in the base.pkg folder
<?xml version="1.0" encoding="utf-8"?>
<installer-gui-script minSpecVersion="2">
<pkg-ref id="com.coolprogram.binfile">
<bundle-version/>
</pkg-ref>
<options customize="never" require-scripts="false" hostArchitectures="x86_64,arm64"/>
<volume-check>
<allowed-os-versions>
<os-version min="10.15"/>
</allowed-os-versions>
</volume-check>
<choices-outline>
<line choice="default">
<line choice="com.coolprogram.binfile"/>
</line>
</choices-outline>
<choice id="default"/>
<choice id="com.coolprogram.binfile" visible="false">
<pkg-ref id="com.coolprogram.binfile"/>
</choice>
<pkg-ref id="com.coolprogram.binfile" version="1.0.0" onConclusion="none" installKBytes="SIZE_PLACEHOLDER" auth="Root">#base.pkg</pkg-ref>
<product id="com.coolprogram.binfile.1.0.0"/>
</installer-gui-script>
<?xml version="1.0" encoding="utf-8"?>
<pkg-info overwrite-permissions="true" relocatable="false" identifier="com.coolprogram.binfile" postinstall-action="none" version="1.0.0" format-version="2" auth="root">
<payload installKBytes="SIZE_PLACEHOLDER" numberOfFiles="COUNT_PLACEHOLDER"/>
<bundle-version/>
<upgrade-bundle/>
<update-bundle/>
<atomic-update-bundle/>
<strict-identifier/>
<relocate/>
<scripts>
<preinstall file="./preinstall"/>
<postinstall file="./postinstall"/>
</scripts>
</pkg-info>
Next, we will pack the flat folder
cd ./flat && xar --compression none -cf './coolprogram.pkg' *
Signing packages requires using both the Developer ID Installer and the Apple Worldwide Developer Relations Certification Authority certificates.
Note: in our example, we have our certificate password protected
In our case, the first one is called installer_cert.p12 and the latter certroot.cer
again, we will create the pem and the der files for the signing process
openssl pkcs12 -in installer_cert.p12 -password pass:<PASSWORD> -certfile certroot.cer -nodes | openssl rsa -out key.pem
openssl pkcs12 -in installer_cert.p12 -password pass:<PASSWORD> -certfile certroot.cer -nodes | openssl x509 -outform der -out cert.der
Now that we got the certificates, we can sign the coolprogram.pkg file
xar --sign -f ./coolprogram.pkg --sig-size 256 --cert-loc cert.der --digestinfo-to-sign digestinfo.dat
openssl pkeyutl -sign -inkey key.pem -in digestinfo.dat -out signature.dat
xar --inject-sig signature.dat -f ./coolprogram.pkg
The last step we have to do is to notarize the package with the Apple store API.
We will first create a json file with the 3 items we fetched earlier – the issuer ID, the API key and the private key, in our example the private key is called notarize.p8
rcodesign encode-app-store-connect-api-key -o key.json <ISSUER_ID> <API_KEY> notarize.p8
Finally, we will use the json file we created to call the apple store api and notarize our pkg file
rcodesign notary-submit --api-key-path key.json --wait ./coolprogram.pkg
The output should look something like:
creating Notary API submission for coolprogram.pkg (sha256: .............)
created submission ID: .....-1d8f-....-9c90-........
resolving AWS S3 configuration from Apple-provided credentials
uploading asset to s3://notary-submissions-prod/prod/.....-1d8f-.....-9c90-......
(you may see additional log output from S3 client)
S3 upload completed successfully
waiting up to 600s for package upload ......-1d8f-......-9c90-........ to finish processing
poll state after 0s: InProgress
poll state after 3s: InProgress
poll state after 7s: InProgress
poll state after 10s: InProgress
poll state after 13s: InProgress
poll state after 17s: Accepted
Notary API Server has finished processing the uploaded asset
fetching notarization log for .....-1d8f-.-.....9c90-.......
notary log> {
notary log> "archiveFilename": "coolprogram.pkg",
notary log> "issues": null,
notary log> "jobId": ".....-1d8f-.....-9c90-.....",
notary log> "logFormatVersion": 1,
notary log> "sha256": "..........",
notary log> "status": "Accepted",
notary log> "statusCode": 0,
notary log> "statusSummary": "Ready for distribution",
notary log> "ticketContents": [
notary log> {
notary log> "cdhash": "........",
notary log> "digestAlgorithm": "SHA-1",
notary log> "path": "coolprogram.pkg"
notary log> },
notary log> {
notary log> "arch": "arm64",
notary log> "cdhash": ".....",
notary log> "digestAlgorithm": "SHA-256",
notary log> "path": "coolprogram.pkg/base.pkg Contents/Payload/opt/coolprogram/binfile"
notary log> }
notary log> ],
notary log> "uploadDate": "2022-12-05T08:07:50.134Z"
notary log> }
And that’s it!
You can now distribute the coolprogram.pkg file to anyone without any issues!
Reference links:
https://gist.github.com/jcward/d08b33fc3e6c5f90c18437956e5ccc35
http://bomutils.dyndns.org/tutorial.html