One stop solution for setting up CI and CD system for iOS applications using Jenkins and Xcode - A complete reference guide (Part 1)

Hi,

In a quest to streamline the product delivery process to QA and to app-store, I decided to set up a CI and CD system using Jenkins at my current company. Being a senior iOS developer, although I had experience working with CI systems before, I never had set up one on my own before. Being aware of Jenkins and Xcode's sour relationship, I knew I was up for a war :) and my beloved Xcode and Jenkins did not disappoint me a bit.
So, I decided to write a blog for those of you, who decide to go down the same road as I did.

What is CI ?
Continuous Integration (CI) is a development practice, which aids in improving the product quality by imposing frequent code push to the centralized repo, thereby making the code available to testers for testing more frequently and in smaller chunks. This approach helps in identifying and backtracking the bug, which makes it easier for the developers to fix them quickly as well.

What is CD?
Continuous delivery is the further extension of CI. On clearing all the automated tests, the system creates a build and deploys it to various environments automatically. This makes the executables available for testing, and helps in improving the transparency of development process as everybody is aware of current state of the build.

How will you achieve CI and CD?
Simple enough! At the high level, all you need to do is,

1. Set up a branch and hook the Jenkins to monitor the branch. 
2. Every time someone makes a push to the branch, Jenkins pulls the code from the remote branch and builds the code using Xcode command line tools.
3. Jenkins then runs the unit tests written by developers to monitor the sanity of the code. (You can add the automated scripts written by your testers, as well, as a Jenkins job here). 
4. If code fails to clear even a single unit test, a mail is sent out to developers to warn them about the instability of the code.
5. If everything goes fine, Jenkins creates a .ipa file of the product and deploys it on iTunes connect, thereby making the build available to TestFlight.   

Sounds easy, isn't it? Trust me, it is indeed easy to implement it...just follow the tutorial entirely and you won't be disappointed! :)

To keep things simple, let's break the process it in two parts. In this part, let's install Jenkins, configure it, and modify the Xcode project settings to make it work with Jenkins.

And here's taking a deeper dive into the first part...

Step 1: Checking if JAVA exists and installing JDK 1.8 if it does not exists in your Jenkins machine.

This is one of the most important steps. Needles to say, I learnt it a very hard way. Most of the tutorials out there say, Jenkins needs JAVA but they do not mention that Jenkins expects you to have specifically JDK 1.8. Here is the link to official Jenkins Page, which clearly mentions that,

"At this point, Jenkins does not yet support Java 9 development releases. (Date : 2 December 2017)"

As JDK 1.9 is available as the latest version, just like me you too might be using it. So, remember to downgrade to JDK 1.8 to support Jenkins until Jenkins becomes compatible with the higher JDK versions.

To check if Java is installed in your system and to test the version of JDK installed in your system, open the terminal and type

If in case your system does not have JDK installed, Go to Oracle official website, select MacOS and download the dmg. On downloading the dmg, double click open it and follow the instructions to install JDK. Once done run the command above to confirm the installation.

Step 2: Download and Install Jenkins.

To download the Jenkins, visit the official website of Jenkins, scroll to long term support, select the MacOSX. This should start the dmg download immediately.

After you download the .pkg file, double click on it and the installer pops up. Follow the instructions to install the Jenkins. At the end of it your browser should automatically open up a new tab and show 

As it clearly shows, in order to confirm that Jenkins is being installed by the admin of the machine, the initial password is written in secret folder. In order to unlock the Jenkins you need to copy the content of the file and paste it here. In order to paste the content use 

Now, simply right-click and paste the copied password, and then click on Continue. On the next page, select the install suggested plugins. Jenkins takes a decent amount of time to download and install the plugins. Until then, grab a cup of tea, sit back and relax.

Once the download completes, Jenkins asks you to create an admin user. Go ahead and create your first admin account on Jenkins. Remember to save the username and password for future use. 

After entering the details, click on Save And Finish.  If everything goes fine, you should be able to see the Jenkins Dashboard.

Step 3: A dirty secret of Jenkins you should be aware of (shhhh, don't tell anybody. After all it's a bloody secret)

When you install Jenkins,  it creates a new anonymous user account of type Admin. (In case, you see a anonymous account created with Standard account type, you will have to change the account type to admin). You should be able to see it in System Preferences -> Users & Groups.

Now that we know, Jenkins runs in its own User and user has admin permission, we should switch to this user account to configure Jenkins further. But before we do that let's work out some more things being in current account, so that transition becomes smooth. 

Let's go ahead and rename this account, Am gonna call it as Jenkins. In order to rename the account, right-click on the account and select Advanced Options.

Jenkins creates an account without a password, so let's create a password for this user account. In order to set the password click on Reset password. Enter the new password and confirm it.

If in case, Jenkins created a Standard account (instead of the type, Admin), check the Allow user to administer this computer check box to make it an Admin account.

Step 4: Download and install Xcode on your Jenkins machine.

If your Mac does not already have Xcode installed, go ahead and download the Xcode from Apple's website. 

Step 5: Export your developer/distribution certificates and their private keys from KeyChain. 

No matter, whether you are setting up Jenkins on the same machine as your development machine (for testing purposes), or you are setting it up on an altogether different machine, you need to export your developer/distribution certificates. 

This is important because, Jenkins has its own account, Obviously the Keychain instance for Jenkins account and the Keychain instance for your current account will be different. So in order for Jenkins to be able to access your distribution certificate you need to copy it in Jenkins' own Keychain. Hence exporting is necessary.

As I had already mentioned in the beginning of the tutorial that we will be setting up the Jenkins to upload the build to iTunesConnect, and iTunesConnect will not accept .ipa, code signed with development certificate, so there is no point in exporting the development certificate from the context of this blog. Hence, I'll be exporting only the distribution certificate.

In order to export the distribution certificate, follow Keychain -> login -> Certificates. Select the distribution certificate of yours and then right click and select export. Enter the password and confirm password to encrypt the exported certificate. 

Each development/destribution certificate will have its private key counterpart in your key chain. Combination of private key and certificate is necessary to code sign your application. So we will have to export the keys as well. In order to export the keys, repeat the same procedure on the corresponding private key.

Now we need to pass these exported certificates to the Jenkins account. Use either air drop, gmail or pen drive/hard disk to copy these certificates. If in case you have both Jenkins and your development account on the same machine, simply copy these files to the Jenkins shared folder.

Path to Jenkins shared folder is /Users/Shared/Jenkins/Home. 

Step 6: Log in to Jenkins Account and continue further configuration.

Now that we have everything we need, it's time to say goodbye to the current user account. Hereon we continue our configuration in the Jenkins' account.

Step 7: Add the distribution certificate and private key exported to Jenkins Keychain.

In order to add the certificates to Keychain, simply double-click on the certificate. When prompted to enter the password, use the same password that you used while exporting the certificate. 

As of now, there seems to be some bug in the UI of Keychains and hence the added certificate might not reflect immediately. You might have to kill and restart the Keychain to see the added certificate and its key.

Many folks face issue with the Keychain UI. On double-clicking the certificate, sometimes, Keychain complains that it could not add your certificate. If you are one of such unlucky developers, don't worry, I have a solution for that as well.

The issue seems to be only a UI bug and you can circumvent the issue by turning to our old pal Terminal.

Now open up your Keychain and make sure your distribution profile and private keys are added to Keychain.

Step 8: Setting up SSH key with Github/Bitbucket.

In order for your Jenkins system to pull your code from Github/bitbucket you need to authenticate Jenkins with Github/Bitbucket. Now there are multiple ways, one that I prefer is setting up SSH key.

Though for the purpose of this tutorial, I'll be using BitBucket as the steps are identical to Github.  

As a first step, we need to generate the RSA private and public keys to authenticate the Jenkins system with Github/Bitbucket. In order to generate RSA keys, open your terminal and enter 

Once you are done with it, you should be able to see two files named id_rsa and id_rsa.pub in your ~/.ssh folder.

You will copy the content of id_rsa.pub (RSA public key) and add it to your Github/Bitbucket account. In order to copy the content of id_rsa.pub use

Now login to your GitHub / Bitbucket account and add the copied RSA public key. In order to add the SSH key to GitHub follow Github help page.

In order to add SSH key to Bitbucket, go to your Account -> bitbucket settings -> Security -> SSH Keys. Click on Add key and paste the copied RSA public key. That's all. If you still feel confused follow the official Bitbucket help page.

Finally in order to test the SSH connection open your terminal and type 

If everything goes fine, you should be able to see "You can use git or hg to connect to Bitbucket" That's it...now your Jenkins System can pull the code from Github/Bitbucket whenever it wants.

Step 9: Clone your repo to further configure your Project Settings to make it work with Jenkins.

Now that you have configured your system to use SSH connection go ahead clone your repo. This repo will not be used by Jenkins. We will use this repo to modify the project settings to make it work with Jenkins and will push the changes back to the remote repo. You can change the project settings on any system and you need not clone the repo on your Jenkins machine, but there is a very good reason why I am asking you to do it on the Jenkins machine. Once the changes are done and you push the code back to the remote repo, you can delete the local repo from the Jenkins machine.

Step 10: Login to Xcode and sync all the provisioning profiles.

Fire up your Xcode. Go to Preferences -> Accounts. Add your developer account, and click on Download Manual Profiles to download all the provisioning profiles associated with your developer account.

Step 11: Change your Project from Automatic Code signing to Manual Code signing.
I know that Xcode9 command tools now supports automatic code sign with the usage of -allowProvisioningUpdates. However, I have not had a chance to work with the automatic code sign yet, and I've heard that many developers are facing issues with it. That's why I plan to take some time to try it out myself, and will then update this post accordingly. 

So, for now, if your project happens to use Automatic Code sign, then you will have to change it to Manual Code Sign. If your project is already manual code signed, then you can skip this step.

In order to change to Manual Code Sign, you need to create a Distribution Provisioning Profile. To create one, Go to developer portal, select provisioning profiles and create a new distribution provisioning profile. Don't forget to add your distribution certificate (that you just imported and added to Jenkins Keychain) to the distribution profile.

Once done, come back to Xcode, go to preferences -> Accounts, and click on Download Manual Profiles to download the newly created provisioning profile. Now open your project setting for each target in your project and uncheck the automatically manage code signing. Select the provisioning profile that you have just created from the drop-down next to Provisioning Profile. 

In case, you don't see the provisioning profile you just created, download the provisioning profile from the developer portal, and double-click on it to install it on your machine. Once you do that, you should be able to see the provisioning profile in the drop-down automatically. 

Once you select the provisioning profile, Team and Signing Certificate should be selected automatically. Else, try selecting it on your own. 

Once done, try running the project just to confirm all your code signing changes are fine.

Step 12: Placing Provisioning profile in proper folder and finding its UUID.

If in case you happened to skip step 10/11, then you might not have the provisioning profile in its place. Xcode command that we gonna use soon will try to look for provisioning profile at /Users/Shared/Jenkins/Library/MobileDevice/Provisioning Profiles. In order for your xcodebuild command to run fine you must add the provisioning profile to  Provisioning Profiles folder.

This you can do in two ways.

1. Use your Xcode and sync all your manual provisioning profile as mentioned in Step 10.

2. Download the provisioning profile manually and place it at path /Users/Shared/Jenkins/Library/MobileDevice/Provisioning Profiles. You might find that Folder MobileDevice/Provisioning Profiles does not exists, in such case you have to create one on your own. Because it involves loads of manual work I prefer using Xcode.

We need to find the ID of the provisioning profit that we jus created. ID of the provisioning profile is nothing but the part of file name prior to .mobileprovision extension. You can open up /Users/Shared/Jenkins/Library/MobileDevice/Provisioning Profiles folder and you should see at least one Provisioning profile by now.

If there is only one, then there is no confusion, simply copy the file name till .mobileprovision extension.

If there are multiple mobile provisioning profiles, then it becomes difficult to identify which is the correct provisioning profile or if you have installed provisioning manually even then file name will not tell you the ID of provisioning profile.

In such cases, download the provisioning profile manually from developer portal and open it using TextEdit, search for key <key>UUID</key> value for this key will be the identifier of your provisioning profile. Copy it. We will use this key in next step.

Step 13: Create a ExportOptions.plist to assist xcodebuild tool
Starting from Xcode 9, you need to pass ExportOptions.plist as a runtime argument to  xcodebuild command to archive and export your ipa. This ExportOptions.plist is a simple plist which contains the configuration details for archiving and exporting ipa. Few such configuration details are

1. SigningStyle : Manual or Automatic
2. Method : app-store, adhoc, development
3. ProvisioningProfiles : Name of provisioning profile/id of provisioning profile
4. TeamID : Team ID associated with your distribution certificate.

Though the structure of ExportOptions.plist is very simple, there seems to be a tons of confusion about it among developers. Thanks to Apple, for changing the structure of ExportOption.plist significantly from Xcode 8 to Xcode 9 there by creating a huge confusion among developers. Here is a copy of working ExportOptions.plist.

Though it looks simple, in case you are wondering how I figured it out, I followed this tutorial . The hack of actually exporting an ipa using Xcode 9 and then copying the content of ExportOptions.plist was amazing and helped me a lot to understand the structure of it.

Add this file to your projects root folder and now you are in good place to commit your code back from your local repo to remote. 

Step 14: Fire up your Jenkins and and join the pieces  together to set up our CI.

Though this is a Jenkins integration tutorial, other than installation part we haven't talked much about Jenkins or about its configuration. To be frank, its simple the biggest pain was Xcode and we dealt with it already. 

Open your web browser and enter localhost:8080, this should open Jenkins webpage and ask you to login. On successfully logging in, click on New Items, and enter the name of your project in the textField. Select the Freestyle project and click OK

On creating a new Item,  check This project is parameterized option and select String Parameter. In the name field enter branch and in default value field enter the name of the branch from which you would want Jenkins to pull the code from. By default it always pulls the code from master.

scroll to Source Code managed meant and select Git. Enter the SSH repository URL for  Github/Bitbucket. Point to consider here is to use SSH url and not HTTPS url because we have set up our Jenkins machine to use SSH connection in Step 8. Once you add the URL you should see something like

Clearly, its complaining that Jenkins could not access your Github repo. You need to add credentials to allow Jenkins to access the repo.

To do that click on add, when the popup appears select the Kind to SSH username with Private key. Enter your Github/Bitbucket username in username field. 

Select Enter directly option for Private Key, now copy the content of /Users/Shared/Jenkins/.ssh/id_rsa using pbcopy and paste it here.

Enter the pass phrase you used, while creating RSA key in the pass phrase field. 

Once done it should look like

Now click on Add and return to the Source Code Management section and select credential that you just created from drop down. Give it sometime and all the errors should disappear. 

Now scroll down and select the branch, your Jenkins should pull the code from. We have already added a variable named Branch to point to the branch we need earlier using This project is parameterized option remember? Simply use that variable now.

There you go! Now your Jenkins is fully configured to pull the code from the branch you specified whenever it needs.

Before we wind up this part, Lets do one last thing. We would want Jenkins to pull the code from Github/Bitbucket every time somebody pushes the code to your specified branch don't we? Now we can do it with WebHook plugin, But because not all of us have a public IP on our Jenkins machine, we might not be able to set up Webhook so as an alternative we will use Poll SCM.

We will set up our Jenkins system to poll the Github/Bitbucket branch we specified at a regular interval and check the hash map of the previous commit. If there is change in hash map (That means some new code has been pushed to Branch) Jenkins will pull the code base and will continue with its build operations. To do that simply scroll to Build Triggers and check the Poll SCM and enter H/5 * * * *

Thats all. Now your Jenkins will poll your specified branch every 5 minutes and check if there is any new code being pushed. If yes, it will pull the code and build it else it will continue to ignore it. Thats great!! Isn't it?

Thats all for Part 1. Now we have Jenkins installed and completely configured to work with our Xcode project. But the actual build job yet to be added to make it work. Believe me thats the easiest part. See you in Part 2.