In the early days, Humi developers implemented Jenkins to run our unit tests as part of a CI/CD process. As an open source tool, Jenkins quickly became an additional product for our team to support, especially before DevOps resources were hired. Spending a lot of time tweaking and scripting Jenkins eventually led us to try out Atlassian Bamboo, a more graphical user interface (GUI) focused build tool.
Atlassian Bamboo worked well for us for a few years, enabling team members to visualize their build steps, work with existing plugins, and expand runner capabilities. As we continued to scale out the system, we realized the GUI-focused system had some limitations compare to other yaml configuration based offerings. In addition, the pricing structure was based around the number of runners being used, which scaled quickly with the needs of our team.
While assessing some of the options available to us, we came across GitHub Actions. Integrating our build pipelines directly into the tool developers already use for source control and change review made a lot of sense. The ability for team members to contribute to the yaml configurations using the same tools, and the ability to build on top of existing "actions" offered by the community seemed like great advantages over other CI/CD tools.
Moving our Atlassian Bamboo pipeline to GitHub Actions was a fairly straightforward task. We were able to leverage some existing GitHub "actions" for common tasks like checking out repositories, running test suites, caching dependencies, and communicating with AWS.
We were able to further improve our pipeline by collaborating with many individuals across the team inside GitHub, working through pull requests as we would with any other application changes.
We were additionally able to reuse some of our custom implementation as private GitHub "actions"
With the ability to leverage unlimited runners in the cloud (compared to the fixed, price per runner Atlassian Bamboo model) we were able to enable some of our end-to-end (E2E) test suites to run on every pull request. In the past, we would run changes against a QA environment, taking turns to refresh the data. Now, we are able to spin up our entire platform, run E2E tests, and clear everything up before continuing. Being able to do this on every pull request ensures the process scales with the size and demand of our teams.
As we began demanding more resources from GitHub to run our tests, we quickly saw a win from a cost perspective with hosting our own runners. GitHub Actions offers the tooling required to introduce your own resources to run GitHub Action workflows, allowing you to scale these resources to the demands of your system. Ultimately we ended up paying only for the hours used via AWS either from ec2 instances or elastic containers.
GitHub Actions has allowed us to run more tests across more systems, while being developer centric and budget-friendly. It has provided us more flexibility when used alongside our end-to-end testing framework, and ultimately feels right at home in our development process. We are excited to see the GitHub Actions tool continue to receive new improvements from the GitHub team and we expect to continue leveraging this technology for the foreseeable future.