SOFTWARE ENGINEERING PROCESSES: CODE RELEASE AUTOMATION

In terms of products a company creates, digital applications are unique. Unlike physical products, computer code can change with the end user being none the wiser. Whether software or firmware, modifications can be made without the entire supply chain having to adjust to accommodate the changes. This mutability is an inherent strength, but it can quickly become a danger if not managed properly. While improvements can be made, optimizations found, and revisions implemented instantaneously; so too can a litany of problems arise as complexity increases. Compatibility issues, feature knowledge, functionality regression (breaking something that worked), scalability; together they quickly become overwhelming when dealing with code release automation.

The common solution for this set of problems is to utilize a system in which a revision of the code is labeled, and information related to all the changes since the last label are collated. The revision information and a strict process to track changes are crucial. To create a release manually, a developer must follow a specific set of steps to get to that point. For a Simplexity firmware or software engineer to release code, they must: stamp the code with the new revision, compile a log of all the changes that occurred in this given revision, build the code, perform various unit tests and code analyses, capture metrics about code health, collate the build artifacts, logs, and metrics into one easy to download bundle, and post to a common location for usage.

That is a lot of steps!

As one can see, this can quickly get difficult to manage for even the most organized of developers, as each step must occur for each release which can be time consuming. Building, collating changes, running tests all take time and effort. Coupled with stress that can accompany a rigid release schedule, pressure from stake holders looking to get a revision that has a desired feature, or delays that can occur if an error was found in the above steps and it is easy to see how a step can be missed unintentionally from time to time. This can be exacerbated further by adding on the complexity that comes with having different products with different release behaviors and recipes. Managing a manual release process is just fraught with small openings for error to be introduced into the system.

Luckily, with the advent of source code revision control systems such as git, the workload can be turned from a multi-day affair, to a singular command that can begin the entire code release automation process. Simplexity has a clear, effective process that can be automated easily, freeing up a valuable amount of resources to be applied else. The standardized, automated release process allows for easy use as well as an effective means to wrangle the release process into a central, debuggable, and repeatable environment that will minimize the chance that errors occur.

---

The Code Release Automation Environment

There are four critical resources used in our automated release process. By using a small bit of scripting, you can combine the power of git, Docker, Confluence, and Bitbucket to handle new code releases.

git

An industry standard, git is a way to manage changes across code. Each time code is changed or updated a unique sequence of letters is generated by a secure hash algorithm (SHA). This SHA is associated with a specific message pertaining to what was changed, which are captured in a log. Simplexity uses this SHA in the release package name, as the SHA is a quick way to associate a group of changes. The changes from the previous release SHA to the SHA of the current release are a snapshot of information that can be used to know what modification were made. It is a good way quickly narrow down if a feature was added, existed, or may have been broken by a specific release. git has the capability to ‘branch’ and have multiple code lines that share a common ancestor but make changes independent of each other, each with their own logs. git also allows for behaviors like tagging, which associates a human readable name, like a release version, with a specific SHA.

Docker

Docker is a system that containerizes computer environments. A container is like a virtual computer inside of your computer. This system allows every project to have its own uniquely configured container for just the development tools needed. Simplexity uses containers to ensure that whenever an engineer, or in this case, our release process needs to do work on a given project, they are given an identical environment. No hassle having to mess with set up instructions. No hassle in conflicting environments from prior work. Instead, an environment is automatically generated from a Docker configuration. This ensures a robust, repeatable environment. By Docker-izing our build environment, the system can be easily run from anywhere on any computer, as needed.

Bitbucket

Bitbucket, part of the Atlassian suite of tools, is our centerpiece for the effective release process. Bitbucket is a cloud-based git server but provides additional tools for software engineering. Bitbucket Pipelines (BBP) are a continuous integration tool, a tool that can perform various checks on a code for every git revision made. Using the Bitbucket hosted git along with Pipelines allows Simplexity to create an exact process of steps run to generate build artifacts from the code and perform a series of tests to verify that the code performs as desired before a release is posted.

Confluence

Confluence is another Atlassian tool, a wiki server. Confluence allows Simplexity to organize information in a centralized location we can share with our clients. Confluence plays nicely with Bitbucket through easy to use, scriptable programming interface (API). Simplexity can link directly to commits in Bitbucket to Confluence for easy reference. Simplexity has developed Python script that bundles the build artifacts and test results from a Pipeline run posts the bundle as a release on a new Confluence page.

---

Putting It All Together

These disparate pieces interlock in an intuitive way. When a developer wants to create a new release, on their local system, they simply call their normal process but with the option ‘release_tag’. Simplexity’s code build process then generates a git tag and pushes it to Bitbucket. When a tagged SHA is pushed to the BitBucket server, our BBP script is kicked off. The BBP then instantiates a Docker container. When the container is running, it kicks off a build of the code. This generates a release bundle. The BBP then performs a series of tests. If these tests pass, the release process calls our Python script which copies the release bundle and a changelog to a new Confluence page.

This workflow completely automates the release process and has vast extensibility for each specific product type needed. Simplexity utilizes the power of BBPs to run a series of commands, load a docker instance, build the code, and centralize build artifacts and change logs instantaneously in the cloud. No longer will there be concerns about missing a step, as there is now a defined pattern to follow no matter who must take over.

While the code release automation process software engineers deal with can be a lot to manage, there are certainly tools to help automate the process to save valuable time.