In the series of blogs we’ve explained technical terms, explained how the eventual move to cloud is inevitable, how you setup for success in migration and discussed a few of the post migration steps. The question is what’s next in this continuous improvement path?
If the previous blockers were capacity and internal alignment, then what happens now you have (almost) limitless capacity, and everyone is facing in the same direction?
Put simply you need to operate quicker and release value as soon as possible. Many years ago, I heard countless horror stories of large organisations who would batch all changes up and deploy to production once per quarter. The massive risk is that when something goes wrong you don’t which one of the thousand changes have caused the problem. Manual testing, large complex changes and sacred production environments were recipes for disasters.
Changes to software, code and configuration has no value until its released and in the hands of your customers. One of the ways to be ship more often is by adopting continuous integration (CI) and continuous deployment (CD) to get assets into production safely, quickly, and reliably.
It’s worth breaking CI/CD into the separate components.
Continuous integration is the process of integrating code (or other assets) into a shared repository, making sure that the changes don’t negatively impact each other.
What does this really mean in non-technical terms?
- Developers will work on their version of the code; this will have been branched from the master version of the code
- If you have multiple people working on code, then there is good chance that their code will overlap and impact each other’s work
Think how much fun you use to have combining various Word documents together before multi-editor facility was a thing. Multiply the by several hundred thousand lines of codes
An automated process will run when someone tries to update the master code. This introduces one of the prerequisites to be successful with CI/CD, automated testing, more about this soon.
CI/CD only works if all members of the team practice it, this introduces the most important prerequisite, a change in culture and management support.
If you don’t have cultural change or only part of the team are using CI then the changes to master will be less frequent, bigger less frequent changes give more conflict.
Automated testing
CI relies on testing, the earlier the testing the better and there are tools that help check testing coverage. A key point is that don’t expect to cover all tests with automation, you’ll miss something. When you do find something create a test case for it and feed this back into automated testing. For each piece of new functionality there should be test created for it.
Testing should be an ongoing process.
What happens when a build breaks?
Prioritise the fixing of the build and don’t allow any more merged changes. If you allow more merged changes to your build, then it soon becomes impossible to work out the root cause.
So, you’ve now got a lean development / engineering capability producing tested code, regularly merged and ready for production, now you need to deploy it.
Continuous delivery / deployment
We’ve merged the deployment and delivery together in this section. For a lot of organisations, the delivery of code from developer through automation direct to production via continuous deployment might be a step too far. We’re still seeing continuous delivery with manual deployment to production, for now.
The key is that build process and the long processes for deploying solutions has been automated. The aim is an automated release process, and you can deploy your application any time by clicking a button.
You can then decide when to release, daily, weekly etc. Although if all tests have been performed what’s stopping you go deploying more regularly?
What all this means?
First of all, the process above works for software engineering & data engineering but also works well for advanced analytics and model deployment. Although for analytics the model performance is impossible to review based on code alone, that’s a whole blog series by itself.
Second, you don’t need to do everything on day one. If you already have an application then put in basic unit tests & automated testing, these are the bedrocks of CI/CD.
And finally, take time to celebrate your overall success. If you’ve followed some of the steps in these blogs then you’ll have a cloud strategy, migrated workloads to the cloud, a skills assessment, an understanding of the technology terms, CI/CD and most importantly projects delivering real value to customers quickly, efficiently and reliably.
Authored by:
