We’re back with more information on CI/CD and specifically how to get started and best practices. If you’re already a CI/CD pro you might want to check out our article here about implementing Gatling Enterprise Cloud Version into your CI/CD process. First though, let’s get started with the basics.
What is shift left testing?
In traditional software development, testing was done at the end of a project prior to release. Whether the testing was functional or performance based, all tests would be performed at the end of the development cycle. However, this often results in having to rewrite massive amounts of code at the end of a project as you’ll need to replace any unstable code you’ve written and also any code that you’ve built on top of it. It can also be extremely difficult to actually find what you’ve broken over the course of the project that is actually causing your issues.
The solution to the above problems is Shift Left Testing, or to start testing as early as possible in the project in order to identify potential problems as soon as they are introduced and correct them before building on top of them. The best way to do this is to implement a CI/CD pipeline that includes testing.
Wait, CI/CD… What’s that?
CI/CD stands for Continuous Integration/Continuous Deployment (or Delivery).
First let’s talk about Continuous Integration, this is level one, testing throughout the course of a project. This is all that’s necessary to have shift left testing.
Continuous Delivery is level two, every time you commit tests are run. If the tests are green you’re generating some packages for your application that could possibly go live. However, pushes to production are still manually deployed and not automatically sent. This typically means you’re not ready to have 0 downtime deployments and are scheduling your deployments for a specific time.
Continuous Deployment is level three, every time you make a commit and the tests are green your changes are automatically pushed to production at any time of the day.
So testing in the CI/CD process means that you’re constantly running tests with every deployment or even automating your deployments based on the result of tests when you do a commit. This allows you to be sure everything you release is stable, resilient, and scalable.
All clear? Great! Let’s get into how to get started with the tips.
Start early
In an ideal world you’d be able to implement your functional and performance tests from the start of the build. But wait (you say)! I’ve already started building and I haven’t done any testing! There’s an easy answer for that, go! Now! Stop reading and start setting up tests. Or finish this article then set up your tests but get started, the sooner the better.
Choose a well supported CI/CD tool
There are many open-source and paid CI/CD tools out there that can help you build, deploy and test your projects. It’s important to choose one that is going to work for you and be well supported over the lifetime of your project. Here are a few of our favorites:
Any of the above are great options and as a developer or a member of devops team you’re probably already in one of these ecosystems. The best part about them is that all of them already have native integrations with Gatling or can be integrated easily using our CI script.
Build your test suite
So you know you need to run tests, but what tests should you actually be running? This is a huge topic, probably worthy of its own post but here are a few of the essentials and quick descriptions of the types of tests you should be running in no particular order.
Load tests:
Obviously we’re starting with Gatling’s bread and butter. A great set of load tests will ensure your application is built on solid footing and will scale as you grow. If you want to be a unicorn tomorrow, start load testing in your CI/CD process today. These should be run before each deployment.
Unit tests:
Static code tests of the smallest chunks (units!) of your application to make sure they’re ready to go with no errors. No infrastructure is needed for unit tests and they should be run by a developer before each commit.
Integration tests:
Making sure that all of your units play nicely when combined. These should be run in your CI process with each merge/pull request.
End to End (E2E) tests:
Usually done as a smoke test (one user) running through the total functionality of your application from start to finish. These should be run after every delivery.
There are many many more types of tests you can and probably should integrate into your CI/CD process but this is a great core to get you started.
Automate small, schedule big
Most of your tests might be short (unit testing for example) and these tests should be automated and run every commit. However, especially when it comes to load testing you can have some tests that run for extremely long periods of time, such as a 4 hour soak test. Rather than automating a test like this to run every commit, you may instead want to schedule it to run nightly so you can be testing while you’re not working. Gatling combined with a CI/CD tool can allow you to schedule your tests during your downtime so even when you’re not on the clock you can still be testing!
Run parallel tests
One of the main reasons that the traditional testing model was used is because it limits testing to a one time phase so it’s faster. We know that sacrificing speed for quality though will hurt you in the end. However, you can speed up your CI/CD testing phase by running multiple tests at once. Most of the time your tests are complementary and by running them in parallel you’ll save valuable time and speed up your development cycle.
So there you have it! That should keep you busy in setting up your CI/CD strategy for a while. CI/CD is a massive topic that we’ve written about before so be sure to check out our other CI/CD posts to learn more and check back often as it’s a topic we’ll be revisiting. I hope this has helped and don’t forget to test, succeed, and iterate.
