Shift-Left – (never again) sleepless in DevOps

3 o’clock in the morning. A nasty beep pulls you out of your sleep. Your mobile phone rings. And no, it’s not personal, it’s the job. You work in an online development team and are now doing DevOps. This means that in addition to development, they have also taken on responsibility for operations. This in turn means that you are now also responsible for troubleshooting nightly support cases.

And so you now stare at your mobile phone with a cryptic error message that monitoring has automatically sent. At first glance, you understand only this much: it must be urgent. Someone on the other side of the world cannot use one of your services. You hurriedly go to your PC and dig through the log files. Always on the lookout for the trigger or the cause of the error. At the same time, your mind is racing: Maybe it wasn’t such a good idea to release the new feature on Friday afternoon? But time was breathing down our necks! Or is the error in last week’s API change? But someone must have tested it!

Holistic responsibility for development and operation

Let me say one thing upfront: DevOps is great.¹ A team takes holistic responsibility for the development and operation of their product. With the appropriate practices and principles, software is developed faster, safer and more stable. With the accompanying changes in mindset, behavior and culture, historically grown organisational structures and hurdles are also overcome.

Figure 1: DevOps Lifecycle

• Fewer dependencies,
• higher speed and
• stability are not just buzzwords in the

are not only buzzwords in the DevOps context, but at the same time also the things that ensure a more peaceful sleep.

Shift-Left – The shifting of testing efforts

Continuous Integration (CI)² – a process of continuously merging components into an application and an important part of the DevOps approach – also promises greater speed. One way to achieve good CI is through automated testing. When we talk about good test automation, we do not mean the highest possible number of test cases, but rather the highest possible quality of test cases. High-quality does not mean that the majority of testing activities take place in the test phase after implementation, but rather that the majority takes place much further upstream and then distributed to the other phases. This shifting of testing efforts to earlier phases of the DevOps cycle is also referred to as “shift-left”.

Figure 2: Shift-Left – shifting activities to the left

The combination of DevOps with Shift-Left

From the perspective of quality assurance (QA), I would even like to put forward the thesis that it is only the combination of DevOps principles and a simultaneous shift-left that lets us sleep soundly. To understand this, however, we first need to explain the shift-left approach. To do this, we unfold the “DevOps eight” and thus obtain a route.

Figure 3: Shift-Left-Left in the spanned DevOps lifecycle

On the one hand, it is about shifting QA activities and efforts to the left in the cycle, i.e. earlier in time. Testing is no longer a phase after the build and before the release, but must take place in all phases of the cycle.

In addition to the temporal significance, there is also a “local” significance. One could almost speak of a shift-down. Let’s recall the much-cited test pyramid. The original pyramid by Mike Cohn³ in the version with the levels unit, service and UI.

Figure 4: Execution form vs. integration

Cohn and also Martin Fowler⁴ thus illustrate the ideal-typical distribution of testing efforts. Fowler himself, however, also speaks of the fact that in reality the pyramid likes to resemble an ice-cream cone, because test activities usually take place late and over the surface of the application.

Let’s detach ourselves for a moment from the definition of the pyramid, which also wants to map the execution mode of test automation with the levels, and now assign the levels the meaning of the degree of isolation of the test object.

The further down the pyramid, the smaller the test objects are considered in isolation. As already touched on in the topic of test automation, the shift-left also means a shift of the tests on the pyramid from top to bottom. Away from complex, long-running and maintenance-intensive tests via the UI, towards the earlier, because more local tests of isolated test objects with fast turnaround times.

Earlier testing means we find bugs more often and faster. Smaller and more independent test objects also means we can fix bugs much faster. The few or non-existent dependencies of the test objects also mean that the maintenance of the test cases can be carried out with less effort. These effects together result in a significant cost reduction, or to quote Kent Beck: “Most defects end up costing more than it would have cost to prevent them”.⁵

Isolation with Shift-Left

I would like to explain isolation in more detail. Shift-Left is not only about a large amount of isolated unit tests, but also about the autonomous testing of (micro-)services and interfaces using techniques such as mocking, service virtualisation⁶ and Contract-Testing⁷.

If a high level of test coverage can be realised at these levels, there is automatically no need to test the requirements checked with these test cases at higher levels or even to repeat them implicitly, such as in elaborate End2End tests. This gives us another great advantage as a practical gift: fast feedback.

If we are to test even minor changes again and again quickly, then a manual testing approach makes little sense. By testing early in development, frequently and automatically, we can get feedback on the state and progress of our development artefact at almost any point. And this feedback is also usually more cost-effective due to the reduced complexity resulting from the lack of dependencies.

Conclusion

From the perspective of quality assurance and testing, Shift-Left means: Fast feedback through test early and test often.

I would like to mention one more addition at this point: you have just seen that you can break up the rigid anchoring of the Test phase after coding and sensibly distribute its contents and activities to the left until the Plan phase. The left end of the shift-left thus starts from the end of the Plan phase. If you now extend the scope by this phase and take another step to the left, so to speak, we speak of the so-called Shift-Left-Left (cf. Figure 3).

The requirements definition phase can also benefit from the additional QA perspective.

• How can the acceptance criteria of a user story be tested at all?
• Can testable quality requirements be derived in addition to functional requirements?
• How does the planned procedure fit with the test architecture and test strategy?

These are just a few questions that can be answered as early as possible in the later development process.

In addition, the collaboration of the different disciplines (here usually: requirements, test and software engineering) strengthens the whole-team-quality idea and reduces the friction that can arise during the handovers between the disciplines. Widespread approaches such as Behaviour Driven Development (BDD)⁸ and activities such as the 3-Amigo conversation are certainly worth mentioning here. However, all this offers enough material for a separate blog post.

Back to my thesis: So how does the Shift-Left make you sleep through the night? Put simply: by testing in a more small-scale and at the same time more comprehensive way, you can build up a stronger confidence in the stability of your application and its functionality. You thus reduce the blind spots before the release and minimise the “release pain” in the sense of delivery risk. If, as described in the initial scenario, an error situation nevertheless occurs in production, you can analyse it more quickly because you can localise the causes more quickly. Finally, you can start troubleshooting and restoring the system or functionality sooner. In addition, you also have a lasting positive influence on one of the DevOps key metrics⁹ for the performance of software deployment, the “mean time to recover” (MTTR)¹⁰.

With this in mind, sleep well! 😴

Notes:

If you like the post or want to discuss it, feel free to share it in your network. 

[1] What is DevOps?
[2] Continuous Integration: Improving Software Quality and Reducing Risk (Duvall, Matyas & Glover)
[3] The Forgotten Layer of the Test Automation Pyramid
[4] TestPyramid
[5] Extreme Programming Explained: Embrace Change
[6] Mocks Aren’t Stubs
[7] ContractTest
[8] Introducing Behaviour-Driven Development
[9] Accelerate: The Science Behind Devops: Building and Scaling (Forsgren, Humble, Kim)
[10] What Is Mean Time to Repair? MTTR Explained

Veit Joseph has published another article in the t2informatik Blog:

What does t2informatik actually do? One click and you'll know.