Performance analysis beyond code
The world outside your code
A colleague was given the task of redesigning a label. This label was to be designed using JasperReports and sent to a label printer as a PDF – and not directly in the manufacturer’s printer language as before.
After the tests in the test system were successful, the change was transferred to production. Immediately there was a hail of protests from customers! While printing a label used to take less than 1 second, printing the new label now took about 35 seconds. The colleague optimised his code and was able to reduce the time to 32 seconds. Other colleagues came to the rescue and optimised the printing to 31 seconds. Then an administrator was called in to analyse the network traffic. It quickly became clear that another administrator had installed a new printer driver on the production server. This printer driver asked the printer if there were enough labels available to print (convenience feature!). However, as this feature was deactivated on the productive printers, the printer driver waited until a timeout for a response. Well, this timeout was 30 seconds long! On top of that, there was an implementation error in the driver that caused the print to be sent to the printer despite the lack of a response.
In this blog post we would like to show how you can deal with performance problems in the “world out there”, beyond code.
What is performance anyway?
In computer science, performance describes the efficiency of hardware and software. If you want to assess the performance of a system, you should also know the environment in which it is achieved. Try to sketch a picture of your environment. The important thing is not to create a sketch that is as detailed as possible, but a simple drawing that will help you in a concrete way! The visual appearance is relatively unimportant, what matters is the connections and the understanding gained. Take a maximum of 10 minutes for such a sketch, because if performance problems occur in production, it has to happen quickly.
In the example with the slow label printing, the software system looks like this:
Now take the picture and supplement it with tools that can help you measure performance at the interfaces.
Certainly, there are always points where you cannot find a way to measure performance; it is precisely these “blind spots” that need to be identified.
Where is the time lost and where do you find the relevant users?
If the system “feels” slow, this may be a very subjective feeling on the part of the user. But it is precisely these users who operate a system every day and therefore notice very quickly that something is “not running smoothly”. This means that you should include these users in the performance analysis. But where do you find the relevant users?
Just as we used a sketch to explore the technical environment of label printing, you need an overview of which users can help you find performance problems. You need concrete contact persons. Use the picture of the system environment and derive the corresponding users. Ask yourself, for example:
- Who uses the system at the point where the problem becomes apparent?
- Who else works with the system: at the step before, at the step after?
Talk to real people and listen well
Yes, now you need to talk to real people. Maybe you are afraid to talk to your customer because he has just called you in an angry and unnerved state. Try to put yourself in his shoes. Develop empathy. It’s not that hard, just listening well can do it.
Let’s take a look at the different forms of listening. Often we listen in order to speak ourselves. We lurk for the opportunity to interrupt the other person’s flow of speech and then speak up. But for difficult situations we need
- active and
- empathic listening.
Receptive listening serves to gather information!
Signal that you are attentive and let the other person speak. Ask: “Please tell me what is going on with you right now. I’m all ears!”.
Active listening shows that you want to understand!
Report back what you have heard: “I would like to briefly summarise what I have understood”. “What came across to me was that…”.
Empathic listening picks up on the emotions of the other person!
We take seriously how the other person is feeling at the moment. We express this by reflecting the feelings we perceive in the other person: “That sounds like frustration!” “I hear a lot of anger there!”.
Try to put yourself in the shoes of the user. What impact does the performance problem have on him personally? Really take the time to listen. Your counterpart will feel better understood and taken seriously.
Understand the subject matter
Ask the users if they can identify a specific use case that is running slowly. If you get the answer that everything just works slowly, define a specific one with the users to try to track down the performance problems. Ideally, describe a process that you can then run through yourself.
Use visualisation techniques so that you can talk through the use case step by step with the users. Look for exceptions and alternative processes and note them down in bullet points. Then go through the entire process again to identify the points where performance changes.
Try to get a sense of whether the flow is subject to large fluctuations. Try to find out where and how you can determine the specific time course. In the simplest case, the time is logged at certain points, in other situations it can possibly be determined by tools. (A practical tip: If data is exchanged between different systems, for example, make sure that the clocks of the systems are synchronised. If this is not possible, make a note of the difference between the systems).
Another important point: Remember – even if something works quickly for you in a test environment, this may not be the case for the users (keyword: The world outside your window). If in doubt, it is advisable to go to the specialist department and check the performance on site. This has two major advantages:
- You may come across boundary conditions that you were not aware of, but which could be elementary for the solution of the performance problem.
- And from a purely human point of view, you will notice that this kind of appreciation goes down very well with the users in the specialist department and that they are happy to help you solve the problem.
Make something clear to yourself beforehand: although you are the expert for the software, the people in the specialist department are experts in their job. Printing labels from our example may sound like it requires little to no expertise. However, it can be an essential step in a process that is important for the delivery of an overall service in the business process. The users in the business department are the experts for this. And because of the performance problem of the software, they cannot do their job or can only do it to a limited extent.
So take the users and their expertise seriously and show it. Thank them for giving you an insight into their department and for taking the time to talk to you. In addition, you can consider appropriate questions in advance:
- What exactly do the users do with the system and what job should the software do for them?
- How seriously does the performance problem affect the processes?
- To what extent are the users restricted in doing their actual job?
- How does this affect the users personally? How do they feel about the situation?
You will find that such openness and attitude, as well as appropriate questions, do not diminish your expertise. On the contrary: you thus create trust. With honest interest, you will be perceived even more as an expert in solving the performance problems. You signal that you really want to understand the situation and the challenges in order to find the best solution.
And if you then also have some biscuits with you, all doors will be open for you! 😉
Fixing performance problems can be a big challenge. Often, nothing is what it seems. It sounds logical for developers to make local optimisations to the code – as described in the label printing example. However, it makes more sense to look beyond the end of the nose, to the system context and to possible causes beyond the code. Before optimising the code, it is worth taking a look at “the world out there”. If you ever find yourself in such a situation, try to understand this world and get to know the real causes. The users will certainly help you with this.
Susanne Muehlbauer is a self-employed agile coach and systemic business coach. With passion and a lot of personal commitment, she works with people, teams and organisations on their way to more agility.
Thomas Ronzon has been working as a project manager and senior software developer at w3logistics AG in Dortmund since 2000. He is primarily involved in the modernisation of business-critical logistics applications.
In the magazine JavaSPEKTRUM he regularly reports on new tools for architects (“The Tool Talk”). He also regularly publishes technical articles and speaks at conferences. Thomas is passionate about diving deep into technical aspects, but never loses touch with the technical side of things. With a lot of empathy, experience and concrete suggestions for solutions, he repeatedly bridges the gap between business and IT.