DevOps and microservices architectures closely scrutinised

IT companies such as Uber, Netflix and Spotify have achieved great growth in a relatively short time. This has been made possible in part by the use of microservices architectures developed and operated by autonomous teams (Low 2021; Chacko 2022; Upadhyay 2021). We take a look at these teams in this article and relate them to microservices architectures via the six DevOps principles of DASA (DevOps Agile Skills Association). Uber serves as an illustrative example for us. The company operates a platform technology that allows passengers to communicate directly with drivers to book a transport option. In particular, we are concerned with changes in the system architecture and the description of technical contexts.

Modern design of development teams

Let’s start with a look at the six DevOps principles and the value-creating communication structures in autonomous teams:

Figure 1: Six DevOps principles (based on DASA 2021)

Vertically organised teams are needed to take over responsibility also in the operation of the product. In addition to developing and deploying the product, the team must be able to operate and improve the product. In order to realise these autonomous and vertically organised teams, skills in different areas have to be built up. To achieve this, it is not expedient to bring together individual specialists, but to build a balanced team structure with distributed skills. In addition to expert knowledge, the soft skills of team members and openness to other opinions within the teams are considered particularly important skills (DASA 2021).

Due to the increasing influence of IT on business success, the focus in software development is shifting to a high quality of the product and a short delivery time. For this reason, implementation projects are largely realised with agile project methods (Shore 2021, p. 7-8). As a result, there are often changes in the organisation of development teams in the sense of autonomously acting units.

Hierarchical organisational forms can manifest problems in communication. For example, classic team structures isolate individual teams and unnecessarily complicate communication channels (Skelton et al. 2019, p. 26).

Organisational structures in projects can usually be divided into three different structures:

1. Formal structure (organisational chart)
2. Informal structure (exchanges between individuals in the organisation)
3. Value-creating structure (exchange related to problem solving).

Basically, the best exchange occurs between the informal and value-creating structure. The goal of an organisation should therefore be to promote interaction between value-creating teams and individuals (Pfläging 2014).

Continuing, Conway’s law describes a phenomenon which states that the design of a system is dependent on the organisational structure of the development team (Kwan et al. 2012). In this context, the value-creating structure is particularly decisive for the design of the product. Here, the management level of a company can influence the design of a product by looking at the organisational structure for collaboration and adapting it if necessary (Skelton et al. 2019, pp. 30-31). As a result, the communication structures also change; this principle is called the “Reverse Conway Maneuver” (Skelton et al. 2019, 37-38).

Figure 2: Teams for independent microservices and databases (Skelton et al. 2019, p. 42)

Figure 2 shows the design of teams for the architecture of independent microservices in independent databases. The communication channels are clearly defined and solution-oriented. As a result, non-value-adding communications can be omitted and the efficiency in the implementation of the target architecture is increased. This principle can be applied to a wide variety of target architectures, so that on the one hand a revision of the communication in the development teams can take place, but also an assessment of bottlenecks in the teams can be made.

A brief introduction to microservices

Due to the increasingly complex structure of IT systems, decoupling individual services or functions has proven to be a way to increase the efficiency and stability of the overall system. Using such an architecture of many individual services is described as microservices architecture (Namiot Dmitry et al. 2014).

In the following, we briefly look at the differences in architecture design, go into the technical realisation of a microservices architecture and summarise the advantages and disadvantages of microservices.

Differences between a classical architecture and a microservices architecture

The division of a complex IT system into individual functions is called microservices architecture. Here, the division into functions is a main component of the architecture. Decoupling individual services into loosely coupled functions has advantages in maintainability, testability and deployment. In most cases, these functions are designed around business use cases and developed by small, autonomous development teams (Richardson 2021).

At the same time, breaking down the entire solution into individual functions presents challenges that can create more development effort. First and foremost, developers are burdened by the added complexity of communicating and delivering the individual functions. In addition, the developers and the stakeholders from the specialist area must be prepared for a new test procedure. In the process, the overarching tests change first and foremost, in relation to the technically mapped process (Chen 2018).

A particular challenge is the definition of the system into the individual functions. With the new introduction of the architecture, a design for the separation must be defined before the implementation can be started. Various approaches to partitioning can be used for this purpose:

1. via the use cases of the departments,
2. via the functions used,
3. via the resources and entities used.

With these three approaches, the main aim is to delineate the responsibilities of the services as clearly as possible in order to be able to make targeted improvements and adjustments without influencing other functions (Namiot et al. 2014).

The communication of the separated services plays another major role in the design of the architecture. Figure 3 shows common methods for communication in a microservices architecture.

Figure 3: Microservices communication methods (adapted from Namiot et al. 2014)

The first method shown represents direct invocations of the app by the services. This method has the greatest flexibility in connecting new services. The problem with the direct connection of the services is the high number of calls to the app. This can lead to delays in processing as products become more complex (Namiot et al. 2014). This challenge can be solved by a hybrid processing mode of the calls (synchronous calls with message queues) or another architecture (Karabey et al. 2021). Other approaches to communication in the microservices architecture are gateway and communication bus solutions. This allows calls to be processed in a more structured way. The communication bus solution is often located in machine communication (Internet of Things) in particular (Namiot et al. 2014).

The three approaches to communication between microservices are independent of each other and can offer advantages or disadvantages in different situations. A gateway solution extends the direct communication between apps and services by a software solution that takes over the processing of the sent message and then forwards it specifically to the target systems. A communication bus solution makes the message available to all participants of the system, so that an extension of the communication by new participants is simplified.

In general, the direct connection of the app has the greatest popularity in the implementation of microservices architectures, as it offers a high degree of flexibility and is easy for DevOps teams to develop (Karabey et al. 2021).

Possibilities for the implementation of microservices

When implementing a microservices architecture, it should be noted that a wide variety of technologies are used. It also follows that the skills required for this must be built up in the team before an implementation can start. Here, too, it is important that the previously mentioned 6 DevOps principles are applied by development teams to ensure optimal development. Of course, the operation of the microservices should be considered from the beginning in order to be able to use a structure for monitoring and error handling of the services in case of a resulting complexity of the architecture. (Lauer and Augsten 2018).

It should be noted that a solution is normally composed of a wide variety of tools and is not solved by one technology. Figure 4 shows a selection of tasks in the development of microservices with common providers for implementing the solution.

Figure 4: Common tools for microservices development (adapted from Thorpe 2018)

As a basic subdivision of the technologies, the topic blocks API management and tests, data transfer between services (messaging), monitoring, serverless tools and toolkits for the implementation of microservices spring to mind. It is striking that large cloud providers such as Amazon, Microsoft and Google provide their own products to support development in many areas (Thorpe 2018).

API management and testing of services describes, on the one hand, the management of microservices throughout the system as well as the testing of services during development. The tools presented for this purpose are characterised by a simpler user interface in which it is possible to track which services are active and how often they have been used. In addition, individual test cases can be run through without much effort before the complete implementation. These tools are particularly important for large IT landscapes, so that an overview can be created between the dependencies of the services.

The transfer of data between services describes the orderly processing and distribution to the different destinations. For this purpose, technologies are used that collect all calls and process them in an order – also called a queue. The start and target services are also specified in the queue.

The monitoring of established microservices describes a technological support for the operation of the services. During the use of the services, many individual calls will take place. It must be possible to monitor the services in order to check the calls and to track the exchanged data.

Serverless tools describe cloud solutions from providers that do not require their own environment. According to the “pay-per-use” principle, companies can call up the microservices in the provider solutions and execute the functions without having to bear permanent costs. In addition, the providers offer their own user interface in which the functions can be developed without having to demonstrate in-depth programming knowledge.

The toolkits for building microservices describe a complete solution from providers. They provide many different technologies and are often characterised by open interfaces to which existing technologies can be connected.

Advantages and disadvantages of a microservices architecture

A microservices architecture offers some advantages, but also has some disadvantages.

The availability of different technologies is considered a major advantage. As briefly described in the previous chapter, several technologies are available for different problem areas, so that teams have the freedom of choice and can decide on a suitable technology. This is supported by existing standards in the areas of integration and communication, so that many technologies can be used in parallel (Kappagantula 2018).

The ability to look at the business value of individual services is also an advantage. Services without real value to the business are simply not developed and operated. This feature makes it possible to show the direct impact of microservices on the operational business (Kappagantula 2018).

Other advantages that arise from coupling microservices as loosely as possible are the independent and parallel development and deployment of functions, the ability to provide new software releases on a simplified basis on a regular basis and the higher security of the entire system architecture through encapsulation (Kappagantula 2018).

In summary, a microservices architecture offers the following advantages:

• freedom over technology choice
• defined business value per service
• independent deployment
• regular software release
• increased security through encapsulation
• parallel development and deployment

However, a microservices architecture not only offers advantages, but also has some disadvantages. For example, troubleshooting is made more difficult because microservices are often operated by autonomous product teams, increasing the complexity of the search for cross-product errors (Kappagantula 2018).

A disadvantage – or should we say challenge – can be the high load in a system. Therefore, when scaling microservices, a delayed response time of the system has to be considered (Kappagantula 2018).

And the decoupling of services also harbours central disadvantages. The increased configuration effort in the development of such an architecture should be emphasised. In addition, the security of each individual microservice must be maintained during operation. Overarching security gaps may then have to be solved individually by each product team. Furthermore, data may need to be duplicated due to performance degradation and/or the increased difficulty of exchanging data and code between services (Kappagantula 2018).

So, in summary, a microservices architecture can have the following disadvantages:

• more difficult troubleshooting
• delayed reaction time
• increased configuration effort
• high operational effort (security)
• data dependencies and data duplication
• code exchange between services

A look at the Uber microservices architecture

Let’s now take a look at the Uber microservices architecture. In the first system architecture of Uber, a classic monolithic approach was used. At the time, this was the simplest implementation option for creating a prototype in a selected city. However, this system architecture was reconsidered in the course of Uber’s expansion due to various obstacles and problems. Especially in operation, scalability was not guaranteed and the continuous integration and further development of the system could not be ensured under the system architecture (Low 2021). Figure 5 shows the chosen monolithic approach and schematically the modern microservices architecture.

Figure 5: Uber system architectures in comparison (Adapted from Kappagantula 2018)

The monolithic architecture presented here consists of three basic components. A REST API was designed for communication between the driver and passenger so that the fastest possible exchange of data between the two individuals is ensured. In addition, three adapters were used at the beginning, which performed individual actions. These include sending emails to the app users, payment via app and billing. The data was all merged and processed in a database. The development took place in technology-related teams (Kappagantula 2018).

The individual functions for passenger management, billing and notifications were merged into one framework. Uber confirmed that this fabric of functions caused very high dependencies in the further development and troubleshooting of the individual solutions. In addition, the system architecture made it difficult to develop new functions (Shivang 2019).

To solve these challenges, it was decided that the monolithic architecture would be replaced by a microservices architecture. The right side of Figure 5 shows the microservices architecture. Each element of the architecture is maintained and further developed by a separate team. An API gateway is used. This organises the communication between driver and passenger to the individual services. The figure primarily shows the division of the adapters from the monolithic architecture into individual services, as well as the expansion of the architecture to include additional services in order to achieve the greatest possible independence between the services (Kappagantula 2018).

By dividing into the user interface and the management of drivers and passengers, it becomes possible to make adjustments to the company’s appearance in the app or on the web without influencing the logic of the management (Shivang 2019).

In addition, a high scalability of the individual functions is achieved by splitting them into the individual services. In this way, resources for passenger management, which is used much more frequently than the payment process, can be better organised and utilised (Shivang 2019).

This approach has also enabled Uber to orchestrate 2,200 microservices without creating critical dependencies in the development and troubleshooting of the individual services. Uber now relies on a domain-oriented microservices architecture (DOMA). Uber’s aim here is not to look at the individual services, but to combine groups of services as domains and organise them as simply as possible in order to counteract the disadvantages of microservices architecture already described (Gluck 2020).

In Figure 6 you can see how different technologies are used by the new architecture. Uber has consciously decided to give the teams freedom over the technologies used (Shivang 2019).

Figure 6: Excerpt of Uber technology use (based on Yang 2022)

In addition to monitoring through Flink, error handling through ELK and the database systems Hadoop and Apache Hive, Kafka is primarily used as a communication service. In addition, Edge Gateway is used as a self-developed API management technology to organise the individual services. These two technologies are particularly relevant for the microservices architecture (Yang 2022).

Apache Kafka is an open source technology for processing and forwarding messages. Kafka communicates via the TCP protocol and can be used as a messaging service, for tracking the activities of a website, for monitoring the processed data and for logging the calls (Apache Kafka 2021).

Uber uses Kafka as a cornerstone of its system architecture. The company also claims to have one of the largest implementations of Apache Kafka in use. Trillions of messages in the petabyte range are processed daily. The routing of messages between driver and passenger also takes place via Apache Kafka. Primarily, Kafka is used by Uber as a messaging service. Furthermore, logging and monitoring of Kafka are used to ensure traceability in case of errors (Yang 2022).

Since 2014, Uber has focused on a versatile use of APIs in the context of microservices. The focus is also on the documentation of the individual endpoints. A separate API management tool was developed for this purpose. Edge Gateway deals with the documentation and organisation of the individual APIs. Since 2018, Edge Gateway has been used for all Uber companies and not just for the classic business model. The technology allows the responsible team and the basic function of the API to be presented across the board (Thangavelu 2020).

Conclusion

When looking at the available literature, it quickly becomes clear that a microservices architecture also entails a change in the organisation. Academic papers and also Uber as an example company show that a change in the system architecture has a great influence on the entire organisation of the IT areas of a company (Skelton et al. 2019; Low 2021).

The influence of team organisation on the success of a new architecture was also found at Uber, so that autonomous working on different functions was made possible through different technologies. Uber’s self-developed gateway solution enables all teams to document microservices in a comparable way, while different technologies are used in the development of microservices (Thangavelu 2020).

At Uber, it became apparent after the prototype phase that a monolithic architecture was not a solution for the company’s digital business model. Further development and scaling had such a fast pace that work had to be done in different places at the same time. The introduction of the microservices architecture with existing and self-developed technologies led to the desired adaptability of the system.

It should be noted that the microservices architecture significantly increases the number of technologies used. On the one hand, this shows an advantage of microservices, because each developer can decide independently which solution he prefers. On the other hand, however, it also creates a confusing use of technology, which causes additional effort in the communication of services. Uber has recognised this and creates an overview of the dependencies of the 2,200 microservices with the help of DOMA.

The Uber example also shows some disadvantages of the microservices architecture. During the expansion and new development of the services, it was discovered, for example, that in addition to the high number of technologies, the large number of services contributes to the confusion of the system landscape. This increases the effort required to operate the system. Both factors cause problems, especially in communication between the services.

In practice, a constant change in the technologies used can also be seen. Large companies in particular are now increasingly investing in platforms to support microservices architectures – but that is a topic for another article.

Notes:

If you like the article or want to discuss it, please feel free to share it with your network.

Apache Kafka (2021): DOCUMENTATION: Getting Started. https://kafka.apache.org/documentation/#introduction
Chacko, Alan (2022): System Design of Spotify. https://iq.opengenus.org/system-design-of-spotify/
Chen, Lianping (2018): Microservices: Architecting for Continuous Delivery and DevOps. In: 2018 IEEE International Conference on Software Architecture (ICSA): IEEE.
DASA (2021): DASA DEVOPS PRINCIPLES. DevOps is about experiences, ideas and culture to create high-performing IT organizations. https://www.dasa.org/about/dasa-devops-principles/
Gluck, Adam (2020): Introducing Domain-Oriented Microservice Architecture.
Kappagantula, Sahiti (2018): Microservice Architecture — Learn, Build, and Deploy Applications. Get a better understanding of microservice architecture and, as an example of its benefits, how Uber broke down their monolith into microservices. https://dzone.com/articles/microservice-architecture-learn-build-and-deploy-a
Karabey Aksakalli, Işıl; Çelik, Turgay; Can, Ahmet Burak; Teki̇nerdoğan, Bedir (2021): Deployment and communication patterns in microservice architectures: A systematic literature review. In: Journal of Systems and Software 180, S. 111014. DOI: 10.1016/j.jss.2021.111014.
Kwan, Irwin; Cataldo, Marcelo; Damian, Daniela (2012): Conway’s Law Revisited: The Evidence for a Task-Based Perspective. In: IEEE Softw. 29 (1), S. 90–93. DOI: 10.1109/ms.2012.3.
Lauer, Georg; Augsten, Stephan (2018): Microservices – Ein Einstieg in die Praxis. https://www.dev-insider.de/microservices-ein-einstieg-in-die-praxis-a-671171/
Low, Jin (2021): Uber System Architecture. https://medium.com/interviewnoodle/uber-system-architecture-40201134aaea
Namiot Dmitry; Sneps-Sneppe Manfred; Dmitry, Namiot; Manfred, Sneps-Sneppe (2014): On micro-services architecture. In: International Journal of Open Information Technologies 2 (9), S. 24–27. https://cyberleninka.ru/article/n/on-micro-services-architecture
Pfläging, Niels (2014): Organize for complexity. http://www.scrummaster.dk/lib/agileleanlibrary/people/nielspflaeging/organize%20for%20complexity%20lookinside.pdf
Richardson, Chris (2021): What are microservices? https://microservices.io/index.html
Shivang (2019): An Insight Into How Uber Scaled From A Monolith To A Microservice Architecture. https://www.scaleyourapp.com/an-insight-into-how-uber-scaled-from-a-monolith-to-a-microservice-architecture/
Shore, James (2021): The art of agile development. Unter Mitarbeit von Diana Larsen, Gitte Klitgaard und Shane Warden. Second edition. Beijing, Boston, Farnham, Sebastopol, Tokyo: O’Reilly.
Skelton, Matthew; Pais, Manuel; Malan, Ruth (2019): Team topologies. Organizing business and technology teams for fast flow. Portland, Oregon: IT Revolution.
Thangavelu, Madan (2020): Designing Edge Gateway, Uber’s API Lifecycle Management Platform. https://eng.uber.com/gatewayuberapi/
Thorpe, Stefan (2018): Top Tools for Building Microservices on All Levels. These tools come together to help you orchestrate your microservices’ messaging, testing, monitoring, and more. https://dzone.com/articles/30top-tools-for-building-microservices-on-all-leve
Tozzi, Chrsitopher (2022): How to Construct the Ideal DevOps Team Structure. https://www.itprotoday.com/devops/how-construct-ideal-devops-team-structure
Upadhyay, Anu (2021): System Design Netflix – A Complete Architecture. https://www.geeksforgeeks.org/system-design-netflix-a-complete-architecture/
Yang, Yang (2022): Presto® on Apache Kafka® At Uber Scale. https://eng.uber.com/presto-on-apache-kafka-at-uber-scale/

This article is a joint work by Dierk Soellner and Adrian Schaub.

Dierk Soellner has published three other posts on the t2informatik Blog:

What does t2informatik actually do? One click and you'll know.