Kafka - the asynchronous microservices runtime for state, scale and performance (N. Avery)

Today's world operates under a new set of constraints and tries to solve problems that were never thought possible. Everything has changed. In this talk, I discuss the origins of how Apache Kafka came to be, but also how it now underpins some of the worlds largest microservice eco systems. Kafka and the microservice moment gathered momentum at the same time, Kafka for streams or data-in-motion and microservices as a new, fine grained architectural principle. The thorn in the side for microservices has always been about the ability to handle, state, and scale. Kafka, meanwhile has always been good at distributing state, which means, that now as we build microservices on top of Kafka, there is a natural affinity between them. I will cover the evolution of microservices until the point of today and then work through how we can build stateful scalable microservices using Kafka. I will break down how scaling state works, and also how it can be queried with ‘interactive queries’. Lastly, I cover key architectural principles of: o Scalable business processing o Run it on rails: Instrumentation and monitoring o Control flow patterns (start, stop, pause) o Error handling strategies