Request-Response with REST/HTTP vs. Data Streaming with Apache Kafka

Request-response communication with REST / HTTP is simple, well understood, and supported by most technologies, products, and SaaS cloud services. Contrarily, data streaming with Apache Kafka is a fundamental change to process data continuously. HTTP and Kafka complement each other in various ways. This post explores the architectures and use cases to leverage request-response together with data streaming in the control plane for management or in the data plane for producing and consuming events.

(Originally posted on Kai Waehner’s blog: “Request-Response with REST/HTTP vs. Data Streaming with Apache Kafka”… Stay informed about new blog posts by subscribing to my newsletter)

Request-response (HTTP) versus data streaming (Apache Kafka)

Prior to discussing the relationship between HTTP/REST and Apache Kafka, let’s explore the concepts behind both. Traditionally, request-response and data streaming are two different paradigms.

Request-response with REST/HTTP

The following characteristics make HTTP so prevalent in software engineering for request-response (aka request-reply) communication:

• The foundation of data communication for the World Wide Web
• The standard application layer protocol in the internet protocol suite, commonly known as TCP/IP
• Simple and well understood
• Supported by most open source frameworks, proprietary products, and SaaS cloud services
• Pre-defined API with GET, POST, PUT, and DELETE commands
• Typically synchronous communication, but chunked transfer encoding (i.e., streaming) is also possible
• Point-to-point message exchange between two applications (like a client and server or two independent microservices)

Data streaming with Apache Kafka

HTTP is about communication between two applications. On the contrary, data streaming is much more than just data communication between a client and a server. Hence, data streaming platforms like Apache Kafka have very different characteristics: