Episode 393: Jay Kreps on Enterprise Integration Architecture with a Kafka Event Log

Filed in Episodes by on December 18, 2019 0 Comments
Facebooktwitterlinkedin

Jay Kreps, CEO of Confluent discusses an enterprise integration architecture organized around an event log. Robert Blumen spoke with Jay about the N-squared problem of data integration; how LinkedIn tried and failed to solve the integration problem;  the nature of events; the enterprise event schema; schema definition languages; the use of an event log in single node and distributed databases; how the concept of an event log was generalized from a database to the enterprise; the initial development of Kafka; the evolution of Kafka in the last four years; the addition of a SQL-like interface to Kakfa; operational requirements around the use of Kafka as event log; how programs get their events into Kakfa; adding new databases to the Kafak-centric architecture; applications that consume and produce internal events; correctness and how to handle bad events.

Related Links


 

SE Radio theme: “Broken Reality” by Kevin MacLeod (incompetech.com — Licensed under Creative Commons: By Attribution 3.0)

Tags: , , , , , , , , ,