Episode 424: Sean Knapp on Dataflow Pipeline Automation

Filed in Episodes by on September 2, 2020 0 Comments

Sean Knapp, CEO of Ascend.io, discusses data pipelines and data pipeline automation.  Sean spoke with Host Robert Blumen about the ubiquity of data pipelines; what data pipelines do; where the data comes from, how it is transformed, where it goes; and what it is used for (analytics, machine learning, reporting, alerting, business intelligence).  Semi-automated and ad-hoc automation.  Costly manual recovery from failure modes.  Partial failures and bulk redo.  Pipeline automation.  Why automate?  The orchestration layer; architecture of the orchestration layer.  What type of state does the orchestration layer keep? Failure modes and optimizing redo.  Monitoring pipelines.  Privacy and pipelines.  Pipeline automation-as a-services.

Related Links


SE Radio theme: “Broken Reality” by Kevin MacLeod (incompetech.com — Licensed under Creative Commons: By Attribution 3.0)

Tags: , , , , , , ,