Three real world data pipelines in action - workflow automation with Stonebranch

Sdílet
Vložit
  • čas přidán 29. 07. 2024
  • In this video, I explore three real-world data pipelines used by major companies, thanks to Stonebranch's workflow automation platform. See how data is transferred, validated, and processed across AWS, Azure, SAP, Databricks, Snowflake, Informatica and more. Stonebranch is a great alternative to Airflow, but it also works together with Airflow.
    ▶ Success Stories: bit.ly/3RIaaWW
    ▶ Data Engineering Academy: bit.ly/3R13ao0
    Timestamps:
    00:00 Introduction
    00:53 Stonebranch introduction
    02:17 Architecture
    03:29 Agents & firewalls explained
    05:21 From dev to production
    06:11 Tasks & Integrations
    08:04 In-memory streaming
    08:43 Task triggers & scheduling
    09:30 Event Driven Workflows
    12:10 Pipeline 1: Data Ops example
    16:19 Pipeline 2: SAP commissions & Slack approval
    18:57 Pipeline 3: Huge pipeline with mainframe & cloud
    22:14 Starting & Monitoring Pipelines
    24:56 Takeaways - pros & cons
    28:10 Call for comments
    #dataengineering #workflowautomation #bigdata
  • Věda a technologie

Komentáře • 6