Evolution of Real-time User Engagement Event Consumption at Pinterest | Current 2023

Pinterest handles billions of user engagement events daily that are consumed by hundreds of real time apps via a few Apache Kafka® clusters. In this talk, Lu Liu and Heng Zhang discuss how they used Apache Flink® to standardize their processing logic and improve data quality, leading to multi-million dollar infrastructure savings and increased user engagement.
CHAPTERS
00:00 - Intro
01:26 - What is Pinterest
03:00 - Event Processing Architecture
05:15 - Architecture Evolution
08:42 - Problem Solving
17:55 - Platinum Event Stream
20:53 - Technical Architecture
22:03 - Flink Application
22:45 - Splitter
23:02 - Filtering
23:42 - Splitting
28:29 - Learnings
30:11 - Data Quality
31:09 - Cost Efficiency
32:01 - Conclusion
32:36 - Ongoing Efforts
Speakers: Lu Liu and Heng Zhang
--
ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion - designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.
#current2023 #apachekafka #kafka #confluent