Building a Real-Time Analytics Database • Tim Berglund • GOTO 2023

This presentation was recorded at GOTO Chicago 2023. #GOTOcon #GOTOchgo
gotochgo.com
Tim Berglund - VP DevRel at StarTree & Author of "Gradle Beyond the Basics" ‪@tlberglund‬‪@StarTree‬
ORIGINAL TALK TITLE
Building a Real-Time Analytics Database: A 'Choose Your Own Adventure' Journey
RESOURCES
pinot.apache.org
/ startreedata
/ startreedata
dev.startree.ai
stree.ai/slack
Tim
timberglund.com
/ tlberglund
/ tlberglund
ABSTRACT
Have you ever stopped to think about how to build a database? The thing is, there isn't just one way, as we can see by the massive number of data infrastructure options we have to choose from. It's a nonstop series of tradeoffs, each motivated by the constraints the database wants to satisfy. An in-memory transactional database would be one thing. A general-purpose, single-server relational database would be another. A low-latency, horizontally scalable analytics database would be...the journey we're going to take.
In this talk, we'll start by picking a data model, make decisions about serialization and storage, choose indexing strategies, pick a query language, and figure out how to scale, eventually ending up with something that looks remarkably like Apache Pinot, a real-time analytics database. Pinot was built on a journey like this, always optimized for ultra low-latency, user-facing analytics at scale. In the real world, Pinot is used by applications like LinkedIn and UberEats to expose the state of the system not just to internal decision-makers, but to the users of the system itself, including all of us people who consumers of analytical queries. By focusing on the internals of Pinot and the tradeoffs made along the way to build a database of its kind, we'll see how it enables a new class of applications that every user of a system into a decision maker. [...]
Download slides and read the full abstract here:
gotochgo.com/2...
RECOMMENDED BOOKS
Tim Berglund • Gradle Beyond the Basics • amzn.to/3fSjfMD
Tim Berglund & Matthew McCullough • Building and Testing with Gradle • amzn.to/3VaBY6g
Mark Needham • Building Real-Time Analytics Systems • amzn.to/41AOZJd
Gwen Shapira, Todd Palino, Rajini Sivaram & Krit Petty • Kafka: The Definitive Guide • amzn.to/41AVlrO
Adi Polak • Scaling Machine Learning with Spark • amzn.to/3N9vx1H
/ gotocon
/ goto-
/ gotoconferences
#ApachePinot #Analytics #RealTime #RealTimeAnalytics #TimBerglund #StarTree #StarTreeCloud #Cloud #ApachePinotTutorial #ApachePinotTraining #OLAP #OLTP #LowLatency #ApacheZooKeeper #ApacheHelix #Hadoop #ApacheSpark
Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket at gotopia.tech
Sign up for updates and specials at gotopia.tech/n...
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
www.youtube.co...

[@anytcl]

This really doesnt at all teach you anything about building a real time analytics database
It was an introduction about different database concepts, some of which might be used in real time databases
but he doesn't tell you anything about how to build one

[@don5oan]

+1