Arpit Bhayani
Arpit Bhayani
  • 189
  • 2 334 989
How Converged Indexes makes all kinds of queries efficient | Rockset Internals
System Design for SDE-2 and above: arpitbhayani.me/masterclass
System Design for Beginners: arpitbhayani.me/sys-design
Redis Internals: arpitbhayani.me/redis
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani
# Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: czcams.com/video/o7qLKfILuD8/video.html
Designing Microservices: czcams.com/video/JPj6mhVLQN0/video.html
Database Engineering: czcams.com/video/-htbah3eCYg/video.html&pp=gAQBiAQB
Concurrency In-depth: czcams.com/video/2PjlaUnrAMQ/video.html
Research paper dissections: czcams.com/video/LXhgFAZroG8/video.html
Outage Dissections: czcams.com/video/LeT_s-UFw-U/video.html
Hash Table Internals: czcams.com/video/jjW8w8ED3Ns/video.html
BitTorrent Internals: czcams.com/video/v7cR0ZolaUA/video.html
# Things you will find amusing
Knowledge Base: arpitbhayani.me/knowledge-base
Bookshelf: arpitbhayani.me/bookshelf
Papershelf: arpitbhayani.me/papershelf
# Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: linkedin.com/in/arpitbhayani
Twitter: arpit_bhayani
Weekly Newsletter: arpit.substack.com
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.
zhlédnutí: 3 145

Video

Architecture deep-dive of a Real-time Analytics Database | Rockset Internals
zhlédnutí 8KPřed 21 hodinou
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani Kicking off a super interesting series on the internals of Rockset - a real-time search and analytic...
How JioCinema live streams IPL to 20 million concurrent devices w/ Prachi Sharma | Ep 7
zhlédnutí 140KPřed 14 dny
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani # Recommended videos and playlists If you liked this video, you will find the following videos and p...
How Reddit designed their metadata store to serve 100k req/sec at p99 of 17ms
zhlédnutí 23KPřed 21 dnem
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani # Recommended videos and playlists If you liked this video, you will find the following videos and p...
How PostgreSQL generates all possible query execution plans before choosing the best one.
zhlédnutí 21KPřed 28 dny
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani # Recommended videos and playlists If you liked this video, you will find the following videos and p...
How nested loop, hash, and merge joins work.
zhlédnutí 25KPřed měsícem
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In this video, I delved into the essential concept of joins in databases, exploring the algorithms u...
How PostgreSQL parses the query and constructs the Parse Tree?
zhlédnutí 18KPřed měsícem
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In the previous video, we delved into the client connection process in the PostgreSQL database. In t...
PostgreSQL connection management and per-client process model explained
zhlédnutí 16KPřed měsícem
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In this video, I delved deep into Postgres' connection management, exploring its unique process per ...
How DNS really works and how it scales infinitely?
zhlédnutí 18KPřed měsícem
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In the video, I delved into how DNS works, from converting human-readable domain names like google.c...
Understanding Proxy, Forward Proxy, and Reverse Proxy
zhlédnutí 23KPřed 2 měsíci
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In this video, I delved into the concept of proxies, explaining their purpose, types, and practical ...
What happens when you type a URL into your browser?
zhlédnutí 19KPřed 2 měsíci
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In this video, I delved into the fascinating process of what happens when we type a URL into our bro...
Everything you need to know about CockroachDB w/ Ben Darnell | Ep 6
zhlédnutí 12KPřed 2 měsíci
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In a video, I shared my journey of enhancing my Go programming skills by delving into CockroachDB's ...
What exactly is the HTTP protocol and how to write a new one from scratch?
zhlédnutí 20KPřed 2 měsíci
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In this video, I delved into the concept of protocols, focusing on HTTP as a widely used example. I ...
How indexes work in Distributed Databases, their trade-offs, and challenges
zhlédnutí 17KPřed 3 měsíci
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In the video, I explained how indexing works in a distributed database like DynamoDB and the challen...
Understanding Database Trade-offs - The RUM Conjecture - Paper Explained
zhlédnutí 10KPřed 3 měsíci
System Design for SDE-2 and above: arpitbhayani.me/masterclass System Design for Beginners: arpitbhayani.me/sys-design Redis Internals: arpitbhayani.me/redis Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters. Sign up and get 40% off - app.codecrafters.io/join?via=arpitbbhayani In the video, I dissect a research paper called "The Rum Conjecture," which explores the trade-offs ...
The Google File System - Paper Explained
zhlédnutí 20KPřed 3 měsíci
The Google File System - Paper Explained
Why thread pools even exist? and how to implement them?
zhlédnutí 30KPřed 3 měsíci
Why thread pools even exist? and how to implement them?
Picking the right string column datatype - CHAR vs VARCHAR vs TEXT
zhlédnutí 11KPřed 4 měsíci
Picking the right string column datatype - CHAR vs VARCHAR vs TEXT
The compression algorithm that powers all Time-series Databases
zhlédnutí 8KPřed 4 měsíci
The compression algorithm that powers all Time-series Databases
How databases store and transmit integers efficiently using varint encoding
zhlédnutí 9KPřed 4 měsíci
How databases store and transmit integers efficiently using varint encoding
How Pass by Value and Pass By Pointer work internally? Peeking into the assembly code to find out
zhlédnutí 11KPřed 4 měsíci
How Pass by Value and Pass By Pointer work internally? Peeking into the assembly code to find out
How to start your writing journey as a software engineer
zhlédnutí 21KPřed 6 měsíci
How to start your writing journey as a software engineer
Everything you need to know about Read Uncommitted Isolation Level
zhlédnutí 7KPřed 6 měsíci
Everything you need to know about Read Uncommitted Isolation Level
Disabling Garbage Collection helped Instagram save money and get better performance
zhlédnutí 20KPřed 7 měsíci
Disabling Garbage Collection helped Instagram save money and get better performance
How Google designed their Highly Available Load Balancers using Anycast
zhlédnutí 14KPřed 7 měsíci
How Google designed their Highly Available Load Balancers using Anycast
CAP Theorem - From the First Principles
zhlédnutí 22KPřed 7 měsíci
CAP Theorem - From the First Principles
How Dukaan moved out of Cloud and on to Bare Metal w/ Subhash | Ep 5
zhlédnutí 189KPřed 7 měsíci
How Dukaan moved out of Cloud and on to Bare Metal w/ Subhash | Ep 5
Navigating Massive Codebases - From the First Principles
zhlédnutí 7KPřed 8 měsíci
Navigating Massive Codebases - From the First Principles
Bloom Filters - From the First Principles
zhlédnutí 6KPřed 8 měsíci
Bloom Filters - From the First Principles
If I were to learn a new programming language
zhlédnutí 21KPřed 9 měsíci
If I were to learn a new programming language

Komentáře

  • @heyhelloitsme
    @heyhelloitsme Před hodinou

    Excellent discussion. Very apt questions and deep routed knowledge answers.

  • @aakashvaghela7365
    @aakashvaghela7365 Před 2 hodinami

    what happens when we refresh a page??

    • @AsliEngineering
      @AsliEngineering Před 34 minutami

      Depends on what has been cached locally. Worst case, the exact same flow.

  • @mystiqkc
    @mystiqkc Před 5 hodinami

    I looove this kind of explanation. i.e stepping back and discussing the scenarios behind why something came up to be. Thanks a lot for these videos man!

  • @vikasarya7329
    @vikasarya7329 Před 6 hodinami

    i literally checked my playback back speed.

  • @valerianpereira4691
    @valerianpereira4691 Před 13 hodinami

    Energy packed podcast by Subhash

  • @saswatsaubhagyarout3480
    @saswatsaubhagyarout3480 Před 15 hodinami

    What a video! mind-blowing. Thank you for this.

  • @n-keerthi
    @n-keerthi Před 17 hodinami

    On jio cinema its number of views altogether, not concurrent views as it was in hot star. This is how people are fooled. In one match it was showing 48Cr. I'm damn sure ipl has no that many audience

  • @Polbose
    @Polbose Před 21 hodinou

    One of a kind, always great content!

  • @rahulsangvikar7973
    @rahulsangvikar7973 Před 22 hodinami

    Absolutely amazing video, however, I'd like to point out a slightly different way of writing a lock-free fair implementation. Say you're using n threads. Start ith thread from i, and increment the counter by n each time. This way you will always calculate isPrime for a unique number in all the threads and the numbers will be fairly distributed among all of them.

  • @rahulsangvikar7973
    @rahulsangvikar7973 Před 22 hodinami

    Why do we need 2 mov micro instructions? Why couldn't we just use something like addl $1, %rdi as a single micro instruction? Also, adding just 1 to a number "binarily" is extremely efficient right? It's much faster than adding any other number, so why couldn't we handle this as a special case?

  • @avinash21899
    @avinash21899 Před dnem

    18th feb 2024 3rd test day 4, india 2nd inning the viewers showing on tv screen was 52.8Cr. can someone explain this. I have a picture of this.

  • @samj9822
    @samj9822 Před dnem

    Did you get coding questions when you were interviewed for Staff role at Google ? I am curious as you were already so senior at Unacademy at that time. Have you shared your interview experience anywhere, would like to refer. Shall be thankful for your response.......

  • @venky1922
    @venky1922 Před dnem

    Solid Podcast Mind blowing..

  • @tarangchikhalia
    @tarangchikhalia Před dnem

    Thanks for this amazing explanation. I am curious to know how the insertion and deletion is handled. You mentioned the index is ordered. What happens if the new object is added/deleted. Does the index reorder? Is it using the linkedlist data structure for the index to make the insertion/deletion efficient?

    • @AsliEngineering
      @AsliEngineering Před dnem

      RocksDB is the storage layer. Read about LSM trees. Gist: buffered in mem hash table and flushed on disk as KV pairs ordered by keys, these are SSTables. Everytime a new SST files are created and then multiple are merged in O(n). To go deeper, read about LSM trees.

    • @tarangchikhalia
      @tarangchikhalia Před dnem

      Thanks @@AsliEngineering. I will look into it..

  • @user-fi1vj7dl6p
    @user-fi1vj7dl6p Před dnem

    by far the best one with so much problem statements and explained in a nice and proficient manner

  • @HusainSyedAli
    @HusainSyedAli Před dnem

    What a session🎉, lot more details in a less time, we wish more these kind of sessions across all other company cloud operations, thanks Team, great job.

  • @kuldeepaher4937
    @kuldeepaher4937 Před dnem

    Thala for a reason

  • @42Siren
    @42Siren Před dnem

    This is so amazing !

  • @imhiteshgarg
    @imhiteshgarg Před dnem

    Hi Arpit, Thanks for this video and your replies on my curiousities. In this video, I could observe that the problem started when we added too many nodes too quickly in the cluster. So the process should be as follows? - Add a node, wait for it to find zookeeper node as cluster leader. - Only when new node has registered its zookeeper node, then we add another node so that no second leader can be elected?

  • @rajsrisg
    @rajsrisg Před dnem

    Wonderful video Arpit. Prachi was amazing in her explanations. Great asset to JioCinema.

  • @imhiteshgarg
    @imhiteshgarg Před dnem

    Hi Arpit, Nice video as always. I just wanted to ask that if we introduce message brokers between every service then doesn't it cause delays in complete processing of a request?

    • @AsliEngineering
      @AsliEngineering Před dnem

      Yes. It does. And you do it when it is to do and process stuff asynchronously.

  • @Miles-co5xm
    @Miles-co5xm Před dnem

    This is the answer to over engineering of things which don't even go live.

  • @ishajindal7862
    @ishajindal7862 Před dnem

    Hi Arpit, How does the aggregator know which leaf node can execute a particular query? Are all leaf nodes able to execute all the queries?

    • @AsliEngineering
      @AsliEngineering Před dnem

      They fan out across all relevant leaf nodes. I will be discussing this very same thing in the next video. This looks counter intuitive, but the tradeoff makes sense.

  • @LeoLeo-nx5gi
    @LeoLeo-nx5gi Před dnem

    Hi Arpit, thanks a ton!! You explained it in such an awesome way!! Thanks for such grt insights....

  • @SahilBhosale08
    @SahilBhosale08 Před dnem

    Thanks for this amazing podcast , this is the best content and best insights

  • @abhishekkrishna5978

    He is already talking in 2x

  • @sourabhhsethii7292

    Classic!

  • @srijanverma5238
    @srijanverma5238 Před 2 dny

    great video @AsliEngineering ! One question - consider a scenario in which an "election" is in progress via LCR algorithm. The node with the highest UUID dies soon after passing its message. That means, the highest UUID keeps circling (however, its node does not exist) . What happens in this case ?

  • @CyberSoundStation
    @CyberSoundStation Před 2 dny

    Thanks for explaining. It looks like the converged indexing thing slows down writes more than regular indexing because it's maintaining columnar, row, and search indexes. But it makes the query execution engine happy, getting all the good stuff from both indexing schemes.

  • @saijayavinothtvs2588

    Thanks...have a check on StarTree Index

  • @AsliEngineering
    @AsliEngineering Před 2 dny

    Here's the link to Rockset, give it a shot - rockset.com/

  • @AsliEngineering
    @AsliEngineering Před 2 dny

    Here's the link to Rockset, give it a shot - rockset.com/

  • @vipulsharma5140
    @vipulsharma5140 Před 2 dny

    This is such a great and simple explanation of partitioning and sharding of a database. Would love to watch the video on partitioning strategies when it is uploaded.

  • @nagesh007
    @nagesh007 Před 2 dny

    Nice

  • @user-pb2wh8iw9m
    @user-pb2wh8iw9m Před 2 dny

    I tried ignoring it several times as voice pace is too fast of the "Genius Guy Subhash" but always getting stick to this podcast :p Totally insane, soch raha hu jeevan waste kar diya apna maine faltu ke kaamon mein being a developer

  • @f.e.febriansyahpusatedukas8816

    15:15 im still dont get the urgent reason (or each advantages) of using the endianess

  • @prateeknayak-si9ik
    @prateeknayak-si9ik Před 2 dny

    @arpit if you can get sumone from zerodha or any broker to understand trading platform system design

  • @nagesh007
    @nagesh007 Před 2 dny

    Nice

  • @nagesh007
    @nagesh007 Před 2 dny

    Nice

  • @nagesh007
    @nagesh007 Před 2 dny

    Awesome 😍

  • @xikusyaa
    @xikusyaa Před 2 dny

    Jump start is really amazing part!

  • @timepasschannel3163

    Missing Dish TV days 😢

  • @IOFLOOD
    @IOFLOOD Před 2 dny

    It's been great working with Subhash. Really glad to be a part of his impressive operation.

  • @advaitdocvita
    @advaitdocvita Před 2 dny

    Heyy arpit can you please make a video on how to choose just one thing and master it. Like I want to learn Golang and then move to system design and in depth concepts for backends so can you please make a structured video for the same ? Would be great 😃

    • @AsliEngineering
      @AsliEngineering Před 2 dny

      There is no secret sauce. Just pick one and learn without getting distracted. It is easy to be distracted with so much going around. But you really need to be focused on one and learning it from a single resource or at max two resources. For example, I am so tempted to do and learn so many things right now, but I have parked all of them and putting my entire energy in learning concurrency in-and-out in Python. Hope this helps. btw, the video would have been this one line :)

    • @advaitdocvita
      @advaitdocvita Před 2 dny

      @@AsliEngineering sure Arpit thank you so much for this 😃 Will stick to golang and learn all the concepts.. also great videos from you, sir love it..

  • @chiragjain9575
    @chiragjain9575 Před 2 dny

    Thala for a reason

  • @nagesh007
    @nagesh007 Před 3 dny

    Awesome 😍

  • @ajayChauhan-nt5xm
    @ajayChauhan-nt5xm Před 3 dny

    23:00 so the feature is just P2 if customer are not complaining. For us at Games24x7 everything is P0, user experience is everything. Good to know few of the strategies share, kudo for organising this.

  • @Aditya_Vyas
    @Aditya_Vyas Před 3 dny

    Arpit never forgets it ❤