How do nested loop, hash, and merge joins work? Databases for Developers Performance #7
Vložit
- čas přidán 25. 06. 2020
- There are three key algorithms use to combine rows from two tables:
* Nested Loops
* Hash Join
* Merge Join
Learn how these work in this video
Need help with SQL?
Ask us over on AskTOM: asktom.oracle.com
Twitter: / chrisrsaxon
Daily SQL Twitter tips: / sqldaily
All Things SQL blog: blogs.oracle.com/sql/
Test your SQL Skills on the Oracle Dev Gym: devgym.oracle.com/
============================
The Magic of SQL with Chris Saxon
Copyright © 2020 Oracle and/or its affiliates. Oracle is a registered trademark of Oracle and/or its affiliates. All rights reserved. Other names may be registered trademarks of their respective owners. Oracle disclaims any warranties or representations as to the accuracy or completeness of this recording, demonstration, and/or written materials (the “Materials”). The Materials are provided “as is” without any warranty of any kind, either express or implied, including without limitation warranties or merchantability, fitness for a particular purpose, and non-infringement. - Věda a technologie
bro, not exaggerating, your way of teaching is legendary, keep making videos on different topics and ur views will reach the top.
Hi, I am really surprised you got so less views. You have explained something I took years to understand. Subscribed.
Thanks Kapil, glad this helped you understand :)
Exactly your videos are highly underrated on youtube
Very much agree your explanation was spot on ...and solid too..
I feel so lucky to have come across this video on my second day of SQL home study. Looks like I saved myself years of mystification/confusion!
You're welcome, glad this helped!
I'm an Oracle DBA for 30 years and your explanation is the best I ever seen !!! Congrats
You're welcome Peter - glad you found this useful!
Thank you very much! This is hands-down the easiest and most concise explanations I've seen!
Great vidéo!
My sum up:
Definitions:
[from other source]The Optimiser will decide which table will be the _inner_ or _outer_ table.
- The outer table is the source of rows to match against the inner table. It is usually read from disk.
- The inner table is the table that is probed for matches. It is usually held in memory, is usually the source table for hashing, and if possible, is the smallest table of the two being joined.
Nested Loops:
TL;DR : good if you only have a small subset of rows to join from the outer table, AND you have an index on the inner table.
For each row in the outer table, it will look for all the row in the inner table. Without any index, you get a complexity of o(number of rows in Table1 * number of rows in Table2).
This is very inefficient, unless you only have a small number of rows from the outer table to join (or a small subset of the outer table to join), AND you have an index on the joining column of the inner table. Thanks to the index of the inner table, the nested loop (that looks for the matching row in the inner table) will be fast. And as there are few rows triggering the nested loop, the whole operation will be pretty Note that if you also have an index for the outer table, finding the subset of rows that we want to join is even faster; but this is optional.
If the number of rows to join from the outer table is small, but not that small, the Optimiser may have a hard time deciding whether to use a Nested Loop or Hash Join, so having up to date statistics is important.
Merge Join:
TL;DR: good even if you have lots of rows to join, but requires an index on the outer table to be efficient.
Both tables are first sorted by the joining column (¿sorted in memory?), and then joined. The sorting allows to read the the inner table bit by bit : the RDMS reeds the 1st row from the sorted outer table, and then looks in the inner table for matching rows. When the next row no longer matches, it means that there won’t be any other matching row, as the table is sorted. This allows to only have to read each table once. Merge joins are thus efficient even if many rows have to be joined. But the sorting operation is expensive… Sorting both tables means a complexity roughly of o(#Table1 * Log #Table1 + #Table2 * Log #Table2)
However, if you have an index for the outer table (on the joining column), then only the inner table will have to be sorted. Indeed, indexes are sorted, so the RDBMS will use the outer table index to read it in a sorted manner. Sadly, even if you have an index for the inner table, Oracle DB will still have to sort it [10:28].
Hash Join:
TL;DR: works only for equality joins (e.g. not for “joincolumTable1 > joincolumnTable2”). Good for large amount of rows to join. It is the most efficient most of the time. Complexity is o(number of rows in Table1 + number of rows in Table2)
A hash table of table1 is computed in memory, and then the value of each of table2 is hashed, and an equality of hash value is looked for in the hash table.
This is amazing how you explain and make it simple. Thank you very much!
Woah this is good stuff. Easily one of the best explaination I seen in a long time. Clear and Concise. The anim also nicely done. Totally can visualise and relate in split seconds. Kudos!
You're welcome, glad you found this useful :)
Great explanation! A word of note for those who were also not understanding (like me) due to the suit system ; The suit strength goes clubs < diamonds < hearts < spades
I just found out that apparently that there's two systems of suit strength :
clubs < diamonds < hearts < spades (alphabetical)
and
diamonds < clubs < hearts < spades (alternating)
I know the latter. I didn't know the alphabetical order even existed.
I was just ordering alphabetically for this video! Thanks for digging out the other suit sorting method
Perfect video! You can explain things that everyone can understand it!
Been doing MSSQL for 20 years and this was by far the best explanation of these joins. Thanks!
You're welcome!
I am incredibly grateful to Chris for their invaluable tutorials on hash join, merge join, and nested loop. These concepts always seemed daunting to me when I encountered them in execution plans, but their clear and concise explanations have helped me gain a much deeper understanding. Their expertise and dedication to educating others are truly commendable. A big thank you for demystifying these complex topics and making them accessible to all!
You're welcome
Excellent explanation of join strategies. And your use of decks of cards helps to visualize each strategy in your head for a better understanding of each one. Thank you.
You're welcome, glad you found this useful :)
How you can explain complex matter and in the entertaining way ?! Congratulations ! You nailed it !
You're welcome, glad you found these useful and enjoyable :)
Great explanation and the example with the deck of cards is brilliant
Excellent explaination Chris Saxon
Fantastic explanation. You made it so easy to understand . Thanks
Excellent Explanation. I don't think even those who wrote the optimizer's algorithms could have explained it this lucid and simple.
Thanks! Glad you found this informative :)
Great explanation! Thank you so much for sharing your knowledge.
First time came across such good explanation of these 3 kind of joins. Kudos !!!
This is a core question of DBA Interviews. Very well explained. Heartiest thanks Sir
You're welcome; glad this helps!
This channel is pure gold. Thx a ton.
Thanks :)
This is an amazing level of detail. I'm so glad I found your channel. It is priceless. Thank you so much for sharing all this knowledge with us.
You're welcome, great to hear you find this useful :)
Best explanation! Thank you sir!
Your content (and presentation) is excellent, thank you for your work!
You're welcome; glad you found this useful
Wow, Easy to understand with a detailed explanation. Thank you
You're welcome, thanks!
It's truely magical. Thank you
that was super easy to understand such complicated concepts and with good English for all people. Thank you
You're welcome, thanks!
The best explanation that I have ever seen.
This is really magic , I could understand all of it with an engaging interest. Thanks for sharing.
Great to hear :)
Probalby the best explanation on this I have ever seen
Absolutely Loved the way you explained. More power to you. Subscribed. :)
Thanks very much for the detailed study.
Thank you so much. You make my day. Best explanation ever.
this is freaking awesome i learnt something in depth one of the best channel i have seen so far
i think using hints we can command the optimizer we want this join type while doing certain query using pinned explain plan
Thanks!
You have to be careful using hints. To ensure the optimizer picks a particular plan, typically you need LOTS of hints to ensure it does this. When using Oracle Database we recommend you use SQL Plan Management (SQL profiles & baselines) to manage plans instead.
Such a good video!! Finally making Joins make sense for me!
Glad this helped :)
This was really good, thank you.
very informative, easily and affectively explains the three join strategies
Great, glad you found this informative :)
Thanks to yt and you for recommending and giving a beautiful explanation on this topic ❤️ choosing cards was best....was unable to understand from theory session from various sources
Thanks and welcome
Great explanation! Thank You
this video helps me to understand the joining strategies as well helps to make sense & reason out the usage of some of the complex concepts used in Apache Spark, fortunate to find this video, fantastic explanation, very easy to understand the concepts that felt really confusing, thank you very much :)
You're welcome! Glad you found this useful
This is one the bests videos about this subject. Thank you a lot!!!
Thanks; glad it helped
Very nice Chris. I’ve watched some other video in the past from Connor, but this one is much more elaborated. I was wondering if you could explain in one video of this series about bloom filters. Thank you again for doing this. Excellent work.
Thanks Franky! I've added bloom filters to my list of things to create videos about ;)
This was a very good explanation. Thanks!
Best explanation for joins ever 👍👍
This is really outstanding session.
Thanks bro , Excellent Explanation
this was extremely useful! Thank you!
such clear explanation, thank you sir
very simple explaining..good work bro
This might be the first comment that I have ever make in youtube :) I am already familiar to join types but watching this was quite nice. you explained it so simple and I love it Chris 👍👍👍. I would like to add this video to my blog also if it is okay for you. Nice moustache by the way :)
Cool, glad you enjoyed this
wow this is awesome! Thanks for sharing this video.
Thank you so much for this video. It really helped me for my interview.
You're welcome; glad this helped you
Great video, thank you.
Awesome way of explaining the joins 👍.
You're welcome, thanks!
such a pleasure to watch, thanks for the video
You're welcome, glad you enjoyed it!
awesome, great and terrefic explanation. so underrated !
Thanks!
i always confused in this, But after your card explaction i understand way u explained. Subscribed
You're welcome!
this is super good. just for my understanding, you said when we just have few card (5) from outer deck to match nested loop becomes faster as it can start matching from the first card however, hash join still needs to create hash table for all 52 cards from outer table. why should it create hash table for 52 instead of 5, assuming the filter is already applied and it knows those 5 rows already.
Thanks
Perhaps I wasn't clear on this - I was thinking of a Top-N, "get the first 5 rows then stop" query. Instead of a where clause that only matches 5 rows, the result set (could) be bigger. But we'll stop as soon as we've returned 5 rows.
Because nested loops join rows immediately, it can stop as soon as it reads 5 rows from the outer table (assuming they all join to a row in the inner table). A hash join always builds the hash table on the whole outer table first.
If the where clause identifies 5 rows from the outer table, you're right the hash join will only build on these 5 rows.
It was unclear for me too, thanks for the explanation.
Absolutely incredible Big Cheer to this guy
your explanation is beyond amazing bravo
Thanks; glad you found this useful!
Absolute gem!!
Thank you for well explained. Excellent!
You're welcome, thanks!
Nicely explained!
Thanks a lot for your awesome videos!
Very Good explanation
thank you, highly appreciated
This is GOLD ❤❤❤❤
this was magic thank you
This video deserves more views
Thanks, glad you found it useful :)
Excellent, Thank you :)
Great learning, Thank you.
You're welcome!
Awesome video! Love it!!!
Perfect explanation👏🏻
Thanks!
Great vid 😀
Saved my a**. Really nice video.
Great content
Really wonderful!
Very well explained !
You have got less likes .. your video deserves lot of likes and appreciation as Its really outstanding👌🏼
You're welcome, glad you enjoyed this - please share it with anyone you think would benefit!
excellent video
Great video
usefull explanation ! thanks a lot!
I don't think anyone can explain better than this.
Very nice video 👍
you nailed it👌👌
Thank you so much.
You're welcome!
I'm lucky to found this treasure.
Awesome Explanation.
I said "Yaaar Kamaaal" when watching this video.
Which mean " Dudeee, Terriffic"
:) Great to hear!
you are very good teacher !!!!!
Thanks :)
Excellent thanks alot
You're welcome
I got addicted to see deeper and deeper in SQL now Thanks for the such nice explanation
Thank you for this clear explanation! Subscribed! One question: if both tables are indexed and we use a Merge Join, you said that it would still sort the second table. Could you elaborate on the reason behind it? Excellent content!
"It just does!" Sorry, I don't know the exact reason why Oracle Database always sorts the second table. It may be that this changes in the future.
U r a gem
when we compare the complexity we only compared based on the number of matches however the hash join also needs to build hash table, how can we estimate the cost of that to compare with other two.
Thank you for a clear explanation! But I am still wondering why the merge sort needs to start on the previous value if we know it was already joined with the last value from the outer deck. Is there something I have missed? Thanks :)
There could be many rows with the same value. Going back to the previous value is simple way to ensure you always capture them all. It also works for greater/less than comparisons - if the join is C1 > C2 then you'll be on the last row/value in the inner deck after the first row from the outer. But likely need to revisit most of the rows in the inner deck when you go to the second in the outer.
Refreshing
this video shd be viral among all advance database students
Then please share it :)
God level..
👏🏻👏🏻👏🏻
Finest video on this topic. How is it possible that it was hidden for so long? Just a suggestion, I think you should include NESTED Loops, Merge join, Hash join in the description so more people searching on youtube will come across this. include more hashtags and all. Thanks for the video, if possible please explain REGEX.
Thanks - these terms are already in the description though; what exactly are you suggesting I do differently?
REGEX is a big topic! Maybe I'll cover it one day ;)
@@TheMagicofSQL sry I wrote description, I meant video title.
I have a question, when you say to sort the values of the two decks, you mean by creating a clustered index on each bucket or am i missing something?
No - just that the database has to order all the rows from each table on the join columns. Like when you add an ORDER BY to a SELECT statement.