Explaining EVERY Sorting Algorithm (part 1)
Vložit
- čas přidán 19. 05. 2024
- There are lots of sorting algorithms and some of them are hard to understand, so in this series I will explain all of them, starting in part 1 with those that I consider most important to understand.
corrections/clarifications: none so far
Chapters:
0:00 Intro
1:04 Selection Sort
1:35 Double Selection Sort
2:01 Insertion Sort
2:38 Binary Insertion Sort
3:27 Bubble Sort
3:59 Shaker Sort
4:21 Asymptotic Notation
7:11 Finding Time Complexity
8:57 Quick Sort
11:22 Merge Sort
12:40 Stability
13:42 Space Complexity
15:28 Heap Sort
18:17 Comb Sort
19:35 Shell Sort
20:59 Radix Sort
24:59 MSD Radix Sort
25:42 Bucket Sort
28:34 Counting Sort
29:52 Spaghetti Sort
30:34 Gravity Sort
32:00 Pancake Sort
33:15 Bogo Sort
34:25 Outro - Věda a technologie
twitter.com/kuvina_4
Hybrid sort - a combination of quick sort, heap sort and insertion sort, see std::sort
Timsort - a hybrid of merger sort and insertion sort
strings in radex sort.
But can J do 8t myself
?
My favorite sorting algorithm is "don't sort". Its where you figure out a way to solve your problem without sorting the list
Redefine Sort:
step 1: redefine the sort qualifications so that the list you have is sorted
step 2: marvel at your "sorted" list
Delete Sort
step 1: delete all data
step 2: claim that it's all sorted
step 3: realize that you could say the same for it being not sorted
step 4: panic
Replace Sort
Step 1: replace the data with the set 1, 2, 3
step 2: profit
@@therealelement75no the empty list is definitely sorted
Step 1 do step two
Step 2 refer back to step number one
Or
Step 1:Go to step 3 ignore step 2
Step 2:go to step 1 ignore step 3
Step 3:go to step 2 ignore step 1
I propose what I have heard referred to as "Stalin Sort":
in O(n) remove any element which would cause the array to be unsorted.
Unlike most sorting algorithms it doesn't preserve the contents of an array, but the result is indeed sorted.
you've heard of merge sort, now get ready for purge sort
@@Kuvinaneck yeah, it's purging time!
Lol. This sounds like an algorithm created by an AI when you don't specify the requirements explicitly.
Order(1) sort: set array size to 0.
You know, I wonder, what would happen if, instead of deleting the list forever, you stored the deleted values to another array? Then recursively sorted that purged list, until you're left with a bunch of sublists. Then, simply insert each element in every sublist in the list from the recursion before it, until eventually the list is sorted?
I think my favorite sorting algorithm is Miracle Sort. The steps are as follows:
Check if the array is sorted. If not, check if the array is sorted. Repeat until sorted.
Also heck yeah, enbies represent! Happy Pride!
Mfw random radiation-induced bit flips
@@DaemonWorx its best case time complexity is still O(1) because there's always that small chance radiation flips all the bits in perfect order
You'd have to recreate so many SM64 solar bit mutations in order for that to be even a fraction as viable as bogosort
@@DMZZ_DZDMit makes bogobogosort look like the speed of light
We need to have Quantum Bogosort in the next one; it's too good of a joke algorithm to not include. Great video! :D
Yeah it's like "what? you checked them one at a time? Just do every possible permutation simultaneously and grab the sorted one ez"
@@warriorsabe1792 Wait, I thought it was "shuffle list, then if not sorted, destroy the universe". Assuming the Many-Worlds interpretation of the universe holds, then all universes where the shuffle did not sort the list will be destroyed, so all surviving universes will have the list sorted in O(n) time and O(1) space complexity.
I watched this expecting it
69th like
@@KinuTheDragonin some cases it's O(1). Just don't check if it's sorted, assume it is, and only destroy the universe if there is an error.
Pattern defeating quicksort. Also if you're looking for "joke" ones: sleepsort. It'd may be interesting to go into algorithms designed with L1/L2 cache and branches/branchless in mind.
For optimizing cache you'll want to look into "Cache-oblivious algorithms" which is a way of analyzing loads & stores in big O notation (and probabilistic cache misses), the currently most known optimal algorithm for this is Funnelsort (which is a small improvement on Merge sort). Really Quick Sort & Merge Sort are both nearly optimal (which you probably guessed) but at least we have a model to tell us "why" quick sort is so good.
In Bogo Sort, instead of permuting the elements randomly, you could just permute the elements sequencially (think of permuting the indexes in lexicographical order, then checking if the reordering sorts the list). It is still O(x!), but at least is guaranteed to sort in finite time.
It's like the fourth time I rewatch that series, seriously this is the best resource to intermediary understanding of sorting.
Binary insertion is actually like one of the if not the best way to sort physical objects, because the reason it is slowed down is shifting is O(n), but with real objects, you can shift everything at once for O(1), like in sorting cards you look for the right spot to put a card and then just stick it in there.
with all this knowledge i learned that the optimal sorting algorithem for all these animations is simply returning list(range(n)) since after assuming so many wonderful things (n uniform whole numbers between 1 and n) you already know the answer
I come here to learn more about the radix sort, but end up watching all of the video. It is very interesting, thank you for making this
wow this video was super interesting! I somewhat recently programmed an application that sorted tiles in a 2D grid, and coding a fair amount of algorithms made me feel like I at least had a somewhat good grasp of how everything worked, but seeing this video I was really surprised at how deep the subject is, and how much I was missing! Amazing video!
That was a great explanation of big O notation, which doesn't make it sound illogical!
Oh my gosh, I was just getting recommended a bunch of sorting algorithm ASMR(?) videos and was trying to find something to explain each of the sorting algorithms, but none were really helping me.
Haven't watched the video yet, but you've done such a good job explaining other complex topics, that I'm pretty sure this one will be of great help to me.
I look forward to the next part!
There's another algorithm I'm not sure is in the end list (since I don't remember the name), but it had to do with a _very particular_ sort of memory device that was able to use chains of charge pump cells to be able to rapidly interleave and shuffle lists together. The algorithm wasn't particularly good asymptotically, but the amount of silicon gate area required for each cell was just _tiny_ and the device could be run at clock speeds fast enough to more than keep up in practical terms.
kinda makes me think of what hardware might look like for impractical but asymptotically or even thermodynamically optimal sorting if foregoing solid state
I couldn't help but think of Liam Neeson…
A very particular sort of memory. The sort of memory that's a nightmare for unsorted lists…
Commenting for the algorithm!
The sorting algorithm
The space/time issue was a huge one for me when I did embedded programming. I used to write code for things as slow as 8mhz 8-bit hardware, with a few kb to a few hundred kb of ram. Merge sort was the usual candidate, but there were actually times that we would say stability be damned and free memory addresses from those twin sorted arrays as soon as they were in the final array, and then just return the final array.
I'm literally playing a programming game right now and I'm thinking which sorting algorithm is the easiest to implement. It could never be a better timing for this video to come out lol. Apart from that, probably good thing to refresh memory and likely learn something new so you are instantly getting a watch time from me 👀
If I need to sort something & I don't care about optimization, I use what I call NaïveSort
It's selection sort, except instead of swapping elements, you copy the element into a temporary list & delete the original, then at the end you replace the original list with the temporary
It's literally just worse than SelectionSort, which is already pretty bad, but it's dead easy to implement -- maybe even the easiest of them all
@@aformofmatter8913 always loved selection sort for how stupidly easy it is to implement, literally just call min and swap in a loop and that's it, but imo yours feels needlessly sophisticated, in a way. Like, for me it is easier to just call swap than to create a new container. Although I see why you like it, it's an interesting approach
What game?
@@elnico5623 it was probably "Turing complete". It's not a programming game per se, instead it teaches you to create a computer from logic gates and then you program it. It's available on steam and is very fun, pretty much a sandbox for any digital logic simulation
I put this video knowing it would knock me out, worked like a charm. When I woke up, I re-watched the entire thing because it’s was such an interesting video
20:08 You really thought you could sneak that in there and have nobody notice
This is an amazing video, it's like those "sorting algorithms visualized" videos, but with a concise explanation of how each one works. Thank you, it really helped me understand them.
Amazing video, really! I was not expecting this high quality, but here it is! You gained a new subscriber.
The world needs more CZcamsrs like you and less CZcamsrs of the kind you don't even want to ever find.
There are a couple of algorithms used on really old, serial access (i.e. tape) backing store.
The first is the Fibonacci sort:
Augment the list to make a fibonacci number (say F[n]) of items and split them between two tapes, the first (tape A) containing F[n-1] and the second (tape B) F[n-2] items.
Each item on these tapes can be considered a sorted list of length one.
Merge sort the first list on tapes A and B onto a third tape (tape C), and repeat for F[n-2] times.
Now, rewind tapes B and C. Tape A has F[n-1] - F[n-2] = F[n-3] lists left. This becomes the new tape B. Tape B is finished, so can be used as the new tape C. Tape C has F[n-1] sorted lists, so can be the new Tape A.
Repeat this until there is just one sorted list.
The other one is an extension of this, called the cascade sort, where, instead of just three tapes, there are k tapes, and tapes 1 to k-1 are merged to tape k, then the tapes are cycled, so tape 1 becomes the new tape 2, tape 2 becomes the new tape 3, …, tape k - 1 becomes the new tape k and the tape k becomes the new tape 1.
Just stumbled upon this and gotta say it's really well made! Keep it up :)
20:08 AMONGUS
in-place merge sort would've been a nice mention. Also the two ways of quicksort partitioning. And in terms of joke sorts, demon sort is one of the more interesting ones :)
And then there would be threading optimization with odd-even sort and merge sort… ah, but I haven't seen the second video yet. Great work, anyway!
20:08 I love you for this
brilliant video and your voice is really nice too. will definitely rewatch next year when i have an intro to programming class :)
I wanted to see how Radix works, and this video explained it so well. I'm only really a beginner at coding, and I had an "OHHHH that makes sense" moment in this
30:00 its really cool to see algorithms that only work in the real world! im really glad you included these
ive been using bubble sort to maintain the sortedness of arrays, works pretty well if nothing huge needs to change
20:09 AMONG US on purple bar
This is a very good explanation of these sorts! Are you planning to make the code for the visualisations public?
I am considering it, but as of right now, the code is a mess with no comments, so I would like to make it readable before that.
amazing video you explain everything so well
host is teaching me more math then school ever did, and making it interesting
Wow, this video is a really good work, congratulations!
Fluxsort and Quadsort would be pretty interesting to cover as well for the part 2!
Hey! Loved this video :). Small observation I would make is when you're comparing algorithms, maybe use a gradient from green to red to order their complexity :)
Here's an interesting variation on the Bogo sort: it will only randomize groups of sorted sequences, then at next check, it will do it again with the new group. Will this be better or worse than regular Bogo? The intuitive answer would be yes, but how would you prove it?
My approach would be that there are situation where the list is piecewise sorted and the randomization sorts it right, so it can be a reduction in time complexity, though inconsistent because it only works in one condition but it's also a significant increase in space complexity because you need to keep track of which are the sorted lists.
When it comes to Bucket sort I'm guessing you can preserve the efficiency of algorithm on non evenly distributed arrays if you know what distribution your data is expected to be, so that you can put more buckets on the densest parts of the array. Also, I wonder if you could do multiple steps of evenly distributed buckets, but only split buckets that are above a certain size. My guess is that it would boil down to another algorithm.
Cant wait to see thanos sort. If the array isnt sorted, remove half the items at random. Repeat until the array is sorted
This is amazing. Thank you
20:09 *a m o g u s*
Bogo has always been my favorite. Not many sorts can go from "1st try" to "heat death of the universe."
I've written a node.js package for sorting in the past and just went back to mess with it again a little and I found out that I wasn't benchmarking node's native sort method for Arrays correctly (it was sorting the input array in-place and then in subsequent runs already had the pre-sorted input). Now I wanna rewrite my package and try double selection sort while I'm at it.
Loved the visual style
Yaay, SFML! I used it last semester for some assignments but haven’t ever seen it named in a project someone else has done, so I wondered if it was commonly used or not.
This video is really helpful. Keep going bro 🔥🔥
That pancake sort might actually work on some kind of strange doubly-linked list if each link specifies which link of the following element to continue with
Fun fact: a quantum bogosort would in theory always resolve in 1 step (if done correctly)
22:42
For best performance you would count the digits without copying the array to the second buffer, after counting you would do the sort copy to the other buffer, then count, sort copy back, and keep going until one of the two arrays has the final sorted result
Computational complexity is the same, but it's more performant
I feel like the statement radix can only sort integers should come with an asterisk. Shouldn't you be able to map any object to an integer and achieve the same results? This would require some preprocessing if it's not baked in to the object itself but could still be done quickly in some cases. For example if you're sorting cities by country->city you could generate an integer encoding for that data to then perform radix on.
the best video ive seen on this subject
I think you don't need merge sort to use the additional array, you can do all in the first one with pointers
No matter how many times I get it explained to me I can’t ever fully understand big O notation. Which is funny because back in the day I was an avid sorting algorithm enjoyer, having made over 90 sorts myself.
Corection at 9:35: Shouldn't the pivot be a random value in the list that we swap to be the first value? That way, we make it less likely that some preoccuring pattern in the data will cause us to get the worst case runtime.
mine: you have list A and B, make list B numbers s-b (smallest number to biggest number in list A), then go through list B and remove any items in list B not in list A, because we made list B s-b, it was sorted, then remove all the ones not in A and we only have the ones in A
I'm not complaining about the content, because that is ACE, but i'd low cut the vocal track with a EQ or HPF at minimum 85hz. maybe you can even cut it at 120-130hz based on tonal features of your voice, do it post process, not pre.
Just a little free tip that will make the sound more consistent through the series. (that is, if you use the same microphone, and the same room ofc) There is a very audible deep humming which i suspect you can just filter away.
I'm saving your videos for when it's 3 am and can't sleep
20:08 among...
I was dying when I saw that lmao
Awesome stuff!
I was just about to cover this topic, but you did it first.
my fav sorting is cosmic ray sorting, which basically waits for the cosmic rays to flip the bits and somehow the array will be sorted lol
A little while back I saw a video on BOGO BOGOsort (if I recall the name correctly). Iirc it's some kind of bogosort nested in another bogosort, but I forgot the details
In terms of complexity sleep sort is actually weirdly interesting to discuss. Just sleep as long as the number is, and it will print the items likely in order, assuming the minimum difference in sleep is greater than any expectable jitter in the threading handling. While the time complexity is O(n), the efford is actually barely anything. While certainly in the list of impractical sorting algorithms, it is actually implementable at least.
This explained sorts in plain language, much better than others have done. Nice flag, too (0:06).
2:35 - A Numberphile video showed that Selection and Insertion are the same sort but reversed.
20:00 - That's not the only way to do Shell Sort. If a pair is swapped at the current gap, compare to the left at the same gap size, repeat until you reach the start of the list. It guarantees that all items at the current gap size will be in order, moving items which might have been missed with a larger gap on previous iterations. Example with gap of three: 4,7,2,9,1,6,5,8,0,3
compare 4 & 9; compare 9 & 5, swap, compare 4 & 5; compare 9 & 3, swap, compare 5 & 3, swap, compare 4 & 3
From now on I'll exclusively use BOGO sort for everything.
in the discussion of radix and related sorts, I fail to see how, in practical implementation, the range can be anything other than O(1), since any practical number representation has a fixed bit count (short of arbitrary precision data types, but who actually uses those). A radix256 sort of a 32bit number can be done in 4 passes, no variance. It doesn't matter if the 32 bit values are int or float, since floats can be turned into a value-ordered sequence of integers by a simple bitwise operation during the first pass, that is easily undone during the last pass (forward: flip all bits of negative numbers and just sign bit of positives).
Otherwise a very clear, concise and well presented review of sorting. Thank you. Moving on to ep.2 now...
Radex sort is where it went over my head I think. Could be that 20 minutes of thinking is my limit for one sitting though.
in languages with pointers, you can make a list where each element has a pointer to the next element. using this structure, you can shift the entire array by using the pointers and not repeated reading and writing.
isn't pointer arithmetic like a taboo?
Nice. Subscribed ❤
I think this is probably the best explanation of Quick Sort I've encountered. Finally got my head around it after all these years.
💛🤍💜🖤
33:40 I had the subs on and it said:
"*Bogosaur* is unique because it's not guaranteed to sort the list on finite time." 😂
Amazing video!
Well that escalated quickly
Have you heard about our Lord and savior, Faith Sort? This simple algorithm is sometimes considered a joke, because it starts from the assumption that the list is perfectly sorted when we receive it, and never actively changes the order of the sort while always insisting that the list is properly sorted. If the order of the list happens to change between Faith Sorts, it is still sorted as it always was and always will be. Amen.
I believe in the sort now. Amen.
one thing possibly missing from the obscure sorts list is examples program synthesis and machine learning producing novel algorithms which are not easily explained
Sleep Sort, where you make a thread for each item, let them sleep for that, then they will wake up in order
What if you use the `median of the medians` algorithm in the implementation of the quicksort to almost always be sure that you will split into 2 equal subsequences? Would it improve its time complexity in the worst case ?
I accidentally subscribed.
*+1 sub*
Great video!
Thank you!
Very cool thank you so very much
Nuclear Bomb Sort: sets EVERY element in the array to 0 and checks if it's sorted
This, is technically O(n) time, and actually has a space complexity of, 0
But it is less useful than stalin sort
new idea, scan the list, consider that the way its supposed to be sorted, boom
I like the vid! something you may want to clarify about binary search insert is that in theory it's faster in practice it's not. The check is O(nlog(n)) but the act of inserting it in the location and shifting makes it O(n^2) still. So it's actually not better.
"Words are just 26 bit numbers"
God i've never thought about it like that.
I assume the thumbnail is probably heapsort? I am kind of curious. Even though it really doesn't matter LOL
Is a sorting algorhythm possible where the average time is so bad it's a function like Grahams function, TREE or SCG?
If your only definition of a sorting alg is that it takes a sortable array as input and returns a sorted array, then yes. You can write an algorithm that performs bogosort (time complexity n*n! average) but also on each step j compute tree(j). This will do nothing to the array but will add computation that scales with the tree function. The final complexity will run O(TREE(n*n!)) average case with O(n) best case and an undefined worst case because bogosort has the ability to run infinitely. I'm pretty sure any algorithm that computes the tree sequence is indeed O(TREE(n)) but I'm not positive.
Chuck Norris sort: it is sorted
My favourite sorting algorithm is the O(0) (or would it be O(1)?) Ba Sing Sort, aka "there is no out of place value in the array". You don't do anything, and the array is already sorted. Trust me.
How is merge sort n space complexity? I can make a program that does it without using extra memory!
13:43 actually, selection sort can be stable, depending on how you find the smallest/largest number, and how you implement the algorithm
99% of bogo sorters quit right before they’re about to sort the list
masterful presentation, lots nuance is packed into every part
This is completely unrelated to anything, but you might be interested to know that your first name "Kuvina" means "as/with pictures" in Finnish. Fitting for a visual medium like YT :)
I've gotten comments about it before, but I never realized how fitting it is!
20:09 I see you
Me when seeing gravity sort somehow have everything go into place (particularly on a circular visualization): oh so that makes everything fall to its correct place
“Gravity sort *insert stuff blah blah blah it uses gravity ok*”
Me: oh it literally does that
I think there is a problem with the way you described quicksort. If the pivot happens to be the largest element in the list, A will scroll all the way to the end of the list and meet B without making any swaps. Then if you place the pivot to the left of them, the element to the right of the pivot (where A and B meet) is going to remain smaller than the pivot, but also be on the wrong side of the pivot, with no chance of making ever getting sorted.
You're right, there is a discrepancy between that and my code. My code goes like this:
int quickPart(int from, int to)
{
int piv = a[from];
int c = from + 1;
int d = to;
while (c < to && a[c] < piv)
{
check(c);
c++;
}
while (d > from && a[d] > piv)
{
check(d);
d--;
}
while (c < d)
{
switcheroo(c, d);
show();
c++;
d--;
while (c < to && a[c] < piv)
{
c++;
}
while (d > from && a[d] > piv)
{
d--;
}
}
switcheroo(from, d);
return d;
}
A goes right until it finds a piece >= the pivot (or if it passes B)
B goes left until it finds a piece