The Awesome New Queue of .NET 6 That You Skipped

Sdílet
Vložit
  • čas přidán 5. 03. 2023
  • Check out my courses: dometrain.com
    Become a Patreon and get source code access: / nickchapsas
    Hello everybody I'm Nick and in this video I will introduce you to the new queue type of .NET 6, called PriorityQueue.
    Workshops: bit.ly/nickworkshops
    Don't forget to comment, like and subscribe :)
    Social Media:
    Follow me on GitHub: bit.ly/ChapsasGitHub
    Follow me on Twitter: bit.ly/ChapsasTwitter
    Connect on LinkedIn: bit.ly/ChapsasLinkedIn
    Keep coding merch: keepcoding.shop
    #csharp #dotnet

Komentáře • 196

  • @Xankill3r
    @Xankill3r Před rokem +59

    As a game developer Priority Queues come up very frequently for me. Mostly in the context of pathfinding.

    • @jwmoffat
      @jwmoffat Před rokem

      For the cases where you do have ties, does it affect your results on which one is dequeued first? I'd imagine for pathfinding that any of them would be valid and that it wouldn't matter, but curious if you had more insight.

    • @YotaXP
      @YotaXP Před rokem +6

      @@jwmoffat It might be the difference between whether a character steps north then east vs east then north, but as long as it is deterministic, I think that should be acceptable.

    • @Andrew90046zero
      @Andrew90046zero Před rokem +1

      Interesting, never thought of that.

    • @Xankill3r
      @Xankill3r Před rokem

      @@jwmoffat exactly what YotaXP said. It's only important that the outcome be deterministic.

    • @shuvbhowmickbestin
      @shuvbhowmickbestin Před 3 měsíci

      graphs algorithms?

  • @modernkennnern
    @modernkennnern Před rokem +78

    It's very weird that a queue isn't actually a queue - i.e. a FIFO

    • @protox4
      @protox4 Před rokem +5

      You can force it to be FIFO if you want. Just wrap it in another class that increments a counter every time an item is enqueued, and implement the IComparer to use the item's current counter to compare if the other comparer is equal.
      [Edit] Or, just using a timestamp like Nick showed.

    • @iGexogen
      @iGexogen Před rokem +6

      It is like queue in real life where some "just ask" people can insert themselves at the beginning))

    • @protox4
      @protox4 Před rokem +2

      @@iGexogen I think he's talking about the fact that items with the same priority are not FIFO, not the fact that higher priority items jump the line.

    • @srijonroy7084
      @srijonroy7084 Před rokem +9

      It's a heap.

    • @dotvkab
      @dotvkab Před rokem

      @@protox4 and say bye bye performance

  • @RoySalisbury
    @RoySalisbury Před rokem +3

    About 15 years ago (.net 2.0) I had someone write a PriorityQueue for me that worked almost exactly the same way, but also had the ability to "re-adjust" what got dequeued based on how long it was sitting in the queue. The problem with a standard priority queue is that items can get starved out and never get dequeued. We had it so that if a lower priority item had been passed over for dequeue after a set period of time, it would get moved up in priority (an internal priority) so that it would eventually get dequeud.
    It was not a complex algorithm, but it was not trivial either. The guy that wrote it for me actually had written numerous books on algorithms.

  •  Před rokem +3

    I've actually written multithreaded applications where the queue is used to communicate between components running in separate threads. There I even had a need for a priority queue where you could send out-of-band messages to a component. If you use BlockingCollection it was relatively easy as you can await multiple queues at once.

  • @domportera
    @domportera Před rokem +7

    I use a queue or stack very often in irl tech "experience" and game development. super clutch for just cycling through a task list in order. I even made a little "cyclic queue" class for whenever a perpetual cycle is warranted :)

  • @iGexogen
    @iGexogen Před rokem +8

    I think this collection is not intended to be used that way, it can be useful in cases where order within priority is insignificant. For this use case I'd rather implement some class having ordered dictionary of queues under the hood, this solution adds some allocation overhead but simple and obvious.

    • @dotvkab
      @dotvkab Před rokem

      So PriorityHeap is more obvious name for this

  • @adrian_franczak
    @adrian_franczak Před rokem +8

    Shouldn’t there be x.Item2.CompareTo(y.Item2)?

  • @adamoneil7435
    @adamoneil7435 Před rokem +1

    I didn't know about this -- thank you so much. I've used queues only a little bit in the wild.

  • @waynehamberg9248
    @waynehamberg9248 Před 11 měsíci

    The one queue class that I would like to see Microsoft add is a TimedQueue that triggers an event when at a specific time to do a specific task. I want that class to be it's own thread with a priority so if I have 2 events at the same time it will run the higher priority first. Or spin that task on a thread with a higher priority if two threads are launched at the same time with different priorities.

  • @NawfalHasan
    @NawfalHasan Před rokem +1

    So basically this is a combination of SortedSet and Queue.

  • @sotsch9280
    @sotsch9280 Před rokem +8

    A priority Queue behaves like an 'ordinary' binary heap. In Game development often used for much stuff (A* Pathfinding, TurnBased Style ganes, AI decisions prioritizing, etc.), as well as stacks (Backtracking etc.)

  • @Blaisem
    @Blaisem Před rokem

    Queue and stack types I don't use much either, but as you said, it's incredibly clutch when you do need it and imo leads to elegant coding solutions to their niche use cases. I don't have a need for the priority queue at this time, but it's definitely good to know about it, and I liked the demonstration of workarounds to dig more functionality out of it.

  • @masonwheeler6536
    @masonwheeler6536 Před rokem +3

    I've worked on a lot of codebases, and I don't think anyone was ever using the big, messy multiple queues "solution" described at the start of this video. Typically people just used a different priority queue implementation. The Heap/Priority Queue is a pretty well-known algorithm; the only real difference here is that Microsoft has (finally finally finally!) included an official version in .NET.

    • @owensigurdson8610
      @owensigurdson8610 Před rokem +1

      I wish they just use normal names for things however (i.e. Heap).

    • @ShikaIE
      @ShikaIE Před 4 měsíci +1

      I had the same thought. Literally wondering “isn’t this just a heap?”

  • @TheAceInfinity
    @TheAceInfinity Před rokem

    You could probably also wrap this to use a type 'OrderedPriorityItem(T Priority, long Order)' where T is constrained to be 'IComparable' and go from there. Create buckets with T as the lookup and Interlocked.Increment. The wrapper could allow for a comparer to be passed in via the constructor to conform to how the original priority queue works, since then it would be usable with more types, especially those that you don't own. It's a bit of work, but that would be my initial attempt to approach it, probably while having to tweak some things.
    edit: Actually much easier than I thought if you create a wrapper that takes in a comparer of the primary element type and deals with incrementing the internal order.

  • @osman3404
    @osman3404 Před rokem

    LOVED THE VIDEO … but rarely use queues but this video gave me an idea how I can use it more often

  • @jerryme1225
    @jerryme1225 Před rokem

    I have used a priority queue in a branch and bound optimization algorithm. It is blazing FAST! It uses a binary tree sort when items are added to the queue. No sorting is done at any other point. The dequeue process is to take lowest value node by using a depth-first search. The code pretty much knows where the lowest value and can find it in a few steps. It is also very fast. The downside for the basic implementation is that priority values cannot be changed once an item has been added. One could add a method to update a priority value by (1) marking a node as invalidated, and (2) add the object again with the updated priority. When dequeuing, the method will have to check for deleted nodes, ignoring them, continue to do so until it finds a node that is not invalid. I don't know how much of this might have been implemented in .Net 6. But I'm looking forward to trying it out. Thanks Nick!

  • @davidr5000
    @davidr5000 Před rokem +17

    For me that seems like you need to know a lot of the internal workings. If a new developer looks at this code he probably does not know why the Stopwatch and the Statuscomparer is needed. If i had this requirement i would create a class that has internally a *_queues Map* and has the public interface *addItem(Status status, string user)* that adds the item to the specific queue or creates a knew on and adds it to the internal map. And also the *dequeue()* method where i can implement the sorting of my Statuses however i want and return in the right order. So in this example i would not use it but thanks for sharing :)

    • @bluesillybeard
      @bluesillybeard Před rokem +1

      That is a better way to do it in this case, but he's showing how to use a priority queue.
      To be fair he could have used an example that suits the use of a priority queue better.

    • @iGexogen
      @iGexogen Před rokem +1

      OrderedDictionary will be better choise. It is costly to order statuses on every dequeue operation, much more efficient to iterate on already ordered keys and without probing an unexistent variants.

  • @modulsyd9222
    @modulsyd9222 Před rokem +8

    In Microsoft’s implementation, I miss the Update method on the priority of an item. This is for example needed in a shortest path algorithm.

    • @protox4
      @protox4 Před rokem +2

      There's an open issue to add that.

  • @DredTather
    @DredTather Před rokem

    I had to write a priority queue in c# about 10 years ago. The priority queue is really useful for geometry algorithms.

  •  Před rokem +1

    Since the priority queue has nothing to do with FIFO, I prefer to rely on multiple queues that I dequeue as I want e.g. this loop :
    dequeuing 3 times in the gold queue then 2 times in the silver queue then 1 time in the bronze queue.
    var bronzeQueue = new Queue();
    var silverQueue = new Queue();
    var goldQueue = new Queue();
    bronzeQueue.Enqueue("Bob");
    bronzeQueue.Enqueue("Boris");
    bronzeQueue.Enqueue("Buzz");
    silverQueue.Enqueue("Sam");
    silverQueue.Enqueue("Sandy");
    silverQueue.Enqueue("Saul");
    silverQueue.Enqueue("Silv");
    goldQueue.Enqueue("Gal");
    goldQueue.Enqueue("Garry");
    goldQueue.Enqueue("Gus");
    goldQueue.Enqueue("Gustave");
    var queues = new (Queue queue,string name)[] {(bronzeQueue,"Bronze"),(silverQueue,"Silver"),(goldQueue,"Gold")};
    while (bronzeQueue.Count>0 || silverQueue.Count>0 || goldQueue.Count>0) {
    for (int q=queues.Length-1; q>=0 ;q--) {
    int max = Math.Min(queues[q].queue.Count,q+1);
    for (int i=0; i

  • @QwDragon
    @QwDragon Před rokem +2

    That's strage that gitting element from queue doen't return priority.
    By the way, without priorities the first code could've been simplified by using array of queues:
    var queues = new [] { platinumQueue, goldQueue, normalQueue };
    while (!cts.IsCancellationRequested)
     for (var queue in queues)
      ...

  • @danylokd
    @danylokd Před rokem

    If order is important, you could also write a wrapper around sorted dictionary of queues.

  • @refactorear
    @refactorear Před rokem

    I had the pleasure of using PriorityQueue in one of the Advent of Code puzzles of last year. I could have used in two different puzzles but I didn't know about it until halfway the competence.

  • @alexkazz0
    @alexkazz0 Před rokem

    Thank you, this is very useful!

  • @alexclark6777
    @alexclark6777 Před rokem +1

    I'm starting to see why I skipped it, after watching this. Seems like they're violating the fundamental principle of a queue - FIFO ordering - just to be able to "group" by priority. I'm not sure what causes the ordering to work that way but a Dictionary would seem to solve this same problem without messing up the ordering?

  • @ToadieBog
    @ToadieBog Před rokem

    I didn't even know about it!
    Queues...not used very often.

  • @vasilisplavos
    @vasilisplavos Před rokem

    Thank you, Nick, awesome video! I am just wondering why not start queue priorities from 0 instead of 1?

  • @shazmunchdylbertoid
    @shazmunchdylbertoid Před rokem +1

    huh I never thought of doing a singleton like that
    before anyone says it yes it's probably semi standard but I just haven't really had the need before - there are a few other ways you could do this eg a private read-only instance in the class you're running your code in or similar.

  • @kostasgkoutis8534
    @kostasgkoutis8534 Před rokem

    Yeah, I was prototyping with this, and I noticed the issue right away.. I came to the same conclusions as you did..

  • @tarsala1995
    @tarsala1995 Před rokem

    I suspect that one additional reason to introduce PriorityQueue is to use C# in coding interviews. During my interview to MS I was kind of forced to use Java (which means to learn as well) in the interview due to this very common data structure missing in C#.

  • @benjaminclehmann
    @benjaminclehmann Před rokem +4

    The fact that it doesn't maintain insertion order doesn't strike me as that weird for a priority queue, it matches what they do in a lot of other languages. Insertion order may be meaningful for your example with card tiers, but very often people are using priority queues because it's how you implement an algorithm, and in that case insertion order isn't normally important.
    That said, it doesn't have the option to update an item's priority, which is needed by an awful lot of algorithms. It's a disappointing oversight, since it appears it's not good for when your business logic wants the semantics of a priority queue, and it's not good for algorithms which are most efficiently implemented as priority queues.

  • @ivaniliev93
    @ivaniliev93 Před rokem

    Finally.... I used to use the WinIntellect collections or copy some experimental implementation

  • @TheFinagle
    @TheFinagle Před rokem

    Theory Question: if you had a priority queue, but needed to prevent blocking on lower priorities is there a way to handle that? for example a way to request the next Gold, even if There are Platinum's remaining. I know that doesn't fit this example, but also a frequent concern in other areas where priority queues are used.
    If not this seems to me like a very rudimentary implementation that will hopefully be improved or expanded on in the future.

  • @Speiger
    @Speiger Před rokem

    Thanks for showing me that one.
    Time to implement that into my lIbrary since i am pretty sure java doesn't have that one :)
    Sry about the GH ping ^^

  • @_iPilot
    @_iPilot Před rokem

    Moreover, i even used to implement priority queue on my own in C++ using binary trees.

  • @bluesillybeard
    @bluesillybeard Před rokem

    I would personally call it "OrderedCollection", (due to the fact that it doesn't quite behave as one would expect), but honestly not a huge deal as long as it's clearly documented.

  • @foamtoaster9742
    @foamtoaster9742 Před rokem

    Hi, just wondering with the enums, if you just invert the order of the enum in the way you type it out, so platinum at the top etc. wouldnt that make the comparison work ?

  • @smaroulis
    @smaroulis Před rokem

    never used, because I wasn't aware, always created my own queuing systems. Nice to know

  • @Subjective0
    @Subjective0 Před rokem

    The behaviour matches the data type as I have learned it, it’s made to give the item with the highest/lowest of a certain priority value, but it has unpredictable behaviour if you give everything the same priority. So I wouldn’t say that it’s a weird or misleading implementation that they made.

  • @crazyfox55
    @crazyfox55 Před rokem

    I'm curious if the same priority items eventually come out, or might there be items that were added first might stay in the queue for an arbitrarily long time?

  • @KeyboardKrieger
    @KeyboardKrieger Před rokem

    I have to test it first, but maybe if it's a more realistic example with a UserEntryObject which stores priority and timestamp and the comparer it will get less complex

  • @Soraphis91
    @Soraphis91 Před rokem

    this might have been not the best example :/
    you could just go with your first code, put all the queues in an internal list of your worker, and iterate the list. That way you don't have to add that much code when adding another queue, but you save yourself creating the tuple for ordering, you save yourself creating the comparer and so on.

    • @nickchapsas
      @nickchapsas  Před rokem +2

      It would be a bad idea to do so because if I need N queues I need N changes to that original class and if the hot path is always the last queue then I have consistently bad performance.

  • @KNHSynths
    @KNHSynths Před rokem

    I am not convinced by this new guy.
    To summarize, we start with a simple C# code that poses problems of evolutionary maintenance but that can be understood and corrected by a basic developer, to end with a complex C# code that is also full of pitfalls but it takes a senior developer to understand and maintain it...
    I will not advise this approach to my clients, that's clear.
    Thanks for the video, always well done!

  • @alexanderdell2623
    @alexanderdell2623 Před rokem

    Hi Nick, can you make a video about concurrent collection and concurrency in general?

  • @xniyana9956
    @xniyana9956 Před rokem +2

    I've used queues and stacks fairly often for various lower level algorithms like processing TCP/IP messages or processing syntax trees.

    • @nickchapsas
      @nickchapsas  Před rokem +4

      This and gaming are the two use cases I’ve seen it personally the most

  • @androidsavior
    @androidsavior Před 8 měsíci

    Do you have an example where we can use the queue in a real world web application ? thank you

  • @DarrellTunnell
    @DarrellTunnell Před rokem +4

    I actually prefer your first code. 1. It's easier to see what is going on. 2. No unexpected behavior like losing FIFO as per this new queue which you've worked around by misusing the priority buckets imho putting each item into its own unique priority as far as the data structure is concerned. 3 you can easily make that initial code more extensible if you want to allow more priorities it doesn't take much work with DI, and services available it's not difficult to imagine how that might be done.

  • @lovyNOM
    @lovyNOM Před rokem

    I use Queues fairly often, I haven't needed a PriorityQueue yet though

  • @jchandra74
    @jchandra74 Před rokem

    How do you use this with real world queue like Azure Storage Queue or AWS queue? You have something that read from that queue, figure out what the priority and requeue it using this and spit out to a real queue again to be consumed downstream like reprioritizing real queue in memory to put into another real queue?

  • @JoeIrizarry88
    @JoeIrizarry88 Před rokem

    Video for your experience & opinion about Pulumi - CDK style libraries in general even. Pros/Cons for traditional declarative terraform, ARM, CloudFormation, etc.

  • @malcolmchambers4934
    @malcolmchambers4934 Před rokem

    A queue that fails to preserve the entry order is not a queue.
    I am actually rather glad that I did not know about this. As the single case I had to do something like this I Processed queue 1 then queue 2 each time I was fetching from the queued items, and the order of entry was significant. I doubt that if I had used a priority queue which is not a queue that I would have caught the defect.
    My use case was threaded where the consumer was independent of the injection process. Queue 1 was in effect a queue jump mechanism.

  • @oleggolovkov957
    @oleggolovkov957 Před rokem +2

    Yeah but 3 queues provided you with O(1) time for both enqueue and dequeue while the PriorityQueue needs O(logN) for both since this is implemented as a heap, this is also the reason why you don't get the order of items with same priority you expect, this happens naturally when implementing array-backed heap

    • @nickchapsas
      @nickchapsas  Před rokem

      Actually it is O(N) where N is the number of queues you need to check which ends up being worse

    • @jupahe6448
      @jupahe6448 Před rokem

      @@nickchapsas I thought N represents the number of inputs
      sure the number of queues is a factor but so is the time it takes for the operation in the first place, big O Notation is only meant as a way to predict how much the resource usage increases (if you think of time as a resource) when the number of inputs increases
      If you would need a queue for every X inputs O(N) would be correct, but in this case I don’t see the number of queues increasing in such a way

    • @nickchapsas
      @nickchapsas  Před rokem +1

      You got a point actually. N*O(1) is a better way to describe it. When you have the same data structure N times then you still need to take that increase into account. It obviously scales better in comparison to logn for this specific usecase because the algorithms themselves are o1 as pointed out but if you have 1000 queue types and your hot path is on the last one then that’s not efficient neither in performance nor code upkeep and memory.

    • @oleggolovkov957
      @oleggolovkov957 Před rokem

      @@nickchapsas sure but the example was about 3 queues and so was my comment. On practice one should of course benchmark his scenario because up to some extent N FIFO queues is still better than a single priority queue

    • @cyril113
      @cyril113 Před rokem +2

      It's c * O(1) where c is some constant. Using n is confusing because it's used for input size. There is a constant for every algorithm but we ignore it in the big O notation. So in theory c1 * O(1) is always better than c2 * O(log n). But yeah it's irrelevant in practice and one should benchmark both implementations to find out of what's really better for realistic n that doesn't tend towards infinity. I'm getting triggered when people bring up big O notation as an argument

  • @humanesque
    @humanesque Před rokem

    What's the real effective difference between this and just using an enumeration of queues?

  • @PbPomper
    @PbPomper Před rokem +1

    I don't see the benefit unless there is some sort of optional parameter that allows you to enforce proper dequeueing. This workaround only makes things more difficult to read.

  • @ThatGuyT
    @ThatGuyT Před rokem

    IIRC as a small caveat: this PriorityQueue does not allow re-prioritizing (if that's even a word)

  • @kondziossj3
    @kondziossj3 Před rokem +1

    Hi Nick
    What's your opinion about TPL Dataflow?
    Did you use it?

    • @reikooters
      @reikooters Před rokem

      I've used it now and then for around 8 years whenever I've had to create an application which is always running and processes a queue. I find it useful when I want to have a queue in which I want the items to be processed in parallel, but with a rate limit, and when items could take varying lengths of time to be processed. e.g. the application is constantly checking a database for new queue items, then adding items to the queue to be processed, but I want a maximum of say 6 items being processed at once, but occasionally one item takes a while. So I'm mostly using BufferBlock and ActionBlock. I haven't really had the need to create a whole "pipeline" with TransformBlocks etc.
      Using TPL Dataflow works much better than what a naive approach may look like, e.g.: 1) query database for items 2) process all the items in parallel with some max degree of parallelism 3) wait for them to finish 4) wait for some seconds before checking database again 5) repeat - because if you have one item which is much slower than the rest, then you end up with only one "worker" actually doing work at the end until it finishes before querying for more items again. Instead with TPL Dataflow you would query the database, queue them to the ActionBlock, then wait before querying again, meanwhile you just let the ActionBlock keep processing the items as quick as it can with all its workers.

    • @kondziossj3
      @kondziossj3 Před rokem +1

      ​ @reikooters I use it sometimes
      but I feel like I it's too complicated sometimes
      (e.g. last pipeline have like ~20 steps)
      Overall it's good
      I mostly like it for preventing to make a "spike" operations
      so situation when you make a lot of operations and you wait for finish all of them and then go to next
      but... that means when there is 30 sek of processing and then I want to save all things then the CosmosDB or any other storage have bad time for few secods
      so... it's overall normalize the usage of resources

  • @krccmsitp2884
    @krccmsitp2884 Před rokem

    I actually didn't catch that a "PriorityQueue" was introduced in .NET 6. I find its behavior a bit strange, as it does not meet the expectations. However, I also need a PriorityQueue very rarely and once implemented my own many years ago.

  • @dennycrane2938
    @dennycrane2938 Před rokem

    I basically only use queues for avoiding recursion. I'd like something like this for business systems, but then you wouldn't do it in code like this, it would need to be a function of a durable bus of some kind. I'm not sure what I would use this for... maybe something that was fully self contained like a game or something.

  • @anonimxwz
    @anonimxwz Před rokem

    I only use queues in some niche algorithms

  • @hanssobeseir3765
    @hanssobeseir3765 Před rokem

    PriorityQueue is an algorithm, optimized for getting highest priority in fixed time an inserting in order log(n). So if that is what you need, then use it. Everything you said and did is correct and nice, but you missed the essence of the priority queue.

  • @fenderrexfender
    @fenderrexfender Před rokem

    Fun data type

  • @Keanomy
    @Keanomy Před rokem

    The amount of times I've heard "Hello guys, I'm naked." is disturbing.
    Appreciate the videos, been really helpful in my journey of getting back into C#!

  • @jlchappell
    @jlchappell Před rokem

    I’m confused. PriorityQueue says it’s out for .Net6/7/8, and I remember reading about it more than a year ago. What’s new about it?

  • @Freakhealer
    @Freakhealer Před rokem +2

    I totally appreciate the comparer class method but your issue with the enum isn't really an issue.. is it? You could add them from higher priority instead of assigning values and if you need new status place it in the right place and that's it.

    • @nickchapsas
      @nickchapsas  Před rokem +2

      And then when I need to add a new highest priority one what do I do? Push every enum one value down and break everything. It's bad design to base enum ordering based on enum values.

    • @Freakhealer
      @Freakhealer Před rokem +2

      @@nickchapsas if you say so, but my assumption was that it would be used as an enum not as a int value. So the only thing that would matter would be the order of the enum

    • @jackoberto01
      @jackoberto01 Před rokem +3

      Moving enum values around is bad practice and could break existing serialized data if it's stored with the enums int value. You could store the enum values as a string but that's not ideal either

  • @thebitterbeginning
    @thebitterbeginning Před rokem +13

    Performant or not, PriorityQueue() is simply not a Queue. It's useless. Nice job, MS.
    Thanks for all you do, Nick. Awesome, as usual!

    • @bluesillybeard
      @bluesillybeard Před rokem +2

      Yeah, I don't like how they called it a queue when it's really not a queue at all.
      A better name would be "SortedSet", or "PrioritySet", since that better describes what it does.

    • @sotsch9280
      @sotsch9280 Před rokem

      Binary heap!!!!!!!!!!!!!!!

    • @bluesillybeard
      @bluesillybeard Před rokem

      @@sotsch9280 That would work for sure (and would be pretty fast), although I don't think C# has a built-in class for that, so it would have to be programmed first.

    • @iodred
      @iodred Před rokem

      @@bluesillybeard What they're saying is that's what a "Priority Queue" implementation is based on - a binary heap. Since a binary heap rebalances itself upon insert or remove ops, the FIFO order within individual priority levels is usually disturbed.

    • @Blaisem
      @Blaisem Před rokem +1

      @@bluesillybeard You can't call it a set when it allows duplicates

  • @haydensprogramming6766

    Nice "It's a British thing" easter egg

  • @AlexanderBelikov
    @AlexanderBelikov Před rokem

    Super useful, as always! Thank you, Nick!

  • @Max_Jacoby
    @Max_Jacoby Před rokem

    Next time in a queue to a cashier at McDonalds I'll go first and declare to everybody that order is not guaranteed.

  • @yoniarkin9040
    @yoniarkin9040 Před rokem +1

    How does it handle starvation of low priority items?

    • @protox4
      @protox4 Před rokem

      It doesn't, by default. But you can yourself by implementing the priority with the comparer, and increment a counter on each enqueue. If it reaches a certain threshold, the older elements become higher priority. (This same mechanism can be used to ensure FIFO ordering.)

  •  Před rokem

    ?? That is such a strange thing that items with same priority are not guaranteed to be in same sequence. They should have just called it SortedQueue and it would be much more clear. Thank you for the video Nick. I will probably never use this component.

  • @jolkert_
    @jolkert_ Před rokem

    imo if FIFO within the same priority isnt guaranteed, i just dont think it should be called a "queue" at that point

  • @peculiar-coding-endeavours

    The biggest surprise was that .NET didn't have a priority queue prior...

  • @nomadshiba
    @nomadshiba Před rokem

    wish in c# there was a mode that forces you to write "fast" code

  • @paviad
    @paviad Před rokem +1

    Isn't there a big performance hit when you use a completely unique priority for each and every item in the queue?

    • @iGexogen
      @iGexogen Před rokem +1

      No, under the hood it uses one array for whole items, it doesnt allocate N data structures for N existent priorities, but I consider it is still bad choise for use case demonstrated in video.

    • @ecchioni
      @ecchioni Před rokem

      It's an Nlog(N) space and time complexity. Nlog(N) space becase the heapifying function is most likely recursive.

  • @BinaryFreak
    @BinaryFreak Před rokem

    Why use a Comparer and not just turn around the enum (Platinum, Gold, Normal)?

  • @poteb
    @poteb Před rokem

    PriorityQueue isn't sealed so I would make a subclass of it and wrap all the specific stuff I need for my use case in there.

  • @AlexanderEndless
    @AlexanderEndless Před rokem

    I won't skip it, I will de-queue it ;)

  • @nanvlad
    @nanvlad Před rokem

    Why would we need to create 3 queues? If they're processed at the same "window" I would create a single queue with items, where every Item has a Priority type (Vip, Platinum, Gold, Usual) and reorder items on every queue change (Item enqueued or dequeued) - first by Priority type and then by added time

    • @nickchapsas
      @nickchapsas  Před rokem +6

      Reordering a queue defeats its purpose. It is extremely wasteful especially in each queue change. You should never do that.

    • @nanvlad
      @nanvlad Před rokem

      @@nickchapsas maybe I used a wrong statement. I mean when we take a next item for processing we can calculate it by type/time and take something like MaxBy(i => i.Type)/MinBy(i => i.TimeStamp) or use a custom equality comparer like you showed in a video with PriorityQueue. I assume the similar approach is implemented in .net6 PriorityQueue class, but can be implemented manually for previous versions

    • @shanehebert396
      @shanehebert396 Před rokem +1

      @@nanvlad you can do it that way but every dequeue will be O(n) since it has to find max/min. You should probably be using a List or something as well since you will have to remove from in the middle of the structure.

    • @gbjbaanb
      @gbjbaanb Před rokem +2

      What you should do, if you don't want 3 queues is to create a dictionary of queues. It's the same thing but easier to cope with. Never reorder items in the queue (frankly, why is that even possible in a queue class?)

    • @JerryFederspiel
      @JerryFederspiel Před rokem

      @@gbjbaanb
      A dictionary of queues could cover a lot of cases that a priority queue does. One case that a dictionary of queues would not do so well on is pathfinding. In pathfinding and other planning algorithms, your priority keys are doubles- they can take on a huge number of possible values- and you want to dequeue the potential routes or plans that are lowest-cost first so you explore and find the lowest-cost total solutions first.
      As for why the priority queue would allow two items of the same priority to be dequeued out of temporal order: I think the point is that the priority queue allows you to *completely* control the order in which items are dequeued. The time that two items are enqueued only matters in terms of their dequeue order *if you tell the queue that*. Otherwise, if you say that the priority of item1 is 3 and the priority of item2 is 3, then you are telling the priority queue that the order of those two items _does not matter to you_ and it is free to choose to dequeue them in whatever order suits its own purposes.

  • @nomadshiba
    @nomadshiba Před rokem

    really cool but i think the first example you showed wasn't a good example of why we need a new queue type
    because the first example you showed can be abstracted away easily, so you dont have write a huge while loop.
    also with enum instead of writing numbers next to them you can just put the higher priority, higher in the enum (if your enums dont go to the database, and they should never go to the database anyway, because enums change)
    but i know you are just trying to show an example of how to use the thing

  • @7200D2KJA
    @7200D2KJA Před rokem

    Why not just encode the queue time epoch as an int?

  • @aragorntjuh
    @aragorntjuh Před rokem

    I like the idea of a priority queue but the code doesn't end up looking that clean like this

  • @arithex
    @arithex Před rokem

    Isn't "PriorityQueue" a.k.a. heap? (As in "MinHeap" or "MaxHeap" data structure -- viz. where the memory is allocated contiguously?) That's a fairly niche data structure.
    One of the biggest problems with it is, you can't really index within it, because elements move around / swap positions.. so, you can *only* ever remove from the tip of the heap, never any other element. I've seen uses which end up adding an 'isDeleted' flag to the element type, and ignoring those elements later.. but then, that opens a whole other world of problems.
    I hate the name "PriorityQueue" because it evokes a confusing concept of "ordered list of fifo queues" like in your gold/platinum ticketing example. As you discovered, this is not that. :)

  • @theonlywallrus
    @theonlywallrus Před rokem

    StAtus (long A). It's a british thing!

  • @shoooozzzz
    @shoooozzzz Před rokem

    wait I'm confused. Priority Queues, aka min heaps, are not new to .NET. I've been using them to fail algorithm interviews for years!

  • @Suriprofz
    @Suriprofz Před rokem

    now a real usecase

    • @nickchapsas
      @nickchapsas  Před rokem +1

      They are heavily used in gaming pathfinding to find the optimal path. Just read the rest of the comments

  • @MehdiArmachi
    @MehdiArmachi Před rokem

    How is the cancellation token breaking the while loop without calling ".cancel" ?

    • @nickchapsas
      @nickchapsas  Před rokem +1

      By thowing an exception

    • @protox4
      @protox4 Před rokem

      It's not. The loop runs forever because he never cancels it. He just used that for demonstrative purposes to show how you should implement a real pump.

    • @shanehebert396
      @shanehebert396 Před rokem

      @@protox4 I'd probably want to make the pump blocking rather than polling, though.

    • @protox4
      @protox4 Před rokem +1

      @@shanehebert396 Yes, but that's kind of out of scope for showing a queue. I don't know why Nick even bothered with the cancelation token in this, tbh.

    • @shanehebert396
      @shanehebert396 Před rokem

      @@protox4 Yep, I agree. Still, it was simple enough to set up to give an idea of what to do. It could have been a while (true) { } but this was like dipping a toe into a sort of real case scenario.

  • @mrx10001
    @mrx10001 Před rokem

    Microsoft should just make a 2nd priorityQueue one with guaranteed order.(or via parameter in constructor). Also this better be in the summary of the class, because that's mega un-obvious behaviour.

  • @davidmartensson273
    @davidmartensson273 Před rokem +1

    Since they call it a queue, I would have expected it to keep the other within the priorities to, especially in the case of the example of tickets where you definitely want to make sure the first one to try to book within a priority will get first chance.
    They should have named it something else, PriorityBuckets or similar since that is more to the point in my opinion.
    Yes you "can" solve it with adding timestamps but that is make it quite a lot less easy to use.
    I would have preferred if they changed it around so the default was do keep the order with each priority and add an option to increase performance by ignoring order, that would in my opinion make more sense since its likely that the customer that really needs the extra performance will check all options and find it, the more novice that just want to use it out of convenience is much more likely to not notice and create bugs in the code.

    • @qj0n
      @qj0n Před rokem

      'Priority Queue' is an official, well-established name for this data structure for like at least 60 or 70 years and keeping the order was never obligatory. You'll find this definition in every academic book on data structures. The last thing we needed was Microsoft inventing a new name for something known for so long

    • @davidmartensson273
      @davidmartensson273 Před rokem

      @@qj0n That might be, but I did not go the academic path and for me a queue keeps its order, and the other Qeues in .NET does keep order so it still feels strange.
      I can accept that if this is a well known structure MS should not change its behavior, but it should still be easier to get it to behave as other queues in my opinion.
      And if I check the Wikipedia page on Priority queue it states "In some implementations, if two elements have the same priority, they are served in the same order that they were enqueued in. In other implementations, the order of elements with the same priority is undefined. "
      So its not that it always is undefined, its just that its not mandatory to keep order, so MS could have had the default be to keep the order and offer a high performance option that does not.
      Unless the wikipedia article is dead wrong on this :)

    • @davidmartensson273
      @davidmartensson273 Před rokem

      @@qj0n I also think that the need to have elements in order within priority depends on if the priorities are an unbounded range or a small set of values.
      In the example with maybe 3-5 different priority levels I would very much assume you want to keep order within the priorities but for more algorithmic use where priority can be much more fine grained and possibly almost unique, keeping order within the priorities are much less important.

  • @cruz1ale
    @cruz1ale Před rokem

    Who names their child "Poke"? That's just asking for being bullied at school.

  • @tmhchacham
    @tmhchacham Před rokem +2

    Writing you're own comparer seems like added complexity for what could have been a simple solution: renumber the enum. As long as the code only uses the enum, renumbering the enum should not be a problem. Indeed, that's one of the main reason to use an enum! If you do not like renumbering, for whatever reason, the traditional way to do that (learnt it in apple basic) is to number them 10, 20, 30, instead of 1, 2, 3. That allows for modifications. So, if you added titanium, it would become 5. This allows for a few edits. Of course, if you expect many edits, just change the numbering to 100 ,200, 300, or 1000, 2000, 3000. This is so much simpler than writing your own comparer.

    • @nickchapsas
      @nickchapsas  Před rokem +6

      This is one of the most common bad practices when enums are used. It looks like it makes sense on the surface but you are adding the responsibility on something that shouldn't have it. The enum shouldn't know how its supposed to be ordered for one specific scenario and the values shouldn't reflect that. What if you want ascending and not descending? Now you have to know the values. What if someone is added on the top? Now you might break consumers. I've seen it go wrong in some way almost every time it is used. And what happens when you don't use enums but instead you use static readonly fields. You fallback to the comparer. Never do that. Simler doesn't mean better. And I would argue that this isn't simple. It is naive.

    • @tmhchacham
      @tmhchacham Před rokem +1

      @@nickchapsas I hear what you are saying, but i do not understand it at all. I don't have a lot of experience with large teams though, so i just need to be consistent with myself.
      "The enum shouldn't know how its supposed to be ordered for one specific scenario" In your example, the enum was specifically created for this scenario, that is, for priority. So, of course it should have numbers! (At least that's the way i see it.) Also, when anyone else looks at the code and sees the numbers, it will click immediately that the numbers are important. Not so with a comparer. The person needs to read it, figure out why you put it there, and even then not everyone will get it. Numbers make it dead simple.
      "And what happens when you don't use enums but instead you use static readonly fields. You fallback to the comparer." Agreed. But that's not an enum. Indeed, the enum (with numbers) solves this.
      ---
      That being said, you are far more experienced than me (i love your videos and learn so much). Please prove me wrong so i can learn.

    • @jackoberto01
      @jackoberto01 Před rokem +1

      Code based on hard coded enum values is not a good idea in my opinion. A better way would be to translate the enum value to their correct values within a method or similarly in this case using a comparer. A comparer like this is easier to write and maintain than modifying the enum definition

    • @tmhchacham
      @tmhchacham Před rokem

      @@jackoberto01 I've seen way too any people not understand anything that isn't simple. I'll disagree with you on with is easier to write an maintain. A numbered list can be maintained by anyone, even a non-programmer. A comparer needs to be _understood_, and thus requires a programmer, and not an idiot either.
      Please believe me when i say i am not trying to be contrarian. It's just so "obvious" to me that a numbered enum is better, that's its going to take a "proof" to convince me otherwise. That being said, it's what i want you to do, so i can learn. Fwiw, i think i've used a numbered enum just once. Never really had too many cases where i might have needed one.

    • @shanehebert396
      @shanehebert396 Před rokem +1

      ​@@tmhchacham the main issue comes if you need to persist the data containing the enum. If you have a priority enum where HIGH gets automatically defined as 1 and then you write that to a database or something, then if you add a SUPER_HIGH value that automatically gets defined as 1, if you read in the persisted data, all the priorities will shift. You could store the text version of the priority but that can be an issue if the enumerated value is renamed (change HIGH to HIGHER) but at least there are text to enum converters already... if you write the string "HIGH" in the database, for example, and when the string "HIGH" is read back out and converted, it doesn't care that when it was written the integer value was 1 but it's now 8... the conversion will get the right value. In either case, you could have a data migration step to convert all the priorities as appropriate, but that may be an issue if, for example, you write to a JSON or XML file today and then attempt to process it next year (you'll need versioning and all that).
      In the case where the integer value for the enumerated value isn't used and there is an explicit comparator function, then yeah, that also comes with its own maintenance as the enum and the comparator function have to stay in sync.
      However, for enum values that are only ever used during runtime, this may not be as big of an issue because the whole system should be recompiled anyway.

  • @frankfernandez3608
    @frankfernandez3608 Před rokem

    I seriously need to stop solving everything with a list

  • @kaoeferreira9933
    @kaoeferreira9933 Před rokem

    Great video Nick, I suggest please allow put legend in your videos ! So people can legend and be easier to understand what you saying cause you speak to fast

    • @krccmsitp2884
      @krccmsitp2884 Před rokem

      By legend, you mean captions/subtitles, I guess? Just click on "CC" to see auto-generated ones.

    • @kaoeferreira9933
      @kaoeferreira9933 Před rokem +1

      ​@@krccmsitp2884 Yeah, I use the 'CC' auto generated but in some many videos is confusing, doesn't 'translate' well

  • @chswin
    @chswin Před rokem

    Honestly, these in-memory queues have such limited use…

  • @gbjbaanb
    @gbjbaanb Před rokem +2

    When is a queue not a queue? When you use a Microsoft "queue" class.
    This isn't a performance tweak, this is broken. If they wanted non queue behaviour, they should have called it something else.

    • @nickchapsas
      @nickchapsas  Před rokem +2

      This is actually how Priority Queues work in other languages too. Java has the same time for example

    • @Basement-Science
      @Basement-Science Před rokem

      I agree, this pretty much ruins it. At that point you almost may as well implement it yourself.

    • @protox4
      @protox4 Před rokem

      It's just a simple heap structure. Very common. .Net developers used to have to write it themselves, which was a shame for a common collection structure.

  • @MrMattberry1
    @MrMattberry1 Před rokem

    Looked overly complicated, I would have just done some di and a factory returning the queues

  • @Dustyy01
    @Dustyy01 Před rokem +2

    The real question here is why Mike is not a YT Member?

  • @magashkinson
    @magashkinson Před rokem

    I didn't

  • @LogicException
    @LogicException Před rokem

    Cool, a non thread safe priority queue... meh.

  • @kiztime1234
    @kiztime1234 Před rokem

    "The Awesome New Queue of .NET 6", is fine as a title without the dickish "That You Skipped" appended on the end.

  • @michaelinsberg2185
    @michaelinsberg2185 Před rokem

    Never use this queue