Why Computers Can't Count Sometimes
Vložit
- čas přidán 11. 11. 2018
- Sometimes, numbers on sites like CZcams and Twitter jump up and down; subscriber counts lag, like-counts bounce all over the place. Why is it so hard for computers to count? To answer that, we need to talk about threading, eventual consistency, and caching.
Thanks to my proofreading team, and to Tomek on camera!
The Cambridge Centre for Computing History: www.computinghistory.org.uk/
🟥 MORE FROM TOM: www.tomscott.com/
(you can find contact details and social links there too)
📰 WEEKLY NEWSLETTER with good stuff from the rest of the internet: www.tomscott.com/newsletter/
❓ LATERAL, free weekly podcast: lateralcast.com/ / lateralcast
➕ TOM SCOTT PLUS: / tomscottplus
👥 THE TECHNICAL DIFFICULTIES: / techdif
This was a really difficult script to write! Folks who know 'eventual consistency' as a more formal term may be annoyed that I'm using it a bit loosely here. Hopefully this works as an explainer for folks who are new to it, though!
Good job
I'm impressed by how well you covered caching, database server timeouts/lag, ACID, etc in lay mans terms @Tom
A good example of what you mentioned was 6 views but 300 likes on this video
View count : 48 Likes : 172
The Concurrent Counting Problem is so important, yet it's often neglected by many Computer Sciences courses. I imagine this video will be useful in CS courses too.
For people interested, I highly recommend the "Is Parallel Programming Hard, And, If So, What Can You Do About It?" free online book, from the Linux Kernel developers. It has a whole chapter dedicated to concurrent counting, showing various approaches to the problem.
"Computers are just overgrown calculators"
Something I will say to an AI in the future just to insult it.
And then it will whoop your ass with its digital Kung Fu.
actually, AI is the same thing as computing. AI that is actually intelligent is pure fiction, and may always be, certainly far beyond our imaginings. . It doesn't' matter that people make unfounded predictions. It's actually much farther beyond our comprehension that we generally think. Our dramatic underestimation of that gap is a product of some combination of popularized 'intellectual anthropomorphication' of AI, and just incomplete and inadequate thinking about what intelligence must mean as a word in order to be distinct from calculation, memory, filtering, etc. Consider we can't remotely, in our widest dreams, imagine a rational human language, not remotely. But for a computer that was somehow magically intelligent, such a thing would be required, it seems. But that's just part of the problem. AI 'learning' is in essence just filtering of filtering. Anyone that says or thinks there's anything about AI that is actually intelligent, is destroying the meaning of the word intelligence. IQ test scores for AI must be zero to be correct (they are often not, and this is a complete failure of test giving). Intelligence must mean 'conceptualization' aka 'cognition'. But it does not have to be 'accurate' or 'competent' it just has to be able to 'conceptualize' to be intelligent, even it gets everything wrong. So, dreaming and imagining is a product of intelligence. Many lower animals must be intelligent, it gets hard the lower down the scale to say where intelligence starts in life forms. There can be unintelligent responses to stimuli in creatures and plants, that are of course still useful.
@Christopher Armstrong brain is no a calculator
@@eltouristoduo one of rare but precise explanation what is actually wrong with all this AI hype. Thank you.
@@eltouristoduo You know this a joke right. No need to write an essay.
"Eventual Consistency" is the method I use for getting enough sleep. I'm sure it'll get there one day.
@@musekek3935 That went from 0 to dark in no time
@@musekek3935 nope, connect a few wires and you'll see. Sleep _/\√^v~^\_. Death is more a ______________. Eventual consistency of sleep would be a pattern /\/\/\/\/\/\. Otherwise its saying that a server's eventual consistency is reached when you discontinue its power supply.
TLDR (attempt to pull the thread back from the dark side)
@@andrevanrooyen6232 To be fair, ______________ is very consistent.
I can relate with that
Whilst watching this at midnight... same bro, same
My eyes: see 67 and 68
Me: ah yes, it's all coming together now
Just skirted 69 (lmao)
Hey we need one more like here
Leonard Winchester we need one less
computer: miscounts and doesn't quite make it to 69
me: this is outrageous. it's unfair.
@@jmmip202 d e s p i c a b l e
I had to explain this to a client once. You did a much better job at it. We had thousands of people per minute interacting with a map and deciding where to place their "donation". There was a counter on the page showing how many "donations" have been made so far and it ran into the millions. We had dozens of AWS Instances, 15 or so RDS Read replicas, and Elasticache servers running to handle the traffic. But the testers (who were sitting next to each other) kept reporting minor discrepancies in their numbers when glancing across at the other tester's screens. I kept trying to explain the concept of "eventual consistency" and that although each individual user should still see the number going up on their screen, two users might just see different figures. But we would still eventually sort it all out and get the down-to-the-penny accurate results at the end of everything. I wish I could have shown them this video.
I'm dealing with something like this lately too, a client asking why their internal activity counter was not matching Google Analytics real-time viewer counts. Need a Tom Scott video on the difference between users and sessions too now :D
A programmer is going to the grocery store and his wife tells him, "Buy a gallon of milk, and if there are eggs, buy a dozen." So the programmer goes, buys everything, and drives back to his house. Upon arrival, his wife angrily asks him, "Why did you get 12 gallons of milk?" The programmer says, "There were eggs!"
Tommy Callaway It’s not so much an issue with programming as an ambiguity of grammar.
@@KnakuanaRka r/woosh
Exception: Item "dozen" not found.
g.swell dead sub. Poor usage.
Bad programmer. Should have come back home with 13 gallons of milk.
*Tom Scott*
*Tom Scott Returns*
*Tom Scott Forever*
*Tom Scott and Robin*
*Tom Scott Begins*
*The Dark Programmer*
*The Dark Programmer Rises*
*Tom Scott vs. VSauce: Dawn of Knowledge*
Tom Scott 2: the electric boogaloo
Tom Scott Chriatmas Special (direct to DVD)
2 Tom 2 Scott
You forgot the best one that no one acknowledges, Tom Scott, Mask of the Phantasm
Tom Scott The Video Game (contains Downloadable Content)
8 minutes, one take, not a single unplanned "um" or "ah": Tom is so damn good at this! How he doesn't have a lucrative TV presenting career is amazing to me. I can only assume he doesn't want one and that he's very happy with what he already has.
you can't be in charge, unless you are THE DUMBEST PERSON IN THE ROOM.
(Dubya was POTUS! Rump was POTUS!! Rudy was MAYOR OF NYC! and so forth...)
RICH KIDS (who never worked a day in their lives) RUN EVERY ATOM OF OUR LIVES, cuz daddy made some calls.
NONE are smart or qualified or have any grip on reality. MONEY IS ALL THE CARE ABOUT.
They are sociopaths who are a threat to all society.
EX:
nothing is easier than hiring Anchors for CNN or CBS or FOX....who are SMART and charming.
Yet none of them can do this?!?!
MERIT DOESN'T EXIST.
I read this right as he said "uh" (4:27). Still an incredible feat, and technically the 'uh' there is more of a way of making the example sound more "friendly" by making it seem like something he came up with on the spot even though we all have the tacit understanding that it was already written into the script, but that was kind of a funny coincidence.
"Why computers can't count sometimes"
Let's count with Microsoft
1, 2, 3, 95, 98
@@zsin128 missing a couple in there, the real joke is where is 9?
Gaben is a computer confirmed!
Half-life 1, 2, alyx,..
Don’t forget:
Xbox, Xbox 360, Xbox One, Xbox One X
2000, pi, am space, 8, 10 YAAAAAAAAAAAAAY
@@JollyOldCanuck No, they run XP.
Great video! We get people asking about this topic all the time!
social bald
Tapstorm lmao
Tapstorm you’re bald
Bald
who u
I wish everyone saying "lmao 15 likes 0 views youtube is high 😂😂😂" saw this video
I wish everyone spamming the current year would stop and remove themselves from the site
@@phutureproof 2019
It's almost 2020 here.
Can we please have an official youtube taskforce that just links this video on those comments?
@@hazeltree7738 reporting for duty
K-pop community needs to see this, so they can stop complaining about views disappearing.
megalexantros , I agree, getting so sick of seeing "stop using emojis they delete views" on every new BTS video!
right. & they say "uR nOt a tRu bLiNk if u DoNt lIkE oNe oF thEm!1!1!" As if we can't have criticism.
I actually still have some doubts, would love for someone to explain further. I understand that too many ppl watching the same video at a time requieres yotube to register such data and finally proccess it and update the view count, but still I don't understand what provoques some views to be "deleted". How can youtube determine the view was not a real one, and make the number disappear? I say "disappear" cuz I've seen (putting an example) a view count going from 10.2 M, and then decreasing to 10.1M. I really don't want to come and be seen as an hard-headed kpoper, but I trully want to understand, please.
@@ribonette_ I would also like to have a bit more info regarding that phenomenon. As I understand it, the multiple hits can make the video gain more views than it should as a result of "bad calculations" before the algorithm finalises the real view count.
I understand however that this is a very crude way of seeing this and there's a lot more than meets the eye.
Nah, it's useless, they wouldn't understand
One of the most surprising things as I learned about IT is how much of this stuff that seems to "just work" really doesn't, but only provides the constant illusion that it does to the front end user. I never thought a calculation like this could be this challenging.
when the client gives 2 requirements: host 4K videos quickly and have a view counter.
Dev team says: okay we can implement the video stream in a week but the counter will take us 3 months.
This kind of "hard thing is easy and easy thing is hard" happens so much in development.
In accounting too. The complex reports are easy. It's the "easy" reports that take forever to compile correctly.
Client: I don't understand! It's just a number, the easiest part!
*Client is found dead in the morning*
That's because of almighty CDN providers that provide us with the videostrem itself. But the real-time counting have to be processed through distributed and highly sophisticated back-end system.
Andre van Rooyen How?
same reason why video isn't the problem when doing videochat. Picture didn't arrive? Part of an image is corrupted? You lose signal for half a second? Image arrives in wrong order? No problem, you still get the context.
But missing audio fragments or getting them in wrong order? That's gonna screw our weird human brain, that part has to be perfectly consistent.
I've learned so many random things thanks to this channel. These are types of the channels that deserve a CZcams Premium show.
Skills With Phil yes!!
Hey, i know your channel!
thats why its so great that vsauce has one
Yes, this infotainment should only be available to rich people.
@@JoshSweetvale You are so poor that you don't have credit card for free trial? xD
As a developer, I know about caching and basically everything you talk about, but it is still so interesting and I still learn from it. Especially how to explain things to other people. Thanks!
Its also a good reason to confirm that the "last one" of an item on ebay, Amazon, etc. is actually "still there" when ordered, or else you may be emailing the refund department as well. For the same reasons just described.
Anyone remember the old "under 301 club"?! :)
Ye
Yep
Yepp
Ye
Y
This channel is my favorite ever. I just found it a couple weeks ago and have been watching 10 videos a day, while I work and exercise. It’s very interesting and oddly calming
i really appreciate you and your team's ability to take something fairly advanced like this and put it into terms newcomers can understand.
I just want to complement you, Tom, on your narration. I've noticed the One-take approach before on your videos, but this one was impressive. It can't be easy to keep your inflection and pacing interesting for such a long take, let alone doing it in such a conversational tone.
*compliment
*remembers ‘Former UN Secretary General Ban Ki Moon’* (or something like that)
@@dirrdevil he's the anti-tom
does everything in >1 takes
cant explain anything
not bothered to do anything difficult for something simple
(search: define complement)
Q. Why did the multithreaded chicken cross the street?
A. To the get other to side
Notation Polish Reverse good soul the for is!
Easy to fix, I'll just enable atomicy:
A. To get to the other side!
Q. Why did the multithreaded chicken cross the street?
T No lie, my brain autocorrected that entire sentence and it took me way too long to see the punchline
@@woodfur00 wow same
!edis rehto eht ot teg oT
?daor eht ssorc nekcihc rettamitna eht did yhW
right at 7:47, when you said "All out of sync--", my video started to buffer, and I mistook it for comedic effect... It was real. And you sir? You've made me laugh for the sheer fact of how perfect it was to buffer at that moment. Brava, your video makes its own jokes.
Did a Brit just say, "math?"
Whoa
It's because he's adjusting his video for an international audience.
But does half the world say math? We say maths in Australia and I'm sure most other Commonwealth countries.
@@michaelclark3192 probably depends on what India says. That's going to be the driving factor.
@@michaelclark3192 not Canada
@@jessers1712 we say maths
As a distributed systems engineer at Apple, one of the interview questions I ask candidates is to describe the differences in design considerations when building a write heavy vs read heavy system.
This video goes into enough detail to pass that part of the interview with flying colors. Very cool to see it explained in such a simple and easily understandable way.
That is a great explanation, but there is one more thing at work! Many large social media sites deliberately send out slightly randomised numbers to make it impossible for clients to see if their views and votes are counted.
This is to enable so-called Shadowbanning, where an account's comments and votes are no longer counted, but the user is not supposed to find out. With this method they can ban vote bot accounts without the owner noticing it (which would simply cause them to delete and replace the bot) thereby reducing the viability of vote manipulation. On some platforms it is even used to deal with generally toxic users.
Wow, is this true? If so, I learned something interesting today!
It's true. Except that enabling shadowbanning is just a side effect and not the goal...
While we're on the topic, YT itself is a great example of this principle. Just take a look at the recent Diablo Immortal scandal to see these algorithms at work.
CZcams does this with comments too. If you post too many comments in a short period of time, others won't be able to see your last comments, even if you can.
This comment needs to be more visible. What an ingenious method to solve multiple problems at once. I never knew this existed, but it makes sense once explained.
"Counting things accurately is really difficult"
Pennsylvania would like a word
I still feel that deep within my soul.
Eventual Consistency applies to elections too, whether people like it or not.
Pennsylvanian here. Math hard.
Nevada's still counting!
I wish every K-pop stan malding about how youtube "takes away views" from their revered idols would watch this video
Same
I wish they have a working brain at least
You think they're intelligent enough to understand this stuff? Pfft
As a kpop fan myself I do consider sending this video if I see fans complaining. Most those complaining are kids tho they just don't know how anything works in life
They could not understand this video.
I didn't expect to find database concurrency, multithreading, caching and page replacement. Thank you for bringing computer science to an easier understanding. When you're too deep into it, you don't honestly realize why people are confused.
There are some systems that get _easier_ with scaling, not harder. For example, the behavior of a crowd is easier to predict than the behavior of an individual. The ideal gas law is awesomely simple and effective, but only works when you lump together a trillion trillion particles. Larger ecosystems are more resilient and self-sufficient than smaller ones. And so on.
Fun fact of the day.
also fun fact of the day - this all boils down to statistics and normal distribution
It all depends on the perspective.
Is it easier to predict the behaviour for every person in a crowd or for a single person?
Does every single particle behave as expected when lumping together a trillion trillion particles?
Applying your logic to the video's topic, it would be easier to count many likes than a few as you can approximate the real count.
@@ThePhoenix107 Yeah, actually, you are. If you're going to be ten likes off anyway, you're better off being wrong ten likes in a million than ten likes in twenty. Same absolute error, smaller relative error.
@@ThePhoenix107 It doesn't depend on the perspective. The more units of anything you have, the easier it is to predict its average behavior. It's actually called The Law of Large Numbers, mostly any phenomenon will default to normal distribution with specific mean and dispersion values the more trials you go through with. Look it up.
And yes, the experience and behavior of a single person would be considered "one trial" in relation top observing the crowd, indeed.
You forgot to mention reality itself! Physics is easy to calculate in common scales, but if you shrink down to quantum scales, it just doesn't work!
I'm a system's engineering. You succinctly stated what I was shouting at the video when you posed to the question. I love your channel. Scale is the problem
Not when you have solutions like Hedera.
The problem is not scale but education, the solution already exist, people just need to wake up to it.
Your subscribers is pie rn and it’s such perfection
Your editing is so on-point in these videos... it really helps to make what you're talking about easier to understand.
Fancy seeing you here! :)
Hey friend!! Small world!
Actually idk if you would call it editing those stuff we saw were simple created in AE
*_Overgrown calculators_*
*_Ambitious Abacus_*
*_Pretentious Pen & Paper_*
*Sleep is just extended blink*
lightning-trapping rocks
*Flying is just overgrown jumps*
PC: I can run graphically heavy games at 1080p 150 fps
Me:cool cool, how many people have watched this video
PC: *Windows shutdown*
With the xp shutdown sound as well
Ahhh there the problem - Windoze.
Its a horrible o/s.
Ms expects users to identify bugs
@@rty1955 Servers mostly run Linux😑😑
@@codeIMperfect haha Thats because there is a REASON for that
@@rty1955 Oooh🤣🤣
watching this (and more specifically, the ticketmaster queue reference) post-taylor swift fiasco felt like getting a cruel war flashback of sorts
seeing ticketmaster anywhere is like a jumpscare now 😭
Good ol' 301 views.
no 301, views have nothing to do with this
301 views was/is a thing because CZcams likes to verify that traffic to a particular video is normal and not botted or anything of the sort. They freeze the view count at 301 until it can be verified.
That's a "x > 300" vs "x >= 300" issue in their verification algorithm.
Landon Buttars but there's a very popular video whose views are stuck on 301 even though it has crossed 2 million views. It's "Why do CZcams video view gets stuck at 301" by *NUMBERPHILE* . I know it sounds like a meme but this is true
Edward Haas i watched it almost an year ago(probably) it was stuck at 301.
I want a video with the introduction "Hey VSauce, Tom here"
TSauce.
Hey, VSics, Tom here.
@@BEN-ys6gu why kevin?
3:36 the corrupted view counter animation has some things to say. Bj+ch
Okay next time I get the question "What is a computer?" You know im writing "an overgrown calculator"
Finally, something we can link on those "omg more likes than views" comments!
I could've sworn there was already a video covering that, but I've never managed to find it.
That's also because Likes are instant, but Views require at least some amount of the video to be watched to count.
I didn't think of that. Good point.
1. there is caching and syncing of Likes too. Just recently I had encountered that I put a Like an a video (which had 0 Likes), but then decided to remove it... A while later I came back to it, having decided to put the like back on, and was surprised to see that there was 1 Like already displayed. I clicked it, it said 2, refreshed the page, it was 1 again. Apparently it still had my old revoked Like in the system. I tried it a couple of times to make sure it wasn't coincidence. After a day or so it was completely sorted out.
2. It's not just that there is some amount of the video that has to be watched in order for it to count as a View (though that is probably true), it also is that there is a verification process going on, which takes a time. One of the things that process checks for is the number of seconds actually watched, but that is only one of several things, and CZcams keeps the actual details a secret as to prevent people from gaming the system (or at least making it harder).
"Why do CZcams views freeze at 301?" from Numberphile is explains that. Maybe this is the one you watched
@@manyaxxtr Apparently they have gotten rid of this feature. As long as they’re confident it is a genuine view, they will add it immediately. Meanwhile, anything that is possibly suspect will get queued up as a provisional view, and added later if it later checks out.
There's also vote fuzzing, which intentionally give as an inaccurate number to discourage manipulation by making it harder to tell whether it actually works or not.
3 years later, I believe CZcams does this when telling CZcamsrs how many subscribers they have, usually only imprecise by like 1 subscriber though
As someone who finds computers very confusing, I want to thank you for making this simple for me to understand. Now if only I had a few hundred or so lessons like this one, I may begin to comprehend the basics of computing!
If there is one thing that i understood over the years, Real-time operations are not good.
When I worked for Big Software, one of our products had a notoriously inaccurate progress bar. Our lead programmer explained why making the progress bar more accurate was a "bottom of the list" priority item: "The task that the computer is showing the progress bar takes up about 98% of the computer's processing power, with about half of what's left (1% of the total processing power) going to figuring out progress and animating the progress bar. If we want to make that progress bar more accurate, we have to take processor time away from the task (which is supposed to be most important,) to give it to the progress bar animation. This will make the actual task take longer, in a logarithmic curve to where the task will take forever and the progress bar will show an infinite amount of time remaining, if we let it."
That's because if the routines to calculate an estimated 100% have to take into account what impact the progress update code takes... This leads to infinity.
Hahah Big Software
I prefer the loading bars that dont try to give you a time frame but instead just has a byte counter of what is loaded in so far
Easy fix in two words. "Please wait" Oh and add optional spinning dots that do 1.7 rotations before resetting the animation.
As a programmer, that's BS or a sign of a stupidly implemented system.
The issue needs fixing properly so real work isn't delayed
FIRST!!
I actually was. My local server just held onto my comment for a while.
yeah, along with many other people
If only you'd said that second part with an edit.
Fester Blats it’s a joke
I had a feeling sb will say this
@Fester Blats ㄇㄍㄨㄜ
I know all that you're talking about in this video at a deep level, being a veteran web developer. However your communication style, examples and explanations had me see some things in a new light, even after 20 years of doing this for a living. Great job!
I have 10 years of different experience in IT (fullstack, devops, ml), but I still like to watch such materials of Tom's, with basics of an IT, because I know, that I will learn at least one thing or acknowledge certain thing or get an idea. T.Hanks ;) no, seriously. Thanks Tom:)
This would be an ironic video in which to wonder why there are more likes than views.
Well that's because CZcams waits for you to watch a set amount of a video for a bit before it counts the view.
Likes are instantaneous, the second a video is liked, the like shows up.
@@OctaHeart r/woosh
@Fireork Not only that, but once you get over 301 views, each view also have to be verified, if its a legit view as well, unlike the likes (no pun intended)
@@GummieI that system was discontinued a long time ago
Why Computers Can't Count Sometimes
I am always astounded by how well you present in one single take. Not having jump cuts all the time is really refreshing
I'm gonna start using "eventual consistency" as an excuse to slack off now. Thanks, Tom!
Having had to scale some of my web apps from a little single-threaded prototype to a set of deployed (sometimes asynchronous) microservices over a bunch of virtual servers and containers with cache, I had to learn all this on the spot, and you summarize it very well here. I will keep this video in my pocket for the next time I get asked such questions.
There were 48 views and 21 comments when I started watching this (3min after the video was posted). A few minutes later I refreshed the page; there were 24 comments and there were 179 likes but the number of views stayed stuck at 48 views - fewer views than likes due to sync lag - nice illustration of your point.
That was almost certainly the fact that CZcams counts a view when half a video is watched, whereas likes are (effectively) instantaneous. This means that if a load of people like a video as soon as they saw it, then watched it, the views would wait for half the video length
@@owenconnors1 Not true, you most definitely don't have to watch half of the video to count it as a view. That would be an awful lot of views lost, think about the ads in the beginning and the end that some CZcamsrs do and everybody skips. Some say "The generally-accepted number for watch time to count as a view on CZcams is actually a full 30 seconds" others say "CZcams's definition of a view is a viewer initiated intentional play of a video". I know that I had posted a video, and my family member watched maybe 10 seconds of it and it was added to the count.
@@tonyrulez69, at this point we don't know for sure because the algorithm is specifically designed so we can't know. That's because there is real money in play, and people will certainly try to abuse that system.
Bear in mind that CZcams pauses view counts, usually near 300, to do some validation work to make sure that the views are legitimate; some people try to artificially increase view count to defraud CZcams. But I expect that those pauses happen later as well if they spot suspicious activity.
The best best practical example of all the system design interviews I've ever had
Well, this video... IS AWESOME! I am really surprised and amazed at Tom Scott’s performance! Really learned a lot!
Wow. Covered this in Uni a few weeks ago, and you managed to summarise two 1 hour lectures into under 10 minutes. Sure, some things are lost, but I am still impressed.
3:39 "...and given that CZcams has to work out not just views, but ad revenue and money, it has _got_ to be accurate."
12/10, best joke
The title made me expect a video on floating point calculations and instead I got horizontal scalability. Well played, far more practically important to explain.
Really nice video Tom - and great use of graphics, very helpful!
This is perfect timing because I have been binge watching your videos!!
Same
Same
It really do be like that sometimemes...
...well, most of the time actually.
What counts as a youtube view? Watching 100% of the video? What if I skip 10 seconds? 🤔 What if I have the video on loop over night, does the CZcamsr get all my views for his revenue?
Google doesn't really talk about the algorithim behind the view count but for AdSense you need to watch ~30 seconds to add a view in a standard video so that gives you an idea. You can only give one view per IP address every 6-8 hours so looping doesn't do anything, in fact if you loop a video you can be put in the low quality / bot category and don't give any revenue to your youtuber.
@@mmdrdd interesting, thanks!
@@mmdrdd What if i watch 3 seconds of a 6 second video does that count as a view?
Alex
Even if it does there will be no revenue, thus it doesn't really matter....
Alex no
*Sees thumbnail*
Me: We were on the verge of greatness...
We were this close
It's at 69 now
Great video, I love how you simplify complicated topics
I created the infrastructure from the ground up on two high traffic, big data systems, and this is similar to how I used to do things, however I just stuck to the caches, and had a couple of replicated queues that hit one replicated database at some point. In this case, it was the database doing the queueing up of the big bundles, and the read-only database clones that were being asked for bundles of data to be calculated. Still an amazing subject to cover, and you make it really simple to understand!
OMG Ti-994A! I still have a few with their original boxes too! I loved them so much. They were very fun to learn basic cassette tape recording programming when I was 7 years old. They even had basic voice synth out of the box.
Hands down, this channel is one of the best on youtube
For several big systems, a versioning system is used so that if several threads try to modify data in a database at the same time, the system checks the base version and the modified version of the data to see if things are done out of order. However, it can lead to conflicts in the commits (same base version for two commits with different modified versions).
Thank you for a cracking explanation of something that sounds simple, but really isn't.
And yes, it may not have been 100% accurate at every point, but eventually everyone will get the right idea.
"Computers are just overgrown calculators." Ada Lovelace turns in her grave...
I have to say the visual overlays of concepts Tom is talking about are fantastic. I've had to instruct some coworkers on how to program efficiently on a particular controller, and always had problems explaining conceptual and virtual processes without getting a lot of blank stares. (Note, I'm not a teacher and have had no desire to be one, but when the word comes down from above that you're going to teach a class on how to program something...)
To be fair, they didn't even have powerpoint when I had to teach the infrequent class, but I really could have used the animations like those on this video.
I like the way he explains things. Simple graphics added to the video very smoothly - it helps us visualise the concepts
Machining systems also usually single--threaded. If there's more than one processor they'll use a master-slave system (the master requesting/sending information to each slave along the same single thread) or if there's more than one master via token-passing, each master being assinged a set time to send & recieve. That way you can ensure you got the numbers right, and your robot arm isn't still in the machine when it starts grinding.
Thanks for the video, really great to hear an example of how counting can get messed up :D
An amazing amount of work goes into making something look smooth, informative and useful. This was an AMAZING amount of work, done really, very, very well. Thank you, Sir.
I love how you're one of the few youtube channels left that don't have sponsors.
Amazing how complicated things can be, yet appear incredibly simple.
Great video! Learning about how computer systems work is tough, and your videos do a great job of walking through complex ideas in plain English, that doesn’t feel like it’s talking down to the audience.
I love it when Tom explains something. I makes it sooo easy to understand! ❤️
Watching these videos feels like taking a class on a niche situation
Your editing skills are special. Very simple but effective
4:30 I see United Airlines forgot to implement this feature.
Airlines do it deliberately because they have a statistical model for how many people don't show. If they over book they can give someone a partial refund and bump them to the next flight. There isn't a next show for a touring musician.
B T 2 David Dao joke? 🤣
Aww... CPC464... getting all nostalgic there! 😢 My second computer after a Tandy TRS80. Because of this I'm now a woodworker.
I'm feeling nostalgia for the TI-99/4 (or 4A) - my uncle used to work for them
The TI99/4A catched my attention also, I had one once and still regret I had given it away, as well as my CPC-464.
I've been watching Tom Scott for years and only just realised the red t-shirt is "his thing".
I do databases since 20 years and I love your explanation. Will point all beginners here.
Thank god, I was looking for something good to watch then you uploaded. Saved me from watching some boring video on CZcams
CAP Theorem -- Consistency, Availability, Partition tolerance -- Pick 2! Since partition tolerance is rudimentary property of a distributed system, you can either choose consistency or availability :)
Wait what? Where is this theorem from?
Great video. Simple explanations for seriously complex concepts (at scale).
Love the background. I recognized two computers I had when I was young :)
About queues though, you made it sound a bit like queuing is a bad idea at the beginning. It's not, all software at scale is tied together with buffers (caches) and queues. The scheduling and handling is fairly advanced, but it still is just queues and buffers that allow software to be loosely coupled and survive the occasional software or hardware crash.
interesting! I always just kinda assumed that was lag on my end when things like view counts went up and down, different updates going through out of order
I think it's fair to say that counting accurately, _quickly,_ is the difficult part. If numbers are flooding in faster than your counter can count, which is certainly possible with how big the Internet can make things, you're gonna have a bad time (or some clever engineering).
@Growtopia Mods not all computers run at the same speed, or even at high speeds for that matter.
Also, the human population is a big number, but what’s important here is the rate at which the values change. The approximate net change in population is +2 per second. You’re right, most computers can handle that rate. But when Tom Scott uploads a video, a small percentage of his (currently) 2.1 million subscribers are notified that he’s posted a video. Let’s assume that 10% are notified, and that 10% of those notified will open the video within 10 seconds. That comes out to 1,000 people clicking on the video per second. And there are thousands of CZcamsrs, and hundreds of thousands of viewers, scaling way, way beyond the rate of our measly population’s growth rate. That’s what CZcams’s servers have to count.
Typically excellent video and amazing backdrop...remember the home micros very well.
7:35 What an incredible graphic! Really puts things in perspective.
"This was a triumph! I'm making a note here: HUGE SUCCESS!!!"
Did you have to adjust the camera to the screens in the background? Or did it just work
PAL TV at 25i matches the frame rate of 50Hz, but NTSC with a rate of 29.97i doesn't perfectly match with a 60Hz frame rate. Getting good video from a CRT can be difficult, and may require adjusting shutter settings.
Nice profile pic
@@jasperkooij1891 What a Save!
One thing I love about Tom's channel is that I can find good questions like that one in the comments section. Normally the comment section is overwhelmed by jokes so that the interesting questions drown.
@@thefrosty1925 Good idea, ima go play that now.
This channel really is phenomenal
Love the TI 99 4a and Sord M5 to the left of Tom Scott, I had both as a child.
gonna link this video to all the comments making fun of youtube for having more likes than views or more comments than views
that would actually be very helpful! I just shared it with my friends and now are having a nice debate about how this algorithm works C:
Cassandra Database Engineer checking in, this was a wonderful explanation!!
You made it very clear, Tom! If only all my programmers had it all in mind all the time...
Nice Video, but what's even more amazing is that I learn more from your videos, than from my school lessons