Coin Flip Cheaters: A game from Primer
Vložit
- čas přidán 8. 04. 2022
- Play at primerlearning.org
Or on Google Play: play.google.com/store/apps/de...
For discussion and updates
- Discord: / discord
- Twitter: @primerlearning
- Reddit: r/primerlearning
Plush blobs and other merch: store.dftba.com/collections/p...
Support these videos on Patreon: / primerlearning
Made with Unity
github.com/Helpsypoo/PrimerUnity
Made possible by support through Patreon:
Anthony Eufemio
Jon Mundle
Spline
Zachariah Richard Fournier
Vladimir Duchenchuk
Roy & BreAnna Steves
Shayn Osborn
Jeremy
Guguke
Anders Fjeldvær
Luc Cedric R.
Erik Broeders
Kairui Wang
Sean Barker
Eric Helps
Stevie Hryciw
Tim Barber
Ben Kamens
Bez Noferesti
Kieran Boulton
vigo oudenaarden
Andrew Lang
Flavio Kindler
Cowit
John Shaffstall
Alba Caparros-Roissard
Sam Shaw
Jeff Linse
Timothy Furman
Kevo
Gabriele Siino
Ghost Goat
abledbody
Alex Garber
Krum Valkov
Matthäus Pawelczyk
OuroCat
Jacob Luedecke
ketura
Sam Van Cise
Kevin Holesh
Daniel Kjellevold Steinsland
Daniel Schramm
Jason Prado
Jeff
Garrett
Brian Cloutier
Rikard Eide
Christy Serbus
after three hours of playing the game I've realized you've essentially created a slot machine for Bayesian statisticians
Funny, I wrote a Bayesian script in R to play this game before seeing this comment.
E
@@EEEEEEEE in what language do you speak? Why did you just type “and”
@@wow-roblox8370 he speaks English and don’t trust google auto translate lmao
Yaratmak ALLAH CELLE CELALÜ'ye mahsustur. LA İLAHE İLLALLAH
Wait, he’s using plushies to “pay” us to play his game, this game is helping gather data about how you sometimes have to trade cost and accuracy in data sets…
Well played, well played
the easiest way to catch a cheater: it wears makeup
@@Blox117 or probably a mask
@@majorgeneralrahul6298 the makeup is the mask. also, high heels
@@Blox117ok i know you’re sad that sharon cheated on you and took the kids or whatever but please get a therapist instead of doing whatever this is
@@luna010 see even you know they cheat a lot
Kinda neat, I wonder (statistically) how sure one could be with 15-16 flips per blob
Well stay tuned for the next video (actually fairly soon)
@@PrimerBlobs I'll look forward to it!
@@PrimerBlobs Wait what? A quick upload time, who really are you ‘Primer’?
Not that certain at all. Sample size is still really small
Given no other knowledge, you cannot gain any minimum amount of knowledge about whether the coin is fair or biased no matter how many times you flip it. That's because the unfair coin could be chosen to have a bias of 0.5+epsilon for any epsilon. Since epsilon can be made arbitrarily small, the likelihood ratio (between fairness and unfairness) per coin flip is also arbitrarily small.
His existential dread over sad findings of human behavior might be over.
Never
@@PrimerBlobs well, here we go again!
One thing! Just one thing! Please tell IT to me: WHY tf do I have so many fans even though no CZcamsr is unprettier than I am? WORLDWIDE!!!! WHY??? Tell me, dear nöi
@@AxxLAfriku I'm guessing you are a bot.
@@2008bunnybigenderflux you are correct
As a side note, I've found myself biasing against certain blobs more. For a while, it felt like Sunglasses (both Star sunglasses and normal ones) blobs were cheating more than other blobs, so I was biasing against them.
So.. you’re racist?
lol I didn't even think that appearance might count
This is part of what is sent to the server. I'm excited to see how this shakes out.
@@PrimerBlobs ooooh. Can't wait to see the biases everyone has.
Hehe I only looked at the numbers, it didn't even occur to me that the blobs could be a factor! Really interesting to see how different people might play the game.
damn those guys at the top of the leaderboard sure are good, and also committed! ironic that a game called "catch the cheaters!" has no anti-cheat lol. well done anyways - it is a really nice concept to see you covering
I mean the highest people i see are around 800. I hit 80 before I intentionally failed so I don't think thats unrealistically high. Were there higher values before that got removed?
@@DollyBowman I got to 170 score with 173 flips left before the game froze (I didn't listen to it and played the browser version on my phone, it got very laggy at around 200 total blobs), so I'd say 800 isn't unrealistic for someone who was dedicated to it and got good luck
@@DollyBowman yes, people had hacked the game within the first hour or two, and had scores such as: 69420, 10000000000, etc. I'm guessing it's been fixed now?
@@s.t.-1094 Those are the blatant ones.
The trick is to cheat with a realistic enough score that still puts you at the top of the leaderboard.
So I wrote a simple AI for this problem and looked at the distribution of scores.
n = 20
mean = 162.8
median = 15
max = 1758 (wow I don't have the patience for that)
min = 2
"Cheater coins have a 75% chance of landing heads"
This is important info.
this *really* changes the game
oh yeah I thought they had a random value between 50 and 100% which would have made it really hard to guess a cheater that was at like 55%
Yeah, I didn't know that when I tried it, but whatevs.
It's amazing how important it is and how important that number is. I brute forced the optimal strategy and calculated the expected value per round (per blob).
It's -0.128856. That means games will eventually end, even with perfect play because there's a slight downward trend.
But it you change the cheater coin from 75% to 75.27% and play with the optimal strategy for that scenario, the expected value becomes 0.002713. This would allow for games to continue infinitely.
The same thing happens if you make the reward 16 or the penalty 29 or change the chance of the blob cheating. It's crazy how close this game is to being endless, but it isn't.
@@daniel.lupton Isn't the 75.27% version still likely to end due to a stroke of bad luck? Because a stroke of bad luck immediately ends the game, so like, if you could continue even wiith negative points you'd be guaranteed to eventually go positive again, but since you get immediately stopped there are still high odds you lose anyway right?
May I recommend, if you decide to update it, to add a hard mode where cheaters can have a coin loaded either to heads or tails? I wonder how that'd change things
Same here. Cuz rn if first 5 flips are tails, the blob is preety much guaranteed to be fair
Fun idea! Though we already don't know a lot, since we don't know how much the coins are weighted. Is it a random value between 51:49 and 100:0, or always around 75%? I feel like it would make the 30 coin penalty a bit harsh and turn the whole thing into more of an RNG-based guessing game to also randomize the direction of the weight...
EDIT: Nevermind, it's in the rules that it's 75%. Still, 30 coins would be harsh...
@@majascha3414 in the game it says it's 75%
@@fanjason6137 I had one with 4 tails and 1 head, and it was a cheater
If all cheaters could be cheating either towards heads or tails, with the same percentage chance and the same percentage weighting, it would make it impossible to have a strategy at all. You'd basically just be blindly guessing.
This reminds me of a math teacher [a real one] where he proved to his students the variety in outcomes would be more than they thought.
He gave them a homework assignment of doing & tallying 200 coinflips. He told them them didn't have to bother with the coin [i.e. they could "cheat"], but he would know who did do the assignment properly & who didn't.
He guessed right right every time, mainly because the "cheaters" would never include enough 7, 8, & 9 consecutive results that would always come up when you 200 coinflips.
He is sure a good teacher
and then someone copied results from randomizer, but made it biased
I could just imagine the kid who did it, felt the results did not make sense and than edited his rolls.
I love this so much. Human perception of random systems is fundamentally flawed, and I thoroughly expect this "game" to produce a dataset that bears this out. Fischer Yates with a 64-bit Mersenne Twister is the gold standard of pseudorandom number generation for a reason, and is simply as close to truly random as random can get without going quantum with it. 99.9% of the time when someone claims something isn't "random", what they really mean is they "FEEL" it isn't random, that it isn't consistently inconsistent. I don't recall who, but a very smart person once said "if something feels random, it probably isn't." Truly random systems have streaks, and aren't prone to even distributions, especially in such small datasets. Love this. Love, love, love this.
The PCG family of random number generators are superior to Mersenne Twister. Have been for years now.
Yeah, confronting people with this is the main reason for the game. Not trying to teach anything in particular with it except that you can't trust your gut for probability judgements. (of course sometimes that's all you really have in which case I hope you're well calibrated)
You see this all the time with TCG/CCGs, where people will be like “oh I must not have shuffled properly, it’s acting similarly to last time” and much more
@Kaitlyn L As a Magic player, this is exactly why I have researched so much about random systems, and why I am so excited for the next video!
I'm not gonna talk about my strategy, unless you guys want to, but at the start I knew that it was risky and its weakness would be a streak of unlikely flips. And said streak was exactly what got my otherwise pretty good run to end. I was averaging 0.84 points per guess (1 guess every ~2(+/- 1) seconds) and having around 100 flips remaining (25-75% being 80-145 flips, with a max of 205) when a few consecutive blobs brought me down to my knees. Lost with 103 points. I didn't pay attention to my average flips per guess, tho. As a matter of fact, it'd be cool to have all these statistics at the end screen. Points/guess, guess/second, (flips_spent+guess_reward)/guess...
After over 7 months, the legend came back!
With a game
This is epic
Countryballs Brasileiras!
Then we have to wait another 7 months
On some Android devices, the app may not remember your personal top scores if you close and open the app. If you see this, please comment here or on the play store listing to let me know what device you're on to help me track down the issue.
If you do see this, your scores and drawing entries are still stored in the database, though.
Ok :D
Hey, for the game, I think it would be better if the instructions were on the side, cause Unity disables scrolling and even on an 1080p resolution I couldn't see the controls til i zoomed out. (On chrome)
Can you make an option to not be stored on your database, and not be on the leader board? I'd like to play offline...
I could also become the database, I have 1 terabit / second internet. Storage space? Well my FLASH DRIVE is 1 terabyte
Offline mode would only be possible for the android app though...
@@bitonic589 You don't need to submit a name or email. It will automatically store anonymous play data though. I did that because I want to collect it to analyze for the main video.
@@PrimerBlobs oh ok
I was literally watching your videos last night and thinking "Man, he hasn't uploaded in a while, huh" Thanks for the video, Primer!
SAME!!!
not last night but I look and was like "oh no- I JUST LEARNED ABOUT THIS CHANNEL AND WATCHED ALL HIS VIDEOS IN A DAY ToT"
@@olivebre4170 LOL I DID THE SAME THING WHEN I FOUND HIM XD
this thing nerd-sniped me to hell and back
Hi Xidnaf, suprisingly you are alive!
Yooo xidnaf! Surprised to see you here
yoo xidnaf
Congrats on the great score!
@@PrimerBlobs thanks ^^
Just got recommended the evolution videos a few weeks back. Working thru them. Immediately was hooked and then bummed that it looked like the channel went quiet. Happy to see you’re still making videos!
This is a great demonstration of Exploitation vs Exploration. This is a common design-of-experiments problem where you want to make a decision with a minimum number of test data. For instance, if you only project 2500 people will click on your advertisement, but you use the first 2000 people to test whichof your 2 ads is the most effective, that only leaves 500 expected people to apply your data results to (ie showing them the best of the two). You essentially lose value on the 1000 people you showed the less effective ad to.
In this game, every flip (collection of data) has a cost or weight attached to it. You want to minimize your flips spent while maintaining an average number of flips gained that is greater than or equal to your flip cost. Such a strategy may have a tolerance that allows for a certain number of incorrect classifications depending on the cost of incorrect classification and the gains incurred by a correct guess.
Oh man, I love your blob animations and getting to see them do their thing at the press of a button will be very gratifying :)
Holy crap I was just revisiting this channel. What great timing!
I've been looking at your channel every odd week in exasperation. Glad to see another video!
It’s always great when you upload!
Excited to see what the full video is about! Multi-armed bandit perhaps?
Love primer videos. Not only do they have crazy interesting concepts, those blobs have brought more happiness into my life than literally anything else.
this is such good timing, came out just after i finished studying hypothesis testing/ type 1 and 2 errors in stats :)
I love the hat blobs, you are an excellent content creator. I hope you keep making content, keep doing it at your own pace
One video per year but still amazing work
Hey hey, I've been managing two!
@@PrimerBlobs Loool, that was not a critic btw. Way better then the ytubers that post everyday without quality, love to watch your videos man!
discovered your channel during hte first wave of the pandemic and wait for every single one of your videos since then and i'm watching them in the hour after they come out
I went off the deep end with this game.
It's deceptively simple. I did a a full brute force statistical analysis of the optimal strategy. The expected value per round is exactly -0.128856.
This means, on average, per blob, if you play perfectly, you'll lose 0.13 flips. This is great because it's about the "house odds" of most casino games. It also means if you play long enough, even perfectly, the game will eventually end.
What's really crazy about it is how precise these numbers are. If you change the reward to 16 or the penalty to 29, the expected value is positive (allowing infinite games)
If you change the bias of the cheater's coin from 0.75 to 0.7527 the expected value is positive. Games can last forever. 0.0027 is all it takes.
All these numbers, 0.5, .75, 15, 30. They're so round you'd assume they'd been picked arbitrarily. But if they were anything else the game wouldn't work. It's on the knife's edge of being infinite or being unplayably harsh. There's something beautiful about that. Well worth spending half my easter revising statistics and optimising algorithms.
To work this out I wrote a recursive algorithm that simulated the expected value of every "position" where there's fewer that 400 head or tails. It assumes you play optimally all the way to the 400 flips point then works backwards all the way to the start. (If you work forward you need to evaluate 2^400 games, so back-propagating is the only method that is computationally feasible). Once you work out the optimal strategy you can just store the moves and simulate games. Here's the results of 1M games.
n = 1000000
median = 77
mean = 698.4
stddev = 3404.5
max = 220228
With a stddev ~5x the mean, it's an extremely skewed distribution. Most of the games end before 77 points but some outliers go out to the 10s of thousands. Wild.
I really like this game. When I first saw it I thought it was just a toy to explain conditional probability (and it is) but there's so much complexity hidden underneath such simple mechanics.
Wow I feel inferior. My algorithm can only manage an average score in the high tens. (I only have a sample size of 100 games, though.)
But I can confirm the parameter stuff. Changing the reward to 16, the penalty to 29, or the biased coin's probability to .754 produces a positive E.V.
@@TroyVan6654 With a sample size that small, high 10s isn't bad. Most of the weight from the average comes from the massive outliers. I ran two independent 1M game runs. One had a median of 77, another had a median of 40. Same algorithm just different rng seed. You'd think at 1M runs, at least the median would be fairly stable but even that fluctuates. So with 100 runs, it's anyone's guess what you'll score.
I should update this. I realised my play strategy is based on the idea that you have 400 coins and are trying to maximise the return. When you only have 1 or 2 flips left, the decision changes.
One day I'll write up a full solution. But making the AI consider its option better at low flips seems to have modestly boosted the mean. ~ 20 or so. Enough that it's worth including in a "perfect" strategy.
I got 704.267 for E.V. with the perfect strategy. I solved this by using a linear approximation for the tail F(n)=F(1000)+6.134419315491878*(n-1000) where F(n) - expected points with n coins left.
Your videos are so interesting to watch!
Please make some more :D
Now I'm thinking... What happens if you try to cheat in a game about cheaters?
Depending on how the game is built it could be trivial (solution or score is calculated/stored client-side) or almost impossible (a server handles everything, you just submit answers).
Let's hope it's the second to make it fair ;)
If the NSA wanted to get the high score, I'm sure they could do it, but I'll guess it's not trivial. :)
The top of the global leaderboard is a guy with a score of 69420, who is around 69000 points higher than the second place participant. I think we can safely say this has happened.
@@PrimerBlobs Someone has cheated just a few minutes ago
Probably not trivial, but seemed easy enough for someone to do it in the first hour
@@whetlands Yeah that's me... it was surprisingly easy. @Primer sorry for putting that on the leaderboard, didn't think it would really work. Remove it please.
@@whetlands The guy (previously "Anonymous blob" changed his avatar and name now to "Is there any Anti-Cheat?". So he definetly is able to influence the scoreboard... What a shame, I had the second score with 284 points :D
Flipping great game!
But…are you going to check the leaderboard for cheats using statistics? :)
Mmmmmaybe. :)
@@PrimerBlobs Bring on the meta!
@@DrAndrewSteele It's looking like this may indeed be necessary.
@@DrAndrewSteele I got 182 with this strategy: Flip 1x if H, continue; Else guess fair. Flip 2x: if HH, guess Cheater; Else continue. Flip 3x, if HTT then guess fair; Else continue. Flip 4x, continue. Flip 5x, if 4 Heads and 1 Tail, then guess cheater; if 2 Heads and 3 Tails, guess fair; else continue. Flip 6x if 3 Heads and 3 Tails, guess fair. Else continue.
It's not ideal but it's an okay strategy I think.
False Positive = 25.4%
False Negative = 34.1%
@@PrimerBlobs is it normal for 50% of the 5heads:0tails blobs to be fair? 3 of those improbable fair blobs came to me in a row and killed me instantly.
It has been happening before this one but, when three of the first three blobs did that, I was like
Woah, just one blob doing that is one in 32. And this is probably something wrong with your code.
Or am I just unlucky to have more fair players with 5:0 than cheaters
Having corrected homework for a statistics course that taught the concept of confidence limits in probability quite alright.
Well timed video.
HE’S BACK, AND DECAPITATED AS WELL
Edit: Just realized he wasn’t decapitated, his shirt just blends so well into the background.
you blind
Decapitated implies he had a body to begin with. :)
Debodied?
Probably better if which side the cheater tries to cheat with is random.
Aside from that, really cool concept
Well sort of. It actually changes the math for this problem and im not sure if thats the intention.
@@dondonnysson4973 my only concern is that if the results sckew tails, then you know it's fair almost by default, which I feels takes a bit away from the intrigue of the perception of fairness
It would make the game harder, but as a result would probably make it a worse example to start learning from since you generally want to start with the simplest possible example that covers the given problem.
Statistically speaking, it's already impossible to break even in this game in the long run. If the bias could go either way, the reward/penalty would have to be significantly adjusted to make the game remotely playable.
I really like the purity of this game. The numbers are simple. The rules are simple. But the mathematics is still genuinely interesting and often unintuitive.
For example, after 3 tails, you can expect a fair player, but if you flip another coin and get a heads, your confidence goes down. You have more information but the expected value of your move gets worse.
This concept is so simple but so unintuitive. I think making the game more complex might hide these simple insights behind a wall of statistics.
I'm gonna play it just because the blobbies look so darn cute - great animation!
I fell in love in this game. Simple, satisfying and helps to gather data for nice cause.
The return of the king
i feel like the game would be more interesting (and realistic) if the cheaters could cheat into either direction
I think we're supposed to assume getting heads is a positive outcome for the blobs, not that their goal is to trick us. Being able to cheat with tails would be equivalent to a student actively getting wrong answers just so they can brag that they cheated on a test without getting caught.
maybe more interesting, but oddly, less realistic.
When people cheat IRL, whether in a game, or in taxes, there is only one direction they cheat in.
If any blob could just decide to cheat in either direction, then there wouldn't be a way to correctly guess which ones are cheating without spending significantly more coins, since small sample sizes are pretty unpredictable. The way it is now is pretty balanced. If you aren't stupid, you can keep it going forever, however it gets boring after scoring over 100.
So glad to see you upload again!
I just subbed, and theres a new video, such nice timing!
I just accidentally realized that you can really abuse the fact that this is a game if you want highscores, by just guessing the first few rounds without flipping coins at all and then continuing normally once you get a lucky streak... I just wanted to lose quickly and clicked cheater a few times, and miraculously got them all right ^^'
I did the same and ended up doubling my coins before actually playing.....but the pattern in playing that I used was pretty trivial and lasted quite awhile. I got bored to continue.
first time i did this, i guessed correctly 9 times in a row, and each was a deliberate choice based on the previous results, not just random. if i went from there, i could have gone pretty far, but i kept guessing and died with 19 points
This game is brilliant! For the leaderboard, I would love it if you would add a histogram distribution of all scores with an arrow or line showing where you are, like how leaderboards work in mini motorways or the zacktronic games. That way you can get a sense of your percentile at a glance.
It would also be cool to have different modes that score the 4 outcomes differently. If the consequence for cheating is extreme, the cost of injustice for a false positive would be high, so you'd need a strategy that only accused cheaters if the evidence is beyond a reasonable doubt, even if you let some moderate cheaters slip through the cracks. On the other hand, the objective could be quality control, and it is important that only honest blobs make it through, even if some honest blobs get mislabeled as cheaters, like medical trials or quantum physics experiments.
I really enjoy this channel because he explains complex ideas simply so it makes me feel smart
If you spam click the cheater / fair button while you're losing, you can accidentally "vote" on an extra, unshown blob. This results in impossible remaining flips such as -40, and also doesn't present the reset button or an option to enter the high score. It does not matter whether animations are disabled or not.
It's frustrating when they get 5 heads in a row, and it turns out they're not a cheater. The chances of that happening with a fair coin are 3.1%.
And therein lies the rub - It's the Gamblers Fallacy. You can't rely on previous results to inform the probability of the next. A fair coin flip can only ever have one of two results, and every flip is always 50/50.
@@gloriouslumi I don't think that applies here. I might need to use a different formula, and maybe the odds of it being a cheater are less than 97%, but most of the time it is a safe bet to immediately label them cheater.
I guess the upcoming video will tell us the proper formula. I look forward to it. I had trouble working out the right balance of risk. Got to a score of like 70.
A fair player has a about a 3% chance to go 5 in a row, but a cheater only has 24%, so about 1 in 9 times you see a 5:0, it's fair player.
@@PrimerBlobs Thanks, wasn't sure how to calculate the relative likelihood. An 89% chance of success is pretty good; it seems to a positive investment. I used to know the formulas for this sort of risk-assessment, but now they're tricky to find online.
Looking forward to the video, to serve as a good example of this sort of risk calculation.
It's a 12.5% chance of being fair. You have to consider, it's also unlikely for a cheater to get that many heads and then compare the probabilities.
SOLUTION(?)
(1) You can get the probabilities using the Bayes' theorem. After some rearrangement, I have p(fair|h,t)=4^(h+t)/(4^(h+t)+3^h * 2^(h+t)) where h is the number of heads and t is the number of tails.
(2) You have to decide how close to one has to be p(fair|h,t) to call "fair". I tried to find the value analytically, but for me it is too difficult. So I will program a simulation and try different values.
(3) My small program tells me that the optimal value is close to call fair blob when p(fair|h,t)>0.75 and call "cheat blob" when p(cheat|h,t)>0.75. Then I get averages close to 850 blobs!!!
I believe that the next step is to get the expected value of choosing "fair", choosing "cheater", and choosing "flip one more", and choose whichever value gives you the most expected number of flips
I calculated the optimal solution computationally (enough that the tiny rounding errors have no impact on the decision). I don't know if there is a definitive analytical answer.
I ran this through 1M trials and the average is about 700. But it's such a volatile game, even 100k runs had fluctuating averages. Actually check this out. Here's the result of two 1M game runs using exactly the same code:
n = 1000000
median = 77
mean = 698.4
stddev = 3404.5
max = 220228
n = 1000000
median = 46
mean = 703.0
stddev = 3490.9
max = 213537
The median is all over the place even after 1M runs.
The optimal strategy is to consider not just the expected value of you're current guess but also the expected values of the next guess (and so on, recursively). You have to consider if you get a heads or a tails (weighted by the odds of each given your current assumptions). Then, if that new expected value is more than 1 flip higher than your current expected value, it's worth flipping again. It doesn't matter if your current expected value is bad, it matters if the next guess is any better. These two facts coincide more often than not, so a naive strategy often works pretty well, but it's a fun thought exercise to really min/max the results.
This channel is a real inspiration
If this was a secret ad for those plushies, it worked because I just bought one. Keep up the great work!
The best strategy by far is to just flip once and if it's heads guess cheater if it's tails guess fair I got 122 second try lol.
You should get a mean score of 40 if you continue this strategy forever. Actually closer to 39 but who counts
Definitely the fastest strategy. There's value in that. If you can play faster than anyone else you have a better chance of a lucky winning streak. But it's also definitely sub-optimal.
It actually is the best strategy probabilistically (at least with my simulation). I've programmed a simulation that ensures that if you follow this rule you'll end up getting right a 90% of the cheaters.
@@howdyimflowey4341 I mean, that's just not true. A heads has a .75/(.5+.75) = 3/5 = 60% chance of being a cheater. A tails has a .5/(.25+.5) = 2/3 = 60% chance of being fair.
That means whatever the result you have a 60% chance of guessing right after 1 flip. (which is not 90%)
But you need a 66.667% chance to break even (because a loss costs your twice as much as a win gives you). The remaining 6.66% of the 45 point delta = 3 flips lost. (and that's not counting the flip you spent)
So this strategy will lose you 4 flips, per blob, on average. I don't know how you set up your simulation but it's either wrong or you're lying because a very simple math check demonstrates how wrong it is.
Can you please add a way to speed up the labeling animation? And a way to automate labeling? I know that if I get x H and y T then I'll always label z, for many different pairs of x and y values. At this point it's probably faster to reimplement the game to validate the strategy locally, set up screen capture, ocr and automated clicking to run my algorithm than to manually click through. I'm at 235 score and 448 flips left, thinking of abandoning the run, while the top leaderboard score is over 900. Fun mini game at start but clicking through gets tedious quickly.
The blobs are amazing!! A great way to gain statistics while making it fun!!! So many of the simulations I’d love to play!
In my first game of this, my strategy was purely based on flipping zero times, once or twice, and then using my guessing powers to guess which one it was. It worked effectively, leading me to the 150s and 10 score, but then I had a major losing streak.
SO MUCH FUN!
This is really fascinating and I am excited by this and the next topic!!
He rarely updates videos, but when he does oh are they good.
No way! You haven't upload for 7 months, but the day after I find you and subscribe you uploads again! Where you just waiting on me this whole time? Lol
But seriously, glad to see you upload another video! I fully expect there to be big waits between uploads, but I'm excited to see more whenever you have them ready!
Happens to me too all the time
IT'S BEEN A WHILE I'VE MISSED YOUR CONTENT
I have been waiting for another video for a long time!
Probably not but I’ll still watch anyways
Ok, first strategy, use 10 coins and based on the results, try to get 15 flips. If you are still not sure, it's better to spend another 5 and lose less.
That's not a good strategy. If you start with tails-tails you don't gain more value by doing additional tests (or at least, the value is less than what you spend), same if you get heads-heads-heads. You need to flip the coins one by one, do a bayesian analysis of your current result, then a bayesian analysis for one extra throw as a heads, and an extra throw as a tails. Do a weighted average according to your estimated probability that it'll be heads or tails, and calculate whether that extra information is worth 1/45 of a correct answer.
@@GregTom2 To do one better, you should apply this recursively. Test the expected value of a guess vs the weighted expected value of the best case (flip or guess) of the next move. There's diminishing returns here, but a couple of iteration definitely change the results of your gameplay.
A curious result I found is that for any turn after the first, there are 2 and exactly 2 counts for heads that require another flip. For every other possibility, you should take a guess. I'd have thought that range would grow or shrink, at least for edge cases if not as a general trend. But at least for 200 turns that's always the case. The more I play with this the more surprised I am by the results.
Now I'm just reading comments and sharing insights. :p
At last another upload! Thank you 🙏🏼.
i would love to see more of these games that are based on the other simulations.
I was wrong!
L imagine being wrong
Hahaha! It wasn’t a year!
Looking at the leaderboard I'd guess the leaders are scripting or have an insane amount of time on their hands.
I stumbled across this while I was looking for something to practice the new Excel Lambda function with. Long story short, I've written a solver in Excel. In my simulated results, it can get scores over 100 before my tables runs out of entries. It assumes a 50% chance of a cheater and a 20% chance of the coin being "dropped". (Are these numbers listed anywhere?)
My methodology was pretty simple. You want to increase the expected value of a correct answer, but only while the increase is worth the cost of the flip. It's a stateless game, so all you really need to consider is how many heads and tails you've seen in this round.
So I started by determining the odds of getting a given combination of heads (H) and tails (T) for both a fair coin and a biased coin. This is done using a binomial distribution. (built in to excel)
From their you can determine the chance that the blob is cheating. This is determined using Baye's Theorem, which in this case is just a fancy word for a weighted average.
Now you know the chance of a cheater for any given distribution. You need a 66% certainty either way to break even. But what we really want to do is maximise certainty while minimizing the flips. The real question is whether to flip once more, or take a guess.
The easiest way to approach this is to consider the expected outcomes of each choice. If you take a guess now, you will win or lose based on the known expected value. If you flip once more, you'll lose 1 flip and then either 1) lose the coin or 2) get a heads or 3) get a tails. If you get a heads or tails you'll know more about the round but might actually reduce your certainty in your position. You should consider the expected value of the resulting positions (more on that in a bit). You could also assume a 50/50 chance of heads or tails when calculating your expected value, but you actually know more than that. There's a chance the blob is a cheater which will increase the odds of getting a heads.
So we can now make an informed decision about whether we're better off guessing now or trying our odds with a flip and guessing then. But that still leaves out a big piece of the puzzle. What if we keep flipping, can our odds improve and how does that inform our immediate decision? The standard tool in the programmer's toolbox for this kind of problem is recursion. Now, I'm sure a mathematician could produce some infinite series and prove it converges to a simple function, but I'm not a mathematician so I'm going to make the computer do the thinking.
It turns out, if you recursively evaluated the expected value at each state, you're encouraged to flip more than if you naively take the expected value of a guess at each state. As you add more levels to the recursion, it does have diminishing returns (it converges) but not as quickly as you might expect, so it's probably worth optimising.
What's also surprising, from my findings, there's also no definitive "end" to a flipping sequence. If the distribution hovers around the midpoint of the two expected distributions, it stays in your best interest to keep flipping. This is counterintuitive and I'm not sure I've gotten that entirely right (again, not mathematician).
Still, whether I've done the mathematics right or not, my little excel script seems capable or almost limitless play. I'll have to take a crack at the leader board. This was a great thought experiment and even though it's dealing with entry level probability material it did genuinely challenge me to think creatively and the results surprised me. It also gave me a great excuse to abuse Excel and use its functional programming in ways it was never intended. Good stuff.
Update: I rewrote it in Python because Excel formulas aren't a real language. It has high scores of over 100k but most of the games end before hitting 100. Has a mean score of 700 with a standard deviation around 6k. Figure that out Gauss.
I've since actually tried to play the game with my human brain, and I seem to be doing consistently better with my more conservative approach at low flip counts. So maybe there's a way to factor that in to the AI. But it's hard to get a decent sample size. Even Python can run 10k games in 30 seconds. After 20 minutes I've hit 200 points on my first game, maybe I've just lucked out so far.
@@daniel.lupton Mean around 690 is what the optimal strategy gets when not looking at remaining coins, but you need a few million games for to have low enough variance.
@@ITR Yeah, I rewrote it in C to simulated more games. I've done a few million and get similar results. I haven't considered remaining coins properly yet. I modified the AI to play more conservatively at low coin counts but that always negatively impacted the mean. The only thing I haven't tried is limiting the recursion depth to remaining coins. That could significantly improve the performance when there's only a handful of coins left. If I play with it again, I'll try that.
So glad to see you back!
Thanks for making this game!
I played this game and I have a prediction: when that the blob is playing fair sometimes the outcomes come out balanced, but often enough there are more tails than heads. What if sometimes, when the game tells us the blob is fair, the coin is biased towards the tails, but Primer is not revealing it to teach us something about psychology once he shows us the data.
If I had to guess the video is about p values and so the blob being always biased towards heads when it cheats is so the statistics is simpler and you only need to do a one tailed test rather than a two tailed test.
The app became unplayably laggy when the number of blobs became too high, please fix this.
Always a good day when I see a new Primer video
this is awesome, i didn't think he would upload again to be honest
Thank you for making this amazing game. It made me do a lot of thinking and essentially write a bot (which I have never done before) to play the game and select "fair" or "cheater" based on certain outcomes.
It was a great journey. I started with simulating probabilities, and then understanding that they can actually be calculated mathematically. Then, I went for the highest accuracy, without realising that value-wise it would be a poor decision. And then after focusing on value, my best "formula" to success was to make a choice every time when the value of a current choice is higher than the mean of the possible next two choices (for example: if current situation is 0 heads & 1 tails, the value is -26, and the mean value of the two possible future situations, which are 1 heads & 1 tails and 0 heads & 2 tails is -27, which means that after having flipped tails only once, it's "best" to select "fair" immediately. I did not expect this when I first coded it).
I know little of Bayesian satistics and math overall, so I may either be writing nonsense or child's thoughts for some of you, but I enjoyed the journey, and I am grateful to Primer for making this inspiring me to answer the challenge :)
The highest score I got after 6 attempts is ~400
What a cool analogy, I look forward to the video!
Can already tell this will be great
Bro your videos are amazingggg I have like zero brain cells but I promise that watching these have doubled that 💛
just remembered this channel existed and remembered at a perfect time
OH MY GOSH HES BACK!!!
I almost crapped myself when I saw this in my notifications.
After 7 months you made a video FREAKING UPLOAD DUDE
I can't believe I am so late! I know you don't post often but I absolutely love your videos
Looking forward to the next vid after a long time
You're alive. Amazing.
Can't wait to see the new video :)! Really nice topic.
statistical hypothesis testing with the binomial distribution looks like it’ll come in great here !!!
my boi back with some fire vids
Good too see u back😀
It has been a loooong time :)
Nice to See you :)
Love the concept cool video Justin :)
I made a spreadsheet with probabilities to achieve each result (e.g. 3 heads, 2 tails) as a fair and cheating blob. Then I used these numbers to determine the ratio of fair to all blobs who would get to that result which is the same as the a priori probability that a blob plays fair. I then calculated the expected return if you guessed at this very moment. Then I checked that value against the expected return if you kept throwing the coin (back-propagating this value from the maximum throws evaluated) including the cost for more coin flips. Comparing these two values then tells me whether it's worth to continue to play or to guess right there. What to guess is then easy since I already know the probability for fair (and therefore also cheating) blob and just guess the higher one.
I only checked for up to 25 coin tosses with the same result and force a guess afterwards. That happens rarely enough that it shouldn't have a huge impact on the final performance, even if there are some fringe cases where contiuing to play is marginally better.
Because the probabilities to get to each state become a bit wonky with the strategy employed and I didn't 100% trust my own math there, I wrote a simulation of the game and had a simulated player play it with my strategy. It ended up with an average score of 659 after 200,000 games played. Notably, the very best of those 200,000 games had a score of 175,703 - so massive outliers are definitely possible.
CZcams doesn't like links, so I can't really link the spreadsheet, but here's an excerpt (only up to 10 of one result, the full thing goes to 25) of the strategy employed (order of throws doesn't matter, just sum of heads and tails seen):
Guess Fair at (H|T): 0|2, 1|3, 2|3, 3|4, 4|4, 5|5, 6|6, 7|6, 8|7, 9|7, 10|8
Guess Cheater at (H|T): 3|0, 5|1, 7|2, 8|3, 10|4
If you want to follow it, be sure to not skip any of those by flipping the coin too often but do keep flipping if you are between two guess values (e.g. you guess at 3|0 and 3|4, so you keep flipping at 3|1, 3|2, and 3|3).
Keep up the good work man love the videos
Wow, powerful and insightful!
thank you for doing this
First William Osman and now Primer. What a great day!
I love your videos so much! Your amaaaazing!
I love the blob accessories
Can't wait for the next video!
I love the little detail on how the coin can fall off the plate
Wow, I've always loved your channel, and only now I learned that you use Unity to create your videos! I would love to see a backstage video on how you do it!
You came back at the right time.
Can't wait to see your next video!
The legend came back again
Since you're coding in unity for most of these projects I would love to see you package some of the stuff you've made into a broader "game".
Your videos feel like the foundations for the type of game everyone wanted spore to be and it could be a great ancillary revenue source for you, maybe allow the download as a Patreon tier?
Big project for sure, but your videos and work is so high quality I'd love for you to get the money you deserve out of it.
finally these videos are too good
Yay! A new primer video!