We Ruined the BGG Rating System
Vložit
- čas přidán 7. 07. 2024
- Talking about the BGG rating system and how I believe it has become almost useless as people purposely tank games for reasons other than gameplay.
0:00 Intro
0:58 BGG Rating System
3:35 Gloomhaven vs Brass Birmingham
4:42 Feeling Slighted
6:38 Compensating for Overrated Games
7:58 Political
8:55 Your Thoughts and Opinions? - Hry
Another innocuous reason I've seen is people re-rating (and mostly downgrading) games after many years based on how often they still play it. Legacy/campaign games suffer the most from this, even Tom Vassel mentions doing this sort or re-rating and I've always been baffled. I trust my own rating when a game is fresh on my mind, bit of course fondness fades with memory and you aren't meant to replay legacy/campaign games. Also, just because I do not play Azul anymore (just an example), doesn't mean the quality of the gameplay has actually changed. It is just that I played it a LOT and no longer feel the same enthusiasm.
did they remove the rating descriptions, bc if they're still there changing your rating over time is valid.
This is the problem so often with data in general and therefore with drawing conclusions from it. "Garbage in, garbage out". Data can be intentionally corrupted as in the examples you gave, but it is frequently unintentional too. For example, no two people view a "7" rating as meaning the exact same thing. Thankfully in the case of BGG there is usually a large enough sample size of reviews, plus additional context available, information from other sources like CZcams, independent reviewers, etc. that a discerning person can educate themselves and make the best decisions for themselves. It is often good practice to ignore a percentage of the very top and very bottom ratings as fanatical or biased, and focus attention on reviews the "middle" in the 5-7 range. Those are most likely to provide realistic praise and respectful critique.
Good subject to discuss. One YT content creator mentioned that whenever she sees a Kickstarter game BGG rating, she automatically deducts one point. That’s because backers of crowdfunded games tend to have a positive bias toward that game. After all, many cost hundreds of dollars plus. She added that in the past she had been naively duped by such high ratings. If one can wait, a more accurate rating assessment of a game comes after a couple of thousand ratings. And it is good to read the 4-6 ratings comments, as well as the 7-9 ratings. It can prove helpful to learn about why the game might not be a buy candidate. Another example is when a new art style is in a reprint and not liked by the OP owners. At first “Village: Big Box” was a victim of a lot of 1 ratings battled by others giving it a 10. Now things seem to have settled down, enough people have rated the game and it is at 7.7 compared to the original which is at 7.5. I think that is a legit rating now, since the “Big Box” version includes two well liked expansions, plus a brand new one. So, good value for a solid Euro game having a long “ track record” and some unique aspects to it.
I don't think it's accurate that crowdfunded games have a positive bias. There are plenty of examples of expensive crowdfunded games that aren't rated highly. With expensive games (whether crowdfunded or not) the rating is a reflection of selection bias. With expensive purchases, there's more research done to ensure you're buying something that you'll like.
Are people really arguing the ratings for games like Brass: Birmingham, Gloomhaven, Eclipse, Nemesis, Root, Too Many Bones, Everdell, Blood Rage, etc. are of the game's price?. People that pay a lot for a product that doesn't meet their expectations tend to complain the loudest. I know that I'm extra critical of expensive games, because the game needs to justify the price tag. No one cares if a $5 game is simply OK.
There's a group of gamers that have a bias against crowdfunding, and they'll conjure reasons why people would be more biased about those purchases than a retail purchase. None of these people have any data backing up their statement. They simply want it to be true, and their echo chamber repeats it as if it were true.
As a general rule, I pay zero attention to BGG ratings. I find them less than useful and the way people will use them as a weapon ensures they will never be an accurate measure of anything.
Agree with your assessment. No matter, I only used it when first started gaming. Now I use it as a catalog when looking for a particular game. I do my own research to determine if this is the game I want. Great video.
I agree with you, but at this point I use it for my own library of ratings for games I have played and don't care about the ratings on games I have not played/do not own. If I am interested in a game I couldn't care less what other people have to say about it. I will pick it up, play it, rate it for myself then get rid of it or keep it. Ratings bombing has been around forever so I simply do not care about them anymore.
I agree completely. I never really use the rating to help decide whether to get a game or not. Sometimes, I might read/watch a couple of reviews to see if there is a pattern of issues or something, but crowd sourced numbers are useless. The rating from a reviewer that has a proven history of similar taste to me would be more valuable.
It’s always been a flawed system since it’s mainly hardcore gamers that rate them and it’s heavier games that move to the top. If casual gamers like myself rated every game then a lot more games would be higher that are more midweight I believe. Based on the wording for the ratings on BGG a 10 means you will always play it. I give a lot of my games 8-10 because I would play them always or if someone suggests them. Every group has its toxic fan base no matter what it is.
It's a hobbiest site with a bias for Euros and heavy games. The more you align with that group, the more valuable the average ratings for games will be.
The usability for the site isn't great. It's not built for casual gamers. Plus, it's a community that isn't particularly friendly to newcomers. They can be over-critical elitists.
I've always thought a board game site on par with Flickchart would likely provide more interesting rankings than BGG.
All great and valid points! Is there a better alternative out there though? If I'm researching a game, I don't depend on merely the number, but rather I'll read through those who rated it 8, 9, and 10... and then I'll read some of the 1's. I'm almost always able to dismiss the 1's because the comments aren't a true reflection of the game (for some of the reasons you listed, and others).
If people just took the time and looked at what the rating number means it would make the system work.
10: "outstanding, will always enjoy playing"
1: "awful" defies games description"
7: "good, usually willing to play "
etc etc.
The rating tries to illustrate the feeling of players but people treat it like they are scoring an exam.
I couldn’t agree more. This needs to be fixed which is not that easy as views on games are subjective and should remain so. One way to fix this would be if rating outliers get discounted in some form. Eg if everyone on BGG gets a “critic” rating between 0 and 1. So you get penalised if you make Rating outliers and the impact of your rating gets lowered. Comparing your rating with the statistical rating distribution should do the job, if you are x standard deviation from the average rating then your rating is classified as an outlier and your own critic ratings get penalised.
Those are good examples, but they are exceptions. I think most people do rate games as per the guidelines so ‘the wisdom of crowds’ does work for most situations. Like all data analysis, it is good to look for biases and sampling issues before using results. For example, I know that for most games with small ownership bases, esp KS, there is a bias upwards due the ‘endowment effect’. I use the rating as a starting point, and go deeper into the comments and numbers when it really matters.
The best approach, in general is to “try before you buy.” Thanks.
I agree entirely with everything you said. But there are even more reasons not to trust BGG ratings. You can take a good game and put it in front of someone who doesn't like that type of game and they're going to give it a low rating. I feel like this may be more common that people realize.
A while back several friends were talking about how great Guild of Merchant Explorers was. I played it with them without knowing anything about it.. There were some good ideas in there, but there was absolutely no player interaction. I might as well have been playing a solo game. So I gave it a 5. Of course there are people who don't want player interaction and for them this game may be great.
If I had taken time to learn more about the game before I played it, I probably would have tried to avoid it. Maybe I shouldn't have even rated it?
I am not huge into board games but you have to treat your backers well. They are supporting your games development. Give them the best and give it to them first. It is not a big deal.
I think they should acquire people to comment on the ratings they give. Most of the time I only read ratings that have comments, and I find it way more helpful than just looking at the average score.
Participation in the ratings isn't very high. Most games don't even break 10k ratings. The people that do attempt to manipulate the ratings with 1s and 10s tend to leave comments. So, you'd be decreasing the pool of honest ratings, resulting in average ratings that are shifted even more by these users.
If we assume that all the Gloomhaven 1's are fake ratings, then that's 720 out of 53,120 ratings. It's not a statistically relevant number of users. Why punish the people that are rating honestly when the bad actors have an insignificant impact on the overall rating?
Absolute Rating Systems are a thing of the past for me. I even entered into the Linux World when Distrowatch was no longer taken as seriously.
I hope someone finds out a way to have a good Preference-Based Rating System; where gamers rate games of a certain genre they like and rate it if it meets the satisfaction of fans that like that particular genre.
What we need is certified critic review like we might find in Rotten Tomatoes. I absolutely do not care what the average Joe thinks of something. Give me opinions based on vast experience, from people that are trying hard to avoid their biases.
The only way to fix this is to rate things as honestly as possible.
I mostly look at the complexity rating, and maybe make sure the rating isn't like super low.
Good video, but all ratings systems that are based on personal opinion are useless.
If you think about it, an aggregate rating system only needs 3 options. Yes, no, and neutral. That's it. From there you can work out a 1-10 score.
i’ve never rated anything below a 5, and only one game a 10 - though most people rate it at a 7 - but my 10 rating reflects how much i love playing this game i’ve owned since 2006! it’s not about a game being the most perfectly designed euro in the world, it’s about my personal enjoyment.
I think your points are all sound, except one: what's the evidence that these biases are getting worse over time? Haven't they always been intrinsic to the system?
I would say one clear thing you could look at is kickstarter and how now it is one of the major driving forces in board games. People will back a game and instantly go on the games page and rate it, sometimes years before they will ever have a chance to even play the game. Some games will get hundreds of ratings before it gets released.
I was wondering recently the same, mostly because of Mythic games. I wanted to see how good is 6 siege and all I have seen is 1s. What is meant towards the company not the game itself so that was actually bothering me. BGG should have ratings for games AND companies so ppl can go and complain there instead of badly polluting a games rating because of other reasons than how fun is a game. I understand, I would be pissed too, but would not rate the game down because of a company. I would rate the company down for sure, just like in google ratings.
This is definitely an interesting idea
I think another issue with the rating system is that the people who rate more games and are actively buying games, rate their games based on few plays, so it is very likely that they rate the game, while they don't really know the game. on the other hand, people who play their games to death, and have played enough to have a fair judgment are in the minority, and their opinions are added to the mix very late.
better that people who play many games rate than people playing one game forever.
@@trbry. It is not better or worse, that's just a logical consequence. There is nothing ideal in this world.
In 14 years on BGG, I've rated 591 games (not counting expansions, etc.). I've never rated anything a "1", only a single game got a score of "2" and twelve games received a "3". Obviously, people are free to treat the rating system however they like but these are the reasons I don't take BGG rating seriously. It's unfortunate because a metric that could potentially be useful is basically rendered meaningless by various agendas that have little-to-nothing to do with the game itself.
a frustrating trend for me is people rating games all in that very truncated 5-8 area you talk about. it's a 10-point scale, use it! It's to the point that borderline-unplayable games have ratings like 6.5 and great games have a 7.5 because people who never made it past grade school think that anything below a 7 is a "failure" and a 10 is "perfect"
People giving scores other than based on merit does happen, but IMHO I don't think it's a big problem, only a small one.
For example, if I go to Gloomhaven, there are 700 ratings of 1, and only 360 ratings of 2, which is odd since I expect scores to have a bell shape, with fewer 1's than 2's. BUT, though there are 700 ratings of 1, there are 25,000 of 10! Even if we expected 200 ratings of 1, that would leave 500 ratings left, which is only 2% of the total scores which would only have a very minor influence. My point is Gloomhaven at this point is still number 3, and regardless of whether is 1 or 3 or 7, the fact that you're dealing with a top game in the land means that any cutt-throat behaviour is swallowed by the sheer number of people voting. Who cares whether it's fighting for top space with one or two others, I really don't care if Bezos, Gates, or Musk have more money, you just know they're too rich for their own good. In this sense the rating system works, because a game not be 'exact' but every game is generally in the area you'd expect.
Another example, people complaining that Wingspan is overrated is neither here nor there. People can't judge what others like, and if the majority of people like Wingspan then complaining about it is like complaining that people shouldn't like drama tv because you don't like drama tv. That's not how things work.
So regardless of why people vote and all the little problems, in general it works because all these things even out and if you want a good game then you know you can look at the first 20 or 50 games and know that the majority of people also like these games. So the system is not 'ruined,' it works exactly the way it should do.
Personally I have given a 10 for Frostpunk and Spirit Island (one kickstarter, one not). I don't view those scores as exaggerated. Frostpunk has an X-factor which makes it a gaming experience I think everyone should have, and Spirit Island has a depth of strategy that more than compensates for any weaknesses. Being hard in itself is not a flaw otherwise every game that is hard would have a low score, and both games have extremely immersive and integrative aesthetics. And at the end of the day, they are 10's for me, even if they aren't 10's for others.
I'm not at all talking about people who honestly rate the games. If you think a game is a 1 or a 10, no big deal. Everyone has different opinions. But there are campaigns to try to deflate or inflate certain games and people will even comment, "its an ok game but rating a 1 cuz its overrated." I know certain games in the top 100 aren't going to get too affected by this. But let's say a game has less than 1000 ratings and something happens where 50 or 100 people rate it a 1 for something outside the actual game. That will drastically hurt that game and company.
Also, I don't think this is a big deal at all, just an interesting topic. I do find myself relying way less on the ratings of games these days and more so investigate into the comments of why people rated a game where.
What do you think the good game rating system would be?
I am asking because I was thinking a lot about the same, and in a way you are right that 1 is extreme bad, but if you give 1 only to the worst of worst in material quality and rules and all, then we have a useless point of the scale. There is no game nowadays that would get a 1, nor a 10 honestly. Then basically we are left with a 8-point scale instead of 10. Due to averaging and people having different styles and likes, a system like that would just mass produce a lot of 5.5 games.
I totally agree that rage voting and bought votes are bad, but how could we build a voting system that is informative? How a could a single (or few) number(s) tell that if you like Brass but hate Secret Hitler, then these are the games taht you would like and these you would hate. But if you are the minority who does not like Brass, then maybe you should like tehse because all Brass-haters loved this and this and that games. Because it is either that Brass is a world champion game, or it is just so that the Brass fans are overrepresented on BGG.
It’s a shame the system is full of spite and pumping ratings. I try to go by the rating system’s explanation of the number values (clicking the ? icon next to the box that describes roughly what each number should represent). I think that’s a fairly good structure when used properly, but that’s asking a lot from people. 😒
bgg is good to have a look at what is out there. but a rating does not make any sense in my oppinion because every boardgame is a matter of taste. if i dont like a specific kind of game that does not mean these kind of games are bad, its just not my kinda game but it can still be a solid 9. i only use bgg to stay informed what new games are out on the market and get a clue what kinda game it is.
Why would you rate a game you have never played?! This is BGGs rating in a nut shell.
Outside of the general bias from the users of the site the ratings are fine. If your game preferences align with BGG, then the ratings are likely even more useful. For every 1 or 10 rating to manipulate the ranks, there's 100 "true" ratings. The signal to noise is relatively low, and outside of people who take stock in the ranks the system works fine.
Does it really matter that Wingspan has a bunch of 1 ratings when it's ranked #35 of 26,222 ranked games? When you look at how big the "pool" is of board games on BGG, anything within the Top 1,000 is a very good game. It doesn't guarantee that it's a good game for you, but it's clearly popular and enjoyed by a large number of gamers to rank that high.
The same goes for IMDB. Bunch of trolls on there giving 1s to IP's they 'hate'.
The scores are weighted on BGG. Bombing 10s and 1s doesn’t have as much strength as giving it a 4-8. Also older scores and weighed lower as well. I don’t think you took into account HOW BGG actually tallies the scores.
I just hope everyone has fun, that's why I rate everything a five.
A game is really really good, you give it a 9 or 10, or like 9.1 or something. You really want to play a game/play it a lot, you give it a 9 or 10. I struggle to give a game I really enjoy to play 7 or 8, even if I like to a play another game more, or think another game is really a better game. BGG has mixed the two into one: How good the game is, and how much you want to play it.
My opinion is that you should be able to give a game many ratings, like 8: A) How much you like to play it, and B) How good the game is in the main categories: 1) Family 2) Strategy 3) Abstract 4) Thematic 5) Party 6) Children's 7) Customizable 8) War. You think a game is a superb children's game, but don't like to play it that much yourself? You should be able to show this!
It was never a good system, it has just gotten worse. Or the community has, since a lot of this is the fault of people moreso than the system.
For one thing, there is always the "cult of the new" which IMO badly skewers a lot of ratings.
Secondly, there are biases both toward certain types of games, and against other types of games by a significant portion of the community which, in large part, is representative of a snobbish attitude by some as to which games are "worthy" of their attention, and which are beneath their dignity. Much like the "established" arts community which dictates what's "in" and what is not, often not based on the merit of the work, but on practically anything else.
Also, I never base my decision as to whether or not I will purchase a game based on the word of a reviewer. A review may spark interest, but I only purchase games I've either already played myself or have watched others play.
Likewise, all of my ratings are based upon how much I want to play the game given the aforementioned criteria.