Pete's back and discusses a very interesting topic... Reviews, they aren't always THAT reliable...
There’s a particular concept about reviews that I really hate, one that seems most prevalent in game reviews: number scores. Oh, how I hate number scores. IGN.com among is the worst offenders. Now, don’t take this the wrong way. IGN (whatever that stands for, I’m too lazy to look it up) has lots of great coverage. I especially applaud their Nintendo team’s effort to push lesser known third party Wii games.
I first noticed strange incongruities with their review system when they did their Red Steel review. Now I’m not apologizing for the shit storm that is Red Steel. But when I read Matt Cassamisina’s write-up of the game, his words really read like the game deserves a 4 out of 10, but even he seems to be aware of the review problem because that isn’t the score he gave it. He gave it a 6. Seriously, read his review. He more or less crushes the game, but then gives a score that I think most people can agree is above average.
Cassamasina even cites IGN’s review policy saying that a rating of 6 means the game has “more blemishes than strengths.” So… I guess with IGN a 5 isn’t considered average? But wait. On their review info page they say that, “A 5.0 game is the epitome of ‘middle of the road.’” I don’t know about you, but to me “middle of the road” means an equal amount of faults and strengths, which seems pretty average.
So why is that a score of 6.0 has “more blemishes than strengths?” Shouldn’t that be the description of a 4.0? Because “more faults than strengths” (IGN actually uses the word “faults” on their review info page) suggests below average, and as I said before, the Red Steel review reads like it’s a below average game. That is, until, Cassamasina says that Red Steel actually has “a great deal more strengths than it does weaknesses, but its one weakness is a whopper.” That doesn’t make sense. You give a game a 6 saying that a 6 has more faults, but then go on to say that the game in question has more strengths, but apparently its biggest flaw is so big that the game deserves a score that actually means it has more faults? What the fuck?
Another thing I hate about number scores is decimals. Check out Mark Bozon’s review of Dragon Quest V. It’s a total rave, for the most part, but he scores the game at 8.9. Now that’s something I don’t quite understand. If the game is great, why not give it a straight 9.0? It says right there that the overall score is not an average.
Quite in fact, if you added the five scores and divided by five (meaning, if you figured out the mean average of the game, and yes, I know that I have way too much free time on my hands) you’d come up with 8.7. So why give the game an 8.9? What is the criteria that constitutes one tenth of a point? How can a game be one tenth of a point better or worse than another game? Now, if you said that each of the five scores represents two points apiece and the overall score was the sum of those scores, that would make sense. But this doesn’t.
It doesn’t stop with IGN. Take 1Up and the now-defunct EGM (God rest its soul). Their justification in switching from a number score system to a letter grade system was mainly because people apparently couldn’t get over the fact that 5 is average. I guess their readers thought that 7 is average, because in school, 70 percent is considered average. So they started grading games like a teacher grades a paper. But if you now say that the average score is a C, which is a 70 percent, then aren’t you basically admitting that a 7 is average? Doesn’t that defeat the purpose of what you set out to do in the first place? Especially since 1Up does not actually rate their games on a percentage scale, it’s just the letter grade. But then 7 and 70 are two completely different numbers. Metacritic may not think so, but they’re not fooling me. They seem to think that a score of a B+ equals… 83? How do they figure that? I would think that a score of A+ equals 100. So A is 95 and A- is 90. That would mean that a B+ equals 89, not 83.
So why 83? What’s the reasoning behind that (if there is any)? If B+ equals 83 then what does B and B- equal? 82 and 81?
Metacritic’s system seems very random and disorganized, like there isn’t a universal system by which the scores of other sites and publications are translated into Metacritic’s system of 0 to 100 (and if they had a universal system, that would also mean they would have to use the exact same sites and publications in their listings every single time to remain consistent). Need another example? Look at the way they translate GamePro’s scores. GamePro rates on a scale of 1 to 5 stars, 1 is bad, 5 is good, and I’m assuming 3 is average. But Metatcritic translates a GamePro score of 3 into a Metacritic score of 60.
Sure, 3 twenty dollar bills is sixty dollars (3 equals 60, in that case), but if 3 is average, shouldn’t it translate into a 50 on Metacritic? Unless GamePro counts 0 stars as a score, then technically they’d be rating on a scale of 1-6, or 0-5, or… something. Whatever.
The basic summary of all this is that websites and magazines and critics in general really ought to get rid of the number score system altogether. I understand if the scores made it easier in the past for the idiots who don’t want to read the whole review, but now thanks to YouTube and others, video reviews now eliminate that problem. I don’t even like ScrewAttack’s system of “Buy it, Rent it, F it” because, on the disjointed mess that is Metacritic, they would still turn that into 100, 50, 0, or, because of their overall lack of logic, maybe it would be 100, 57 and 3. But anyway, there’s my rant about number scores. Thanks for giving me some ears to yell in.
Read more...
