Home

 › 

Articles

 › 

Metacritic Can Get Game Developers Fired

Metacritic Can Get Game Developers Fired

I’ve long talked about how I am against review scores. I use scores in my reviews as it is Cheat Code Central’s policy and, hey, if I don’t include them I usually get complaints in the comments about not including them. People like to judge games by a number, what can I say?

But overall, this point of view has just been a personal one. I don’t find anything ethically wrong with review scores, so to speak. I just think that complex opinions can’t accurately be boiled down to a single numerical score.

However, as my time in the gaming industry has gone on, I have become increasingly aware that Metacritic is a lot more than just a handy site that averages all review scores together. I’ve been told that people have lost their jobs over a bad Metacritic score, and by “bad” I mean 8 or lower. Yes, if your game was rated any lower than the top 20 th percentile, then you can be out on the street. There is something fundamentally and horribly wrong with that, especially because it’s not hard to get below an 8 on Metacritic.

First of all, not every single review outlet uses the same metrics. Metacrtic always boils things down to a 1 out of 10 score, but if you aren’t using that scale, things can get screwed up pretty quick as Metacritic attempts to convert everything numerically.

Let’s take our site, for example. We use our entire 1-5 scale. When I give something a 4 out of 5, I mean that it’s a pretty good game, worth playing by all means. But when Metacritic scales that up to ten, it just multiplies everything by two, and suddenly I’ve given a game an 8 out of 10, which doesn’t feel as glowing as I intended.

The more wonky your review scale, the harder it becomes to convert. Famitsu scores on a 0 to 40 scale. A 30 out of 40 is still a good game, but when Metacritic scales it, the game has gotten a 7.5. Similarly, any website that uses 1 to 4 stars also counts as a 7.5 when they give a game 3 out of 4, which otherwise would be a glowing rating!

Then there’s problems with how averages work mathematically. It’s very easy to drag a review score down with one or two horrible scores (this is why you’ll frequently see the best and worst score dropped when scores are averaged in events like the Olympics.) But reviews are, of course, subjective, so a game isn’t naturally going to appeal to everyone who plays it. It takes only a couple outlying reviewers to ruin a game’s average, which can be devastating to development team.

Not to mention, the times in which Metacritic scores are “checked” tend to be a day or two after the game’s release. In other words they are based on scores that, for all intents and purposes, were done on a tight schedule with little to no in-depth gameplay or replay. Further reviews trickle in over time, but by then impressions are already made. Take Alien: Isolation , for example. If you were to believe its early Metacritic rating, you would think that it wasn’t that great of a game due to IGN’s mediocre review, but as time went on more and more outlets reviewed it well and opinion of the game turned more favorable.

And even if people should be fired for bad review scores, who should be fired? Should the character modeler in Sonic Boom be fired because the gameplay was so abhorrent? The character models looked just fine. They did their job to the best of their ability, but they stand to lose their job along with anyone else because the game was so bad. Game development teams aren’t one person, they are many doing different jobs, and if only one does there job poorly, you could see a flop, and then everyone is affected.

Metacritic Can Get Game Developers Fired

So yeah, I have to say that I now have an ethical problem with review scores. Frankly, the knowledge that low Metacritic scores can cost developers their job is devastating to reviewers. I know many reviewers that have felt pressure to push their scores toward a agreed upon “average” set by their outlet because they are afraid of stepping on any developer’s toes, and even more who artificially inflated their score because they didn’t want anyone to get fired. This makes all reviewers less likely to review truthfully, and is an amount of power that frankly we shouldn’t have.

Honestly, I’d love for the day to come when we no longer use scores in any reviews. But until then, at least some sort of review standardization is needed. I’d love to see us actually grade games the way students are graded on papers, everything from an A+ to an F. These grades are easily understandable and translatable to something we all went through. As are near perfect, Bs are good, Cs is passable but needs improvement, Ds are just short of failure, and Fs are failures all around. At the very least, that gives us a more abstract rating system that might make people get fired less due to Metacritic scores.

In the end, we game journalists are critics. It’s our job to critique a piece of work as independently from other factors as possible. We strive to review a game based on its own merits and flaws alone. But basing the future of a game developer’s employment off Metacritic scores puts us in a weird position. Even if we consciously tell ourselves “No, I will not let this effect my opinion” it will inevitably do so subconsciously.

To top