You Can’t Fake Quality, But That Never Stops Them from Trying


Several years ago, I worked in a development department of a major publisher. For its recent holiday season, it had, with a couple isolated exceptions, mostly released a cavalcade of licensed-IP, rushed and otherwise lackluster titles (sound familiar?). At a subsequent all-hands meeting of the corporate headquarters, the president of the company took the stage and thanked us for our tireless efforts. The company, he said, was doing better than ever before; we had beat earnings expectations, shown positive revenue growth, and pleased Wall Street once again. But not everything was rosy: we had proven time and again that we could make money, but our products were getting pummeled in the press – hurting the company’s brand in the process. This, he said, had to change.

The president outlined his vision: now that we had tackled the basic business of making and releasing games, our next big focus was to be quality. It wasn’t enough to just get the titles on store shelves and into people’s homes. We also had to ensure they were good games. This meant taking a hard look at titles in development and deciding whether or not to continue with them. It meant being careful when choosing partners with which to work. And above all, it meant raising the reputation of the publisher’s name, so that when people saw its logo on the box they knew they wouldn’t be getting something broken.

Being young, naïve, and with no sense of irony or sarcasm at the time (perhaps I exaggerate), I bought into this speech. I was impressed that he had candidly admitted that many of the games his company had shipped were pretty much terrible. He mentioned the importance of making choices and doing things in which one could believe. Finally, he’d pointed out that making high-quality games was good business besides. His words were resonating with me, and I walked out of the meeting hopeful that, if not immediately, in a few years I might be working at the kind of place that wasn’t just an industry player by dint of size but also respect and perhaps one day even cultural cachet.

A while later, I was having lunch with an acquaintance who was a veteran of the company and the industry at large. He had been with this particular publisher through its darkest and tiniest days, and had been in the game industry long enough to have worked on titles that were stored entirely in a couple meager kilobytes. The subject of our conversation turned to my current and his former employer, and I told him about the speech, saying I felt the president really seemed to understand things. He laughed in response and said, “Matthew, they do this ‘quality’ thing every couple years. It must have happened at least a half-dozen times while I was there. Every time, they say they really need to improve quality and start these big initiatives centered around raising the caliber of their titles. You’ll see.”

* * *

Quality is hard to measure and control even in cases where creative judgment and opinion don’t enter the equation, such as in car manufacturing or construction. But the subjective nature of entertainment, with all its opinions, whims, and fashions, complicates things even further. As William Goldman wrote famously of the film business: nobody knows anything. Just when someone thinks they’ve figured out the secret formula to consistently create great entertainment, the ingredients suddenly collide in a way previously unforeseen. Expectations are dashed, creators we believed in are found to be maddeningly inconsistent. Maybe audiences have just gotten bored and are waiting for something fresh and unexpected to take the day. The rules change all the time.

In order to try to raise the quality bar of its titles, the major publisher I worked for instituted, along with many of its peers in the industry, a system of bonuses for production employees based in part on average game review scores, as calculated by sites like GameRankings and Metacritic. This incentive, in theory, pushed employees to achieve the greatness in our games that the president had identified as our goal. But this meant equating a Metacritic score, which averages the numerical scores of dozens of reviews in diverse publications, to real intrinsic quality.

Armed with the knowledge that higher review scores meant more money for them, game producers were thus encouraged to identify the elements that reviewers seemed to most notice and most like – detailed graphics, scripted set piece battles, “robust” online multiplayer, “player choice,” and more, more of everything. Like a food company performing a taste test to find out that people basically like the saltiest, greasiest variation of anything and adjusting its product lineup accordingly, the big publishers struggled to stuff as much of those key elements as possible into every game they funded. Multiplayer modes were suddenly tacked on late in development. More missions and weapons were added to bulk up their offering – to be created by outsource partners. Level-based games suddenly turned into open-world games.

Before you cry in despair, keep in mind that all these people wanted in the end was the best game possible – or, more precisely, the best-reviewed game possible.

* * * 

The president who made the big speech about quality left the company a few years later (as did I). His successor might have realized, or known all along, that low-quality movie-license games that happen to be on store shelves on the day of the movie release often make significantly more money than great games based on older properties, because that’s what the company continued doing, to the benefit of its financial performance. Abetted by well-executed sequels to its own properties, it remains to this day a great business success story.

Microsoft, for its part, recently announced that low scores from Metacritic will be part of the criteria used to remove titles from availability on its Xbox Live Arcade download service. I can easily imagine the meeting that took place in Redmond as they rolled out a strategy to focus on quality.

16 Comments