How to Tell a Hard Landing, with Mass Effect
Anyone who’s worked in the game industry for some time has been on a project that came in under the wire– so much so, in fact, that this is unfortunately many people’s exclusive experience with shipping games. Aside from the obvious negative effect this has on our quality of life, and our desirability as an industry for the world’s best creative talent, the last-minute cuts and the seemingly heroic hacks show through in the shipping product as well. Here are some elements to look for in determining how a game’s production went, from smooth sailing (mythical as that may be) to the worst bare-knuckle stress and drama.
Before I begin, I’d like to note that Mass Effect is a great game that I thoroughly enjoyed. I use it as an example here because it’s fresh in my mind, and because I get the feeling that it was meant to be more– maybe much more– than what ended up on the gold disc. I don’t know anyone personally who works at BioWare Edmonton, so everything you’ll read here is just a guess on my part. I mean them no ill– Mass Effect is all the more a triumph, and the team all the more laudable, for overcoming the adversities I’m certain the project faced.
Undue or odd simplicity.
Simplicity isn’t a bad goal. But there’s a difference between elegantly simple and oddly, strangely simple. When a game’s design has elements that vary wildly in their level of detail, odds are good that it wasn’t meant to be that way: game designers, like other artists, like to see balance in their creations.
For example, the items of Mass Effect are limited to weapons, armor, upgrades, and medkits. No other items can be obtained or inspected (quest items just show up in your quest log). There is no loot other than finding more weapons, and these automatically get better (in a completely linear progression: Assault Rifle I, Assault Rifle II, and so on) as you advance. Because they’re the only thing you can find, you’ll soon have piles of pistols and shotguns, which you can sell for cash to buy– what? The only thing you can buy is more weapons.
As one of my colleagues at work put it, the lack of any other kind of item than these in the game really detracts from the realism of the world. Imagine driving through a desolate ice field on a distant planet, picking up some debris on your scan, making your way to it and finding an old crashed probe, and finally, opening it up to find… a sniper rifle, of all things. It just doesn’t work that well. I feel safe proposing that BioWare, one of the leaders of the RPG genre, did not originally intend it to be this way.
When you find a very simple system like this– one in a game that in other areas pulls out all the stops to create a richly detailed world– there’s a good chance it was implemented under great duress, as deadlines drew near and the parties involved realized what they originally wanted wouldn’t happen. Either there was just no time left, or the original design wasn’t working and nobody realized it until too late. At the risk of sounding dramatic, these moments are when dreams die. You can sometimes spot their remains in what ends up shipping.
cf. any of the actual open-world gameplay in the big publisher “open-world” superhero games like Spider-Man 3 or Superman Returns; the canned, timed button-press moments in Call of Duty 3; and Final Fantasy XII’s License Board (in an admission of its weakness, the License Board was later replaced with a more fleshed-out character advancement system in a Japanese re-release).
Significant differences between demos and the actual game.
Good game developers don’t set out to lie about their games, and so when they create a demonstration to show the public one year out from shipping, we should assume they are making their best-effort guess as to what the game is really going to be like when it’s done. Obviously, things don’t always go as planned. When the final game diverges significantly from what was shown in the demonstration, you can bet the project experienced sharp course corrections.
About a year ago, a promotional movie for Mass Effect showcased numerous features that were nowhere to be seen when the actual game shipped. The Galaxy Map featured beautifully smooth zooming between levels and planets in dramatic three-dimensional orbits. The combat engine allowed the player to direct the squad to specific individual locations with an interface that paused the on-screen action, and the player could even take control of each member at will. Certain conversation lines seemed interruptible by the player.
Unfortunately, none of these features can be seen in the final game. In BioWare’s defense, a hugely important part of the creative process is taking out things that don’t work. But, making these decisions with one year to go in the production schedule is already cutting it close, let alone when they may have actually been made, sometime between the demo and the release to manufacturing. And these decisions are costly, especially on the well-being of the team. Because these big games are comprised of complex, interrelated systems, anytime a mid-course correction is made in one feature, many more features and plans need to be evaluated in the light of the modification. Flexibility is a good thing, but too many changes, too late, brings exhaustion as the developers try to work out the cascading effect of change after change on the game’s overall design and experience.
In a worst-case scenario, the project is changing so much that not everyone is fully up to speed on what the game actually is anymore; people make guesses, work gets wasted or thrown out, and tempers flare. As I stated above, I don’t know that this was the case in particular during the development of Mass Effect. But I do know that the easiest, most enjoyable projects are the ones where clear definition is achieved early enough that everyone can plan accordingly. The larger the team, the more critically important this becomes.
cf. This is similar to Halo 2’s demonstration at E3 2003, which featured an entire level that never made it into the final game. Half Life 2’s press materials for the same E3 featured some elements, such as the strange blue tentacle, that disappeared before release.
Sudden narrative leaps and loose ends.
The player reaches one point during the main story of Mass Effect where it seems as though a major decision is about be made. It’s presented as though the choice will have far-reaching consequences on the fate of the galaxy. But after it is made, the effects of this decision are never felt; in fact, the event is never referred to again after that particular mission is over. There are discussions the player can have with certain characters that seem to be leading somewhere, but the dialogue tree suddenly runs out of options, and conversation comes to an abrupt halt. And instead of the player finally figuring who the real villain is and what it wants, the villain simply shows up about two-thirds of the way through and explains itself directly to you, for no benefit to itself and seemingly no reason.
I’ve seen my fair share of emergency story patchwork up close, and each of these moments telegraph that aura. Stories in video games face adversity from many different directions. Most designers and players don’t like overly long cutscenes, and cinematics are often the first to get cut in any case. Scripts need to be adjusted to accommodate what’s doable in-engine and what the artists have time to create and animate. Worst of all, if a mission or sequence designed to reveal something important is cut at the last second, the same plot development needs to be squeezed into a place it wasn’t originally intended to be: a quick conversation, a disembodied voice over the radio, a wall of text.
Carrying the knowledge of how much better something could have been, or how the story of one’s game does indeed make more sense than players think it does, can be a burden at times. But that, ironically, is one of the saving graces of sequels (as they will inevitably come for Mass Effect). A sequel offers developers the chance to make Game 2 what Game 1 was supposed to be, and in do things in Game 3 that they talked about doing in Game 2. Let’s not pretend that isn’t nice sometimes– a sequel can be the perfect venue to refine a specific type of craft.
cf. Indigo Prophecy’s confusing jumps two-thirds of the way through the story; Knights of the Old Republic II: The Sith Lords’ hurried, abrupt conclusion; and Xenogears’ baffling second disc, where the game gives up on towns or cutscenes and simply teleports the player from dungeon to dungeon– explaining what happened in the meantime with scrolling text.
Loading and streaming issues.
Loading, streaming and caching issues all come from a problem set that has ensnared a sizable number of projects in this console generation. Because art assets are more detailed than before, files are larger in size, and thus there is more to load; compounded with the rise and popularity of open-world gameplay, ostensibly any of those assets needs to be able to be loaded at any time. Finally, to add insult to this injury, the Xbox 360 is a step backwards from the original Xbox in this respect, which featured a guaranteed hard disc that could be used for caching frequently-used data. Xbox 360 developers need to account for players who have no hard disc, and must stream directly off the disc as much as possible.
As BioWare’s first game on the Xbox 360 platform, it’s reasonable to expect some technological teething problems. One of the first things noticeable in Mass Effect is the tremendous amount of texture-popping that goes on as the player enters new areas. Occasionally, even just running from place to place results in a loading message appearing while the disc spins wildly inside. It’s clear that the popping wasn’t something the artists knew in advance to work around, since there are often shots in cutscenes that don’t even last as long as it takes for the texture to pop in: by the time the texture has actually loaded, the camera isn’t even on the object any longer.
Good systems can be designed to minimize load times and texture popping. However, developers frequently underestimate the amount of work this takes. Ideally, a plan to address the issue would be in place from the very beginning, allowing the artists to construct objects and environments that work in lock-step with the technology. Unfortunately, too often this doesn’t happen, for any number of reasons– perhaps the technology just wasn’t in place at all at the time, or the artists and engineers didn’t communicate well, or there was just gross negligence on the part of everyone involved.
Past a certain point (much earlier than most people realize), it’s too late to go back and change your environments, because the designers have already designed everything around the spaces they have. So the engineers are left holding the bag attempting to optimize loading and streaming for these assets after the fact, when they may not have been built for good streaming in the first place.
cf. Many Unreal-engine based titles, not just Mass Effect, show noticeable texture popping (BioShock et. al.), I don’t know if this is due to the way the engine works or the way certain developers have made use of it. Halo 2’s infamous cinematic texture pops are a telltale sign of last-minute content creation.
Lots of bugs.
I saved the most obvious one for last– overly buggy games are a clear sign of a project slamming into its release date at a higher velocity than what an ideal situation would dictate. Reviewers sometimes take it upon themselves to berate the QA (Quality Assurance), or test, department for the bugginess, but these people should be given a break: just because a game is buggy doesn’t automatically mean it is the fault of the testers. When a consumer encounters a bug in a game he or she has bought off the store shelf, the chances are fairly good that a description of that exact problem lurks somewhere inside a database on a developer or publisher server– found, detailed, and reported by one of the game industry’s testing legion.
Fixing bugs as the release date creeps closer is a delicate balancing act. Sometimes the bug is just found too late, because the system that is being tested came in late itself (see above). Other times, to address a bug would mean changing how something important works, which could cause more problems than it solves. It’s difficult to predict all the ways one change could affect everything else in the game, and so sometimes the best thing to do, in the case of something minor, is to let it go.
Of course, bugs in a game can indeed be the fault of the test department, too. It’s possible the issue just wasn’t found by them. Or, if the person in charge of test on a title is a pushover, his objections to release on the grounds of big bugs lurking in the code may be easily overruled or ignored by a producer. Most of the time, the financial considerations of missing a ship date will trump all but the biggest quality issues.
Mass Effect, being the huge game that it is, presents a challenge from a testing perspective, so it’s no wonder that bugs are rather easy to find. Playing normally, I got stuck on geometry several times, with no way out but to reload an earlier save. I flipped my supposedly un-flippable Mako vehicle. A step that I had not yet taken was registered as completed in my quest log. None of these bugs severely impacted my ability to enjoy the game, but they do affect the “fit and finish” of the final product. I have never worked on a game where we didn’t dearly wish we had just a few more weeks to address some of our biggest pet peeve issues.
cf. Both The Elder Scrolls III and IV, Morrowind and Oblivion, for all kinds of miscellaneous problems that were discovered, detailed, and exploited by the fan community; BioWare’s own earlier Knights of the Old Republic for similar issues; and, to be fair, just about any big, ambitious game.
Update 11/28/07: Though I felt I went to some length to state that I really enjoyed Mass Effect and think of it as a great game, it seems some people feel the main idea of this piece is “Mass Effect sucks”. It isn’t; my point is more that the Mass Effect team probably went through a very difficult period finishing and shipping this game. So, if you feel motivated to defend the game in response to this article, I recommend using that energy instead to thank the people at BioWare (if they’re back from their vacations) for the hardships they endured for the sake of the game. They’ll appreciate it.
Commentary | November 24, 2007
Reader Comments (39)
No offense dude but that entire survey you conducted wasn't very good.
I know 10 people (including me) that don't have any errors you seems to have.
That must mean 100% of Mass Effect Users don't have the problem!
See what was wrong with that survey? Now apply it to yours and it's the same thing.
As a game programmer I'd like to make a small comment on the bugs section:I find the biggest problem with removing bugs from software is related to the publisher or whoever is overseeing the schedule of development. Most programmers I know are aware that the earlier a bug is fixed the easier it's going to be to fix. I've worked on projects where we've constantly been fixing bugs as we introduce them and everything has gone well. Unfortunately it takes only a single big, unmovable, milestone (E3 anyone) to add so many bugs that'd you miss the next milestone fixing them all. This is only compounded closer to release when there is more pressure to complete the product AND less time to fix the remaining bugs.
It basically comes down to the fact that publishers often want every feature in a build without giving extra time. This is often impractical and the games industry is far worse than other industries I've seen in this regard.
Though i can ignore popin, loading and gameplay issues.
But i was expecting Planets with good sdequests.Longer substantial mainstoryline.Only 5 main planets, and game got interested in virmire,ilos and then you get final level.
No tough choices like wrex one, only 2 to 3 choices.
Choices are not like desuex,rawdanger, wayofsamurai etc that can change entire course.
All in all much of the exploration stuff was SUPERFICIAL.
Though Graphics atleast outstanding.
There was so much stuff that didn't feel quite right, when other bits were just amazing, and although it didn't impact on the game at all, still left a sense of incompleteness to it.
Mass Effect included a few things I didn't expect, but dishearteningly, jettisoned - as you point out, probably for a variety of reasons - a lot I did.
That being said, I still absolutely loved the game. I disagree about the gripes some people have with the story; I think this was some of the best storyTELLING I've experienced since Legacy of Kain. If only Amy Henning could finish the legacy.
Spoilers:(Are you referring to the Krogan/Wrex decision on Virmire? the Rachni Queen? the decision to sacrifice one of your squad?)
With the possible exception of the Krogan cure for the genophage, I didn't think any of the other events were abruptly cut short or left un-integrated.
We all have deadlines to meet, be that as a game programmer, a web developer or a graphic designer - that's the crazy pace we all live our lives at these days. At least in today's modern 'next-gen' gaming developers can issue a patch to fix things up, no excuse for being lazy mind you.
I wonder if Microsoft or Bioware is more to blame?
Had they waiting until the game was "complete" (let's say come end of 1st quarter or beginning of 2nd) it would probably be a much better and more complete game, but their sales may also have been much less.
If only Grand Theft Auto 4 had shipped this holiday we may have seen the complete Mass Effect in early 2008.
The Money factor always comes before quality of the product. Companies want to put their product on the shelves, no matter how flawed it is, so they can begin reaping the financial rewards.They sure don't make'm like they used to.
I agree with everything you said...the major headache that is not visible/apparent to gamers at all is that making a game is very expensive, and takes a lot of time.
Honestly, publishers are really only concerned with one thing, the bottom line. They are very wary about making decisions that affect that bottom line, mainly allowing more time or pushing out the release date (brutally increasing the cost of development). Another effect of delaying the release on a project is that it affects the timeliness of the project, especially when you have a wildcat marketing department who has been over-hyping your game for years.
The thing that publishers seemingly are not as concerned about (but is arguably the most important thing) is the actual quality of the game. It takes copious amounts of iteration to make a game "feel" right, and publishers have no concept of this (it's a young industry, but come on, have we learned nothing yet from 30+ years of making games?). They don't realize that that also affects the bottom line (positively), although not immediately, because it can be the difference of 1 to several points off on review scores, which, whether we like it or not, have been shown in most cases to affect the sales of a game drastically. Especially, again, when the game has been over-hyped for years, and gamers' expectations are unrealistically high as a result.
I also agree about the dreaded E3 demo...IMO E3 is/was the biggest waste for developers, who work extra long and hard to get everything into the demo that they want people to see, sacrificing stability, organization, and general quality of code to make it happen. Unless a game that is near completion is being showcased, most of the extensive work done to get the E3 demo out the door and onto the floor ends up being throwaway work that either needs to be re-done or gets scrapped altogether...very hard on the developer's quality of life, as well as morale when you are pumping everything you've got into a ton of work that ultimately ends up being a waste of time.
More and more I believe that publishers should not show *anything* until, well, almost everything is finished. This would prevent gamers from feeling like they are getting gypped when they see awesome things in the demo that aren't delivered on, and the developers would get to stop wasting time to showcase a bunch of things that will probably never make into the final game anyway.
Whew, didn't intend for that to be so long winded. At any rate, Mass Effect is one of the best titles to come out since, well, probably KOTOR, and I enjoyed playing it intensely, in spite of the the myriad technical issues present in the final release of the game. I really hope EA takes a good hard look at what people are saying and give BioWare the adequate time and resources to really nail it the next time around, as they (and the game) will certainly be all the better for it.
"it's a young industry, but come on, have we learned nothing yet from 30+ years of making games?."
I think you hit the nail on the head here. It keeps coming back to the fact that software engineering is still (relative to other engineering disiplines) in it's infancy, and game development is even younger.
I left out my E3 anecdote as my post was getting long, but I thought I'd share it now:
I worked on a game a few years back and we had to push a build out for E3. The lead and I coded for something like 30 hours straight to get the features in for the build. They all made it in, however we spent THREE entire weeks fixing the bugs we added. I remember looking at a piece of code I had written and couldn't for the life of me remember writing it.
I think for those who don't work in the game industry, might have a harder time realizing how much pain we go through to bring out a finished product in a timely fashion.
In some of our games, we obviously released some that did have minor bugs since we had to get product out at a set date. Some of our customers did notice them but were not bothered by it much.
There were times when we had major bugs in the game(s) that would require us to make major revisions and thus I had to alter some of my original art assets for that particular game (frustrating to say the least).
I remember working long hours (one time i worked 36 hours straight) just to hit our deadline. I really sympathize with people who work in our industry and I completely understand why certain bugs are left in the game and in fact I'm more forgiving on minor bugs such as the ones you mentioned.
Great work on a great game!
Is/was the pure distillation of what Bioware wanted to create, and by proxy what we as gamers came to expect from the project, actually a generation ahead of itself?