Scott Hartsman, a noteworthy MMO producer and personal favorite of mine, has put together a rather wonderful article on his blog regarding some of the problems with MMO betas and launches in general. It is worth the read but I do have to warn that he is as long winded as I am. Unlike me, however, it is all directed and to the point! If you’re just looking for the high points though here is what stuck out at me.
I have never been a fan of public betas. I still live in the old days of secret beta invites to players who’re known to be good testers. Of course in reality that sometimes meant the wrong people got the invites. We generally did alright and, with a few exceptions, I like think companies benefited from having us around. Scott has a bit of a different take on that, however, and I can’t really fault the argument.
It’s not a 100% correlation – But in general, the bigger the beta, the more confidence. The later and smaller the beta, the less confidence, and the higher internal pressure (usually driven by the cost) to get something, anything out the door, as a hail mary.
In particular Scott is talking about how this practice looks to consumers and I think he is dead on. While I might not judge an MMO by the size of its beta I know many players that would. It is funny how that comes up in several walks of life.
That said, he also elaborates on why betas don’t really involve a whole lot of actual testing. It comes down to two major points that are well reasoned. The first is one that should be obvious: starting over, at least in MMOs these days, is not that big of a deal. It was previously believed that players would not stand losing their characters at the end of beta. In the “glory days” of EQ I can totally agree. Any progress was a lot of progress. These days, however, I just don’t see it. Max level characters are so much easier to achieve. It is a travesty I say (but that is tangent)!
WoW proved this to be a fallacy. For a good game, it turns out that there’s not just a higher tolerance for starting over than anyone imagined, but many people are actively interested in repeating their progress once a beta is over. Counterintuitive at first, but it does make a lot of sense. To a significant part of the core MMO audience…
Starting over feels like cheating.
He is exactly right. When I’ve played a beta I feel less lost on day one. I focus in on what I know worked for me before and I catch up my “hard earned gains” in a quarter of the time it took the first go round. This is one of the things that I enjoy most about beta. In a beta I test things consistently. I meander around and I don’t feel rushed. I recognize that that “rushed” feeling is self inflicted but even in retirement I’m still a “the game starts at max level” guy. Beta lets me stop and take the time I want to. It also means I am more forgiving of bugs and my reports about them are more detailed.
Tying into this, however, is one of the major industry issues that MMOs face: long development time that allows for little in the way of going backwards. Before designers ever get to actually create content they have to wait on the world and tool builders to finish. This leads to a long time of waiting and making plans on paper. Helmuth von Moltke the Elder is attributed with the quote “No battle plan survives contact with the enemy” and I think it is fitting for MMOs. Plans are made but the actual implementation of them comes so late in the game it is neigh impossible to make core changes and you might find out your product just isn’t fun.
The classic problem is compounded by the fact that designers are forced to continue progressively building more and more (on paper) upon unproven hypotheses (also on paper), until they end up with a 1,000+ page document of “Here’s the game we’re going to make once the architecture is in place.”
Scott talks about this in length and it is important to know if you’re big into the industry or just want to understand why something didn’t go as you anticipated. Steve Danuser talks about this in his article. Essentially, if a core pillar of your game isn’t right you’re left with limited options at that stage of the game. That is why, in my experience, big change can often come so slowly.
Ultimately I think some of the basic issues of work environment and work flow need to be addressed. Do you get productivity out of designers who have insane deadlines and are expected to work in a “60 hours is our corporate culture” environment? It is my opinion that you don’t and I think a lot of people would agree. The work flow is also somewhat skewed. I am, in no way, suggesting that world and tool building isn’t important but that isn’t what players see. Limiting the time for content design and basing that design on nothing more than “plans” won’t work that well.
Going forwards I imagine leaders in the industry are going to learn and adapt to these trends. Look at 38 Studios, for instance. Instead of starting a huge hype train they are being awfully tight lipped about their product. This is not actually a normal practice in the industry. I am also curious as to how their work flow is. We already know that they let Ryan lounge around and play ping pong all the time. I have to imagine it isn’t strictly following the previous mentioned model! With any industry happy, healthy, and less stressed employees produce better work. It is why my company has mandatory vacation. Perhaps the MMO industry should take a page from other technology firms. It is my guess that we’ll see better products and better products mean more money.
No 38 Studio employees were harmed or actually slandered in the writing of this article. This is mostly because Ryan is a lot bigger than me and I have a glass jaw.
All characters are © 2007 - 2010