From: on a mountain in Idaho
Does anyone test the patches before they are released? Is that Steve's job as well?
We don't use the updater to get the patches, so we can't test that.
For the patches themselves, I usually try to test it through a whole turn. Problem however is, that MWIF has so many probable outcomes (especially in naval combat) that it is impossible to test everything, so we try to test those things Steve has fixed.
Unfortunately, there is always the possibility that fixing one thing might cause another thing to go wrong if a certains situation occurs. Not knowing what and when is the major problem here...
But slowly things are going to get better and better. There is real progress measurable, especially in solitair. But I agree that it is slow, very slow...
I don't envy you guys considering the complexity of the game and all the options involved. In the final months leading up to the release of WitE I was spending 12 to 14 hour days testing, testing, and more testing. This game is probably 10 times more complex than WitE so having a well developed test plan is pretty key.
Thing is, a complex or long game can never be 100% (or even close) to being tested. E.g. how many times did a beta tester play a whole game of WITE in beta phase? I would be surprised if ever. Problem with MWIF you have the game length AND the not-made-for-computer game complexity too. The testers will only ever be able to do so much I'm afraid...
I agree beta testers can only do so much, we went thru quite a few in the few years I was on that program, only a handful stuck it out from beginning to end. Burnout was pretty high.
We had different testers test different things. Leo did a lot of the long term game play by going auto with the AI and running the game thru completion on dozens of instances on various scenarios. Certain aspects are tested in this case like game balance, ai, production, game sequence, etc. We did have a few players get close to the end of the 41 campaign but all the smaller scenarios were played through.