LoBaron -> RE: Exercise in frustration - No obvert please (10/13/2012 11:37:29 AM)
This looks like a good sandbox, JocMeister.
Still, do not underestimate how many details you need to address before you can
safely assume that the results you see are related to the question you raised.
Taking your sandbox as an example we could raise the question whether units assigned
to the same HQ have an increased chance of coordination.
For this we need to reduce the impact of other factors adverse to or supporting coordination
to an absolute minimum. This is quite a lot of work, and usually more as is invested in such a test
- which again leads to false results.
Lets see what needs to be done (from the top of my head, so I can surely leave something out):
- Your sandbox with the option to either use a 90 AS baseforce or an HQa with the same AS
- A level 9+ AF to minimise any adverse impact from AF. This must be a constant.
- A target at normal range, I would chose something in the 4-5 hex range. This must be a constant.
- Two squadrons of the same plane model to eliminate coordination factors like plane type, service and cruise speed
(e.g. 2 B24D1 units)
- Standard altitude for the tests, e.g. 10000ft
- Normalized peader attributes (all values at 50)
- Normalized pilot attributes (all values at 50)
- Normalized unit Morale (99) and fatigue (0)
- Normalized HQ attributes (leaders, exp,...)
- No CAP over target (because otherwise seemingly uncoordinated attacks could be a result of the CAP, not the strike)
Then you need to accept that there are some random factors you cannot influence completely:
- Random skill/morale/exp increase/decreases
- Random plane crashes, maintenance...
These factors can influence your results at random, the only way to eliminate them as a factor is:
do as many testruns as possible.
This is not close to everything but you get the picture.
Now you need to decide what you want to observe. Based on the above question I would try something like this:
HQ present, all groups set to the present HQ
HQ present, all groups set to a different HQ
HQ present, groups set to variing different HQs
HQ absent, all groups set to the same HQ
HQ absent, groups set to variing different HQs
Then you need to formulate your expected result considering HQ assignement influences coordination.
(what difference you expect to see between the different tests)
And finally you need to be aware that the coordination factor you are researching is one of about 12-14
all influencing coordination, enanced by a couple of randoms.
So you need the delta between the different tests to be extremely small, Iīd expect it to be a lot smaller than
uncoverable by 10 consecutive testruns in each test. Probably about 40-50 repetitions for each test are required
to be able to draw conclusive results.
That stuff is often underestimated and done wrong (e.g. looking at results before previousely formulating expectations), not only here but all
around the world. Thats why so many crap statistics are produced every second and fed to the public, having no significance at all.