Soviet Air Leader Head-to Head Test

[M60A3TTS]

I was curious to see how many more aircraft would sortie in a ground strike given the air rating of a certain Air Army Leader.

For this test, I consistently launched the same 523 bombers of 16th Air Army in July 1944 to see how many would be sent out. The manual says a higher rated air leader will insure more planes fly out for a strike.



Time to find out how much more.

For the test, I used Marshal of Aviation A. Golovanov, air rating 7 and General Leytenant of Aviation G. Tupikov, air rating 4.



A total of 20 missions were flown under each leader. The same target was chosen each time. Casualties weren't tracked, just what % flew under each air leader.

I will post results, but any guesses as to how many bombers of the 523 eligible hit the target under each leader?

[BrianG]

my guess 8% v 4%

[loki100]

isn't the other side that there is a cap on how many go ... in effect set by the computer's interpretation of how many are needed modified by your doctrine settings.

So if you hit the cap, it doesn't matter how good/bad your leaders are ... no more fly.

Maybe a bit like Hitman's test with morale gain on win/loss, all a good leader does is to ensure you get it right in marginal situations.

[HITMAN202]

Loki I've been looking at a ton of leader-influenced unit factors, MP's, Adjusted combat value, morale/experience gains, fatigue, SU committed (I have not done one on air ground support/attack) and clearly the better the leader the more impact/benefit, but the random "die roll" that sets the base value of whatever the leader influences has such a dramatic range of minimum/maximum values that any leader influence is minimal. I suspect the impact on Soviet leader on #'s of plane committed is negligible.

Granted each small influence that contributes to total attack/defense strength matters, but in reality each individual leader influence is minimal.

[HITMAN202]

In the above example comparing the air value Golovanov 7 and Tupikov 4, I think you probably have a 15-25 % average increase (it may take 100-200 test runs to get a statistical significant # ) but the real influence on the modified offensive combat value (for air ground support) or ground attack (degree of disruption that lessens modified defense value... at least that's how I think it works.. Morveal will affirm or inform.) is slight. How much ??? For the Soviets very little, 1-3%, because of the normal large unmodified offense value (so many damn units in the attack.)

Now the type of plane is important. For example the Ju88's kick butt for German air support/ground attack, but the overall impact, is still not huge (maybe 5 % or so.) This is a guess, but an educated one. (I'm not as uneducated or dull as Flaviusx assumed in another thread.)

[M60A3TTS]

Under Golovanov, about 40% of the bombers flew to the target. In over half the cases more than 200 went in.

Under Tupikov, it was 32.5%, and in only two cases did more than 200 fly. In raw numbers the difference averaged about 40 aircraft, a 1944 air regiment.

That would imply about a 2.6% increase in commit rate for every step up in air rating.

[chaos45]

I havent done the same level of statistics you gentlemen have on the game but I know/have seen better leaders impact the battles alot.

Leaders with above 7 morale/7 mech or infantry depending greatly influence ground combat- you often see it in doubling of total CV both on defense and attack. A crappy leader wont get those results ever normally and in fact often lowers your starting CV it seems. This is just from my observations of looking at my ground battle reports over the last 12+ months of game play vs Pelton- German leaders are a huge edge, I mean huge. The best Soviet leaders dont even come close to the best German leaders, plus the Germans have more of the best 7+ skill leaders.

[HITMAN202]

Actually M60 32 % to a 40% increase is 8/32 or 25%. I guessed 15-25 %. Not bad.

Chaos I think you are being deceived by the huge range of the base value, which is then influenced by the leader's rating. Morvael will affirm or inform

[loki100]

ORIGINAL: HITMAN202

Loki I've been looking at a ton of leader-influenced unit factors, MP's, Adjusted combat value, morale/experience gains, fatigue, SU committed (I have not done one on air ground support/attack) and clearly the better the leader the more impact/benefit, but the random "die roll" that sets the base value of whatever the leader influences has such a dramatic range of minimum/maximum values that any leader influence is minimal. I suspect the impact on Soviet leader on #'s of plane committed is negligible.

Granted each small influence that contributes to total attack/defense strength matters, but in reality each individual leader influence is minimal.

Hi

as I've said I really admire the sort of tests you've been setting up but I think that you are making a basic error in your comparisons. Hard to explain without getting into statistical theory but essentially the approach you are using depends on both data sets ideally having a normal distribution, at worst having the same skewed distribution.

problem with this game (and many real life social stats) is that there is an upper limit that can't be breached. So if your two data series have different mean values (to be expected) and one is closer to the upper limit value, then you are very unlikely to be comparing similar distributions. That may explain why you are often finding marginal differences as there is very little variance range available. I can explain this better in terms of formal statistical reasoning but I don't think this is really the place for a page or so of quasi mathematical proofs [;)]

[M60A3TTS]

I'm not a statistician, nor am I into quantum theory, or nuclear physics. I just want to know how much better results I get with an air 4 guy vs air 7 guy. End of story.

[loki100]

ORIGINAL: M60A3TTS

I'm not a statistician, nor am I into quantum theory, or nuclear physics. I just want to know how much better results I get with an air 4 guy vs air 7 guy. End of story.

aye I realise that, and as with Hitman I really appreciate the efforts you put in (as I am far too lazy to do my own). But the problem is you are 'doing statistics' and if you make errors in interpretation then you'll draw the wrong lessons.

In effect, I think you've shown that the results scale with ability, which is good and what I've sort of assumed from the results of dumping some of the total muppets in favour of the ok cadre of air commanders.

But what you haven't shown (and this is a common and easy error) if (a) the gain is statistically valid (I think it is) or (b) really what you are claiming. The latter is important with the Soviets with so many mediocre commanders.

In this case, if regardless of commander, you can't fly more than x planes (computer calculation of need modified by your air doctrine), then 'not quite a muppet' may be every bit as useful as someone who you would indeed 'send out to get the messages'. Especially late game when the Soviets have a lot of planes available.

[M60A3TTS]

Actually part of the limitation of ground attack is there's a finite # of regiments you can commit in total, 22 I believe, using the manual selection method. The air doctrine is only for AI controlled activities. If I reduce the % from 300 to 100, I get the same commit results.

[HITMAN202]

Loki I fully agree with your comments in that we really don't know how the computer "works" the values, but we still can run a specific combat a large # of times and look at the trend. A problem with M60's test above is that he is replaying a saved turn that has preset "die rolls", so to speak, that are replayed since it is a saved turn. Morveal commented on this in an earlier thread. His point is that by replaying a turn, you are reusing the same die roll set for that particular saved turn. Thus the numbers aren't truly random, even if you change the order of resolving each combat.



What I do is only run trials by restarting the first turn, thus getting random numbers.



I am no statistician either, and agree that the base values are skewed (no perfect bell shaped curve) but if you isolate a infantry division and run a large number of attacks with and without a leader you can compare adjusted combat values and come to some conclusion. That's what I have done.

[HITMAN202]

Statisticians are strange birds (well, my lineage may be a strange avian species) but I admit I waste a number of hours playing around with stuff and reading AAR's (loki, yours are A-1). But on the other hand I'm not spending dozens of hours on a server nervously biting my nails.

[M60A3TTS]

ORIGINAL: HITMAN202

Loki I fully agree with your comments in that we really don't know how the computer "works" the values, but we still can run a specific combat a large # of times and look at the trend. A problem with M60's test above is that he is replaying a saved turn that has preset "die rolls", so to speak, that are replayed since it is a saved turn. Morveal commented on this in an earlier thread. His point is that by replaying a turn, you are reusing the same die roll set for that particular saved turn. Thus the numbers aren't truly random, even if you change the order of resolving each combat.



What I do is only run trials by restarting the first turn, thus getting random numbers.



I am no statistician either, and agree that the base values are skewed (no perfect bell shaped curve) but if you isolate a infantry division and run a large number of attacks with and without a leader you can compare adjusted combat values and come to some conclusion. That's what I have done.

So if I do the same thing over 20 consecutive turns, you are saying my results will be different?

[HITMAN202]

Sorry for not answering sooner, my job can be crazy and lack of sleep, etc.

Two points.

1) First your trial may be "underpowered." It's a statistical term to say you didn't run enough tests to see a difference, or in this case, the real difference in the benefits of different strength air leaders. Yes to your question to my first point. Likely you would see a major difference if you would run 20 trials with 20 tests each. I feel that 100 test run would be better. There is a method to determine how many tests within a trial is needed to determine if there is a statistically significant difference but I'm now over my head, so to speak, in explaining something I know little about. But lets say I have a new drug that I claim will increase the survival of lung cancer patients by 10 %. Statistically there is a formula to determine how many patients would be needed in a trial to see if my claim is true or not. So many medical studies are bs because they do not meet statistical significance.. ie. they are 'underpowered."

2) Second point. If I understand morvael correctly in an earlier thread, for each turn the computer generates a "set" of die rolls that are used for subsequent combats, leader support, etc. By saving and replaying a turn, you are using that same "set" of die rolls, which are not random. When morvael explained this (again if understand him correctly) I quickly argued (to myself) that since you can randomly select the order of combat (to use a simple example) you in effect can run a unique trial when reloading a saved turn. But in reality (as I reasoned later) you are using a nonrandom, predetermined set of die rolls that can (that's the killer word) prejudice your trial.

So M60, my friend, to your question in #1's case your are underpowered, in #2's case you are not using random numbers and possibly coming to wrong conclusions based on a statistically flawed trail.

BTW M60, you will be also near the top of my Soviet butt kicking list when I come out of retirement.

[M60A3TTS]

As they say, "Be careful what you wish for. You may just get it."

[HITMAN202]

My butt kicked ??? Very possibly but the fight will be fun as you are one of the top Soviet players.

BTW, did a make sense and answer your question ???

[M60A3TTS]

It's commonly understood the greater the sample size, the more accurate the results. By the same token whether I ask 100 or 1000 people in an affluent area of Dallas and then a middle class suburb in NYC who they will vote for in the next Presidential election, I'm not sure that will predict anything. What surprised me from the test was the best Soviet air leader can't get even half his bomber force to the target. And I doubt 100 or even 1000 repetitions will prove otherwise. It may move the needle 2-3 percentage points but I bet little to no more.

If you ask people who have been around this game a while what they think of the air war, the common answer is, "it's broken". But I think few can explain how it's broken. Part of improving individual gameplay is to minimize the inherent broken elements of the game and enhance those that are not broken, or at least not broken as much as other elements. So my takeaway from this particular test is, when it comes to this facet of the air leader value, specifically on the secondary fronts, any air leader will do. Replacing a 4 air guy with a 6 air guy on the Volkhov Front will just never be in my playbook.