Results 2 - Air Unit Training

[mussey]

OK, shoveled the snow, fed the birds, and ran more definitive Air Unit training tests.

1) Test Question: "Do air units with higher pilot to a/c ratio train as fast as a lower ratio units?"
I tested 8 fighter units, at San Fransisco, over a 34 day period. All units have ranges set to 0. All have air leaders w/ 53 Leadership, and similar Inspiration ranging from 50 - 56.

Here are the results (% increase of levels over 34 days):

# Sqdns...# a/c...# pilots....ratio.........Exp...........Air........Defn
___4_____71____71_____1:1______32%___123%___73%
___2_____50____66_____1:1.3____29%___111%___71%
___2_____24____66_____1:1.7____17%____90%___52%

Conclusion: The lower the ratio, the faster the training, though obviously with fewer possible pilots trained.


2) Test Question: "Do fighter air units training on Sweep, train faster than units on Escort?"
I tested 6 fighter units, at Seattle, over a 34 day period. All units had their ranges set to 0. All had air leaders w/ 56 leadership, and similar inspiration ranging from 52 - 56.

Here are the results:

Mission.....# Sqdns...# a/c...# pilots....ratio.........Exp...........Air........Defn
Sweep/15k'___3_____70____70_____1:1_____32%___122%___81%
Escort/15k'____3_____75____75_____1:1____32%___120%___77%

Conclusion: Units training on Sweep, have a slight increase in Defn levels. But it also falls within a margin of error, and may prove to be elusive.


3) Test Question: "Do air units with higer rated leaders train faster than units with lower ones?"

I tested 4 fighter units, at Pearl Harbor, over a 34 day period. 2 sqdns had leaders of 50 leadership, and Inspiration of 40 and 47. The other 2 units had 36 Leadership, and Inspiration of 40. All on Training Escort @ 15k' at 0 range.

Here are the results:

Leaders....# Sqdns...# a/c...# pilots....ratio.........Exp...........Air........Defn
High (50)___2_____50____50_____1:1_____29%___112%___70%
Low (36)____2_____50____50_____1:1____30%___117%___69%

Conclusion: There is no consistent difference between units w/ higher rated leaders and units with lower rated ones. It's possible that other leader qualities besides Leadership & Inspiration have an impact on training, if so, I could not detect it.

Let me know what you think.

[Tokyo Boy]

good work. I copy past these results in my personal training guide

[Numdydar]

The leader qualities of the last test are too low for both sets. Try much higher leader skills of 70+ and see what happens. You may need to go in the editor to setup some leaders with high enough skills though.

[tiemanjw]

cool work.

What were the raw increases in experience? (such as started at 30, was 40 at the end of the test)

[tiemanjw]

ok, with respect to airframe to pilot ratio, assuming your pilots started at about 25 exp, they would gain 8, 7.25, and 4.25 exp / month (estimated). Assuming these move up to 50 exp linearly (not sure, but it seems close without doing much testing on that), than it would take 3.125, 3.445, and 5.88 months to "graduate" the classes at 50 exp. Normalized for 6 months, you could graduate 192, 230, and 280 pilots in 6 months. Sooo... it looks like the optimal airframe to pilot ratio is as low as you can get it. For the allies this means (in practical terms), that you should always fill your pilot rosters to max, but it is more important to "spread the wealth" with respect to airframes. As the war starts off and you are scraping the bottom of the barrel for airframes, get as many groups as you can on training and spread out the airframes as much as possible. As the war goes on, and your factories are drowning you putting them (or rotating off the front lines) fill out your training groups.
Short answer, always have max pilots in a training squadron - fill up with airframes as you get them.

[btd64]

What about airfield size? In my current AI game of Andy Mac design, I am finding that the larger the field the quicker they train. The number of units at the base did not seem to be a factor. I also set units at 100% CAP and they "sometimes" train quicker than in training. I have put a squadron with 38 exp and 42 air into combat and had them train quick. SINK OR SWIM. But they had 55 exp and 58 air after 4 weeks. High losses of aircraft, but didn't lose many pilots....MY

[CaptBeefheart]

Thanks for doing this work.

One thing I always wondered was the presence of a 70 EXP, 70 AIR pilot in the training squadron, whether as the commander or someone else. It would be cool if you could test for that (i.e. all noob pilots vs. all noob plus one veteran). Also, I think the leadership value in #3 might be a bit low: maybe 60 leadership would be a better test parameter.

Cheers,
CC

[geofflambert]

I think having a well trained pilot in the group is beneficial, regardless of whether he's the commander. But by how much? It could be a good idea in some circumstances but not others.

I think it's generally a good idea to use 1/3 of the group's capacity in aircraft, but that somewhat depends on how many planes you have that are only fit for trainers. If they're frontline aircraft I would say definitely keep them to 1/3.

[tiemanjw]

ORIGINAL: Commander Cody

Thanks for doing this work.

One thing I always wondered was the presence of a 70 EXP, 70 AIR pilot in the training squadron, whether as the commander or someone else. It would be cool if you could test for that (i.e. all noob pilots vs. all noob plus one veteran). Also, I think the leadership value in #3 might be a bit low: maybe 60 leadership would be a better test parameter.

Cheers,
CC

Made no difference... Either a better leader or a well trained / experienced pilot. See my AAR for the classified details. (disclaimer: this is "training" up to 50 exp. Beyond that - training or actual fighting - I have no idea).

[CaptBeefheart]

ORIGINAL: tiemanj

ORIGINAL: Commander Cody

Thanks for doing this work.

One thing I always wondered was the presence of a 70 EXP, 70 AIR pilot in the training squadron, whether as the commander or someone else. It would be cool if you could test for that (i.e. all noob pilots vs. all noob plus one veteran). Also, I think the leadership value in #3 might be a bit low: maybe 60 leadership would be a better test parameter.

Cheers,
CC

Made no difference... Either a better leader or a well trained / experienced pilot. See my AAR for the classified details. (disclaimer: this is "training" up to 50 exp. Beyond that - training or actual fighting - I have no idea).

Thanks for that. It looks like leadership and having a veteran on the roster have no effect whatsoever . Out of curiosity, did you check those numbers again 45 days later to see if the trend continued?

Cheers,
CC

[Trugrit]

I don't believe that is true,

The key is "instructors" not leaders.

You need to go back to the original documentation on this.

In your game folder there is a folder called manuals.
In that folder is a PDF called "Pilot Management Addendum - v1094b"

Scroll down to number 7 on training and read it carefully.

[tiemanjw]

ORIGINAL: Commander Cody

ORIGINAL: tiemanj

ORIGINAL: Commander Cody

Thanks for doing this work.

One thing I always wondered was the presence of a 70 EXP, 70 AIR pilot in the training squadron, whether as the commander or someone else. It would be cool if you could test for that (i.e. all noob pilots vs. all noob plus one veteran). Also, I think the leadership value in #3 might be a bit low: maybe 60 leadership would be a better test parameter.

Cheers,
CC

Made no difference... Either a better leader or a well trained / experienced pilot. See my AAR for the classified details. (disclaimer: this is "training" up to 50 exp. Beyond that - training or actual fighting - I have no idea).

Thanks for that. It looks like leadership and having a veteran on the roster have no effect whatsoever . Out of curiosity, did you check those numbers again 45 days later to see if the trend continued?

Cheers,
CC

yes, up to 50 exp. At 50 exp there may be a 1 point difference developing, but I haven't yet compared all the numbers (and I haven't posted it yet). Standby.

[tiemanjw]

ORIGINAL: Trugrit


I don't believe that is true,

The key is "instructors" not leaders.

You need to go back to the original documentation on this.

In your game folder there is a folder called manuals.
In that folder is a PDF called "Pilot Management Addendum - v1094b"

Scroll down to number 7 on training and read it carefully.

you don't need to believe it... I'm just reporting what the numbers say. Feel free to run additional tests to contradict the results.

I've read (and just now re-read) the pilot addendum, and here is my (imperfect, for sure) interpretation in light of these results:

1. if the pilot’s experience is less 50 (plus pilot’s missions and kills) and less
than the overall group experience level

This, I think, would normalize the pilots - such that lower exp pilots would "catch up", and keep the pilots clustered together in experience. The test results show this with all but about 2-3% of pilots within 2 points.

2. if the pilot’s experience is less 50 (plus pilot’s missions and kills) and less
than the leader’s skill

So it appears that so long as leaders (leadership?) skill is above 50 (or whatever you are training to), this would make no effect. My tests would not show this.

3. if pilot is in a dedicated Training group with some Instructors (adds higher
increments to accumulators)

This is probably the one you refer to. I agree that a reading of this suggests that "instructions" would help. However, I do not see this in my results. I do remember Alfred (sorry if I am misunderstanding you, Alfred) saying that the pilot addendum is a bit outdated, and that it was written when there was something in the game called a dedicated training group. I have not been playing long enough to have seen this. Perhaps this mechanic was left on the cutting room floor when training groups were removed.(?)

4. if pilot is in a dedicated Training group without Instructors (higher success
and slightly higher increments than a normal group)

Again, no training groups, so this may be outdated.

5. if pilot is in a group with some training percent (number of veteran pilots
(experience of 80+) increases chance of successful training)

This one, as well, seems to suggest what you are saying should be true. I can't explain this one. The only 2 things I can come up with are
a) the effect is so small as to be meaningless when averaged over a training cycle
b) my "veteran" pilots are all in "group reserve" to prevent training accidents. Perhaps this makes a difference?

[Alfred]

These "tests" will not allow for reverse engineering the algorithms.  By themselves they can however provide some tentative support for relationship factors.  What I see primarily from these tests is back up for what I have said on numerous occasions viz that there is a lot of common knowledge which is just wrong but nonetheless regularly trotted out.
 
There is very little testing done on this forum which has any real validity.  Generally, they simply are not rigorous enough.  Some years ago, Symon who was the key dev in setting up the AE test development program, pointed out that to get statistically valid results one needed 30 test runs and to completely drop back to the desktop after each, just to examine a factor.  With access to the code and the constants, he knew how to set up the test parameters so that only the single variable was actually tested.  In short, a lot of work which is not usually replicated by non devs "testing" the game.
 
As to the pilot addendum being outdated, that is too harsh.  It needs to be intelligently read, as Tiemanj does in post #13, in the light of subsequent game changes.  It still remains fundamental to understanding the game code.  This threads "testing" is consistent with what the addendum says.
 
Alfred 

[tiemanjw]

ORIGINAL: Alfred

These "tests" will not allow for reverse engineering the algorithms.  By themselves they can however provide some tentative support for relationship factors.  What I see primarily from these tests is back up for what I have said on numerous occasions viz that there is a lot of common knowledge which is just wrong but nonetheless regularly trotted out.

There is very little testing done on this forum which has any real validity.  Generally, they simply are not rigorous enough.  Some years ago, Symon who was the key dev in setting up the AE test development program, pointed out that to get statistically valid results one needed 30 test runs and to completely drop back to the desktop after each, just to examine a factor.  With access to the code and the constants, he knew how to set up the test parameters so that only the single variable was actually tested.  In short, a lot of work which is not usually replicated by non devs "testing" the game.

As to the pilot addendum being outdated, that is too harsh.  It needs to be intelligently read, as Tiemanj does in post #13, in the light of subsequent game changes.  It still remains fundamental to understanding the game code.  This threads "testing" is consistent with what the addendum says.

Alfred 

No doubt - I can't speak for others, but I do have other things to do besides this game (at least I keep telling myself that). So I can't afford time-wise to put the necessary rigor into this to get statistically valid results - even if I could get all the data and know/isolate all the variables.

But I would argue that this testing has value. Just because they don't reverse engineer the games algorithms (which I have no intention of doing) they do suggest the following when training pilots:
fill up your pilot rosters to the max. If you subscribe to the idea that it is a long war and you need many pilots - the best way (probably) is to max out your pilot rosters.
If you have enough airframes to fill out all the TO&Es for training squadrons, do it. Each extra airframe helps train quicker, but at diminishing returns. If you do not have enough airframes, I would argue that it is best to spread the wealth as much as possible
It is [probably] not worth the PP cost (and opportunity cost) to put good leaders into training squadrons. You (or at least I) never have enough of either, and it appears that if there is a benefit to this, it is minimal.
It is [probably] not worth the effort (in mouse clicks) and opportunity cost to pull in high experienced pilots to help train the rookies (caveat: only tested with the exp pilots in group reserve)

Of course the data is what it is, and I think I've fully explained my methodology (in my AAR, there are a few details I don't want completely public yet - sorry Lowpie). If someone wants clarification, thinks something is wrong, has contradicting observations, or amplifying information - I love the discussion, and I'm always open to learning something.

[mussey]

ORIGINAL: tiemanj

cool work.

What were the raw increases in experience? (such as started at 30, was 40 at the end of the test)

Good question. On average they started at about 29/29/29 (Exper./ Air/ Defn.) and finished about 35/60/49.

What Alfred mentioned above is important, that multiple tests need to be done to test out these observations. A study is only valid and reliable if it can be duplicated. I think, however, that the #1 study (ac to airframe) is quite revealing. While the Sweep to Escort needs more testing, especially at other altitudes, maybe at 100' (?) instead of 15k. The leadership issue is confounding me, and quite frankly needs a more sophisticated approach in future tests. Anyone???

Things to remember:
- more testing needed to prove/disprove...
- this was only with fighters. Other types may lead to different results...
- Leadership studies were conducted with sqdns at Rookie status (below 50), with leaders <56. Perhaps results may differ with higher levels...

Thanks for excellent comments.

[mussey]

Also, I wonder what the result would be if a sqdn with 0 ac was filled to max pilots...? [8|]

[geofflambert]

ORIGINAL: tiemanj


but I do have other things to do besides this game (at least I keep telling myself that).

It is only a game.

There is nothing that is not the game.

Time is subordinate to the game.

Space makes room for the game.

[Trugrit]

Tiemanj,

I was replying to Commander Cody when he said:
“It looks like leadership and having a veteran on the roster have no effect whatsoever”
I don’t believe that is true because the addendum says that is not true in regard to veterans
80+ experience in the group. That is what I was talking about.
I could be wrong.

A longer test would help.
Because the addendum says you can’t “generally” get to 70 without combat, I don’t think you could get a long enough running test.

What does the addendum mean by “generally”?
Does that mean that there is a way to get over 70 without combat?
Is the addendum blowing smoke up my………?

If there is a path for getting over 70 without combat I’m all ears.

Alfred is right about testing.

Also there is this in the addendum:
Points are accumulated for each skill/experience and once a certain level is reached, a
point is added to the actual skill/experience level.
The cutover level is based on the current skill/experience level with some randomness
thrown in.

There is not a good test for “some” randomness without knowing what the “some” is.

How much is “some”? 50/50 or is it like dice? You can only get a probability over a long run but
Over a short run it’s meaningless. In this case we don’t even know what the dice is.

Is 34 throws of the unknown dice long enough? I don’t know.

Just my two cents.

[geofflambert]

I always train fighter pilots to 70 before releasing them to combat, so yes you can. Other specialties I train to the 60's and then put them in action.

I'm presuming that the "70" desired is air combat skill, not overall experience. It's a good thing to get the defense skill as high as you can and you can do that by training them (fighter pilots) to strafe or do low bombing attacks.