Accuracy of combat reports - An analysis

This new stand alone release based on the legendary War in the Pacific from 2 by 3 Games adds significant improvements and changes to enhance game play, improve realism, and increase historical accuracy. With dozens of new features, new art, and engine improvements, War in the Pacific: Admiral's Edition brings you the most realistic and immersive WWII Pacific Theater wargame ever!

Moderators: wdolson, MOD_War-in-the-Pacific-Admirals-Edition

Post Reply
User avatar
Yakface
Posts: 846
Joined: Sat Aug 05, 2006 11:43 am

Accuracy of combat reports - An analysis

Post by Yakface »

I had a concern over sync in my game. Which lead me to do some analysis on the reliability of the ground combat portion of the combat reports.

Method

Modified a scn to put 7 Japanese divisions onto Luzon. They made their way to Clark field, as did most of the Allied army on the island.

Once set-up I ran a series of combats. Each involving 2 days of Japanese attacks, one directly after the other. Notes were made of:

1) The actual AV in the hex
2) The raw AV reported in the combat report*
3) the casualties*

* see image

Then the same with the second day's attack and finally the actual AV in the hex on day 3



Image
Attachments
AV and Cas.jpg
AV and Cas.jpg (42.87 KiB) Viewed 82 times
User avatar
Yakface
Posts: 846
Joined: Sat Aug 05, 2006 11:43 am

RE: Accuracy of combat reports - Analysis

Post by Yakface »

Here are the results + analysis





Editted table to be more explanatory and smaller and a minor error



Image
Attachments
Analysis.jpg
Analysis.jpg (85.58 KiB) Viewed 82 times
User avatar
Yakface
Posts: 846
Joined: Sat Aug 05, 2006 11:43 am

RE: Accuracy of combat reports - Analysis

Post by Yakface »

Explanation of table:

the first 10 lines are the test. The red figures are the actual observations with some number processing in black

Column E: what the expected AV would be if the casualties in the report had been 100% accurate

Column F: the expected AV in column E minus the actual AV in column D. - figure means the report over estimated losses, a positive figure means more casualties inflicted than were reported.

Column G: actual AV divided by reported loss to get a percentage difference.

Column H: how far the results deviated from the reported as a factor of 2. This is treated differently for under reporting or over reporting. As an under-report of 50% is the same in statistical terms as an over report of 100%. Negative and positive figures have been expressed in a way that can be directly compared:

If the actual result was under-reported in the combat report this calcualtion was applied:

=(figure in column G)*100/100-1

If the report exaggerated the casualties:

=100/(figure in column G*100)-1
User avatar
Yakface
Posts: 846
Joined: Sat Aug 05, 2006 11:43 am

RE: Accuracy of combat reports - Analysis

Post by Yakface »

Conclusions:

The reports are moderately accurate.

Specifically:

1) the Raw Av in the hex is equal to the actual AV in the Hex +1 for each combat unit in the hex. There is no randomness - it is spot on (with the +1/unit unusual feature).

NB - this may not be true when defending - the test was just on Japanese attacks.

2) There was over reporting and under-reporting at times. More frequently over reporting. However the results didn't deviate by a factor of greater that .54 in either direction. Ie +54% actual over reported (under-reporting casualties in the combat report) or -35% actual over reported (exaggeration of casualties in the report)

There are some uncertainties in the method. It is a little rough and ready (for example a destroyed squad could have already been disabled, or healthy. depending upon which it may have had an AV value or not so counting all destroyed squads as a loss of AV may be innaccurate. However since the vast majority of casualties were diablements, this introduces only minor uncertainty.)
User avatar
Yakface
Posts: 846
Joined: Sat Aug 05, 2006 11:43 am

RE: Accuracy of combat reports - Analysis

Post by Yakface »

Forgot to say - the result at the bottom is the result from the turn with suspected sync problems.

I don't know if I have enough datapoints (effectively I've got 20: 10 tests, 2 days each) to be able to say with certainty I have a problem, however it looks anomolous (out by a factor of 3.6 compared to a high of .54 in the test). Is there anyone out there with more statistical expertise who is able to say that it is mathematically significant given the size of the sample?
wdolson
Posts: 7648
Joined: Tue Jun 27, 2006 9:56 pm
Location: Near Portland, OR

RE: Accuracy of combat reports - Analysis

Post by wdolson »

There are known problems with the reporting of specific casualties on the ground combat report. We made some improvements during development, but the way it was designed, it was impossible to make it perfect without a very deep rewrite which we didn't have time for.

Bill
WitP AE - Test team lead, programmer
Image
User avatar
witpqs
Posts: 26376
Joined: Mon Oct 04, 2004 7:48 pm
Location: Argleton

RE: Accuracy of combat reports - Analysis

Post by witpqs »

The way I look at it, the staff is restricted to 1940's Morse Code technology for sending combat reports back to me, not to mentioning encryption/decryption. I figure I'm lucky to get reports so quickly!

[:D]
User avatar
Yakface
Posts: 846
Joined: Sat Aug 05, 2006 11:43 am

RE: Accuracy of combat reports - Analysis

Post by Yakface »

There are a few unusual features - for example, motorized support casualties are added to both the total of vehicle and non-combat numbers, so there is a built in exaggeration there which will affect every combat where a motorized support is damaged/destroyed
User avatar
btd64
Posts: 12736
Joined: Sat Jan 23, 2010 12:48 am
Location: Lancaster, OHIO

RE: Accuracy of combat reports - Analysis

Post by btd64 »

Fog of war.[:D]
I had a combat report, several times, different battles, different games, all AI though, say a unit was in a battle when in fact it was either set to defend or in the next hex.
Air battles, I had 12 betty's bombing a base with 14 damaged and 1 killed by flak.
It's not perfect. I like the inaccuracy in the reports. It keeps the game fun and interesting. And coming back for more.[:)]....GP
Intel i7 4.3GHz 10th Gen,16GB Ram,Nvidia GeForce MX330

AKA General Patton

DWU-Beta Tester
TOAW4-Alpha/Beta Tester
DW2-Alpha/Beta Tester
New Game Development Team

"Do everything you ask of those you command"....Gen. George S. Patton
User avatar
witpqs
Posts: 26376
Joined: Mon Oct 04, 2004 7:48 pm
Location: Argleton

RE: Accuracy of combat reports - Analysis

Post by witpqs »

ORIGINAL: General Patton

Fog of war.[:D]
I had a combat report, several times, different battles, different games, all AI though, say a unit was in a battle when in fact it was either set to defend or in the next hex.
Air battles, I had 12 betty's bombing a base with 14 damaged and 1 killed by flak.
It's not perfect. I like the inaccuracy in the reports. It keeps the game fun and interesting. And coming back for more.[:)]....GP
Set to defend, yes they include all units there. In a different hex, wow - I haven't hit that one yet!
User avatar
dr.hal
Posts: 3449
Joined: Sat Jun 03, 2006 12:41 pm
Location: Covington LA via Montreal!

RE: Accuracy of combat reports - Analysis

Post by dr.hal »

Nicely done Yakface, I for one appreciate the effort. Certainly if you want to get "scientific" about it, you would need to increase your data points. But these data that you do provide are enough to give us a feel for what is going on. And in truth, I think it is absolutely "accurate" to be inaccurate, if you get my meaning. As is said above, "fog of war." Additionally it is historically the norm to over exaggerate the damage you do to an enemy. During Vietnam I certainly remember the numbers of dead NVR and Viet Cong being reported and if one were to total those numbers up for the 10 years of conflict, I believe the statisticians stated the result would be more than the male military aged population of North Vietnam. So a "beefed up" number is to be expected. I've always take the reports with a large bucket (not grain) of salt. Thanks for your work. Hal
User avatar
Yakface
Posts: 846
Joined: Sat Aug 05, 2006 11:43 am

RE: Accuracy of combat reports - Analysis

Post by Yakface »

Yeah - I have no problem with the fog of war. As people have said, uncertainty keeps things interesting. It's useful to know what the limits are to that uncertainty. It seems to be that you can expect to have done somewhere between two thirds and 150% of reported casualties
Post Reply

Return to “War in the Pacific: Admiral's Edition”