Coder Diary #6 -- CSlint

John Tiller's Campaign Series exemplifies tactical war-gaming at its finest by bringing you the entire collection of TalonSoft's award-winning campaign series. Containing TalonSoft's West Front, East Front, and Rising Sun platoon-level combat series, as well as all of the official add-ons and expansion packs, the Matrix Edition allows players to dictate the events of World War II from the tumultuous beginning to its climatic conclusion. We are working together with original programmer John Tiller to bring you this updated edition.

Moderators: Jason Petho, Peter Fisla, asiaticus, dogovich

Post Reply
User avatar
berto
Posts: 21461
Joined: Wed Mar 13, 2002 1:15 am
Location: metro Chicago, Illinois, USA
Contact:

Coder Diary #6 -- CSlint

Post by berto »


Coder Diary #6 -- CSlint


In Coder Diary #4, I wrote:
ORIGINAL: berto

Can we automate checking

[*]all units (North Vietnamese, US, ARVN, ..., Israeli, Egyptian, ..., German, ... Russian, ... Japanese, ...)

across

[*]all scenarios

across

[*]all games (EF, WF, RS, ME, VN, ...)

?

But of course. Why, yes we can! [:)]

I have begun developing a set of Cygwin-based tools for

[*]Checking scenario files for internal consistency (so as to prevent, for example, the no-fire unit bug described above).
[*]Checking all game data files for all manner of errors.
[*]Reporting game data in easy-to-read fashion (e.g., csoobrpt.pl).
[*]Converting hexadecimal C++ unit attribute codes to their decimal .oob file equivalents, and vice versa (csflags.pl).
[*]Other purposes and uses.

I call my toolkit "CSlint":
What is CSlint? Akin to the well-known C programming language utility lint. ("Lint: A Unix C language processor which carries out more thorough checks on the code than is usual with C compilers. Lint is named after the bits of fluff it supposedly picks from programs." See here.) Similarly, CSlint is a toolkit of programs designed to discover and report bugs, problems, glitches, anomalies in the Campaign Series game data files.
With the CSlint toolkit, there are all manner of things we can QA check (and all manner of things we can make easier).

And because our computers are so fast, we can run these checks in just a few minutes, across all games. As often as we wish! Create a scenario, edit a unit file, run the QA tools on the spot. Real-time QA, as we develop. Not post-release. [:-]
Here are some glimpses of recent work done on CSlint ...

csoobrpt.pl is a Perl script (program) to report the CS (Campaign Series) OOBs (the .oob files).

One can use csoobrpt.pl to report OOB info straight up, as in:

Robert@roberto ~/cslint
$ ./csoobrpt.pl -g vn -m "huey" -id -carryflags -name -short -nick
P11622 8 UH-1D Huey Recon UH-1D Huey Recon
P11626 8 UH-1D Huey UH-1D Huey
P11627 8 UH-1D Huey UH-1D Huey
P11628 8 UH-1D Huey UH-1D Huey
P10632 32 UH-1B Huey UH-1B Huey
P10633 32 UH-1B Huey UH-1B Huey
P10634 32 UH-1B Huey UH-1B Huey
P10639 8 UH-1D Huey UH-1D Huey
P10640 8 UH-1D Huey UH-1D Huey
P10641 8 UH-1D Huey UH-1D Huey
P10860 16 UH-1D Huey Recon UH-1D Huey Recon
P19621 0 UH-1D Huey UH-1D Huey
P19622 0 UH-1D Huey UH-1D Huey
P19623 0 UH-1D Huey UH-1D Huey

which shows, for the Vietnam War game, the carryflags for all Huey (helicopters).

You can also, combined with standard Linux/Cygwin tools, munge csoobrpt.pl output in interesting ways:

Robert@roberto ~/cslint
$ ./csoobrpt.pl -g vn -f -l -s/ -id -speed -name -short -nick | awk '$3>1000'
platoon10.oob:513: P10304 1200 /Divisional HQ (heli)/ /Division HQ/
platoon10.oob:515: P10306 1200 /Brigade HQ (heli)/ /Brigade HQ/

which reports (for Vietnam War) all units with speed values greater than 1000 -- outliers, and possibly mistaken?

Robert@roberto ~/cslint
$ ./csoobrpt.pl -g vn -f -l -s/ -id -startyear -stopyear -name -short -nick | awk '$3>=62 && $4<=72'
Platoon11.oob:36: P11610 62 64 /M-114/ /M-114/
Platoon11.oob:37: P11611 62 64 /M-114/ /M-114/
Platoon11.oob:249: P11635 62 72 /CH-34 Choctaw/ /CH-34 Choctaw/
Platoon11.oob:250: P11636 62 72 /CH-34 Choctaw/ /CH-34 Choctaw/
Platoon11.oob:251: P11637 62 72 /CH-34 Choctaw/ /CH-34 Choctaw/
platoon10.oob:75: P10073 63 72 /M132 Flamethrowers/ /M132/ /Zippo/
platoon10.oob:466: P10281 62 70 /Reconnaissance 62/ /Reconnaissance 62/ /M14/
platoon10.oob:579: P10664 65 67 /UH-1B Gunship/SS-11/ /UH-1B Gunship/SS-11/
platoon10.oob:604: P10869 64 69 /AC47 Gunship/ /AC47 Gunship/ /Spooky/
platoon10.oob:633: P10524 63 72 /F-100D Supersabre (Rockets)/ /Supersabre/ /Hun/
platoon10.oob:648: P10539 62 66 /F-4B Phantom II/ /Phantom II/
platoon10.oob:649: P10540 62 66 /F-4C Phantom II/ /Phantom II/
platoon19.oob:98: P19128 62 69 /OTO 105mm L5 Pack Howitzer/ /L5 105mm Howitzer/
platoon19.oob:99: P19129 62 69 /OTO 105mm L5 Pack Howitzer/ /L5 105mm Howitzer/

which shows (for Vietnam War) all units available between 1962 & 1972, but not before or after.

Robert@roberto ~/cslint
$ ./csoobrpt.pl -g vn -f -l -s/ -m "sherman" -strength -assault -name -short -nick | awk '$2 > $3'
platoon10.oob:8: 9 8 /M4A3(76) Medium Tanks/ /M4A3(76)/ /Sherman/
platoon10.oob:9: 9 8 /M4A3(76) Medium Tanks/ /M4A3(76)/ /Sherman/
platoon10.oob:10: 9 8 /M4A3(76) Medium Tanks/ /M4A3(76)/ /Sherman/
platoon10.oob:72: 18 9 /POA-CWS-H1 Flamethrower Tanks/ /POA-H1/ /Sherman/
platoon10.oob:73: 20 9 /POA-CWS-H5 Flamethrower Tanks/ /POA-H5/ /Sherman/
platoon10.oob:74: 20 9 /POA-CWS-H5 Flamethrower Tanks/ /POA-H5/ /Sherman/

which shows (for Vietnam War) any Shermans where the strength value exceeds the assault value.

How about Shermans where the strength value is less than the assault value?

Robert@roberto ~/cslint
$ ./csoobrpt.pl -g vn -f -l -s/ -m "sherman" -strength -assault -name -short -nick | awk '$2 < $3'
Platoon08.oob:18: 7 8 /M4 Medium Tanks/ /M4 Sherman/ /Sherman/
Platoon08.oob:19: 7 8 /M4 Medium Tanks/ /M4 Sherman/ /Sherman/
Platoon08.oob:20: 7 8 /M4 Medium Tanks/ /M4 Sherman/ /Sherman/
platoon00.oob:2: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
platoon00.oob:3: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
platoon00.oob:4: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
platoon03.oob:2: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
platoon03.oob:3: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
platoon03.oob:4: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
platoon10.oob:11: 7 10 /M4A3E8 Medium Tanks/ /M4A3E8/ /Sherman/
platoon10.oob:12: 7 10 /M4A3E8 Medium Tanks/ /M4A3E8/ /Sherman/
platoon10.oob:13: 7 10 /M4A3E8 Medium Tanks/ /M4A3E8/ /Sherman/
platoon10.oob:14: 8 10 /M4E5 Medium Tanks/ /M4E5/ /Sherman/
platoon10.oob:15: 8 10 /M4E5 Medium Tanks/ /M4E5/ /Sherman/
platoon10.oob:70: 3 10 /M4A3 Flail Tanks/ /M4A3 Flail/ /Sherman Crab/
platoon10.oob:71: 3 11 /M4A3 Dozer/ /M4A3 Dozers/ /Sherman Dozer/
platoon10.oob:251: 0 6 /Maintenance Recovery M74/ /M74 ARV/ /--- based on a Sherman/

Lots and lots of capability here. csoobrpt.pl is extremely useful for eyeballing and making better sense of the cryptic OOB data.

Another CSlint utility:

csflags.pl is a Perl script (program) to convert platoon*.oob flag values to their flag name equivalents, and vice versa.

What are flag values? These are special OOB (.oob file) numbers that specify various unit attributes and capabilities, for example:

// Unit Other flags.
const int XCanDigIn = 0x00000001;
const int XMineClearing = 0x00000002;
const int XFireHardAI = 0x00000004;
const int XFireSoftAI = 0x00000008;
const int XReconAI = 0x00000010;
const int XAssaultAI = 0x00000020;
const int XRetreatAI = 0x00000040;
const int XCanDamage = 0x00000080;
const int XCarryAny = 0x00000100;
const int XIsDiveBomber = 0x00000200;
const int XPathfinder = 0x00000400;
const int XAirPlane = 0x00000800;
const int XHelicopter = 0x00001000;
const int XRecovery = 0x00004000;
const int XLayVehBridge = 0x00008000;
const int XLayFootBridge = 0x00010000;
const int XSpecOps = 0x00020000;

By means of a nearly inscrutable and -- for us humans -- error-prone process, we can combine these flag values into a single number, e.g., the carry flag value 131, which signifies

const int XLoadable = 0x00000001;
const int XRidable = 0x00000002;
...
const int XPortage = 0x00000080;

Converting hexadecimal flag values to their decimal number equivalents and back again -- technical mumbo jumbo that we humans are not very good at. So I wrote csflags.pl to help us out.

Examples:

Robert@roberto ~/cslint
$ ./csflags.pl -g me -carry 131
XLoadable/XRidable/XPortage

which (for the game Middle East) shows you the meaning of the carryflag value 131.

Robert@roberto ~/cslint
$ ./csflags.pl -g wf -move "XTracked/XHalftrack/XNoLtBridge/XNoMedBridge/XNoHvyBridge"
16485

which (for the game West Front) shows that the combined moveflags XTracked/XHalftrack/XNoLtBridge/XNoMedBridge/XNoHvyBridge have the value 16485.

csflags.pl replaces some incomplete and outdated reference documents that, heretofore, modders have used to set and interpret the unit flags.

And what about the promised .scn (scenario file) checks?

csscnchk.pl is a Perl script (program) to check the scenario (.scn) files for various errors.

robert@roberto ~/cslint
$ ./csscnchk.pl -g vn
ERROR: DienBienPhu-1953_1.scn:44: id 21269 (tracking -1) not found in DienBienPhu-1953_1.org
ERROR: Dong_Vinh_Xanh_1965.scn:167: id 10665 not found in Dong_Vinh_Xanh_1965.org
ERROR: Dong_Vinh_Xanh_1965.scn:329: id 21218 not found in Dong_Vinh_Xanh_1965.org
ERROR: Dong_Vinh_Xanh_1965.scn:362: id 10310 (tracking -1) not found in Dong_Vinh_Xanh_1965.org
ERROR: Hiep_Hoa_1963.scn:47: id 21402 (tracking -1) not found in Hiep_Hoa_1963.org
ERROR: Hiep_Hoa_1963.scn:48: id 21201 (tracking -1) not found in Hiep_Hoa_1963.org
ERROR: Hiep_Hoa_1963.scn:49: id 21049 (tracking -1) not found in Hiep_Hoa_1963.org
ERROR: Hiep_Hoa_1963.scn:50: id 21107 (tracking -1) not found in Hiep_Hoa_1963.org
ERROR: IaDrang-1965.scn:57: id 21209 (tracking -1) not found in IaDrang-1965.org
ERROR: KheSanh-1968.scn:668: id 10272 (tracking -1) not found in KheSanh-1968.org

For the game Vietnam War, the output shows:

[*]the scenario (.scn) file name
[*]the line # where the error occurred
[*]the unit id
[*]'(tracking -1)', if that unit's tracking value was specified that way
[*]the OOB (.org) file name corresponding to the scenario file name

You can also use csscnchk.pl to check a single file you are currently working on, for example:

robert@roberto ~/cslint
$ ./csscnchk.pl -g vn -f Hiep_Hoa_1963.scn
ERROR: Hiep_Hoa_1963.scn:47: id 21402 (tracking -1) not found in Hiep_Hoa_1963.org
ERROR: Hiep_Hoa_1963.scn:48: id 21201 (tracking -1) not found in Hiep_Hoa_1963.org
ERROR: Hiep_Hoa_1963.scn:49: id 21049 (tracking -1) not found in Hiep_Hoa_1963.org
ERROR: Hiep_Hoa_1963.scn:50: id 21107 (tracking -1) not found in Hiep_Hoa_1963.org

(Observe that the csscnchk.pl output above confirms previous analysis about the Hiep_Hoa_1963.scn. See http://www.matrixgames.com/forums/fb.asp?m=3341248.

If you're wondering: No, csscnchk.pl does not use the same command-line code in that earlier example. Those were just proof-of-concept command lines to prove a point. csscnchk.pl is more sophisticated and comprehensive than that.)

Are there erroneous .scn files for the games EF, WF & RS (and ME)? Yes, but I won't report their errors here. You'll have to use csflags.pl yourselves to find out. [:'(]

CSlint is already proving useful for QAing (Quality Assuring) the Campaign Series/Modern Wars game data files. CSlint will be a great benefit for modders -- which is why we anticipate packaging it with all future CS/MW new releases and udpates. (But there's a catch: You'll have to install Cygwin first.)

Aside from CSlint (and post-codebase merge clean-up), still more stuff...

See the attached screenshot.

Image

In the panel

[*]to the far left, you see the standard File menu in all current CS/MW games
[*]in the middle, you see the File menu in Rising Sun for an interim build of that game
[*]to the far right, you see the File menu in the JTS [John Tiller Software] game, The Renaissance

I hadn't really thought about it, but when I went to look, I was quite surprised to discover that, until now, the Campaign Series of games -- to be more precise, their engines (as opposed to their frontends) -- have lacked the File>Open option (to resume saved games), not to mention the File>New option (to launch new games).

Why? It took a bit of detective work, but eventually I found this in some very old legacy CS code, code that had been carried over to more recent CS codebases:

MainMenu MENU {
POPUP MsgFileM {
#if defined(_DEBUG)
MENUITEM MsgNewM, ON_FILENEW
MENUITEM MsgOpenM, ON_FILEOPEN
#endif

MENUITEM MsgSaveM, ON_FILESAVE
MENUITEM MsgSaveAsM, ON_FILESAVEAS
MENUITEM SEPARATOR
MENUITEM MsgReplayM, ON_REPLAY
MENUITEM SEPARATOR
MENUITEM MsgExitM, ON_FILEEXIT
}

(Also other omissions and restrictions, which I won't detail here.)

Ah, so it appears that ON_FILENEW & ON_FILEOPEN were for debugging purposes only; that all new games and save game reloads were to be done the "official way", via the game frontend launcher.

Seeing no reason not to, I have lifted the restriction and have fully implemented both File>New and File>Open:

Image

You may, in all future CS/MW releases, select new games and load saved games directly from the game engine (the actual game program), bypassing the frontend launcher entirely.

Taking it easy in The Man Cave? Nyet! Hard at work as ever.

Until the next time ...
Campaign Series Legion https://cslegion.com/
Campaign Series Lead Coder https://www.matrixgames.com/forums/view ... hp?f=10167
Panzer Campaigns, Panzer Battles Lead Coder https://wargameds.com
User avatar
wings7
Posts: 4586
Joined: Mon Aug 11, 2003 4:59 am
Location: Phoenix, Arizona

RE: Coder Diary #6 -- CSlint

Post by wings7 »

Excellent! [&o]
Please come and join and befriend me at the great Steam portal! There are quite a few Matrix/Slitherine players on Steam! My member page: http://steamcommunity.com/profiles/76561197988402427
User avatar
Crossroads
Posts: 17498
Joined: Sun Jul 05, 2009 8:57 am

RE: Coder Diary #6 -- CSlint

Post by Crossroads »

Enjoying your Coder Diary entries Berto! [:)]
Visit us at: Campaign Series Legion
---
CS: Vietnam 1948-1967 < Available now
CS: Middle East 1948-1985 2.0 < 3.0 In the works
User avatar
XLVIIIPzKorp
Posts: 224
Joined: Tue Oct 24, 2006 12:34 am
Contact:

RE: Coder Diary #6 -- CSlint

Post by XLVIIIPzKorp »

Yep, me too. Always looking for them.
User avatar
junk2drive
Posts: 12856
Joined: Thu Jun 27, 2002 7:27 am
Location: Arizona West Coast

RE: Coder Diary #6 -- CSlint

Post by junk2drive »

bump 6
Conflict of Heroes "Most games are like checkers or chess and some have dice and cards involved too. This game plays like checkers but you think like chess and the dice and cards can change everything in real time."
Post Reply

Return to “John Tiller's Campaign Series”