Matrix Games Forums

Forums  Register  Login  Photo Gallery  Member List  Search  Calendars  FAQ 

My Profile  Inbox  Address Book  My Subscription  My Forums  Log Out

Coder Diary #6 -- CSlint

 
View related threads: (in this forum | in all forums)

Logged in as: Guest
Users viewing this topic: none
  Printable Version
All Forums >> [Current Games From Matrix.] >> [World War II] >> John Tiller's Campaign Series >> Coder Diary #6 -- CSlint Page: [1]
Login
Message << Older Topic   Newer Topic >>
Coder Diary #6 -- CSlint - 7/4/2013 11:53:28 AM   
berto


Posts: 18267
Joined: 3/13/2002
From: metro Chicago, Illinois, USA
Status: offline

Coder Diary #6 -- CSlint


In Coder Diary #4, I wrote:

quote:

ORIGINAL: berto

Can we automate checking

  • all units (North Vietnamese, US, ARVN, ..., Israeli, Egyptian, ..., German, ... Russian, ... Japanese, ...)

    across

  • all scenarios

    across

  • all games (EF, WF, RS, ME, VN, ...)

    ?

    But of course. Why, yes we can!

    I have begun developing a set of Cygwin-based tools for

  • Checking scenario files for internal consistency (so as to prevent, for example, the no-fire unit bug described above).
  • Checking all game data files for all manner of errors.
  • Reporting game data in easy-to-read fashion (e.g., csoobrpt.pl).
  • Converting hexadecimal C++ unit attribute codes to their decimal .oob file equivalents, and vice versa (csflags.pl).
  • Other purposes and uses.

    I call my toolkit "CSlint":

    quote:

    What is CSlint? Akin to the well-known C programming language utility lint. ("Lint: A Unix C language processor which carries out more thorough checks on the code than is usual with C compilers. Lint is named after the bits of fluff it supposedly picks from programs." See here.) Similarly, CSlint is a toolkit of programs designed to discover and report bugs, problems, glitches, anomalies in the Campaign Series game data files.

    With the CSlint toolkit, there are all manner of things we can QA check (and all manner of things we can make easier).

    And because our computers are so fast, we can run these checks in just a few minutes, across all games. As often as we wish! Create a scenario, edit a unit file, run the QA tools on the spot. Real-time QA, as we develop. Not post-release.

  • Here are some glimpses of recent work done on CSlint ...

    csoobrpt.pl is a Perl script (program) to report the CS (Campaign Series) OOBs (the .oob files).

    One can use csoobrpt.pl to report OOB info straight up, as in:



    Robert@roberto ~/cslint
    $ ./csoobrpt.pl -g vn -m "huey" -id -carryflags -name -short -nick
    P11622 8 UH-1D Huey Recon UH-1D Huey Recon
    P11626 8 UH-1D Huey UH-1D Huey
    P11627 8 UH-1D Huey UH-1D Huey
    P11628 8 UH-1D Huey UH-1D Huey
    P10632 32 UH-1B Huey UH-1B Huey
    P10633 32 UH-1B Huey UH-1B Huey
    P10634 32 UH-1B Huey UH-1B Huey
    P10639 8 UH-1D Huey UH-1D Huey
    P10640 8 UH-1D Huey UH-1D Huey
    P10641 8 UH-1D Huey UH-1D Huey
    P10860 16 UH-1D Huey Recon UH-1D Huey Recon
    P19621 0 UH-1D Huey UH-1D Huey
    P19622 0 UH-1D Huey UH-1D Huey
    P19623 0 UH-1D Huey UH-1D Huey



    which shows, for the Vietnam War game, the carryflags for all Huey (helicopters).

    You can also, combined with standard Linux/Cygwin tools, munge csoobrpt.pl output in interesting ways:



    Robert@roberto ~/cslint
    $ ./csoobrpt.pl -g vn -f -l -s/ -id -speed -name -short -nick | awk '$3>1000'
    platoon10.oob:513: P10304 1200 /Divisional HQ (heli)/ /Division HQ/
    platoon10.oob:515: P10306 1200 /Brigade HQ (heli)/ /Brigade HQ/



    which reports (for Vietnam War) all units with speed values greater than 1000 -- outliers, and possibly mistaken?



    Robert@roberto ~/cslint
    $ ./csoobrpt.pl -g vn -f -l -s/ -id -startyear -stopyear -name -short -nick | awk '$3>=62 && $4<=72'
    Platoon11.oob:36: P11610 62 64 /M-114/ /M-114/
    Platoon11.oob:37: P11611 62 64 /M-114/ /M-114/
    Platoon11.oob:249: P11635 62 72 /CH-34 Choctaw/ /CH-34 Choctaw/
    Platoon11.oob:250: P11636 62 72 /CH-34 Choctaw/ /CH-34 Choctaw/
    Platoon11.oob:251: P11637 62 72 /CH-34 Choctaw/ /CH-34 Choctaw/
    platoon10.oob:75: P10073 63 72 /M132 Flamethrowers/ /M132/ /Zippo/
    platoon10.oob:466: P10281 62 70 /Reconnaissance 62/ /Reconnaissance 62/ /M14/
    platoon10.oob:579: P10664 65 67 /UH-1B Gunship/SS-11/ /UH-1B Gunship/SS-11/
    platoon10.oob:604: P10869 64 69 /AC47 Gunship/ /AC47 Gunship/ /Spooky/
    platoon10.oob:633: P10524 63 72 /F-100D Supersabre (Rockets)/ /Supersabre/ /Hun/
    platoon10.oob:648: P10539 62 66 /F-4B Phantom II/ /Phantom II/
    platoon10.oob:649: P10540 62 66 /F-4C Phantom II/ /Phantom II/
    platoon19.oob:98: P19128 62 69 /OTO 105mm L5 Pack Howitzer/ /L5 105mm Howitzer/
    platoon19.oob:99: P19129 62 69 /OTO 105mm L5 Pack Howitzer/ /L5 105mm Howitzer/



    which shows (for Vietnam War) all units available between 1962 & 1972, but not before or after.



    Robert@roberto ~/cslint
    $ ./csoobrpt.pl -g vn -f -l -s/ -m "sherman" -strength -assault -name -short -nick | awk '$2 > $3'
    platoon10.oob:8: 9 8 /M4A3(76) Medium Tanks/ /M4A3(76)/ /Sherman/
    platoon10.oob:9: 9 8 /M4A3(76) Medium Tanks/ /M4A3(76)/ /Sherman/
    platoon10.oob:10: 9 8 /M4A3(76) Medium Tanks/ /M4A3(76)/ /Sherman/
    platoon10.oob:72: 18 9 /POA-CWS-H1 Flamethrower Tanks/ /POA-H1/ /Sherman/
    platoon10.oob:73: 20 9 /POA-CWS-H5 Flamethrower Tanks/ /POA-H5/ /Sherman/
    platoon10.oob:74: 20 9 /POA-CWS-H5 Flamethrower Tanks/ /POA-H5/ /Sherman/



    which shows (for Vietnam War) any Shermans where the strength value exceeds the assault value.

    How about Shermans where the strength value is less than the assault value?



    Robert@roberto ~/cslint
    $ ./csoobrpt.pl -g vn -f -l -s/ -m "sherman" -strength -assault -name -short -nick | awk '$2 < $3'
    Platoon08.oob:18: 7 8 /M4 Medium Tanks/ /M4 Sherman/ /Sherman/
    Platoon08.oob:19: 7 8 /M4 Medium Tanks/ /M4 Sherman/ /Sherman/
    Platoon08.oob:20: 7 8 /M4 Medium Tanks/ /M4 Sherman/ /Sherman/
    platoon00.oob:2: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
    platoon00.oob:3: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
    platoon00.oob:4: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
    platoon03.oob:2: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
    platoon03.oob:3: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
    platoon03.oob:4: 7 8 /M4 Medium Tanks/ /PLA M4 Med. Tanks/ /Sherman/
    platoon10.oob:11: 7 10 /M4A3E8 Medium Tanks/ /M4A3E8/ /Sherman/
    platoon10.oob:12: 7 10 /M4A3E8 Medium Tanks/ /M4A3E8/ /Sherman/
    platoon10.oob:13: 7 10 /M4A3E8 Medium Tanks/ /M4A3E8/ /Sherman/
    platoon10.oob:14: 8 10 /M4E5 Medium Tanks/ /M4E5/ /Sherman/
    platoon10.oob:15: 8 10 /M4E5 Medium Tanks/ /M4E5/ /Sherman/
    platoon10.oob:70: 3 10 /M4A3 Flail Tanks/ /M4A3 Flail/ /Sherman Crab/
    platoon10.oob:71: 3 11 /M4A3 Dozer/ /M4A3 Dozers/ /Sherman Dozer/
    platoon10.oob:251: 0 6 /Maintenance Recovery M74/ /M74 ARV/ /--- based on a Sherman/



    Lots and lots of capability here. csoobrpt.pl is extremely useful for eyeballing and making better sense of the cryptic OOB data.

    Another CSlint utility:

    csflags.pl is a Perl script (program) to convert platoon*.oob flag values to their flag name equivalents, and vice versa.

    What are flag values? These are special OOB (.oob file) numbers that specify various unit attributes and capabilities, for example:



    // Unit Other flags.
    const int XCanDigIn = 0x00000001;
    const int XMineClearing = 0x00000002;
    const int XFireHardAI = 0x00000004;
    const int XFireSoftAI = 0x00000008;
    const int XReconAI = 0x00000010;
    const int XAssaultAI = 0x00000020;
    const int XRetreatAI = 0x00000040;
    const int XCanDamage = 0x00000080;
    const int XCarryAny = 0x00000100;
    const int XIsDiveBomber = 0x00000200;
    const int XPathfinder = 0x00000400;
    const int XAirPlane = 0x00000800;
    const int XHelicopter = 0x00001000;
    const int XRecovery = 0x00004000;
    const int XLayVehBridge = 0x00008000;
    const int XLayFootBridge = 0x00010000;
    const int XSpecOps = 0x00020000;



    By means of a nearly inscrutable and -- for us humans -- error-prone process, we can combine these flag values into a single number, e.g., the carry flag value 131, which signifies



    const int XLoadable = 0x00000001;
    const int XRidable = 0x00000002;
    ...
    const int XPortage = 0x00000080;



    Converting hexadecimal flag values to their decimal number equivalents and back again -- technical mumbo jumbo that we humans are not very good at. So I wrote csflags.pl to help us out.

    Examples:



    Robert@roberto ~/cslint
    $ ./csflags.pl -g me -carry 131
    XLoadable/XRidable/XPortage



    which (for the game Middle East) shows you the meaning of the carryflag value 131.



    Robert@roberto ~/cslint
    $ ./csflags.pl -g wf -move "XTracked/XHalftrack/XNoLtBridge/XNoMedBridge/XNoHvyBridge"
    16485



    which (for the game West Front) shows that the combined moveflags XTracked/XHalftrack/XNoLtBridge/XNoMedBridge/XNoHvyBridge have the value 16485.

    csflags.pl replaces some incomplete and outdated reference documents that, heretofore, modders have used to set and interpret the unit flags.

    And what about the promised .scn (scenario file) checks?

    csscnchk.pl is a Perl script (program) to check the scenario (.scn) files for various errors.



    robert@roberto ~/cslint
    $ ./csscnchk.pl -g vn
    ERROR: DienBienPhu-1953_1.scn:44: id 21269 (tracking -1) not found in DienBienPhu-1953_1.org
    ERROR: Dong_Vinh_Xanh_1965.scn:167: id 10665 not found in Dong_Vinh_Xanh_1965.org
    ERROR: Dong_Vinh_Xanh_1965.scn:329: id 21218 not found in Dong_Vinh_Xanh_1965.org
    ERROR: Dong_Vinh_Xanh_1965.scn:362: id 10310 (tracking -1) not found in Dong_Vinh_Xanh_1965.org
    ERROR: Hiep_Hoa_1963.scn:47: id 21402 (tracking -1) not found in Hiep_Hoa_1963.org
    ERROR: Hiep_Hoa_1963.scn:48: id 21201 (tracking -1) not found in Hiep_Hoa_1963.org
    ERROR: Hiep_Hoa_1963.scn:49: id 21049 (tracking -1) not found in Hiep_Hoa_1963.org
    ERROR: Hiep_Hoa_1963.scn:50: id 21107 (tracking -1) not found in Hiep_Hoa_1963.org
    ERROR: IaDrang-1965.scn:57: id 21209 (tracking -1) not found in IaDrang-1965.org
    ERROR: KheSanh-1968.scn:668: id 10272 (tracking -1) not found in KheSanh-1968.org



    For the game Vietnam War, the output shows:

  • the scenario (.scn) file name
  • the line # where the error occurred
  • the unit id
  • '(tracking -1)', if that unit's tracking value was specified that way
  • the OOB (.org) file name corresponding to the scenario file name

    You can also use csscnchk.pl to check a single file you are currently working on, for example:



    robert@roberto ~/cslint
    $ ./csscnchk.pl -g vn -f Hiep_Hoa_1963.scn
    ERROR: Hiep_Hoa_1963.scn:47: id 21402 (tracking -1) not found in Hiep_Hoa_1963.org
    ERROR: Hiep_Hoa_1963.scn:48: id 21201 (tracking -1) not found in Hiep_Hoa_1963.org
    ERROR: Hiep_Hoa_1963.scn:49: id 21049 (tracking -1) not found in Hiep_Hoa_1963.org
    ERROR: Hiep_Hoa_1963.scn:50: id 21107 (tracking -1) not found in Hiep_Hoa_1963.org



    (Observe that the csscnchk.pl output above confirms previous analysis about the Hiep_Hoa_1963.scn. See http://www.matrixgames.com/forums/fb.asp?m=3341248.

    If you're wondering: No, csscnchk.pl does not use the same command-line code in that earlier example. Those were just proof-of-concept command lines to prove a point. csscnchk.pl is more sophisticated and comprehensive than that.)

    Are there erroneous .scn files for the games EF, WF & RS (and ME)? Yes, but I won't report their errors here. You'll have to use csflags.pl yourselves to find out.

    CSlint is already proving useful for QAing (Quality Assuring) the Campaign Series/Modern Wars game data files. CSlint will be a great benefit for modders -- which is why we anticipate packaging it with all future CS/MW new releases and udpates. (But there's a catch: You'll have to install Cygwin first.)

    Aside from CSlint (and post-codebase merge clean-up), still more stuff...

    See the attached screenshot.




    In the panel

  • to the far left, you see the standard File menu in all current CS/MW games
  • in the middle, you see the File menu in Rising Sun for an interim build of that game
  • to the far right, you see the File menu in the JTS [John Tiller Software] game, The Renaissance

    I hadn't really thought about it, but when I went to look, I was quite surprised to discover that, until now, the Campaign Series of games -- to be more precise, their engines (as opposed to their frontends) -- have lacked the File>Open option (to resume saved games), not to mention the File>New option (to launch new games).

    Why? It took a bit of detective work, but eventually I found this in some very old legacy CS code, code that had been carried over to more recent CS codebases:



    MainMenu MENU {
    POPUP MsgFileM {
    #if defined(_DEBUG)
    MENUITEM MsgNewM, ON_FILENEW
    MENUITEM MsgOpenM, ON_FILEOPEN
    #endif

    MENUITEM MsgSaveM, ON_FILESAVE
    MENUITEM MsgSaveAsM, ON_FILESAVEAS
    MENUITEM SEPARATOR
    MENUITEM MsgReplayM, ON_REPLAY
    MENUITEM SEPARATOR
    MENUITEM MsgExitM, ON_FILEEXIT
    }



    (Also other omissions and restrictions, which I won't detail here.)

    Ah, so it appears that ON_FILENEW & ON_FILEOPEN were for debugging purposes only; that all new games and save game reloads were to be done the "official way", via the game frontend launcher.

    Seeing no reason not to, I have lifted the restriction and have fully implemented both File>New and File>Open:




    You may, in all future CS/MW releases, select new games and load saved games directly from the game engine (the actual game program), bypassing the frontend launcher entirely.

    Taking it easy in The Man Cave? Nyet! Hard at work as ever.

    Until the next time ...

    _____________________________

  • Post #: 1
    RE: Coder Diary #6 -- CSlint - 7/4/2013 7:40:31 PM   
    wings7


    Posts: 4608
    Joined: 8/11/2003
    From: Phoenix, Arizona
    Status: offline
    Excellent!

    (in reply to berto)
    Post #: 2
    RE: Coder Diary #6 -- CSlint - 7/5/2013 2:07:02 PM   
    Crossroads


    Posts: 15151
    Joined: 7/5/2009
    Status: online
    Enjoying your Coder Diary entries Berto!

    _____________________________

    Visit us at: Campaign Series Legion
    ---
    CS: Vietnam | CS: East Front 1939-1941 IN-THE-WORKS
    CS: Middle East 1948-1985 Fully reimaged v2.0 available now!

    (in reply to wings7)
    Post #: 3
    RE: Coder Diary #6 -- CSlint - 7/5/2013 11:16:24 PM   
    XLVIIIPzKorp


    Posts: 220
    Joined: 10/24/2006
    Status: offline
    Yep, me too. Always looking for them.

    (in reply to Crossroads)
    Post #: 4
    RE: Coder Diary #6 -- CSlint - 11/3/2013 12:15:35 AM   
    junk2drive


    Posts: 12907
    Joined: 6/27/2002
    From: Arizona West Coast
    Status: offline
    bump 6

    (in reply to XLVIIIPzKorp)
    Post #: 5
    Page:   [1]
    All Forums >> [Current Games From Matrix.] >> [World War II] >> John Tiller's Campaign Series >> Coder Diary #6 -- CSlint Page: [1]
    Jump to:





    New Messages No New Messages
    Hot Topic w/ New Messages Hot Topic w/o New Messages
    Locked w/ New Messages Locked w/o New Messages
     Post New Thread
     Reply to Message
     Post New Poll
     Submit Vote
     Delete My Own Post
     Delete My Own Thread
     Rate Posts


    Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI

    0.111