Matrix Games Forums

Forums  Register  Login  Photo Gallery  Member List  Search  Calendars  FAQ 

My Profile  Inbox  Address Book  My Subscription  My Forums  Log Out

Coder Diary #24 -- UID Slots Expansion

 
View related threads: (in this forum | in all forums)

Logged in as: Guest
Users viewing this topic: none
  Printable Version
All Forums >> [Current Games From Matrix.] >> [World War II] >> John Tiller's Campaign Series >> Coder Diary #24 -- UID Slots Expansion Page: [1]
Login
Message << Older Topic   Newer Topic >>
Coder Diary #24 -- UID Slots Expansion - 11/3/2014 10:13:49 AM   
berto


Posts: 19699
Joined: 3/13/2002
From: metro Chicago, Illinois, USA
Status: offline

Coder Diary #24 -- UID Slots Expansion, Part 1


Another very long Coder Diary. But you might want to skim through it and stick around until the very end...

Yet another foundational issue:

In the Campaign Series, units are identified by ID number, or UID. For example, in West Front, in the German platoon01.oob file, we have this



P01018 4 7 13 40 0 0 46 24 45 2 4 1 11 0 1 766 9 P01018 7 33 16 1 0 15 43 7 53 12 21 8 7 PzKpfw V Medium Tanks , Pz V , Panther



where for the PzKpfw V Medium Tank, its UID# is 01018.

Let's break the UID# down. In

01018

  • the leading 01 (green digits) signifies the nation ID#, in this case the German nation ID# 01
  • the middle 0 (blue digit) signifies the unit type, in this case the Tanks, Light Tanks, Self-propelled Gun category 0
  • the trailing 18 (red digits) signifies that this is the 18th unit of this type, for this nation

    Let's take a look at the (revised) Platoon Codes.

    quote:

    ORIGINAL: Jason Petho

    Here is the latest Platoon coding for NEW ARMIES that are for the Campaign Series.

    This is a work in progress!

    PLATOON CODES
    Jason Petho -- February 4, 2009

    PXX001 – PXX099 Tanks, Light Tanks, Self-propelled Gun (as SU-100)
    PXX101 – PXX199 Artillery (Self-propelled and towed)
    PXX201 – PXX299 Infantry (All types)
    PXX301 – PXX399 Headquarters (All types)
    PXX401 – PXX499 Leaders (All types)
    PXX501 – PXX599 Off-map Airplanes (All types)
    PXX601 – PXX619 Reconnaissance Vehicles (Armoured Cars, Recce Jeeps, etc)
    PXX620 – PXX669 Helicopters and ON MAP aircraft (All types)
    PXX670 – PXX699 Naval Units (Boats, Landing Craft, etc)
    PXX701 – PXX799 Transport units (All types of Trucks, APC’s, IFV’s, etc)
    PXX801 – PXX879 Antiaircraft (Self-propelled, towed, man portable)
    PXX880 – PXX899 Misc Buildings, special units (Factories, News Crews, etc)
    PXX901 – PXX999 Antitank (Self-propelled, towed, man portable)

    As you can see, for most unit categories, we are limited to at most 100 units per category, since available UIDs for each nation, each category range 0-99 only.

    Are 100 units per category enough? Seems not, as Jason placed this concern near the top of his Middle East wish list. How so?

    In the game Middle East, let's see how many units there are, for each nation, in the first (Tanks, Light Tanks, Self-propelled Gun) category:



    [root@berto middle_east]# ls *[Pp]latoon*.oob
    Platoon08.oob Platoon22.oob Platoon25.oob Platoon28.oob
    Platoon09.oob Platoon23.oob Platoon26.oob Platoon29.oob
    Platoon14.oob Platoon24.oob Platoon27.oob Platoon30.oob

    [root@berto middle_east]# for n in 08 09 14 22 23 24 25 26 27 28 29 30; do echo $n; awk '{print $1}' [Pp]latoon$n.oob | egrep "P$n[0]" | sort | uniq | wc -l; echo; done
    08
    97

    09
    26

    14
    63

    22
    67

    23
    47

    24
    33

    25
    28

    26
    38

    27
    11

    28
    64

    29
    17

    30
    23



    Hmm, for nation 08, France, there are 97 different Tanks etc. units. Nearing the 100 limit!

    For nation 22, Israel, it's not so bad -- 67 Tanks etc. units. We've still got some slack there.

    What about infantry?

    PXX201 – PXX299 Infantry (All types)

    In Middle East, let's check their unit counts:



    [root@berto middle_east]# for n in 08 09 14 22 23 24 25 26 27 28 29 30; do echo $n; awk '{print $1}' [Pp]latoon$n.oob | egrep "P$n[2]" | sort | uniq | wc -l; echo; done
    08
    95

    09
    68

    14
    47

    22
    89

    23
    77

    24
    56

    25
    56

    26
    65

    27
    44

    28
    49

    29
    49

    30
    53



    Hmm. Israel now has 89 different Infantry units, nearing the 100 maximum. And France? (France does play a limited role in the game Middle East.) France has 95 different Infantry units. Nearly running out of free slots!

    Let's take a quick look at East Front...



    [root@berto East Front]# for n in 00 01 02 03 04 05 06 07 08 09 10 13 17 24 26 30 37 39 40; do echo $n; awk '{print $1}' platoon$n.oob | egrep "P$n[0]" | sort | uniq | wc -l; echo; done
    00
    94

    01
    99

    ...

    09
    98

    10
    89

    ...

    24
    94

    ...

    40
    96



    For unit type Tanks etc., category 0, for nations Russia (00), Germany (01), Britain (09), the U.S. (10), Soviet (24), and Allied Germany (40), unit counts are near 90 or above. Nearly running out of free slots here too!

    In fact, for Germany (01), we are effectively at the limit. You want to add some more Tank units to the German OOB? Sorry, no can do! Unless you retire some other units, thereby freeing up their UID slots. Which will likely break scenarios having those retired/substituted units.

    In East Front, for the Infantry category, the situation is no better. For several nations, we are near the maximum number of Infantry units allowed. Same story for West Front and Rising Sun.

    When this game was first designed, it was likely thought: 100 units per category? That's more than enough.

    Except who could have foreseen that, nearly 20 years and 1,200+ scenarios later, this game series would still be going strong? And that 100 units per category is now a tight and worrisome constraint?

    What to do?

    What if we take the easy way out and simply steal unit slots from one category to another? For example, maybe borrow some free slots from

    PXX670 – PXX699 Naval Units (Boats, Landing Craft, etc)

    and devote them to Tanks, or Infantry, or wherever we have the need.

    Bad idea. Category codes aren't just for show; the game engine employs them to differentiate among units for different purposes (tanks vs. infantry vs. leaders vs. amphibians vs...). So we can't just reapply UIDs from one category to another willy nilly.

    quote:

    Let's break the UID# down. In

    01018

  • the leading 01 (green digits) signifies the nation ID#, in this case the German nation ID# 01
  • the middle 0 (blue digit) signifies the unit type, in this case the Tanks, Light Tanks, Self-propelled Gun category 0
  • the trailing 18 (red digits) signifies that this is the 18th unit of this type, for this nation

  • What if instead we increase the number of slot digits from 2 to 3, so that for example

    01018

    becomes

    010018

    That is, by prepending an extra digit to the current slot values, we increase the available slots from 0-99, 100 slots, to 0-999, 1000 slots.

    Hey, great idea! Let's get to work converting all the many .oob and .org and .scn and ... thousands of unit .bmp files (that have to be renamed).



    OMG! Many thousands upon thousands of UIDs in hundreds and hundreds of data files (all games) that we have to tweak by hand. Thousands of file renamings too. Shudder at the amount of work that entails! Imagine all the errors that such a mistake-prone manual editing operation would introduce.

    Hopeless, yes?

    [continued in next post...]

    < Message edited by berto -- 11/3/2014 2:00:12 PM >


    _____________________________

  • Post #: 1
    RE: Coder Diary #24 -- UID Slots Expansion - 11/3/2014 10:15:12 AM   
    berto


    Posts: 19699
    Joined: 3/13/2002
    From: metro Chicago, Illinois, USA
    Status: offline

    [... continued from previous post]


    Coder Diary #24 -- UID Slots Expansion, Part 2


    No! Not with "a bit" of programming know-how. With the right programming, and some cleverness, we can automate this gargantuan task. Witness:

    quote:

    ORIGINAL: berto

    Following are instructions on how to convert your Middle East installation to the new UID expanded slots (000-999) scheme.

    ...

    Open a Cygwin terminal (at least one).

    Change directory to the copy of your middle_east directory:



    $ cd "/cygdrive/c/Temp/Middle East/middle_east"



    ...

    In your middle_east directory, list the *.pl (Perl) scripts you will use to do the conversions:



    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ ls -l *.pl
    -rwx------+ 1 Robert None 677 Oct 1 02:44 fixpuid.pl
    -rwx------+ 1 Robert None 2802 Oct 3 14:48 fixsuid.pl
    -rwx------+ 1 Robert None 687 Oct 2 14:11 mkfixfnm.pl
    -rwx------+ 1 Robert None 800 Oct 2 14:14 mkfixuid.pl



    Run the mkfixfnm.pl script, to make the program that will do the actual filename fixing:



    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ ./mkfixfnm.pl

    now run: $ ./fixfnm



    As you can see, when finished, the make script tells you to run the indicated made program.

    If you prepend 'time' to that command, when finished, it will tell you how long it took for the program to run (in my case 3 mins., 18 secs.). The 'time' is optional.



    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ time ./fixfnm

    real 3m18.385s
    user 0m22.901s
    sys 1m38.915s



    Similarly, now run the script to make the program that will do the UID conversions:



    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ ./mkfixuid.pl

    now run: $ ./fixuid



    Again, when finished, it tells you what to do next: run the fixuid script as indicated (prepending 'time' if you wish):



    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ time ./fixuid

    real 0m28.798s
    user 0m5.690s
    sys 0m13.503s



    Now do some checks to see that the file renamings and UID conversions indeed all worked. (Feel free to copy and paste my commands posted here into your own Cygwin terminal.)



    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ ls -l U*.bmp | head
    -rwx------+ 1 Robert None 931944 Oct 22 1998 Unpaved3d.bmp
    -rwx------+ 1 Robert None 233796 Oct 22 1998 Unpaved4d.bmp
    -rwx------+ 1 Robert None 59856 Oct 22 1998 Unpaved5d.bmp
    -rwx------+ 1 Robert None 16908 Jun 9 1998 Up080001.bmp
    -rwx------+ 1 Robert None 18360 Sep 10 1998 Up080002.bmp
    -rwx------+ 1 Robert None 20662 Feb 14 2009 Up080003.bmp
    -rwx------+ 1 Robert None 12310 Sep 1 09:05 Up080004.bmp
    -rwx------+ 1 Robert None 11920 Sep 10 1998 Up080005.bmp
    -rwx------+ 1 Robert None 33750 Mar 25 2008 Up080006.bmp
    -rwx------+ 1 Robert None 51958 Nov 23 2008 Up080007.bmp



    You will see that all of the Up*.bmp files are of the form: Up######.bmp. That is, 6 digits in the file name, where before there were only five.

    Then, check the beginnings ('head -n 20') of various data files. You should verify that in every displayed case, unit IDs are prepended with the letter 'P' (new to the .scn & .scl files), and that every unit ID is also of the form P###### (6 digits):



    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ head -n 20 "Operation Yiftach.scn"
    17
    Operation Yiftach
    0 0 0 1-20[n]
    0 0 0 0 1-20[1]
    0 0 10 4 70 65 60 75
    0
    50 100 300 400 0
    100 70 0 0
    1RGTCVKQPa;KHVCEJoOCR
    1RGTCVKQPa;KHVCEJoQTI
    3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
    3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
    1 5 15 0
    0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 1 1 1 1 1 1 0 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1

    0 11 6 P222003 412 1 1 6 100 5 0 0 3 3 3 3 0 -1 -1
    0 12 3 P262037 463 8 8 6 100 6 0 0 3 3 3 3 0 -1 -1
    2 12 3 16
    5 12 3 25 26
    0 13 6 P222002 411 1 1 6 100 5 0 0 3 3 3 3 0 -1 -1

    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ head -n 20 "Operation Yiftach.org"
    1
    479
    5 48
    B2612201 429 4 4th Infantry Battalion
    P264003 456 6 Al-Mowaled
    P263007 430 4 4th Inf Battalion HQ
    C2602201 431 4 1st Rifle Company
    P262001 432 4 1st Platoon
    P262001 433 4 2nd Platoon
    P262001 434 4 3rd Platoon
    P261005 435 4 Mortars
    P262047 436 4 HMG Squad
    End
    C2602201 437 4 2nd Rifle Company
    P262001 438 4 1st Platoon
    P262001 439 4 2nd Platoon
    P262001 440 4 3rd Platoon
    P261005 441 4 Mortars
    P262047 442 4 HMG Squad
    End

    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ head -n 20 Platoon22.oob
    [Israel]
    P220064 2 5 5 40 0 0 56 26 28 2 4 1 11 0 1 6 5 P220001 5 33 0 1 0 15 48 1 51 12 7 4 4 Cromwell III, Cromwell III ,
    P220001 2 5 5 40 0 0 56 26 28 2 4 1 11 0 1 6 5 P220001 5 33 0 1 0 15 48 1 51 12 7 4 4 Cromwell IV, Cromwell IV ,
    P220002 3 4 6 50 0 0 28 9 13 1 2 1 10 0 1 6 0 P220002 4 33 0 1 0 15 48 1 51 12 6 3 3 Hotchkiss H-39 , H-39 ,
    P220003 1 4 6 50 0 0 28 9 13 1 2 1 10 0 1 6 0 P220002 4 33 0 1 0 15 48 1 51 12 6 3 3 Hotchkiss H-39 , H-39 ,
    P220004 3 4 6 50 0 0 20 13 10 1 2 1 10 0 1 6 3 P220004 3 33 0 1 0 15 48 1 51 12 6 4 4 R35 Light Tanks , R35 ,
    P220005 1 4 6 50 0 0 20 13 10 1 2 1 10 0 1 6 3 P220004 3 33 0 1 0 15 48 1 51 12 6 4 4 R35 Light Tanks , R35 ,
    P220006 3 7 8 40 0 0 38 15 31 3 4 1 7 0 1 6 1 P220006 6 33 16 1 0 15 48 1 51 12 9 5 4 Sherman Krupp , Sherman Krupp ,
    P220007 2 7 8 40 0 0 38 15 31 3 4 1 7 0 1 6 1 P220006 6 33 16 1 0 15 48 1 51 12 9 5 4 Sherman Krupp , Sherman Krupp ,
    P220065 5 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220008 6 33 16 1 0 15 48 1 60 12 9 5 4 Sherman M3 , Sherman M3 , Alef
    P220008 4 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220008 6 33 16 1 0 15 48 1 60 12 9 5 4 Sherman M3 , Sherman M3 , Alef
    P220009 3 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220008 6 33 16 1 0 15 48 1 60 12 9 5 4 Sherman M3 , Sherman M3 , Alef
    P220010 2 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220008 6 33 16 1 0 15 48 1 60 12 9 5 4 Sherman M3 , Sherman M3 , Alef
    P220011 1 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220008 6 33 16 1 0 15 48 1 60 12 9 5 4 Sherman M3 , Sherman M3 , Alef
    P220066 5 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220012 5 33 16 1 0 15 50 1 69 12 9 5 4 Sherman M1 , Sherman M1 ,
    P220012 4 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220012 5 33 16 1 0 15 50 1 69 12 9 5 4 Sherman M1 , Sherman M1 ,
    P220013 3 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220012 5 33 16 1 0 15 50 1 69 12 9 5 4 Sherman M1 , Sherman M1 ,
    P220014 2 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220012 5 33 16 1 0 15 50 1 69 12 9 5 4 Sherman M1 , Sherman M1 ,
    P220015 1 7 8 35 0 0 38 15 31 3 4 1 11 0 1 6 2 P220012 5 33 16 1 0 15 50 1 69 12 9 5 4 Sherman M1 , Sherman M1 ,
    P220016 4 7 8 35 0 0 40 19 32 3 4 1 11 0 1 6 2 P220016 5 33 16 1 0 15 51 1 75 12 9 5 4 Super Sherman M1 , Super Sherman M1 ,

    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ head -n 20 weapon.pdt
    P080001 9 7 5
    10 5 3 2 1 1
    P080002 9 7 5
    10 4 3 2 1 1
    P080003 18 17 16 14 12 10 8 6
    12 12 11 11 10 10 9 9 8 8 7 6
    P080004 2
    10 8 6 4
    P080005 8 5 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
    12 12 12 12 11 11 11 11 10 10 10 10 9 9 9 9 8 8 8 8 8 7 7 7 7 7 7 6 6 6 6 6 6
    P080006 2
    9 7 5 3
    P080007 2
    9 7 5 3
    P080008 9 7 5
    10 4 3 2 1 1
    P080009 5 4 3
    5 4 3 2 1
    P080010 2
    5 2

    Robert@roberto /cygdrive/c/Temp/Middle East/middle_east
    $ head -n 20 unittext.txt
    P080001
    The M8 Armored Car was a U.S. armored car used by the French. Formed into sections of three vehicles, the M8 was used by the French armored reconnaissance battalions. The M8 was an excellent armored car. It was fast, had a good range of operation, and it was mechanically reliable.

    ARMAMENT
    1 x 37mm gun
    1 x 7.62mm machinegun
    1 x 12.7mm machinegun
    MAX ARMOR
    19mm
    WEIGHT
    7.9 tonnes


    P080002
    The M5A1 Light Tank was a U.S. light tank used by the French. Developed from the successful M3 design, the M5A1 was, by all accounts, an excellent light tank. Combing speed, mechanical reliability and an automatic hydraulic transmission, the M5 performed well in whatever role it was assigned. The French received 230 of these tanks via lend-lease.

    ARMAMENT
    1 x 37mm gun
    3 x 7.62mm machineguns
    MAX ARMOR




    ...

    I tried to make this as turn-key, push-button easy as possible. Here's how easy it was for Jason:

    quote:

    ORIGINAL: Jason Petho

    Tada!




    Does it work? Yes! With just a few simple commands (forgetting about the not so simple coding I had to do first, in both writing the Perl conversion scripts, not to mention adapting the game engine code to the revised scheme), in just ~10 minutes, we were able to convert all the many thousands of UIDs in the many dozens of Middle East data files, also automatedly rename thousands of .bmp files.

    In actual game play, there are no reported errors or glitches. It all seems to work!

    Of course we can, in due time, apply this same technique to up the available UID slots from 0-99 to 0-999 for all other games in the Series, including East Front, West Front, Rising Sun, and beyond. Requiring less than half an hour for each game!

    By upping the unit category unit count from 100 to 1000, for new unit creation, the sky's the limit!

    This whole UID slots expansion business is just one more example of the many structural, foundational issues we have had to address in modernizing the Campaign Series of games. Again, who could have foreseen way back when that, twenty years later, there would be so many units, so many new nations, and new games in the Series?

    So we devote the needed time and effort to fixing the creaking foundations and plumbing and wiring etc. in this old house, er, game series.

    Tedious, boring stuff for the most part. Certainly not glamorous. Not very much fun...

    Oh, before I end this, here's another little something we've been working on. Here, from Middle East, are actual in-game screenshots, actual size:





    Until the next time...

    < Message edited by berto -- 11/3/2014 11:54:29 AM >


    _____________________________


    (in reply to berto)
    Post #: 2
    RE: Coder Diary #24 -- UID Slots Expansion - 11/3/2014 12:09:55 PM   
    wings7


    Posts: 4608
    Joined: 8/11/2003
    From: Phoenix, Arizona
    Status: offline
    Thanks and keep up the good work!

    Patrick

    _____________________________

    Please come and join and befriend me at the great Steam portal! There are quite a few Matrix/Slitherine players on Steam! My member page: http://steamcommunity.com/profiles/76561197988402427

    (in reply to berto)
    Post #: 3
    RE: Coder Diary #24 -- UID Slots Expansion - 11/3/2014 5:33:05 PM   
    Crossroads


    Posts: 16208
    Joined: 7/5/2009
    Status: online

    quote:

    ORIGINAL: berto
    When this game was first designed, it was likely thought: 100 units per category? That's more than enough.

    Except who could have foreseen that, nearly 20 years and 1,200+ scenarios later, this game series would still be going strong? And that 100 units per category is now a tight and worrisome constraint?


    Well here's hoping, in 20 years time, this game will still be going strong, and some one is writing the same about 1000 unit types not being enough


    _____________________________

    Visit us at: Campaign Series Legion
    ---
    CS: Vietnam | CS: East Front 1939-1941 IN-THE-WORKS
    CS: Middle East 1948-1985 Fully reimaged v2.0 available now!

    (in reply to berto)
    Post #: 4
    Page:   [1]
    All Forums >> [Current Games From Matrix.] >> [World War II] >> John Tiller's Campaign Series >> Coder Diary #24 -- UID Slots Expansion Page: [1]
    Jump to:





    New Messages No New Messages
    Hot Topic w/ New Messages Hot Topic w/o New Messages
    Locked w/ New Messages Locked w/o New Messages
     Post New Thread
     Reply to Message
     Post New Poll
     Submit Vote
     Delete My Own Post
     Delete My Own Thread
     Rate Posts


    Forum Software © ASPPlayground.NET Advanced Edition 2.4.5 ANSI

    0.129