How do you edit the PWHEX file?

Please post here for questions and discussion about scenario design, art and sound modding and the game editor for WITP Admiral's Edition.

Moderators: wdolson, MOD_War-in-the-Pacific-Admirals-Edition

User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: How do you edit the PWHEX file?

Post by RyanCrierie »

Basically how it works is simple. It analyzes your map file, and then outputs them in a graphical file format (multiple PNG files), so that all you have to do is use a program like photoshop (or the later versions of paint shop pro which are 64 bit) is "paint" what you want to edit; then save the graphical image.

Then a complimentary program analyzes that graphic image and alters the hex file accordingly.
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: How do you edit the PWHEX file?

Post by Dili »

You paint hexes and hexsides in a certain colors -explained in manual what each color means- to say to the repacker program the modified information it should inject in pwhexe.dat file.

Example:
So you want to modify Tokyo Terrain Type from Heavy Urban to Forest- you replace the heavy urban color which is Red (RGB 255,0,0) to Forest which is Green(RGB 0,255,0). Then you save the image in DUMP directory and use MAPGEN_REPACK.exe to inject that new information into pwhexe.dat from the image you painted just by replacing the color.

It can be seen in Terrain Type map i posted the red color of London,Berlin and others which says to repacker program those hexes should be Heavy Urban.

It is basically changing the pwhexe.dat information by painting/changing color in hexes and hexsides. There are several maps. A map for Malaria, another for Weather, etc...

Another paint application good for this is the freeware Krita. https://krita.org/en/
el cid again
Posts: 16980
Joined: Mon Oct 10, 2005 4:40 pm

RE: How do you edit the PWHEX file?

Post by el cid again »

Now I have refreshed my memory on pwhexe.dat editing

The newest form of the pwhexe.dat editor is only needed for modern
laptops or monitors with unusually wide screens - by historical standards.
It (version 8) permits moving to see anything in the editor. Version 7
permits one to see everything at once - but only on a screen more square
than the newest types are.
el cid again
Posts: 16980
Joined: Mon Oct 10, 2005 4:40 pm

RE: How do you edit the PWHEX file?

Post by el cid again »

Irfan View is powerful, and free. It permits very detail editing of almost any form of graphics imaginable.

ORIGINAL: RyanCrierie

Basically how it works is simple. It analyzes your map file, and then outputs them in a graphical file format (multiple PNG files), so that all you have to do is use a program like photoshop (or the later versions of paint shop pro which are 64 bit) is "paint" what you want to edit; then save the graphical image.

Then a complimentary program analyzes that graphic image and alters the hex file accordingly.
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: How do you edit the PWHEX file?

Post by Dili »

Can you paint with Irfanview? From its webpage you can't, it is an useful viewer/batcher like xnview and others. Cid are you sure you understood how Mapgen works?

First Mapgen unpacks the map in PNG graphic files maps - each of these maps have a function: weather,terrain, strait, road, rail... these graphic file have an overlay of hex and hexsides for the user to paint on with colors that the Mapgen repack can interpret and inject in pwhexe.dat

See the Via Balbia in Libya above? can you paint that in irfanview?

el cid again
Posts: 16980
Joined: Mon Oct 10, 2005 4:40 pm

RE: How do you edit the PWHEX file?

Post by el cid again »

Irfanview is not a paint program. But I "paint" using it using various tools. You can copy and paste at the
pixel level. You can draw and fill. There are probably other ways. It is a very sophisticated program
AND its author supports you via email in English (never mind he is German) for free. There are books on it -
but only in Polish and German last I knew.

I never heard of Mapgen - so I do not understand it. I also manage vast numbers of files and only make map art
when needed - in the original game style - because no one else is supporting me in this area. Originally I teamed
with an artist. When he died, I teamed with a different one. He is now too sick to draw. An offer to do the map
system was started but never completed. So I use "temporary" maps just modifying what is in stock art - to show
critical features not in other mods (like sea ice cover, the locations of roads and railroads, and colored letter
codes to clue players about something special in a hex). If I tried to do an entire map system, it would take
years - but I might one day. I am getting better. Note I HAVE modified the strategic map to show RHS entry-exit
points and I DO use a lot of third party art, some of which affects maps to a degree.

Since I am not trying to do entire maps from scratch, I don't have to paint much. I find the ability to put what
I want at the pixel level is powerful enough for almost anything. I even create entirely new airplane art by
modifying some similar aircraft at the pixel level. I had to do that when we wanted some types that no one else
has done. I have only one type I am unable to draw - it is very unusual in shape - a rather fine late war
Japanese transport - Ki-97. I use a historical charcoal drawing - the only extant image - for the side. I did
create an acceptable top.

I do not see Via Balboa although I see Libya. Sounds like "Balboa Road" - but I don't see roads. But I can
and do draw roads - and did today.


ORIGINAL: Dili

Can you paint with Irfanview? From its webpage you can't, it is an useful viewer/batcher like xnview and others. Cid are you sure you understood how Mapgen works?

First Mapgen unpacks the map in PNG graphic files maps - each of these maps have a function: weather,terrain, strait, road, rail... these graphic file have an overlay of hex and hexsides for the user to paint on with colors that the Mapgen repack can interpret and inject in pwhexe.dat

See the Via Balbia in Libya above? can you paint that in irfanview?

Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: How do you edit the PWHEX file?

Post by Dili »

With MAPGEN we do not paint the map, we do paint the map code.


Via Balbia in post #18 See those brownish painted hexsides ? It means major road.


el cid again
Posts: 16980
Joined: Mon Oct 10, 2005 4:40 pm

Obscure codes in the electronic map

Post by el cid again »

I look at AE as a system. Originally an electronics technician and
machine language programmer, and a pioneer of early digital computers
(even pre-electronic digital computers, which used relays), I think in
terms of how the program works more than in terms of art. Most of my
time involved with maps is down at the level of the codes which define
the "real map" from the computer's point of view. Art is simply what
players see and has essentially nothing to do with how the program works.

Reworking some pwhexe.dat files to fold in improvements designed for
RHS Level II, I discovered I needed to look at weather zone, country code
and sub-map numbers in an area where these were not defined by stock.
Essentially RHS Level I is based on stock maps and ignores these codes
for "off map areas." But RHS Level II uses Andrew Brown's "extended map"
art - and needs to have corresponding pwhexe.dat files for three rows of
hexes. Unlike Andrew's pwhexe.dat files - mine make every visible hex
in his map art functional.

Because AE was not properly documented by its original programmers, there
are probably functions for these codes that we do not know about. In any
case, it is good professional practice to define every field as it should
be in the context of the design. This means that every function using a field
will work as well as possible (to the extent the code functions as designed),
and that any future code seeking to exploit that field data will also work as
intended.

Because of these codes, and many others, I am skeptical about the idea of using
the AE engine with non-AE map systems. However, to the extent it can work, it
is probably a good idea to think about the values in these (and other fields).
In the context of modifying an AE map of some kind, it is certainly a good idea.
Do not ignore these codes, particularly in the original "off map area" where their
definitions will not work for a new on map area.
el cid again
Posts: 16980
Joined: Mon Oct 10, 2005 4:40 pm

Reply to Dili:

Post by el cid again »

Dili: The problem is that code in scripts needs to be neutralized for a non WITP game.
Otherwise all sorts of weird effects will happen. Probably a "dummy, neutral" script
can be made using the editor. For a non-Pacific game, I think this is a minimal
requirement. Better would be an entirely new script FOR THE GAME you design. Doing
that well might take a couple of years of work.

Sid



It might be more generally useful if there was a generator for scripts. If one
wants a new game (not WITP) - one should kill the script and generate a new one.
The existing ones are sufficiently "dumb" (AI is a misleading name) that we might
be better off if we could create new ones from scratch.

Changing what the map looks like is purely cosmetic and does not change the basic
functioning of the game. When I alter a map, it is for game function reasons, not
cosmetic reasons. The only reason I care about map art is to help players know what
is in a given place?

If automatic code generators can be adapted, is there a way to make new scripts easier
than by hand?

ORIGINAL: Dili

With MAPGEN we do not paint the map, we do paint the map code.


Via Balbia in post #18 See those brownish painted hexsides ? It means major road.


Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: How do you edit the PWHEX file?

Post by Dili »

El cid read my words, this has nothing to do with map art. MAPGEN converts the map CODE into colors, we paint those colors as we like to change the CODE and MAPGEN repacks it.

I don't know about your question, maybe a specific example of an idea you have in mind.

However, to the extent it can work, it
is probably a good idea to think about the values in these (and other fields).
In the context of modifying an AE map of some kind, it is certainly a good idea.

yes that is why i am using the more common map zones as well, i made it on propose that Tokyo hex 114/60 is a coastal Italian city that is an important naval base.
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: Obscure codes in the electronic map

Post by RyanCrierie »

ORIGINAL: el cid again

I look at AE as a system. Originally an electronics technician and
machine language programmer, and a pioneer of early digital computers
(even pre-electronic digital computers, which used relays), I think in
terms of how the program works more than in terms of art. Most of my
time involved with maps is down at the level of the codes which define
the "real map" from the computer's point of view. Art is simply what
players see and has essentially nothing to do with how the program works.

I'll make this clear(er).

MAPGEN is a program designed to read that underlying binary data contained in pwhexe.dat and then render it graphically.

This is some code from MAPGEN's map dumping program:

ReadByte = [HexFileData.read(1)] # Read One Byte from file as a character.

if ReadByte == [b'\x00']: #x00 Deep Ocean
MapImage.paste(Hex_Blue_1, (int(PIXLOC_X_MODIFIER),int(PIXLOC_Y_MODIFIER)), Hex_AlphaChannel)
if ReadByte == [b'\x01']: #x01 Clear
MapImage.paste(Hex_Green_4, (int(PIXLOC_X_MODIFIER),int(PIXLOC_Y_MODIFIER)), Hex_AlphaChannel)
if ReadByte == [b'\x02']: #x02 Jungle
MapImage.paste(Hex_Green_2, (int(PIXLOC_X_MODIFIER),int(PIXLOC_Y_MODIFIER)), Hex_AlphaChannel)
if ReadByte == [b'\x03']: #x03 Mountain
MapImage.paste(Hex_Brown_1, (int(PIXLOC_X_MODIFIER),int(PIXLOC_Y_MODIFIER)), Hex_AlphaChannel)


Basically, this snippet reads one byte from pwhexe.dat; and it then does different things:

If the byte has a hexadecimal value of x00 (aka deep ocean); it pastes the "BLUE 1" hex.

If it's got a hexadecimal value of x02 (Jungle), it pastes "GREEN 2" hex.

The repacker does this in reverse. It looks at the graphical image and converts the GREEN 2 color into hexadecimal value of x02 and puts that into the binary file.
el cid again
Posts: 16980
Joined: Mon Oct 10, 2005 4:40 pm

Mapgen

Post by el cid again »

How does this work with respect to roads and railroads?

How does this work with respect to coastal terrain. In
such terrain, pwhexe.dat codes whatever the LAND is - say
woods. But there is also WATER in the hex. And the water
might be light blue (shallow) or dark blue (deep).

Since RHS needs different maps for every season, this might
be a solution. Someone in Europe said he would make such a
program, and said it would be fun and easy - but he gave up.
But in principle, it should be possible.

Where does one find this program?
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: Mapgen

Post by Dili »

Mapgen extracts one map for railroads hexes, other map for roads hexes, other map for weather, etc...
How does this work with respect to coastal terrain. In
such terrain, pwhexe.dat codes whatever the LAND is - say
woods. But there is also WATER in the hex. And the water
might be light blue (shallow) or dark blue (deep).

You paint the LAND,SEA type in called by Mapgen:TERRAIN_BIG map

You paint the coastal instead in HEX_TYPE_BIG map.
Image

It defines the coastal hexes by painting blue clear ones.

Go to Mapgen topic and download the manual.
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: Mapgen

Post by Dili »

This is the rail map. It is obviously separated from road map. To CODE paint the roads you need another map.

Image
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: Mapgen

Post by RyanCrierie »

Here's how you edit in MAPGEN (i'm about to release block II).

This is a railnetwork for australia.

You can see how I've crudely drawn across it with photoshop (or paint shop pro)

Image
Attachments
RailNetworkExample.gif
RailNetworkExample.gif (58 KiB) Viewed 192 times
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: Mapgen

Post by RyanCrierie »

Same location, but in game via hotkeys following analysis/repacking of the graphical file:

Image
Attachments
RailNetworkExample2.gif
RailNetworkExample2.gif (175.64 KiB) Viewed 192 times
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: Mapgen

Post by RyanCrierie »

Same area displayed in Photoshop, with two layers:

1.) Actual in-game artwork that you see

2.) The rail data layer extracted into graphical format via MAPGEN.

I highly recommend the use of a 64-bit graphics editing program (newer version of Photoshop or Paint Shop Pro whatever they call PSP now) because it takes 218 MB of memory for each “layer” of detail in the map if you edit the full 9000x7000+ pixel map, and that adds up fast.

Image
Attachments
PhotoshopExample.gif
PhotoshopExample.gif (280.25 KiB) Viewed 192 times
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: Mapgen

Post by RyanCrierie »

MAPGEN Block II 19.6 MB ZIP

Designed to be unzipped to your WITP:AE main directory and run from there.

Source code is included in /SOURCE, documentation is in /DOCS

Files generated will be placed into /DUMP.

The program will read the files from /DUMP when rebuilding PWHEXE.dat

I've included a zipped up copy of the original PWHEXE.dat from the latest WITPAE because you won't back it up.

MAPGEN DUMP: Dumps the PWHEXE.dat file into different graphical layers.

MAPGEN REPACK: Repacks the graphical files back into the PWHEXE.DAT file.

MAPGEN MAPART: Combines the 41(!) BMP files in the \ART directory for the in-game map into one really big PNG file which is also pixel accurate with the dump graphics files that DUMP generates. Also splits the big png files back into 41 BMP files when you're done editing the big file in photoshop.

LANGUAGE USED: Python 3.x; using the following libraries:

Colorama 0.3.7
Pillow 4.0.0

and packaged into executables by PyInstaller 3.2.1 (that's why each file is so big, it includes an entire python distribution instead of just what the program needs to run, unlike most programming langauges that build executables)
RichardAckermann
Posts: 270
Joined: Fri Dec 04, 2015 12:07 pm

RE: Mapgen

Post by RichardAckermann »

I need help with the PWhexe data format, and would require an expert to give me some info on it.
Like:

struct HexDataType
{
INT8 Terraintype
INT16 NE hex roadtype
INT16 etc..
};

And how many sets of this data exist for each line of hexes in the game map.

I need the info to be able to import WITP AE scenarios into my WITP AE 1 1/2 game engine.
dwbradley
Posts: 197
Joined: Sun Mar 21, 2004 1:17 am

RE: Mapgen

Post by dwbradley »

ORIGINAL: RichardAckermann

I need help with the PWhexe data format, and would require an expert to give me some info on it.
Like:

struct HexDataType
{
INT8 Terraintype
INT16 NE hex roadtype
INT16 etc..
};

And how many sets of this data exist for each line of hexes in the game map.

I need the info to be able to import WITP AE scenarios into my WITP AE 1 1/2 game engine.

The info in this package may be of use to you.
https://drive.google.com/file/d/1ip5AGA ... sp=sharing

Cheers
Dave Bradley
Post Reply

Return to “Scenario Design and Modding”