MAPGEN (A WITP:AE Map Dumper/Repacker)

Please post here for questions and discussion about scenario design, art and sound modding and the game editor for WITP Admiral's Edition.

Moderators: wdolson, MOD_War-in-the-Pacific-Admirals-Edition

User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Previously seen on the Ice Age AE thread (HERE)

Gents, here is a very early alpha release of MAPGEN (for lack of a better name).

It's not a fully featured set, but it should work on a lot of things.

Current feature set is:

Image

Link to my website for 24~ MB ZIP download

I apologize if the download size is a bit big, but I included the exact Win32 installer binaries for the programming development environment I used to make it to assist with others who may want to work on it; and/or to reduce bug issues from incompatible builds.

Anyway.....

***STANDARD DISCLAIMER***
This SOFTWARE PRODUCT is provided by THE PROVIDER "as is" and "with all faults." THE PROVIDER makes no representations or warranties of any kind concerning the safety, suitability, lack of viruses, inaccuracies, typographical errors, or other harmful components of this SOFTWARE PRODUCT. There are inherent dangers in the use of any software, and you are solely responsible for determining whether this SOFTWARE PRODUCT is compatible with your equipment and other software installed on your equipment. You are also solely responsible for the protection of your equipment and backup of your data, and THE PROVIDER will not be liable for any damages you may suffer in connection with using, modifying, or distributing this SOFTWARE PRODUCT.

****************************************************************

Description of MAPGEN:

Gents, here is a very early alpha release of MAPGEN (for lack of a better name).

##########REALLY BIG NOTE#####################
## REMEMBER TO BACK UP YOUR ##
## PWZONE.DAT ##
## PWZLINK.DAT ##
## PWHEXE.DAT ##
## FILES BEFORE DOING ANYTHING ELSE ##
## TO AVOID A REINSTALL\REPATCH OF WITP:AE! ##
##############################################

It dumps WITP:AE PWHEXE.dat information to a graphics file, where it can be easily edited with
Paint Shop Pro or Photoshop (and seamlessly overlaid over satellite maps etc so that everything
all matches up), and then it repacks the information back into PWHEXE.dat via analyzing certain
pixels of interest on the graphics file.

It's written in Python 2.7 using the PIL image library for image manipulation.

This release currently requires a Python 2.7 with PIL development environment installed
on your computer.

##########REALLY BIG NOTE##################################################
## If you have a virtual machine, you should install Python into that VM
## and execute MAPGEN and Python from the VM, to avoid pooping up your
## main computer OS with various programming language call-outs in the
## registry. It's what I do when I program.
############################################################################

I would have liked to do a repackaged EXE file using a Python to EXE program that's out there,
but there are some bugs involved with PIL and that program.

Long term objectives:

1.) Add more DUMP/REPACK files (see MAPGEN_FEATURES.gif for what's not done etc)

2.) Maybe make it into an executable, so it doesn't require installation of the Python development
environment and PIL on a target computer.

3.) Maybe run it off my website, as it's a python script, meaning it could run on a webserver? But I don't
know about that, as it would require 15 MB of uploaded graphics files to repack a map. :-\

**************FILE FOLDER LAYOUT**********************

\Dev
A primitive early development item for procedural map generation "Drunkard's Walk" is here.
You may disregard this.

\Docs
Some useful documents are here. (AE Map Documentation, some hexfile information and RGB color information).

\Libraries
This includes the Windows-32 installation binaries for the Python development environment I used
to develop MAP GEN. They are:

PIL-1.1.7.win32-py2.7.exe
python-2.7.12.msi

If you don't trust the files in this download, you can download them from the following places:

PIL (http://www.pythonware.com/products/pil/)
http://effbot.org/downloads/PIL-1.1.7.win32-py2.7.exe

Python:
https://www.python.org/ftp/python/2.7.1 ... 2.7.12.msi

\PreDumped
This folder contains already pre-dumped graphics images from the latest
official PWHEXE.DAT.

\Scripts
This contains the programs for dumping and repacking the map(s).

Once you've installed Python 2.7.12 and PIL 1.1.7 you should just click on a Python file to run it:

e.g

MAPGEN_COASTWATCH_JAP_DUMP.py

##################################################
############BIG NOTE##############################

At this current level of development, there's little error protection.

it assumes you have

WITP_AE_Grid_Black_32Bit.tif
WITP_AE_Grid_Blank_32Bit.tif

and

pwhexe.dat

inside your /Scripts folder.

If not, it probably breaks.

##################################################
############BIG NOTE II###########################

Because users are users; I've included a copy of the latest
WITP:AE map files in:

PWHEX_Pristine.zip

in /Scripts. I'm unsure of the legality of this; but it
should help save you from having to reinstall your game
should something mess up.
Attachments
MAPGEN_Features.gif
MAPGEN_Features.gif (11.24 KiB) Viewed 1139 times
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Terrain Hex Type Dumped.

This is a resample down from the original 9900 x 8000 pixel image for web display.

Image

The 9900x8000 pixel size of the dumped graphics files kind of necessitates having a heavy duty
64-bit version of Photoshop to handle it all; but it also means you can cut and paste direct
map graphics images from WITP:AE over the dumped graphics near seamlessly.

Imagevia Imgflip GIF Maker
Attachments
TerrainTypesDUMP.gif
TerrainTypesDUMP.gif (135.99 KiB) Viewed 1134 times
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by Dili »

Wonderful. Ryan i am making a Mediterranean map, this would help me in graphics alone or also would also help me in editing the hexes? i mean changing sea hexes to land hexes etc. From the list you have above it seem the hex terrain is repackable.


Wouldn't this work with Krita? For some stuff i found it faster than Photoshop.
cardas
Posts: 184
Joined: Fri Apr 08, 2016 1:01 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by cardas »

Great work, I will certainly test this out in a bit. I've actually thought about coding up an map editor (along with an improved scenario editor) but unlike you I'm lazy and never gotten to it.

I haven't actually tested it yet, so the following comment are assumptions; If you intend to do a fully functional editing tool then you really need to create either a full interface or more than one bitmap. Doing basic terrain data is fine with a single one, but I at least imagine that it will become an issue if you want to edit all hex information (stacking limit 0-255, nation, Entry zone ID etc.).

Note that currently you indicate that Hexside Straits are repackable, but not Hex Strait Size. But if you have a Hexside Strait then the Hex Strait also needs to be set for the Hexside Strait to actually function! E.g. if hex 100, 100 has Hexside Strait East: 01 (Wide), then hex 100, 100 should also have Hex Strait Size: 01 (Wide).
Actually I'm not sure if Hexside Strait and Hex Strait has to be the same, it might still function if one is 02 (Narrow) and the other is 01 (Wide). I'm certain it won't work if Hex Strait Size is 00 (None) though.

I'm not looking forward to editing such a large bitmap either. I got a much newer laptop that is my "main" computer, however I generally do all my graphic stuff on a 10+ year old desktop with 2 GB RAM (and a CRT screen... yeah). From personal experience I can say that it's possible to edit such a large bitmap regardless if you are patient though [>:]
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Note that currently you indicate that Hexside Straits are repackable, but not Hex Strait Size. But if you have a Hexside Strait then the Hex Strait also needs to be set for the Hexside Strait to actually function!

Hex Side Strait Dumping capability is TBD. I decided to release something now; rather than wait until it's perfect, in order to get feedback.
E.g. if hex 100, 100 has Hexside Strait East: 01 (Wide), then hex 100, 100 should also have Hex Strait Size: 01 (Wide).

This is something i found out while testing hexside rivers. [:'(]
I'm not looking forward to editing such a large bitmap either. I got a much newer laptop that is my "main" computer, however I generally do all my graphic stuff on a 10+ year old desktop with 2 GB RAM (and a CRT screen... yeah). From personal experience I can say that it's possible to edit such a large bitmap regardless if you are patient though [>:]

I think I can help you there though. I just completed a study of the WITP:AE BMP map files.

I think I can "cut up" the huge 9000~ pixel map in much the same way the official in game bitmap images were cut up, for people with older computers like you and then restitch it together;.
Attachments
AE_Graphic..p_Format.zip
(489.89 KiB) Downloaded 18 times
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

I've just also completed tests on Python 3.4.4; which required me to update the image library from PIL to Pillow, a code fork of PIL; which kind of broke some things and will require some re-work to make my code work again -- I just got a limited proof of concept working again in 3.4.4/Pillow.

But the good news. The change to the newer libraries was to get PyInstaller support, which will enable me to build an executable with everything needed bundled into it; so you won't need to install Python or the dependencies onto your computer; which is good. Only downside is a simple program is about 8~ MB.

But I guess this is OK in a world where we can devote 400~ MB to the map images for WITP. [:'(]
cardas
Posts: 184
Joined: Fri Apr 08, 2016 1:01 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by cardas »

I think I can help you there though. I just completed a study of the WITP:AE BMP map files.

I think I can "cut up" the huge 9000~ pixel map in much the same way the official in game bitmap images were cut up, for people with older computers like you and then restitch it together;.
That'd make it much easier to work with. The conceptual idea on how to code that doesn't really seem difficult either (says the lazy one who hasn't even bothered to take a stab at it).

But the good news. The change to the newer libraries was to get PyInstaller support, which will enable me to build an executable with everything needed bundled into it; so you won't need to install Python or the dependencies onto your computer; which is good. Only downside is a simple program is about 8~ MB.
Having Python installed is no issue for me. An standalone executable is still much more user friendly, so good choice.

But I guess this is OK in a world where we can devote 400~ MB to the map images for WITP.
Weeeeell about that... [:'(] No, even the (in computer era terms) fossil I'm talking about has enough space to spare for that.
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by Dili »

Any reason you don't work in new laptop? it would be night and day.
cardas
Posts: 184
Joined: Fri Apr 08, 2016 1:01 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by cardas »

Only Windows machine I got around and I'm not about to install any Windows virtual machine on the laptop. I could of course still do the graphics stuff on the laptop, however I prefer to keep all the WitP stuff on the Windows machine as that's where I'll play the game anyway. Furthermore I often have something else going on on the laptop at the same time.
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Got my code ported from 2.7/PIL to 3.4.4/Pillow and the result is significantly slower than PIL. [:@]

As in 110 seconds to dump an image versus about 10. [:@]

But there is a solution -- I tested two methods by running them 300 times to provide a good baseline

"Draw each pixel individually" (current solution) takes about 1.77 seconds per 300 hexes.

whereas

"Paste a hex image in (Alternate solution)" takes about 0.05 seconds per 300 hexes.

It's about a 35.4x speed difference, or the difference between dumping a map in 276 seconds versus 7.8 seconds.
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by Dili »

Okay cardas.
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

After much cursing, and some avenues that didn't pan out; I managed to get the dump/draw speed back...mostly.

It takes 25~ "in compiler" seconds to dump a map now in Python 3.4, versus about 10~ for Python 2.7; but I can live with a mere 2x computational time increase, as opposed to 12x; especially with the abilities python 3.4 gives me.
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Here's a super alpha of a compiled executable that does one thing:

Dumps Terrain Hex Types.

It's more of a "proof of concept" to see if there are any unforeseen issues.

Link to 9 MB ZIP

Apologies for the size, but it packs a near complete python interpreter into it to let it run on your systems without any "additional dlls".

Anyway; I used it to dump the old "War in the West" mod at:

https://tanciky.cz/uv/witw_english/scre ... ?page=maps

And it looks pretty neat; and you can see how the creator simply pasted it over the existing WITP:AE map.


Image
Attachments
WITW_Extracted.gif
WITW_Extracted.gif (112.22 KiB) Viewed 1130 times
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Super Alpha v2 of the dumper for Terrain Types only:

LINK

8.3~ MB ZIP; no longer needs a pre-compiled map TIFF image; instead, it builds one from scratch, and speed has been vastly improved from 20~ seconds for a map build down to about 3.5~ seconds.
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

3 March 2017 Alpha:

5.8~ MB ZIP LINK

A bit faster plus smaller now; and will dump all hex-types. (Terrain Type, Terrain Hex Type, Malaria zone, etc)

(Hex Sides are TBD refactored).
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by Dili »

Hi Ryan

It shows the Ms-dos windows and then it appears dead. No exe running in task manager.

I have a windows 7, nvidia GPU from 2013.

Edit: i could catch the Cmd Window.


Attachments
Clipboard-1.jpg
Clipboard-1.jpg (48.52 KiB) Viewed 1135 times
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Dili, it's because it's looking for PWHEXE.DAT to dump the map data from, but since it can't find it; it crash/closes.

Copy PWHEXE.dat from your WITP:AE folder to the folder you have MAPGEN in and it should work.

I need to make this a bit more user friendly. :-P
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by Dili »

Thanks, still no go. I have a permission error: Error no13 so does not write the png file.

I tried to fix it in executable properties i had to unblock the exe and i also run it has administrator but the result is the same.
User avatar
RyanCrierie
Posts: 1321
Joined: Fri Oct 14, 2005 7:15 am
Contact:

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by RyanCrierie »

Give me a chance to try this on a W7 laptop I have to see if I can reproduce the error.
Dili
Posts: 4713
Joined: Fri Sep 10, 2004 4:33 pm

RE: MAPGEN (A WITP:AE Map Dumper/Repacker)

Post by Dili »

Thanks.
Post Reply

Return to “Scenario Design and Modding”