MAPGEN (A WITP:AE Map Dumper/Repacker)
Moderators: wdolson, MOD_War-in-the-Pacific-Admirals-Edition
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
MAPGEN (A WITP:AE Map Dumper/Repacker)
Previously seen on the Ice Age AE thread (HERE)
Gents, here is a very early alpha release of MAPGEN (for lack of a better name).
It's not a fully featured set, but it should work on a lot of things.
Current feature set is:
Link to my website for 24~ MB ZIP download
I apologize if the download size is a bit big, but I included the exact Win32 installer binaries for the programming development environment I used to make it to assist with others who may want to work on it; and/or to reduce bug issues from incompatible builds.
Anyway.....
***STANDARD DISCLAIMER***
This SOFTWARE PRODUCT is provided by THE PROVIDER "as is" and "with all faults." THE PROVIDER makes no representations or warranties of any kind concerning the safety, suitability, lack of viruses, inaccuracies, typographical errors, or other harmful components of this SOFTWARE PRODUCT. There are inherent dangers in the use of any software, and you are solely responsible for determining whether this SOFTWARE PRODUCT is compatible with your equipment and other software installed on your equipment. You are also solely responsible for the protection of your equipment and backup of your data, and THE PROVIDER will not be liable for any damages you may suffer in connection with using, modifying, or distributing this SOFTWARE PRODUCT.
****************************************************************
Description of MAPGEN:
Gents, here is a very early alpha release of MAPGEN (for lack of a better name).
##########REALLY BIG NOTE#####################
## REMEMBER TO BACK UP YOUR ##
## PWZONE.DAT ##
## PWZLINK.DAT ##
## PWHEXE.DAT ##
## FILES BEFORE DOING ANYTHING ELSE ##
## TO AVOID A REINSTALL\REPATCH OF WITP:AE! ##
##############################################
It dumps WITP:AE PWHEXE.dat information to a graphics file, where it can be easily edited with
Paint Shop Pro or Photoshop (and seamlessly overlaid over satellite maps etc so that everything
all matches up), and then it repacks the information back into PWHEXE.dat via analyzing certain
pixels of interest on the graphics file.
It's written in Python 2.7 using the PIL image library for image manipulation.
This release currently requires a Python 2.7 with PIL development environment installed
on your computer.
##########REALLY BIG NOTE##################################################
## If you have a virtual machine, you should install Python into that VM
## and execute MAPGEN and Python from the VM, to avoid pooping up your
## main computer OS with various programming language call-outs in the
## registry. It's what I do when I program.
############################################################################
I would have liked to do a repackaged EXE file using a Python to EXE program that's out there,
but there are some bugs involved with PIL and that program.
Long term objectives:
1.) Add more DUMP/REPACK files (see MAPGEN_FEATURES.gif for what's not done etc)
2.) Maybe make it into an executable, so it doesn't require installation of the Python development
environment and PIL on a target computer.
3.) Maybe run it off my website, as it's a python script, meaning it could run on a webserver? But I don't
know about that, as it would require 15 MB of uploaded graphics files to repack a map. :-\
**************FILE FOLDER LAYOUT**********************
\Dev
A primitive early development item for procedural map generation "Drunkard's Walk" is here.
You may disregard this.
\Docs
Some useful documents are here. (AE Map Documentation, some hexfile information and RGB color information).
\Libraries
This includes the Windows-32 installation binaries for the Python development environment I used
to develop MAP GEN. They are:
PIL-1.1.7.win32-py2.7.exe
python-2.7.12.msi
If you don't trust the files in this download, you can download them from the following places:
PIL (http://www.pythonware.com/products/pil/)
http://effbot.org/downloads/PIL-1.1.7.win32-py2.7.exe
Python:
https://www.python.org/ftp/python/2.7.1 ... 2.7.12.msi
\PreDumped
This folder contains already pre-dumped graphics images from the latest
official PWHEXE.DAT.
\Scripts
This contains the programs for dumping and repacking the map(s).
Once you've installed Python 2.7.12 and PIL 1.1.7 you should just click on a Python file to run it:
e.g
MAPGEN_COASTWATCH_JAP_DUMP.py
##################################################
############BIG NOTE##############################
At this current level of development, there's little error protection.
it assumes you have
WITP_AE_Grid_Black_32Bit.tif
WITP_AE_Grid_Blank_32Bit.tif
and
pwhexe.dat
inside your /Scripts folder.
If not, it probably breaks.
##################################################
############BIG NOTE II###########################
Because users are users; I've included a copy of the latest
WITP:AE map files in:
PWHEX_Pristine.zip
in /Scripts. I'm unsure of the legality of this; but it
should help save you from having to reinstall your game
should something mess up.
Gents, here is a very early alpha release of MAPGEN (for lack of a better name).
It's not a fully featured set, but it should work on a lot of things.
Current feature set is:
Link to my website for 24~ MB ZIP download
I apologize if the download size is a bit big, but I included the exact Win32 installer binaries for the programming development environment I used to make it to assist with others who may want to work on it; and/or to reduce bug issues from incompatible builds.
Anyway.....
***STANDARD DISCLAIMER***
This SOFTWARE PRODUCT is provided by THE PROVIDER "as is" and "with all faults." THE PROVIDER makes no representations or warranties of any kind concerning the safety, suitability, lack of viruses, inaccuracies, typographical errors, or other harmful components of this SOFTWARE PRODUCT. There are inherent dangers in the use of any software, and you are solely responsible for determining whether this SOFTWARE PRODUCT is compatible with your equipment and other software installed on your equipment. You are also solely responsible for the protection of your equipment and backup of your data, and THE PROVIDER will not be liable for any damages you may suffer in connection with using, modifying, or distributing this SOFTWARE PRODUCT.
****************************************************************
Description of MAPGEN:
Gents, here is a very early alpha release of MAPGEN (for lack of a better name).
##########REALLY BIG NOTE#####################
## REMEMBER TO BACK UP YOUR ##
## PWZONE.DAT ##
## PWZLINK.DAT ##
## PWHEXE.DAT ##
## FILES BEFORE DOING ANYTHING ELSE ##
## TO AVOID A REINSTALL\REPATCH OF WITP:AE! ##
##############################################
It dumps WITP:AE PWHEXE.dat information to a graphics file, where it can be easily edited with
Paint Shop Pro or Photoshop (and seamlessly overlaid over satellite maps etc so that everything
all matches up), and then it repacks the information back into PWHEXE.dat via analyzing certain
pixels of interest on the graphics file.
It's written in Python 2.7 using the PIL image library for image manipulation.
This release currently requires a Python 2.7 with PIL development environment installed
on your computer.
##########REALLY BIG NOTE##################################################
## If you have a virtual machine, you should install Python into that VM
## and execute MAPGEN and Python from the VM, to avoid pooping up your
## main computer OS with various programming language call-outs in the
## registry. It's what I do when I program.
############################################################################
I would have liked to do a repackaged EXE file using a Python to EXE program that's out there,
but there are some bugs involved with PIL and that program.
Long term objectives:
1.) Add more DUMP/REPACK files (see MAPGEN_FEATURES.gif for what's not done etc)
2.) Maybe make it into an executable, so it doesn't require installation of the Python development
environment and PIL on a target computer.
3.) Maybe run it off my website, as it's a python script, meaning it could run on a webserver? But I don't
know about that, as it would require 15 MB of uploaded graphics files to repack a map. :-\
**************FILE FOLDER LAYOUT**********************
\Dev
A primitive early development item for procedural map generation "Drunkard's Walk" is here.
You may disregard this.
\Docs
Some useful documents are here. (AE Map Documentation, some hexfile information and RGB color information).
\Libraries
This includes the Windows-32 installation binaries for the Python development environment I used
to develop MAP GEN. They are:
PIL-1.1.7.win32-py2.7.exe
python-2.7.12.msi
If you don't trust the files in this download, you can download them from the following places:
PIL (http://www.pythonware.com/products/pil/)
http://effbot.org/downloads/PIL-1.1.7.win32-py2.7.exe
Python:
https://www.python.org/ftp/python/2.7.1 ... 2.7.12.msi
\PreDumped
This folder contains already pre-dumped graphics images from the latest
official PWHEXE.DAT.
\Scripts
This contains the programs for dumping and repacking the map(s).
Once you've installed Python 2.7.12 and PIL 1.1.7 you should just click on a Python file to run it:
e.g
MAPGEN_COASTWATCH_JAP_DUMP.py
##################################################
############BIG NOTE##############################
At this current level of development, there's little error protection.
it assumes you have
WITP_AE_Grid_Black_32Bit.tif
WITP_AE_Grid_Blank_32Bit.tif
and
pwhexe.dat
inside your /Scripts folder.
If not, it probably breaks.
##################################################
############BIG NOTE II###########################
Because users are users; I've included a copy of the latest
WITP:AE map files in:
PWHEX_Pristine.zip
in /Scripts. I'm unsure of the legality of this; but it
should help save you from having to reinstall your game
should something mess up.
- Attachments
-
- MAPGEN_Features.gif (11.24 KiB) Viewed 1139 times
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Terrain Hex Type Dumped.
This is a resample down from the original 9900 x 8000 pixel image for web display.
The 9900x8000 pixel size of the dumped graphics files kind of necessitates having a heavy duty
64-bit version of Photoshop to handle it all; but it also means you can cut and paste direct
map graphics images from WITP:AE over the dumped graphics near seamlessly.
via Imgflip GIF Maker
This is a resample down from the original 9900 x 8000 pixel image for web display.
The 9900x8000 pixel size of the dumped graphics files kind of necessitates having a heavy duty
64-bit version of Photoshop to handle it all; but it also means you can cut and paste direct
map graphics images from WITP:AE over the dumped graphics near seamlessly.
via Imgflip GIF Maker
- Attachments
-
- TerrainTypesDUMP.gif (135.99 KiB) Viewed 1134 times
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Wonderful. Ryan i am making a Mediterranean map, this would help me in graphics alone or also would also help me in editing the hexes? i mean changing sea hexes to land hexes etc. From the list you have above it seem the hex terrain is repackable.
Wouldn't this work with Krita? For some stuff i found it faster than Photoshop.
Wouldn't this work with Krita? For some stuff i found it faster than Photoshop.
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Great work, I will certainly test this out in a bit. I've actually thought about coding up an map editor (along with an improved scenario editor) but unlike you I'm lazy and never gotten to it.
I haven't actually tested it yet, so the following comment are assumptions; If you intend to do a fully functional editing tool then you really need to create either a full interface or more than one bitmap. Doing basic terrain data is fine with a single one, but I at least imagine that it will become an issue if you want to edit all hex information (stacking limit 0-255, nation, Entry zone ID etc.).
Note that currently you indicate that Hexside Straits are repackable, but not Hex Strait Size. But if you have a Hexside Strait then the Hex Strait also needs to be set for the Hexside Strait to actually function! E.g. if hex 100, 100 has Hexside Strait East: 01 (Wide), then hex 100, 100 should also have Hex Strait Size: 01 (Wide).
Actually I'm not sure if Hexside Strait and Hex Strait has to be the same, it might still function if one is 02 (Narrow) and the other is 01 (Wide). I'm certain it won't work if Hex Strait Size is 00 (None) though.
I'm not looking forward to editing such a large bitmap either. I got a much newer laptop that is my "main" computer, however I generally do all my graphic stuff on a 10+ year old desktop with 2 GB RAM (and a CRT screen... yeah). From personal experience I can say that it's possible to edit such a large bitmap regardless if you are patient though [>:]
I haven't actually tested it yet, so the following comment are assumptions; If you intend to do a fully functional editing tool then you really need to create either a full interface or more than one bitmap. Doing basic terrain data is fine with a single one, but I at least imagine that it will become an issue if you want to edit all hex information (stacking limit 0-255, nation, Entry zone ID etc.).
Note that currently you indicate that Hexside Straits are repackable, but not Hex Strait Size. But if you have a Hexside Strait then the Hex Strait also needs to be set for the Hexside Strait to actually function! E.g. if hex 100, 100 has Hexside Strait East: 01 (Wide), then hex 100, 100 should also have Hex Strait Size: 01 (Wide).
Actually I'm not sure if Hexside Strait and Hex Strait has to be the same, it might still function if one is 02 (Narrow) and the other is 01 (Wide). I'm certain it won't work if Hex Strait Size is 00 (None) though.
I'm not looking forward to editing such a large bitmap either. I got a much newer laptop that is my "main" computer, however I generally do all my graphic stuff on a 10+ year old desktop with 2 GB RAM (and a CRT screen... yeah). From personal experience I can say that it's possible to edit such a large bitmap regardless if you are patient though [>:]
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Note that currently you indicate that Hexside Straits are repackable, but not Hex Strait Size. But if you have a Hexside Strait then the Hex Strait also needs to be set for the Hexside Strait to actually function!
Hex Side Strait Dumping capability is TBD. I decided to release something now; rather than wait until it's perfect, in order to get feedback.
E.g. if hex 100, 100 has Hexside Strait East: 01 (Wide), then hex 100, 100 should also have Hex Strait Size: 01 (Wide).
This is something i found out while testing hexside rivers. [:'(]
I'm not looking forward to editing such a large bitmap either. I got a much newer laptop that is my "main" computer, however I generally do all my graphic stuff on a 10+ year old desktop with 2 GB RAM (and a CRT screen... yeah). From personal experience I can say that it's possible to edit such a large bitmap regardless if you are patient though [>:]
I think I can help you there though. I just completed a study of the WITP:AE BMP map files.
I think I can "cut up" the huge 9000~ pixel map in much the same way the official in game bitmap images were cut up, for people with older computers like you and then restitch it together;.
- Attachments
-
- AE_Graphic..p_Format.zip
- (489.89 KiB) Downloaded 18 times
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
I've just also completed tests on Python 3.4.4; which required me to update the image library from PIL to Pillow, a code fork of PIL; which kind of broke some things and will require some re-work to make my code work again -- I just got a limited proof of concept working again in 3.4.4/Pillow.
But the good news. The change to the newer libraries was to get PyInstaller support, which will enable me to build an executable with everything needed bundled into it; so you won't need to install Python or the dependencies onto your computer; which is good. Only downside is a simple program is about 8~ MB.
But I guess this is OK in a world where we can devote 400~ MB to the map images for WITP. [:'(]
But the good news. The change to the newer libraries was to get PyInstaller support, which will enable me to build an executable with everything needed bundled into it; so you won't need to install Python or the dependencies onto your computer; which is good. Only downside is a simple program is about 8~ MB.
But I guess this is OK in a world where we can devote 400~ MB to the map images for WITP. [:'(]
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
That'd make it much easier to work with. The conceptual idea on how to code that doesn't really seem difficult either (says the lazy one who hasn't even bothered to take a stab at it).I think I can help you there though. I just completed a study of the WITP:AE BMP map files.
I think I can "cut up" the huge 9000~ pixel map in much the same way the official in game bitmap images were cut up, for people with older computers like you and then restitch it together;.
Having Python installed is no issue for me. An standalone executable is still much more user friendly, so good choice.But the good news. The change to the newer libraries was to get PyInstaller support, which will enable me to build an executable with everything needed bundled into it; so you won't need to install Python or the dependencies onto your computer; which is good. Only downside is a simple program is about 8~ MB.
Weeeeell about that... [:'(] No, even the (in computer era terms) fossil I'm talking about has enough space to spare for that.But I guess this is OK in a world where we can devote 400~ MB to the map images for WITP.
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Any reason you don't work in new laptop? it would be night and day.
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Only Windows machine I got around and I'm not about to install any Windows virtual machine on the laptop. I could of course still do the graphics stuff on the laptop, however I prefer to keep all the WitP stuff on the Windows machine as that's where I'll play the game anyway. Furthermore I often have something else going on on the laptop at the same time.
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Got my code ported from 2.7/PIL to 3.4.4/Pillow and the result is significantly slower than PIL. [:@]
As in 110 seconds to dump an image versus about 10. [:@]
But there is a solution -- I tested two methods by running them 300 times to provide a good baseline
"Draw each pixel individually" (current solution) takes about 1.77 seconds per 300 hexes.
whereas
"Paste a hex image in (Alternate solution)" takes about 0.05 seconds per 300 hexes.
It's about a 35.4x speed difference, or the difference between dumping a map in 276 seconds versus 7.8 seconds.
As in 110 seconds to dump an image versus about 10. [:@]
But there is a solution -- I tested two methods by running them 300 times to provide a good baseline
"Draw each pixel individually" (current solution) takes about 1.77 seconds per 300 hexes.
whereas
"Paste a hex image in (Alternate solution)" takes about 0.05 seconds per 300 hexes.
It's about a 35.4x speed difference, or the difference between dumping a map in 276 seconds versus 7.8 seconds.
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Okay cardas.
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
After much cursing, and some avenues that didn't pan out; I managed to get the dump/draw speed back...mostly.
It takes 25~ "in compiler" seconds to dump a map now in Python 3.4, versus about 10~ for Python 2.7; but I can live with a mere 2x computational time increase, as opposed to 12x; especially with the abilities python 3.4 gives me.
It takes 25~ "in compiler" seconds to dump a map now in Python 3.4, versus about 10~ for Python 2.7; but I can live with a mere 2x computational time increase, as opposed to 12x; especially with the abilities python 3.4 gives me.
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Here's a super alpha of a compiled executable that does one thing:
Dumps Terrain Hex Types.
It's more of a "proof of concept" to see if there are any unforeseen issues.
Link to 9 MB ZIP
Apologies for the size, but it packs a near complete python interpreter into it to let it run on your systems without any "additional dlls".
Anyway; I used it to dump the old "War in the West" mod at:
https://tanciky.cz/uv/witw_english/scre ... ?page=maps
And it looks pretty neat; and you can see how the creator simply pasted it over the existing WITP:AE map.
Dumps Terrain Hex Types.
It's more of a "proof of concept" to see if there are any unforeseen issues.
Link to 9 MB ZIP
Apologies for the size, but it packs a near complete python interpreter into it to let it run on your systems without any "additional dlls".
Anyway; I used it to dump the old "War in the West" mod at:
https://tanciky.cz/uv/witw_english/scre ... ?page=maps
And it looks pretty neat; and you can see how the creator simply pasted it over the existing WITP:AE map.
- Attachments
-
- WITW_Extracted.gif (112.22 KiB) Viewed 1130 times
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Super Alpha v2 of the dumper for Terrain Types only:
LINK
8.3~ MB ZIP; no longer needs a pre-compiled map TIFF image; instead, it builds one from scratch, and speed has been vastly improved from 20~ seconds for a map build down to about 3.5~ seconds.
LINK
8.3~ MB ZIP; no longer needs a pre-compiled map TIFF image; instead, it builds one from scratch, and speed has been vastly improved from 20~ seconds for a map build down to about 3.5~ seconds.
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
3 March 2017 Alpha:
5.8~ MB ZIP LINK
A bit faster plus smaller now; and will dump all hex-types. (Terrain Type, Terrain Hex Type, Malaria zone, etc)
(Hex Sides are TBD refactored).
5.8~ MB ZIP LINK
A bit faster plus smaller now; and will dump all hex-types. (Terrain Type, Terrain Hex Type, Malaria zone, etc)
(Hex Sides are TBD refactored).
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Hi Ryan
It shows the Ms-dos windows and then it appears dead. No exe running in task manager.
I have a windows 7, nvidia GPU from 2013.
Edit: i could catch the Cmd Window.
It shows the Ms-dos windows and then it appears dead. No exe running in task manager.
I have a windows 7, nvidia GPU from 2013.
Edit: i could catch the Cmd Window.
- Attachments
-
- Clipboard-1.jpg (48.52 KiB) Viewed 1135 times
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Dili, it's because it's looking for PWHEXE.DAT to dump the map data from, but since it can't find it; it crash/closes.
Copy PWHEXE.dat from your WITP:AE folder to the folder you have MAPGEN in and it should work.
I need to make this a bit more user friendly.
Copy PWHEXE.dat from your WITP:AE folder to the folder you have MAPGEN in and it should work.
I need to make this a bit more user friendly.
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Thanks, still no go. I have a permission error: Error no13 so does not write the png file.
I tried to fix it in executable properties i had to unblock the exe and i also run it has administrator but the result is the same.
I tried to fix it in executable properties i had to unblock the exe and i also run it has administrator but the result is the same.
- RyanCrierie
- Posts: 1321
- Joined: Fri Oct 14, 2005 7:15 am
- Contact:
RE: MAPGEN (A WITP:AE Map Dumper/Repacker)
Give me a chance to try this on a W7 laptop I have to see if I can reproduce the error.