The lag and choppiness in lategame - holy **** !

[Timotheus]

So I finally got to late game.

I have a somewhat decent rig, 8GB RAM, AMD 8350 8 Core Processor, 4GHz

64 bit OS.

Windows 8.


Late game, 700 stars, the game is unplayable for me.

Yes, everybody, I have read and implemented the "solutions" such as they are - nebulae off, framerate unchecked and put a big fat "1" in it, civilian ships not showing, etc etc.

It is a slideshow.

Looking at my cores, my one core is at 100% utility, while the other ones are barely used at all.

Since I do love this game [8D], is there some fix I can do to continue playing my late game as I have a very interesting situation for my empire.

Right now, though, this is unplayable - extremely choppy, a slideshow, FPS of perhaps 1 per second... or less.

And this despite the fact that in Task Manager I am running this process with REALTIME, highest priority.

Any help would be appreciated.

[PsyKoSnake]

Reduce the speed of the game.

[Timotheus]

True, I did to 0.5x.

Still choppy though... plus, the game SHOULD be playable on normal speed, IMHO.

I wonder what folks experience is for late game DW. Do you run it at 0.5x speed? Does your rig have no problems?

Is there (hopefully) some tweak I can do in file editing to make it less of a problem?

[Buio]

Turn off "Show Empire Influence". Make sure to zoom out so you don't show icons for civilian ships. For example use Delete hotkey, then Page Up x2 (fast taps).

But yeah, the game is still very choppy late game despite efforts to improve it throughout 3 expansions and a lot of patches.

[Osito]

It's a feature not a bug. It's because you want to play with 700+ stars and the inevitable number of planets and moons and what-have-you. It's why pretty much every 4x4 space game out there seriously limits the number of star systems and planets you can have in your game. And it's why the ones who don't do that lag like f*****y. It's why you play CIV 5 on a huge map and spend 'minutes' on each turn towards the end of the game. It's why I never seem to play most strategy games beyond a certain point, unless I've reeled in the options. On the other hand, you learn to accept it, because what we have now is so much more than what we used to have. There is a certain irony here: I remember playing CIV1 and CIV2 virtually lag-free when they first came out, and I loved those games, but I wanted so much more. Now I have so much more, but I also have the lag. Is there any truly modern large scale strategy 4x4 available that is (1) a good game and (2) doesn't lag middle-to-end game? If there is, let me know, as I want to play it.

Edit: Ok, before anyone says it, yes there are games that don't lag, like CKII and EUIV (not for me anyway). So I was talking BS.

[CyclopsSlayer]

Hmmm...
My system is Win7 64bit sp1. Intel i7 3770 8core (3.4GHz) 16GB ram. So your system should be similar.

Do you have it set to auto-distribute the loads across your cores? On mine Core 0 is at about 60%, Cores 2/4/6 at 20% and Core 1 at 15%. Cores 5/7 are listed as in reserve. 5.6GB Ram used.

51 years into my current game, 1400 stars, 15x15 sector, all ships displayed, 19 other base empires, the Shakturi active, 58 colonies in 50 systems, 380 Military ships, 186 State units, 1626 Private sector, still playing on 1x speed and FRAPs is showing me still at 30 (flickers between 26 and 33) FPS @ 1920x1080.

[Flinkebeinchen]

@Timotheus
What graphic card do you have?

I have no problems with my system running late games 1400 stars. Win 7 64, Intel 2500k 4x 3.3GHz, 8GB RAM, ATI HD 6870.

Or maybe Distant Worlds just doesn't like Win8? Hmmmm [X(]

[Haree78]

My system isn't as good as yours and I don't have that kind of problem with 1400 stars.

[Canute0]

I don't think it's a question of the graphic card, since DW isn't a 3d game and not so hungry about graphical power.

DW is more a database game, positions of hundret of ships and planets need to be updated each tick. So it's more a questions about CPU,Motherboard, RAM and harddrive.
But since your systems looks like an upperclass one, you shouldn't have a problem.
DW isn't optimized for multicore, like the other 95% of the windows programs, so it is normal that just one core is used mainly.
You checked how the useage was spread over the cores, but did you look at the taskmanager if DW was the main one who use all the power ?

[Buio]

ORIGINAL: Haree78
My system isn't as good as yours and I don't have that kind of problem with 1400 stars.

What date are you playing? I don't believe that DW runs better on older hardware. Number of stars doesn't matter, it's how many colonies and ships you and all opponents have together.

I've tried it on several computers, from laptop to powerful desktop, with Windows 7 and Windows 8, and the late game slowness is apparent on all.

[Icemania]

Timotheus, agree with Buio, where I've had the biggest problems is games with lots of independents and colonies. If they are relatively scarce the late game slows but is playable. That said, I haven't been past the 60 year mark in a long time ...

[scotten_usa]

ORIGINAL: CyclopsSlayer
Do you have it set to auto-distribute the loads across your cores? On mine Core 0 is at about 60%, Cores 2/4/6 at 20% and Core 1 at 15%. Cores 5/7 are listed as in reserve. 5.6GB Ram used.

How do you tell Windows to auto-distribute CPU load across multiple cores?

[Canute0]

Open the task manager, then select a process then you can select what core the process should use (it is below the priority, i use german can't say how it is called at engl.)

But only a few applications works better just with one core, most "modern" ones works better when you let windows assign them.

[CyclopsSlayer]

ORIGINAL: ScottenChi

ORIGINAL: CyclopsSlayer
Do you have it set to auto-distribute the loads across your cores? On mine Core 0 is at about 60%, Cores 2/4/6 at 20% and Core 1 at 15%. Cores 5/7 are listed as in reserve. 5.6GB Ram used.

How do you tell Windows to auto-distribute CPU load across multiple cores?

Under Win7/Vista and I would assume Win8 it should be automatic. Under XP it just packs the first available core unless told otherwise. Core i, and AMD Bulldozers actually use that for power saving as they unload one of the physical cores and then will turn it off to save power.

[Kayoz]

ORIGINAL: Canute
DW is more a database game

WTF?!?!? Any game that uses a database to handle the storage of elements actively used in the game will run unbelievably slow.

ORIGINAL: Canute
So it's more a questions about CPU,Motherboard, RAM and harddrive.

No, it's about the efficiency of the code. If you write sloppy, inefficient code, your program will be as efficient and speedy as Congress. If you do things like skip range engagement checking by manually computing distances instead of using collision mechanics in the graphics card - you're wasting CPU time doing work that the GPU could do more efficiently and quickly.

I can't even imagine what you mean by the HDD or motherboard. If you're touching the HDD regularly in a game, you might as well chop your own head off as far as game design is concerned. How the motherboard factors into this is really beyond me. Are you suggesting that Elliot should write hardware-specific optimized code ... in C#?

ORIGINAL: Canute
DW isn't optimized for multicore, like the other 95% of the windows programs, so it is normal that just one core is used mainly.

It's called threading. Not optimization. There's some overlap between the two, but DW's speed problem is far less it's optimization (ie: the efficiency of the code) than it's single-threaded design foundation. Elliot could only make so many speed gains by optimizing his code - whereas if DW's architecture were built from the start with multi-threading in mind, the comparable speed improvement would be far more dramatic. I rather suspect, from the rare interviews he's had, that DW's life and success far outstripped anything he imagined at the beginning - when his decisions were based on a much smaller and much more personal project.

That said, it's possible that Elliot is doing a load of floating point math - but I can't imagine he'd do that.

[Spidey]

WTF?!?!? Any game that uses a database to handle the storage of elements actively used in the game will run unbelievably slow.

A database is an organized collection of data which means that any system that keeps order in arrays with data can be considered a database. You don't need to implement support for some SQL-variant for your data structure to be an improvised database.

Anyway, I'm pretty sure Canute is simply saying that looking up data, updating it, and storing it is a much heavier task in DW than the display of graphics. There's no physics engine, there's no advanced shadows, no light ray projection fluff, no reflective surfaces, and in fact no detailed surfaces whatsoever. There are, however, quite easily tens of thousands of ships moving to and from. And so it becomes a "database game".

No, it's about the efficiency of the code. If you write sloppy, inefficient code, your program will be as efficient and speedy as Congress. If you do things like skip range engagement checking by manually computing distances instead of using collision mechanics in the graphics card - you're wasting CPU time doing work that the GPU could do more efficiently and quickly.

Indeed. But before you start suggesting things like using CUDA cores for some of the calculations, please do remember that some of us are playing on legacy hardware. HDD usage comes into play for the same reasons, doesn't it? My system has 3 gb of ram. Could all the game data as well as all the temporary calculations fit into that along with Windows and the bunch of other things that runs on computers today? It would be a stretch, wouldn't it? Now consider that the official minimum for DW is what, 1 gb? How do you fit DW into that without keeping a bunch of things on the HDD and loading it as necessary? Or would you argue that anyone without 8+ gb ram and a GPU that eats more power than my microwave shouldn't play a game at all?

It's called threading. Not optimization. There's some overlap between the two, but DW's speed problem is far less it's optimization (ie: the efficiency of the code) than it's single-threaded design foundation.

Is it really fair to blame someone for saying "optimize for multicore" on a game board, considering how often the term is used? And it's my understanding that you optimize for multiple cores by threading when possible, so isn't he essentially saying the same thing you're saying, even if he's less specific about it?

Let me add, as an aside, that in my discipline the word "optimization" is quite meaningless unless you specify a goal. I'm an economist so "optimize" can be in terms of simplicity, ease of understanding, accuracy, speed, cost, or some objective function. In a computer science setting, I can imagine you might need to optimize for power usage, low CPU usage, low memory usage, speed, compatibility with legacy systemsm, or some combination.

I'm not a computer scientist so I don't know if CS terminology is radically different, but to me "optimize" simply means "achieve better performance in terms of some factor". This is to say, "optimize" doesn't have to be in terms of efficiency, particularly not when you explicitly specify some other factor.

[Erik Rutins]

ORIGINAL: Timotheus
So I finally got to late game.
I have a somewhat decent rig, 8GB RAM, AMD 8350 8 Core Processor, 4GHz
64 bit OS.
Windows 8.

Which version of Distant Worlds are you running?

Regards,

- Erik

[Kayoz]

ORIGINAL: Spidey
There are, however, quite easily tens of thousands of ships moving to and from. And so it becomes a "database game".

With that definition of "database game", any game using a large quantity of numbers qualifies. I could accept "glorified spreadsheet game" - such as MOO3 has been described as. But every database I know of requires the information to be written to the HDD - that's where it's stored as a requirement. A spreadsheet not so.

If you really want to quibble about semantics, then I ask that you find a -SINGLE- computer game that doesn't deal with numbers, their storage and manipulation - since that's the only requirement for the "database game" moniker. Tic-Tac-Toe would easily qualify under your "database game" description.

ORIGINAL: Spidey
HDD usage comes into play for the same reasons, doesn't it?

When you're saving your game, would you expect any different? Yes, some HDD access is inevitable.

I'm not sure - depends on how Elliot wrote it. But it's in .Net, so a lot of HDD access might be going on without his control. Again, the HDD usage question depends on what you're referring to. Without being more specific, I doubt even Elliot could venture a guess.

ORIGINAL: Spidey
My system has 3 gb of ram. Could all the game data as well as all the temporary calculations fit into that along with Windows and the bunch of other things that runs on computers today?

In short - yes it could fit in 3GB. A lot of data can be abstracted - particularly where the player isn't watching. You can rebuild the system when the player changes his view so he's actually looking there as necessary by using the seed(s) used to generate it. Similar things throughout the game will cut down on the memory footprint.

But you mentioned "other things"...

How long is a piece of string? If all the "other things" consists of is notepad - then yes. If you're running Maya and working with a large complex model - then probably not. Your question makes as much sense as "can I buy a car?". Are you talking about a 1985 Honda Civic, or a 2014 Ferrari? If you're a Wall Street banker pulling in 7 figures - you might not be able to buy one if all your cash is in the Cayman Islands and/or you're in debt up to your eyeballs.

Like I said - how long is a piece of string?

ORIGINAL: Spidey
Now consider that the official minimum for DW is what, 1 gb? How do you fit DW into that without keeping a bunch of things on the HDD and loading it as necessary?

Turn off all the graphics options. Play in a minimum sized galaxy with one opponent. It's the minimum and will likely run quite happily. Remember, they wrote minimum specifications. You can run it - that's all they guarantee. I rather doubt Erik would agree that the minimum specifications applies to a huge galaxy, maxed out opponents, pirates everywhere and all the bells and whistles on.

ORIGINAL: Spidey
Or would you argue that anyone without 8+ gb ram and a GPU that eats more power than my microwave shouldn't play a game at all?

Power consumption depends on a lot of things. Your GPU is the least of your concerns because DW makes very little use of the GPU. There are numerous threads on DW's processor load distribution (near none) and GPU utilization (diddly).

That said, I'm quite confident my box draws significantly less power than some microwaves.

ORIGINAL: Spidey
Is it really fair to blame someone for saying "optimize for multicore" on a game board, considering how often the term is used? And it's my understanding that you optimize for multiple cores by threading when possible, so isn't he essentially saying the same thing you're saying, even if he's less specific about it?

1. Did I blame him? Did I attack him? I did no such thing. I find it interesting that you're implying some sort of attack from me when I stated nothing impugning his statement. My statement was merely correcting him on his misunderstanding of the technology and terminology. I was trying to educate him. I believe anyone familiar with my posts will agree that if I wanted to flame him for his statement, I would do so explicitly. I'm not exactly known for being diplomatic or my use of kind words in confrontations.
2. The term "optimize" is quite common. Do a search of the forums. You'll find it's used hundreds of times.
3. No, we're not saying the same thing. I can't say with any confidence what he was trying to communicate, as he clearly doesn't understand the issues or technology that he's talking about. But what I am confident of is that he's not on the same page as me. Not even the same book.

ORIGINAL: Spidey
Let me add, as an aside, that in my discipline the word "optimization" is quite meaningless unless you specify a goal.

The OP specified the problem as frame-rate and responsiveness of the game. The goal is quite clear.

ORIGINAL: Spidey
In a computer science setting, I can imagine you might need to optimize for power usage, low CPU usage, low memory usage, speed, compatibility with legacy systemsm, or some combination.

I've never heard of optimization being used to describe legacy support.

But yes, there are different parameters by which you can optimize. Not that such has anything to do with the OP's issue - which is performance.

ORIGINAL: Spidey
"optimize" doesn't have to be in terms of efficiency, particularly not when you explicitly specify some other factor.

The parameters for discussion of optimization is quite clear in the OP - performance. I don't understand how you can contend that the OP was unclear. Why are you trying to cloud the issue when the OP's issues are crystal clear?