Latest Posts

Topic: Unlimited resources

Junke1990

Topic Opener
Joined: 2009-02-22, 17:57
Posts: 10
Ranking
Pry about Widelands
Posted at: 2011-06-21, 18:27

I'm running on Ubuntu 64 bit with a intel D530 (dual @ 1.8GHz) and 8 GB of ram and I'm having trouble too. I'm having huge framedrops when playing a map for a few hours. When I point the view at a space where I don't have any building the FPS will go up a lot, sometimes by 2 to 3 times the FPS of when viewing my buildings and characters.


Top Quote
Astuur
Avatar
Joined: 2009-02-28, 09:08
Posts: 733
Ranking
One Elder of Players
Location: Frankfurt / Germany
Posted at: 2011-06-21, 18:28

Thanks SirVer for the detailed answers.

I have tried to close as many programs as possible and many services to free RAM; (6 GB are installed) but about 900 MB are taken by varous things in the background.
Then I have tried to load the last but one save from the 512x512 game.
It worked, but WL takes another 924 MB and (after2 min so not shown) 12,5 % CPU power ( 8 cores, so that is 1 core totally)

Interresting for you?


Being no programmer, I apologize for all my suggestions that imply undue workload and for other misjudgements due to lack of expertise or relevant skills.
I am on Win32, have no means to compile, and rely on prefabricated distributions (Thanks to Tino).

Top Quote
SirVer

Joined: 2009-02-19, 14:18
Posts: 1445
Ranking
One Elder of Players
Location: Germany - Munich
Posted at: 2011-06-22, 09:27

Yes, it shows that indeed the CPU is the bottle neck and not the memory. The process should be able to address at least another GB or even more on 64bit. Is this with !OpenGL rendering enabled?


Top Quote
martin

Joined: 2011-01-13, 13:21
Posts: 65
Ranking
Likes to be here
Posted at: 2011-06-22, 09:36

Wow! 6GB and yet a slide show? And why do your background tasks take 900MB? That's more than twice my RAM (minus the shared part). However, I cannot complain. Widelands runs fluently on my laptop - at least for the small maps and short games (I never tried a 512x512 map). Until now, I thought widelands' main memory usage would be due to the graphics (keeping all animation frames of all objects with 4 Bytes per Pixel). But it seems that there is another large memory consumer on the way that seems to need even more. So can you additionaly please tell something about

  • the time this game is running (total time as well as the time from loading to slideshow),
  • the tribe you are playing,
  • quantity, type and AI of the other tribes,
  • the size of your territory on the map,
  • estimation of the number of objects on the map (incl. buildings, roads, flags, people, wares, trees, animals, etc.) - don't count exactly, just estimate face-wink.png
  • your screen resolution

Btw: interesting picture. I guess the plateau in the CPU usage graph could be the loading process - would be strange if it was different. I think at least for loading/preparing a game, more than one CPU core could be used. Is there any graph for the HDD usage?


Top Quote
Astuur
Avatar
Joined: 2009-02-28, 09:08
Posts: 733
Ranking
One Elder of Players
Location: Frankfurt / Germany
Posted at: 2011-06-23, 14:43

You want to make me work, Martin, huh? face-smile.png
Well, all right: First of all - this is from Win XP SP3 (32) OS - so it's not 6GB (that's the machine) but max 3.
@ SirVer - no Open Gl was not enabled (tried meanwhile, doesn't s change a thing for performance)
Please look here (http://wl.widelands.org/forum/post/5180). Some of your questions are answered there.

And why do your background tasks take 900MB?

That's my usual setup. I can free some more, but not much AFAIK, though I don't know if I could kill a few services more: Here

Tribe is Empire (Playing red), as you can see.

the time this game is running (total time as well as the time from loading to slideshow),

This is the time the game has been running -- it takes about a minute, maybe less, until it starts to get real jerky.

Tribe is Empire using red.

quantity, type and AI of the other tribes,

Umpf... the trouble is, I don't know where I could look up the AI settings of my enemies.
There were 8 parties on this map, I tried to have all tribes and all AI settings (except: none).
Obvioulsy I must have left out 2 of them. But I dont know which ones they were - and please don't ask about the color assignment. face-smile.png
I think I had 4 teams (setup along the tribes) All I can say, is that my modest neighbor to the west is also playing empire and is in my team. If you really need this, I can try and load the replay and see what I can find out.

the size of your territory on the map,

See Here

your screen resolution

I played the game with different resolutions (as I usually do). When realms get bigger, I switch to a higher resolution. I started with 800x600 (full screen) and then went to 1900x1160, windowed) all 32bit.

Now for the hard part:

estimation of the number of objects on the map (incl. buildings, roads, flags, people, wares, trees, animals, etc.) - don't count exactly, just estimate

I pass that back to you. face-smile.png You have seen the map and territories. All roads are fully flagged, double carriers. I'd gladly give you the replay, but you might not be able to load it.

Here are some statistics that might help you with the estimation.

Land Workers Buidlings Wares Productivity Csualties Kills Military buildings lost Military buildings defeated Civilian buildings lost Military
Hope, you can figure out something. Obviously my machine is not up to huge WL games, and that's a pitty. I have another single core hyperthreading machine with 3,4 GHz (vs. 2,66 on this quad). and might be doing better with this. Nobody really needs games this huge, but they're still fun to play. What I really want to know, is whether this memory and CPU consumption is plausible. (I guess that's what you are trying to find out) And the next question for SirVer is, whether it might be optimized (or in fact, if anybody is willing and able to do that.) :) Thanks for explainig things to me and answering my questions face-smile.png

Edited: 2011-06-23, 14:44

Being no programmer, I apologize for all my suggestions that imply undue workload and for other misjudgements due to lack of expertise or relevant skills.
I am on Win32, have no means to compile, and rely on prefabricated distributions (Thanks to Tino).

Top Quote
player123

Joined: 2011-09-03, 05:08
Posts: 11
Ranking
Pry about Widelands
Posted at: 2011-09-10, 07:06

I've experienced the sort of slowdown to a crawl that Astuur mentions, and that wasn't on a 512 x 512 map, but on the riverlands map with all possible opponents enabled (7 total players from memory). It slowed down from about 18 hours onward.

My system is an I7 2600K with 8gb of memory, Windows 7 64 bit, so no shortage of computing resources to throw at the game.

I suspect there are three areas to look to find the slowdowns.

1) the messaging system often takes forever to respond to clicking on a message even on a new, small map, find what is causing the holdup here and you'll likely find what slows down the rest of the game.

2) the playback feature. How do you disable it? I'm sure that on large maps with lots of time passed it must impact the game speed with writing all the game events to disk all the time

3) disk access in general. Its seems the game halts briefly when writing to disk (you notice this when autosaving in a long game on an old map). You should try and find a way to background read/write to disc.


Top Quote
SirVer

Joined: 2009-02-19, 14:18
Posts: 1445
Ranking
One Elder of Players
Location: Germany - Munich
Posted at: 2011-09-10, 08:27

Hi,

before someone jumps to implementation conclusion from that, let me comment. First: never optimize without profiling first!

1) the messaging system often takes forever to respond to clicking on a message even on a new, small map, find what is causing the holdup here and you'll likely find what slows down the rest of the game.

I have never seen this, can you give a quick way to reproduce this problem?

2) the playback feature. How do you disable it? I'm sure that on large maps with lots of time passed it must impact the game speed with writing all the game events to disk all the time

A complete replay is only a few kilobytes. There is no processing involved, only dumping raw player commands to disk. This takes no time at all. Disk access is buffered and handled by the OS and will never interrupt the program while it only outputs streaming data. You can't disable it, because it makes no sense to disable it.

3) disk access in general. Its seems the game halts briefly when writing to disk (you notice this when autosaving in a long game on an old map). You should try and find a way to background read/write to disc.

The problem here is the serialization of all game objects when saving. Due to pointers and cyclic dependencies this is non trivial and takes a while. This has nothing to do with disc access, while there is a correlation between disk access and hiccups, there is no causality. It would hold even if you just simulated a write. Also this one is not the general problem of the 'crawl' problem discussed in this thread but only happens while saving.

Generally, your post - while surely posted with good intentions - angered me. There is no way to find performance problem in a complex program just by using it. The assumption that 'you'll likely find what slows down the rest of the game' by just playing widelands is .... strange. Please be aware that all developers are pretty good at what they do, finding the grind in a complex program needs profiling which takes a significant amount of time, the reason this has not yet been fixed is that noone was able to muster this time. Also, whenever you post your opinion about solutions to problem, it is wise to back your words with data. In fact it is a necessity if you expect someone to take action on your claims.

If you want to provide valuable information about this problem, learn to use oprofile or valgrind; make a careful analysis of their output and post again with the information which methods are the culprits.


Top Quote
player123

Joined: 2011-09-03, 05:08
Posts: 11
Ranking
Pry about Widelands
Posted at: 2011-09-10, 09:48

I'll just respond to the first bit about messaging, start a new game and build/do things that will triggers messages. Then open the message pane and try clicking on either the message text or the tickbox beside the message. It often takes a few seconds for me. Perhaps it is something inherent to the windows port, I don't know, I'm not a linux user (another reason why I wouldn't even attempt to play with the source code).

As I said at the top of the post the game slows way down for me also with horrid mouse lag. When playing a largish map for long periods of time.

If you want to get angry over things I point out, that is your problem, not mine, nothing I can do about it, only you can control your temper. I presume oprofile and valgrind are developer debug tools for linux, which again, I don't use.

Edited: 2011-09-10, 09:49

Top Quote
Venatrix
Avatar
Joined: 2010-10-05, 19:31
Posts: 449
Ranking
Tribe Member
Location: Germany
Posted at: 2011-09-10, 13:17

player123 wrote: I presume oprofile and valgrind are developer debug tools for linux, which again, I don't use.

As I see, yes, they are. But there exist Windows profiling tools, too. Even for free. So you can feel free to use them.

PS: I think, SirVer’s right to say, that you can’t judge, where the slowing down happens if you don’t really know the code. And even the coders don’t seem to be able to find the problem(s). So please don’t be that rude to someone who tells you his opinion, especially if he’s in the code. You want us to be polite, too, right?


Two is the oddest prime.

Top Quote