View Issue Details
ID | Project | Category | Date Submitted | Last Update | |
---|---|---|---|---|---|
0022090 | AI War 2 | Crash/Exception | Nov 4, 2019 9:04 pm | Nov 7, 2019 1:07 pm | |
Reporter | NRSirLimbo | Assigned To | Chris_McElligottPark | ||
Status | resolved | Resolution | fixed | ||
Product Version | 1.003 Sortable Objectives | ||||
Fixed in Version | 1.006 Freedom Of Fleet Line Combination | ||||
Summary | 0022090: Resource Leak | ||||
Description | Found a resource leak within the game where the RAM kept increasing seemingly without limit. Since the game didn't crash, only got slower and slower, I was able to make a screenshot of the debug readout in the escape menu and create a savegame. An error seemingly related to issue 0021648 appeared right beforehand. For those who can't read German, that's 12.3 GB reserved RAM and 9 GB actively used. | ||||
Tags | No tags attached. | ||||
|
|
|
Can you post your ArcenDebugLog.txt from your PlayerData folder? I suspect the leak might be something along the lines of tons of error messages popping out for some reason... Thanks! |
|
Oops, forgot that among the other stuff |
|
Ran into it again. This time the PC had a bluescreen with MemoryManagement (gonna guess it's because of the leak...) |
|
There is definitely some sort of bug in the metal flow planning, related to "Error! PlannedMetalFlow from pool was already internalIsInUseNow" and "Danger! CurrentFrameNumber changed from 34405 to 34406 during metalFlowContext!" I'm not sure if that's related to the UI, or what. Have you by any chance been using the metal flows window that you get when clicking the metal display in the top resource bar? If you don't use that, does this still happen? I'm surprised more people aren't running into this, so I'm not sure what's different in your savegame or your style of play, one or the other. Either way we need to fix it, but just trying to narrow it down. Thanks! |
|
Oh, am I also correct in that the second time you ran into this (and got a bluescreen), you were in a new savegame and not the same old one? Looks like a fresh one was generated. I guess it's possible this is related to a specific faction you have enabled and some bug in them or some bug in a specific combo of ships. That save you posted is pretty large, so I'll have to look into it and see what's going on. |
|
When I open this savegame, I get super smooth memory usage and no leaks when I am just panning around the battlefield and not really doing anything. When I open up the metal flows interface, I have yet to get any exceptions, BUT the memory starts climbing like crazy (20mb a second or so, on average, but not wholly consistently and sometimes with some drops). Closing the metal flows interface causes the usage to go back down some, but not remotely to where it had been. The number of "raw budgets" does seem to start going up inappropriately at some point, too, possibly after opening the metal flows window. In general the number of "raw budgets" seems incorrectly high (171k and counting after about 5 minutes on 10x fast forward, with 2.9m total). So there seems to be a leak in that area, possibly only somewhat related to the interface itself. Badger, if you see this note, I'll look into this one since it spans several areas that are not all related to you. |
|
To answer a few questions: I usually don't use the metal flow interface and as far as I can recall I never had it open when these errors occur. Every time a new update comes out I abandon the current game and I create a new one so there's no false positives on bugs or exceptions. The leak I ran in today was on a fresh save created just today on version 1.005. I don't frequently check the RAM usage, but it seems to spike way harder than "only" 20mb/s, or maybe I just start to notice it at some point. What I could imagine is that there's some kind of infinite loop in there that simply mass-triggers the "small" leak? I'll keep my eyes out when I play on how much RAM the game uses over time. I have been noticing a slow in performance over gameplay, though that could also be due to higher ship numbers. What I have noticed at one point is a massive decrease in performance when I open the galaxy map. I have yet to check if that has anything to do with what we're tracking down, but it went from no number displayed to 0000143:0000050-60% when opened and then ticked back up to 0000154:0000070-80%, probably full performance again when closed. Also, is there any way I can trigger a thread position state dump or however you call them (as in a file which writes down where in which code line all the threads currently are)? That might help locate the infinite loop, provided there even is one. Lastly, I encountered these minor errors before, I do in just about ever savegame that survives for more than an hour, but not directly before it went out of control. |
|
IDK what is up with mantis and tilde, but I meant to say 50-60% and 70-80% up there. |
|
My guess with the ~ (sorry for off topic) is that it makes it go the the comment number after the ~. So 0000151:0000056 would be issue whatever:000056 |
|
Correct on all those notes about ~. Coming in 1.006: * Fixed a pretty nasty memory leak that could happen around planned metal flows, particularly given that they were being used from a single pool but for two different threading areas. ** This was something that would affect things even when the metal flows UI wasn't open, because of race conditions between two threads. It was also causing some errors and other race conditions in general, all of which are now fixed. ** It was a very unusual sort of situation with a very special-case data structure (the time-based pool), and that was just... really an edge case thing. It also was not really causing errors or a memory leak in all cases, even though it was in some. Why it would be particularly severe for the one player who reported it, while nobody else had crashes related to this, is mildly baffling. But that's the nature of race conditions for you, honestly. ** Thanks to NRSirLimbo for reporting. There's still another leak, though, so I need to also hunt that one down. |
|
* Fixed a couple of memory leaks and other things that caused excessive memory churn while actively viewing the metal flows interface. These were things that would be given back to the player and thus not truly a memory leak in general; it was leaking on the heap and then coming back later. |
|
Okay, coming tomorrow: * The last of the memory-leaky-type things related to metal flows is now fixed. This one was introduced possibly as far back as September 2018 by Chris, or possibly more recently when changing another bit of code to make a copy per list instead of having a reference in another list. At any rate, we were creating a metric ton of extra metal flow data that was then getting churned out of the memory pool by the garbage collector, and generally wasting some CPU time and efficiency. Now things properly get returned to the pool. ** On the metal flows screen, there is still some climbing RAM that you see for a while, but then it stabilizes and eventually drops. That seems to be mostly related to temporarily cached sprite quads being used by TextMeshPro, and with such large amounts of text that changes frequently that's probably unavoidable. It's not a true memory leak anymore, and doesn't spike more than something like 60mb at peak in our current tests, where it was hundreds of mb in the same tests previously. Thanks! |
|
I like the sound of that :) Good work! |
|
Thanks! Fingers crossed. :) |
Date Modified | Username | Field | Change |
---|---|---|---|
Nov 4, 2019 9:04 pm | NRSirLimbo | New Issue | |
Nov 4, 2019 9:04 pm | NRSirLimbo | File Added: 20191105023707_1.jpg | |
Nov 4, 2019 9:04 pm | NRSirLimbo | File Added: 20191105024742_1.jpg | |
Nov 4, 2019 9:04 pm | NRSirLimbo | File Added: resource leak.png | |
Nov 4, 2019 9:04 pm | NRSirLimbo | File Added: s2 resource leak.savemet | |
Nov 4, 2019 9:04 pm | NRSirLimbo | File Added: s2 resource leak.save | |
Nov 4, 2019 9:45 pm | Chris_McElligottPark | Note Added: 0054349 | |
Nov 4, 2019 10:21 pm | NRSirLimbo | File Added: ArcenDebugLog.txt | |
Nov 4, 2019 10:21 pm | NRSirLimbo | Note Added: 0054356 | |
Nov 6, 2019 12:12 pm | NRSirLimbo | File Added: ArcenDebugLog-2.txt | |
Nov 6, 2019 12:12 pm | NRSirLimbo | Note Added: 0054392 | |
Nov 6, 2019 1:28 pm | Chris_McElligottPark | Note Added: 0054393 | |
Nov 6, 2019 1:30 pm | Chris_McElligottPark | Note Added: 0054394 | |
Nov 6, 2019 1:47 pm | Chris_McElligottPark | Note Added: 0054395 | |
Nov 6, 2019 2:20 pm | NRSirLimbo | Note Added: 0054396 | |
Nov 6, 2019 2:20 pm | NRSirLimbo | Note Added: 0054397 | |
Nov 6, 2019 2:25 pm | Ovalcircle | Note Added: 0054398 | |
Nov 6, 2019 2:37 pm | BadgerBadger | Assigned To | => Chris_McElligottPark |
Nov 6, 2019 2:37 pm | BadgerBadger | Status | new => assigned |
Nov 6, 2019 7:28 pm | Chris_McElligottPark | Note Added: 0054413 | |
Nov 6, 2019 7:56 pm | Chris_McElligottPark | Note Added: 0054414 | |
Nov 6, 2019 8:37 pm | Chris_McElligottPark | Status | assigned => resolved |
Nov 6, 2019 8:37 pm | Chris_McElligottPark | Resolution | open => fixed |
Nov 6, 2019 8:37 pm | Chris_McElligottPark | Fixed in Version | => 1.006 Freedom Of Fleet Line Combination |
Nov 6, 2019 8:37 pm | Chris_McElligottPark | Note Added: 0054415 | |
Nov 7, 2019 6:15 am | NRSirLimbo | Note Added: 0054419 | |
Nov 7, 2019 1:07 pm | Chris_McElligottPark | Note Added: 0054427 |