View Issue Details

IDProjectCategoryLast Update
0020320AI War 2Crash/ExceptionSep 19, 2018 12:45 pm
ReporterRocketAssistedPuffin Assigned ToBadgerBadger  
Status resolvedResolutionfixed 
Product Version0.773 Terrorized By An Infinite Loop 
Fixed in Version0.773 Terrorized By An Infinite Loop 
Summary0020320: Game freeze
DescriptionMaking a report here for some organisation.

Loaded the save. Got about two minutes in, hit TAB to open galaxy map - game freezes. Put some tracing on and got a log with somethings actually in it. Also have that hidden screen image put here as well.

Dev build.

Continuing testing.
TagsNo tags attached.

Activities

RocketAssistedPuffin

Sep 17, 2018 8:25 am

reporter  

ArcenDebugLog.txt (1,332,751 bytes)
Current2.save (1,786,466 bytes)
Current2.savemet (44 bytes)
HiddenScreen.jpg (76,903 bytes)   
HiddenScreen.jpg (76,903 bytes)   

RocketAssistedPuffin

Sep 17, 2018 8:31 am

reporter   ~0049314

Using that save procs it guaranteed within about 1, 2 minutes, each time I try. Starting a new game and letting it sit for a bit so far hasn't.

I'll try a new game today and progress through it. See if that isolates it to a broken save.

It's not a specific time it occurs, and I don't need to open the galaxy map either.

RocketAssistedPuffin

Sep 17, 2018 9:25 am

reporter   ~0049315

New game froze about 9 minutes into simulation time.

Chris_McElligottPark

Sep 17, 2018 12:10 pm

administrator   ~0049316

This is in the dev builds, correct? On the old savegame, does it still do the freeze in the very latest versions? There was some questionable data in there that the game now handles more gracefully. I suspect that this sort of data was getting generated after spending a while in the game and then gumming things up. I'm currently not getting the freeze on that save, but I've only let it run about 3 minutes.

RocketAssistedPuffin

Sep 17, 2018 12:21 pm

reporter   ~0049317

I shall test not much later today. Yes, dev builds. I'll most likely be reporting from that from now on.

3 minutes would've hit it by now, since it does it consistently at around 2. We'll see.

Chris_McElligottPark

Sep 17, 2018 12:24 pm

administrator   ~0049318

Sounds good, thanks!

BadgerBadger

Sep 17, 2018 12:51 pm

manager   ~0049319

I let the game run at max speed without really doing anything, and I observed the game freeze at 3h 7m. I don't see any obvious errors in the unity log or arcen debug log.

Chris_McElligottPark

Sep 17, 2018 1:06 pm

administrator   ~0049322

Funnily enough I just ran into it in another save. I'll see what I can figure out...

Chris_McElligottPark

Sep 17, 2018 1:24 pm

administrator   ~0049323

I fixed what was one possible infinite loop (frankly the syntax on it is confusing and from a quick read I'm not sure how it ever worked), but I'm not wholly convinced that was what the trigger was. I have yet to see it since, but it was very intermittent for me anyhow.

Chris_McElligottPark

Sep 17, 2018 1:38 pm

administrator   ~0049324

....aaaand it froze again. Will continue scouring the code.

Chris_McElligottPark

Sep 17, 2018 4:32 pm

administrator   ~0049329

I don't seem to be able to cause this anymore, not since improving the GetSpeed() logic. I can't fathom how that would have been related, but perhaps the cumulative number of changes caused it to get fixed up in some fashion. The GOOD news is that if this happens again, while I'm running it in the standalone game (versus the unity editor), I can now get proper stack traces on any and all active threads. So that's a win compared to earlier today. I still can't get that while in the unity editor, which is how I usually run the game, so it's possible that I might run into it at some point and be unable to debug it at that time. But hopefully someone out of us will have a good repro case at that point.

...and then I clicked back to the game and it froze once again. But I didn't have the debugger attached, facepalm.

RocketAssistedPuffin

Sep 17, 2018 5:51 pm

reporter   ~0049331

Can confirm it's still here.

Chris_McElligottPark

Sep 18, 2018 10:25 am

administrator   ~0049332

-bad news on infinite loop detection... trying to do the following does NOT work:
1. In mono, it is not possible to get a StackTrace object from another thread than the one you are presently on. The method is there like in .NET, but it has an obsolete flag noting it isn't implemented and is a mono limitation.
2. The next idea was to trigger an exception on the main thread by calling Thread.Abort() on it from another thread, but this only kinda-sorta works; it generates the exception, but with no stack trace, which is a definite issue.
3. The only remaining option is thus to try to inject bad data into the main thread such that it will die on its own wherever it is. That won't hit every case of a possible infinite loop, and it's not graceful in the slightest, but it would work unless simply no central logic was being referenced.
4. Another option would be to have EVERY loop check a central bool every iteration and die if it's false... but that's rather time-intensive and a (very) mild waste of CPU as well. It also would not catch thread deadlocks, only infinite loops. Which is true of option 3, as well.
5. Last option would be for us to go back and see if we can find the specific svn revision where the locks were introduced and then deduce from that what the problem is and fix it.

I'll follow up via email. I'd like to try option 5 to start out with, if possible, because I fear this may be a deadlock.

BadgerBadger

Sep 19, 2018 12:45 pm

manager   ~0049343

This is the infamous infinite loop

Issue History

Date Modified Username Field Change
Sep 17, 2018 8:25 am RocketAssistedPuffin New Issue
Sep 17, 2018 8:25 am RocketAssistedPuffin File Added: ArcenDebugLog.txt
Sep 17, 2018 8:25 am RocketAssistedPuffin File Added: Current2.save
Sep 17, 2018 8:25 am RocketAssistedPuffin File Added: Current2.savemet
Sep 17, 2018 8:25 am RocketAssistedPuffin File Added: HiddenScreen.jpg
Sep 17, 2018 8:31 am RocketAssistedPuffin Note Added: 0049314
Sep 17, 2018 9:25 am RocketAssistedPuffin Note Added: 0049315
Sep 17, 2018 12:10 pm Chris_McElligottPark Note Added: 0049316
Sep 17, 2018 12:21 pm RocketAssistedPuffin Note Added: 0049317
Sep 17, 2018 12:24 pm Chris_McElligottPark Note Added: 0049318
Sep 17, 2018 12:51 pm BadgerBadger Note Added: 0049319
Sep 17, 2018 1:06 pm Chris_McElligottPark Note Added: 0049322
Sep 17, 2018 1:24 pm Chris_McElligottPark Note Added: 0049323
Sep 17, 2018 1:38 pm Chris_McElligottPark Note Added: 0049324
Sep 17, 2018 4:32 pm Chris_McElligottPark Note Added: 0049329
Sep 17, 2018 5:51 pm RocketAssistedPuffin Note Added: 0049331
Sep 18, 2018 10:25 am Chris_McElligottPark Note Added: 0049332
Sep 19, 2018 12:45 pm BadgerBadger Assigned To => BadgerBadger
Sep 19, 2018 12:45 pm BadgerBadger Status new => resolved
Sep 19, 2018 12:45 pm BadgerBadger Resolution open => fixed
Sep 19, 2018 12:45 pm BadgerBadger Fixed in Version => 0.773 Terrorized By An Infinite Loop
Sep 19, 2018 12:45 pm BadgerBadger Note Added: 0049343