Ping steadily increasing to client for one map, but not another. Not ded svr; limited to 3 plyrs.

[Sum Ting Wong]

Been playing A19.3 for many hrs, and building on map "Pregen03."  I'm in CA, and usually play with 1 friend (through Steam, not dedicated) who lives in PA. 

No problems 'til recently. 

We played last night.  On the map I mainly use (Pregen03) his ping starts at 150 and keeps slowly rising well past 175000, unless he disconnects and reconnects, which only lasts for a few minutes until the lag becomes unplayable. 

We did a few tests.  If I load map "Navezgane," his ping is steady at 75, with no appreciable change.

As soon as I load the Pregen map, his ping starts around 150, and rises steadily.

 

A few facts:

     We both have very recent high end computer builds, with top tier graphics cards.

     We are connecting through Steam, not a dedicated server.

     We both have fiber, with no other issues regarding lag, slow downloads, etc.

     We have played the Pregen03 map for MONTHS, with no noticeable lag until the past 5 days.

     I have created a new building in the Pregen03 map, using CM menu, so not sure if this plays a part in the degredation.

     Even if I disable Blood Moon, and there is little activity in the map, his ping starts high, and steadily increases past 175000.

     The degradation on his (PA) connection does NOT affect my playability in any way, but I'm hosting.

     We are both using A19.3(b6).

     Not to beat a dead horse, but it only has an issue on 1 map, and not the other, and the issue started within the last week which is, coincidentally, when I started making a few items using the CM.

 

Any help would be appreciated!

 

     

 

[SylenThunder]

Have your friend with the issue ensure that the client is excluded from his security software. This is covered in the Support FAQ thread that is pinned at the top. Outside of that, I'd need logs. Instructions for that are also Pinned.

[Sum Ting Wong]

 

Thanks for the quick reply.

My friend is using Windows 10, and the AV is just stock Defender.  He's added exceptions for the firewall, and turned off Defender, with no change.

I fully read the FAQ, and searched for similar issues.  I've been building computers since 1986, and am comfortable providing any info you need, so feel free to ask, if it will help debug.

I've attached the last few log files, as I'm not sure which are pertinent.

Please let me know if you need further info.

 

I appreciate your time.

D!

output_log__2021-01-17__01-19-14.txt output_log__2021-01-17__02-09-23.txt output_log__2021-01-17__02-10-59.txt output_log__2021-01-19__14-25-57.txt output_log__2021-01-19__20-26-36.txt output_log__2021-01-19__20-48-08.txt

[meganoth]

It would have been interesting to also see the logfile of your friends client. But your logfiles already show an interesting thing. In 4 of the six there are hundreds of lines like this:

 

(Filename: C:\buildslave\unity\build\Modules/Audio/Public/AudioSource.cpp Line: 459)

Can not play a disabled audio source
 

 

In the other two strangley there is none of this, but this error turns up repeatedly.

 

(Filename: C:\buildslave\physx\build\PhysX\Source\PhysXCooking\src\convex\QuickHullConvexHullLib.cpp Line: 937)

[Physics.PhysX] QuickHullConvexHullLib::findSimplex: Simplex input points appers to be coplanar.
 

 

I have seen the second error message come up a few times, it seems rather harmless itself but points to savegame data that is damaged in some way (my impression from only a few samples and my rather faulty memory, I'm not at all sure about this "fact").

 

The first one seems harmless as well, but having such a message in a larger quantity could maybe thwart the network code in the same thread from operating correctly. But why isn't it there in two of the logs and the problem still occurs?

 

Apart from that I see nothing unusal in your logs

[Liesel Weppen]

If it is a corrupted map file, it may be location dependent. Probably it vanishes, if you move away from wherever you are. And/or you can figure out a region where it occurs?

 

You can also try starting a fresh game, just to look if it is a general problem or map dependent.

[Sum Ting Wong]

I can verify 100% it is map dependent.  We played the problem map (ping steadily increases), then another undeveloped map, and his ping was steady at 75 (from across the country).  So there is something up with this map.

 

Oddly, it didn't start until I built a new structure using the creative menu.  We have been developing this map for months, with no issues.  Last week I decided to try building one structure using items from the creative menu, and that seems to coincide with the ping issue.

 

Is there a way to get a definition of the two recurring faults?

 

 

I will get the log files from my friend's (client) computer.

Thanks again for the help!

[meganoth]

With "map" Liesel meant a region inside the game world. The whole game world is segmented into rectangular areas, each one is a separate file in the Region directory of the save game.

 

Your friend should run or drive a mile away from the spot he is at now and you should do the same.  If the pings turn normal again, you can delete just one region file and the rest of the world is saved.

 

Normally using the creative menue should not do anything harmful, lots of people seem to do it. Doesn't mean there couldn't be some seldom-used item in there nobody has tested thoroughly.

 

 

[Sum Ting Wong]

3 hours ago, meganoth said:

With "map" Liesel meant a region inside the game world. The whole game world is segmented into rectangular areas, each one is a separate file in the Region directory of the save game.

 

Your friend should run or drive a mile away from the spot he is at now and you should do the same.  If the pings turn normal again, you can delete just one region file and the rest of the world is saved.

 

Normally using the creative menue should not do anything harmful, lots of people seem to do it. Doesn't mean there couldn't be some seldom-used item in there nobody has tested thoroughly.

 

 

 

 

So THIS was good info.  To test, I just started a game, and had my friend log in.  He spawned near me, which was near the problem area.  Immediately his ping shot up.  We both got in a jeep when his ping hit around 55000.  By then he was lagging so much, that I had driven away about a mile, with him sitting next to me, and his screen showed he hadn't even got in the jeep yet.  I parked about 1.5 miles from the problem area, and watched the ping.  It stayed high until his character caught up to my current game state and location, then the ping instantly dropped. 

 

We drove closer and closer to the newest structure without any change in ping (around 90).  I parked in front of the two older structures, with no change in ping.   I was about 75 yds from the newer structure, and it had not yet materialized on screen.  Ping was normal.  I very slowly approached the structure.  As soon as it materialized, the ping started going out of control.  

 

I shut down the game shortly after.

Looking at the log file, I'm guessing it's this warning that's most relevant:

(Filename: C:\buildslave\physx\build\PhysX\Source\PhysXCooking\src\convex\QuickHullConvexHullLib.cpp Line: 937)

[Physics.PhysX] QuickHullConvexHullLib::findSimplex: Simplex input points appers to be coplanar.

 

I've attached both the host (mine) log file, and the client log file from my friend's computer.

>>> I looked at my computer when the lag started to shoot up, and it was at 2:29pm.  The client file shows 5:29pm, as he's in PA.

 

We decided to try it again, but making a much shorter log file.  I spawned, and moved to a short distance away from the new structure, just out of range of it materializing.

My buddy spawned close to the problem area, the ping shot up, and he ran over to where I was, and the ping lowered.  We both got in the jeep, and drove right to the structure, watched the ping climb for a few seconds, and then shut down the game.

 

It is DEFINITELY that warning that just I posted above.  I included the second set of log files, as they are much smaller, and easier to see the issue.

 

So...knowing that, is there a way to debug this fault?  I would hope TFP might be interested, as I've done nothing spectacular, like weird objects or enormous structures.

 

 

 

 

 

output_log__2021-01-22__17-05-52-Client.txt output_log__2021-01-22__14-05-01-Host.txt output_log__2021-01-22__18-58-41-Client-2nd Attempt.txt output_log__2021-01-22__16-00-30-Host-2nd Attempt.txt

[meganoth]

@Jugginator may be interested in look into it if he got time. If that is the case he probably needs the complete savegame. So make a backup before changing anything.

 

[Sum Ting Wong]

Thanks, I just saved it.

That's a large Savegame file.   The world (Navezgane) which I've only used for a short time is 62 MB, and the one I've developed is 1.16 GB.

[Jugginator]

The coplanar error is typically related to electric fence posts; can you upload the region file(s) of the base you have with fence posts setup (i.e. the laggy area)?

 

To do this:

• dm into console
• press F3
• Tab
• Click Ch in the new window opened with F3
• You'll see the region location info in the new chunk window
• As you move it'll update if you move into another region. Just go to all areas of your base and note the numbers. That will be a file with the same name located inside the Regions folder.

 

Alternatively, just post your whole save folder and the coordinates of your base.

[Sum Ting Wong]

 

Thanks for the help.

I've uploaded the specific region file.

Oddly, there are no electric fence posts in this region..

 

Please let me know what you find, when you have time.

 

Appreciated!

 

> EDIT - While walking around in the world, I just realized how large the regions are, so I included a pic of the problem area, with the coordinates shown from the DM.

r.-2.-3.7rg

[Jugginator]

Thank you very much; I didn't realize how much wiring you had, so to make sure I get it right can I have your power.dat file please?

 

On another note, deleting the power.dat will likely cure the issue (just make sure you take engines and ammo out of stuff first, that's contained in that file).

 

@Sum Ting Wong

[Sum Ting Wong]

 

Here's the power.dat file.

Thanks for the info on how to reset without losing all the work.

It's not a tragedy if I have to reset the region, just a lot of work.

I figured if it can help find bugs for you guys, it's a win-win.

 

If you have a way of identifying how I caused it, that would be helpful, to avoid the same mistake in the future.

 

Thanks again!

power.dat

[Jugginator]

18 minutes ago, Sum Ting Wong said:

 

Here's the power.dat file.

Thanks for the info on how to reset without losing all the work.

It's not a tragedy if I have to reset the region, just a lot of work.

I figured if it can help find bugs for you guys, it's a win-win.

 

If you have a way of identifying how I caused it, that would be helpful, to avoid the same mistake in the future.

Specifically, the issue is in one of the wired things, I've never heard/seen of anything but an electric fence causing it though. Don't have to reset the region likely, just the power.dat, which contains just the wire hookups/gas/engine/ammo. Deleting that should work, no need for a region wipe

 

The issue seems to have crept back up after being fixed once, I'll report back when I find something

[SylenThunder]

4 hours ago, Jugginator said:

Specifically, the issue is in one of the wired things, I've never heard/seen of anything but an electric fence causing it though. Don't have to reset the region likely, just the power.dat, which contains just the wire hookups/gas/engine/ammo. Deleting that should work, no need for a region wipe

 

The issue seems to have crept back up after being fixed once, I'll report back when I find something

I have heard of a large number of turrets causing it, but we're talking about people who have 30-50 turrets.

[Sum Ting Wong]

 

Actually, there are over 40 turrets...

 

Just hoping @Jugginator is able to root out the problem...

I luv turrets ;>)

 

Not that I don't like to have to gun down my own blood moon horde but, occasionally, it's fun to watch a s4itload of turrets do the heavy lifting while u and ur buds "pick and choose."

[Jugginator]

@Sum Ting Wong

 

This, as with the fence post null ref, is rather elusive. I tried looking for differences in the wiring and imediately eliminated a few null refs, lowered it from 19 to 13 in fact, , but it's one of those "seen this 4 times in a row that's the issue, oh wait, now it's this... and it didn't work here" things lol.

 

For now, if you don't want to rewire everything, you can unhook/re-hook the turrets. I have actually completely removed the null refs by removing turrets in two different spots. Going through and narrowing down all of them would take waaay longer than just re-hooking them, so here are the two areas (note that simply re-wiring will work, I just removed them to be quick about it, although to be sure you can pickup and place them back down then wire them back up):

 

 

Spoiler

 

Hopefully we can get the root cause of this issue narrowed down and fixed asap. I'm glad you brought this up since (at least I) haven't seen it with turrets.

 

Honestly, I set up and ran extensive 64 or 32-spawning hordes max game stage for some time with between 20 and 50 turrets in three different tests and haven't had this, definitely not common. Pretty cool base, by the way

[Sum Ting Wong]

 

Cool; at least I don't need to completely tear down, as it goes many many levels down.

 

I will give this a shot and let you know what happens.

 

Thanks for spending the time to look at it!

 

[Jugginator]

Ofc you're welcome. Hopefully we can get this sorted out.

 

I am most curious if this resolves your friend's issue. Hopefully it does.

[Sum Ting Wong]

@Jugginator

 

So, here's an interesting update.  Last night I removed all the engines from the generator banks, and the ammo from all the turrets.

I then shut down the game, and removed the power.dat file.  I restarted the game, and my friend from PA logged on.

 

The ping was normal (95) and I installed all the engines (30) back into the generator banks.  Without wiring, I decided to be procedural about debugging the cause, so I fired up the generators with no wiring.  Interestingly, the ping did go up to about 300, then fluctuated, and slowly came down to a little over 100.

I wired up 1/3 of the turrets, with the generators off, and the ping was normal; maybe just slightly high.

BUT, as I turned on each generator, the ping went higher and higher.  With the first generator on, it went to around 1,500.  When the second one was turned on, the ping went to around 5,500.  With the third one on, the ping was slowly climbing, and I shut everything off and called it quits when the ping was over 20k, as my friend was lagging so far behind, he couldn't effectively wire anything further.

 

If you have any ideas for debugging on my end (software tools, logging tools) I'd be willing to do some legwork to help sort it out.  

I could just delete the region and start over, but I suspect it will happen to others, who may be unable, or unwilling to take the time to report it.

 

As always, thanks for the time to help, I'd imagine you're spread thinly playing whack-a-mole with issues like mine.

 

 

[Jugginator]

What in the world lol.

 

As for debugging... I'd use perfmon to monitor resources, see if your (host) and client's CPU/RAM/Networking (I suspect networking) are being hogged up, one of those will narrow it down a bit.

 

That is really crazy yet interesting thanks for doing that and reporting. You're definitely right about what-a-mole issues as of recent LOL, have quite a few on my plate, but I think I'll put the base on a world on dedi and see whats up. I'll test my in-house client-server combo then try with another tester.

 

@Sum Ting Wong Hey you said you were wiring then said your friend was wiring, did the ping get to over 20k after he started wiring things himself?