Native Linux server (with management scripts)

[jdragun]

OK. I don't get it.

 

I modified startserver.sh to

#if [ "$(uname -m)" == "x86_64" ]; then    ./7DaysToDieServer.x86_64 -logfile 7DaysToDieServer_Data/output_log__`date +%Y-%m-%d__%H-%M-%S`.txt -quit -batchmode -nographics -dedicated $PARAMS
#else
#    ./7DaysToDieServer.x86 -logfile 7DaysToDieServer_Data/output_log__`date +%Y-%m-%d__%H-%M-%S`.txt -quit -batchmode -nographics -dedicated $PARAMS
fi

And yet the log is still showing 32-bit.

 

Going to have to start a fresh box up and test on it I guess.

 

Edit: New box, fresh install, and still I get...

2016-02-22T21:19:41 0.087 INF Version: Alpha 13.8 (b7) Compatibility Version: Alpha 13.8, Build: Linux 32 Bit

2016-02-22T21:19:41 0.087 INF System information:

2016-02-22T21:19:41 0.129 INF OS: Linux 4.2 Ubuntu 15.10 64bit

>.<

 

I am also running the dedi mods and the start.sh calls the 32 bit version. Is there a reason for that? I am running Ubuntu Server 64, and don't know if I should just edit the start.sh to call the 64 bit version or not. Will that effect anything? What is the major diffrence between the 32 and 64 versions other than mem use?

 

Do you still have to edit the startdedicated.sh in the /home/sdtd/engine/ dir?

[Devidian]

I am also running the dedi mods and the start.sh calls the 32 bit version. Is there a reason for that? I am running Ubuntu Server 64, and don't know if I should just edit the start.sh to call the 64 bit version or not. Will that effect anything? What is the major diffrence between the 32 and 64 versions other than mem use?

 

Do you still have to edit the startdedicated.sh in the /home/sdtd/engine/ dir?

 

This is a good question, mine is starting up in 32 Bit by default also:

 

2016-02-25T09:15:04 0.055 INF Awake2016-02-25T09:15:04 0.128 INF Version: Alpha 13.8 (b7) Compatibility Version: Alpha 13.8, Build: Linux 32 Bit
2016-02-25T09:15:04 0.129 INF System information:
2016-02-25T09:15:04 0.443 INF    OS: Linux 3.13 Ubuntu 14.04 64bit
2016-02-25T09:15:04 0.445 INF    CPU: Dual-Core AMD Opteron(tm) Processor 1214 HE (cores: 2)
2016-02-25T09:15:04 0.445 INF    RAM: 3936 MB
2016-02-25T09:15:04 0.445 INF    GPU: Null Device (128 MB)
2016-02-25T09:15:04 0.451 INF    Graphics API: NULL 1.0 [1.0] (shader level 2.0)

[Maza]

Hello Alloc

 

I have a dedicated server with the following settings:

 

CPU Intel I3, 8Gb RAM, Linux OS Ubutu Server 15, with 15 slots, use server manager of Envul and features constant falls with the following error:

 

/usr/local/lib/7dtd/common.sh: line 201: 28650 Dead nc 127.0.0.1 $ TEL_PORT 0 <& 3

Edited March 3, 2016 by Maza (see edit history)

[Alloc]

No, just vanilla. I have a xui mod running some weeks now but the crashes like this were happening before too. Not often but they happen.

Have you been monitoring the memory consumption / size of the virtual address space of the process? If you don't get this soon after starting the server but only later on when people have played on it and you only access the web panel after that for the first time it is probably just a memory issue.

 

 

Currently it uses xhr requests (polling) not web-sockets.

Ah, ok, gotcha. Yeah, of course this could be done. But I don't see that much of an audience that would benefit from this. Or maybe there would be in terms of server managers or something. Would require a lot of extra time though

But of course it's open to everyone to have a go at implementing something like that

 

 

edit: by the way, can't you configurate the web-server to push an empty transparent map tile on missing files ? the console 404 warnings ...

Yeah, probably a good idea.

 

 

...

I hope my english was good enought to explain my thoughts - otherwise i prefer german

Heh, no issues there

As I said above though ... this would just add another layer of work on my end while not giving any benefit to the integrated web panel. Of course it sounds like a good idea but I suppose someone else will have to have a go at this.

 

 

 

 

I am also running the dedi mods and the start.sh calls the 32 bit version. Is there a reason for that? ... What is the major diffrence between the 32 and 64 versions other than mem use?

As has been said a few times these pages: 32 bit is fully working, 64 bit not (probably only the serverlist part, but haven't looked any further).

 

Do you still have to edit the startdedicated.sh in the /home/sdtd/engine/ dir?

No, it's not used.

 

 

 

 

... features constant falls with the following error:

 

/usr/local/lib/7dtd/common.sh: line 201: 28650 Dead nc 127.0.0.1 $ TEL_PORT 0 <& 3

Suppose the server isn't usable at this moment anymore? Probably plain crash (or at least hangup) due to the memory restrictions. 15 slots sounds a bit much for a 32 bit server instance.

[zigstum]

Editing WebMap

 

Mr Alloc

I've tried changing the index.html file @ /home/sdtd/engine/Mods/Allocs_WebAndMapRendering/webserver/index.html, but it shows no change on the map page itself. Specifically, I want make the side pane an off-canvas menu type div.

Is the web page the result of a compiled process?

How can I make changes?

Is it possible to run the webmap in a standard apache2 instance rather than the embedded mono server?

Thanks,

Zig.

[Maza]

 

 

Suppose the server isn't usable at this moment anymore? Probably plain crash (or at least hangup) due to the memory restrictions. 15 slots sounds a bit much for a 32 bit server instance.

 

 

Thank Alloc,

 

My server is 64 bit, and has 8gb ram, the server down with less than 10 players, this occurs often, other occasion he just not running without displaying error.

 

Decreased to 13 slots. Usually the server is online with 14 players at the same time with 7 players.

 

I monitored the consumption of memory with 10 players reaches 48% of memory consumption.

 

Errors:

 

/usr/local/lib/7dtd/common.sh: line 201: 28650 dead nc 127.0.0.1 $TEL_PORT 0<&3.

 

 

Instance: mazaserver

Status: NOT running

Game info:
   Server name:    Maza Server BR [shop/Tp/Home/Loteria/Etc...]
   Password:
   Max players:    13
   World:          Random Gen

Network info:
   Port:           25000
   Public:         true

 

 

Statics with 7 players in server

 

top - 21:02:00 up  9:00,  2 users,  load average: 1,65, 1,19, 1,08
Tasks: 119 total,   1 running, 118 sleeping,   0 stopped,   0 zombie
%Cpu(s): 24,6 us,  2,5 sy,  0,0 ni, 72,5 id,  0,4 wa,  0,0 hi,  0,1 si,  0,0 st
KiB Mem:   8086908 total,  8018028 used,    68880 free,    19244 buffers
KiB Swap:  8298492 total,  6067340 used,  2231152 free.   385520 cached Mem

 PID USER  PR  NI   VIRT     RES     SHR S    %CPU   %MEM    TIME+ COMMAND
20779 sdtd  20  0    2682616  2,125g  29364 S  107,5  27,6  20:45.11 7DaysToDieServe

Edited March 6, 2016 by Maza (see edit history)

[zigstum]

Oddly, the changes I made to the index page are now showing (22hrs later 0_o), I will do some more testing

[Alloc]

Is the web page the result of a compiled process? How can I make changes?

No, it's directly served, you can edit it at any time as it's currently not cached.

 

Is it possible to run the webmap in a standard apache2 instance rather than the embedded mono server?

Yeah, but you will either have to reverse proxy the API / map requests to the game or modify the files so they access the API / map from a different location (the game server).

 

 

 

 

My server is 64 bit

Yeah, the server, but the 7dtd server process most likely is not (that is unless you modified the scripts to start as 64 bit).

[Maza]

 

 

Yeah, the server, but the 7dtd server process most likely is not (that is unless you modified the scripts to start as 64 bit).

 

 

Ok, thank you Alloc

 

How to modify my 7dtd server process for 64-bit ?

 

startserver.sh :

 

#!/bin/sh
cd "`dirname "$0"`"

PARAMS=$@

CONFIGFILE=
while test $# -gt 0
do
       if [ `echo $1 | cut -c 1-12` = "-configfile=" ]; then
               CONFIGFILE=`echo $1 | cut -c 13-`
       fi
       shift
done

if [ "$CONFIGFILE" = "" ]; then
       echo "No config file specified. Call this script like this:"
       echo "  ./startserver.sh -configfile=serverconfig.xml"
       exit
else
       if [ -f "$CONFIGFILE" ]; then
               echo Using config file: $CONFIGFILE
       else
               echo "Specified config file $CONFIGFILE does not exist."
               exit
       fi
fi

export LD_LIBRARY_PATH=.
#export MALLOC_CHECK_=0

#if [ "$(uname -m)" == "x86_64" ]; then
#       ./7DaysToDieServer.x86_64 -logfile 7DaysToDieServer_Data/output_log__`date +%Y-%m-%d__%H-%M-%S`.txt -quit -batchmode -nographics -dedicated $PARAMS
#else
       ./7DaysToDieServer.x86 -logfile 7DaysToDieServer_Data/output_log__`date +%Y-%m-%d__%H-%M-%S`.txt -quit -batchmode -nographics -dedicated $PARAMS
#fi

Edited March 6, 2016 by Maza (see edit history)

[Devidian]

Have you been monitoring the memory consumption / size of the virtual address space of the process? If you don't get this soon after starting the server but only later on when people have played on it and you only access the web panel after that for the first time it is probably just a memory issue.

 

Maybe i should install my node monitoring app that i developed for my company's cloud for memory monitoring... but i never saw memory (virt) above 3GB. The Last Entry before the crash was:

 

2016-02-12T02:25:20 232914.094 INF Time: 3873.49m FPS: 52.37 Heap: 1001.7MB Max: 2541.2MB Chunks: 1508 CGO: 2 Ply: 0 Zom: 0 Ent: 0 (1154) Items: 716 CO: 1

 

Ah, ok, gotcha. Yeah, of course this could be done. But I don't see that much of an audience that would benefit from this. Or maybe there would be in terms of server managers or something. Would require a lot of extra time though

But of course it's open to everyone to have a go at implementing something like that

 

For multi-managers for example yeah. The benefit may be that you would just have to make the api and let the community create several different forms of web-managements. Ok sure at first there may be a little bit more time for you to spend on the web-sockets api, but as soon as it works you could easily add new features and the developers from the community could add it too their web interfaces.

Servers that are hosting multiple instances also wouldn't have the webserver overhead on each instance.

[SylenThunder]

Ok, this is for Alloc, or anyone else who has some idea of what in the world is going on.

 

Wiped my server.

Installed Ubuntu 15.10 Server

Installed the Pre-requisites.

Installed the server.

Ran the updateengine, updatescripts, and updatefixes.

Created the instance, started the server. It's peachy.

Shut server down, and edit usr/local/lib/7dtd/commands/start.sh to make it run in 64-bit.

Start server, it crashes.

Gives the following error...

2016-03-17T13:13:27 21.110 INF StartAsServer2016-03-17T13:13:27 21.118 INF createWorld: Navezgane, Navezgane, GameModeSurvivalMP
Exception: Block with name 'rockResource02' not found!
 at WorldBiomes.E (System.String ) [0x00000] in <filename unknown>:0 
 at WorldBiomes.D (Byte , System.String , System.Xml.XmlElement , Boolean ) [0x00000] in <filename unknown>:0 
 at WorldBiomes.D (Byte , System.String , System.Xml.XmlElement , Boolean ) [0x00000] in <filename unknown>:0 
 at WorldBiomes.G (System.Xml.XmlDocument , Boolean ) [0x00000] in <filename unknown>:0 
 at WorldBiomes..ctor (System.Xml.XmlDocument _genxml, Boolean _instantiateReferences) [0x00000] in <filename unknown>:0 
 at GameManager.WH (System.String , System.String ) [0x00000] in <filename unknown>:0 
 at GameManager.ON () [0x00000] in <filename unknown>:0 
 at GameManager.StartGame () [0x00000] in <filename unknown>:0 
 at ConnectionManager.Net_ServerInitialized () [0x00000] in <filename unknown>:0 
 at NetworkCommon.StartServers (System.String _password) [0x00000] in <filename unknown>:0 
 at MainMenuMono.Start () [0x00000] in <filename unknown>:0 

(Filename:  Line: -1)

Switch it back to 32-bit.

Start server, and it runs just peachy.

 

Any clues why I'm getting a blocks.xml error when running 64-bit?

[Alloc]

...

Any clues why I'm getting a blocks.xml error when running 64-bit?

 

That's interesting ... Can you post / send me a full log of both 32 bit (running fine) and 64 bit (with that error)?

[SylenThunder]

That's interesting ... Can you post / send me a full log of both 32 bit (running fine) and 64 bit (with that error)?

Sure thing.

 

32-bit

First Run

http://pastebin.com/zKc0J3eP

 

Second Run

http://pastebin.com/tNqwF0Ar (Removed some of the CBSM queries to shrink the size for pastebin.)

 

 

64-bit

http://pastebin.com/HRrNMNCW

 

I posted both 32-bit runs because the first one I started, saw in the log it loaded 32-bit, and then shut it down. The second run was after the attempt to run 64-bit and getting the error.

 

It's really strange because no changes were made to any of the xml files at all, and I was successfully running this on 64-bit before I wiped my system.

[Devidian]

Hi there,

 

first of all - good that your fixes do some backup. I had some russian cheaters on my server a few days ago, just banned them and restored one of the backups exactly before they joined

But there i noticed that every backup had atleast 2.5gb in size, first backup from 19.02.2016 every hour ... but it only has a total of ~53gb currently. I dont see any linking when i ll the directory, so whats going on there ?

 

Maybe this just causes my next error message:

 

2016-03-20T21:52:42 122267.693 INF GMSG: Devidian joined the game2016-03-20T21:52:42 122267.693 INF Player connected, entityid=171, name=Devidian, steamid=76561197972223708, ip=37.24.153.246
2016-03-20T21:52:42 122267.693 INF Player set to online: 76561197972223708
2016-03-20T21:52:44 122269.979 ERR Could not save file '/home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map': Sharing violation on path /home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/765
61197972223708.map
2016-03-20T21:52:44 122269.979 EXC Sharing violation on path /home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map
IOException: Sharing violation on path /home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map
 at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) [0x00000] in <filename unknown>:0
 at System.IO.FileStream..ctor (System.String path, FileMode mode) [0x00000] in <filename unknown>:0
 at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode)
 at DatabaseWithFixedDS`2[system.Int32,System.UInt16[]].Save (System.String _dir, System.String _filename) [0x00000] in <filename unknown>:0
UnityEngine.Debug:Internal_LogException(Exception, Object)
UnityEngine.Debug:LogException(Exception)
Logger:masterLogException(Exception)
Logger:Exception(Exception)
Log:Exception(Exception)
DatabaseWithFixedDS`2:Save(String, String)
MapChunkDatabase:SaveAsync(TaskInfo)
ThreadManager:S(Object)


(Filename:  Line: -1)

 

This doesn't seem to crash the server or anything, just noticed this in the logs.

File Permissions are as follows:

 

-rw-r--r-- 1 sdtd sdtd 3,3K Mنr 20 10:39 76561198291405125.ttp

 

(just took a random file, all have the same user/permissions)

 

 

=============

I had one crash again, while the mentioned cheaters where online.

 

[s_API FAIL] SteamAPI_Init() failed; SteamAPI_IsSteamRunning() failed.Setting breakpad minidump AppID = 251570


Native stacktrace:


       /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x8960f) [0xf6dfe60f]
       /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x21a20) [0xf6d96a20]
       [0xf7750420]
       /home/sdtd/engine/7DaysToDieServer.x86() [0x8f35fb2]
       /home/sdtd/engine/7DaysToDieServer.x86() [0x8f286e6]
       /home/sdtd/engine/7DaysToDieServer.x86() [0x8f28c32]
       /home/sdtd/engine/7DaysToDieServer.x86() [0x8f2bde9]
       /lib32/libpthread.so.0(+0x6f59) [0xf772df59]
       /lib32/libc.so.6(clone+0x5e) [0xf750fc4e]


Debug info from gdb:


Child Domain

 

Also noticed some other unnormal logs:

 

2016-03-19T06:13:35 22411.718 INF AIDirector: wandering horde zombie '[type=EntityZombie, name=zombieArlene, id=117886]' reached pitstop and will wander around for awhile looking for trouble.Failed setting triangles. Some indices are referencing out of bounds vertices. IndexCount: 744, VertexCount: 161


(Filename:  Line: 1158)


2016-03-19T06:13:48 22424.129 INF AIDirector: wandering horde zombie '[type=EntityZombie, name=zombieNurse, id=117778]' is being removed from horde control.
Failed setting triangles. Some indices are referencing out of bounds vertices. IndexCount: 69, VertexCount: 219


(Filename:  Line: 1158)


2016-03-19T06:13:50 22426.708 INF Removing observed entity 40

 

2016-03-19T06:30:06 23402.768 INF Player with ID 115707 has stack for "shotgunShell" greater than allowed (1767 > 250)Couldn't send RPC function 'RPC_RawData'




(Filename:  Line: 2480)


Couldn't send RPC function 'RPC_RawData'




(Filename:  Line: 2480)


2016-03-19T06:30:32 23428.730 INF [NET] PlayerDisconnected EntityID=115039, PlayerID='76561198204006159', OwnerID='76561198204006159', PlayerName='Moscow'

 

 

2016-03-19T09:35:59 34555.681 INF Time: 572.85m FPS: 43.81 Heap: 2837.0MB Max: [b]2967.0MB[/b] Chunks: 1050 CGO: 48 Ply: 3 Zom: 3 Ent: 9 (308) Items: 291 CO: 32016-03-19T09:36:11 34567.713 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250)
2016-03-19T09:36:11 34567.716 INF Player with ID 115039 has stack for "wood" greater than allowed (15899 > 1000)
2016-03-19T09:36:11 34567.717 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500)
2016-03-19T09:36:11 34567.717 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000)
2016-03-19T09:36:11 34567.718 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500)
OutOfMemoryException: Out of memory
 at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int)
 at NetConnectionRPC..ctor (Int32 _channel, .ClientInfo _clientInfo, INetworkClient _netClient) [0x00000] in <filename unknown>:0
 at NetworkServerUnity.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0
 at Steam.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0


(Filename:  Line: -1)


2016-03-19T09:36:29 34585.784 INF Time: 573.35m FPS: 51.00 Heap: 2679.8MB Max: [b]2967.0MB[/b] Chunks: 1050 CGO: 33 Ply: 3 Zom: 1 Ent: 6 (301) Items: 291 CO: 3
2016-03-19T09:36:41 34597.761 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250)
2016-03-19T09:36:41 34597.761 INF Player with ID 115039 has stack for "wood" greater than allowed (15899 > 1000)
2016-03-19T09:36:41 34597.761 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500)
2016-03-19T09:36:41 34597.762 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000)
2016-03-19T09:36:41 34597.762 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500)
2016-03-19T09:36:51 34607.072 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250)
2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "wood" greater than allowed (15899 > 1000)
2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500)
2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000)
2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500)
2016-03-19T09:36:59 34615.776 INF Time: 573.85m FPS: 53.78 Heap: [b]2870.6MB[/b] Max: 2967.0MB Chunks: 1050 CGO: 33 Ply: 3 Zom: 1 Ent: 7 (302) Items: 291 CO: 3
2016-03-19T09:37:21 34637.159 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250)
2016-03-19T09:37:21 34637.159 INF Player with ID 115039 has stack for "wood" greater than allowed (15879 > 1000)
2016-03-19T09:37:21 34637.160 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500)
2016-03-19T09:37:21 34637.160 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000)
2016-03-19T09:37:21 34637.160 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500)
2016-03-19T09:37:29 34645.845 INF Time: 574.35m FPS: 46.92 Heap: 2655.8MB Max: 2967.0MB Chunks: 1050 CGO: 40 Ply: 3 Zom: 1 Ent: 7 (302) Items: 291 CO: 3
2016-03-19T09:37:40 34656.207 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250)
2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "wood" greater than allowed (15879 > 1000)
2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500)
2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000)
2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500)
OutOfMemoryException: Out of memory
 at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int)
 at NetConnectionRPC..ctor (Int32 _channel, .ClientInfo _clientInfo, INetworkClient _netClient) [0x00000] in <filename unknown>:0
 at NetworkServerUnity.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0
 at Steam.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0


(Filename:  Line: -1)


2016-03-19T09:37:54 34670.263 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250)
2016-03-19T09:37:54 34670.265 INF Player with ID 115039 has stack for "wood" greater than allowed (15879 > 1000)
2016-03-19T09:37:54 34670.265 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500)
2016-03-19T09:37:54 34670.265 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000)
2016-03-19T09:37:54 34670.266 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500)
2016-03-19T09:37:59 34675.984 INF Time: 574.85m FPS: 55.46 Heap: 2632.8MB Max: [b]2967.0MB[/b] Chunks: 1050 CGO: 33 Ply: 3 Zom: 1 Ent: 9 (302) Items: 291 CO: 3

 

here you can see an OOM Exception while a friend of mine is trieing to connect - the Max here is less than 3GB so why out of memory, 32Bit ? We already played with 6 player on the same server without any problems, the cheaters were only 4.

Maybe the excessive cheating is the reason?

 

That was it for now

 

Greetings

Devidian

[Alloc]

64-bit

http://pastebin.com/HRrNMNCW

The actual error is in line 103. No idea why this is happening though, might be related to the still unknown occasional crashes on Unity 5.2.* builds on Linux. Sorry, no further idea right now. But at least it will most likely get better with A14

 

 

 

 

But there i noticed that every backup had atleast 2.5gb in size, first backup from 19.02.2016 every hour ... but it only has a total of ~53gb currently. I dont see any linking when i ll the directory, so whats going on there ?

That's because the backup (done by rsync) uses hard links. Basically all files that are identical between two (or more) consecutive backups share the same disk area that's why the total backup size doesn't increase as much as it would with all new copies each time

 

 

 

Maybe this just causes my next error message:

...
2016-03-20T21:52:44 122269.979 ERR Could not save file '/home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map': Sharing violation on path /home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map
...

This doesn't seem to crash the server or anything, just noticed this in the logs.

This is indeed not critical for the most part (worst case you might lose some of the player map exploration progress) and definitely won't cause any stability issues with the server.

 

 

 

I had one crash again, while the mentioned cheaters where online.

 

Native stacktrace:
       /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x8960f) [0xf6dfe60f]
...

Most likely related to the OOM later on.

 

 

 

Also noticed some other unnormal logs:

 

2016-03-19T06:13:35 22411.718 INF AIDirector: wandering horde zombie '[type=EntityZombie, name=zombieArlene, id=117886]' reached pitstop and will wander around for awhile looking for trouble.
Failed setting triangles. Some indices are referencing out of bounds vertices. IndexCount: 744, VertexCount: 161

2016-03-19T06:13:48 22424.129 INF AIDirector: wandering horde zombie '[type=EntityZombie, name=zombieNurse, id=117778]' is being removed from horde control.
Failed setting triangles. Some indices are referencing out of bounds vertices. IndexCount: 69, VertexCount: 219

Probably not dramatic either but we'll have a look.

 

 

 

2016-03-19T06:30:06 23402.768 INF Player with ID 115707 has stack for "shotgunShell" greater than allowed (1767 > 250)
Couldn't send RPC function 'RPC_RawData'
...
2016-03-19T06:30:32 23428.730 INF [NET] PlayerDisconnected EntityID=115039, PlayerID='76561198204006159', OwnerID='76561198204006159', PlayerName='Moscow'

First line: "Anti-cheat" detection of my mod (player probably used an exploit or just happened to stumble on a bug so he got a stack of an item with a stack size that's not achievable normally).

"Couldn't send": No harm, player disconnected so the server couldn't send some data.

 

 

 

...
OutOfMemoryException: Out of memory
...

This is where it starts to get completely irrelevant: If there's an OOM everything after that is basically just random and the server's no longer in a stable state (and typically crashes at the same instant).

 

 

 

the Max here is less than 3GB so why out of memory, 32Bit ?

What the info line shows is just Mono's heap but Unity uses some more resources for it's native stuff so you were more likely at 3.5 GiB or something and typically the limit for 32 bit processes is somewhere around 3.5 to 3.8 GiB.

 

 

Maybe the excessive cheating is the reason?

Could have amplified the problems but in the end it's just hitting the memory limit for whatever reason after some playtime

 

 

Regards,

Chris

[Devidian]

Oh yeha well, forgot linux has something like hardlinks That makes sense.

 

The server was fresh started at night so they really hit the memory limit fast (about 2 hours they where online) normally the server runs days until i restart it because of logs going above 30k.

 

Would be cool if the server fixes would recognize OOM and restart the server automatically.

Just saw that i had overcommitmemory at 0, changed it to 1, maybe that helps over short pikes.

 

Greetings,

Devidian

 

PS: what about a buff that everybody gets automatically who has greater stack sizes than allowed - it should set speed to 0 until there is no more stack above limit

[Alloc]

Would be cool if the server fixes would recognize OOM and restart the server automatically.

The fixes are part of the server process, they can't do anything like that. The scripts on the other hand could but as there's not even a guarantee to what shows up in the log on such occasions it doesn't make a lot of sense either. Easiest is still just creating a cron job that regularly "starts" the server as it will only do anything if the server is actually no longer running.

 

 

PS: what about a buff that everybody gets automatically who has greater stack sizes than allowed - it should set speed to 0 until there is no more stack above limit

Something like that might be possible as an option in the future. Combined with a warning to the user this actually sounds quite interesting

[Devidian]

Easiest is still just creating a cron job that regularly "starts" the server as it will only do anything if the server is actually no longer running.

 

This would start the server again yes, but the "dead" instance would still be running as it always does when i look at the processes. I always had to manually kill the old process as it still consumes memory, before i restart.

 

Thats one reason why i thought it may be good to outsource the web-component and just have a sort of websocket api. If the api isn't responding for a time but the process is running, the web component could kill the process and restart or atleast give information on the frontend with a button to kill/restart.

 

I'm currently working on a project with meteor which runs with node/mongodb and has some really cool features like realtime sync with the mongodb<->frontend. So if there would be just a possibility that the game itself publishes all data to a mongodb, another app (must not be meteor) could fetch those data. The only thing to manage then would be an api to pass commands to the server (this could also be done by connecting to telnet i think)

[SylenThunder]

I am at a complete loss here. I do not understand what has gone wrong at all.

 

I updated everything, I re-created my world. I get...

root@7DTD:~# 7dtd.sh start Home
Failed!
root@7DTD:~# 

 

SO I figure it's something in the much tinkering I did with 64-bit or Ubuntu 15 when I was playing with a13.8.

 

I wipe the server completely.

Install Ubuntu server 14 LTS.

Install the server using Alloc's scripts.

Create the instance.

Edit the config.xml in /home/sdtd/instances/Home/config.xml to remove the loot and craft timer lines.

Start the server.

root@7DTD:~# 7dtd.sh start Home
Failed!
root@7DTD:~# 

 

We're as vanilla as it gets here, and I cannot for the life of me get this thing to start.

 

 

Any ideas?

[palmic]

Check /home/sdtd if you have engine there,

If not:

7dtd.sh updatescripts && 7dtd.sh updatefixes && 7dtd.sh updateengine

[palmic]

--already fixed in this thread, please delete me, author--

Edited March 27, 2016 by palmic
fix (see edit history)

[SylenThunder]

yeah, it's there. ran all of the commands before starting, just to make sure everything was updated.

 

no dice.

 

And no log is generated since it takes maybe 2 seconds to fail.

[palmic]

--already fixed in this thread, please delete me, author--

Edited March 27, 2016 by palmic (see edit history)

[SylenThunder]

Fully upgraded debian, full https://7dtd.illy.bz/ rewipe, fresh new instance cannot start.

"ERR Error parsing configfile property 'CraftTimer': Could not parse config value '1'"

(this is generated by server management script)

log output

 

HOTFIX: remove nodes with CraftTimer & LootTimer variables from config.xml

From my first post on the issue

 

...

Edit the config.xml in /home/sdtd/instances/Home/config.xml to remove the loot and craft timer lines.

...

 

I'm still tinkering, but it's still not making any progress. If it would load enough to create some kind of log, that would be really helpful. >.<

[palmic]

Bwahah.. ok, iam deleting my post

 

Try to call runtime manually:

/home/sdtd/engine/7DaysToDieServer.x86_64 -logfile /home/sdtd/instances/$instancename/logs/2016-03-27_16-21-08_output_log.txt -configfile=/home/sdtd/instances/$instancename/config.xml