Devidian
-
Posts
80 -
Joined
-
Days Won
1
Content Type
Profiles
Forums
Store
Articles
A20 Bugs
A21 Bugs
Events
Posts posted by Devidian
-
-
Easiest is still just creating a cron job that regularly "starts" the server as it will only do anything if the server is actually no longer running.
This would start the server again yes, but the "dead" instance would still be running as it always does when i look at the processes. I always had to manually kill the old process as it still consumes memory, before i restart.
Thats one reason why i thought it may be good to outsource the web-component and just have a sort of websocket api. If the api isn't responding for a time but the process is running, the web component could kill the process and restart or atleast give information on the frontend with a button to kill/restart.
I'm currently working on a project with meteor which runs with node/mongodb and has some really cool features like realtime sync with the mongodb<->frontend. So if there would be just a possibility that the game itself publishes all data to a mongodb, another app (must not be meteor) could fetch those data. The only thing to manage then would be an api to pass commands to the server (this could also be done by connecting to telnet i think)
0 -
Oh yeha well, forgot linux has something like hardlinks
That makes sense.The server was fresh started at night so they really hit the memory limit fast (about 2 hours they where online) normally the server runs days until i restart it because of logs going above 30k.
Would be cool if the server fixes would recognize OOM and restart the server automatically.
Just saw that i had overcommitmemory at 0, changed it to 1, maybe that helps over short pikes.
Greetings,
Devidian
PS: what about a buff that everybody gets automatically who has greater stack sizes than allowed - it should set speed to 0 until there is no more stack above limit
0 -
Hi there,
first of all - good that your fixes do some backup. I had some russian cheaters on my server a few days ago, just banned them and restored one of the backups exactly before they joined
But there i noticed that every backup had atleast 2.5gb in size, first backup from 19.02.2016 every hour ... but it only has a total of ~53gb currently. I dont see any linking when i ll the directory, so whats going on there ?
Maybe this just causes my next error message:
2016-03-20T21:52:42 122267.693 INF GMSG: Devidian joined the game2016-03-20T21:52:42 122267.693 INF Player connected, entityid=171, name=Devidian, steamid=76561197972223708, ip=37.24.153.246 2016-03-20T21:52:42 122267.693 INF Player set to online: 76561197972223708 2016-03-20T21:52:44 122269.979 ERR Could not save file '/home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map': Sharing violation on path /home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/765 61197972223708.map 2016-03-20T21:52:44 122269.979 EXC Sharing violation on path /home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map IOException: Sharing violation on path /home/sdtd/instances/OZCOOPII/Random Gen/A13.6OMEGAROCKZ/Player/76561197972223708.map at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) [0x00000] in <filename unknown>:0 at System.IO.FileStream..ctor (System.String path, FileMode mode) [0x00000] in <filename unknown>:0 at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode) at DatabaseWithFixedDS`2[system.Int32,System.UInt16[]].Save (System.String _dir, System.String _filename) [0x00000] in <filename unknown>:0 UnityEngine.Debug:Internal_LogException(Exception, Object) UnityEngine.Debug:LogException(Exception) Logger:masterLogException(Exception) Logger:Exception(Exception) Log:Exception(Exception) DatabaseWithFixedDS`2:Save(String, String) MapChunkDatabase:SaveAsync(TaskInfo) ThreadManager:S(Object) (Filename: Line: -1)
This doesn't seem to crash the server or anything, just noticed this in the logs.
File Permissions are as follows:
-rw-r--r-- 1 sdtd sdtd 3,3K Mنr 20 10:39 76561198291405125.ttp
(just took a random file, all have the same user/permissions)
=============
I had one crash again, while the mentioned cheaters where online.
[s_API FAIL] SteamAPI_Init() failed; SteamAPI_IsSteamRunning() failed.Setting breakpad minidump AppID = 251570 Native stacktrace: /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x8960f) [0xf6dfe60f] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x21a20) [0xf6d96a20] [0xf7750420] /home/sdtd/engine/7DaysToDieServer.x86() [0x8f35fb2] /home/sdtd/engine/7DaysToDieServer.x86() [0x8f286e6] /home/sdtd/engine/7DaysToDieServer.x86() [0x8f28c32] /home/sdtd/engine/7DaysToDieServer.x86() [0x8f2bde9] /lib32/libpthread.so.0(+0x6f59) [0xf772df59] /lib32/libc.so.6(clone+0x5e) [0xf750fc4e] Debug info from gdb: Child DomainAlso noticed some other unnormal logs:
2016-03-19T06:13:35 22411.718 INF AIDirector: wandering horde zombie '[type=EntityZombie, name=zombieArlene, id=117886]' reached pitstop and will wander around for awhile looking for trouble.Failed setting triangles. Some indices are referencing out of bounds vertices. IndexCount: 744, VertexCount: 161 (Filename: Line: 1158) 2016-03-19T06:13:48 22424.129 INF AIDirector: wandering horde zombie '[type=EntityZombie, name=zombieNurse, id=117778]' is being removed from horde control. Failed setting triangles. Some indices are referencing out of bounds vertices. IndexCount: 69, VertexCount: 219 (Filename: Line: 1158) 2016-03-19T06:13:50 22426.708 INF Removing observed entity 40
2016-03-19T06:30:06 23402.768 INF Player with ID 115707 has stack for "shotgunShell" greater than allowed (1767 > 250)Couldn't send RPC function 'RPC_RawData' (Filename: Line: 2480) Couldn't send RPC function 'RPC_RawData' (Filename: Line: 2480) 2016-03-19T06:30:32 23428.730 INF [NET] PlayerDisconnected EntityID=115039, PlayerID='76561198204006159', OwnerID='76561198204006159', PlayerName='Moscow'
2016-03-19T09:35:59 34555.681 INF Time: 572.85m FPS: 43.81 Heap: 2837.0MB Max: [b]2967.0MB[/b] Chunks: 1050 CGO: 48 Ply: 3 Zom: 3 Ent: 9 (308) Items: 291 CO: 32016-03-19T09:36:11 34567.713 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250) 2016-03-19T09:36:11 34567.716 INF Player with ID 115039 has stack for "wood" greater than allowed (15899 > 1000) 2016-03-19T09:36:11 34567.717 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500) 2016-03-19T09:36:11 34567.717 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000) 2016-03-19T09:36:11 34567.718 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500) OutOfMemoryException: Out of memory at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int) at NetConnectionRPC..ctor (Int32 _channel, .ClientInfo _clientInfo, INetworkClient _netClient) [0x00000] in <filename unknown>:0 at NetworkServerUnity.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0 at Steam.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0 (Filename: Line: -1) 2016-03-19T09:36:29 34585.784 INF Time: 573.35m FPS: 51.00 Heap: 2679.8MB Max: [b]2967.0MB[/b] Chunks: 1050 CGO: 33 Ply: 3 Zom: 1 Ent: 6 (301) Items: 291 CO: 3 2016-03-19T09:36:41 34597.761 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250) 2016-03-19T09:36:41 34597.761 INF Player with ID 115039 has stack for "wood" greater than allowed (15899 > 1000) 2016-03-19T09:36:41 34597.761 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500) 2016-03-19T09:36:41 34597.762 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000) 2016-03-19T09:36:41 34597.762 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500) 2016-03-19T09:36:51 34607.072 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250) 2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "wood" greater than allowed (15899 > 1000) 2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500) 2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000) 2016-03-19T09:36:51 34607.073 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500) 2016-03-19T09:36:59 34615.776 INF Time: 573.85m FPS: 53.78 Heap: [b]2870.6MB[/b] Max: 2967.0MB Chunks: 1050 CGO: 33 Ply: 3 Zom: 1 Ent: 7 (302) Items: 291 CO: 3 2016-03-19T09:37:21 34637.159 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250) 2016-03-19T09:37:21 34637.159 INF Player with ID 115039 has stack for "wood" greater than allowed (15879 > 1000) 2016-03-19T09:37:21 34637.160 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500) 2016-03-19T09:37:21 34637.160 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000) 2016-03-19T09:37:21 34637.160 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500) 2016-03-19T09:37:29 34645.845 INF Time: 574.35m FPS: 46.92 Heap: 2655.8MB Max: 2967.0MB Chunks: 1050 CGO: 40 Ply: 3 Zom: 1 Ent: 7 (302) Items: 291 CO: 3 2016-03-19T09:37:40 34656.207 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250) 2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "wood" greater than allowed (15879 > 1000) 2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500) 2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000) 2016-03-19T09:37:40 34656.212 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500) OutOfMemoryException: Out of memory at (wrapper managed-to-native) object:__icall_wrapper_mono_array_new_specific (intptr,int) at NetConnectionRPC..ctor (Int32 _channel, .ClientInfo _clientInfo, INetworkClient _netClient) [0x00000] in <filename unknown>:0 at NetworkServerUnity.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0 at Steam.OnPlayerConnected (NetworkPlayer _networkPlayer) [0x00000] in <filename unknown>:0 (Filename: Line: -1) 2016-03-19T09:37:54 34670.263 INF Player with ID 115039 has stack for "shotgunShell" greater than allowed (1126 > 250) 2016-03-19T09:37:54 34670.265 INF Player with ID 115039 has stack for "wood" greater than allowed (15879 > 1000) 2016-03-19T09:37:54 34670.265 INF Player with ID 115039 has stack for "forgedSteel" greater than allowed (1660 > 500) 2016-03-19T09:37:54 34670.265 INF Player with ID 115039 has stack for "gasCan" greater than allowed (9600 > 1000) 2016-03-19T09:37:54 34670.266 INF Player with ID 115039 has stack for "forgedIron" greater than allowed (994 > 500) 2016-03-19T09:37:59 34675.984 INF Time: 574.85m FPS: 55.46 Heap: 2632.8MB Max: [b]2967.0MB[/b] Chunks: 1050 CGO: 33 Ply: 3 Zom: 1 Ent: 9 (302) Items: 291 CO: 3
here you can see an OOM Exception while a friend of mine is trieing to connect - the Max here is less than 3GB so why out of memory, 32Bit ? We already played with 6 player on the same server without any problems, the cheaters were only 4.
Maybe the excessive cheating is the reason?
That was it for now
Greetings
Devidian
0 -
Have you been monitoring the memory consumption / size of the virtual address space of the process? If you don't get this soon after starting the server but only later on when people have played on it and you only access the web panel after that for the first time it is probably just a memory issue.
Maybe i should install my node monitoring app that i developed for my company's cloud for memory monitoring... but i never saw memory (virt) above 3GB. The Last Entry before the crash was:
2016-02-12T02:25:20 232914.094 INF Time: 3873.49m FPS: 52.37 Heap: 1001.7MB Max: 2541.2MB Chunks: 1508 CGO: 2 Ply: 0 Zom: 0 Ent: 0 (1154) Items: 716 CO: 1
Ah, ok, gotcha. Yeah, of course this could be done. But I don't see that much of an audience that would benefit from this. Or maybe there would be in terms of server managers or something. Would require a lot of extra time though
But of course it's open to everyone to have a go at implementing something like that
For multi-managers for example yeah. The benefit may be that you would just have to make the api and let the community create several different forms of web-managements. Ok sure at first there may be a little bit more time for you to spend on the web-sockets api, but as soon as it works you could easily add new features and the developers from the community could add it too their web interfaces.
Servers that are hosting multiple instances also wouldn't have the webserver overhead on each instance.
0 -
I am also running the dedi mods and the start.sh calls the 32 bit version. Is there a reason for that? I am running Ubuntu Server 64, and don't know if I should just edit the start.sh to call the 64 bit version or not. Will that effect anything? What is the major diffrence between the 32 and 64 versions other than mem use?
Do you still have to edit the startdedicated.sh in the /home/sdtd/engine/ dir?
This is a good question, mine is starting up in 32 Bit by default also:
2016-02-25T09:15:04 0.055 INF Awake2016-02-25T09:15:04 0.128 INF Version: Alpha 13.8 (b7) Compatibility Version: Alpha 13.8, Build: Linux 32 Bit 2016-02-25T09:15:04 0.129 INF System information: 2016-02-25T09:15:04 0.443 INF OS: Linux 3.13 Ubuntu 14.04 64bit 2016-02-25T09:15:04 0.445 INF CPU: Dual-Core AMD Opteron(tm) Processor 1214 HE (cores: 2) 2016-02-25T09:15:04 0.445 INF RAM: 3936 MB 2016-02-25T09:15:04 0.445 INF GPU: Null Device (128 MB) 2016-02-25T09:15:04 0.451 INF Graphics API: NULL 1.0 [1.0] (shader level 2.0)
0 -
Ok now i'm home from office i may explain a little bit more what i would like to have.
The goal behind that is splitting web and game server, so if the game server stops working the webserver would still run and say "instance not running" if the webserver crashes on the other side, the game server is still running.
Another benefit would be having only 1 webserver that may handle multiple sources (gameserver) on the same machine.
There may be 2 ways i'm currently thinking how i would realize this. The first i call "data-dump" where the game-server just dumps all data to a special directory that could look like:
data\players.json // all exposed player data
data\entitys.json // all current alive entitys (zombies/animals)
data\map\* // exported map like currently (with the tiles as images, maybe as copy from the original exported map)
data\*.json // everything else to expose, like claims, logs, server (instance) configuration, player permission levels, etc
Then on the (external) web service there could be configured the base path to instances directory so it could automaticly scan for all status.json files in the instance data directory's or the pathes could manually configured. In node.js for example i could use file change watcher that always reload the data on any file changes. I wont go more into details here, because this isn't my preferred way.
=====
A Better solution would be what i call "web-socket-service" that may also handle user permissions. So the server runs something like socket.io for node.js (only built-in) where a web-service (either another backend process, or a frontend lib like socket.io) can connect to and register with the steam id. Then for example i could join rooms like "player.location" that pushes any player location changes on change to the room (where multiple user could be registred to) if i have the right permission level. Also i could with the right permissions emit commands to the server or join rooms like "chat" where i can follow the game chat live.
This way only the map tiles would need to be created and maybe copied (to not access the original files that frequently are write accessed by the game)
In both scenarios there wont be any web-server overhead per instance, just a way to push the data either to files or to web-sockets.
I hope my english was good enought to explain my thoughts - otherwise i prefer german
0 -
Are you running any mods on that server? Especially additional items / item icons?
No idea why it would crash in the destroy method unless it's already having memory issues at that point.
No, just vanilla. I have a xui mod running some weeks now but the crashes like this were happening before too. Not often but they happen.
You mean like the API it already has and the web frontend uses to get the required data?
ؤhm,.. wait... no. Currently it uses xhr requests (polling) not web-sockets. Web sockets is a more real time connection, the client gets a permananet connection and the server pushes any changed data. More like publish/subscribe...
More like (pseudocode)
ws = connect(localhost:port).
...
ws.on('playerList',dosomething);
..
ws.emit('subscribeToPlayerList');
...
edit: by the way, can't you configurate the web-server to push an empty transparent map tile on missing files ? the console 404 warnings ... (i dont like that
)I know how to configurate nginx to do that but as you use your own ...
0 -
The output regarding the certificate is "normal" behaviour. Also doubt the webserver part itself is directly responsible for the crash, more likely some sideeffect in combination with the core code or just something simple like OOM. Did you have a look at the *top* of the log file? Depending on what's causing problems the Linux client sometimes overwrites the content there with error messages.
You are right, the error is on top of the log file, funny
Here it is:
[s_API FAIL] SteamAPI_Init() failed; SteamAPI_IsSteamRunning() failed.Setting breakpad minidump AppID = 251570 Stacktrace: at (wrapper managed-to-native) UnityEngine.Object.Destroy (UnityEngine.Object,single) <0x00004> at (wrapper managed-to-native) UnityEngine.Object.Destroy (UnityEngine.Object,single) <0x00004> at UnityEngine.Object.Destroy (UnityEngine.Object) <0x00023> at AllocsFixes.NetConnections.Servers.Web.Handlers.ItemIconHandler.LoadIcons () <0x008bb> at AllocsFixes.NetConnections.Servers.Web.Handlers.ItemIconHandler.HandleRequest (System.Net.HttpListenerRequest,System.Net.HttpListenerResponse,AllocsFixes.NetConnections.Servers.Web.WebConnection,int) <0x00022> at AllocsFixes.NetConnections.Servers.Web.Web.HandleRequest (System.IAsyncResult) <0x003c0> at System.Net.ListenerAsyncResult.InvokeCallback (object) <0x0004f> at (wrapper runtime-invoke) object.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <0x00046> Native stacktrace: /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x8960f) [0xf6e6660f] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x21a63) [0xf6dfea63] [0xf77b8420] /home/sdtd/engine/libstdc++.so.6(_ZSt29_Rb_tree_insert_and_rebalancebPSt18_Rb_tree_node_baseS0_RS_+0x98) [0xf76fe9f8] /home/sdtd/engine/7DaysToDieServer.x86() [0x81fe106] /home/sdtd/engine/7DaysToDieServer.x86() [0x81fd8cf] /home/sdtd/engine/7DaysToDieServer.x86() [0x8201ef8] /home/sdtd/engine/7DaysToDieServer.x86() [0x85444b3] [0xf5cc33f0] [0xf5cc33ac] [0xead63d6c] [0xead632cb] [0xebc4d5a9] [0xebc4d1d0] [0xf6b440bf] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x25a1b) [0xf6e02a1b] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(mono_runtime_invoke+0x66) [0xf6f03aa5] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(mono_runtime_invoke_array+0x53b) [0xf6f084f6] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x12b822) [0xf6f08822] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x153ca4) [0xf6f30ca4] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x15413e) [0xf6f3113e] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x153b0d) [0xf6f30b0d] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x17d0fb) [0xf6f5a0fb] /home/sdtd/engine/7DaysToDieServer_Data/Mono/x86/libmono.so(+0x19e868) [0xf6f7b868] /lib32/libpthread.so.0(+0x6f59) [0xf7795f59] /lib32/libc.so.6(clone+0x5e) [0xf7577c4e] Debug info from gdb: ================================================================= Got a SIGSEGV while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= IP address: 127.0.0.1 IP address: 85.214.212.32 eac_server.so [x86] :: OnLoad() STEAMPS3 - AsyncTCPSocket created STEAMPS3 - AsncTCPSocket destroyed STEAMPS3 - AsyncTCPSocket created STEAMPS3 - AsncTCPSocket destroyedAnother thing, i thought about building my own WebInterface (with node.js/socket.io etc.) and how nice it would be to have a native game "web-sockets" like api that runs on localhost:xxx. So web-service and game service would be divided in 2 tiers. I could look into it myself and make my own mod but 1. its long time ago i was into C# (currently only in web based languages) and 2. it would be cool to have something like this from "official" side
What do you think about this ?
0 -
Hi,
my Server randomly crashes when connecting to the web interface. I know this some time now but didn't manage yet to copy the log file into the forum.
2016-02-22T17:00:50 265747.915 INF Steam certificate error: RemoteCertificateChainErrors 2016-02-22T17:00:51 265748.839 INF Steam OpenID login from 217.225.77.226:62797 with ID 76561197972223708, permission level 0
This are the last entry's before the server stops working. In htop its still running but with ...
7dtd.sh status OZCOOPII
it keeps telling its offline...
Instance: OZCOOPII Status: NOT running Game info: Server name: [DE] Omega-Zirkel [COOP][PUBLIC TS3][sIRMOD] Password: Max players: 8 World: Random Gen
Seems all network connections are gone away. Any idea ?
UPDATE:
After htop F9 and (re)start i connected to the web gui again and looked at the output log. The certificate error is also in there but not stopping the server.
2016-02-22T17:42:50 2239.520 INF Steam certificate error: RemoteCertificateChainErrors 2016-02-22T17:42:51 2240.490 INF Steam OpenID login from 217.225.77.226:64067 with ID 76561197972223708, permission level 0 2016-02-22T17:43:09 2258.718 INF Web:IconHandler: Icons loaded - 5961 ms
Greetings,
Devidian
0 -
So far no. I'm far from being a designer or anything like that so I have to search for fitting icons (both in terms of content as well as licensing) or it will end up ugly as hell
Also: Wrong thread.
Whops, sorry had multiple tabs open
Ok well, then i'll try myself editing this when i get some free time.
0 -
Hey Alloc,
are there any plans adding different icons for different animals/zombies on the map ?
Greetings,
Devidian
0
That makes sense.
Native Linux server (with management scripts)
in Multiplayer
Posted
The 2 Threads are confusing
Just a side question (not important): there is an issue with permanent hostiles on map even if no player is online, do you know about this ? Just saw this again yesterday(today/currently) on my servers map. I noticed this in A11 too but never mentioned this here.