Jump to content
Alloc

Native Linux server (with management scripts)

Recommended Posts

Hi there,

 

as it was suggested to me I open a new thread on my management scripts and native Linux server support. This is basically following up on the thread Dedicated Server without hacky workarounds though this is in Linux Bugs where it might be overlooked.

 

I created a full set of management scripts for a Linux based 7dtd server. This allows basic operations like starting/stopping as well as advanced things like a few event hooks.

 

(This also includes native Linux engine files so you do not have to use workarounds like Wine to run 7dtd. Unfortunately these files are 32 Bit only so far but at least they can be run on 64 Bit hosts without any problems.) No longer valid, 7dtd provides "built in" 32 bit and 64 bit Linux support since A14

 

For the full documentation on the management scripts go to https://7dtd.illy.bz/.

 

Regards,

Alloc

Edited by Alloc (see edit history)
  • Like 6

Share this post


Link to post
Share on other sites

Thank you so much for this, haven't even checked out the linux bugs forum as I assumed it would be empty. :) I was waiting for a native linux build so I could set up a server without having to deal with WINE or run a Windows server, but now I can start experimenting right away. Once again, thank you! :)

Share this post


Link to post
Share on other sites

Thank you very much! I have tried to setup a dedicated server with this on Ubuntu 14.04 and it works for the most part. I had to install some of the deps but thats ok. There is also a couple issues I am having which I was wondering if you could help with.

 

1. When I do 7dtd.sh updateengine I get some things that look like errors. This is the output: http://pastebin.com/PUWxQwbw

What's odd is that there is no /home/buildbot on my machine so I'm not sure why it shows that.

 

2. In the instance log I get this line a bunch: Fallback handler could not load library /home/sdtd/engine/7DaysToDie_Data/Mono/x86/SteamworksNative

I get some others as well that say they are harmless but this one doesn't say it's harmless. Plus when installing it said something about SteamworksNative.dll not being found.

 

3. I cannot get the server to show up in the public servers list even though I have set it as public. I can however connect to it if I add it manually to favourites.

 

I couldn't find anywhere which ports I should open. I researched and found that only ports 25000-25002 UDP are required. I ran this command: ufw allow 25000:25003/udp

I can connect with that but I still don't see it in the server list. I have tried doing: ufw allow 25000:25003/tcp

But that still doesn't work. I have also tried: ufw allow 35000:35003/udp and ufw allow 35000:35003/tcp

But that still doesn't work. When the server starts it shows using another port that randomly changes and port 27036 which never changes and I have tried allowing that through tcp and udp but it still doesn't work. So maybe you could give some straight up iptables rules to setup the server from scratch or something that makes it show up in the server list?

 

That's about all I have noticed so far. The only other thing I'm wondering is if this will be lower performance than a straight up windows server or maybe better. The server is running Ubuntu 14.04 and has 32 gigs of RAM, SSD drives and Intel Xeon E3 1245v2 4 core/8 thread processor. Do you have any recommended player limit to set for such a server?

 

Thanks again!

Share this post


Link to post
Share on other sites
1. When I do 7dtd.sh updateengine I get some things that look like errors. This is the output: http://pastebin.com/PUWxQwbw

What's odd is that there is no /home/buildbot on my machine so I'm not sure why it shows that.

The path shown in this error refers to the location Valve built this program. It has nothing to do with your system. Also it can be simply ignored, I don't know why it shows up on some systems (did also show on my Debian 6/Squeeze test machine, but neither on my deployment machine with Debian 7/Wheezy nor on my Fedora 20 test machine ...).

 

2. In the instance log I get this line a bunch: Fallback handler could not load library /home/sdtd/engine/7DaysToDie_Data/Mono/x86/SteamworksNative

Those entries are ok.

 

I get some others as well that say they are harmless but this one doesn't say it's harmless. Plus when installing it said something about SteamworksNative.dll not being found.

What do you mean by "when installing it"?

 

3. I cannot get the server to show up in the public servers list even though I have set it as public. I can however connect to it if I add it manually to favourites.

Sounds like you got a wrong combination of the Steamworks files. Could you pastebin the whole output.log from the instance you were trying to run?

 

I couldn't find anywhere which ports I should open. I researched and found that only ports 25000-25002 UDP are required. I ran this command: ufw allow 25000:25003/udp

I can connect with that but I still don't see it in the server list. I have tried doing: ufw allow 25000:25003/tcp

But that still doesn't work. I have also tried: ufw allow 35000:35003/udp and ufw allow 35000:35003/tcp

But that still doesn't work. When the server starts it shows using another port that randomly changes and port 27036 which never changes and I have tried allowing that through tcp and udp but it still doesn't work. So maybe you could give some straight up iptables rules to setup the server from scratch or something that makes it show up in the server list?

I only have the following ports:

- in: 25000-25002/UDP

- out: 20,21,80,443/TCP

- (and DNS, NTP but that shouldn't matter ;) )

I think it uses HTTP for the server list so port 80 should be enough.

 

That's about all I have noticed so far. The only other thing I'm wondering is if this will be lower performance than a straight up windows server or maybe better. The server is running Ubuntu 14.04 and has 32 gigs of RAM, SSD drives and Intel Xeon E3 1245v2 4 core/8 thread processor. Do you have any recommended player limit to set for such a server?

I don't know about a comparison to a Windows based server but I'd suspect it to perform a little bit better as most things perform better on Linux regarding raw CPU / IO power and you obviously don't need graphics power on the server ;)

Also don't have any numbers for required resources for a given player count, only hosting a server for me and two friends.

 

Regards,

Alloc

Share this post


Link to post
Share on other sites

Hi there,

 

By installing I just meant when running the boostrap.sh file.

 

I just created a brand new instance just to test and it has the exact same problem with not showing on the public servers list. I just tested and I can still connect to it. So maybe it's something messed with the files like you say but I don't know what. I know I had some issues with Steamguard at the start and had to disable it so I could even get the updateengine part to login to Steam. Maybe that somehow messed it up?

 

Here is the output.log from after I started the instance: http://pastebin.com/9KKxHm6D

 

I have now allowed port 80 and it makes no difference so I don't think that's it.

 

If I could somehow uninstall and start over that would be ok if that's what has to happen for it to work. I had it do backups. I'm just not 100% sure which files to remove and whatnot because I'm pretty sure it has put some in other places on the server.

 

Also, if you have a donation button somewhere for Paypal I would send you a donation =)

 

Thanks!

Share this post


Link to post
Share on other sites

Some of the existing scripts were a bit too "Ubunty-ese" specially the service/deamon commands. That is one area on improvement. Personally I did not bother to do it as pims promised that native Linux servers was "almost ready" though that statement last a lot of credibility over the past months...

Share this post


Link to post
Share on other sites
By installing I just meant when running the boostrap.sh file.

Hm, shouldn't complain about Steamworks at that point as 7dtd is not run during bootstrap. If you should rerun bootstrap and get that again please show me the log :)

 

I just created a brand new instance just to test and it has the exact same problem with not showing on the public servers list. I just tested and I can still connect to it. So maybe it's something messed with the files like you say but I don't know what. I know I had some issues with Steamguard at the start and had to disable it so I could even get the updateengine part to login to Steam. Maybe that somehow messed it up?

Should not interfere here. Also disabling Steamguard should not be required, you should simply get a mail with a confirmation code which has to be entered during the update-process. (Have done so like 30 times the last week ;) ) If there's any error showing please also send me a copy of that.

 

Here is the output.log from after I started the instance: http://pastebin.com/9KKxHm6D

We've got the problem in line 96:

2014.06.04 03:25:50: EntryPointNotFoundException: GameServer_LoggedOn

This should not happen anymore ... When did you first run the bootstrapper? After I created this thread or before?

 

 

If I could somehow uninstall and start over that would be ok if that's what has to happen for it to work. I had it do backups. I'm just not 100% sure which files to remove and whatnot because I'm pretty sure it has put some in other places on the server.

Basically you could delete everything within the users folder (but not the backups-folder if you used that of course ;) ). SteamCMD probably also created a folder in the root's home, namely /root/Steam. You can also simply rerun the bootstrapper without deleting anything which will skip redownloading SteamCMD and 7dtd but do the other steps of the installation again.

 

Also, if you have a donation button somewhere for Paypal I would send you a donation =)

Thank you, I really appreciate that. Still I'm doing this for free because I myself use this and I also refresh/deepen my knowledge on Bash scripting ;)

 

 

@Mike:

Some of the existing scripts were a bit too "Ubunty-ese" specially the service/deamon commands.

To be a bit nit-picky it was made for Debian in the first place ;)

Obviously that should make it run mostly without problems on Ubuntu and other Debian derivatives too.

 

But by now I'm also testing the whole thing on Fedora so it should work on most systems. Basically the only thing I don't check is the startup of the server at system boot time but according to the information on the net systemd can use sysvinit-style scripts so it shouldn't be any problem either.

 

The only thing that I really have it depend on is a Bash and of course 32 Bit libraries (gcc, ld-lib) which is simply caused by the fact that SteamCMD is not available as 64 Bit executable.

 

Personally I did not bother to do it as pims promised that native Linux servers was "almost ready" though that statement last a lot of credibility over the past months...

Hm ... "over the past months"? Unfortunately it's more like "over the past years" by now ;)

But as long as we're able to do this stuff ourselves I won't complain, they got other things to do too.

 

 

Regards,

Alloc

Share this post


Link to post
Share on other sites

Thank you very much for your help! I got it to work by deleting everything and starting over. I deleted those folders you mentioned and the stuff in /tmp that I found. Woohoo! =)

Share this post


Link to post
Share on other sites

As I thought probably just an old file somewhere :)

 

Glad it works for you now.

Share this post


Link to post
Share on other sites

Great news and excellent timing for these scripts as far as I am concerned.

I have been running a 7dtd server on Debian 7.4 since Alpha 5.

 

In Alpha 8.4 and 8.5 I have had very frequent server crashes. I've gone over the hardware to ensure there is no problem there, all is good.

Time to wipe the files and start fresh I think. So thank you for the effort and time you have put into this, I for one appreciate it.

Share this post


Link to post
Share on other sites

If you use the instance edit feature it's called "Item spawn menu" under "Generic Options".

Share this post


Link to post
Share on other sites

server runs very stable no crashes but telnet seems to be buggy

 

SocketException: The socket has been shut down

at System.Net.Sockets.Socket.Send (System.Byte[] buf, Int32 offset, Int32 size, SocketFlags flags) [0x00000] in <filename unknown>:0

at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0

Rethrow as IOException: Write failure

at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0

at System.IO.Stream.WriteByte (Byte value) [0x00000] in <filename unknown>:0

at NetTelnetServer.WriteToClient (System.String _line) [0x00000] in <filename unknown>:0

at ហ.ᡤᙃ (ᡡᙃ ᙂ) [0x00000] in <filename unknown>:0

at ហ.ᝉᙂ (System.String _line) [0x00000] in <filename unknown>:0

at ConnectionManager.RPC_ServerCommand (System.String _playerName, System.String _cmd, NetworkMessageInfo _networkInfo) [0x00000] in <filename unknown>:0

 

(Filename: Line: -1)

 

im using frontrunnerteks servermanager maybe the manager crash the telnet server i dont know

Share this post


Link to post
Share on other sites

Hm, the only way I ever got telnet to crash was when I did not properly disconnect, i.e. closing the connection instead of letting the server close it by sending the exit command (this *always* made it crash for me). Don't know what FRT does with his SM though ;)

Share this post


Link to post
Share on other sites
Hm, the only way I ever got telnet to crash was when I did not properly disconnect, i.e. closing the connection instead of letting the server close it by sending the exit command (this *always* made it crash for me). Don't know what FRT does with his SM though ;)

 

not a big deal i change to webconnect.

but can anyone tryout the blacklist it does not work under linux im trying to ban myself but no luck, its not a config problem the same config under windows works got the message youre banned ....

 

<blacklisted steamID="765611979xxxxxxxx" unbandate="5/10/2015 12:21:07 AM"/>

 

TY

Share this post


Link to post
Share on other sites
<blacklisted steamID="765611979xxxxxxxx" unbandate="5/10/2015 12:21:07 AM"/>

Looks like TFP did not explicitly specify how that date should be formatted so it's different in the Linux build. Maybe even depending on the systems locale.

With a german locale it looks like this:

<blacklisted steamID="76561198xxxxxx" unbandate="05.06.2014 01:56:34" />

Share this post


Link to post
Share on other sites
Hm, the only way I ever got telnet to crash was when I did not properly disconnect, i.e. closing the connection instead of letting the server close it by sending the exit command (this *always* made it crash for me). Don't know what FRT does with his SM though ;)

 

Yeah, now it does that. In early 8.* telnet would stop letting me connect - I could connect the first time but if i didn't properly `exit` it wouldn't let me connect again.

Share this post


Link to post
Share on other sites

I only have the following ports:

- in: 25000-25002/UDP

- out: 20,21,80,443/TCP

- (and DNS, NTP but that shouldn't matter ;) )

I think it uses HTTP for the server list so port 80 should be enough.

 

You don't need the outgoing http(s) ports. I didn't found usage of 20 and 21.

You have to allow the outgoing source query engine ports around 25000 to list your server. I can deliever exact ports in the next days.

Share this post


Link to post
Share on other sites

20/21 is listed because those are open by my firewall for updates ;)

I just listed everything that was open. Though it looks like I got the wrong state (tried with different port combinations). Looks like it really is something between 27000 and 27099 UDP outgoing (ports used by Valve games).

Share this post


Link to post
Share on other sites

Monitored the traffic ... Looks like it's:

Source port: Baseport+2 (i.e. 25002 for baseport 25000)

Dest port: 27017 - 27020

Protocol: UDP

 

Don't know if only one of the dest ports would be enough, but that matches what Valve documents here:

Which ports do I need to open on my router or firewall for Steam?

 

Your network must be configured to allow Steam access to the following ports (in order from highest to lowest priority for QoS users):

 

Steam Client

  • UDP 27000 to 27015 inclusive (Game client traffic)
  • UDP 27015 to 27030 inclusive (Typically Matchmaking and HLTV)
  • TCP 27014 to 27050 inclusive (Steam downloads)

 

So I would recommend going for dport 27015-27030.

 

Regards,

Alloc

Share this post


Link to post
Share on other sites
Looks like TFP did not explicitly specify how that date should be formatted so it's different in the Linux build. Maybe even depending on the systems locale.

With a german locale it looks like this:

<blacklisted steamID="76561198xxxxxx" unbandate="05.06.2014 01:56:34" />

 

Ok, so automatic unban just does not work atm (not caused by the Linux engine!): http://7daystodie.com/forums/showthread.php?11220-Does-automatic-unban-work

 

The format is actually not relevant and both variants seen in this thread work from that point of view, it's just that the 7dtd engine simply does not care about that unban date ;)

 

Regards,

Alloc

Share this post


Link to post
Share on other sites
Monitored the traffic ... Looks like it's:

Source port: Baseport+2 (i.e. 25002 for baseport 25000)

Dest port: 27017 - 27020

Protocol: UDP

 

Don't know if only one of the dest ports would be enough, but that matches what Valve documents here:

 

 

So I would recommend going for dport 27015-27030.

 

Regards,

Alloc

 

its steamport 27036

 

IPV4 UDP 125u Empfangen auf *:27036

Share this post


Link to post
Share on other sites
its steamport 27036

Hm, so the range is even wider ... On my machine it never sent (or received) anything on that port, only on the given range (27017-27020) ;)

 

 

Update of the scripts to v13

Added scripts-update detection+install feature so you don't have to download the management scripts manually each time there is an update ;)

Share this post


Link to post
Share on other sites

I had an issue with file permissions. Seems it has used user 1000 which is a different user than sdtd on my system and it has seemingly chowned a lot of things to that user for some reason and messed a bunch of things up until I chowned them back to root. I think these included the /home, /usr, /var and /etc folders as well as the folders used or created by the bootstrap script where it installed files. For example /usr, /usr/local, /usr/local/bin, /usr/local/lib, /usr/local/lib/7dtd were all owned by user 1000. So was the /etc, /etc/cron.d, and the /etc/cron.d/7dtd-backup file and whatever else there was. It's very odd considering that not only is this not the right user but it shouldn't be touching those folders in the first place.

 

I realize everything probably shouldn't run as root but it seems this script requires it. Luckily I don't really have anything else on this server other then some inactive stuff I was testing before. If it ever ends up not running as root then there are some things in /home/sdtd that are owned by root which may cause issues too but I'm sure that would get sorted out.

 

So anyway, once I fixed that I created a cronjob to remove old backups. I simply modified the 7dtd-backup cron file that was added and added another cron to it.

 

5 * * * * root nice -n 19 find /home/sdtd/backup/* -maxdepth 0 -type d -mmin +1440 -exec rm -rf {} \; > /dev/null 2>&1

 

Make sure to leave a blank line at the end of the cron file.

 

It removes any backups that are older than 24 hours. Change the number -mmin +1440 to remove files older than that number of minutes. You just have to make sure to download them before they get deleted in case you ever need them.

 

I changed the times because I don't like running things exactly at the top of the hour since other cronjobs will often run then and potentially cause some lag. I also added nice to everything to help prevent lag as well.

 

It might be good to just rsync them off to another server but it might be nice if it compressed them first to save bandwidth and disk space. I tried to add a cron to do this but something in the backup script is conflicting with it. Probably the way you copy the folder and then rsync it. So if it was going to compress it then it would have to be compressed in there somehow which would probably be better anyway. I'm not sure exactly what it's doing there with the rsync parameters but I know the backups work because I used one.

Share this post


Link to post
Share on other sites

Hm, it the scripts do ensure the permissions of some files/folders are set correctly but they should not touch anything outside SDTD_BASE or /usr/local/lib/7dtd (besides the 7dtd-files in bin, etc, cron.d, bash_completion but those are relevant to these scripts only anyway).

The only thing that I could imagine could interfere with other folders could be the un-tar-ing of the mangament scripts. But I never got any problems with that either. Will dig into that though.

 

Regarding running as root: Basically the scripts could all be made to be runnable by the user for the game itself (i.e. "sdtd" in default installs). But as the user is changed for starting the engine anyway I didn't see much of an advantage in that. Also e.g. updatescripts would not work without root permissions. Also by running as root a lot of stuff could be made root-(write)-only later on (e.g. the whole engine folder if there's nothing left that wants to open files for writing in there) or the backups. This would reduce the risk of the engine damage anything if there was a security hole.

 

 

Backups: The backup system requires a few changes anyway as I was going to implement a few limits (ticket 12). I will also include an option to have backups compressed (though that obviously would negate the positive effects of hardlinking stuff ;) ).

 

Running backups at different priority is a good idea, will add that too :)

 

As far as transferring backups to other targets: You could write a backup hook for that purpose (see Hooks). This would be run directly after a backup was created and you would get the folder passed as parameter so you could simply compress/rsync it or whatever you want to (compression will be added directly next release, probably later today).

 

 

Regards,

Alloc

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...