Jump to content
giKoN

AntiCheat Update Considerations

Recommended Posts

Its only closed because of Prisma.  No other reason.  I am considering switching it to public at some point.

 

Share this post


Link to post
Share on other sites
23 hours ago, giKoN said:

@Smegzor

 

try making your server send this to a client and check if you find the server getting a response against it:


_cInfo.SendPackage(new NetPackageConsoleCmdClient().Setup("dm", true));

 

This should be an easy proxy without altering client dll. In the end, it should be the same result. 

 

Regarding closed source I really would like to say something but in the interest of this topic i refrain from doing so.

 

From what other people have seen believes this issue effects pvp,pve,modded or unmodded servers.  "It's a one way direction package and the clent processes the package<-This is where the hack is." What i would like your opinion on is if this is something that can be fixed by modders or NEEDS TO BE DONE BY THE PIMPS? Everything ive seen shows that the only people who can fix this is the Devs. I personally feel this security threat should be #1 thing to fix right now.... I'm just wanting to get your thoughts about that-thanks.

 

Share this post


Link to post
Share on other sites

Hi all,

 

first of all always nice to see people care about security :)


Of course development does not work like we have a switch somewhere saying "Prevent all cheating / hacking" that we just never cared to turn on because we hate everyone.

 

That said, every game has always been and will always be prone to cheating. Even big companies which spent millions of bucks into fighting it, like Blizzard (not Activision Blizzard, haven't been following their activities for a long time anymore), never were able to fully stop it from happening. Even games like WoW that are mostly server authoritative, which makes it a lot easier to fight cheating, have not been able to fully prevent it.


What we currently have in place is for the most part EAC, preventing users from modifying their game client. So unless someone actually bypasses their protection - and I'm currently not aware of any tool successfully doing so - we have to assume the client does nothing we don't allow it to. So for example forging wrong netpackets is not plausible on EAC protected servers. Of course if anyone becomes aware of any bypass that actually works ... please report it (either to us or directly to EAC / Kamu). That's the only way such things can be fixed.
If you run a server *without* EAC in place *nothing* is ever going to stop client side cheating. Thinking that testing for a client's "fly mode switch" (if we had one that was transferred by default) would help anything is wrong. If you aren't totally dumb/lazy as a hacker you'd simply make it so that such a flag would not be reported to the server. Same for checking player's y-coordinate. If I was hacking my client I would simply send the terrain height as my position's y to the server while locally flying well above / below ground. So, relying on *any* data coming from a client that can not be considered unmodified (through means like EAC protection) is not helping to fight cheating on a server. (Which is also the reason as to why we do not support client side code mods yet, as that takes more measures to make sure they can't be abused for cheating)

 

The server on the other hand is fully in the hand of whoever runs it so it can be used to cheat by the owner. Which means as a user I have to somewhat trust the admins that they don't abuse their power. But that's how it always will be for public run servers, no way to stop that part.

 

The third major component involved is the network between the two first parties. And yes, people who do have the knowledge could use that to achieve some stuff they should not be able to. Haven't looked at the network protocol in some time, so can't say if just injecting packets would work without any issues, changing them on the other hand is pretty much a given. Using this to gain advantages for yourself should be possible, running commands on the server definitely is not though. We're going to look into closing up that side of things though if it's actively being exploited by now.

 


In general: If you know of any publicly available cheats (e.g. exploits without using specific tools) or hacks (tools modifying the game while running EAC, tools for changing / injecting network traffic etc) report them. That way we can investigate them and get things fixed.

 

 


Now for some responses to more specific talks in this thread... (Sorry for not using proper quotes, the forum software does not seem to let me use bb-code and making a big post with multi-quotes gets annoying here ;) )

 

 

@giKoN

Quote

Assemblies are barely checked for their individual operations [...]

Not sure if that's what you mean, but making sure the assemblies are not modified is exactly what EAC does.

 

 

Quote

[...] FlyMode is client side only, NoCollision is client side only. The worst part is, there are no back checks and the servers do not request updates to confirm states with clients. [...]

As stated above, even if those things were reported to the server it would not help. People who can enable those modes will also be able to prevent those states from being reported to the server.

 

 

Quote

Another server tonight has had its serveradmin.xml corrupted and emptied out.

Quote

You can as client already interact with serveradmin xml with basic commands, admin add, ban add etc.. You can thus also send a corrupt netpackage to alter the serveradmin.xml in a way that it gets corrupted and regenerated fresh without entries is my guess.

I'd really like to see an example of that. Not saying there can't be bugs in the code making this possible, but not aware of anything yet. The admin data is managed by the server, the clients can only interact with them through the means of the respective commands (admin, ban, etc). Running such commands on the server is protected by the permission levels of the users vs the commands. Now, if an admin missed that lower number means higher access level and e.g. switched the two around (e.g. perm level for "admin" = 1000, perm level of regular users = 0) it would mean everyone could access this data. But I doubt that's the case as it's not the defaults and admins are generally aware of that stuff when looking at the example and defaults. (PS: Running commands on client side, which is probably what some people here are talking about, won't affect the server's admin settings)

 

 

@Grandpa Minion

Quote

[...] hackers have figured out how to access commands to control a server client side and is such a threat they now have the ability to ban who ever they wish, corrupt current admin files and permissions.

As stated above for giKoN and in the introducing parts, I don't see this happening. But I'll gladly look into this if you can provide more information :)

 

 

@giKoN

Quote

[...] please keep in mind that it is also possible to spoof steam id's on entry.

Any proof on that one? This is should be fairly impossible thanks to the way Steam authentication works.

 

 

@CH1LLV1LLE

Quote

I promise this is not some attempt to discredit you, like I said before, I know with 100% certainty that it is you mod that has exposed the admin console to any player in the server and not just in a little way, they have full access to everything an admin could do. See attached for proof.

Not seeing any attached proof. Though what I can state: I looked at the code for that mod (thanks @giKoN !) and unless giKoN removed parts that were responsible for such issues (and I highly doubt that) there's nothing in that mod that interferes with the way the in-game console works.

 

 

@meganoth

Quote

[...] and replace the own steam-id with that of the admin in any package that sends a console command.

No, the console does not work like that. The sender of a console command is identified by his connection, and this connection is only accepted after succesful Steam authentication. So you can't just alter data on the command package to get yourself authenticated for commands.
Now, *if* you were able to figure out the connection details (source IP + port at least) of another existing connection of an actual admin who's online you *might* be able to fake a package you send to look like it came from the admin. But unless you're on the same local network that's like arguing the NSA was hacking 7DtD because they don't like us ;)

 

 

If you or parts of your message didn't get mentioned explicitly above and isn't covered with what's here (or what I said sounds wrong) feel free to bring it up. Also note that repro steps, information on tools etc help incredibly with getting stuff more secure.

 

 

Cheers,

Chris

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites
28 minutes ago, Alloc said:

Of course development does not work like we have a switch somewhere saying "Prevent all cheating / hacking"

You...uh...interested in buying one? I know a guy. 😉

Share this post


Link to post
Share on other sites

Thanks Alloc, it is a relief to know that work will be done on NetPackages. I do think it's necessary at this point.

And if DM/CM on dedicated can be solely controlled by the dedicated instead of client it would definitely help. 

 

I hope I can deliver some proof of concept for the rest, there's several tutorials for Minecraft and other games on how it can be achieved in general, unfortunately as Ch1lly pointed out im still an amateur ;) 

Share this post


Link to post
Share on other sites

Maybe I will post a quick summary of what happened while the thread was hidden (which happened per my request to not have too many details showing, so please don't start arguing with the mods on that):
 

Since the thread started, I know of at least 14 Servers which have been attacked, sometimes the worlds have been entirely corrupted to the point of having to start a fresh seed, some have had their serveradmin.xml restored to vanilla, some had seen their own players banned for activities based on spoofing.

 

On the technical side, we were able to provide the proof of concept within approx. 1-2 days. While a group of coders OUT OF THE COMMUNITY takes on the huge challenge to try and add verification steps to each netpackage coming towards the server, we were however shown, that besides the obvious flaw in the netcode, EAC has been bypassed as well. 

 

One of the hackers has succesfully showcased the full toolbox he has available on our EAC protected servers, you can imagine what unlocked edit mode on a live server can do. While the work on netpackage security might even protect against some of the action of the EAC bypassing hackers, it is obvious that this is a battle which we can't win if the dev team does not start to take the netcode seriously. When it comes to EAC, invest in more than the lowest tier please, for us. Implement heartbeat checks to ensure integrity of client files and operations while the player is established.

 

Anyone who goes through the game in detail can easily see how this game is coded for singleplayer - and to make it work for dedicated, you exchange a bunch of unsecure packages. However, in the course of this current project not only do we see that the vast amount of packages being exchanged from client to server to client is unnecessary and creates additional performance issues, it also bares the risk of clients being able impersonate (spoof) other players on the server. 

 

This has gone out of hand. And if 7 days to die does not get an overhaul to its netcode, it might as well just shut down its dedicated server branch.

 

 

Share this post


Link to post
Share on other sites
2 hours ago, giKoN said:

it might as well just shut down its dedicated server branch.

Well...there are quite a few dedicated servers that are set up for families and friends and trusted groups to play with each other and they have a lot of fun and don't have the same issues as open public servers that cater to strangers on the internet. So I think calling for a shut down of all dedicated servers is probably premature just because the way you are using them is problematic. Don't get me wrong, I hope the problems you are facing from hackers are able to get fixed at some point but if they never do and you have to close up shop, there will still be people having a rewarding online experience within their trusted groups on dedicated servers. 

5 hours ago, Grandpa Minion said:

i watched the a20 stream last night madmole and prime claimed they had no clue about the hacker problem happening on servers.

That's because that isn't their area of focus. The developer who is aware and does have a clue already responded in this thread. TFP is not the borg collective where every member of the team instantly knows about every issue in every other team member's area and frankly, the problem with the hackers that you are describing is not at the top of the priority list. Sorry.

Edited by Roland (see edit history)

Share this post


Link to post
Share on other sites
4 hours ago, Roland said:

Well...there are quite a few dedicated servers that are set up for families and friends and trusted groups to play with each other and they have a lot of fun and don't have the same issues as open public servers that cater to strangers on the internet. So I think calling for a shut down of all dedicated servers is probably premature just because the way you are using them is problematic. Don't get me wrong, I hope the problems WE are facing from hackers are able to get fixed at some point but if they never do and you have to close up shop, there will still be people having a rewarding online experience within their trusted groups on dedicated servers. 

 

I will try not to deviate the topic more than necessary but I do hope you at some point get to realize how much a comment like this may upset the one or the other. 

 

I have corrected your statement in the quote to give you a hint where you could start.

  • Like 1

Share this post


Link to post
Share on other sites

I honestly dont care much about anti cheat systems as long as its not something shady what permanently runs in the background throttling the system or something like Easy Anti-Cheat whats latest update has successfully managed to kill of several members of the playerbase of all games what uses their system.

 

Whenever it gets updated or not shouldnt be a priority at this stage.

Edited by Solomon (see edit history)

Share this post


Link to post
Share on other sites

A workaround could be found in filtering the api for the source of requests. I.E. Admin API would be accessible only through localhost, so if an admin needs to access the api via client app, client app should be in same network as the server, so admin would connect through a protected vpn whatsoever to the server to be able to execute admin commands. Easy solution I guess.

Share this post


Link to post
Share on other sites

I would just like to add a couple of things if I may...

As a server owner I find some of the comments above regarding this being *our* hacking problem and this not being a priority to be concerning to say the least.

We run a large 7 days community with 3 active servers and 2000+ members in our discord, We also run the largest facebook 7 days to die pc group with over 10000 members. Our focus when we started many years ago was to encourage people to play 7 days to die and love it as much as we all do. We have successfully built up a great community and constantly encourage new players join our servers and the broader 7 days to die community, And then in turn they get their friends to play and so on, Fueling game sales and helping keep the money coming in and giving you guys the ability to develop the game to the stage it's at today. We are happy with how the game has been developing and would like to see it continue to develop even further in the future.

These latest hacks that have been hitting  all of the online servers threatens to derail not just our community but the broader online community as a whole, I know of at least 20 large servers that have all been targeted and they are just the ones I know of, no doubt there would be many many more. If left unchecked and all of these large communities die you will be loosing many 1000's of players and the flow on effect will be the loss of modders, coders, server manager makers and so many smart and talented people that have given so much to the community over many years. 

This may sound all doom and gloom but you guys need to understand that this threat is very real and needs to be addresses sooner rather than later, If the last week is anything to go by in a few weeks from now if these hackers continue to be aggressive and destroy online servers it will be a disaster.

Below is a list of things that I have seen happen just on our servers alone-

  • Non admin players able to get access to the debug and creative menu's without setting off our bot alerts that *used* to be able to detect when they entered these modes.
  • Non admin players being able to put dev items (or any item) in legit players inventories thus giving the legit players a ban for unauthorized items in their inventory.  
  • Non admin players being able to clone their names and stat's of other players thus making it hard for server admins to ban the correct people.
  • Non admin players appearing as admin with the admin star next to their name.
  • Serveradmin.xml files being wiped clean of admins and ban list's.
  • Non admin players being able to reset large chunks of maps, wiping out player made bases that they have spent many hours working on.

These hackers have pretty much control over everything an admin does and a few other things on top of that.

I have many log files, screen shots and even video of these hackers in action, I am very willing to pass all of this information onto the relevant people if proof is what is required ,Just let me know and I can send it over.

I hope you guys don't take this the wrong way, we are passionate about this game and would like to help you guys grow it for many years to come.

 

Thankyou.

 


   

  • Like 2

Share this post


Link to post
Share on other sites

Just scroll up to where the developer responded and provide the information he asked for. Working productively with the developer is going to be the most effective way to get what you want. 
 

I wasn’t saying security isn’t important. I was saying that calling for a shut down of all dedicated servers if the problem isn’t immediately fixed is melodramatic and unproductive. 

  • Like 2

Share this post


Link to post
Share on other sites
16 hours ago, giKoN said:

Maybe I will post a quick summary of what happened while the thread was hidden (which happened per my request to not have too many details showing, so please don't start arguing with the mods on that):
 

Sigh, and yet multiple people during the streamwere ripping into TFP claiming Mods were deleting threads etc.

Share this post


Link to post
Share on other sites
9 hours ago, giKoN said:

I hope the problems WE are facing from hackers

Oh okay. kumbaya.

Share this post


Link to post
Share on other sites
On 10/11/2020 at 9:46 AM, giKoN said:

 

 

Shat happens, back up server and hope for the best, i back up usually twice a day....

Share this post


Link to post
Share on other sites
1 hour ago, kokuojeiku said:

Shat happens, back up server and hope for the best, i back up usually twice a day....

For those of us who self-host on Linux, this is stupid easy, and a no-brainer. All the servers our group hosts have incremental backups run hourly to dedicated storage. For people on Windows this isn't as feasible, but is still possible with the right software.

 

However the grand majority of people hosting are just renting from a service. For them, running regular backups is a nightmare, results in a fair amount of server downtime allotted to running backups, or is rather expensive when supported by the host. Restoring under these conditions is equally arduous.

 

Share this post


Link to post
Share on other sites
21 hours ago, kokuojeiku said:

Shat happens, back up server and hope for the best, i back up usually twice a day....

Thank you for that awesome idea. That solves all the problems. This thread can be closed now and all necessary investments into netpackage security will no longer be required.

Share this post


Link to post
Share on other sites
On 10/16/2020 at 9:11 AM, Alloc said:

 

@giKoN

Not sure if that's what you mean, but making sure the assemblies are not modified is exactly what EAC does.

I'd really like to see an example of that. Not saying there can't be bugs in the code making this possible, but not aware of anything yet. 

 

@Grandpa Minion

I don't see this happening. But I'll gladly look into this if you can provide more information :)

 

 

@giKoN

Any proof on that one? This is should be fairly impossible thanks to the way Steam authentication works.

 

 

@CH1LLV1LLE

Not seeing any attached proof. 

 

 

 

If you or parts of your message didn't get mentioned explicitly above and isn't covered with what's here (or what I said sounds wrong) feel free to bring it up. Also note that repro steps, information on tools etc help incredibly with getting stuff more secure.

 

 

Cheers,

Chris

 

Guys, @Alloc posted this almost a month ago and there has been zero response to it unless you are privately communicating with him. It is not enough to come on here and just say there is a problem. You need to bring proof: provide logs, show video, post reproducible steps to make the hacks work, etc. Chat bombing the dev stream is far far less effective in getting a fix you say you want than working with a developer who has invited you to send him usable information.

 

I agree that it is unproductive for others to come on here and minimize the issue by saying "shat happens" but lets get some actual traffic here reporting and posting usable proof like @Alloc asked for. If there is no proof or reproducible steps that can be provided then unfortunately the issue will go on.

Share this post


Link to post
Share on other sites

The proof is the netcode. 

 

Everything that has been mentioned prior we are able to provide more details for as soon as there is reasonable interest. We do share a discord group with Allocs and Hated but the communication has rather been one-sided. I think one entire tool was shared with Allocs which shows just how easy it is to do whatever you want on a dedi server.

 

By now:

* we have fixed serveradmin.xml corruption which is caused by invalid characters (missing check on save&load). Anyone joining and getting banned with < > &  " ' characters in their name will wipe out your serveradmin file. 

* we have fixed netpackage connect/disconnect spam which crashes servers all around the world. This is a workaround fix - needs to be adressed properly within the dev team. (Sharing my video as client, not the one performing the connect spam - the active connect spam video was shared in your testing discord)

* we have identified a few netpackages which we can add additional verification layers to the instigator/sender id's matching the id that's being sent in.

* for many of the netpackages we will not be able to add such verification layer. Sender and EntityID so often don't match - sometimes for good reasons. Sometimes it seems pretty random. 

 

To be honest, open dedi multiplayer simply is a @%$#show right now. 

 

 >

Share this post


Link to post
Share on other sites

What was that video supposed to show? I know you said you weren't the one performing the test but why is this video here?

Share this post


Link to post
Share on other sites
1 hour ago, JCrook1028 said:

What was that video supposed to show? I know you said you weren't the one performing the test but why is this video here?

It shows how the game handles netpackage spam for connect/disconnect packages. It attributes resources to the spam prior to checking for validity/steam auth, thus, all valid packages get delayed (ping) - if done with high enough frequency/long enough the CPU will cave in. RAM load is increased significantly too. We did this with relatively small bursts to test the concept.

 

the crash ptentially causes world saves to go corrupt and desync between client files. 
 

the spam was performed locally. 


 

 

Share this post


Link to post
Share on other sites
24 minutes ago, giKoN said:

It shows how the game handles netpackage spam for connect/disconnect packages. It attributes resources to the spam prior to checking for validity/steam auth, thus, all valid packages get delayed (ping) - if done with high enough frequency/long enough the CPU will cave in. RAM load is increased significantly too. We did this with relatively small bursts to test the concept.

 

the crash ptentially causes world saves to go corrupt and desync between client files. 
 

the spam was performed locally. 


 

 

Unless I'm missing something It really doesn't.... it just shows that there is high ping.   I believe you when you say what it happening but the video doesn't really show anything other than high ping. 

Share this post


Link to post
Share on other sites

Well, i'm testing my client side performance during the attack. So yees this doesnt show anything with regards to the CPU of the server. The proper information is available in the pimps testers discord.

 

However, the ping is crucial given that it was performed locally - thus, no attack on the network itself. It's simple, the game isn't dropping the invalid connect/disconnect requests as it should and instead allocates ressources. 

 

But as I mentioned, this is just one of the endless possibilities we have right now to manipulate on netcode layer. 

  • Like 1

Share this post


Link to post
Share on other sites
4 hours ago, JCrook1028 said:

What was that video supposed to show? I know you said you weren't the one performing the test but why is this video here?

Election night PTSD...

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...