@Riamus : Oh, I like that idea, will try it out tomorrow (no free time today), thanks! I'm kinda hesitant to delete the meshes though, as it will probably also deleted my base that is close to it?
@Jugginator I will try to put some blocks at or close to the ghost tower as you said, but it will be hard to pinpoint the exact location (as it disappears when I get close). I did already post a log in my earlier reply, here it is again: https://pastebin.com/pB5vJEEw , in case you have time to have a look
Lastly, I have a gut-feeling this only happens when visiting a undiscovered part of the map for the first time and build something. If I find the time, I will try to replicate it.