Important The Crashes and Our Progress

Discussion in 'TTT Discussion' started by Opalium, Apr 15, 2018.

  1. ink

    ink Genuine Happiness VIP Silver

    Man both the EUs are the like only servers in well... EU
     
  2. Xproplayer

    Xproplayer VIP Silver

    More like EU servers are the smaller ones compared to US therefore no one cares if you fuck them up cause ur the testing pig

    But hopefully this breakthrough will do some shit. While you're at it fix the stupid weapon balances, remove the event code, and enable weekly fun times where all the crazy weapons are enabled.
     
    • Dumb Dumb x 4
    • Optimistic Optimistic x 3
    • Funny Funny x 2
    • Agree Agree x 1
  3. Notice how a majority of the players that complained in this thread and left negative comments are either banned or inactive.

    EDIT:

    I feel what's really hurting the community is the suggestions part. I feel like if a good portion of the suggestions were filtered through like once a month by a group of people, then we'd clear out most of the bullshit and be able to pick better content for the servers.

    (plus I'd be all for funding server improvements, I know you staff guys are volunteers and I appreciate the work you do)
     
    • Winner Winner x 2
    • Optimistic Optimistic x 1
  4. Carned

    Carned ✘o ✘o VIP Silver Emerald


    https://www.seriousgmod.com/threads/introducing-public-test-server.48009/
     
    • Funny Funny x 1
  5. Daddy Nexxus

    Daddy Nexxus Toxi-Fessional VIP

    2/4 people who rated this dumb are banned, good luck suckers.
     
    • Funny Funny x 2
    • Winner Winner x 1
    • Useful Useful x 1
  6. Juice Juice™

    Juice Juice™ VIP Emerald Bronze

    All the members who have been banned, their petty obsession with this place brought them together. Through petty knowledge on scripts and Gmod, they have created a Frankenstein patchwork of code known as "Toonberg", to catapult the server into a sense of frustration and annoyance.


    This wasn't very clever, but I posted it anyways
     
    • Like Like x 1
  7. Opalium

    Opalium Stay Awesome Banned VIP Silver

    Hello players.
    I have some more news to share on the matter at hand - and sadly, it's not what I hoped for.

    With the help of Facepunch, we were able to track down the issue and pinpoint the cause. We tracked the source of the issue to two different bugs in the engine (more specifically, in the srcds binary), that can randomly happen under certain conditions and cause the server to freeze completely.

    For those who are interested, here are the details:
    • The first problem lies in the map cleanup function, which is called by TTT at the end of each round to reset the map to its initial state. This happens during the round end->preparing round transition, which - I suppose you already noticed - is when most of the "crashes" happen. The cause of this is the engine hitting a bugged condition during that process, which causes it to enter an endless loop. This means that the engine is now "trapped" in an infinite loop, and it cannot ever exit it to continue processing other things. This, in turn, means that the server locks up completely, no longer responding to clients or to commands. Then, as you can imagine, the clients lose connection, as the server stops responding to them.
      Considering the time this began to happen, this is most likely due to a bug in the server build that was pushed at the February update - which is around when those crashes began.
      .
    • The second problem lies in a much older physics bug in the engine, most likely related to collisions. This is yet another case of the engine entering an endless loop - this time because of two entities with specific collision groups colliding into each other and hitting a bug in the engine code related to the calculation of the physics object origin points. This, again, leads to the server locking up and the clients timing out. This bug was actually present for a very long time, but with the former bug now present in the code, the frequency of the latter one being hit seems to have increased as well.
      .
    • In both cases, the server remains frozen until restarted. This is where a script we wrote over a year ago comes in - it monitors the srcds process and forcefully restarts it if it does not respond after multiple tries.

    Unfortunately, because these are engine bugs, it significantly limits what we as a community can do to solve it. While we have different ways to try and work around it through lua and basic binary modules, bugs and issues in the engine binaries are at the hand of Facepunch, the game developers. Unlike the glua framework, GMod's binaries are kept private. This means we don't have a way to access the engine code, let alone fix those bugs ourselves.

    We did, thankfully, find a partial solution for the second bug through some lua wizardry and a few good guesses: POOP - Physics Origin Overload Protection. It's a lua script I wrote that helps combat the issue by "neutralizing" impossible physics conditions that lead to the lockup (more specifically, the collision origin bug - hence the name). Sadly, this script can only do so much, and while it already reduced the amount of non-cleanup freezes by a lot (it's live on EU1 and EU2 right now), it's not going to solve all of them. Still, it helps remedy the issue and will be pushed to the rest of our servers once I finish working on it.

    Then, there is the former bug - which seems to be completely out of our hands. None of our attempts to work around the issue through lua were successful, which leaves us with no much choice on the matter. As it stands now, the only option we have left is to wait for Facepunch to pick up the glove and resolve the issue by fixing the bug.

    We are not alone in this, mind you: more and more servers around the world, especially TTT ones, are reporting an increasing number of crashes with these characteristics. Just a quick search of the Facepunch forums can reveal many others who suffer from this bug. During our communication with Facepunch, they have also acknowledged this. It is therefore clear that the issue is becoming a pressing matter - which, hopefully, will increase its priority for the game developers.

    While this is quite unfortunate news, it's still important for me to let you, our players, know this. It seems that, as much as I hate to say it, we will have to bear with the crashes for a while longer as we wait for Facepunch to work their magic. It is therefore only fair you will know the full details of it.
    As you can understand, there is no ETA or any way to predict what will happen next. The ball is in Facepunch's court now, and we can only hope they will figure this out quickly. We will continue to communicate with them as much as we can to ensure the issue is properly prioritized and to supply any help or data they may need for it.
    And even though this issue may be pending for a while, we are advancing on other fronts as usual: we're only days away from the launch of the PTS, with the first batch containing many great treats, and we are also continuing to work on some behind-the-scenes projects - which I cannot reveal just yet - that will sooner or later add even more value to our community. With this in mind, it's safe to say the future of our community is not threatened by the issue.


    With all that said, I'd also like to thank you, our players, for bearing with us during all this and for the trust you place in us. Indeed, the crashes are frustrating, but even with them, there is so much more to enjoy here in SGM, and I know even this won't stop all the rest our community has to offer.

    Stay awesome,
    ~Opalium
     
    • Informative x 6
    • Friendly x 5
    • Like x 4
    • Winner x 4
    • Funny x 2
  8. Juice Juice™

    Juice Juice™ VIP Emerald Bronze

    I question how many ideas you went through before settling on POOP as a name.
     
    • Funny Funny x 2
  9. Opalium

    Opalium Stay Awesome Banned VIP Silver

    You probably don't want to know. :whistle:
     
    • Disagree Disagree x 1
    • Funny Funny x 1
  10. Juice Juice™

    Juice Juice™ VIP Emerald Bronze

    Bish tell me!
     
    • Funny Funny x 1
  11. Cash

    Cash I staff the proper way Banned VIP

    The other ideas must have been real shitty
     
    • Dislike Dislike x 1
    • Funny Funny x 1
  12. Juice Juice™

    Juice Juice™ VIP Emerald Bronze

    You are so lame
     
    • Agree Agree x 1
    • Optimistic Optimistic x 1
  13. Daddy Nexxus

    Daddy Nexxus Toxi-Fessional VIP

    So.. Can you just fix it? Thanks.
     
    • Winner x 2
    • Agree x 1
    • Disagree x 1
    • Bad Spelling x 1
    • Dumb x 1
  14. Xproplayer

    Xproplayer VIP Silver

    Thats very professional, glad to see our staff taking their job seriously...
     
    • Agree Agree x 1
    • Creative Creative x 1
    • Dumb Dumb x 1