TERA Online dev tracker
All devs
Viewing dev posts for topic: recent-server-stability-issues
Dreamstorm has been disabled in an effort to improve some of the issues individual players are reporting.

We'll be posting a full summary of recent network incidents that impacted TERA's overall network performance as well as details surrounding a compensation package tomorrow (12/15/2016).

We appreciate all the reports players have submitted in this thread. They will aid in our continued investigation and monitoring of TERA's server performance going forward. In order to separate major network incidents from isolated, player-specific issues, we are locking this thread as it has served its purpose.
@Spacecats i preformed a traceroute to the server listed in the "latency issue" area of the support, where should i send that information to assist getting this situation resolved? there's a nice little 200ms rt jump from one that you guys may want to look in to

Thank you. Can you send it to Community@enmasse.com with the title "Sever traceroute" with the file attached. From there, it can be forwarded to the Operations team. There's always a chance that info could help identify a problem area they haven't seen yet.
Eiiji wrote: »
just finished RMHM with 500 to 900ms on slayer, yay :D

Where are you playing from?
We are aware of an issue affecting the servers right now and are investigating. It appears to be a traffic spike at an upstream node.
Hey all. Looking into the "Ticket not found" launcher problem. I'll post again if I get more info on it. I've also got the Ops team looking into the Dreamstorm lag that continues after the server adjustment.
There was an ISP traffic issue that was reported to us this morning. It affected multiple regions and lasted approx 20-25 minutes beginning around 8:45 a.m. (not long after servers came up from maintenance). Unfortunately there's nothing we can do in these situations, but we've been assured by our ISP that action has been taken to make sure things stabilize.
Our Operations team is aware of the current issues and is investigating.
Confirmed that servers are now back up.
We are investigating a new spike in disconnects that started at about 3:40 pm PST. We believe it may be related to the networking issues we saw earlier today. It looks like AV is about to crash and players are having trouble logging into the launcher. The operations team is looking at the problem now.
We've just received a confirmation from our data center host in Chicago that there is a network connectivity problem in that region that is disrupting connections to servers hosted there. For En Masse, that includes TERA, our log in servers, and most of our web servers.
We started seeing another spike in disconnects starting at 9:00 am PST and a second set at 9:30 am PST. The operations team is looking into the problem and trying to find the cause. It looks like people are starting to connect normally once again.
Server traffic is now getting back to normal so I want to take some time to give a brief update. It seems that the mass disconnects over the last several days can can be correlated to massive, brief surges in data being sent to our data center around the same time. We are working on tracking down the source of these data surges as well as ways to block them from happening.

Essentially all our inbound bandwidth is consumed when these surges happen so large batches of players are disconnected or see huge lag spikes.
We are seeing a rise in the number of disconnects right now (starting around 12:12 pm PST) and we are investigating.
@Spacecats @Denommenator thanks for the update with the status on the lag situation. Is there anything we can do on our end to help, like trace route or our ISP info we can give to assist? I personally don't have lag or DC issues much myself but I see it happen to others on my screen and it's quite annoying to say the least.

We'll let you know in this thread if there's any info that will help the Ops team with their investigation. Right now nothing comes to mind. Thanks very much for offering!
Reserving to communicate further information.
Hi all,

There are a lot of threads about this topic already. We've read all your posts, but I'll be closing/merging those threads so we can get conversation all in one stickied place (right here).

We're definitely aware of how much trouble the servers have been experiencing lately. Every indication that we've received thus far is that the cause of the instability is external, either brought on by attacks on our data center or general internet congestion, but that's really no excuse. The prolonged instability the game has suffered lately is unacceptable for an En Masse game and we know you guys agree.

We have our Operations team looking at this with the highest urgency. We'll be meeting directly with the affected service providers today in hopes of narrowing down exactly what the problem is. Denommenator and I will post in this thread when we know more. Obviously it's super frustrating to experience lag and disconnects like this even when you know it's being investigated, but we ask that you try to stay patient while we pursue a concrete and hopefully lasting resolution. Once we're confident that some resolution has been reached, we'll begin to discuss the appropriate compensation for our players. Thanks all!
You can contact Tera Online dev tracker at contact@teradevtracker.com - Privacy policy - Tera Online dev tracker is not affiliated with Tera Online or En Masse entertainment.