Extreme lag

Community Support
,
12

Unplayable lag on all the EU servers today at approximately 10:15 PM UTC.

I took a packet capture today… here is an excerpt of what I’m seeing:

There seems to be a whole lot of individual requests, maybe those are the HTTP requests you were talking about.

I believe the constant connection I see would be the websocket then (I’m not able to dissect the packets because of encryption and I don’t have to tools to capture the secrets). On this connection I am seeing packet loss with the server.

image
image1222×277 22.1 KB

In this case I believe it’s Gloviathosa, is trying to send me some data. The flow is from Glovia to my PC.

I see out of order segments, which cause me to send duplicate acks to enable TCP fast retransmissions. This means a packet was lost in transit from Gloviathosa to me. I do eventually receive the missing packet(s) (in many cases as fast as possible… the very next packet). The missing packets are marked as retransmissions meaning packets are not getting delayed. They are getting dropped.

This is causing my RTT to spike though, which is especially bad because the server is located far away in Germany (I’m in the states).

image
image1692×574 65.1 KB

So my RTT Is on average very low ~168ms (fast as heck for Europe). Anyway it looks like 1 packet drop is causing a delay up to 510 ms. I guess that’s not a great latency, but honestly… it seems the game network code should be able to handle that. It does explain why the “unplayable connection” screen is showing.

Does something special happen when it’s deemed unplayable? What I’m observing is the game is warping me back to a previous position (maybe due to some code related to “unplayable connection” it’s probably the last acknowledged position from the server). And then I’m warped forward to where the server “thinks I am” (which is where my client was before).

image

Honestly if #3 (the client moving me backwards) never happened, the problem would be resolved. My connection is otherwise fine. Dropping 1 packet in TCP is a pretty normal situation. It’s the mechanism used for flow control.

Another option might be, don’t let the client snap the position around during “unplayable connection”. Even if my player was forced to stand still in that scenario it would feel much better than warping too a fro.

1 Like
13

@lucadeltodecso As a follow up I’ll take some captures of what a normal play session looks like on a US server see what that looks like.

14

“Another option might be, don’t let the client snap the position around during “unplayable connection”. Even if my player was forced to stand still in that scenario it would feel much better than warping too a fro.”

there is an option in the game settings under network for just that (pause inputs something something)

15

Interesting. I can’t do any tests right now because I can’t even connect to the server. The USA servers are perfect though.

image

I pinged the server and got a response. 167 ms RTT with 75% packet loss.

I should also mention that ~85% of the time I have no issues even on the EU servers. This is an exceptional event but it does seem to coincide with a lot of other folks having connection issues as well (judging from the forum).

16

its because steam is down right now

17

I just logged into Gyosha Ophin just fine. Is EU steam down?

18

was for maintaince… its up again now

19

Not sure that’s related. I’ve been able to log in and log out of Boundless just fine several times in fact over the last hour, since I’m on the US server side. Glovia still “down” from my side:

image
image1524×612 372 KB

Also saw the message “Too many active connections” “Timed out…” and “Game sever down”

1 Like
20

meanwhile pinging the same server here from Japan I get a nice stable 240ms and 0 packet loss over a few minutes, can also connect to the server fine in-game.

21

Well clearly you’re using the routers on the right side of the server and I’m using the one’s on the left : P

22

literally yes :stuck_out_tongue: (at least I assume my traffic is not being routed the wrong way via america…)

23

image

VPN through Virginia (I can connect just fine).

24

image

Anddd back to my local ISP

25

Which does certainly provide more evidence for what I’ve said quite a few times so far, that it is a problem with the routing between the isp and aws (hence location and vpn both have an effect)

26

I feel like I’m standing right next to you looking at the same thing.

27

Well I can’t say there is not a problem, I highly suspect there is. But if it’s between AWS and Comcast, that’s probably a concern no?

28

You very well might be. lol

29

West Coast Player on Comcast

20181113165732_1
20181113165732_12560×1440 384 KB
20181113165751_1
20181113165751_12560×1440 363 KB
20181113165758_1
20181113165758_12560×1440 383 KB
20181113165829_1
20181113165829_12560×1440 387 KB
20181113165838_1
20181113165838_12560×1440 383 KB
20181113165655_1
20181113165655_12560×1440 388 KB
20181113165702_1
20181113165702_12560×1440 369 KB

Constantly crashing on any server that will let me on it as well.

Finata is also down.

1 Like
30

Honestly if yall are running this on C5s’ you’re spending a good bit of real life coin. You should be able to call up AWS and get someone to assist.

An alternative is that you can (and maybe should) be having us connect to country local proxies for game traffic. That way you have near 100% control of issues like this.

It is a unique challenge with the architecture of the game, and the desire to connect all the servers across the world. I honestly think that’s soo cool. But it is unique. I think if I had any say over the network design of things, that’s exactly what I would do.

31

Was also standing near the portal talking to some folks. A handful telling me they had the same issues. Mostly Comcast and ATT.

Let me know if you need more measurements. I’m happy to help… deal with inter-corporate network problems like this all the time.

2 Comcast, 3 ATT so far