PM Lag

[Alwin]

My PMs sometimes lag by 5 min or more. I imagine this is because I have a large enough number of PM channels, that cycling through takes 5 minutes or more. This is much worse for other players who’ve been around longer (and shorter) than me.

I will imagine that “scalability” means that chat should work well, even if you have 10 alts that are a part of 10 guilds each and some wanton number of PM channels. And based on the discussion below, it should work like “client checking a letterbox for mail.”

Somehow or other, say/shout/whisper seems to work fine. Guild chat is funky because multiple people receive each message. Maybe the following approach will work anyway for guild because it is limited to 100 channels (10 alts, 10 guilds each)

My suggestion is that when user A sends a message to user B, a Helper channel is updated on the server’s end. So, there’s an update to A’s PM channel, B’s PM channel, and A’s Helper channel, and B’s Helper channel. Helper channel is simply a helper tool which records the timestamp and identifier of any PM channel that has been updated. Rather than cycling randomly through PM channels looking for new messages, the client just asks Helper Channel if there have been any new PMs based on the timestamp in the log, and if it finds any, THEN it requests a real update for a channel based on the identifier in the log. When those real updates are received, old data in Helper channel older than some buffer time can be deleted, since it’s just a helper tool. Also, if there are two updates for channel A listed in Helper Channel, you just keep the most recent one.

Helper channel means temporarily storing roughly 2x more chat data in the worst case scenario (where every channel literally just has 1 message, and Helper channel has <= 1 message for every channel). But you can reduce stress on the backend by 2 or more (reduce rate of checks) while increasing responsiveness (since you should get PM updates at an equal rate of checking Helper).

I’m not a computer science guy so this probably has lots of mistakes I’m not seeing, but I’m trying to suggest a simple way to solve the PM lag problem while not changing too much about how chat currently works.

@lucadeltodecso

its not quite so simple as that.

before we updated 10 channels every 30 seconds’ish and it just round-robined choosing the 10 “least recently updated” with no extra criteria + an extra channel (or many on notifications which is actually a composit of many channels, one per alt + a universal announcements + a username level notifications channel) when not on #all to be faster, now it updates 1 channel every every 2.5s’ish, but which channel it picks to update is based on a lot of criteria:

• how active is the channel? when was the last received/sent message? [ more active → expect there to be new messages again soon so update more frequently ]
• are you scrolled up in the channel looking back at older messages? [ looking back at history → care less about new messages arriving right now ]
• are you filtering on a specific channel? [ update more frequently ]
• how long since this channel was updated? [ if channel has just been updated, give it a break for a bit… and helps channels that get prioritised down to eventually update still ]
• how many unread messages are there in the channel? [ if you arent bothering to read the channel, then update less ]
• what type of channel is it? PM/Guild channels expected to be “more important” and updated more frequently than Universal channels, updated more frequently than Notification channels for your alts, updated more frequently than Guild-Log channels, updated more frequently than “universal” notification channels [ we very very rarely send a message out on these as devs ]

The result aims to not be any more “expensive” on the backend, but being much more responsive in general use-cases, eg you send a pm to someone: activity is higher, so is updated more frequently expecting you to get a response. If a pm is sent to you, once it is received itll be updated more frequently for a while expecting more messages to come soon. If you are actively PM’ing someone, even on the #all channel, it should be much more real-time than before where it would ALWAYS be slow as it was wasting time updating channels you hadnt interacted with in a really long time for no good reason [ and the bugs in the system meant that whenever it wasnt updating a specific channel, itd end up missing new messages in that period too making it much more useless… ]

Quote source

Quote from Release 239: Lovestruck and Prototype Creative Mode!:

@lucadeltodecso

The original reason was that the messaging system was developed for async communication via the letterbox.

As usual, requirements change and gradually it became more and more “a chat system” but alreay developed ontop of a normal database that does not support publish/subscribe semantics that are needed for real time chat.

It wouldn’t be a small amount of work to change the backend, so as it stands that is not going to happen in the near future.

[Alwin]

tl;dr

I think PMs lag because client checks all channels in turn. For PMs, I suggest way for client to frequently check 1 centralized channel for faster updates using fewer updates.

[Huntsman]

Quite agree, been a thorn in the side ever since they added Guilds (same update pattern).

[Damagekai]

The same whit guild messages