site banner

Culture War Roundup for the week of September 2, 2024

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

5
Jump in the discussion.

No email address required.

Details of specific technical issues with the spec, for those interested (this isn't CW but since you asked). @SubstantialFrivolity @sarker @confuciuscorndog @TequilaMockingbird Unfortunately it's difficult to explain the shortcomings of the protocol in layman's terms because the protocol itself is hard to understand in layman's terms.

What matrix is supposed to be: Fundamentally, matrix advertises itself as an “eventually consistent” datastore, but in essence can be described as a distributed state machine.  When operating correctly, events are sent to the distributed machine which mutates the state. In other words, matrix is a distributed computer/virtual machine that calculates who is in a room, what their permissions are, and which messages should be displayed in the room. What it's supposed to do is akin to the operating model described in “practical byzantine fault tolerance”. Auth events are particularly important mutations, because they change which mutations are valid in the future

What matrix is in practice:

Matrix is not eventually consistent: Matrix has no method to guarantee that servers will eventually reach a consistent state.  In practice, servers do not reach consistency but instead diverge increasingly over time.  There is no mechanism for servers to tell if they are missing events.  Prev events is a weak measure, because with larger rooms where most members are lurking at any given time, the missed even is much less likely to be referenced.  In the “ideal” world of open federation, each homeserver will only be represented by a handful or even a single user in a given room.  If events are dropped in transit, or a homeserver spends some time offline, the homeservers will likely never recover those lost events.

Room membership breaks spaces, profiles-as-rooms, etc. In practice many spaces are used simply as a readonly way to distribute a list of public rooms.  With auth rules and past state snapshots, spaces with a large membership become a terrible drain on homeserver resources despite the actual information being transmitted being next to nothing. Profiles as rooms would face an even tougher challenge because the people who need to view a user’s profile are the superset of all other users in any room that the user is in. Peeking over federation to achieve one-to-many communication is a horrendous hack, and likely would never get implemented.

Availability during partitions is a bug, not a feature: Matrix is designed to attempt to function even if some homeservers are partitioned from all of the rest of the homeservers.  Fundamentally, this means that each side of the partition will have events that contradict events on the other side of the partition.  If this results in a permanent split brain, this is worse than a temporary loss of availability, because the room is now broken forever.

State resolution implies time travel/retroactive changes: The very idea of state resolution is to address contradictory events, and attempt to come up with a consistent set of events that multiple servers agree on (though as shown in the above item, this doesn’t work in practice).  This means the erasure of some contradictory events, and a revision of history for the server that believed in those erased events.

Matrix does not guard against time travel by hostile homeservers with power level: Users who currently and in the past have the power level to do so and control a hostile homeserver can rewrite history by sending events timestamped in the past and with appropriate DAG pointers.  This will also blow up the resource consumption on all other homeservers in the room, depending on how far back the rewrite happened

Calculating retroactive state with nonlinear history is intractable and ambiguous

State resolution complexity is a consequence of a lack of consensus, not a way to achieve consensus

/sync transmits a diff of the entire snapshot of the state to the client, when the client never even needs the full state. In every other messaging app model the client queries for a view of exactly what the user needs to see on-the-fly. Computing the state diff for sync is computationally intensive for no productive purpose.

Seem to me that you need to clarify what your requirements are.

So matrix advertises itself as "eventually consistent" but is that an actual design goal or is it just words words words courtesy of some MBA? "Consistant" is a very different requirement from "secure" or "fault tolerant" with very different trade-offs. From your description it sounds like the designers have priorized fault tolerance and portability over other concerns, that doesn't necessarily mean it's "broken".

It is a design goal and it's laid out as a claim in their technical specifications (which don't look like mba marketing materials to me)

https://spec.matrix.org/latest/

Eventually-consistent cryptographically secure synchronisation of room state across a global open network of federated servers and services

https://matrix.org/docs/older/faq/

at its core Matrix can be thought of as an eventually consistent global JSON database

Nevertheless, I see no world where having agreement who is and isn't in a room isn't a very basic requirement.