site banner
PaperclipPerfector  7mo ago (text post)   25758 thread views

Culture War Roundup for the week of April 1, 2024

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

11
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

The Many Eyes theory of software development worked. This was an incredibly subtle attack that few developers would have been able to catch, by an adversary willing to put years into developing trust and sneaking exploit in piecemeal.

I've watched a lot of doomerist takes on this one claiming that this proves many-eyes doesn't work, but I think it proves just the opposite. This was perhaps the most sophisticated attack on an open source repo ever witnessed, waged against an extremely vulnerable target, and even then it didn't come even close to broad penetration before it was stopped. Despite being obvious it bears laboring that it wouldn't have been possible for our Hero Without a Cape to uncover it if he wasn't able to access the sources.

If I had to guess, I would suppose that glowing agencies the world round are taking note of what's happened here and lowering their expectations of what's possible to accomplish within the open source world. Introducing subtle bugs and hoping they don't get fixed may be as ambitious as one can get.

That being said, I'm not sure that the doomerism is bad. The tendency to overreact may very well serve to make open source more anti-fragile. Absolutely everyone in this space is now thinking about how to make attacks like this more difficult at every step.

This was perhaps the most sophisticated attack on an open source repo ever witnessed, waged against an extremely vulnerable target, and even then it didn't come even close to broad penetration before it was stopped.

Witnessed is a little important, here; I'm not as sure as TheGrugq that this isn't the first try at this, if only because no one's found (and reported) a historical example yet, but I'm still very far from confident it is the first. And it did get really close: I've got an Arch laptop that has one part of the payload.

Despite being obvious it bears laboring that it wouldn't have been possible for our Hero Without a Cape to uncover it if he wasn't able to access the sources.

That's... not entirely clear. Visible-source seems to have helped track down the whole story, as did the development discussions that happened in public (though what about e-mail/discord?), but the initial discovery seems like it was entirely separate from any source-diving, and a lot of the attack never had its source available until people began decompiling it.

The tendency to overreact may very well serve to make open source more anti-fragile. Absolutely everyone in this space is now thinking about how to make attacks like this more difficult at every step.

Yeah, that part is encouraging; I've definitely seen places (not just in code! aviation!) where people look at circumstances like this and consider it sign the were enough redundancy, rather than enough redundancy for this time. I think it's tempting to focus a little too much on the mechanical aspects, but that's more a streetlamp effect than an philosophical decision.

Visible-source seems to have helped track down the whole story, as did the development discussions that happened in public (though what about e-mail/discord?), but the initial discovery seems like it was entirely separate from any source-diving, and a lot of the attack never had its source available until people began decompiling it.

There isn't any evidence directly supporting this, but I saw a claim that this entire claim of discovery ("slow SSH connections") could easily enough be parallel construction prompted by The Powers That Be (tm) aware of this effort for other reasons -- which range from "we know our adversaries are trying to insert this code" to "we run our own audits but don't want to reveal their details directly". Even in such a situation it's a bit unclear who the parties would be (nation-state intelligence agencies, possibly certain large corporations independent of their respective governments). The obvious claim would be something like "NSA launders data to Microsoft to foil North Korean hacking attempts", but "China foils NSA backdoor attempts" isn't completely implausible either.

That said, I can only imagine that sneaking a backdoor like this into proprietary build systems would be even less likely to be detected: the pool of people inspecting the Windows build system is much smaller than those looking at (or at least able to look at) libxz and it's (arcane, but somewhat industry standard) autoconf scripts.

Also, this is the sort of thing that I have a vested interest in as a long-time personal and professional Linux user. I have the skills ("a very particular set of skills") to follow the details of issues like this, but there isn't yet any organized way to actually contribute them. I'd be willing to spend maybe a few hours a month auditing code and reviewing changes to "boring" system packages, but I'd never think to look at libxz specifically, or to get enough "street cred" for people to actually take any feedback I have seriously. And even then, this particular issue is underhanded enough to be hard to spot even when an expert is looking right at it. Does anyone have any suggestions for getting involved like this?

Witnessed is a little important, here

I think the glass half full perspective is more accurate here. Sure, it wasn't detected at the earliest possible time—the second it was committed—but it was only in the most bleeding edge releases of a select few base distributions for a few weeks before it got sniffed out. For such a sophisticated attack, that's lightning fast. Stuxnet took about five years and infected around a hundred thousand machines before it was uncovered. Sure, it's possible that this sample size of one is unrepresentative of the whole distribution of this event repeated a thousand times, but that's less likely and strikes me as somewhat catastrophizing. As someone noted below, we don't know that this wasn't an attack from an AGI sitting in OpenAI's basement plotting to kill us all as we speak.

Visible-source seems to have helped track down the whole story

How would he have tracked down the backdoor without the repo? It seems to me that without it all he would have is some CPU benchmarks and some valgrind errors. What would he have done with that other than submit a bug report to the company that actually had sources, which could be ignored or "fixed" at their discretion?

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats