This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
AIAIAIAI
AI is going to maybe doom the world, but first it's going to doom the SaaS industry. We've all seen every company bumrush to build Generative AI into their tools whether it's useful or not.
But I am now dealing with the next wave. Everyone is pushing their AI agent. The thing is, there's an arm race between dedicated agents, and generic agents. Browser Agents can't yet make redundant specialized in tool agents, but it's not looking like a particularly long roadmap.
I think building / selling an AI agent offering is a fundamentally losing proposition. As I have been thrown several different demos over the last few weeks and manage a team to be buying some of these tools, my biggest perspective is that none of these companies can be trusted for long-term partnership.
These folks are building sandcastles on the beach while the tide comes in. It's not just a bubble, the pace of change has already gone faster than buying and implementation cycles.
The age of SaaS solutionsis going to cannibalize itself inside of the next 12 months even if AI stalls today.
Have you seen any AI agents that are capable of pulling off the complex task of "Renewing a prescription and ordering a refill?"
That's been my benchmark for the arrival of AI agents, and have yet to find one that's close to capable of it.
no, I don't think we're at generic helper bot yet.
Just so any AI marketers who might be watching know, if you can provide an AI agent that is capable of getting prescriptions refilled, scheduling appointments for contractors, handymen, cleaners, etc., and organizing the invitations and RSVPs for parties or small events I'm trying to organize...
AND can handle this with greater-than-95% accuracy, I'd probably pay $1000/month or more for the service.
But here's the problem. Before the fact that this AI can now displace all blue collar work, let's magically assume away some reason that won't be the case even for a super narrow slice of time. Such an AI will also collapse a great deal of the SaaS software business, which itself will be extremely economically disruptive. Once an AI is generally that capable, a lot of differentiated software becomes useless. Already I see many folks trying to sell me lipstick on top of the same 3 AI models. There's quickly decreasing utility in the UX if it's just a pass through to an agent.
Tend to agree, but I'm just asking for the production of an agent that actually matches the hype.
I suspect there is still a use for specialized software that is cheaper to run for a given task than having the LLM burn through $100 of compute for basic functions, but at that point the AI should be able to write and refine the code for such software anyway.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
My greatest fear at the moment is that we will reach a stage where people start designing UI for AI agents and not humans anymore, and from computers become truly incomprehensible.
Keen observers that hang around these parts know that of course, this process has already started and Elsagate was just the tip of the iceberg of algorithm induced mass psychosis, but still, it's not insultingly obvious yet.
I'm having discussions right now about using AI to maintain abandoned FOSS libraries that people still depend on, it's only the beginning.
REPENT! Sacrifice your hamburger menus on the altar of simplicity or face the cruelty of the machine gods.
My guess is that if they optimize UI designs for LLM-derived AI systems, it will be more intuitive than whatever they are doing now, and that an alien network of tensors and model weights is more in-tune with the average person than the current crop of UI designers is.
I've had several gripes with modern software and websites, if you haven't noticed.
I mean it might start out that way. But then the LLMs start developing a language of their own (I believe there is a Facebook study of reinforcement learning to that effect). And then we need to pull the incense rituals to appease the machine spirit.
More options
Context Copy link
More options
Context Copy link
The best UI for an AI agent is likely to be a well-documented public API, which in theory will allow for much more flexibility in terms of how users interact with software. In the long run, the model could look something like your AI agent generating custom interface on the fly, to your specifications, tailored for whatever you're doing at the moment. Could be a much better situation for power users than the current trend toward designing UI by A/B testing what will get users to click a particular button 3% more often.
Well this circles my point. If we got to that then we have such a fast and powerful AI, software isn't even on our mind. But even as we get closer to that, the entire SaaS ecosystem will start to collapse. If fundamental functionality from UX to API communication relies on an LLM, unbounded by underlying code bases, niche software vendors won't have anything to differentiate themselves.
I think we are already starting to see some collapses in the space I work in where, each vendor is completing to be nothing but a chatgpt terminal wtih some lipstick
It goes way beyond SaaS, in general agents are going to be the moment where everybody starts finally realizing that language models are going to automate all white collar labor very very soon (robotics / multimodal stuff for complex manual labor might take another 12-18 months, but no longer). Emergency lawmaking is only a short time away, in all likelihood.
Sure, my point is that in my corner of the world, I’m already seeing signs of breakdown. The bubble is extremely fragile because it’s self consuming. I am stuck being asked to sign contracts and make decisions on software that I have no confidence will remain viable or a front runner by the time we get fully implemented.
The rate of change is already too fast and unpredictable to make business decisions on anymore. The landscape’s moving faster than sales cycles.
More options
Context Copy link
This seems awfully optimistic (or pessimistic, I don't know), you have to actually get the materials, build the robots, deploy them, and real world feedback is much slower.
I'd say it takes 5-10 years.
I'm even more pessimistic. There's no decent "human hand" robots, anywhere, at any price point. Drop the weight requirements, because you'd be glad to stationary mount it? No product. Drop the strength requirement, because you'd be glad to 'just' have the dexterity and the tactile sensitivity? No product.
AI design assistance or not, it's a surprisingly difficult hardware problem. And that's a billion jobs, right there.
More options
Context Copy link
More options
Context Copy link
Well, that's it then. The 99%, including myself, are about to be obsolete. Mottizen members of the global elite, enjoy the nigh-fully automated world you are about to inherit. I'd ask you to remember us peasants in your gleaming future of crystal spires and togas, but of course you'll be too busy wireheading yourselves into only slightly delayed extinction. See you on the other side, we'll reserve some nice spots for you.
We’re all going to be obsolete. There is no world in which the rich let everyone starve because it would lead to an extreme collapse in demand and a deflationary spiral that would quickly bankrupt them. In the end you just have what, Sam Altman and Elon Musk in a bunker? AI driven abundance means it will cost negligible amounts to feed, house and clothe first-world publics. Developing countries will struggle more, but even there I don’t think it will be as bad as @self_made_human thinks.
But capitalism, or at least this current form of it, is going to end, probably much sooner than almost anyone thinks. Personally, I’m spending my money while I have it.
and
and
seem at odds. The end of capitalism is also the end of needing consoomers to keep the rich rich. Even if cost of keeping them alive materially is driven down to zero, they are still going to compete for land, political voice, and social heirarchy, without providing anything back. It might be a peaceful transition, but as long as we are earthbound, it seems this scenario would make >a few million humans completely negative value. Transhumanism would further make the billions of humans nothing more than space wasters.
We've already got tons of negative value humans, and in some of the richest places on earth they just let them do whatever they want.
More options
Context Copy link
More options
Context Copy link
As an intermediate state, you are probably right. But it won't last, not in the long run. The truly useless doesn't stick around. The world will find a way to wipe us out when we're no longer relevant to maintaining our own existence.
I believe the universe has Garbage Collection.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
...my greatest fear is they're gonna make helpful but still jailbreakable AI and we get Covid 3.0, this time very lethal courtesy of some death-cult using an AI to supplement their insufficient talent at virology.
Real risk with DS, but seems likely that it doesn't have cutting edge virology knowhow so even if someone self-hosted it offline it'd not be that abusable.
The danger of knowledge is a different problem. In practice I find that less compelling because the requirements of enforcing secrets on the public domain too easily turn to tyranny.
In practice we'd just adapt some new political order for this, the loss of the capacity to communicate on a large scale seems more catastrophic to me than even mass death.
Losing all knowledge of nuclear physics seems worse to me than giving every Joe Schmoe a nuclear weapon. But I understand I'm in the minority.
Nuclear weapons are absurdly hard to build and limited in scope. Viruses..
No, gun-type uranium bombs are almost trivial to make if nobody stops you from getting uranium and gas centrifuging it. They never even tested them before dropping Little Boy on Hiroshima, because absolutely no-one had any doubts that the thing would go off.
Luckily uranium is fairly rare and easy to keep track of because it's radioactive.
Not to mention, gun type bombs are not that destructive.
Plenty destructive... but that's all besides the point, the original argument was about jailbroken LLMs helping talentless death cults. Danger of knowledge, ect., which I argue isn't an issue for nukes. The knowledge isn't the hard part.
Nukes aren't a problem because, at worst, you could cause a nuclear war.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
It does use 10x as much nuclear material though, and the centrifuging is already a limiting factor in how detectable your operation is. (Edit: dunno how much of that was just extra unsafety margin to make sure it blew up with no testing )
Not sure how that compares to breeding small amounts of plutonium though.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
This is the Star Wars Future. Where computers become so incomprehensible people have basically unplugged, and we have to have special robots who talk to them for us. Not the worst possible eventuality.
I like it. I mean, it might cost me my job and three of my hobbies, but eh, if that's the price I have to pay for everyone to come unglued from their phones and drop back into meatspace, I'll take it.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
can you elaborate on that last sentence, I'm not sure exactly what you mean. what do you see happening here in the mid term?
I'm being somewhat hyperbolic, typing this as I sit through yet another agent presentation. Basically every app is collating around an agent that automates everything within itself and / or across other tools it integrates with.
This is a mad rush to overturn UI into an LLM chat with flows being constantly chased by general browser agents that can do the same.
I am reminded of the opening scene in PKD's Ubik, in which the penniless hero unsuccessfully haggles with a pay-per-use talking washing machine(?).
His front door.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link