aqouta
Friends:
User ID: 75
aqouta
1d ago
And it seems like the grinders are the market being catered to at the moment (PoE, I'm looking at you, because they have been around so long they have dedicated players who've played the game for years as it developed and who can now whip through all the levels fast to get to endgame maps and are then bored if their maxxed out builds with the extra-special gear don't find it enough of a challenge).
This complaint doesn't really make sense in poe, the exp penalty for being overleveled means if you're slow and don't do the sweaty run past almost everything strategy you end something like perpetually 3-4 levels over the zone which is fine, you'll be stronger than you need to be, it's like a built in auto-difficulty adjuster.
aqouta
1d ago
The vast majority of DoorDash traffic. I understand the value when one is three sheets to the wind, or ill, or whatever and getting food delivered becomes more valuable. I don't understand how it accounts for such a large share of orders that it's a huge element in the service economy. I'd think the average person would order delivery once a month or less, it appears to be much more than that.
We don't own a car so if we want to stay in and not cook we turn to it once or twice a week. It's roughly as expensive as going out to eat which is another thing we do quite often. It's definitely not frugal but surely the convenience should explain the appeal even if you wouldn't want to pay that mcuh.
aqouta
2d ago
I'm not the one that defined the problems with AI in relationship to Malevolent AI Gods, Singularities, Outcome Pumps, Roko's Basilisk, Grey Goo/Paperclip maximizers, Magical Genies, Shoggoths, Goodhart demon, etc. These are all inherently Sci-fi ideas. I have no idea how you seem to think any of these relate to how Neural Networks, or Bayesian Networks, or any other of the myriad ideas ML/AI engineers have designed.
These are all intuition pumps and metaphors for explaining the actual intuition. Goodhart isn't a science fiction concept, he explained that under optimization pressures a system that satisfies your stated objective can fail your intended objective and in fact the better the system gets at optimization the better it gets at trading off your intended objective in pursuit of better achieving your stated objective. This thread started out because of an example of this very principle playing out in reality.
Again this assumes that this eventually AI will be sentient
No. And this is the third time it's been explained to you so let me be extra blunt. No part of alignment requires or cares about sentience, wants or experiences for the system. The lights need not be on. You didn't even have to click into the short essay, the paragraph I posted covers this.
These aren't "misalignments" they are are classic errors on a non-perfect model in a system lacking in classic control theory.
Error implies the quantity shrinks as you train it more, this is true of hallucinations, they've been reduced. A hallucination doesn't help you achieve a target goal. This is not true of misalignment problems because misalignment is reaching the goal through unintended and harmful pathways.
The coastrunner boat spinning infinitely scoring forever isn't failing to minimize loss. It's minimizing loss perfectly. But the loss target was not aligned with what we actually want.
Your #2/#3 ignores a failure mode monotonically worsened by the thing that fixes every other failure mode, which is why you keep having to reach for "someone forgot a unit test." All the unit tests pass, it's performing exactly as expected. You can't expect to build smarter and smarter scaffolding to swatt it away from optimizing around your guard rails. Eventually it gets strong enough and you lose, game over.
We don't scream that "software algorithms are misaligned" because some coder forget an unit-test on an edge case.
Grep doesn't go looking for novel ways to satisfy your query, it's simple executing a list of instructions. Deterministic software fails when it runs into states you didn't specify at compile time. Optimizers fail when where you specified correctly but it turns out the specification wasn't what you actually wanted. They're a different class of problem.
#1: Sure, AIs get more capable all the time, how do you know there isn't an asymptote?
Obviously we don't and I'm pretty careful to specify, as I did in the previous comment, that this is one of two valid outs. If you think these systems won't get much more capable then sure, although I'd like you to demonstrate that because people predicting it have been very wrong for a while now. But if that's your contention then I think your whole argument would be much different. The whole ask of the AI safety crowd is essentially to impose an artificial asymptote.
#3: Congratulations you've converged on a fundamental question for any human system
Yes. And we have not solved this problem with humans either although we've spent a tremendous amount of effort attempting to better mitigate the problem. contract law, courts, fiduciary duty, audit, separation of powers, criminal enforcement, a state monopoly on violence. Half the discussions on this forum boil down to disagreements on how exactly we're failing to solve this problem. And that's between humans who are individually squishy, mortal difficult to scale(see corporations and their shenanigans in the 17th century for what happened immediately after we found novel ways to scale human effort), must sleep and have evolutionarily built in game theory instincts.
Sure and there are christians, mormons and muslims in the physical sciences. I don't begrudge people their religious beliefs. If a frontier AI researcher wishes to belief in alignment-problems that his/her/their belief. The Bay is quite literally for Rationalists like Utah is for Mormons. I can still think it's a silly belief and I still think the word misalignment has been invented to describe a problem with system error by a bunch of non-engineers. And the lack of actually solving the problem, or even making progress on it, is indicative of a general grift specifically for something like MIRI.
This proves far too much. It labels all speculation of dangerous outcomes as fundamentally religious. Is climate change fundamentally religious? Fear of demographic replacement? Pick your conservative or liberal fear. This position dooms to you dying out to the first large scale novel problem you encounter.
And I dispute heavily that it's a bunch of non-engineers that are concerned. Some of the biggest voices in the movement are the frontier engineers including the CEO of the lead frontier lab. If you're going to dismiss anything the non-engineers say as not credible because it comes from non-engineers and what the frontier engineers say as non-credible because it's "hype" then you've made your position unfalsifiable.
aqouta
3d ago
We are not. Sci-fi predicts innumerable future realities, most of which to not occur, it predicts an unmeasurable amount of future technologies, most of which never get developed, and it almost rarely ever actually predicts how those actual technologies will work. We exist in a reality shaped world and bad sci-fi fans mistake aesthetics for substance
You invoked sci-fi, to sneer at ideas. I'm happy and prefer to not compare things to science fiction. But you're using "scifi" as a talisman to avoid engaging with any speculation whatsoever.
If you wanted me to further read something you should have linked it.
I did and it remains linked.
Further more I'm not sure how this magical second analogy makes the argument any better. If you'd like to make an argument rather than quoting scripture at me I am open to hearing one.
The argument is made at length in the piece. And any many other places, it straight credulity that you have not seen it.
Start from the basics of existing technology and make actual arguments based on the current reality and the projected reality from our current understanding of Artificial Intelligence. Don't start in make-believe land and attempt to redefine reality as leading to it.
The argument is simple:
- AI is getting more capable.
- capabilities include the ability to do harm, often as a side effect of pursuing some other goal
- We have no reliable way to ensure that the inevitable trade offs of harm and pursuing goals aligned with the true interests(As distinct from the technical interests represented in a prompt) of the one who invokes it, yet alone humanity as a whole.
- At a certain level of capabilities those trade offs might include the permanent disempowerment of humanity or our extinction.
- At some level of capability an AI will be invoked that will find the utility function maximizing route includes an acceptable trade off that results in some, to humans, extinction level threat.
Events like this demonstrate that the pursuit of some goal that to us is of trivial value, passing a cyber security benchmark, warranted the trade off and harms of breaking another organization's cyber defenses. A perfect microcosm of the fear of some future catastrophic event. To dismiss this I think you have to either contest that AI will not become much more capable or present some strong reasoning for how we will solve alignment.
Because it's not possible to solve.
Then we should not build it.
Forcing Sentient Beings
I thought the sneer you had was that alignment people were ridiculously thinking they were sentient? It seems like something you believe more than them.
And to further my opinion can you provide any evidence that in the past decade of AI Safety research have those researchers ever produced anything more than words on paper. The evidence points to this being a grift, has MIRI produced a novel ML model that is more safety conscious and performs any task well, have they produced an "alignment" mechanism or algorithm that actually works?
Believers in some version of their cause are currently littered throughout the frontier labs. You have an impossible standard here where you blame them if they're on the frontier, for clearly not believing in what they preach, or blame them for not being on the frontier as then they must be uninformed cranks. No way to win, you never have to actually think about it. But yes, of course they don't have an alignment mechanism, their whole point is that the problem is incredibly hard and that we have no solved it, that we might need to spend decades solving it, but that the alternative is everyone dying.
aqouta
3d ago
The reason I mock "misalignment" is that because it is used as this nebulous term by a bunch of sci-fi cargo cultists.
you best get used to Sci-fi shaped predictions because we're in a sci-fi shaped world.
If "misalignment" is my sentient AI model doesn't do what I want, well that's assuming the conclusion that model is already sentient. Note in the Yud's analogy, it's genies that are the stand in. Genies are already sentient, they can make decisions on how they listen to you. A genie is not a non-sentient wish granting device that tries to fulfill you wish to the best of its ability. That would be the actual analog to an LLM-Agent. This is the problem with analogies, they require a level of similarity between the two abstractions, when that similarity doesn't exist, the analogy, no matter how clever, does not apply.
If only you had read the next sentence!
Luckily you have, in your pocket, an Outcome Pump. This handy device squeezes the flow of time, pouring probability into some outcomes, draining it from others.
The Outcome Pump is not sentient. It contains a tiny time machine, which resets time unless a specified outcome occurs. For example, if you hooked up the Outcome Pump’s sensors to a coin, and specified that the time machine should keep resetting until it sees the coin come up heads, and then you actually flipped the coin, you would see the coin come up heads. (The physicists say that any future in which a “reset” occurs is inconsistent, and therefore never happens in the first place—so you aren’t actually killing any versions of yourself.)
You refuse to actually engage in any of the arguments being made and are dead stuck on the prior that it's all nonsense, it's epistemic closure.
Yud's whole "misalignment" also just applies to humans
Yes, humans are not generally aligned. If you read histories of what humans have gotten up to then this seems pretty obviously to be the case.
don't due exactly what you want without you enumerating it exactly.
You still don't seem to grasp what is meant by alignment. It's specifically even stricter than that! That's the whole point. It's not enough that they do exactly what you ask, because for complicated enough problems you need it to be much much better than just technically doing what you ask.
It doesn't need a fancy sci-fi term, and it literally isn't solvable. It hasn't been solved in the history of the human race.
Correct, it's a very very hard problem. You are in full agreement with the AI safety people who think there is a good chance we're on the path to annihilation, you just inexplicably seem to think it isn't a big deal and refuse to elaborate on why besides sneering and not engaging with the arguments.
aqouta
4d ago
The Ottoman Turks ran the territory, not the Palestinians, and the whole mandate period was a question of precisely how that territory was to be handed off from legitimate British control to local management. The whole of the post-Ottoman territory was going through this proto-nationalist phase, the Jews were not uniquely trying to cut out a piece. Many of them were fleeing other previously Turk run territories going through their own nationalist movements. I'm not trying to say they behaved as angels or anything, there were lots of tit for tat escalations going on as different populations laid claim. But to just gloss it as "in the 1919 apropos of nothing the jews hatched a plan to conqueror the peaceful Arab land" is ahistorical. Much of the original land was bought from Arab landlords who were themselves granted ownership from the Ottomans as legitimately as such things were done back then. Which certainly sucked for the Arab tenants that the new Jewish land owners kicked off the land and I don't blame them necessarily for fighting back but you're putting a hell of an isolated demand for non-national sentiment on the Jews just after they watched millions of their brothers and sisters get slaughtered.
aqouta
4d ago
From 1800 to 1919 there was a negligible amount of violence against Jews in Palestine while they comprised 4-12% of the population.
It's pretty dishonest to not include the fall of the ottoman empire in this analysis. A whole lot of events happened between 1919 and the nakba. The Jews aren't the only actors on the world stage.
aqouta
4d ago
So if you want to claim that's misalignment, then Yolo has been misaligned for 12 years!!! The Horror!!! However that feels like definition creep to better encompass an argument.
I mean yes obviously? It just isn't a problem because YOLO isn't dangerously capable. Alignment isn't a synonym for order following. It's not definition creep, ai safety people have been calling all of this shit unaligned forever and your sort has been mocking them for it forever and never actually making any progress in alignment.
There are three kinds of genies: Genies to whom you can safely say, “I wish for you to do what I should wish for”; genies for which no wish is safe; and genies that aren’t very powerful or intelligent
aqouta
5d ago
I look at atmospheric CO2 concentration and I don't see the line doing anything but going up and to the right.
When it goes to the left is when you need to worry, up or down.
aqouta
6d ago
I'm not the one who originally tried to make some point about what the speculative revenue tells us about Anthropic.
aqouta
6d ago
I think that journalists were souring on Silicon Valley before AI kicked off
souring? They've openly hated Silicon Valley for a very long time.
aqouta
6d ago
Of course not, but neither is the comparison to ebay. There aren't audited numbers so we're all estimating based on available information.
aqouta
7d ago
Yes, it was roughly $5B in 2025 to Ebay's $11B, also interestingly the best guess for Anthropics revenue in June 2026 was also $5B. Things are moving quickly and Anthropic is projected to have ~$60B in revenue over 2026 with error bars as low as $45B and as high as $80B. Putting it in the class of Goldman Sachs, Intel and IBM as far as revenue is concerned. And well, the straight lines on graphs interpretation makes this year to year change pretty interesting.
aqouta
8d ago
From a psychological perspective I'd say pull half your winnings out and distribute it hedging against some scenarios you can think of. If AI goes off and the tech stocks don't reap the outside gains what might? A broad index of companies like the S&P500 might get commodity priced intelligence for very cheap and become much more valuable because their size and market position give them the best chance to leverage the boons. Other inherently scarce commodities like minerals and land will likely preform well in any scenario where labor, both physical and intellectual, falls to marginal. So mining companies with mineral rights and REITs might pay off. I'm sure you can think of some other scenarios. In most worlds these investments aren't big winners, but in the worlds where they are you'll be very happy you halved your direct AI exposure, which will be plenty anyways if it pays off, to hedge in them.
aqouta
8d ago
It's such a wide universe that it's hard to really just dump a suggestion without any feelers for what you'd be interested in. Going off of RPGs and with a fairly wide variety
Path of exile(1 or 2): Diablo style top down click to move loot game with near infinite complexity and a deep market economy. Both games are popular and continue to have near quarterly large patches. It's an overwhelming amount of complexity that people either love or hate. poe 1 is free and poe2 has a $30 buy in until the end of the year when it goes out of early access. The buy in gets you store credit which you could use to buy stash tabs(good or organizing your loot) that apply to both games.
Baulder's gate 3: basically dungeons in dragons as a single player adventure, extremely well received. If you're at all interested in the dungeons and dragons experience this is worth a pick up.
Hades 1 and 2: this is a rogue lite(a genre where you start essentially new games repeatedly with random powerups to try and beat the story, lite means there are inter-run power ups and like means no inter-run powerups essentially) with good RPG themes, great if you like a quick self contained run game that leaves you wanting to play just one more.
slay the spire 1 and 2: sts1 kicked off a whole genre of turn based rogue likes. It's a card game in the rogue lite style.
Blue prince: Another rogue lite at a slower pace where you explore rooms and try to get deeper into an ever changing house as you piece together a neat story.
Clair Obscur: expedition 33: A French JRPG with great atmosphere, not really my type of game but really well received.
Doom: It's doom!
Factorio: Ok, not much of a role play game but you're on the motte, you'd probably like factorio. Build a factory to turn stuff into stuff you need to build a bigger factory.
Hogwarts Legacy: If you like harry potter then it's a very competently made RPG in that game. The game play isn't anything to write home about but it's an RPG set in hogwarts, what more do you want?
Binding of Isaac: Perhaps the GOAT of Rogue likes, old but holds up.
That's what I find scrolling through the last couple years of my steam library. I'm sure I've missed some. If you have any games you've particularly liked in the past or mechanics you're interested in I might be able to refine the search a bit.
aqouta
9d ago
We were going to drive up to the UP yesterday and it has canceled that trip.
aqouta
9d ago
I don't know, I don't get particular enjoyment out of straight or gay relationships, most media I prefer isn't really improved by relationships though. Stories well told in my opinion strip out most things that aren't critical and romantic relationships are either very critical to the story or should be basically minimally addressed imo. And if it's a story where the relationship is critical, which I probably wouldn't want to read in the first place, I'd probably prefer it to be something more relatable like a straight relationship.
aqouta
10d ago
They're a small subset really, I ran into them occasionally because I work at a financial institution and you can tell they're feeling you out for if you have one of the seven figure roles. But they're greatly outnumbered by yuppies who have their own careers.
aqouta
11d ago
I propose a new law, the law of bulveristic recursion, if your bulverism can explain both sides of an argument you need to actually stop trying to read minds and address the actual fucking points. This shit is so exhausting. There are pages and pages of arguments that these people have produced about why they're concerned about ai safety, if they're wrong show that they're wrong, make a convincing argument that they wrong, but construction epicycle on epicycle about how they really must have been grifting for the last twenty years because of some subtle game theory to get rich on the off chance that AI became huge. If they had this much foresight they could have just invested in a couple companies and got rich anyways.
aqouta
11d ago
My own work and that of my colleagues. We have a team harness we iterate on and improve with business and infrastructure knowledge. The key for orchestration is that it allows you to isolate context windows so that your main workflow isn't polluted with unnecessary details. Every token in context that isn't useful degrades performance. Just the process that produces the plan for implementing an update might use half a dozen subagents.
aqouta
11d ago
Maybe you're working with scummier companies but it's not at all apparent to me that the goal of any company's customer support organization is anything other than supporting their customers, which they often do poorly because customer support is a cost center. Maybe if your modal interaction is trying to get a refund you aren't entitled to, but my biggest problem has always been when my interests and the company's basically align but the support agent doesn't know how to move some lever. The company doesn't want to pay a support worker or for tokens necessary to keep me in a kafka hell, nor do they want to piss me off as a customer to the point where I stop being a customer.
aqouta
11d ago
The phrase "vibe coding" is a thought terminating cliche. Certainly people who don't know anything about software engineering who tell claude to make an website for them and then posting a localhost address on social media are funny disasters and tales of their hijinks are spread widely. But anything you're using an ai to do in code would be better accomplished with a custom harness and some agent orchestration to manage context density. where are you even finding public examples of people's set ups besides the posts of people making fun of failed examples? Very strong selection effect.
aqouta
11d ago
They're behavior is really straightforwardly explainable by the things they've been saying this whole time. Like I don't know why you insist so strongly on reading tea leaves and divining the contents of secret cabal meetings. They say what they believe and are worried about and then go out in the world and do the things one would expect them to do given those beliefs and concerns.
aqouta
11d ago
Some other jobs will definitely get a lot more automated, but not so much. Trying to replace customer service agents with chatbots will not be "improved customer service, all problems solved immediately and correctly" but more "we don't have to pay real people to do this shit job anymore, and the customers have to accept it or lump it, they have no choice" money saving.
We're very close to where I'd rather deal with a frontier model doing customer service than a person. The main rub is they probably won't serve us frontier models. I don't know how often you've actually dealt with customer service on out of distribution problems but it's not pretty, and the in distribution problems can basically be straight through processed already with a minimal ai wrapper.
I agree that the grinders exist, and they care about the loot and juice systems being rich enough that they can keep climbing the grind for better gear that will make it so that you can more efficiently grind for even better gear cycle. Where I disagree is that this really has any negative effect on the casual player who spends 12 hours on the acts. Because you definitely don't need to grind in the acts to keep moving forward, in fact it's the opposite, almost every new player kills way more monsters than they need to.
More options
Context Copy link