aqouta
Friends:
User ID: 75
aqouta
1d ago
My own work and that of my colleagues. We have a team harness we iterate on and improve with business and infrastructure knowledge. The key for orchestration is that it allows you to isolate context windows so that your main workflow isn't polluted with unnecessary details. Every token in context that isn't useful degrades performance. Just the process that produces the plan for implementing an update might use half a dozen subagents.
aqouta
1d ago
Maybe you're working with scummier companies but it's not at all apparent to me that the goal of any company's customer support organization is anything other than supporting their customers, which they often do poorly because customer support is a cost center. Maybe if your modal interaction is trying to get a refund you aren't entitled to, but my biggest problem has always been when my interests and the company's basically align but the support agent doesn't know how to move some lever. The company doesn't want to pay a support worker or for tokens necessary to keep me in a kafka hell, nor do they want to piss me off as a customer to the point where I stop being a customer.
aqouta
1d ago
The phrase "vibe coding" is a thought terminating cliche. Certainly people who don't know anything about software engineering who tell claude to make an website for them and then posting a localhost address on social media are funny disasters and tales of their hijinks are spread widely. But anything you're using an ai to do in code would be better accomplished with a custom harness and some agent orchestration to manage context density. where are you even finding public examples of people's set ups besides the posts of people making fun of failed examples? Very strong selection effect.
aqouta
1d ago
They're behavior is really straightforwardly explainable by the things they've been saying this whole time. Like I don't know why you insist so strongly on reading tea leaves and divining the contents of secret cabal meetings. They say what they believe and are worried about and then go out in the world and do the things one would expect them to do given those beliefs and concerns.
aqouta
1d ago
Some other jobs will definitely get a lot more automated, but not so much. Trying to replace customer service agents with chatbots will not be "improved customer service, all problems solved immediately and correctly" but more "we don't have to pay real people to do this shit job anymore, and the customers have to accept it or lump it, they have no choice" money saving.
We're very close to where I'd rather deal with a frontier model doing customer service than a person. The main rub is they probably won't serve us frontier models. I don't know how often you've actually dealt with customer service on out of distribution problems but it's not pretty, and the in distribution problems can basically be straight through processed already with a minimal ai wrapper.
aqouta
1d ago
In the medium term it's all about the harness. You need to have mr. claude digest your project, document every inch of it, have a glossary of terms so it know what you mean when you say threads lag with high comment counts and with tokens measured in the hundreds it can have densely useful context. The breaking down tasks into easily digestible chunks is trivially handled by project documentation and an orchestrator commanding subagents. Building and maintaining these harnesses is much like coding used to be, it takes thinking about the SDLC, the architecture, reacting to failures of assumption about how your agents will interact with the harness and patching those failure modes. It's true that we are not too many turns of improvements from that all being something the models can do themselves if you just ask them to first digest your project.
aqouta
2d ago
I recommend reading Scott's piece that came out today on this very topic.
The difference is in epistemological certainty and scope of actions. The police don't kill without a very high certainty that it is necessary, and even when they do make mistakes the scope of the mistake is that "only" individual people die. This is extremely different to AI safety policy gambling the fate of society on epistemics one or two orders of magnitude less certain than a policeman's threshold to inflict lethal violence.
The class of bullet biting asked of the yud crowd and what produces risk ww3 results is the equivalent of asking "what if enforcing the law on child pornography requires you to arrest a politician but that causes the politician to start world war 3 in order to overturn the state and prevent himself from being brought to justice". No one is prescribing lethal violence until we're many unlikely levels of escalation past where we'd likely go. Even bombing data centers isn't necessarily lethally violence, and to be clear bombing data centers is itself an unlikely far off escalation.
I suppose it depends on how successful you think nuclear non-proliferation actually was. In my view, pretty much every serious nation-state either openly has nuclear weapons or a turn-key program for rapidly obtaining nuclear weapons if neccessary, South Africa is the only country that has ever willingly denuclearised amidst uniquely dysfunctional transition dynamics, and this is all while nuclear weapons have an extremely concrete existential risk profile, no dual-use potential, and are economically net-negative to maintain; none of which apply to compute.
The comparison I'm trying to make to nuclear proliferation is that you can have these multi-lateral treaties with some teeth and they don't seem like they lead unavoidably to some kind of dystopian state or world war three. We can dig into how much they prevented proliferation, and I think a good deal, but there are other wrinkles in that kind of comparison. Most notably that signatories of the treaties that don't push the frontier of AI will still be able to access state of the art inference, just not the ability to push new frontiers. This is like getting all of the benefits of a nuclear umbrella without needing to go through the trouble of enriching uranium. No one has to go without compute, they have to go without absurdly ridiculous amounts of compute that aren't able to be verified aren't working on training a frontier model. They can have the datacenters, they can run their own inference on them. They just need to have some mechanism to verify they aren't doing the very expensive thing that is training a frontier pushing model. And to almost all nations besides the united states this is a pretty sweet deal and arguably China(I would disagree)! Because if there was a race instead of these treaties they would lose the race and in a lot of cases things would go quite badly for them. The game theory here is significantly more tractable than nuclear proliferation.
As far as I am concerned both of these look a lot like unbounded yet finite tyranny, even if such tyrannies might be various degrees of comfortable along the way.
If powerful AI comes about that this is a possible plan then all futures have that character. Your sit back and watch plan included. You're not in any way avoiding it.
I do, in fact, think this thing could kill us all, as I've mentioned a few times in the original post and replies; the same way as many mundane risks could kill me at any moment, and many other existential risks could kill us all as well. As a result, I spend my time grilling and enjoying my life while the going is good, until it inevitably ends one way or another.
Again, say you thought the chance of human eradication in the next 20 years was 20%, like many safety people do, would you still council surrender to that fate?
Implicit in such claims is that "we should use the capabilities army to stop anyone else from mustering an army", hence turning into the bandits that you were so afraid of in the first place.
Preventing others from turning bandit may be something bandits do, but is not centrally the problem of banditry. The legitimate police also do this. One who fights monster should see to it that they themselves do not become monsters, but we have different words for monster fighters and monstrosities for a reason.
the obvious differences between HEU and compute is that there is no justifiable civilian use for HEU
How curious the need for the H in that acronym, because obviously enriched uranium does indeed have civilian use. likewise safety people are happy to allow inference datacenters so long as they, like nuclear power plants, willing to register and make clear they aren't doing weapons grade enrichment/frontier model training. It's absurdly analogous.
Any serious attempt to control compute will be useless at best
Facts asserted not in evidence. I happen to think at best it prevents the destruction of all value in the known universe. The odds of this are of course reasonably disputed even by me, but at best? Come on.
and lead to eternal tyranny / WW3 at worst.
I'm sorry, what's the pathway to ETERNAL tyranny? If you could guarantee our human institutions would endure for an eternity then that's quite the prediction!
No, it is indeed broadly true of 20th century Marxism. Traditional Marxism considers the communist revolution to be a final, eschatological event capable of ending the class struggle, and with the end of class struggle an end to the suffering, exploitation and conflict plaguing humanity for the rest of time.
This is a different type of end. I will again point you at the very important difference between finite and infinite. Infinite does not mean "very large". Marxists as far as I am aware considered a transition to communism inevitable. There was a concept of very bad times spent in the desert not achieving communism that they could accelerate their way through, but human annihilation was not the default path.
while the Scott school is talking about ushering in utopic superintelligence by 2040.
You've very fundamentally misunderstood Scott if this is what you take from the predictions. He's very clear about the point of these plans being to try and lay out a path for the good future while what he's worried about at the bad ends. These are compromises with reality. A desperate attempt to steer us out of race conditions that lead to hell. Scott is not an accelerationist and I believe would be very happy to press a button pausing development. He just reasonably believes no such button exists.
I'm not saying AI safety has lead to anything this bad yet, but when you start talking about nuclear war being acceptable to achieve your aims
This is you punished yud and folks for biting a bullet that you demand they bite. It's an annoying behavior. Again like a libertarian on hearing that you want to ban child pornography demanding that you bite the bullet that you'd kill someone over it because ultimately any law is backed up by the force of the state. It's both at once a childish reduction and a refusal to engage with the on the ground reality of international treaties. Nuclear proliferation is backed up by the threat of WW3 and yet we have no run into WW3. You'd doom us into any stupid tragedy of the commons problem with this nonsense denial of the coordination mechanism that is clearly available to us.
Be significantly more skeptical of inside-view arguments that purport infinite stakes within an imminent timeframe
Be significantly more skeptical of proposals with the potential to cause major harm that assume an imminent inside-view framing with infinite stakes
Be significantly more skeptical of being convinced to do things with a good chance of making the situation worse based on the premises of your inside-view framing
I mean like real suggestions, not vague tone policing. I'm plenty skeptical, that's why I'm at 20% and not 100%. you're interacting with the end product of much skepticism. What this seems to cache out to is "don't believe in AI x-risk" and sorry, that isn't where the evidence points. I'm asking you to actually consider what you'd do if you thought this thing could kill us all, not what you're negotiating for given you don't believe it does.
aqouta
2d ago
I don't think there is any single person here that believes all those things. You're conflating the traditionalists with the black pillers so a lot of those conflict, and both stances combined are probably a minority opinion on here.
aqouta
2d ago
You'd do it with a harness and some md files surely. Have it go through all moderation decisions and distill the general rules into a file or set of files, keep all the base decisions on disc but only pull them into context when relevant. hobbyhorserule.md need only be parsed when the model thinks a post might violate that rule. Basic harness building stuff.
aqouta
3d ago
The difference is that "war" is a concept and "AGI" is, at least theoretically, being physically manifested. If the enemy is "the army about to invade" then you probably don't want to join the enemy army to convince them to turn around.
You're in a war naive society and someone publishes a white paper: "people can use violence to take whatever they want from unarmed populations" You want no one to be dispossessed so you can join the capabilities army explicitly organized around preventing banditry or you can abstain. If everyone like you abstains what do you think the equilibrium on banditry will be? Certainly abstention can be defended under a number of philosophies but joining is a perfectly rational decision.
This is the exact logic that I am saying leads to very bad outcomes. If you believe that the stakes are the infinity of total annihilation, that total annihilation is imminent, and that any leverage thus has infinite value, then you can justify doing anything to gain even miniscule leverage, on nothing but the strength of your own belief. The anime character draining the lives of everyone across the world because they believe that only they can bring the world to salvation is not the protagonist of the story.
I'm responding to this scenario:
I think if you accept the premise that the world is truly doomed on such a short timescale, then having a few more years is better, actually, than trading billions of QALY's for some nebulous idea of influence that has had no empirical benefit.
The reasoning here is faulty in the way I outlined. You're critiquing the assumptions you baked into your own scenario. I would personally inject a lot more nuance instead of these stark terms but I was willing to engage in your hypothetical. My P(doom) is much closer to 20% than the 100% you apply. 20% is still huge and easily worth mitigating with some bilateral treaties. It's you lot who seem to conflate bilateral treaties with like eternal tyranny despite no such tyranny coming from previous treaties.
believe in an imminent Event
I mean not exactly? Communists believe capitalism is inherently unstable and will eventually give birth to communism once buried under the weight of its contradictions or whatever. But they make no real promises of imminence. For that matter AI safety people mostly don't even commit to claiming certainty that LLM architecture will even get there, only that it could and we ought be prepared in case it does.
infinitely high stakes attached to the Event
Not really true of Marxists. As far as I can tell Marx himself estimated the stakes at finite, maybe in the ballpark of alleviating serfdom or avoid depression. nuclear proliferation is a much better comparison. The comparison falls pretty flat in general here.
that they are the only ones capable of making sure the Event leads to utopia
I don't know, kind of if you squint? I'll point out that at least the Yud brand is very happy to keep us at the status quo rather than trying to usher in the eschaton, it's the accelerationists who are really pushing for utopia. Certainly both have a vision of the good, but this is where I think liberalism, christianity, or basically any have this quality.
this set of beliefs lead to the justification of many horrible acts that did not, in fact, meaningfully facilitate a better outcome of the Event.
What horrible acts? Establishing bilateral treaties and, in the same vein of cartoonish libertarian discourse that equates having laws at all with infinite tyranny because noncompliance eventually escalates to violence, we might have to enforce the treaties with violence? This chain is incredibly unconvincing and conflicts violently with the previous clause because it's precisely the opposite group of people who want to do these "horrible acts" as who want to usher in the utopia.
In short this whole comparison looks incredibly forced. It looks a lot like out group homogeneity bias. And I have no earthly idea what your position actually is. You seem to simultaneously want to let it rip because you oppose safety people who want to slow down, while also distrusting the people pushing it forward? What would you actually advise the people worried about ai existential risk do?
Are you in contact with the rdrama folks? They seem to have similar ambitions on a similar architecture and if this kind of project is to succeed it seems a big advantage to start out with a large and varied userbase. And also them being... often objectionable... provides a kind of trial by fire of how we're going to handle tolerating interlocked communities with different norms.
aqouta
3d ago
"Might" is a weak word. Historically, "join your enemies to reform them from within" has pretty much never achieved the desired aims; it seems straightforwardly true that Moloch has won in terms of incentives to push capabilities and accelerate.
Framing this as a simple enemies or allies distinction is sanding away anything useful. Like if the "enemy" was "war" then one might naively believe that creating an army is joining the "enemy". But if what is bad about war is the killing and oppression then you don't prevent that by disarming, only ensure that the people who win are less noble and less capably opposed.
It's already the case thst people accuse those like yud of being an uninvolved crank who doesn't understand the problem because he doesn't have experience building the models. Ai safety people would have less, not more influence if they abstained.
I think if you accept the premise that the world is truly doomed on such a short timescale, then having a few more years is better, actually, than trading billions of QALY's for some nebulous idea of influence that has had no empirical benefit.
Given the infinity of total annihilation any qaly calculation is swamped by even a tiny amount of leverage on probability of averting disaster. You'd need to set the capabilities frontier back some appreciable fraction of the sun's remaining lifespan for this to be a meaningful calculation. Fortunately I think having people in leadership positions of the labs gives us more than a tiny amount of leverage.
Perhaps. It would be easier to engage with this if you elaborated on why you think this is the case.
This is of course vague because you didn't yourself outline what these structural elements are. Seems like some vague notion of the necessity of action to avoid catastrophe. If "we'll be out competed by peer nations who liberalize" is too much of a stretch then we can point to calls to shut down the border lest the nation be dissolved or any number of other calls to action. The calling up of a militia to defend from Indian attacks. The point is that it's silly to compare any movement that wants to act for any reason to communism in order to rub its nose in that feces. You'd leave yourself completely paralyzed.
aqouta
3d ago
Yeah, this is my point. Empirical evidence is that the sum total of AI safety's achievements is, against their own goals, to accelerate the founding of OpenAI/Anthropic and the AI race dynamic, while having made no progress on their actual goals (not even the authors expect Plan A to realistically happen).
After the transformers paper came out it was only a matter of time, this way at least the labs are full of safety people who might be able to prioritize alignment work. The alternative was optimistically we get a year or so more time but no influence.
Right now, we are at the equivalent stage of AI safety, where it is in vogue in certain intellectual circles; we have not yet seen what will happen with AI safety, but it shares many structural similarities to the Marxist movements a hundred years ago.
This prove too much. early liberal, proto-capitalist, thought also shared these structural elements.
aqouta
6d ago
You've managed to avoid having to actually contend with the arguments that they make by hallucinating a totalitarian drive that none of the people in the movement would even personally benefit from. How bilateral international treaties turn into a one world government is left as an exercise for the reader. This is weak stuff.
aqouta
6d ago
Do you or do you not think that AI safety people are primarily concerned with whether AI kills everyone? That's the contention. you claimed that they weren't, no you argue a different position.
aqouta
6d ago
aqouta
6d ago
Honestly all of us just think you guys are making yourself look plainly retarded by making obviously fallacious arguments. If I feel any embarrassment as a result of these posts it's that I'm vaguely associated with the sort of discussion forum with people who post like this. Somehow people whom I have explained the economics of inference too multiple times continue to falsely claim that inference is unprofitable and never will become profitable. And then there's the bulverism, "people who are concerned with ai safety just want regulations" have you ever met literally anyone involved in this field? Seriously? They're one of the highest concentrations of libertarians one could select for outside of an Ayn Rand fan club.
aqouta
7d ago
why waste the universe's time?
What? Is it going to run out? Might as well try.
aqouta
7d ago
it's simply impossible that there can ever be safety if we live in a world where AGI can be created via human hands.
AI Safety is what you do when you come to this as the plain conclusion under normal circumstances and then start thinking about how you might engineer extraordinary circumstances because the alternative is resignation to oblivion.
aqouta
9d ago
Eh, I think it's wrong to think about anyone as "the villain" besides the evil possession magic. Bear is deeply flawed, most centrally he's chronically passive and indecisive. In TLP terms he wants the company to want to pay him more but not to have to ask for it. The only action he ever took to get what he wanted was something he calculated as being totally unable to help, making a throw away wish. Then he's in the relationship he knows something is wrong but chooses to basically ignore it and just go with it because he's cartoonishly incapable of acting. All of that and more is true of him, but after he made the wish that he had no way of knowing would be interpreted monkey's paw style in the worst possible way there was nothing he really could have done besides killing himself. I'm not quite willing to raise "if you don't kill yourself to right a wrong you didn't intentionally commit" to the level of villain. He's a coward, but that's a forgivable sin.
aqouta
13d ago
are you only interested in her because she's green and malleable and can be moulded to what you want?
I agree many women seem to think suspiciously in these terms but it seems like a real theory of mind failure. Men by and large are not looking for the hassel of having to mould a partner into what they want besides a handful of fine adjustments at the end - I have taught my wife the proper way to load a dishwasher.
aqouta
16d ago
I'm a little confused. At least in the Chinese case in most major cities there are "Chinese schools" you can send the kids too that are most analogous to me of the "catholic school" I attended growing up which was a couple hours a week at the catholic school. Where they learn Chinese and cultural stuff. I'm slowly learning some Mandarin and our plan is that the kid(s) will attend Chinese school and be able to speak Chinese but that the language of the household will be English. I've never heard of the concept of a one parent language and it indeed sounds like insanity.
I propose a new law, the law of bulveristic recursion, if your bulverism can explain both sides of an argument you need to actually stop trying to read minds and address the actual fucking points. This shit is so exhausting. There are pages and pages of arguments that these people have produced about why they're concerned about ai safety, if they're wrong show that they're wrong, make a convincing argument that they wrong, but construction epicycle on epicycle about how they really must have been grifting for the last twenty years because of some subtle game theory to get rich on the off chance that AI became huge. If they had this much foresight they could have just invested in a couple companies and got rich anyways.
More options
Context Copy link