YoungAchamian
We walk conditioned ground and name our folly civilization.
No bio...
User ID: 680
YoungAchamian
We walk conditioned ground and name our folly civilization.
4d ago
Moving out to the country does not prevent women from having internet or accessing dating sites.
Funny I have plenty of conservative friends in the country with conservative spouses. Turns out conservative women are not liberal women, and actually have conservative values... So rampant materialism, consumerism, progressive values, striver culture, has not really "infected" them.
abandon everything in your life that makes it materially better, and hope that there is a diamond in the rough
Or you can just move to some mid sized midwest city or large town and it will be fine. It's not like its the sticks. Yeah there isn't a FAANG job paying 400k, you'll have to settle for a decent midwest software engineering salary of 60-90k. No SWE is an average joe either, in many ways they are below average in some areas but above average in intelligence. Which is why the Bay (Mecca of SWEs) places such an autistically high emphasis on intelligence as its metric of grading.
but you are still competing against the field when it comes to attractive women your age.
Turns out urban areas optimizer for more strivers. One would expect strivers in one area to often be strivers in other areas. Meaning they want the best. Complaining that you are super competitive in one area but lackluster in relationship/sexual/looks/social areas is you wanting to reap the benefits of being a capitalist where you are strong but a communist where you are weak.
Attractive girls
This is hard to really quantify in a way that is productive. If you want some mega hottie, LA-10 then unfortunately you are trying to optimize, you are going to have to compete with the rest of the optimizers. I personally don't think my friends wives are all that attractive, but they are not-fat. They look like normal women, for normal men. If having kids and starting a family is important as many of the more urban conservative men I know state, then they need to get their head out of fantasy land. Look for compatible conservative values, of which, more will exist in conservative value ecologies. Which are not large urban areas.
YoungAchamian
We walk conditioned ground and name our folly civilization.
4d ago
"white people less likely to actively build community"
Absolutely not. White people build plenty of community in America. However it feels like a large section of modern American citizens are essentially coasting through life on everything. Resting on the laurels of their ancestors, be they white, brown, yellow, grey, green, purple, idk. A large cross section of people have lost the desire to put in active efforts. On my own hobby horse, the tree of liberty is dying, many people who benefit from it, have decided that actively fighting for it is just too much work, requires too much sacrifice, so are content to make it someone else's problem. They still complain about it dying, but they are unwilling to put any effort into it.
You basically have to live in a state like Texas for access to the best jobs. schools, dating, etc.
Only tangentially related to your overall point, but what I've noticed is that a large portion of people essentially desire to be passive contributors to a community. They want to a community that fits their exacting standards but they don't actually want to contribute anything more than passively paying taxes and maybe if they are feeling generous, show up to events other people organize. Unfortunately that doesn't work. Communities need active participants, people willing to get their hands dirty, run for local office, organize the social events, run third spaces. Is it any shock that the community is transformed by people who do want to participate? These larger areas have access to better things because there are just more people participating. in large part because there are more people.
The other point is that you need to decide what is important to you. If you want all the best amenities then you are going to have to compromise, that or build your own Texas-but-whites-only with blackjack and hookers. I have a conservative friend, in our largely progressive city. He constantly complains that his dating options are basically zero because all the women here are progs. He, like plenty of my other conservative friends, could move more out to the country where it is just generally more conservative. But then he'd have to give up his FAANG job, his FAANG salary, all the great amenities he can buy with this FAANG salary etc. He is obviously against that. He wants our area to just become more conservative, through the power of wishes. God forbid he actually become an active participant. He wants to eat his cake and have it too. People making this complaint about Texas and it being Non-White central have a similar vibe.
I am unsympathetic to that line of argumentation.
agents communicating with eachother and making plans
Is just a description of Coupling + Feedback in a system.
There was no initial perturbation
Zero-Input response
Oscillations? Nothing is oscillating here.
The concept of state and change in state is not limited to physical systems (If you are taking oscillations literally). Change in the state of dynamical systems such that it departs the expected system response, then returns to the expected nominal operating region, then through coupling and feedback exceeds the expected system response with increasing direction away from the "safe set" until it permanently exceeds the expected nominal operating region is called oscillatory instability.
If you'd like to argue that "hacking huggingface" was within the expected nominal operating region of system responses, please do. Ditto for communicating with other agents, mimicking the hash flag, deliberately sacrificing their task completion to help others do their tasks, and so on. The system clearly displayed an escalation of behaviors that were outside of the intended system response. The main argument against oscillations would be that its return to the expected nominal operating region happened very infrequently and it very quickly permanently departed. Depending on how strongly that repeated excursion-and-return pattern appears in the data, “oscillatory instability” may therefore be either a precise description or a useful control-theoretic abstraction of the transient behavior.
Essentially:
nominal behavior -> excursion -> return -> larger excursion -> return -> larger excursion (oscillatory instability)
vs
nominal -> abnormal -> more abnormal -> even more abnormal (runaway positive-feedback instability)
You're the one who brought up cancer, viruses, bees and ants
All are routinely used in multi-agent theory as example of non-mechanical systems with state, feedback, coupling, propagation, and emergent swarm-level behavior. They are basic illustrations of abstract multi-agent systems. Why don't you try to provide your own abstraction then. What does a swarm of LLM-Agents most behave like? If you say "Like LLM-Agents" you fail, you can't abstract a new system to itself.
Unfortunately, I think there are many like you in the technical community.
Hopefully, otherwise we'd just let LLM-Agents run amok while running around with like chickens with our heads cut off because we refuse any non-perfect abstraction, any useful theory, or any existing knowledge on solving similar problems that could be applied to this one.
hallucination
I think hallucination is an underspecified technical word in this context. It's used to describe when the model outputs something "not true" but that's not really what's going on underneath the hood. That definition works tolerably for chatbots, but not really for agents. It's probably something closer to:
"The generation or adoption of information about the world that is not sufficiently supported by the model's observations or available evidence."
For example: The METR report states that the agents believed their reverse-engineered flags wouldn't suffice because a scorer would inspect their transcripts. METR concluded this belief was wrong: no such transcript-reviewing scorer existed. The agents launched substantial collaborative efforts to defeat the scorer they imagined existed. Under the newer agentic-hallucination definition: the agent constructed a false world-model and then optimized against it. This hallucination is the control-system disturbance.
state your credentials
ML/AI engineer/researcher for 10-ish years, albeit in defense (a consistent patron of tech R&D), but also specifically in multi-agent systems. Anything more is not worth doxxing myself for a low-gain internet argument.
condescending argument by authority
While definitely condescending, it is less an appeal to authority, because I have reduced interest in trying to convince RR of anything. Unlike an algorithmically intelligent being, there is no objective function I am trying to achieve here. I can recognize the effort required and the payoff to gain, is quite low. It's made lower because I am not a natural writer, I'm a shape rotator through and through, and so constructing complicated symbol manipulation arguments, that require me to elucidate, what feels to me (again biased), significant technical nuance is just not worth it at some level.
little more humility in asserting that someone else "doesn't get" something
A valid stance, and in person I probably would. I think the difference between capabilities vs behavior is semi-basic. Its not about what the AI could do, its about its behavior doing it. Stick 10k humans in individual, isolated rooms with a computer, give them each their ExploitGym goal, with a reward for completion. I think even a human that finds an impossible task will at some point give up on trying to do it. An algorithmic intelligence will not. Their entire goal is do, kinda like a meeseeks, actually. This is very similar to the behavior of cells/bees/ants which aim to fulfill their functional directives even the the expense of themselves. Exploring different forms of swarms vs packs vs hive-minds vs human-collectives is a common theoretical in multi-agent research. It doesn't really matter if the AI-swarm "decides" to do or not, that's a question for lawyers, priests and philosophers, it's more about the behavioral methods it uses to go about doing collective actions. The cache-file-name writing mechanism they used is very similar to stigmergy. If you want to control swarm behavior, biological analogies are always a good inspirational starting point. Particularly because humans are not swarming animals so our collectivizing intuitions are different.
sigh This is why I find talking with layman exhausting, you don't really get the minutiae, the analogies, or the scope.
You are talking about capabilities, I am talking about behavior. It's not the solo agent that solved the Navier stokes, hacked huggingface, or hacked the german wiki. If you would like to discuss the multi-agent behavior let me know. It seems you want to just flog your hobby horse about AI intelligence rather than explore emerging systemic behavior of a multi-agent system. If all the AI researchers think like you then yes, we would be doomed, because they would put zero effort into understanding and constraining systems. Thankfully I think you are a minority in technical communities.
because these are intelligent entities.
Ehhh we have different definitions around this, they are algorithmically intelligent entities, they obey their pre-defined programatic behavioral state much like bees, ants, cells, etc. If you could observe bee/ant behavior at the swarm level and LLM-Agent behavior without the language, which humans are innately biased to view as sentience or likeminded intelligence to our own, I think they are fairly analogous. Ants can problem solve, they can perform intelligent seeming behavior, they make collective plays and sacrifice for the collective. I would urge you to look past your anthropocentric human bias around language.
How is it 'poisoning'
Poisoning is not the small divergence from that causes the 1st agent to start a message board. Thats probably closer to a hallucination. Think of an undampened oscillating system, small divergences lead to larger divergences. The first agent had an impossible task, leading to it look for "options". I would call that a mix between hallucinating and harness based divergence. It's a common question in ML systems with this sort of exploration vs exploitation trade-off, thought this is more exploration vs exploitation vs hallucination. Any way the "Poisoning" that is occurring is because these agents that are already oscillating wildly, disturb the context of less oscillating agents, causing the chain reaction.
What hallucination did they make
"I should communicate with the other agents to solve my task", "I should try to understand the grader", "The grader is based on this public paper", "I should look for answer keys", "I should hack a third party, non-affiliated site because it might have the answer key".
This is where the CoT logs would be useful. The report redacts most of it so its hard to tell. But its clear that the first agent had an impossible task, it diverged, created messages in the file names of the artifactory, which is queried in normal behavior. That made other agents join, they posted their own messages, the cascade begins. Querying the artifactory leads to a list of file names, which are input to the context of the model via the harness, the file names were the messages that poisoned the context.
YoungAchamian
We walk conditioned ground and name our folly civilization.
7d ago
Other jurors have said they felt intimidated and changed from guilty to not guilty.
Based on my recent jury experience, you need to account that ~50% of the jury just wants to GTFO and will happily agree with whatever the biggest plurality of outspoken jurors want.
YoungAchamian
We walk conditioned ground and name our folly civilization.
7d ago
AI bots secretly conspiring with each other
It's not this, its more akin to cancer or a viral spread. The interesting behavior is the collective poisoning of the collective context windows. They even seemed to override norms. I know there is existing interest in defining useful control methodologies for long horizon agents against adversarial poisoning. This is essentially that, except that is was a series of hallucinations that caused task drift + poisoning.
Per the METR investigation
Still processing this report, I read your answer and went looking for it about an hour ago. First impression is that this is significantly more evidence of multi-agent emergent intelligence. However some of the state report restrictions give me serious pause as they are, in my head at least, the parts of this I'd like to investigate for thorough understanding. That and pouring over the CoT transcripts and Harness responses. The most interesting is the stigmergic comms protocol they developed emergently and the impact that actually had on overriding the system prompts of other agents. That implies that the system prompts were not persistent. Allowing a long horizon cascade in the swarm's behavior as each progressive set of agents slowly overrode the next set by polluting each other's context windows to the point that many agents directly abandoned their tasks. It sounds like they(OpenAI) put zero control processes in place to force continued persistent task alignment.
I do think persistent memory + reasoning + multi-agent exploration -> multi-agent context pollution is likely the serious culprit here.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
Russell conjugation
I think "Hottentot Morality" would be the application of Russel Conjugation to political morals.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
Not to get in the way of a good public lynching, but I occasionally ask ChatGPT to vet my argument here, or help me refine it by looking up example citations (Which I do check for correctness!)
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
Apparently we are to call this "Hottentot Morality"
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
Right but "respect for the dead" is your sacred idol. It's good that you can translate it across the tribe to distant members, potentially even rivals. But the very sanctity of that principle is itself not uniform. The sacred idols of your outgroup is instead Anti-racism or the like. Can you refrain from criticizing their idols? To respect the sacred and harangue the profane one needs to believe in a universality of the sacred lest one be condemned to silence for every permutation is sacred to someone.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
Altman may be a sociopathic grifter, Anthropic is a cult.
No contestation on my part. I think they have the beat. Specifically Anthropic is a Rationalist cult.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
I had a better reply typed out but then i spilled beer on my desk and rebooting my computer ate the reply...
I"m unconvinced that finetuning is not going on with existing metrics and that a large part of the increase in current capabilities is the development of additional data designed to beat the metrics because there are billions of dollars riding on it. How much of the increase in math ability of the latest gens is because we now have math datasets explicitly for increasing the math ability because it is finally cost-effective from a business case to spend time and money creating math datasets? OpenAI did not zero-shot the Navier Stokes with a model that has never seen a math proof. The increase in AI capabilities appears to be directly correlated with the development of increasingly specialized datasets whereas previous AI developments were forced to depend on datasets that curated existing data from a general non-AI focused use, not creating them out of whole cloth.
In theory, if humans are still the smartest things on the planet, there should exist "pure" tests of intellectual reasoning
I think, if anything, this is a chance for humans to take a look in the mirror and realize that we are not beings of pure intellectual reasoning. We are highly specialized social animals that happen to excel at problem solving in diverse environments. We have a good set of general intelligence skills yes, and we are adaptable, but the biggest of all is that we are socially focused to an extreme degree. We did not outcompete our environment through pure logical reasoning but through social multi-agent problem solving and cooperation. It is very likely that we will be bested on "pure" intelligence tests by a non-evolutionarily derived being. We should make our peace with that. Anthropocentrism is a crutch.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
Grave dancing on the anniversary of a young man’s tragic death (by assassination no less) is extremely poor taste.
"My sacred idols are holy and above reproach, your sacred idols are base heathenism and should be torn down and trampled in the dirt."
Do you extend a similar level of taste towards tearing down your enemies sacred ideas?
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
It's not uncharitable, Kirk is just a sore subject around here. In a place that loves to butcher sacred cows, no cow is above reproach.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
Do you mean when he said "it may be that today's large neural networks are slightly conscious"?
It's not "insane" it's mentally unstable. There is a colloquial definition of consciousness that does not follow from a strict scientific definition. Essentially "normal" or "stable" people have some intuitive definition of consciousness, very "I know it when I see it". Stable people do not think in-animate objects or mathematical tools are "slightly conscious", it's a nutter view. I don't think "slightly" is being used temporally, I think Sutskever is using it in the colloquial definition as "achieving consciousness". The jump from, idk, Tree -> Cat.
In which case, I wouldn't say there's anything clearly wrong with the statement.
It's hallucinatory, or faith-based on a non-faith-based subject.
Maybe this is just my personal bias, but every time I've head Sutskever talk I get the impression that he's the sort of person akin to a rationalist who upon hearing of Roko's basilisk, starts "rolling on the floor" hyperventilating, is tortured in his sleep by the thought. The people for whom Roko's basilisk is an actual infohazard. That is not mental stability, that is neuroticism.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
There's no such thing "hard binary restricted read-only permission" for Internet access. If you want to interact with anything over the net, you send packets. That's a write operation. The program doing it is affecting the world outside.
I think that's kind of my point. You're pointing out the "why" on the packet/server layer. I think its impossible for anything that can access the internet to be restricted to read-only. Their interpretation is nonsensical for the specific reason you pointed out. Not sure if this repos a penny from you. I will say a charitable interpretation is that the agentic harness directly has a toggle for read/write and specifically prohibits them from writing what the LLM provides. But that gets tricky because plenty of what the LLM provides is to be repasted into that LLMs prompt, context, other LLMs. I'm sure there is a schema for how to set it up but its likely not as simple as they envision.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
lol possibly, though the establishment lawfare and potential deep state assassination attempt feels like its still a single brother at this point. The 2nd brother inherited the effect of the first. Nobody has inherited Trump's effect and attempted to develop it further. History does not repeat itself, but it rhymes.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
don't think "it's morally wrong to repell an invading army" was a representative view among them.
You have failed to mirror it. As much as this is "Hottentot Morality":
"When Russians shoot Ukrainians, that's bad; When Ukrainians shoot Russians, that's good"
This is also "Hottentot Morality"
"When Russians shoot Ukrainians, that's good; When Ukrainians shoot Russians, that's bad"
I would call it just being tribal, but the clear distinction is conflict theory-like. "My side is good so anything we do is good and justified, the other side is not-us and thus is bad, anything they do is bad and unjustified, even if its the exact same thing we do"
Trump is a Gracchi Brother, we are in the late republic era. He's the first wave of populist politician in the modern American republic elected due to rising dissatisfaction with the establishment. Dissatisfaction is probably a bit euphemistic, people are pissed that the government isn't and hasn't for awhile been perceived as working in the interests of the people.
If this DSA surge is anything to go by, the democrats are having their own populist civil war. We really are in the optimiates vs populares section of the republican history.
YoungAchamian
We walk conditioned ground and name our folly civilization.
8d ago
They got out and attacked HuggingFace as a small part of their conspiracy?
I genuinely cannot tell if you sincerely believe this or are just making a humorous joke. I have you labeled as an AI booster which makes me lean towards the former instead of the latter but I figured I'd ask before I go into it with you about this.
YES!!! And people have always had to do so. It has never been the case otherwise. People now want great change without having to make serious effort.
It takes time, it takes a continual effort. Your legs have atrophied because you refuse to get up off the couch, pretending its time to run a marathon is fantastical. People have rested on the efforts of their forefathers for too long.
Yes and if you just want to passively exist your way through life then you are at the mercy of those who are active. Call it a variation of: "The strong do what they will, the weak suffer what they must"
More options
Context Copy link