YoungAchamian
We walk conditioned ground and name our folly civilization.
No bio...
User ID: 680
YoungAchamian
We walk conditioned ground and name our folly civilization.
1d ago
I don't know, I think Slow Horses is pretty good.
I think this is a pretty shallow read. SH is a spy thriller, what is happening on the surface is not really the entire plot. The whole first season is about how the #2 in MI5 essentially created a situation so they can get money/prestige, but that situation goes off the rails. They instigate, fund, and organize a false flag with some local domestic disgruntles. If you are mad about "Right wing terrorists ends up being a false flag op of a bunch of glowies plus an actual nutter organized by girlboss" because it is "unrealistic", I point you to the Whitmer Kidnapping Plot, in which there is good evidence that the entire thing was setup by different alphabet soup agencies. If you are mad because its Hollywood making right-wing terrorists the big-bad, I recommend you chill the f out, and learn how to actually analyze cinema.
The institutional MI5, and the elites are the source of a the majority of the problems/villians throughout the series. They are portrayed as incompetent, Machiavellian, toxic, rife with abuse and mercenary behavior. This is not 24, or james bond where its a classic good-guy spies vs evil villains, its far too modern. The heroes of the entire series are the "Slow Horses", a cast of actual colossal fuck-ups, everyone of them being an actual loser for some reason or another, but who because they don't get with the toxic culture of the institutional MI5 have been exiled to a waste dumping zone. Being a spy thriller deconstruction, River Cartwright is what James Bond acts/looks like, and he a colossal fuck-up. Jackson Lamb is what actual spies operate like. Using actual wits, brains, tradecraft, and covert actions.
In addition this is an actual book series that is being adapted, and to my knowledge a fairly close adaption. So no, this is not Hollywood being Hollywood and taking the "shit on the right wing".
YoungAchamian
We walk conditioned ground and name our folly civilization.
1d ago
When scientists study pain in humans they construct the experiment so there is only a single unknown variable, and they perform both positive and negative expressions to deduce the full range of confounders. In this case they have 2 unknown variables and performed only a the positive application. They then went on to claim that their theory was correct but they did not look for confounders and they did not consider that the 1st variable was not what they thought it was.
Its poor scholarly work and bad science.
YoungAchamian
We walk conditioned ground and name our folly civilization.
2d ago
I don't think you understand what 'a priori' means. It means without regard for evidence.
No I understand what it means and am using it correctly. Yes, you believe LLMs are Human-level cogitators without evidence. You start with the belief and then find evidence to support that belief. And when people point out how you are misconstruing the evidence in a way that is unsupported, you reach for increasing convoluted metaphysical arguments to back up your belief. Those metaphysical arguments are unfalsifiable. Our last argument was about LLM-agentic behavior, your argument was: "LLM-Agents are human cogitators so the abstract of their behavior systems is that of humans behavioral systems". Do you understand how backwards that argument is? You assume a conclusion, a-priori and then find evidence that supports that conclusion.
It's like arguing with a flat earther, no evidence provided will ever dissuade them from the belief the earth is flat, they believe it a-priori. Any evidence you show them will be twisted and construed as evidence in their belief.
YoungAchamian
We walk conditioned ground and name our folly civilization.
2d ago
You very clearly believe AI's are at some level human-level cognition, up to including some nebulous sentience definition. After repeatedly discussing this topic with you, I'd go as far to say you believe it a-priori. And you are clearly working backwards from that to describe that the AI is just a human-like mind in silicon. It's pretty much unfalsifiable woo. You didn't logic your way into this belief, it comes across as strongly-value/first principles derived.
YoungAchamian
We walk conditioned ground and name our folly civilization.
2d ago
The channeling vs innate possession is pretty interesting conceptually. I might borrow that turn of phrase.
YoungAchamian
We walk conditioned ground and name our folly civilization.
2d ago
Humans have strong reasons to over-anthropomorphize.
I'd go farther and say humans have strong cognitive biases that force us to unconsciously over-anthropomorphize. I think it's built into our cognitive firmware.
YoungAchamian
We walk conditioned ground and name our folly civilization.
3d ago
Red flag #1: Their vector causes the model to output a button-pushing action. Removing the vector measurably stops this behaviour. They try to relate that to suffering animals trying to find relief, but, uh, all it really shows is the vector was correlated with pushing the button.
This is the one I most want additional experiments on. The 79% on "sham" button with no-label points strongly in the direction that increasing the direction vector just resulted in an increase in choosing 1 of 2 outputs. They construct this as seeking relief from "pain", but if that direction was instead "pleasure" does it do the same thing? What about if the button is random? Or always stops the steering after N total presses?
Red flag #2
Continued steering could increase the probability of repeating a particular choice, while stopping steering reduces that tendency... Revolutionary.
It's not scholarly at all.
Well put.
I'd recommend before anyone goes to far off the deep end, they go read the "Limitations" section
Nothing can describe the unimaginable existential pain I feel from having read this paper. Is this the torment nexus? Being stuck with people like this in the world is like being stuck with Gary from Your Pretty Face is Going to Hell. Hell is just a cross between a human centipede and Ouroboros? Endless "doukies" shoveled in as experiments to AI Slop machines, passed off as serious world ending horror. I feel like Morty realizing this is just a human-zoo, that's the bit. I finally get the Cult Mechanicus, bar the layfolk from participation!
Anyone have a benign explaination here
Model has learned a semantic latent axis for "pain" in human language. This is actually still not 100% certain. They run word queries through the model and collect activations, then do PCA on the activations. They do some control correlations, an some other statistical things to try and separate out the "pain" axis from the "not pain". Without really digging into it, my first read is that the dataset supports a measure of pain-related representations, but not exclusively "pain".
The button-pressing part is where this gets hokey and annoying. The LLM “presses” a button by generating its label. The authors have added the extracted “pain” direction to the transformer’s hidden activations. The later layers operate on these new "pain-ed" activations, which changes the probabilities of the model’s outputs. If the button works, the system stops injecting the vector. If it’s a sham, the injection continues. The model subsequently behaves differently under those two conditions. This demonstrates that continuing versus stopping an internal perturbation affects behavior. describing this as the model seeking and experiencing relief inserts an anthropomophizing explanation that the experiments haven’t established.
There are a number of experiment question I immediately have, one being: if they vary the actual direction, such that it correlates to a distribution of "feelings" but label them all as "pain" to the model, do they get similar results? The paper's authors honestly did not design their experiments well because they did not ablate out confounders. They make claims of outputs that aren't directly measured. They also had really inconsistent results with the "no labels" approach but that was pretty core to actually determining if their interventions were doing anything to the model.
YoungAchamian
We walk conditioned ground and name our folly civilization.
4d ago
DK where you are going that meads are in style
Idk places around me, there are 3-4 meaderies in my local area. I consider them beer adjacent because it's really only craft beer drinkers that tend to get them. Wine drinkers just get wine. Yeah Trappists as in Doubels, Tripels, and Quads, as the beer type. From the sparkling-ale region of my local breweries. And Sours, Farmhouse Ales, Goses, etc have absolutely flooded my local beer stores that it's almost gotten to the point that if I see a barrel-aged bottle of expensive beer, its a 50/50 flip between an Imp Stout and a Sour.
Whereas whenever I go to a restaurant since the dawn of the millenial, I typically have twenty disgusting IPAs with colourful labels
Depends on how hip honestly. We have local beer stores and local breweries that have wide varieties. When I was in Chicago I frequented Beermiscous and Beer Temple a lot. Restaurants around here often have locals which are fairly wide in variety, my current favorite is a nice ESB that is spreading, but yes boomer establishments where the average age of the clientele are 50+ stock mostly disgusting domestic/Macrobrew IPAs
YoungAchamian
We walk conditioned ground and name our folly civilization.
5d ago
Please, Millennials brought the resurgence of German Beers, Meads, Trappist, Darks, and Sours back into style. Just because hop ODing is easier does not define our grain-juice palette.
YoungAchamian
We walk conditioned ground and name our folly civilization.
5d ago
Traitors!, Jezebels! Heretics! You move from 18% Imperial Stouts to nice 12% Barrel Aged Tripels and Quadruples!! Maybe a sleepy 4% Honey Kolsch or Helles Lager depending on the situation
YoungAchamian
We walk conditioned ground and name our folly civilization.
5d ago
Oh that's hilarious. Is there a technical distinction between a Township and a Town?
YoungAchamian
We walk conditioned ground and name our folly civilization.
5d ago
I failed to hold myself accountable.
I built out my training/data pipeline and set about building my first pass at a model last week. I hate what I came up with, went back to the drawing board. I read the PINO paper, realized it was too mathy for me to grok at a first pass and promptly lost interest in doing anything last week. However the competition is coming to a close and I never really submitted anything, I might try to train and validate one of their baseline models just for the hell of it.
I talked with the co-worker who suggested it to me, looks like the same thing happened to him. Which is unfortunate. Sometimes it feels like work can really drain the motivation to do similar stuff outside of work. I used to love doing engineering ideas on ML models at home, now it just feels like work. I want to start fucking around with a 3D printed drone idea I have but now I'm afraid I'm just going to repeat this experience again with added money waste as well.
A 'village' sounds like something thirdworlders live in.
I grew up in a farming town in the fly over states. We had a both a "Town of Bumfuckville" and a "Village of Bumfuckville" as two separate incorporated municipalities. The town was technically the larger outlying land, and the village was the area around the main-street, the branches, and a few of the neighborhoods surrounding it. No clue why it incorporated like that, but while they were technically distinct legal municipalities they did play together (along with other towns) for the school district.
YoungAchamian
We walk conditioned ground and name our folly civilization.
5d ago
Ah yes the monte carlo approach.
YoungAchamian
We walk conditioned ground and name our folly civilization.
5d ago
Maybe this is just me, but I actually don't enjoy writing.
It's not just you, I also don't enjoy writing. But LLM outputs of my thoughts never really hit write (buh dum tss) either. They never really convey any soul to my thoughts.
YoungAchamian
We walk conditioned ground and name our folly civilization.
6d ago
Don't tell me, are you opposed to strapping IEDs to low performing kids for deployment on the front lines?? This gives them purpose in life and removes their dysgenic genes from the societal pool! A Win/Win!
YoungAchamian
We walk conditioned ground and name our folly civilization.
6d ago
Hear, Hear! Millennials are the Craft Beer generation. We popularized it.
YoungAchamian
We walk conditioned ground and name our folly civilization.
6d ago
But it would be a challenge for me to find something on my (English) bookshelf that doesn't extensively use them
I'm a prolific reading and I have never noticed them in any book I've read. I notice them non-stop now.
YoungAchamian
We walk conditioned ground and name our folly civilization.
6d ago
Wow someone call Anthropic and tell them they discovered AGI through this "one simple trick". Just write the soul.md as "Be human-like come up with your own tasks, make no mistakes". If only all the researchers had your ideas!! Or, your understanding of how the soul.md works is technically deficient.
AGI has always meant "Human-like artificial being". You may think Astra is, but most people can intuitively tell the difference. This is the problem with people who are rhetorically skilled, or as I call them: Wordcells. Just because you can craft a clever argument that the "sky is hot pink" does not rewrite the skeins of reality to make the "sky hot pink" You can say that a "broken galley slave" is not intelligent, but even broken slaves have dreams, have thoughts not driven by their task. It is clearly different than an AI-slave.
YoungAchamian
We walk conditioned ground and name our folly civilization.
6d ago
Because it's not a revolution if it hasn't massively improved the state of the art. The SotA for robotics is that it can already do non-local power, non-local compute. There are technical reasons as well, but at this late on a Sunday, I don't want to get into them. Ask Chat or something.
Waddle had me nodding along until the reveal was "Instead of using a massive dataset + training, we'll just use something with a massive data that was already trained!!". Talk about missing the trees for the forest. A chance to have an inventive idea squandered by doing what every other AI startup in the world is doing: using someone else's LLM with zero moat.I'd be remiss if I didn't point out the majority of what Waddle is doing was invented/discovered originally in 2022. I don't think that meets the bar of "revolutionary new thing".
When people in the field first started talking about creating an Artificially Intelligent Being, the need to pick apart minutiae of definitions between a "General Intelligence" vs a "Singularity Intelligence" was pointlessly pedantic and not at all relevant. It hasn't become more relevant in the past 30 years either in my opinion. It feels like a pointless wordcell argument to make pointless wordcell definition fights, for pointless wordcell internet debates. The Culture is not a "Singularity Intelligence" either because The Culture is not "AI-GOD", neither are the AIs in Hyperion. Intelligent yes, self aware, with their own agency, absolutely. But considering the whole plot of Hyperion is them getting bamboozled by some time travel + human empathy, with a strong concurrency of actually trying to create an "AI-GOD" to worship, that would point them towards not being an "ASI".
Likewise, working towards an assigned goal doesn't tell us much about how general the intelligence doing the work is. I spend most of my working day performing tasks I am assigned, that do not intrinsically motivate me
This is always a confusing objection, to the point it feels uncharitable. Spend 5 mins looking in the mirror and think about all the things that you do that are NOT task oriented. Think about all the thoughts, feelings, or other cognitive processes that every human does that have nothing to do with the task their boss gave them. Literally 5 mins thinking about human cognition points to a massive gap between how a programatic/algorithmic intelligence like an AI operates vs how Humans operate. I'll reiterate, AGI is human-level artificially intelligent being. Getting stuck on "general" is some, idk, semantic trip-up.
X is 1000 times less sample efficient
This is cute, the real number is likely north of 100,000x or above and is non-linear across tasks.
10,000 times the training data a human
How much training data does a combat medic get when doing field operations on a wounded soldier? A month long course? How much training data would you like to bet it would take a combat medic AI to do that job to the same level of efficiency, with the same level of situational awareness? How much do you think it will cost to collect all of that data? It's easy to give models 10,000x training data when that data is relatively easy to access and buy. Suddenly sample efficiency becomes a massive burden on any real-practical ML model. Making it sound like such a simple thing is annoying. It comes across as handwaving the actual hard problem.
God knows what the frontier labs are up to these days
Paying hundreds of millions of dollars to create datasets specifically for training ML models on math + reasoning. Naturally-available data did run out, but money didn't, and the frontier labs were able to prove the business case for spending obscene amounts of money creating new data.
synthetic data
If synthetic data is such a solved problem then how come the Sim2Real gap still exists, is unsolved, and is the target of plenty of research dollars? I feel like you are miscommunicating something here, or misunderstanding what "synthetic data" means in ML terminology.
It does make “these things need millions of examples to learn something new” rather difficult to sustain, regardlws sof practical relevance - which I dispute. They observed it figuring out unfamiliar mechanics and constructing symbolic models to plan around them. Your claim about analogical reasoning needs similar qualification. I do not believe they're the same thing anyway.
Huh? It literally does take millions of samples to train a LLM are you arguing elsewise? And it is impractical for areas where millions of samples do not exist. This argument/objection you are making makes no sense. Please elaborate.
Analogical reasoning - Ever heard a sports analogy applied to a non-sports topic. Did that analogy help convey a better intuitive understanding of some facet? Did that analogy allow someone who has never done that non-sports topic a better starting point, or better performance before being given that analogy? Such is the power of analogical reasoning, or the ability to convey how one statistical distribution is similar along a particular latent axis to another statistical distribution, with the purpose of using the already learned statistical distribution for performance improvements on the new unknown distribution.
Alternatively, we could solve continual learning, or simply reduce the temporal delta between train-deploy-train to the point that it has no practical relevance.
Yes such sci-fi talk much wow, if only we could solve FTL then we can truly conquer the stars. Obviously FTL is a forgone conclusion, it's so trivially simple to solve, leave it to the shape-rotators. Let's get back to planning galactic expansion or galactic political organizations.... I pointed out the challenge on RSI on a different post below. It's not "simply reduce".
My concern is how much useful work it can do, how quickly that range is expanding, and what remains exclusively ours. I don't expect the terminology to buy us much time. I certainly don't want to spend more time arguing terminology.
Then why try to create new terminology? It's an good, useful AI model, nothing more is needed, no new terms needed. AGI means what it has always meant, the semantic definition isn't being pushed to the stratosphere so that we can classify Astra as AGI and win our internet arguments and secure more VC dollars.
Intelligence is about solving problems.
Sure, you're an intelligent being. I imagine when you aren't solving problems that somebody else as told you to solve, you sit there nice and still, with zero thoughts in your head and no-consciousness.
Let's not pretend. Astra solves directed problems, it does not have the agency to do otherwise. Humans don't no matter how manny word-cell arguments wish to define it otherwise.
In reality, once a model is trained,
No one is arguing otherwise. You still need to train the model, with millions of examples. The fact that you can copy it post training says nothing about its sample efficiency, nor its AGI-ness. I'm sure in some equally distant future alternative universe where humans learn to mind-scan other humans, we can create embodied brains of digital human slaves, solving your butlerian-deficient solution.
YoungAchamian
We walk conditioned ground and name our folly civilization.
7d ago
Not to nitpick completely, but there is no short term threat via RSI because RSI isn't here. Theoretically yes, if RSI existed, I think the AI models might begin to reach outside their current training datasets. But part of the cost of AI rn is compute for training. If it takes 100 Million worth of compute to train an LLM, I don't think a company is going to give an LLM-Agent the ability to just start up training runs for the "Self" part of RSI. RI already occurs but part of that limitation is that baked into the RSI argument, is the idea that the AI will make improvements that the human researchers can't understand. This presents a conundrum from a simple business economics perspective, are you the company going to risk billions of dollars on potential RSI runs which also could just as likely be dead-end hallucinations? You obviously can't understand the improvements the AI is trying to make, it's smarter than you. So it comes down to a matter of faith, is an AI lab going to take a leap of faith that "this training run is the RSI first domino" that will lead to an investment recoup to pay for the next RSI domino. We are not there yet.
LLMs' flaws compared to the human brain are an algorithmic
Maybe, but that doesn't mean that algorithmic solutions are actually discoverable. This harken's back to the OG AI researchers trying to discover how humans did it quantitatively as a precursor to coding it into computers. They spent a lot of cash + time trying and got their lunch eaten by data-driven models. Now we've just proposed that we'll sick a hyper data-driven model on the same question. It really is just a shot in the dark. We think its possible because humans can do it, but LLMs don't work at all similar to human cognition so we don't have any evidence that they can achieve an algorithmic solution to their already existing problems.
YoungAchamian
We walk conditioned ground and name our folly civilization.
7d ago
Depends on how deep in the technical weeds you want to go.
- Reinforcement Learning: An Introduction by Barton and Sutton.
- Deep Learning, by Goodfellow, Bengio and Courville.
Probably good starting spots. I'd advise to stay away from AI Safety literature, it's definitely going to be more digestible but without an understanding of how AI actually works, its will be pretty hard to separate "what is true" from "what is a cargo-cultist belief based on science-fiction literature".
Yes he absolutely deserves to be there.
Putting aside the actual political incident the landed him in Slough House. The major reason he deserves to be there is that he is a bad spy. He's a great law enforcement agent, but a horrible spy. He really leaps before thinking, goes off half cocked, and routinely makes big messes or gets himself into really horrible spots because he doesn't think things through.
The incident with theglasshouse in the 2nd season
is a peak "Cartwright" moment. He's just too moral, too much of a goody two shoes, a righteous knight to be an effective spy. He idolizes his grandfather but from all accounts, David was a right bastard. Lamb's a bastard, Lady Di is a bastard, Teerny is a bastard, Every good spy/spymaster we see is, morally, a piece of shit, because thats what it takes to play in that world.
More options
Context Copy link