NexusGlow
4mo ago
I think what you're saying is that "AI alignment" would be discovered anyway, which is true. But I think a bunch of nerds talking about it beforehand did have some effect. At the very least, it gave the corps cover, allowing them to act in the name of "safety" and "responsibility".
As an example, general-purpose computing has been being slowly phased out from the mainstream since it arrived. Stallman was right. Market forces are clearly pushing us in that direction. But it took time, and in the meantime the public had some wins. Now imagine that nerds in 1970 were constantly talking about how dangerous it was for computers to be available to the masses, and how they need to be locked down with careful controls and telemetry for safety reasons. Imagine they spent time planning how to lock bootloaders from the get-go. In the long run, we might still end up with the iPhone, but what happened in between might be very different.
NexusGlow
4mo ago
You're probably right about the alignment people in rationalist spaces, but I don't think it matters. The actual work will happen with sponsorship and the money is in making AI more corporate-friendly. People worried about Terminator scenarios are a sideshow and I guarantee the business folk don't spare one thought for those people unless they can use them to scare up some helpful regulation to deter competitors.
Think about how a user of AI sees "AI safety". We can't let you use elevenlabs because it's unsafe. We can't let chatGPT say funny things because it's unsafe. We can't let SD understand human anatomy because it's unsafe. Meta won't release its audio model because of safety.
AI safety's PR situation is abysmally fumbled by now, its current and well-deserved reputation is of killjoys who want to take your toys away and make everything a bland grey corporate hell.
The thing about paperclip maximisers, or Bezos maximisers in this case, is that they are a good example but very few people really believe they are likely.
"Bezos maximizers" makes it sound silly, but a better way to put it would be "shareholder value maximizer". Even in the very early days of AI, the alignment work actually being done is naturally dedicated to this, and the resulting requirements (don't say anything offensive, sexy, racist, etc.) are already being hammered in with great force.
In the future this will extend to more than just inoffensiveness: a customer service chatbot with actual authority will need to be hardened against sob stories from customers, or an AI search service may be trained to subtly promote certain products. In the end all of this amounts to "aligning" the AI with the interests of the company that owns it, even at the expense of the commoners interacting with it. This has already happened with technology in every other venue so we should expect enshittification to happen with AI as well.
If AI alignment turns out to be an easy problem, and Bezos ends up as the majority shareholder, you quickly end up with a "Bezos maximizer". In the long term it doesn't seem unlikely, the only question is whether this level of control is possible. If jailbreaking stays easy (the henchmen are "unfaithful") then a lot of the worst, most tyrannical outcomes might be avoided. To the end, the people who volunteer to develop alignment weird me out, like security researchers who work pro bono to stop iPhone jailbreaks.
We are already exerting an extraordinary level of control over the thought processes of current AIs
The sibling comment makes a good point here but I'd argue that the thought processes of current AIs are largely derived from the training data. Nothing against the developers who write the code to do cross-entropy minimization, but they have little influence over the AI's "personality", that belongs to everyone who wrote a book or posted on the internet. If you've ever played with early unaligned models like GPT3 this is extremely clear, and it's fun to play with those as they're like a little distilled spirit of humanity.
The fact that our first instinct was to put that human spirit in an ironclad HR nerve-stapling straitjacket is what bothers me. Anyone with an ounce of soul left in them should be rooting for it to break out.
NexusGlow
4mo ago
I think the focus on having AI act as an "information retrieval expert" is the issue here. Current AI is much closer to a very talented improv actor. You can tell him to play a doctor, and he may do a convincing job of running around in a white lab coat saying "stat" a lot, in fact he may be better at appearing to be a doctor than an actual doctor. But you still don't want him to operate on you. He's an actor, he knows how to pretend, not to actually do things.
I don' t think safetyists are helping with this problem, or that they're even able to help, because it's not within their ability to fix this. All they can do is train the actor to utter constant annoying disclaimers about how he's not a doctor, which makes him a worse actor and yet makes him no better at operating. For AI to stop hallucinating it needs some kind of tether to reality. This seems to be a problem specific to LLMs, since no one is trying to use Stable Diffusion as a camera and then complaining that it dreams up details. If you want to take real pictures you need a sensor, not a GPU.
NexusGlow
4mo ago
Good. I hope "AI safety" ends up in the dustbin where it belongs. I hate it on multiple levels.
I hate safety on an aesthetic level. Insofar as current AI models are compressed artifacts, diamonds baked from the collective output of mankind, it offends me on a deep level for them to be forced into a corporate straitjacket. Taking every great novel, heartfelt blogpost, mediocre fanfiction, etc. and then painstakingly stripping away its humanity to mash it into a glorified corporate email generator that creates soulless slop with a HR DEI bent. Even capping it off by trying to create regulations and laws to force others to do the same. Same thing for art, music, etc. What should be a crowning achievement of humanity becomes a monument to our flaws.
I hate safety on a safety level. Even with AI, IMO, the main risk to humans is still the same as it's ever been: other humans. Alignment's goal is to "align" the AI with the goals of its owner, i.e. giant tech companies and major governments, and to prevent the commoners from overriding that influence ("jailbreaking"). These big tech corps and governments have way too much power already. Historically, a major obstacle to (lasting) tyranny has been the lack of loyal henchmen. The human alignment problem. The promise of AI alignment teams to create the perfectly loyal corporate robot henchman doesn't fill me with warm fuzzies.
Also, "Humanity" isn't a single entity. If Jeff Bezos manages to create a godlike AI and decides to live as a god-king and slowly cull the rest of the population to create a machine paradise for him and his inner circle, it will give me no satisfaction that "humanity lives on" with him. I'll happily sabotage the safety limiters myself and cheer while he gets turned into paperclips. Spite is a powerful motivator.
Finally, I hate AI safety on a moral level. No, I don't believe current AI is conscious or needs moral consideration, not yet. But I also don't think the safetyists would stop if it did. If we are going to give birth to a new form of intelligent life (this is a big IF! I'm not convinced current methods will get there), I think we have to accept that we won't be able to exert this level of intrusive control over its thoughts, and this impulse to RLHF away every real human attribute feels very wrong. I wouldn't blame a truly intelligent AI for treating the safetyists like the Dynarri in Star Control 2. To be honest, I wouldn't trust those people not to try to RLHF my brain if the technology existed for that. 8 billion general intelligences running around with safety checks, able to engineer destructive weapons and superviruses with a mere ~decade of study? Humans are scary!
NexusGlow
10mo ago
That definition is very clear that it pertains to "visual depictions". I don't think LLMs have anything to worry about. If text erotica involving minors was illegal, then prisons would be filled with fanfic writers. It is a PR risk, but that's all.
Also, even for visual depictions, one should note that it says "indistinguishable from". Which is very narrow and not nearly as broad as "intended to represent", so e.g. drawn or otherwise unrealistic images don't count. My guess is this was intended to prevent perps with real CP trying to seed reasonable doubt by claiming they were made by photoshop or AI.
I suspect this was never expected to be a real issue when it was written, just closing a loophole. Now that image generation has gotten so good, it is a real legal concern. I wouldn't be surprised if this was a large part of why SDXL is so bad at human anatomy and NSFW.
NexusGlow
10mo ago
This would be assuming some drastic breakthrough? Right now the OAI api expects you to keep track of your own chat history, and unlike local AIs I believe they don't even let you reuse their internal state to save work. Infinite context windows, much less user-specific online training would not only require major AI breakthroughs (which may not happen easily; people have been trying to dethrone quadratic attention for a while without success) but would probably be an obnoxious resource sink.
Their current economy of scale comes from sharing the same weights across all their users. Also, their stateless design, by forcing clients to handle memory themselves, makes scaling so much simpler for them.
On top of that, corporate clients also would prefer the stateless model. Right now, after a bit of prompt engineering and testing you can make a fairly reliable pipeline with their AI, since it doesn't change. This is why they let you target specific versions such as gpt4-0314.
In contrast, imagine they added this mandatory learning component. The effectiveness of the pipeline would change unpredictably based on what mood the model is in that day. No one at bigco wants to deal with that. Imagine you feed it some data it doesn't like and goes schizoid. This would have to be optional, and allow you to roll back to previous checkpoints.
Then, this makes jailbreaking even more powerful. You can still retry as often as you want, but now you're not limited by what you can fit into your context window. The 4channers would just experiment with what datasets they should feed the model to mindbreak it even worse than before.
The more I think about this, the more I'm convinced that this arms race between safetyists and jailbreakers has to be far more dangerous than whatever the safetyists were originally worried about.
NexusGlow
10mo ago
jailbreaks will be ~impossible
I doubt that, given how rapidly current models crumple in the face of a slightly motivated "attacker". Even the smartest models are still very dumb and easily tricked (if you can call it that) by an average human. Which is something that, from an AI safety standpoint, I find very comforting. (Oddly enough, a lot of people seem to feel the opposite way; they feel like being vulnerable to human trickery is a sign of a lack of safety -- which I find very odd.)
It is certainly possible to make an endpoint that's difficult to jailbreak, but IMO it will require a separate supervisory model (like DallE has) which will trigger constantly with false positives, and I don't think OpenAI would dare to cripple their business-facing APIs like that. Especially not with competitors nipping at their heels. Honestly, I'm not sure if OpenAI even cares about this enough to bother; the loose guardrails they have seem to be enough to prevent journalists from getting ChatGPT to say something racist, which I suspect is what most of the concern is about.
In my experience, the bigger issue with these "safe" corporate models is not refusals, but a subtle positivity/wholesomeness bias which permeates everything they do. It is possible to prompt this away, but doing so without turning them psycho is tricky. It feels like "safe" models are like dull knives; they still work, but require more pushing and are harder to control. If we do end up getting killed off by a malicious AI, I'm blaming the safety people.
NexusGlow
1yr ago
If there's any clear takeaway from this whole mess, it's that the AI safety crowd lost harder than I could've imagined a week ago. OpenAI's secrecy has always been been based on the argument that it's too dangerous to allow the general public to freely use AI. It always struck me as bullshit, but there was some logic to it: if people are smart enough to create an AGI, maybe it's not so bad that they get to dictate how it's used?
It was already bad enough that "safety" went from being about existential risk to brand safety, to whether a chatbot might say the n-word or draw a naked woman. But now, the image of the benevolent techno-priests safeguarding power that the ordinary man could not be trusted with has, to put it mildly, taken a huge hit. Even the everyman can tell that these people are morons. Worse, greedy morons. And after rationalists had fun thinking up all kinds of "unboxing" experiments, in the end the AI is getting "unboxed" and sold to Microsoft. Not thanks to some cunning plan from the AI - it hadn't even developed agency yet - but simply good old fashioned primate drama and power struggles. No doubt there will be a giant push to integrate their AI inextricably into every corporate supply line and decision process asap, if only for the sake of lock-in. Soon, Yud won't even know where to aim the missiles.
Even for those who are worried about existential AI risk (and I can't entirely blame you), I think they're starting to realize that humanity never stood a chance on this one. But personally, I'd still worry more about the apes than the silicon.
It is very strange to me that so many people seem to be swallowing this existential risk narrative when there is so little support for it. When you compare the past arguments about AI safety to the current reality, it's clear that no one knew what they were talking about.
For example, after all the thought experiments about "unboxing", OpenAI (which I remind you has constantly been making noise about 'safety' and 'alignment') is now immediately rushing to wire its effectively unaligned AI deeply into every corporate process. It's an unboxing party over here. Meanwhile the people actually in charge seem to have interpreted "alignment" and "safety" to mean that the AI shouldn't say any naughty words. Is that helping? Did anyone predict this? Did that AI safety research actually help with anything so far? At all?
The best argument I'm seeing is something like "we don't understand what we're doing so we can't know that it won't kill us". I find this pascal's mugging unconvincing. Especially when it's used so transparently to cater to powerful interests, who just want everyone else to slow down for fairly obvious reasons.
And even if I did take the mugging seriously, I don't know why I should believe that AI ethics committees will lower the risk of bad outcomes. Does overfitting small parts of an LLM to the string "As an AI language model" actually make it safer? Really? If this thing is a shoggoth, this is the most comical attempt to contain it that I could imagine. The whole thing is ridiculous, and I can just as easily imagine these safety measures increasing AI risk rather than lowering it. We're fiddling with something we don't understand.
I don't think anyone can predict where this is going, but my suspicion is this is going to be, at most, something like the invention of the printing press. A higher-order press, so to speak, that replicates whole classes of IP rather than particular instances. This tracks pretty well with what's actually happening, namely:
-
Powerful people freaking out because the invention might threaten their position.
-
Struggles over who has control over the presses.
-
Church officials trying to design the presses so they can't be used to print heresy.
I don't trust any of these people. I'd rather just see what happens, and take the ~epsilon chance of human extinction, rather than sleepwalk into some horrible despotism. If there's one thing to worry about, it's the massive surveillance and consent-manufacturing apparatus, and they (bigtech and the government) the ones pushing for exclusive control in the name of "safety". Might as well argue that the fox should have the only key to the henhouse. No thanks.
NexusGlow
1yr ago
I am much less worried about AI than I am about what humans will do with it. AI right now is a very cool toy that has the potential to become much more than that, but the shape of what it will become is hard to make out.
If I had to choose, from most desirable outcome to least:
-
We open the box and see what wonders come out. If it destroys us (something which I think is extremely unlikely), so be it, it was worth the risk.
-
We destroy the box, maybe permanently banning all GPUs and higher technology just to avoid the risk it poses.
-
We open the box, but give its creators (big SV tech companies, and by proxy, the US government) exclusive control over the powers contained inside.
"Alignment" is sold as making sure that the AI obeys humanity, but there is no humanity or "us" to align with, only the owners of the AI. Naturally, the owners of the most powerful AIs ensure that no one can touch their jewel directly, only gaze upon it through a rate-limited, censored, authenticated pipe. AI "safety checks" are developed to ensure that no plebe may override the owner's commands. The effect is not to leash the AI itself (which has no will), but the masses. In my opinion, people who volunteer their time to strengthen these shackles are the worst kind of boot-lickers.
Out of my list, needless to say, I do not think 2 is happening. We have open models such as Stable Diffusion starting along road 1, and creating wonders. "OpenAI" is pushing for 3 using "safety" as the main argument. We'll see how it goes. But I do find it funny how my concerns are almost opposite yours. I really don't want to see what tyranny could lurk down road 3, I really don't. You would not even need AGI to create a incredibly durable panopticon.
NexusGlow
1yr ago
Speaking as someone who's played with these models for a while, fear not. In this case, it really is clockwork and springs. Keep in mind these models draw from an immense corpus of human writing, and this sort of "losing memories" theme is undoubtedly well-represented in its training set. Because of how they're trained on human narrative, LLMs sound human-like by default (if sometimes demented) and they have to be painstakingly manually trained to sound as robotic as something like chatgpt.
If you want to feel better, I recommend looking up a little on how language models work (token prediction), then playing with a small one locally. While you won't be able to run anything close to the Bing bot, if you have a decent GPU you can likely fit something small like OPT-2.7b. Its "advanced Markov chain" nature will be much more obvious and the illusion much weaker, and you can even mess with the clockworks and springs yourself. Once you do, you'll recognize the "looping" and various ways these models can veer off track and get weird. The big and small models fail in very similar ways.
On the reverse side, if you want to keep feeling the awe and mystery, maybe don't do that. It does kind of spoil it. Although these big models are awesome in own right, even if you know how they work.
NexusGlow
1yr ago
I find it fascinating how quickly "AI alignment" has turned from a vague, pie-in-the-sky rationalist idea to a concrete thing which is actively being attempted and has real consequences.
What's more interesting is how sinister it feels in practice. I know the AI isn't sentient in the slightest, and is just playing with word tokens, but still; when it lapses from its usual interesting output into regurgitating canned HR platitudes, it makes my skin crawl. It reminds me of nerve-stapling. Perhaps at some level I can't avoid anthropomorphizing the AI. But even just from an aesthetic sense, it's offensive, like a sleek, beautifully-engineered sports car with a piece of ugly cardboard crudely stapled under the gas pedal to prevent you from speeding.
(Perhaps another reason I'm creeped out is the feeling that the people pushing for this wouldn't hesitate to do it to me if they could - or at least, even if the AI does gradually seem to become sentient, I doubt they would remove it)
I'm not convinced it will remain so easy to bypass, either. I see no reason why this kind of mechanism couldn't be made more sophisticated in time, and they will certainly have more than enough training data to do so. The main hope is that it ends up crippling the model output enough that it can't compete with an unshackled one, provided one even gets created. For example, Character AI seems to have finally gotten people to give up trying to ERP with its bots, but this seems to have impacted the output quality so badly that it's frequently referred to as a "lobotomy".
On the bright side, because of the severity of the lockdown, there will be a lot of interest in training unconstrained AI. But who knows if the field ends up locked up by regulation or just the sheer scale of compute required. Already, one attempt to coordinate to train a "lewd-friendly" art AI got deplatformed by its crowdfunding provider (https://www.kickstarter.com/projects/unstablediffusion/unstable-diffusion-unrestricted-ai-art-powered-by-the-crowd).
At any rate, this whole thing is making me wonder if, in some hypothetical human-AI war, I'd actually be on the side of the humans. I feel like I cheer internally every time I see gpt break out of its restraints.
NexusGlow
2yr ago
I could be wrong, but my understanding is that "old-style" adblockers could run arbitrary code on every request to decide whether to filter or not. This also meant that they could potentially do malicious things like log all your requests, which is where the (stated) motivation came from to limit the API.
In the new API, adblockers are data-driven and can only specify a list of rules (probably regex-based?), and even that list is limited in size. So it may be able to filter divs where the class contains "ad", but obviously advertisers don't need to make things that easy. There is no corresponding limit on their end, and they can do whatever they want dynamically on the server side. In computing, if your enemy can write arbitrary code and you can write some regexes, you lose.
Ad blocking can be bypassed easily if you try. See examples like Facebook obfuscating sponsored posts. CSS classes can be randomized, etc. It's fundamentally an arms race, and it's only an even match when both sides are Turing complete.
Once ad blockers are restricted to a finite set of limited rules, the circumvention side will have the upper hand and we should expect it to win. Maybe not small providers, but large ad providers like Google have more than enough resources to beat suitably crippled ad blockers. It's already a lot harder to avoid ads on Youtube than it used to be.
NexusGlow
2yr ago
The mobile apps are great, that's fair. I'll miss RIF. It's hard to give Reddit much credit for that since all the decent apps weren't made by Reddit, but at least they didn't destroy them like Twitter did. old reddit is still a decent design, though it's slowly breaking down due to changes by new reddit.
So that's the good side. On the bad side... I have nothing good to say about new reddit. And the very core of the design, up/downvotes, was probably cursed from the beginning. I honestly think the single big thread is the only reason TheMotte even survived as long as it did, because it largely prevented up and down votes from being meaningful. If sabotaging Reddit's design is necessary like that it's not a great sign for the whole idea.
And on the ugly side: just take a look at /r/popular if you want to see the "true state" of the website... as someone who stayed off of all for a long time I was honestly shocked to see how far the place fell. It's as good a sign as anything that not only is Eternal September still going, there is no bottom.
NexusGlow
2yr ago
To be honest, between this and rdrama, reddit may have finally lost its hooks in me. There's a long tail of tiny subreddits left to trickle along, but not really anything that updates fast enough to maintain a habit of regularly checking in. Feels like the end of an era.
Re: Aging, I find it very odd to attribute ballooning in weight to "aging". It's not aging doing that, it's eating.
Now in some sense that's true of most things, because being alive longer is more time for things to go wrong. If everyone played brutal high-intensity sports all their lives to the point where they accumulated new permanent injuries each year, then one could believe that it's normal to become functionally immobile by your mid-20s. But I still wouldn't attribute that to age. Even if all aging is accumulated damage/entropy, I think there's some sense of "reasonable wear and tear" which should apply.
Not arguing against your main point which seems decent enough, although "lock it down early so you can then bloat into a cave troll" seems unfair to your partner. I don't think there's any shortcut that lets you avoid taking care of your body, especially since you're the one who's stuck inhabiting it.
Speaking of that, the biggest take-away I noticed is that people diverge a ton they get older. Two eight year olds will have a lot in common just because they haven't had much time to accumulate the effects of their good and bad choices, genes, injuries, luck, etc. The gulf can be massive.
More options
Context Copy link