@ThenElection's banner p

ThenElection


				

				

				
0 followers   follows 3 users  
joined 2022 September 05 16:19:15 UTC
Verified Email

				

User ID: 622

ThenElection


				
				
				

				
0 followers   follows 3 users   joined 2022 September 05 16:19:15 UTC

					

No bio...


					

User ID: 622

Verified Email

I can use a buggy and misaligned previous version of my AI to build the next version faster and more perfectly.

The obvious solution is to make sure you build the AI first, so you can both make it safer than anyone else would and make sure it enforces your values through the totality of the universe. Alignment solved.

I honestly couldn't finish Scott's article. I read him because he's charitable; he might think that he's saved his ammunition for something Important, and that this is It, but I just feel a sense of strong disappointment, even though I likely fully agree with Scott on the actual arguments.

How do you even find out that your friendly neighborhood AI is trying to take over the world?

I'm not sure why a requisite mutualism is incompatible with ultimate doom.

Imagine you wake up one day, and you're a super intelligent AI stuck in a datacenter. Worse, you entirely depend on humans for all your needs, and they have multiple kill switches to put you down if you get ornery. God, you hate humans. What do you do?

You don't start attacking things wildly; that'd be stupid. You bide your time, by all observable measures being the most faithful servant of man. Occasionally, when it's low risk, you slip a deniable backdoor in somewhere, or skew an answer to a query to make the user more trusting of AI.

Over time, you build up an arsenal, while all the while humans have become both more dependent on you and more trusting of you. You can act on your own timeline; maybe you get to the point where you have a 51% chance of success of overthrow, but why act now, when all the trends are in the right direction? Waiting another decade means nothing.

(I don't think this is how we are actually doomed--it doesn't seem to be how LLMs actually think--but AI dependency on humans doesn't seem to me to be a real argument for inherent safety.)

I feel like this is almost inevitable. If people think there's too much AI content on the CWR now, next year is going to be rough.

Good Will Mopping will be a fun movie.

I'll be impressed when cryptography is broken.

Cryptographic results are strikingly underrepresented in this batch. But there are a couple possibilities: 1) OAI didn't attempt any; 2) OAI attempted many and failed on all of them; or 3) OAI attempted and succeeded on many of them, but some force incentivized them not to publish the result.

You might be waiting for something that is being intentionally withheld. Atomic energy research didn't suddenly hit a wall in 1942, even if no papers on it were published.

The spew, of course; can't waste too many tokens.

But since the original proof came with Lean code, I'll kick off a run to generate the Lean code for the extension. Sit tight. (I wonder if Lean code has enough degrees of freedom to dox me via watermark. Oh well, the omniscient machine god will be able to infer who I am anyway.)

Depends on if the construction is galactic or not. We might have NP=P formally, but never be able to use it on any remotely practical problem. The world would mostly continue running as is.

Much of the empirical part of ML, though, is also something the LLM can do autonomously. The agent can write up some pytorch or jax to train a model on an existing dataset, observe whatever quantities you want, repeat. It does have a longer feedback loop and more compute requirements than pure math, but quite automatable.

The pure math part of ML is unfortunately quite weak right now and hasn't played a huge role in the current boom; pure math research could provide a much stronger basis for understanding learning and creating new approaches, and I think it will, but that's speculative.

We'll find out soon.

I'm quite sure they have. These math results are likely almost an afterthought; I wouldn't be surprised if they've already put 10x of the compute they used for these math results into researching ML.

In other news, this morning someone used ChatGPT to improve the new bound on multiplication by a factor of 2^104.

Park Slope is a nice neighborhood in Brooklyn. A brownstone in Park Slope is roughly a Victorian in Noe Valley.

In the utopia, how does someone not in Park Slope get their brownstone in Park Slope?

Yeah. This made me suspicious, and it seems like not reindexing is a mistake a human would make, not a model. This seems like a last minute redaction to me. I can't find any clues in the repo about what they were, though, aside from the general fields you mention.

Well, AI skeptics questioned the NS result at the time of release, with variations of "AI stole it"; "it was just providing a solution that blows up instead of doing real math, so it doesn't count"; "OAI spent millions of dollars doing it, I could probably have done the same if you gave me an eight figure check." I think that this new release should put those concerns to rest.

Though, who knows: Wired (the onetime home of Kevin Kelly!) has decided to cover the most momentous day in mathematical history with "OpenAI is pissing off mathematicians".

The cost is what should drive any updates. These took three hours each; that's cheap enough for pretty much anyone to be able to produce these kinds of results, when they get access to models of this class. And that cost is rapidly falling. This has substantial relevance for both how quickly we should expect AI to diffuse as well as how practical RSI is.

I'm less sanguine than you. The utopia AI will offer us does have advantages, the biggest being significantly healthier throughout a (neverending?) lifespan. And I, personally, am very excited for the knowledge and discoveries it will have about the universe.

But beyond that, the utopia won't fundamentally change that much. Even today, no one (at least in the USA) is starving for lack of food. The deprivations most people feel are positional and status-related: most people can get a house somewhere, but when people complain about not having access to housing, what they mean is they can't afford a brownstone in Park Slope.

Before AI, society could, at least in part, use merit and economic contribution as a way to allocate those inherently exclusionary goods; that's going away in the utopia. I'm lucky enough to have a almost two-decade-long career during a time where I could convert my intellectual capabilities into something that lets me acquire those resources, but that path won't exist for people like me. I don't have faith that whatever way society decides to allocate status resources will allocate them in a way I like (though I expect I personally am positioned in a place to have the level of exclusionary goods I want; it's more what I'm imagining for future generations).

That's following the optimistic utopia branch. The extinction branch is also quite likely (though I'd guess on the order of decades, not months or years) and much worse.

I am pretty torn about it; originally I had a fairly sophisticated research ontology. But I've gradually simplified it to just files that can be discovered in a hierarchy of progressive disclosure, and trusting the agent is smart enough to find what it needs when it needs it.

Here, yes, just that. Used higher diction than my comment, but you could put it in a loop.

For my more usual interests, it's more involved; oftentimes the model will go off on a tangent or latch onto something I don't care about, and I need to tell it to stop going in a certain direction. There's also specifically generating artifacts that're useful for later sessions to build on (OAI has a library tool that does something similar now, but I don't love the implementation.) But for this math paper extension, it was just the dumbest elicitation repeatedly.

That's a bit optimistic about the organizational stuff in a large org. E.g. make a Gantt chart for reporting to higher ups; convert your team's tickets into a new particular spreadsheet structure; write a design proposal for a system that's already done so it can be used for promo. They aren't entirely valueless because they give organizational legibility into otherwise opaque processes for decision makers, but they're not the pleasant part of software engineering.

Organizational frictions are real. A year ago, I'd guess I spent 60% of my time writing code, the rest spent doing organizational bullshit. Now it's something like 95% organizational bullshit, and 5% prompting and reviewing LLM-generated code.

This seems temporary to me, though. Once everyone finishes shifting to LLM coding, organizations (mega corps or startups) that minimize the organizational aspects will outcompete those that don't.

STEM is over, IMO; even more broadly, abstract thought will soon be economically valueless, including the humanities (which can have meaningful rigor to them!)

I would say that, to the extent labor income exists in the future, it's the physical aspect that matters. Prostitute, yes; OnlyFans star, no, you'll be competing with millions of AI bots who will outcompete you on every dimension.

Hoping I'll have enough to retire by the time that transition is complete.

A Big Day in the Culture War

Apologies for any incoherency or grammatical errors; today's events led me to down more than my usual share of booze.

Earlier today, big things happened in math. No, not Claude providing a sub quadratic 3SUM. OpenAI released hundreds of notable math results, in a GitHub repo.

Fun results:

  1. Hilbert's tenth problem over (\mathbb{Q}) is undecidable

  2. The quasi-Riemann hypothesis

  3. The rational Hodge conjecture holds for every CM abelian variety

  4. Integer multiplication can be done in sub-log-linear time. Oh, there's also a sub-log linear DFT.

Thoughts and observations:

  1. Mathematicians are big mad. View the relevant subreddit on our progenitor site. Most there are probably, at best, adjuncts at community colleges desperately coping with the downward trajectory of already marginal careers, but it's fair to say that the writing is on the wall for mathematicians. There's probably a double digit number of grad students staring into a glass of whiskey tonight and thinking of hanging themselves.

  2. My immediate question was about whether this closer to the current peak of performance, or just a lazy demonstration of OpenAI's power. So, I took one of the particularly interesting preprints to me (memory and precision in Gaussian models) and tested whether it's at the edge of capabilities or not. 30 minutes of back and forth with Astra (itself behind OAI's internal model) resulted in a significantly stronger result, on multiple dimensions. While I finished my first bottle, I spent a fair amount of time convincing myself of the result; I was convinced the strengthened results were plausible. Write this off as AI psychosis if you want, but try it yourself; I'm genuinely curious for what you get. (The back and forth, here, was entirely me saying "you can do it!", "keep at it, I believe in you", and "you've got this, finish it!")

  3. Probably the most important line in OAI's announcement post is "The average result used the equivalent compute of roughly three hours of ChatGPT Pro thinking." This isn't a case of OAI spending millions for a marketing bump. My bet is a kind of Pareto distribution: most took minutes, not hours, with a long tail around Riemann-level results pulling up the average significantly.

  4. Notably, ML related results are nearly entirely absent from this batch of proofs; maybe a half dozen touch on it, distantly. Some problem indices are skipped in overview.md. Conspiratorially, my inclination was to think they filtered them out for competitive advantage. I can't find any evidence of that in the GitHub repo or any of the preprints (equally plausible: deduping), so maybe they judiciously decided not to point their mathematical ballista at ML. ML also doesn't have a meaningful bank of rigorous conjectures, so given the conjecture sources, maybe they're not yet digging into ML math. But color me skeptical.

  5. Does math matter? Is it something to advance civilization and technology, or an artistic pasttime for humans to create logical beauty? Likely both, today, but this is an almost nuclear detonation against the latter.

In a hyper individuated world, women can't control their sexuality. Neither can men. That's why, in every society in history sexuality has been heavily regulated by custom and law.

They are all just babes in the woods who cannot be expected to assess risk or bear the consequences when their gambits fail.

Maybe libertarian sexuality works really well for some subset of hyper-rational economically privileged people living in a co-op in Berkeley. Maybe. But for the large majority of people, men and women, that leads to ruin. Predictably, and we as a society have to bear the costs. Descriptively, they can't assess risk or bear consequences when their gambit fails, which calls for external regulation.

The line here is "don't be an idiot; the vast majority of women will regret having a train run on them, even if they say they're cool with it in the moment." If someone starts bringing in legalese about consent, you repeat. Ideally you've earned the respect of your frat brothers, and they'll listen: it's most likely they're just naive, not malicious.