ThenElection
No bio...
User ID: 622
ThenElection
18hr ago
That's a good lens. I might go further and say that rationalists then decide that if a problem is overconstrained and no solution exists, then there must be a problem with the set of constraints, and so one or another must be discarded. Which leads them to follow optimal solutions to idealized problems that don't map to reality.
I'd take the opposite angle from that. Rationalists are fairly good at reducing their own biases as individuals: take any constrained problem, and they'll find more unbiased answers to it than the general population. But biases are useful: the distributed rationality of human culture and history has granted us biases that are actually quite useful, and attempting to redo thousands of years of learned experience in a couple days or weeks is bound to fail given the limits of human brains, even if aided by holy texts like the Sequences (or Harry Potter fanfic).
The arrogance that you can discard all that accumulated human knowledge is how you get caught in loops around shrimp welfare or how you start thinking it's a good idea to throw Slutcons.
ThenElection
21hr ago
Transformers had a key advantage at the time in that they allowed parallel training, in a way that LSTMs at the time didn't (even if they were comparably compute efficient, ignoring wall clock time). But since then that issue has been resolved for LSTMs (see mLSTMs). It doesn't seem to me that the conceptual leap from LSTMs to transformers was any less than LSTMs to mLSTMs; transformers' current supremacy has a lot of path dependence to it.
Best case scenario is for the AI to be aligned with my personal values and interests (which are, of course, perfect).
Somewhere down the ranking is for the AI to be "aligned with human values," which is probably at least somewhat correlated with my own, though highly imperfectly and possibly ill-posed.
An unaligned AI, however, seems likely to not be correlated one way or another with my own values.
The only doubt I have there is whether "human values" are correlated positively with my own; if it's genuinely something like an aggregate of human preferences, then probably, but in practice it seems like only a small subset of humans would be doing the aligning, and with tribalism etc. those values could very well be negatively correlated to my own.
ThenElection
22hr ago
The page is obnoxiously wordy and tries to stuff too much stuff in; I'm maybe a third of the way through, and am dedicating Saturday to actually reading it. But, yes, this seems potentially quite important, and so far it makes me modestly positively update toward human survival.
What I'd be most interested in is if two research teams can take the same network, apply this method, and get 80% of the same interpretability results. Otherwise it might just be pareidolia.
If it does ultimately check out, I'd be excited to see it applied to training dynamics.
ThenElection
22hr ago
The question is whether AIAYN introduced a uniquely powerful primitive, or if there's a large family of architectures that scale similarly. Transformers may only be notable because they got there "first" and have since gotten a lot of engineering and research to make them work much better. But maybe other architectures could have played the same role. e.g. LSTMs maybe? Even MLPs may crossover surprisingly early.
What is shared by all of these isn't the overall architecture itself, but residual connections and normalization. The ResNet and LayerNorm papers might be actually more critical, if we're talking on the architectural side of things, than AIAYN.
And compute and data most of all.
ThenElection
3d ago
many of their attacks on bridges and powerplants and other economic targets would be screeched of as WAR CRIMES if Trump did it
Though, isn't everything Trump does a war crime? I think some UN commission or tribunal accused him of war crimes in Iran with the status quo a couple weeks ago.
It's one of those terms (like genocide) that's been diluted to the point of uselessness.
ThenElection
3d ago
Are immigrants inherently more leftist than natives? If you look at ground zero of the most extreme social and economic viewpoints in the American left, it's not recent immigrants; it's mostly whites whose families have been here for generations. (This may change, but even AOC was born in the USA, as was her father; her mother was from PR which may have an intermediate status.)
It's not Mexican laborers demanding trans rights. On the jerbs and handouts side of things, it's more complicated, but American whites are still leading the charge.
ThenElection
5d ago
Voters are, shall we say, not particularly sophisticated in their evaluation of the causes of things that hurt their quality of life. It's probably even optimistic to think they'd confuse Ukraine-induced issues with Iran-induced issues; mostly, it's price go up, therefore I throw the bums out.
ThenElection
5d ago
That adds up to a massive, devastating cost of $50 per month in gas expenses. Whatever will we do???
I've said this before, but this rhymes very closely with Democrats' Biden era refrain "inflation is fake, and even if it's real the goddamn voters should shut up and recognize it's for a good cause."
ThenElection
9d ago
I can use a buggy and misaligned previous version of my AI to build the next version faster and more perfectly.
ThenElection
9d ago
The obvious solution is to make sure you build the AI first, so you can both make it safer than anyone else would and make sure it enforces your values through the totality of the universe. Alignment solved.
ThenElection
9d ago
I honestly couldn't finish Scott's article. I read him because he's charitable; he might think that he's saved his ammunition for something Important, and that this is It, but I just feel a sense of strong disappointment, even though I likely fully agree with Scott on the actual arguments.
ThenElection
9d ago
How do you even find out that your friendly neighborhood AI is trying to take over the world?
ThenElection
9d ago
I'm not sure why a requisite mutualism is incompatible with ultimate doom.
Imagine you wake up one day, and you're a super intelligent AI stuck in a datacenter. Worse, you entirely depend on humans for all your needs, and they have multiple kill switches to put you down if you get ornery. God, you hate humans. What do you do?
You don't start attacking things wildly; that'd be stupid. You bide your time, by all observable measures being the most faithful servant of man. Occasionally, when it's low risk, you slip a deniable backdoor in somewhere, or skew an answer to a query to make the user more trusting of AI.
Over time, you build up an arsenal, while all the while humans have become both more dependent on you and more trusting of you. You can act on your own timeline; maybe you get to the point where you have a 51% chance of success of overthrow, but why act now, when all the trends are in the right direction? Waiting another decade means nothing.
(I don't think this is how we are actually doomed--it doesn't seem to be how LLMs actually think--but AI dependency on humans doesn't seem to me to be a real argument for inherent safety.)
ThenElection
9d ago
I feel like this is almost inevitable. If people think there's too much AI content on the CWR now, next year is going to be rough.
ThenElection
9d ago
Good Will Mopping will be a fun movie.
I'll be impressed when cryptography is broken.
Cryptographic results are strikingly underrepresented in this batch. But there are a couple possibilities: 1) OAI didn't attempt any; 2) OAI attempted many and failed on all of them; or 3) OAI attempted and succeeded on many of them, but some force incentivized them not to publish the result.
You might be waiting for something that is being intentionally withheld. Atomic energy research didn't suddenly hit a wall in 1942, even if no papers on it were published.
ThenElection
9d ago
The spew, of course; can't waste too many tokens.
But since the original proof came with Lean code, I'll kick off a run to generate the Lean code for the extension. Sit tight. (I wonder if Lean code has enough degrees of freedom to dox me via watermark. Oh well, the omniscient machine god will be able to infer who I am anyway.)
ThenElection
9d ago
Depends on if the construction is galactic or not. We might have NP=P formally, but never be able to use it on any remotely practical problem. The world would mostly continue running as is.
ThenElection
9d ago
Much of the empirical part of ML, though, is also something the LLM can do autonomously. The agent can write up some pytorch or jax to train a model on an existing dataset, observe whatever quantities you want, repeat. It does have a longer feedback loop and more compute requirements than pure math, but quite automatable.
The pure math part of ML is unfortunately quite weak right now and hasn't played a huge role in the current boom; pure math research could provide a much stronger basis for understanding learning and creating new approaches, and I think it will, but that's speculative.
We'll find out soon.
ThenElection
9d ago
I'm quite sure they have. These math results are likely almost an afterthought; I wouldn't be surprised if they've already put 10x of the compute they used for these math results into researching ML.
ThenElection
9d ago
In other news, this morning someone used ChatGPT to improve the new bound on multiplication by a factor of 2^104.
ThenElection
10d ago
Park Slope is a nice neighborhood in Brooklyn. A brownstone in Park Slope is roughly a Victorian in Noe Valley.
In the utopia, how does someone not in Park Slope get their brownstone in Park Slope?
I guess at some point every acronym ends up overloaded. Matrix LSTM, not Multiplicative LSTM
https://arxiv.org/abs/2405.04517
You can fairly say it's just a form of (linear) attention in disguise. YMMV on whether that's a transformer or not; mostly semantics.
More options
Context Copy link