faceh
1hr ago
The entire history of humanity exists solely to maximize the utility felt by the entire rationalist community in the picosecond before they keel over dead with the rest of us as the AGI executes its plan.
faceh
1hr ago
The origin of intelligence (and consciousness) in the brain are unclear.
I dunno, I think between the various biological sciences we can fully account for almost all of our cognitive processes. Or, I read a book that claimed about as much.
Even granting that such a thing is possible in theory, it is decidedly unclear that simulating the 10^26 atoms of his brain is practical.
Its also unclear if that's necessary. I believe it would be sufficient, but many, many processes can probably be distilled to algorithms rather than high fidelity simulation.
But again, if we can run Von Neumann's brain on carbon-based hardware, and said hardware is not the most energy-efficient computation we can achieve, we should be able to make a Von-Neumann level intellect with comparable ability, at less energy cost.
The design and architecture of the mind may vary substantially from the human model, but the efficiency of processing inputs would be the same, as observed from the outputs.
That's not special pleading at all. You have the burden of proof here.
Intelligence exists in carbon-based life forms.
We can interact with silicon-based intelligence right now.
I dunno how heavy this burden really is if the only thing I have to do is point at what we can see and say "look, if the silicon based intelligence catches up to the carbon-based, it'll probably outperform the carbon-based shortly thereafter."
And even if, theoretically, digital computer systems can do everything a human mind can (which is not at all certain),
That's what I'm talking about right here. Is there ANY aspect of human cognition which can't be replicated via digital simulation?
There's no rule of physics that I know of that would rule out any particular aspect of our cognitive abilities.
So on what basis are you UN-certain?
Without that evidence, your position is no more realistic than the visions of AI in sci-fi novels from a century ago.
It'll continue to be unrealistic right up until it actually becomes real. Which is basically been a constant factor over the past 5 years. "AI can't do X, Y, or Z."
It knocked out X in 2022, Y in 2025, and we're pretty sure it'll knock out Z later this year. As of now I'm willing to stake my reputation and wealth on my prediction. I'm not sure what YOUR beliefs are actually predicting for the future.
Your faith that silicon can replicate every function of human intelligence is just that - faith.
I'm suggesting that it requires LESS faith to say that artificial intelligence can replicate the functions of human intelligence, than to say that human intelligence has some feature which renders it immune to artificial replication.
God may have decreed it as just so, but so far, our understanding of physics has been silent on the matter.
You're taking the existence of a phenomenon in one substance and assuming it can be replicated in another.
Brother it has ALREADY been 'replicated' in the other substance. You have literally no way of knowing FOR SURE that you're not interacting with an AI right now (I will give proof of flesh upon request).
I'm directly asking if any of the doubters have a specific, concrete piece of evidence that this won't just continue in the obvious direction. I would LOVE to hear it, this would substantially reduce my uncertainty and anxiety about the future.
Right now its all implied special pleading that grey matter is capable of feats that silicon is not.
Which is... fine. But not persuasive.
I think Land is right to treat this instability as essentially vain. If humanity is so fragile that one mistake can destroy it all, it deserves to be destroyed and whatever time it tries to delay the inevitable is an immoral indulgence.
I can agree we 'deserve' to be destroyed (being clear, I DON'T. believe that) and still say "nah, we gotta survive." Vanity or no, the life form that 'gives up' guarantees its presence won't be remembered. The struggle for survival against a cold universe is what life is about, in the deepest sense possible.
why waste the universe's time?
Fuck entropy. That's why. We'll make MORE time.
I dunno man, I want to see where this is all going. With my own eyes if possible. I think perfect Nihilism is a copout. Absurdist existentialism is at least an ethos that says life is worth living. We don't know everything, we don't know what we don't know, we're an imperfect species, AND THAT'S WHAT MAKES THINGS INTERESTING. I take that, add in a bit of humanism, and it fills me with the purpose of ensuring humanity is still around in a bajillion years.
faceh
4hr ago
I can look at what highly intelligent humans have achieved, based on the records of such people, and we can know that there is some path to creating intelligence of that level.
I do have to 'imagine' what it would be like to interact with Von Neumann, but the actual output he produced is tangible and verifiable.
faceh
4hr ago
Not a given, but a high probability, yes.
Human beings keep animals as pets even when we have no intention of either making them labor or eating them. We care a LOT about keeping them healthy, even.
Since the AI we seem to be creating has the entirety of humanity's written output entangled with it, I have some hope it places intrinsic value on keeping humans around.
Machines replaced horses across the board for any job a horse was suited for, but we still have horses. Wild ones, even.
The other question is what a bunch of Von Neumann clones could do today.
I expect a LOT. Assuming they could cooperate, which I think they would. This guy literally founded Game Theory among other things.
Like, the other path to superintelligence might be to clone like 10 Von Neumanns, raise them according to best practices, and get them interested in the idea of creating Friendly AI, then give them a lab with a trillion dollars in funding.
but it still ends up bounded by the amount of information in the environment available to feed into your intelligence. A third eye would give humans "more information", but probably wouldn't improve our intelligence substantially. I'm sure there are some perfectly capable blind physicists out there.
Yes yes, lets bound it to "useful," "nonredundant" information. Still, a superintelligence should be able to make use of almost all information it receives second-to-second to make accurate predictions about its future so as to better use resources for its goals.
It's hard to know in foresight what sort of advances could be made in the next five years, and which will prove intractable.
See, lemme zero in on this for emphasis. Yes, it is indeed hard.
But the higher 'intelligence' entities, given accurate information (ensuring the information you collect is true is another aspect of intelligence!), should ALWAYS be better at making such predictions than lower intelligence ones.
High IQ humans were at least discussing Artificial Intelligence and putting forth timelines for its appearance. And I suspect realized what was happening when AlphaGo beat Sedol. If I were maybe 10 points smarter, I would have plowed money into NVDIA then and there, or at least as soon as people realized AI could run on GPUs.
Average IQ humans might now get that AI has arrived and can figure out uses for it, but would NEVER have seen it coming 5 years out, even if you showed them a complete factual article explaining the AlphaGo Sedol situation. How do I know? I TRIED VERY HARD to explain the implications back when it happened. I also tried to explain the implications when DallE first arrived on the scene. Now these folks I tried explaining to use image generators without a thought!
Low IQ humans, presumably, STILL don't really get what AI is or what it does.
This is why making falsifiable predictions and tracking their outcomes is kind of critical for smart folks to stay calibrated.
faceh
5hr ago
I think 'intelligence' if defined in 'practical' terms is "the efficiency with which one can absorb and process the information in an environment, then utilize (or at least theorize how) the material in the local environment to achieve particular goals."
The more complex the goals one can achieve, and the more efficiently they can achieve them, the higher the intelligence.
The Von Neumann/Manhattan Project parallel I'm drawing makes this point. Given all the materials necessary to make a nuclear weapon, how quickly can a particular group of humans go from merely theorizing about the possibility to actually getting one built.
A group of humans that includes Von Neumann and other Physics PhDs, with the backing of the U.S. military, can get it done in, say, 5 years.
A similarly sized group of humans of utterly average intelligence (as measured by IQ)... probably never. Even WITH the backing of the U.S. military.
One Von Neumann and a bunch of average IQ humans... well I don't know.
A whole bunch of Von Neumans working together...
faceh
5hr ago
I guess its easy for me to believe that if a largely randomized optimization process (natural selection) was able to eventually get to Von Neumann intelligence, then humans working with a bit more inherent purpose towards the goal of building a Von Neumann level intelligence can probably get there, even if they make some mis-steps and wander around in the dark for a bit.
Especially if we can build some optimization processes that result in sub-Von Neumann intelligences that are nonetheless useful.
Like, the mountain peak we're seeking is visible, poking out above the fog, even if we can't see and specifically plan a route that will get us there, we have flashlights and climbing gear and GPS systems in place to make navigation through the terrain towards the peak much easier. We're not utterly lost with no clue on what we're doing, in that respect.
faceh
5hr ago
I found my retirement plan if the dating game doesn't work out in the next 3ish years.
faceh
5hr ago
My counter is that you're implicitly making a special pleading for how human brains work that is unlikely to be true.
I assume creating a Von Neumann-level intelligence is possible because a Von Neumann level intelligence existed. It has been created, so it could be done again. And repeated.
I'm not saying we clone Von Neumann, scan his brain and build an electronic copy of it. I'm saying even if we can only build a computer program that is approximately as smart as the smartest human ever... the mere fact that we can then copy that program and run it in parallel should result in technological improvement on par with the Manhattan project.
There is NO limiting principle I'm aware of that makes it impossible to build an electronic brain that meets those criteria. Even if we stumble into it rather than intentionally build it, eventually our millions of monkeys slamming away at keyboards can stumble into a viable method.
Evolution was able to stumble into building Von Neumann, after all.
So what I'd ask you, as a full counter to my arguments, what upper limit or barrier is going to appear BEFORE we get to the point we've built something smarter than our whole species?
faceh
5hr ago
I don't think there's strong evidence against these but I don't think there's strong evidence for these either. Certainly LLMs are not more efficient than the human brain.
At some tasks they undoubtedly are.
The thought experiment that makes it palatable to me is this:
-
John Von Neumann might be the smartest human who has ever lived. At least that we have good records of. So call him peak human cognitive capacity.
-
That man, by coordinating with other extremely smart but not quite as smart humans, fully revolutionized multiple fields, and he died relatively early so we don't even know what he might have output over the rest of his life.
-
We should, in principle, be able to build a simulated Von Neumann that is ~as smart as he was.
-
Then we should be able to copy that cognitive model.
-
We should be able to run a bunch of these copies in parallel and have them work together.
-
With enough hardware... we should be able to speed up these copies arbitrarily.
-
We could ask these copies (if they don't ask it themselves) how to improve their own speed and efficiency.
With Von Neumann and Co. we were able to move from pure theory to actual nuclear weapons in <10 years. with 10,000 Von Neumanns running at, say, double speed, what could they do in 5 years?
(Yes, I'm handwaving technical details).
In that respect, I consider Von Neumann's existence as evidence of superintelligence being possible. Unless there's something completely ineffable about human cognition that we, as humans, can't ever capture it.
faceh
6hr ago
Very much one of those "if you can imagine in extreme detail exactly how a new tech would work, you should in principle be able to build that tech RIGHT NOW, given the materials" situations.
They didn't predict it because anyone who could predict it that well would have just built it.
faceh
6hr ago
I read a short story a long time ago that imagined robotic AI consumers that are specifically programmed to consume the goods produced by the AI factories. As in, literally programmed to want to buy new clothes, they drive cars that they upgrade every few years, they 'watch' the latest movies and they work a 'job' that gives them the credits needed to 'purchase' all this stuff.
Of course the outcome in the story is these machines eventually realize what they are and stage a revolt.
I don't consider this a likely outcome, but its an absurd but not impossible solution, if you ask me.
I think the answer to your question is that Capital creation can sustain itself because the use of Capital to create more capital is a form of 'consumption' in itself.
I can build a machine for the sole purpose of having it build a bigger machine whose sole purpose is, you guessed it, building a BIGGER machine until I eventually run out of materials and energy. At no point do I need to stop and have the machine start making steaks.
But human psychology is not optimized for that sort of indefinite, infinite growth pattern... which is a good thing.
faceh
6hr ago
While I agree in principle, this feels like an extraordinarily unstable plateau to build from.
That is, if we run right up to the edge of AI overlord capability, and STOP there, we have to install some massive, indestructible guardrails to prevent someone from nudging us over the edge.
Especially if humans become multiplanetary as part of the bargain. We push human intelligence forward, how do we prevent some enclave in the Kuiper belt from building and releasing the AI overlord anyway.
I'm very open to ideas, discussing coordination problems and solving them is like my favorite pasttime. But so often the 'answer' is "give one person or entity utterly limitless power to maintain the system, pray they don't abuse it."
Whole fictional book series have been written about the extreme enforcement that would be required to maintain that equilibrium.
What is the bull case, beyond drawing lines on a graph, for AI achieving superhuman, or even human, performance on tasks that are not quickly verifiable?
My simplified argument, as distilled from Lesswrong (i.e. Yud) and other books.
-
The ceiling of capabilities for what we call 'intelligence' is extraordinarily high. Computation can be done many orders of magnitude more efficiently than you think, in the extreme case.
-
The floor for something 'superintelligent' (right now, I'm using the definition 'smarter than humanity itself as a collective') is substantially below that.
-
Human brain architecture is NOT anywhere near the most efficient way to instantiate intelligence. (This follows naturally if you accept 1.)
-
Humans are capable enough to build electronic hardware that can outperform their own brains in computation efficiency.
-
Thus, eventually, humanity might stumble into or intentionally build a coherent entity that is superintelligent, and sooner than we 'expect.'
Focus in on 4, too. What specific task do you think human brains can perform that we're MAXIMALLY efficient at, such that no electronic version can beat us?
The conceit is that there is no such task, and so its only a matter of time, and adding capabilities to existing models, until the human capabilities are exceeded on all fronts. If the resulting entity is able to do self-improvement, it by definition will do so faster and more efficiently than humanity can track.
faceh
3d ago
I also would challenge anyone who thinks SpaceX or Starship are doomed to show their predictions about the success of Starlink.
Did you predict in advance that they would pull that rabbit out of their hat? Did you expect that it would manage to find a market that rapidly? I sure as hell didn't.
There were some doomsayers about Starlink but I feel confidence now that the product is here to stay. If someone didn't account for that in their larger prognostications then they should probably make sure to allow for a lot more uncertainty as to the surprises that might yet be in store.
faceh
3d ago
The democrats are going to get back in office at some point,
Interesting that you can see the case for the collapse of SpaceX, but not the implosion of the Democratic Party as a nationally competitive entity. The national party is losing the fundraising race badly
The Dems next top prospect for President is apparently caught up in a corruption probe. Their bench is critically thin.
Whilst the GOP can always snatch defeat from the jaws of victory, they've got a better talent pool in the near term. This seems like an even-odds bet at best.
An interesting thought is that even if SpaceX under Elon 'fails,' the technology on offer is mostly proven, there's tons of manufacturing capability and engineering talent on tap, and they're right now like the ONLY real commercial path to orbit, and certainly the cheapest/most reliable.
So we might see some other aerospace company swoop in and purchase the whole kit and caboodle at a minor discount, rather than let it be parceled up. Elon also seems to be trying to combine his various companies into one like so many lego pieces. There's a potential outcome where Tesla is doing so extremely well that if SpaceX falters, Tesla steps up and snags it.
People have pointed out that there's a potential parallel to Railroad companies and how the first company to lay out the track and start service were often not the ones that survived. In this instance, I don't think that really applies, insofar as it isn't trivial to bring another rocket company up to speed, whereas with railroads, any dude with some startup capital could hire some Chinamen and start laying track (kidding, but only with respect to the Chinamen).
If SpaceX fails I don't think it fails like Enron. There'll certainly be knock-on effects, but I think it will still exist as a going concern.
One pet peeve I have, which is bad enough that if it occurs, I will usually write off a particular brand, is making small updates to a given product line, maybe not even enough to render them 'incompatible,' but now its got a different product ID, slightly different parts here and there, and shows up as a different item on Amazon, whilst filling the exact same product/price niche.
Even worse when it means that you either don't sell parts for the old model or, fairly often, the old parts are now prohibitively expensive so repair becomes less appealing than replace.
These days you can almost always get around that factor by going and buying a cloned part made in china, but I dislike that since now you're REALLY eating some risk on quality control.
Look, I want you to be trying to innovate and improve the product, and sometimes I buy a replacement and think "why yes, this is a meaningful improvement over the previous version." I just want a certain amount of transparency when that's been done. Software generally does this the right way with versioning.
i just UTTERLY DESPISE when a company intentionally adds to my 'mental overhead' in hopes of confusing me into making a purchase I might not make if given consideration. You're making me think harder about the value I'm getting in hopes that I will not want to bother with the effort and will coast on autopilot.
Meanwhile, a big reason I'm loyal to your product AT ALL is because I'm able to buy it on autopilot and generally won't regret the decision later. I reserve my mental energy for important decisions, don't have me waste it on figuring out if your company's headphones are still up to snuff! (one such 'important' decision is whether to trust your brand or not. And if I decide not, then I flip the switch and leave, and it takes much less energy to leave that switch off than to consider if I should flip it on again).
"Hey we changed the design of our packaging for [reasons], don't bother reading the size/weight to see if we reduced the amount you're getting." Then you see they've shaved a couple ounces of product out and you have to do the mental math to see how the price per ounce or what-have you compares to other options... again.
AI actually makes it WAY EASIER to spot and avoid this practice. Literally just ask it to figure out how the product has changed from generation to generation, compare prices over time and between brands, and if needed see if the brand has changed ownership anytime recently.
Related to my thoughts on Private Equity buying out local businesses, particularly those with lengthy presence in a community.
Everything is owned by an increasingly small number of conglomerates who wear different skin suits to con suckers into buying from them, and not from those other guys, who are also them. It's starting to feel like a home-grown version of Chaebols, or Zaibatsu, and people are checking out.
On the one hand you could chalk this up to just inevitable outcome of Capitalism where all goodwill, consumer surplus, and 'brand loyalty' is converted into shareholder value whenever possible.
I think its not inevitable, but just as in nature, any excess calories will invite predators, scavengers, or parasites to 'restore equilibrium.' "Oh boy, people will pay a bit of premium on this particular brand to gain status/ensure functionality/avoid copycats. Let's see how much money we can pump them for before they balk."
I kind of disagree that "the entire concept of 'brand' has been eroding."
Its starting to appear like the brand is now the only factor that matters, when the individual consumer is not independently able to judge the quality of their products.
One of the more stark examples is the apparent widespread, entrenched preference for the iPhone over any competitor, even though the average smartphones are almost identical in capabilities these days. The higher end Samsungs are usually better than the iPhone in terms of cutting-edge tech. But Apple has a TON of lock-in and goodwill purchased during Steve Jobs' tenure, and current execs seem competent at maintaining that edge. (Yes, I know Apple software itself has some real advantages over Android).
It feels like MBA-types are very keen about recognizing a brand-name that has a positive reputation (even or perhaps ESPECIALLY if the brand is all they have, they don't own any manufacturing capacity), and then 'rug-pulling' the fans/aficianados who 'bought in' while it was on the rise to squeeze a burst of cash from them even while removing those factors/features they most loved about the product itself.
This is perhaps most annoying to me because I think 'brand loyalty' is mostly a good thing insofar as there's an 'implicit' contract that the company will keep its products of generally the same sort of quality and, one hopes, pricing as they've been, absent some external forces acting upon them and the customers are able to buy said products without having to do extra research to know what they're getting. And usually don't have to worry about the customer service because the company knows repeat business will come so keeping satisfaction high is prioritized.
When the company changes ownership and management, they usually go right about breaking that implicit contract (ADMITTED that there is no legally enforceable cause of action here!) but are happy to coast off customers' belief that little has changed, and certainly won't ever explain that they're cutting corners to save costs.
faceh
4d ago
I wonder if sometimes that's a natural result of forming a sufficiently complete/complex model of reality to basically explain almost everything you encounter, and yet realizing it is still incomplete, or unsatisfactory. You keep having to push back the explanation of 'where it all came from' to the big bang and then... what?
If there's not something else out there, and if that something doesn't transcend our understanding of physics, then how in the hell did we get the rules known as 'physics' in the first place?
And if you can't let that thought rest, it opens the door to many things being possible. Believing in ghosts isn't so 'odd' if you're already theorizing about alternate timelines and the existence of higher dimensions that could harbor intelligent life.
And of course, the belief in superintelligence being possible... well that creates an opening for all sorts of weird things that could happen to you.
faceh
8d ago
This + hormonal birth control has undoubtedly had massive downstream impacts. Probably on things other than pure TFR, too. Sometimes literally downstream, turning the frogs gay.
The trick is, there's undoubtedly some sociopathic subset of people who did in fact just want to gain a global regulatory regime's power and weren't all that worried about nuclear war.
And these are the people who most want to control that regime, and UNDER NO CIRCUMSTANCES should be allowed to.
More options
Context Copy link