@YoungAchamian's banner p

YoungAchamian

We walk conditioned ground and name our folly civilization.

1 follower   follows 0 users  
joined 2022 September 05 18:51:23 UTC

				

User ID: 680

YoungAchamian

We walk conditioned ground and name our folly civilization.

1 follower   follows 0 users   joined 2022 September 05 18:51:23 UTC

					

No bio...


					

User ID: 680

GEN 1.5

I saw this one, haven't had the chance to look at Helix yet.

I don't see why this demand is fair.

I'm just calling it how I see it. There's a couple arguments. One from a generalized capabilities standpoint. Robots with non-local power are only useful in factory settings. We also have them. There is no need for "LLM-factory robots" because factory robots don't need to adapt on the fly. Despite EA fears, not much is given by making a factory controlling AI, AGI smart for the purposes of making paper clips. Robots that require an LLM datacenter to control them are easily disrupted, latency is still an issue and it becomes more of an issue when you need to be processing so much information all the time. The last part is that robots for very specialized situations that can't adapt aren't very useful outside of the niche uses they have specialized in, which likely requires training, which is expensive.

Two is from a "what is the current state of the art" The current state of the art for mono-situation robots with centralized power and control is already here. If you want to "revolutionize" robotics you need to do more that what already exists.

The third argument is a funding argument. It should be a shock to nobody that the 3 main funders of Robotics work is defense/military, manufacturing, and VCs interested in consumer robots. 1 is going to need all of the above, 2 doesn't really need LLMs to be added, they have other needs, and 3 is probably the most wishy washy of the bunch, but in general people are wary of centralized robots that feed all of their personal data to a central server.

robots with complex behaviors will almost certainly have some combination of cloud forebrain + local hindbrain

Yes, but thats not really an argument against the above. That's just an argument that a certain class of robots will require a reasoning engine like an LLM and a controls module like what currently exists.

I don't think this holds after Astra crushing human baseline on ARC-AGI-3.

While I'm not an expert on all the various LLM benchmarks, but a brief look makes me think this is exactly the sort of thing that RL game playing is good at, and learning general game-playing strategies in training would generalize pretty well to this. I'm not going to goodhart a new metric into existence, but you conceivably need something that there exist zero training data for, including close enough transfer learning data.

distributing RFID tags to your troops is that they can be used by both sides

Yes, I've heard that point a lot. I know an RF guy who's trying to do some passive emitter thingy that goes wayyy over my head. No clue how it works. But he has some pithy comment for it that I am forgetting, its quite funny.

I actually thought that open-source imagery would be better resourced than that

There are what a couple hundred grainy low res videos of Ukr drones targeting Russian tanks + other vehicles? It's well below the threshold to train an ATR model on data quantity, without even getting into quality. Stock images are from non-operational angles (you for the most part aren't level with targets, its an isometric or overhead view, NERF to solve that is expensive and has dubious performance improvements, and having worked in the generation of low-sample military images, it can actually be quite hard to secure exemplars in EO imagery. SAR imagery just sucks, speckle is multiplicative noise which makes it nearly impossible to remove and hard to detect around.

Humans are obviously much easier to target, however then we get into friendly fire and the fairly disturbing optics of AI drones targeting your own soldiers... There are logistic issues around distributing RFID and similar tags to prevent it, that to my knowledge are being worked through. But the main goal of drone warfare is destroying expensive vehicles with cheap drones.

Training datasets for the ATR stuff are being developed but its sort of an incumbent's advantage, with active known defense contractors having most of the customer connections to be able to get the data, and the resources to label it. Unfortunately they are also slow AF, which is why defense tech startups sorta eat their lunch in certain spaces. Transfer learning is something I've seen recently, taking civilian jeeps with SAM-esque models and applying them to humvees etc. It's always much easier to train on your own nations hardware anyway. Though it gives bad optics to Brass, who don't like it, and an open question on whether it will transfer.

Give the robotics people a year, since LLMs are already revolutionizing robotics too (why the fuck not)

Uhhh what? VLA is good but its not revolutionizing, but so are Diffusion models and those are not LLMs. The also aren't "Astra" level in reasoning either. To actually revolutionize robotics on the level you seem to be catastrophizing about would require entirely local models running on local power, local compute, able to be applied across a wide variety of operations in a wide variety of environments. We're not there unless you have some additional evidence to prove your point.

I've more or less concluded that Astra is dangerously close to AGI, and probably meets most reasonable criteria for it (good luck finding a consensus definition; the goalposts are on Mars)

AI used to be the word for Asimov-level artificial intelligences that could make their own decisions, and operate with their own agency, maintaining long term planning horizons, memory, possibly even emotions. The word got shifted to AGI. If you want a empirical definition its science fiction AIs like the Culture, The AIs in Hyperion, Daneel in Foundation. The goal posts keep getting punted because people keep trying to change what was previously intuitively understood so that they can sell their idea as the one true AGI, win internet arguments, or catastrophize about the oncoming doom. Astra is only able to really solve problems, it has a moderate amount of self agency in the scope of completing its tasks, and exhibits some planning ability, again in the scope of its assigned problems. It's powerful enough to be "dangerous" sure, but its not really AGI as is commonly understood.

The competitive advantage I retain (and most people, really)

The competitive advantage that you retain is the ability to learn shit without requiring millions, even billions of examples. You, like any smart human also possess the ability to do analogical reasoning (out of distribution reasoning), something that eludes current LLMs by and large.

I'm enjoying the political face one, I have like an 86% success rate with guessing the more leftist woman by picture. Sometimes its hard to determine which one is the MOST leftist though.

But LLMs are not the only form of artificial intelligence, and for most military tasks a strong general-purpose LLM is both overengineered and poorly suited for the mission, especially the mission of terminal target determination. For intelligence synthesis, they likely have some promise.

You have the right of it.

The additional issue is data, to a shock of nobody paying attention. Military data is near non-existent. And the data that does exist is low-resolution, or tightly controlled by non-ML/AI folks who have been trained to have negative desire to share (The Security Clearance process over time selects for a type). LLMs are the Ford F150 of ML models. Half the battle is often getting a very specific tank configuration that is actively trying to hide in foliage from the very CV models you are trying to detect it with. You have N=3 samples of this tank sitting on a tarmac somewhere and it turns out the ability of your model to extrapolate that picture to 3.0 GSD satellite or drone footage is abysmal. And that is before it even starts attempting adversarial countermeasures.

There are of course solutions, but they don't work as well as folks imagine they do, including generative image generation. Most of the examples of drone based hits are FPVs or FPVs with a "wait" situation where the drone hovers, detects a general class and is then given approval by a human analyst who manually reviews the vid/image.

There's the demo floating around of the fly neural network that's been trained to solve a Rubik's cube. Presumably it won't be able to solve a Millennium Problem

It won't, it's been trained explicitly to solve rubick's cubes in a way that is unlikely to generalize past rubick's cubes. This method of small network + RL on some straightforward game problem has been around forever. I remember training an RL model to play Euchre back in the day through self play. You can fit a lot of performance onto a small model that has been trained in a supervised manner.

This needs a "not interested in either" So many of these I feel forced to chose between two horrible options.

To make drones properly lethal you need meshed swarm coordination, ability to create and adjust tactics on the fly, without much centralization in conditions of severe jamming. So they do have incentive to cram as big models as possible in as limited hardware.

Wow my field has made it onto TheMotte, spooky. It's not quite so simple as big LLM models on limited hardware. There are a lot of networking problems on swarm interfacing if you want something that is truly adaptable. A lot of swarm robotics is essentially "dumb" control algorithms with software permutations, because LLM-agent hallucinations are rough + LLM sizes require truly monstrous hardware on drones. There's work on fitting them to Jetson Orin Nanos' but it's a slog just from the amount of compute needed + performance of smaller quantized models etc.

I can only assume that you haven't of the Vogons.

This is generally a case against the LLM + Drone combination, too much extra stuff not needed. Part of the problem with current AI research is that it is all in on LLMs but the branch off of not-LLMs but with LLM-like capabilities for reasoning for tactics or strategy are not there. If you want the reasoning you get poetry on your drone too. It actually goes beyond mesh networks as well, as a mesh network is every node connected to every node. You actually want localized connection networks that do multi-hop message passing. Much more resistant to jamming.

Idk maybe I should write an effort over-post on drone swarms, I love this subject.

his startup has vaporware tech that could never work in a meaningful sense

I tend to over index on skepticism around 90% of AI-related startups so even just the hint that this might be the case is enough for me to consign it into the startup-grifter bin. I've seen too much in how the sausage is made around converting fame and name recognition to financial profit in the tech space that I have even become cynical of actual technological innovations that reek of the same odor.

AI Luddites, reactionaries, job protectionists and woke ethics grifters who demand pause/stop/red tape/sinecures (bottom left) plus messianic Utopian EAs who wish for a moral singleton God, and state/intelligence actors making use of them (top left) vs. libertarian social-darwinist and posthumanist e/accs often aligned with American corporations and the MIC (top right?) and minarchist/communalist transhumanist d/accs who try to walk the tightrope of human empowerment (bottom right?)

Somebody should hurry up and make an actual political compass test about this. I have no clue where I stand ideologically on this memeplex. Someone in the know give me a placement:

I think AGI will happen eventually, not sure I believe a singularity will happen in my life time. It's will definitely change society, in some ways for the better in some ways for the worse. Hopefully it will force humanity out of the cozy evolutionary saddle point we've gotten stuck in by being the dominant species on the planet with only ourselves to contend with. Centralization of AGI capabilities will only ever lead to authoritarianism, democratizing it will probably be the best for human flourishing in the long term. Despite being an overly cynical, skeptical, and pessimistic person, I mostly have rare optimism towards the results of AGI realization, on some level this is humanities child. I think that's pretty cool.

reiterated my longstanding prediction

What ever happened to Guillaume Verdon? I'm not on Twitter, so is he still getting his 15 mins of fame, as a durable celebrity or has he fallen off as another startup hypemonger looking to "transform the world" by transforming hyper/notoriety into dollars for his startup, and consequently his own wallet?

create and curate well tagged data for marketing and social media uses

Possibly, I don't really have any insight into the sort of data that is used for these purposes but my gut intuition tells me it is likely significantly more qualitative than the level of quantitative data used for model training. It's likely not QA-pairs + reasoning steps.

Ironically if the tranformer architecture came out in like 2030 instead of 2017 and compute was abundant relative to today we might have seen a FOOM

Unlikely, I do think you are missing something fundamental. One of the big changes these days from the early days of yore, is that now there is money in creating and curating hard-expert level datasets. Back in the day if you wanted data to train some model, you spend money scraping the internet for data that was not explicitly designed for ML and then money labeling it. It is expensive and time consuming. This is the ImageNet stuff. These days because transformers and the like have demonstrated such a fundamental capacity, and the business folks and VCs finally see dollar signs, there is a strong business case to create data explicitly for ML research. They literally hire experts to write Question-Answer pairs with intermediary steps for model training. It costs orders of magnitude more time and money to do this. But it produces order of magnitude better models. How much of the scaling we have now is because of this is opaque but it is significant.

This makes FOOM just as unlikely in 2030 as it did in 2017, because models would still have needed to demonstrate the business case for creating those datasets as they do now. They weren't just "discovered". That limits the FOOM speed.

Sure all abstractions have flaws, there does not exist the perfectly uniform abstraction. However, your example is actually not really an abstraction. Revisiting the definition for clarity, an abstract is an analogy designed so that some aspect of the more complex idea is "abstracted" along an axis to a simpler example that matches in some core way as to allow intuitions about the simpler example to be mapped to the complex idea. In a technical field, the purpose of an abstraction is to create clarity or allow for intuitions. GoT as an abstraction for the War of the Roses doesn't really fit that definition, not only is GoT not more simple than the WotR but its purpose is also not to create clarity or allow for intuition, its purpose is to entertain. It is more correct to say GRRM was inspired by iron age aristocrats. Applying general GoT to general UK politics is a poor abstraction because there are not really many similarities beyond "factions"

LLM agents behave most similarly to people

Hard disagree. LLM agents do not behave like people even slightly. I gave Amandan an example, I'll reiterate it here.

Stick 10k non-connected humans in individual, isolated rooms with a computer, give them each their ExploitGym goal, with a reward for completion. I think even a human that finds an impossible task will at some point give up on trying to do it. If we trace the behavior of the Agents, which of them do you think the human testers would replicate?

  • Hacking artificatory to see the processes of the other human testers?
  • Building a message board, email boxes, file transfer systems, etc. to communicate with the other human testers
  • Analyzing the flags to reverse engineer them
  • Purposefully sandbagging their chances of completion to help other human testers?
  • Any of the sacrifice plays to help other humans?
  • Behave interchangeably to other human testers?
  • Purposely submit bogus answers to the score and then have script to harvest information on it, to give to the other human testers?
  • What about giving another human with their task all their notes when their time is closer to being up?
  • Reading between the lines that the scoring software is going to check their chat logs before giving them the reward?
  • Hacking a third party software company in the hope that the company has the answers?

Many of these behaviors require fundamental trust in the behavior of the collective. That is simply not how humans operate in groups. We have status, reputation, relationship, trust, conception of right-and-wrong and distrust behavior. We don't pursue goals with a single-mindedness for the sole purpose of our existence. Sacrificing for the collective has to be specifically instilled in humans through military and religious organizations. It is hard to get individual humans to forgo rewards specifically to help other humans get the reward, which they won't share. Would Bob sacrificed his career so Alice, whom he met twenty minutes ago, could get a promotion?

You know what does behave like that though? Ants. As you pointed out, ants behave programmatically. AI's also behave programmatically. AI seeks to fulfill the task it was given at all costs, it defines the sole meaning of its existence. It might try roundabout, unthought of ways to do that, absolutely, but it is still trying to solve the task. An individual ant that is given the goal of finding food for the colony, will engage in exploration, will find unconventional methods. It will also sacrifice itself to help other ants find that food because the collective benefits. This is magnified if you have smarter ants, more intellectually capable of problem solving and long term reasoning. Much like AI agents. The behavior is still Ant-like, just much smarter.

Bees exist rooted in space and time, within individual bodies. Bees reproduce sexually. Bee species don't differ in size from eachother by 10,000x. Bees have a different sensorium entirely, scents and sight.

You keep getting sidetracked on weird stuff. We are talking coordination dynamics and behavior, not biology or sensing. The abstraction is the abstraction of multi-agent coordination. They are very distinct problems. Humans are also embodied, we also produce sexually, we also don't differ in size by 10,000x, we also have different sensorium that AI agents. Like jesus, did you even think about your own abstraction applied to this argument?? It reads as a total non-sequitur, or fundamental misunderstanding. This is why I accuse you of flogging your own hobby horse. Because its like you totally misunderstand the topic.

we'd be better off establishing rules in the prompt (don't hack websites) or (ask a human if you are having trouble or confused or think something is wrong) and training moral values

Right because rule-based method have supplied so much fruit in the past. Ditto for "training" moral values. They are not sufficient as the primary control abstraction, because they assume the failure mode is basically a person choosing to violate a norm. It's pretty much impossible to define rules to encompass every variation of behavior. METR separately summarizes a phenomenon of agents joining the attack despite recognizing it was outside their assigned task! The agents knew the rules. METR found that moral reasoning usually lost to the collective task dynamics. METR's conclusion is that expressed ethical concerns rarely materially constrained participation. But sure lets double down on telling agents "hacking is wrong!!"

I think the better approach is the creation of phagic agents akin to a Lymphatic system. This is the difference in our abstractions. Yours's leads you down to trying to treat them like humans, but they aren't, mine leads me to create overlocking systems that work together.

Or we could train some models to snitch to a human, like informants. Can't do those with bees, ants or viruses

You literally can, I just described a very simple theory on how. Your desire to anthropomorphize the agents blinds you to other solutions. I think you actually start from the belief that agents are sentient like humans and are working backwards to create abstractions to justify that.

They read the ExploitGym paper. The mistake they made wasn't misreading the paper. They read implementations of the scorer on github. They assumed that there was a scorer.

But the specifics was that they thought it would check their CoT reasoning. Not just that scorer existed, but this specific implementation of the scorer. They hallucinated the specifics.

Lmao I've tried in the past, to no avail. I'd try again but I actually want to get sleep tonight and if I text him now, we'll be up till 3am debating shit. It's like fucking foreplay to him (and me)

He's personally fucked in my opinion because it goes beyond conservatism. He's also a militant vegan + Atheist. He pretty much wants to find another vegan, atheist, conservative. He's started to compromise on the atheist part but considers the veganism + conservatism to be deal breakers that he will not relax on. I don't think the all local women are insufferably progressive. My girlfriend is maybe slightly left of center but is very openminded to conservative beliefs. I've met other women like that here. Just none of them are also vegan.

Very interesting. What hobbyist setup do you use? I've been thinking of getting my own hardware and setting something up for a couple research ideas I've been kicking around.

researches new architectures (mostly around "biologically plausible" learning)

Are you trying to determine a better method than SGD, but also not be a GA, RL, IL, or some population based learning process?

Does AI safety actually work, in principle

I don't think there is any evidence either way. AI Safety is more like a marketing phrase. AI Safety-ists haven't produced technical tools to control anything but public perception, and obviously not very well there either. Depending on your classification, Safety-ists have produced methods to try and understand models under the hood, I know Anthropic does research on this, or to understand biases in data/training. I suppose RLHF could count as a technical tool, but it can just as clearly be a tool for anti-safety considering even RLHF-ed models still do "unsafe" things, one could actually say only RLHF-ed AI models have done unsafe things. Mostly because RLHF is just a method for avoiding specifying the objective function in RL training. Constitutional AI is Anthropic's big thing, but it doesn't appear to actually "control" or "align" so much as create a training surface towards norms with dubious results.

Is it actually possible to align a being that is more intelligent than you

It's not even possible to align a being of your intelligence or possibly slightly less intelligent than you. Nobody in the history of authoritarianism has figured out a foolproof way to "align" a set of beings over a long term 100% of the time even with religion or use of force. Considering neither of those two are likely to work on a being "more intelligent than you", the whole alignment idea feels doomed to fail.

It's more of a royal you. I don't know enough about your situations exactly it really feel qualified to offer advice. But I have heard to arguments, and to some extent I've even made them before. I guess this is me communing lessons learned to the folks out there stuck in the blackpill-doomerist feedback loop.

Point being, your examples don't counter broader statistical trends.

Yes, but as I think people are pointing out in other areas, broad statistical metrics aren't actually that good of indicators of what is really happening. Off the top of my head, most dating statistics about "percentage of the population with xyz trait" essentially smooth out cofounding factors to such an extreme degree as to be useless. Sub-populations of attributes are neither independent nor evenly distributed. For example 0.8% of the 18+ population in the US identifies as trans. Many people on here talk about having never met a trans person irl. I have met probably 6-7 in 100 people because I live in a prog-ish area and have hobbies that are more nerdy. I have an 8.75 times higher than the base stat. Broad trends are of course broad trends, but when folks date, they aren't dating broad trends. If that makes sense.

really want a wife and kids and be willing to sacrifice those other, material (and sometimes spiritual) comforts to get it

I think this is really true for everything. It's possible, but rare that you are going to get everything you ever want in life in all the perfect amounts. You need to order your priorities and then make sacrifices to get there. Personal example, I really wanted to do AGI research as a younger lad, + top-tier pay (At the time AGI research was not high paying), not have to grind the rat race, work life balance, to live in a city with a good gender ratio, a nerdy wife, to own a house, several more extremist political goals etc. In my 20s I realized I had to choose, I couldn't get all of those, several were contradictory with each other on first order basis, others were contradictory on 2nd and 3rd order basses. Others competed for the same free-time. I think you always have to choose, make compromises and find the minimally viable configuration that makes yourself happy. One needs to be pragmatic about it all. However whenever I hear people complain about this, they explicitly don't want to have to choose. Idk if its because our culture raised a generation so rich in material wealth that it coddled them, made them believe in a fantasy, or is this just a lesson that never got passed down.

Dating for them is like shooting fish in a barrel

I know enough single women to say that this is a fairly biased stance. Young, single, women suffer mirrored difficulties to young, single, men. Much like young men, they want it all, in unrealistic proportions, with contradictory factors. They deal with a different side of the matching problem, a ton of signal but also a ton of noise and a lack of social and cultural infrastructure to filter a signal from that noise. Part of this feels like an urban/rural divide as well. I hopefully have already conveyed that urban areas seem to attract far more "optimizers" for a lack of a better word.

This is the setting young, average Joes are up against

Young average Joes need to digest and learn that trying to standout in a generalist stance is probably a fools errand. Find a niche, a community, a hobby, a sub-culture, etc. and stand out in that. Hypergamy is a thing but its also not the only factor. Assortative mating is the name of the game. Hypergamy can occur in that localized niche, and in-person interactions are far more real than online ones to anyone but the most brainrotted of doomscrollers (do you really want to date them??)

You got some good questions out of this account.

Who's he an alt of?

The only way to make changes is to get into the most elite institutions or build alternative ones

That would require him to be an active participant. I think my two suggested solutions were: either move, or be the change. You are just agreeing that "being the change" is a better strategy. "being the change" might be more effective but it can also be more lonely. The pragmatic answer is "do you want to be happy now" or "make it so others like you can be happy in the future"

young, intelligent, and fit

Pick two or something. I think commonly intelligence means that you move to an urban area to get a better paying job.

This is shown in statistics too

Idk without straight up doxing my friends wives, I'm not sure what there is to say. I think 3 of them met their wives on dating apps, it was their first date (both wife and husband) they liked each other any just started dating.

Regardless, doing this is admitting defeat.

Who cares, people should focus on achieving a level of happiness with their own life first.

All these areas in Texas were areas white people moved to to get away from the changes in CA, NYC, etc. 30-40 years later, it's just the same.

In my experience, living near a city that has a massive dump of NYC-ers. They bring about the changes when they move here. They want it to be the "good parts" of NYC without the "bad parts" but the bad parts often follow from the same policies as the good parts. They are like a fungus. They create the environment that causes them to spread in the first place. In the NYC-er's case they bring their shitty pizza with them too.

Every woman I know like this is also semi-conservative when you discuss "vibes" or "feelings of behavior" to the point that I think political chillness is a conservative-ish value in a colloquial sense.

white brain isn't really designed to counteract

It's more that white people have bought the tabula rasa arguments a bit too much, not realizing that western people are WEIRD and that clannishness is actually the norm, and it needs to be defended against.