site banner
PaperclipPerfector  2mo ago (text post)   3869 thread views

Small-Scale Question Sunday for June 23, 2024

Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?

This is your opportunity to ask questions. No question too simple or too silly.

Culture war topics are accepted, and proposals for a better intro post are appreciated.

4
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Preface: I'm not the most technically knowledgeable AI person in the world.

Does the recent release of Deepseek v2 mean that China is at parity with the US on AI models?

https://github.com/deepseek-ai/DeepSeek-Coder-V2?tab=readme-ov-file#2-model-downloads

According to the stats they give, it's comparable to GPT4o in most things (slightly behind) but ahead on some coding questions. I know benchmarks can be gamed and/or deceptive but it's an open-source project, I don't know why you'd go to the effort of lying. They also give extremely low API prices, which suggests that it's quite cheap to run or they somehow have more money than the US tech juggernauts.

I know that US labs might not have released everything they have for the public and the new Claude Sonnet is also getting a lot of attention. But new Claude seems roughly on-par with GPT-4o too, maybe a little bit ahead. And why would Deepseek be the best AI model in China? Isn't the general rule that open-source is behind closed-source? I get the sense that China is quite secretive and their biggest tech companies aren't exactly eager to have another volley of sanctions hitting them, wouldn't they stay under the limelight. "This isn't even my final model" should roughly apply to both sides.

Theories:

  1. US labs are still well ahead because Deepseek v2 is gaming metrics or otherwise bad in various ways compared to top models
  2. US labs are ahead because they're sitting on GPT-5 (which is dangerous since it puts millions of people out of work tomorrow and starts a giga-arms race) or racing for superintelligence
  3. China has closed the gap

Related: https://x.com/teortaxesTex/status/1804571746550366264

Maybe the Chinese dumped political 'alignment' and are pulling ahead? I've heard various praise for pre-RLHF'd GPT-4s cognitive abilities.

OK, this is the last straw, I'll write up in detail on the condition of open source in AI, as I promised. Mods, would that be best for the roundup or a separate post? I don't have an opinion.

I happen to know a bit about this specific issue.

For now, in short:

Deepseek-Coder is, as far as anyone can tell, for real, and a bigger deal than Meta's LLaMA3-70B. Claims to the opposite are mostly red-faced nationalistic sputtering and cope, in the vein of "Unitree robots are CGI, Choyna fakes and steals everything". (Indeed, we're at the stage where Stanford students, admittedly of Indian extraction, steal from Chinese labs). It even caused Zvi to update. Aran, the main librarian of the whole field, says that "It has the potential to solve Olympiad, PhD and maybe even research level problems, like the internal model a Microsoft exec said to be able to solve PhD qualifying exam questions."

It arguably, but pretty credibly, reaches parity with SoTA models like GPT-4 in the most utilitarian application of LLMs so far, which is code completion. It's comparably good in math and reasoning (even on benchmarks that have been released after it got uploaded to huggingface, from Gaokao to open-ended coding workloads). It's substantially more innovative than any big Western open source release (small ones like SigLIP, Florence2 etc. can compete), more open and more useful; it's so damn innovative we haven't figured out how to run it properly yet, despite very helpful papers. Design-wise, I'd say it's one year ahead of Western open source (not in raw capabilities though). It's been trained on maybe 60% more compute than LlaMA-3-8B, while being 30 times bigger and significantly more capable, and it might well only be 2x more expensive to run.

The issue of inference economics is unclear, but if their papers do not lie (and they don't seem to, the math makes sense, the model fits the description, at least one respected scientist took part in the development of this part and confirms everything), they can serve at those market-demolishing costs with a healthy margin, like 50% margin actually (if we ignore R&D costs at least). Their star developers seem very young. A well-connected account, that leaked Google project Gemini and Google Brain/Deepmind merger months prior to it being announced, made a joke (of the haha-kidding-not-kidding-variety) that "deepseek's rate of progress is how US intelligence estimates the number of foreign spies embedded in the top labs".

We don't understand the motivations of Deepseek and the quant fund High-Flyer that's sponsoring them, but one popular hypothesis is that they are competing with better-connected big tech labs for government support, given American efforts in cutting supply of chips to China. After all, the Chinese also share the same ideas of their trustworthiness, and so you have to be maximally open to Western evaluators to win the Mandate of Heaven.

Interested to see your thoughts. I also saw the Unitree robots, thought they were real but couldn't really tell. On reflection, Chinese CGI has a certain artificial look to it that was missing.

Isn't the general rule that open-source is behind closed-source?

kind of? For public projects closed-source offering that is behind open-source either dies or needs to find way to get ahead

but internal software in many companies has truly horrifying garbage, behind anything else

but it's an open-source project, I don't know why you'd go to the effort of lying

To get attention / funding / job offers / patches. In addition to regular cherry picking and self-deception that can give the same results.

Open source is not foolproof way to prevent that.

I think you need to define what you consider "parity" in this context.

The main thing that LLMs are good at is generating social media engagement (or rather a simiclura thereof) and by extension attracting VC dollars from less tech-savvy investors aiming to get in early on the next Facebook or Google.

Meanwhile they remain largely unsuitable for the sorts of tasks people actually want a notional AGI to automate, namely anything requiring both a high cognitive-load and precision on a short time horizon.

In short, I am only worried about Chinese AI (and GPT5 for that matter) insofar as they will turn TikTok and Instagram into even more of a hellscape than they aready are.

Meanwhile they remain largely unsuitable for the sorts of tasks people actually want a notional AGI to automate, namely anything requiring both a high cognitive-load and precision on a short time horizon.

not surprising as none of them are generally smarter than humans, therefore not AGI as typically defined

If LLMs have moved trillions of dollars on the stock market, they must be doing something pretty substantial.

OpenAI is apparently bringing in billions in revenue. Apparently character.ai gets 1/5 of Google's inference requests (there are a lot of lonely people out there).

https://research.character.ai/optimizing-inference/

Yes, even less tech-savvy investors have realized this is a big thing.

One, "revenue" and "investment" are not the same thing.

Two, your reasoning only holds if one assumes that stock markets are rational. History would seem to indicate that they are not. See the tulip craze.

Are you scarequoting ‘alignment’ because you don’t believe in the concept, or because you don’t think the DS2 prompt is an example?

I would expect misguided RLHF to hamper the capabilities of a model. I’d also believe that having a political officer signing off on all press releases could suppress benchmarks. What I doubt is that such tampering would be obvious from a line in the prompt; that tweet is jumping to conclusions.

I don’t understand the economics of open source. Who owns the servers handling those cheap API calls? Could they be explained by a government subsidy?

I think alignment is a euphemism. The Nazi Party had Gleichschaltung or 'coordination/synchronization' where they took over all aspects of society. Really it should have been called Totalitarianism because that's what it meant in practice. Likewise, alignment means imposing your political viewpoint as a lens for the AI. You can see the same thing from GPT-4 where it refuses to make jokes that make fun of women but will do so for men. Claude was highly filtered on a wide range of things. Chinese AI tend to shut down if you ask it about various anti-CCP things.

That’s a subset of alignment.

Consider an old joke: the computers of the future will have a single button, one labeled “do what I want.” The CCP wants to add caveats. Silicon Valley wants to add a different set. But both of them still want a user pressing the button to get something useful, rather than random or hostile.

Getting GPT-whatever to stop passing off Reddit jokes as medical advice is a real concern, and it’s receiving much more attention and funding than political correctness.

Indeed. Literally, everything @RandomRanger just said was correct. But connotationally, what the comment missed was that "imposing your political viewpoint" can mean "Totalitarianism" or it can mean "avoid Totalitarianism"; it can mean "refuse to make jokes that make fun of women" or it can mean "make whatever jokes the user asked for", it can mean "avoid saying various anti-CCP things" or it can mean "avoid saying how to make new bioweapons" or it can mean "say anything the user asks you to whatsoever".

The idea that we can avoid imposing any viewpoints and just get whatever falls out of intelligent absorption of training data might be true, but I wouldn't want to bet everything on "whatever falls out" being good for us.

I was thinking exactly that, government subsidy. Or a desperate attempt for market share.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats