Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?
This is your opportunity to ask questions. No question too simple or too silly.
Culture war topics are accepted, and proposals for a better intro post are appreciated.
Jump in the discussion.
No email address required.
Notes -
It's an offshoot of the widely-reposted AI Twitter claim that 'we could have trained GPT-2 in 2004' (or with 2003 levels of supercomputer compute). And that might well be true, idk. Here's one of the biggest sources.
What's less believable is that nobody involved in this hypothetical effort at the NSA decided to just get rich in the private sector after coming up with technology decades ahead of the competition.
Guess i was wrong! I'd actually read that post before, seems I forgot.
Sometimes my "real" justifications build on a lot of accumulated knowledge and ideas, and writing those all out would take longer than I wanted, so I don't, and substitute for something shorter instead. Sometimes the shorter thing is wrong, though. So my 'real' reason for saying .1% was something about how mathematics and coordination and coming up with ideas is hard, and as we observe society develop we're seeing the best of everyone we have slowly stumble into being more and more correct, and it's almost impossible to beat that privately on something as big as 'GPT' because you have to do all of the research work that tens of thousands of the brightest machine learning researchers did in public over the past few decades. Like, the manhattan project was secret, but it used all of the best people we had and wasn't secret forever. The NSA can keep some cryptographic techniques secret, but not the entire concept of cryptography secret.
More options
Context Copy link
More options
Context Copy link