site banner
naraburns  nihil supernum  1yr ago (text post)   3961 thread views

Friday Fun Thread for March 3, 2023

Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.

4
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

I found an comparison between davinci-003 and chatgpt. Seems like chatgpt is better at some things and worse at others; not a blanket downgrade: https://scale.com/blog/chatgpt-vs-davinci.

Also there is no more data flywheel:

Data submitted through the API is no longer used for service improvements (including model training) unless the organization opts in

How have they cut costs? I've seen answers I don't understand. It doesn't seem like they are necessarily running at a loss:

Quantizing to mixed int8/int4 - 70% hardware reduction and 3x speed increase compared to float16 with essentially no loss in quality.

A*.3/3 = 10% of the cost.

Switch from quadratic to memory efficient attention. 10x-20x increase in batch size.

So we are talking it taking about 1% of the resources and a 10x price reduction - they should be 90% more profitable compared to when they introduced GPT-3.

https://old.reddit.com/r/MachineLearning/comments/11fbccz/d_openai_introduces_chatgpt_and_whisper_apis/jaj1kp3/

I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back. Public price per A100 is $2.25 for one year reservation.

https://news.ycombinator.com/item?id=34986915

Essentially, I had previously thought that better language models will need to be orders of magnitude larger and orders of magnitude more expensive. Now it seems clearer that we're moving towards a future of intelligence too cheap to meter. And "endless stupidity generator"? I just plain disagree, and feel compelled to build a thin client product on top of the API to prove it. There are many jobs that are mostly low-skill word manipulation, and now that you might get that 100x or 1000x cheaper, it opens up yet unimagined opportunities IMO.

I'm not sure how much of this to believe; if it is dumber, that has to be explained somehow.

In my experience, the subjective generic text completion (and iirc perplexity) – is worse, even if most downstream tasks end up being better, and while the mean of sensible instruction-following is higher for RLHF-version, the variance is much lower, so you struggle to get ChatGPT out of its midwit persona in a way that isn't true for davinci (though this may be damage from arbitrary preferences and political bias, rather than the inherent damage of the technique). Opt-in will probably be incentivized.

Even at int4 (which is still dumber in the absolute sense, just more efficient per byte used) a 175B model uses >2 A100-40gb, and if they mix layers it's more. And for a low-latency interactive app, the utilization of 50% sounds rosy – Google with PaLM barely broke 40% in their paper.

Assuming they use 16 bit floating point

We've already assumed int8/4 though, but in any case the bottleneck for interactive use is memory bandwidth, not flops. We'd need more details.

I know there exist (at least on paper) techniques for making inference way, way cheaper, probably even cheaper than you suggest, with minor loss in quality – just not sure they have deployed them yet.

I just plain disagree, and feel compelled to build a thin client product on top of the API to prove it

Here's some free lunch: take LLaMA-65B and rent some GPUs, it'll be 2x cheaper using all the same tricks and allegedly about as good.

There are many jobs that are mostly low-skill word manipulation

I wonder if we will discover that there are many jobs which are, in fact, net negative for the economy only when people doing them get automated away and their sinecures lose lobbying power .

I wonder if we will discover that there are many jobs which are, in fact, net negative for the economy only when people doing them get automated away and their sinecures lose lobbying power.

Already had been discovered.

At the best, about 20% of adult population is keeping all of us alive and comfy, the rest is engaged in digging holes and filling them again (only in comfy offices). UBI is already there, only in the most wasteful and inefficient form imaginable.

Who wants a job not as last desperate attempt to get money, but "to serve mankind", should find employment in water, sewer, power, food production/distribution etc...

I wonder if we will discover that there are many jobs which are, in fact, net negative for the economy only when people doing them get automated away and their sinecures lose lobbying power .

Interesting - which jobs come to mind? HR and environmental reg agencies jump to mind for me.

HR, environmental regulation, personal banking/financial advising, marketing and fundraising, low-risk medical diagnostics, social work, secretaries, most software operations (devops) teams, data analysis, first-pass website content moderation, (indirectly) higher education Arts departments, and some others I'm sure I'm forgetting.

I think the far more interesting thing is that most of the industries in the crosshairs of AI are majority-female; it's the first example of a technology that's going to primarily replace women (nearly all technology ever invented only replaces men) and I think the political effects of that are going to be very, very interesting when you combine it with the fact that a lot of the women currently in those jobs will be single and functionally sterile when they're replaced.

Which reminds me that I should probably buy some Petco stock.

I get a it confused by that list tbh.

A lot of these things are either not useless or not going to be automated in the near term. I might despise advertising and people working with advertising as much as the next dude but you're dreaming of you think AI is going to make this go away.

I shamelessly reposted this to the CW thread if you want to throw this in there. I tend to agree, given that women have social power maybe UBI will actually happen once women are affected?

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats