This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Could anyone actually run these things on a laptop? I know Apple's been up to some wizardry with its new Macbooks. They seem to have unified RAM/graphics memory, so I suppose it could fit on the machine. But fitting a whole AI model in just over 2 kilos? My intuition is that it should burn a whole through your desk or implode like a dying star. The macbook weighs less than a 4090 and has to have storage, cpu, screen, keyboard and so on.
If so, it truly is over for PC. Someone should start making graphics cards with immensely high vram too.
You absolutely could run them on a Macbook, and at decent speeds. Interactive decoding of the sort you'd need with a dialogue-oriented LLM is mainly bottlenecked by memory bandwidth needed to move weights between DRAM and registers, not by compute/processing cores. And Apple Silicon has insane theoretical bandwidth by CPU standards (M1 Pro has 200GB/s bandwidth, M1 Max has 400 and M1 Ultra, available on the Studio with 128Gb RAM, up to 800, vs. 90 for i9-13900K or 54 for the flagship Ryzen 9).
Nvidia cards with the same total memory would still be multiple times faster, though – and with the market full of ones used by miners, likely cheaper. Here's a good blog post on the topic, here's the list of most cost-effective ones. Note, however, that he doesn't worry about interactivity, so regards faster recent cards as more valuable. All things considered, gaymer RTX cards are so much better than A100s and such for many (though not all) tasks that Nvidia has a clause in its contract with datacenters prohibiting their use. 3090 (bandwidth 936 GB/s) is still a decent choice.
Power draw in the moment will be a bitch, of course.
Good post.
I'm surprised Apple makes laptops that powerful. Wouldn't it make more sense to just sell a desktop machine, so you can fit in all that stuff more easily and cool it? The M1 Ultra seems roughly comparable to a 3090, albeit more power-efficient and with lower total processing capacity. But who needs power-efficiency, electricity is cheap. And who needs a mobile 3090? Do San Francisco hipsters really go out to do some video-editing (or AI modelling) in some trendy cafe?
Studio hardware caps out at $8000; Mac Pro, at $50000. I imagine they will make some sort of Apple Silicon Mac Pro that stands above Studio, maybe with another doubling (or two doublings) of the next-generation Ultra. But usually their desktop is a very niche product and isn't refreshed as often. Also, its selling point is customizability that you can't (easily) have with these chips – the ability to get an insane core count, or memory on the scale of a decent Supermicro server, and still in the slick Apple package.
I think at this point it's more profitable for Apple to design and produce an all-around powerful compact SoC that radically improves their already-prestigious laptop line and crush the competition, rather than to fuck around with multi-part systems and market segmentation. They're capitalizing on their years of bespoke ARM engineering. It's everyone else who's making unjustifiably bad CPUs.
For some workloads. Also, TSMC's 5nm vs Samsung's 8nm helps.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link