The Wednesday Wellness threads are meant to encourage users to ask for and provide advice and motivation to improve their lives. It isn't intended as a 'containment thread' and any content which could go here could instead be posted in its own thread. You could post:
-
Requests for advice and / or encouragement. On basically any topic and for any scale of problem.
-
Updates to let us know how you are doing. This provides valuable feedback on past advice / encouragement and will hopefully make people feel a little more motivated to follow through. If you want to be reminded to post your update, see the post titled 'update reminders', below.
-
Advice. This can be in response to a request for advice or just something that you think could be generally useful for many people here.
-
Encouragement. Probably best directed at specific users, but if you feel like just encouraging people in general I don't think anyone is going to object. I don't think I really need to say this, but just to be clear; encouragement should have a generally positive tone and not shame people (if people feel that shame might be an effective tool for motivating people, please discuss this so we can form a group consensus on how to use it rather than just trying it).
Jump in the discussion.
No email address required.
Notes -
GPT-4's writing is wholly unimpressive to me. I mean, it's impressive "for an AI" certainly, it's impressive in the sense that this sort of thing was unfathomable to me a few years ago - but in terms of actually writing a long-form piece that makes me think "hey this is really great, I want to read this", there's basically nothing there at all, among all the examples I've seen.
This is in contrast to the best examples of AI art, which definitely do look good and are pretty nice to look at, despite still having some flaws. This basically matches with my intuitions about what current machine learning techniques should be capable of, because "draw a spatially accurate picture of X in the style of Y" requires less creativity than "write an engaging and thought-provoking novel". Representational art is ultimately a technical problem (and I did recognize this fact many years before I ever thought that AI art would be possible, you'll have to take my word for it) - computers have been drawing pictures for decades. It's just that they used to require a massive precise description of vertices and colors in order to do it, but now we can give the description in natural language. Writing (the best writing anyway, the only writing that I bother reading in the first place) brings you immediately much closer to raw ideation - you can't just construct a product according to technical specification, you have to have novel thoughts at the same time, which is a much harder problem.
It's possible that there's a fundamental "creativity barrier" that the current LLM architecture will never be able to break through - creativity being that X factor that's involved in the production of genuinely original art, or novel scientific discoveries, or influential philosophical ideas. It doesn't seem to me that GPT-4 is any closer to breaking through this barrier than GPT-3 was. I acknowledge that I could be wrong though, and if GPT-5 is able to write a novel as good as Nabokov's Pale Fire then I'll update more strongly towards thinking that scaling transformers alone really will produce AGI.
I have a different, not necessarily conflicting perspective as yours.
Namely, that human writing is shit. No really, 99% of literary output is utter dross barely worth proofreading. Look at the amateurish garbage regularly posted on Reddit writing subs like Writing Prompts as an example.
What strikes you as good writing is the cherry-picked output of tens of thousands of would-be writers, and as such, you're not comparing GPT-4 to the average level of human writing, which it handily exceeds.
As of today, it can't write a proper novel since even a 32k token context window isn't sufficient, but I have gotten very decent prose out of it in my tinkering, especially if you prompt it with the names of existing good human authors.
In other words, GPT-4 is a General Intelligence in the sense that a typical human is a General intelligence, it's simply not strongly superhuman at the task.
I completely agree with you here. Sorry if I didn't make that clear enough. My point was that if all GPT is doing is beating the average or even the above-average writers then that's not really relevant to me, because I wasn't reading that type of writing in the first place. It needs to beat the superior writers to really have an impact.
I imagine that we would have some disagreements over how big the elite actually-worth-reading club is, and who gets to be admitted to it, but I agree with the basic point you're making here.
Actually it can, with recursive prompting. Predictably, the result is not very interesting. But in fairness, there would probably be an immediate improvement if you got rid of all of OpenAI's safety guardrails.
Oh sure, it can write some good sentences. But writing a good sentence is like drawing a good (representational) picture - it's a purely local problem that doesn't have to deal with global conceptual structure. What makes a strong essay or a strong novel is the global structure, and the intricate relationships between local pieces and the overall structure. If there is an inherent limit to the "predict the next token" methodology, it will be found here.
Fair enough! I don't think we have any real disagreement at all then.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link