The Wednesday Wellness threads are meant to encourage users to ask for and provide advice and motivation to improve their lives. It isn't intended as a 'containment thread' and any content which could go here could instead be posted in its own thread. You could post:
-
Requests for advice and / or encouragement. On basically any topic and for any scale of problem.
-
Updates to let us know how you are doing. This provides valuable feedback on past advice / encouragement and will hopefully make people feel a little more motivated to follow through. If you want to be reminded to post your update, see the post titled 'update reminders', below.
-
Advice. This can be in response to a request for advice or just something that you think could be generally useful for many people here.
-
Encouragement. Probably best directed at specific users, but if you feel like just encouraging people in general I don't think anyone is going to object. I don't think I really need to say this, but just to be clear; encouragement should have a generally positive tone and not shame people (if people feel that shame might be an effective tool for motivating people, please discuss this so we can form a group consensus on how to use it rather than just trying it).
Jump in the discussion.
No email address required.
Notes -
FunSearch: Making new discoveries in mathematical sciences using Large Language Models
Another day, another AI breakthrough, this time in maths.
Mathematics has been surprisingly difficult for computers to get a hold of, which would seem weird when you consider they're made of maths themselves. For decades, automated theorem solvers were more of a curiosity rather than anything practical, with the bulk of the problem being, as I understand it, representing intuitive-to-humans (and mathematically valid) concepts/axioms in their programming language. Of late, they've gotten better, and are handy for proof-reading and ensuring no nasty little errors creep into theorems that can stretch pages and boggle even the imagination of talented mathematicians to keep straight in their head.
However, short of brute-forcing certain classes of problems through exhaustive search, novel mathematical insight remained out of the reach of both programs that can't really be called intelligent, as well as proper AI (not that this isn't invaluable, good luck solving some of those by hand).
Until now.
Deepmind, a subsidiary of Google/Alphabet (does anyone call them Alphabet? At least Meta took off), announced FunSearch, which stands for function search (I'm sure somebody finds it fun, I'll settle for interesting).
As of today, the technique (rather than a model, per se, the general approach can be slotted into most LLMs, they used PaLM 2 this time), advanced the SOTA in two particular problems, namely the Cap Set problem and Online Bin-packing.
Of particular interest is the fact that this wasn't done by brute-forcing the space of all possible programs, which as Deepmind notes, is computationally intractable because of inevitable combinatorial explosions.
FunSearch is an iterative procedure; at each >iteration, the system selects some programs from the current pool of programs, which are fed to an LLM. The LLM creatively builds upon these, and generates new programs, which are automatically evaluated. The best ones are added back to the pool of existing programs, creating a self-improving loop. FunSearch uses Google’s PaLM 2, but it is compatible with other LLMs trained on code.
Blah blah blah, I've been awake for 27 hours, my patients are dying, and my patience is dead, I can't be arsed to clean it up further and add additional commentary for the main thread, the "Stochastic Parrot" is better than you are at practical problems, justifying why AI becoming better than us at everything needs a CW angle, fuck me, fuck you, I'm going to take a nap
It would be great to see this post in the Friday Fun thread instead of Wellness Wednesday, but I assume this was your initial purpose and I should put the blame on your sleeplessness.
The wellness part is more of a cry for help as I steadily experience temporary dementia from sleep deprivation. I'm sure "FunSearch" could also work in the fun thread.
More options
Context Copy link
More options
Context Copy link
Reading the paper, doing the set of experiments for one of the problems cost $800-1.4k. Extremely affordable!
This isn't as impressive as 'LLMs good at abstract math' would be, though. This is basically making a million copies of a smart 14 year old, telling them each to randomly tweak programs in ways that seem interesting, and running an evolutionary process on top of that for programs with high scores on some metric. As opposed to taking a LLM and teaching it 1000 math textbooks and then it spontaneously proving new theorems. Which is a thing that this paper, notably, very much doesn't do. But, you know, another paper totally might in 5 years, the field's moving quickly.
But the discovered functions are less triumphs of machine thought and more like random blobs of if statements and additions with a bunch of simple patterns (eg fig 4b, 5b, 6b). Even that's quite useful.
DM claims that the code generated is clear enough that human evaluators can notice obvious patterns and symmetries, which would certainly accelerate things.
As far as I'm concerned, this is only the first step, PaLM 2 isn't a particularly good model, and the fact that this works at all makes it a good bet that I'll only continue to get better.
If I had to set a date on when I expect:
It would be closer to three years, but since you and I are in agreement that it's going to happen, and soon, there's not much else to do but wait and see.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link