site banner
PaperclipPerfector  12d ago (text post)   2338 thread views

Friday Fun Thread for December 6, 2024

Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.

2
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

This week I learned that LLMs are terrible at coming up with quiz questions.

We're having an end-of-the-year party with my team of data engineers, and I want to prepare a quiz for them. I don't want Jeopardy-style "either you know it or you don't, your only hint is the category name" questions, my gold standard is the What? Where? When? show that has questions that:

  • look impossible to answer at first glance
  • have a hint in the question itself
  • don't require knowledge of random trivia
  • can be answered by six smart people in 60 seconds and they can use this fact as a hint

For example, "droste.zip is a small Zip archive that contains two files. One of them is droste.jpg, a very small picture of a Droste hot chocolate poster. Its only purpose is to make the presence of the other file even more impressive. What is the other file in this archive?"

Or, "Charles O'Rear has spent more than 20 years working for National Geographic in exotic locations. But his most expensive and most viewed work is a very simple landscape. Name this photograph."

ChatGPT is hopeless at this. It suggests questions like "What does su in sudo stand for?" even when you explain what you want from it, why its questions are wrong and even give it my handcrafted questions as examples. I thought Russian-language LLMs would be better at it, having a better corpus of competitive W?W?W? questions to learn from, but they were even worse.

Since I sincerely hope none of you work for me, here's some friday fun for you:

  1. Can you answer my questions?
  2. Can you come up with more computer-related questions like these?

Go to db.chgk.info instead of reinventing the wheel. You don’t even have to translate the stuff :)

I know about it, but it doesn't have a filter by theme.

Others have answered correctly.

Here's my effort.

In a well known Jack Nicholson film his character plays or we hear on the soundtrack the following:

Fantasy in F minor

Fantasy & Fugue in D minor

Piano Concerto no 9 in E flat major

Prelude in E minor

Fantasy in D minor

Characterize these and if brave tell me the name of the film.

This is the kind of question I do not want on my quiz, unless it's a movie-themed quiz. While this film has been praised by the critics, I don't think a 1970 movie is on anyone's mind in 2024, unless they are a big movie buff.

But you did know the answer! Winner!

Five Easy Pieces

I didn't, I googled for it.

Question 1: wild guess. Itself?

Question 2: Windows XP default desktop background?

Nice job!

Surely both of these require knowing random trivia?

Charles O'Rear has spent more than 20 years working for National Geographic in exotic locations. But his most expensive and most viewed work is a very simple landscape. Name this photograph.

It has to be windows xp

There is a good paper called Embers of Regression which basically points out that LLMs do not have a consistent intellect. Their ability to perform tasks such as “Write the following in word-reversed order” will change markedly depending on whether the result is a sane/common string or not. So trying to get novel material is hard.

I can’t answer your riddles, I’m afraid. I was never good at puzzles.

Thank you very much for sharing that paper. It is excellent.

Do you have any other "must read" papers on LLMs? Or a link to a good list?

Glad you enjoyed! I subscribe to TLDR AI, it’s surprisingly good.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats