site banner
naraburns  nihil supernum  1yr ago (text post)   33252 thread views

Culture War Roundup for the week of February 20, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

15
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Someone last week posted about how there wasn't any interesting AI-generated music, and my first thought was that there hasn't even been any decent AI music analysis. Here's what I'm talking about: If you study music at the collegiate level (or maybe even at a really good high school), you're going to be asked to transcribe a lot of recordings as part of your coursework. This can be time consuming, but it's fairly straightforward. Suppose you're transcribing a basic pop or R&B song. Listen to the bass part and write it out as sheet music. Listen to the guitar part, etc. After you have all the instruments down go back and add articulation marks, stylistic cues, etc. Figure out the best way to organize it (first and second endings, repeats, codas, etc.). In other words, turn the recording into something you can put in front of a musician and expect them to play. I've done it. It's not that difficult for anyone with a basic knowledge of music and dedication to learning, which is why they expect every musician to be able to do it. YouTuber Adam NEely has done videos where he transcribes pop recordings for a wedding band he's in. There are, of course, some people who can write out the third saxophone part of a big band recording from memory after hearing it once, and these people are rare (though not as rare as you'd think), but most musicians are still pretty good at transcribing.

Computers are absolutely terrible at this. There is software available that purports to do this, some of which is available online for free, some of which is built into commercial music notation software like Sibelius or Finale, and the utility of all of it is fairly limited. It can work, but only when dealing with a simple, clean melody that's reasonably in tune and played with a steady tempo. Put a normal commercial recording into it and the results range from "needs quite a bit of cleanup" to "completely unusable", and at its best it won't include stylistic markings or formatting. At first glance, this should be much easier for the computer than it is for us. We have to listen through 5 instruments playing at once to hear what the acoustic guitar, which is low in the mix to begin with, is doing underneath the big cymbal crash, and separate 2 sax parts playing simultaneously, sometimes in unison, sometimes in harmony. The computer, on the other hand, has access to the entire waveform, and can analyze every individual frequency and amplitude that's on the recording every 1/44,100th of a second.

Except that this is a lot harder than it sounds. As psychologist Albert Bregman puts it:

Your friend digs two narrow channels up from the side of a lake. Each is a few feet long and a few inches wide and they are spaced a few feet apart. Halfway up each one, your friend stretches a handkerchief and fastens it to the sides of the channel. As the waves reach the side of the lake they travel up the channels and cause the two handkerchiefs to go into motion. You are allowed to look only at the handkerchiefs and from their motions to answer a series of questions: How many boats are there on the lake and where are they? Which is the most powerful one? Which one is closer? Is the wind blowing? Has any large object been dropped suddenly into the lake?

And Bregman was just talking about the ability to separate instruments! A lot of transcription requires a reasonable amount of musical knowledge, but even someone who's never picked up an instrument and can't tell a C from an Eb can tell which part is the piano part and which part is the trumpet part. And then there are all the issues related to timing. Take something simple like a fermata, a symbol that instructs the musician to hold the note as long as he feels necessary in a solo piece or until the conductor cuts him off in an ensemble piece. Is the comuter going to be able to intuit from the context of the performance that the note that was held for 3 seconds was a quarter note with a fermata and not just a note held for 5 1/2 beats or however long it was? Will it know that the pause afterward should take place immediately in the music and not to insert rests?

And what about articulations? Staccato quarter notes sound much the same as eighth notes followed by eighth rests. Or possibly sixteenth notes followed by three sixteenth rests. How will the computer decide which to use? Does it matter? Is there really a difference? Well, yeah. A quarter note melody like Mary Had a Little Lamb, with each note played short, is going to read much easier as staccato quarters, since using anything else needlessly complicates things, and doesn't giver the performer (or conductor) the discretion of determining exactly how short the articulation should be. On the other hand, a complex passage requiring precise articulation would look odd with a lone staccato quarter stuck in the middle of it. A musician can use their innate feel and experience as a player to determine what would work best in any given situation. A computer doesn't have this experience to draw on.

How does traditional machine learning even begin to address these problems? One way would be to say, feed it the sheet music for Beethoven's Fifth, and then show it as many recordings of that piece as you can until it figures out that the music lines up with the notation. Then do that for every other piece of music that you can. This would be a pretty simple, straightforward way of doing things, but does anyone really think that you could generate reasonably accurate sheet music to a recording it hadn't heard, or would you just get some weird agglomeration of sheet music it already knows? After all, this method wouldn't give the computer any sense of what each individual component of the music actually does, just vaguely associate it with certain sounds. Alternatively, you could attempt to get it to recognize every note, every combination of notes, every musical instrument and combination of instruments, every stylistic device, etc. The problem here is that you're going to have to first either generate new samples or break existing music down into bite-sized pieces so that the computer can hear lone examples. But then you still have the problem that a lot of musical devices are reliant on context—what's the difference between a solo trumpet playing a middle C whole note at 100 bpm and the same instrument at the same tempo holding a quarter note of the same pitch for the exact same duration? The computer won't be able to tell unless additional context is added.

The problem with most of the AI discourse is that it's too focused on the kind of intelligence that sci-fi tropes have always talked about as the hallmarks of humanity. Getting computers to play strategy games, getting computers to talk, etc. But getting computers to transcribe accurate sheet music from mp3s isn't sexy. If a program came out that could do this, it wouldn't disrupt any economies or cost anyone their jobs, it would just be appreciated by the kinds of people who need to make arrangements of pop songs for cover bands or who want a starting point for their own arrangements, and even then it wouldn't be a game changer, it would just make things a little easier. If most people found out today that such software had been available for the past 20 years, they wouldn't think anything of it. But this software doesn't exist. And, at least to my knowledge, it won't exist for a long time, because it's not sexy and there's no immediate call for it from the marketplace. But if we are ever going to develop anything remotely approaching general AI, such a program has to exist, because general AI, by definition, doesn't exist without it. I would absolutely love a program like this, and until one is available, I'm not going to lose any sleep over AI risk.

How does traditional machine learning even begin to address these problems?

I'm just a stable diffusion hobbyist, but overcoming these challenges sounds a lot like what happens every time I load a picture into the UI and hit 'interrogate'. Currently it provides impressively accurate text descriptions but (admittedly) you can't reverse the process to replicate the original image from the text output. I'm not sure if this is harder than it looks for images*, but for music increasing the resolution from 'description of the piece' to 'full chart transcription for each instrument' seems plausible, quite possibly as a side-effect of text-to-music advances.

*Stable Diffusion's interrogation ability could probably be a lot more powerful already, but afaik it's not really a big focus area because imagegen is much sexier.

How does traditional machine learning even begin to address these problems? One way would be to say, feed it the sheet music for Beethoven's Fifth, and then show it as many recordings of that piece as you can until it figures out that the music lines up with the notation. Then do that for every other piece of music that you can. This would be a pretty simple, straightforward way of doing things, but does anyone really think that you could generate reasonably accurate sheet music to a recording it hadn't heard, or would you just get some weird agglomeration of sheet music it already knows?

Yes, I really think that. Artificial neural nets are really good at identifying higher-order structure from noisy, high-dimensional data. That's why they've had so much success at image-related tasks. All of these objections could just as easily be applied to the problem of identifying objects in a photograph:

After all, this method wouldn't give the computer any sense of what each individual component of the music actually does, just vaguely associate it with certain sounds. Alternatively, you could attempt to get it to recognize every note, every combination of notes, every musical instrument and combination of instruments, every stylistic device, etc. The problem here is that you're going to have to first either generate new samples or break existing music down into bite-sized pieces so that the computer can hear lone examples. But then you still have the problem that a lot of musical devices are reliant on context—what's the difference between a solo trumpet playing a middle C whole note at 100 bpm and the same instrument at the same tempo holding a quarter note of the same pitch for the exact same duration? The computer won't be able to tell unless additional context is added.

A cat can look completely different depending on the context in which it's photographed. Superficially, there's little in common between a close-up photo of the head of a black cat, a tabby cat lying down, a persian cat with a lime rind on its head, a cat in silhouette sitting on a fence, etc. You're telling me you can train an AI on such a messy diversity of images and it can actually learn that these are all cats, and accurately identify cats in photos it's never seen before? But yes, this is something neural nets have been able to do for a while. And they're very good at generalizing outside the range of their training data! An AI can identify a cat wearing a superman cape, or riding a snowboard, even if these are scenarios it never encountered during training.

You answered your own question as to why a good music transcription AI doesn't exist yet. There's little money or glory in it. The time of ML engineers is very expensive. And while the training process you described sounds simple, there's probably a lot of work in building a big enough labelled training corpus, and designing the architecture for a novel task.

That's what people used to say about computers playing Go, drawing pictures or translating texts. How is it going to so X? How is it going to do Y? The answer has turned out to be, "moar compute and moar data". I suspect that transcribing music will end up being solved the same way: version A manages to transcribe the simplest melodies and makes hilarious mistakes, version B makes humanlike mistakes, version C argues that your interpretation of the conductor's intent in this recording is wrong.

I don't know - maybe this is just the wrong target? ChatGPT produces slick text, but existing subtitling programs I know all have their issues with abbreviations, names and foreign words. Maybe that's just because they don't use last-generation AI. But maybe its because it is a more difficult (and more controllable) problem.

I believe the issue will be less singularity or nothing, but rather: is this good enough for purpose to make most creators unemployed?

Computers are absolutely terrible at this. There is software available that purports to do this, some of which is available online for free, some of which is built into commercial music notation software like Sibelius or Finale, and the utility of all of it is fairly limited. It can work, but only when dealing with a simple, clean melody that's reasonably in tune and played with a steady tempo. Put a normal commercial recording into it and the results range from "needs quite a bit of cleanup" to "completely unusable", and at its best it won't include stylistic markings or formatting. At first glance, this should be much easier for the computer than it is for us. We have to listen through 5 instruments playing at once to hear what the acoustic guitar, which is low in the mix to begin with, is doing underneath the big cymbal crash, and separate 2 sax parts playing simultaneously, sometimes in unison, sometimes in harmony. The computer, on the other hand, has access to the entire waveform, and can analyze every individual frequency and amplitude that's on the recording every 1/44,100th of a second.

But we are right at the start of an explosion of AI for all kinds of tasks. The past (and present) is no guide to the future, here.

And Bregman was just talking about the ability to separate instruments! A lot of transcription requires a reasonable amount of musical knowledge, but even someone who's never picked up an instrument and can't tell a C from an Eb can tell which part is the piano part and which part is the trumpet part. And then there are all the issues related to timing. Take something simple like a fermata, a symbol that instructs the musician to hold the note as long as he feels necessary in a solo piece or until the conductor cuts him off in an ensemble piece. Is the comuter going to be able to intuit from the context of the performance that the note that was held for 3 seconds was a quarter note with a fermata and not just a note held for 5 1/2 beats or however long it was? Will it know that the pause afterward should take place immediately in the music and not to insert rests?

And what about articulations? Staccato quarter notes sound much the same as eighth notes followed by eighth rests. Or possibly sixteenth notes followed by three sixteenth rests. How will the computer decide which to use? Does it matter? Is there really a difference? Well, yeah. A quarter note melody like Mary Had a Little Lamb, with each note played short, is going to read much easier as staccato quarters, since using anything else needlessly complicates things, and doesn't giver the performer (or conductor) the discretion of determining exactly how short the articulation should be. On the other hand, a complex passage requiring precise articulation would look odd with a lone staccato quarter stuck in the middle of it. A musician can use their innate feel and experience as a player to determine what would work best in any given situation. A computer doesn't have this experience to draw on.

Machine translation will vary, likely not easily matching what a random music undergrad or postgrad would transcribe. There will be three main angles, and I expect the third to win, handily. (1) is a direct reproduction of the audio input, via sampling (e.g. WAV file) (2) is vector form (not samples or pixels, but how to recreate the image, e.g. MIDI, hold this note for x seconds) (3) will be: send the music to a neural net trained to notate. This thing will have judgment, and it will be poorer initially and improve over time, given (potentially expensive) training.

The real question is, how do you train it? And there are several easy answers; maybe harder answers are more efficient

How does traditional machine learning even begin to address these problems? One way would be to say, feed it the sheet music for Beethoven's Fifth, and then show it as many recordings of that piece as you can until it figures out that the music lines up with the notation. Then do that for every other piece of music that you can. This would be a pretty simple, straightforward way of doing things, but does anyone really think that you could generate reasonably accurate sheet music to a recording it hadn't heard, or would you just get some weird agglomeration of sheet music it already knows? After all, this method wouldn't give the computer any sense of what each individual component of the music actually does, just vaguely associate it with certain sounds.

Why wouldn't it? I imagine there would be issues where it gets stuff weirdly wrong like Stable Diffusion's famously mangled hands or ChatGPT's famously incompetent arithmetic, but I'd expect such a trained model to generally get the sheet music right. No idea how it would compare to a typical real trained human musician at this - my guess would be that it would be similar to how Stable Diffusion can get you images that clearly look very close to something a human artist might draw based on the prompt, just with bizarre artifacts like the aforementioned hands (and eyes, and continuous objects disappearing/getting misaligned when hidden behind stuff, and hair blending into clothes, and clothes blending into skin, and...). I think existing machine learning tools show that the upper limit of precision and accuracy of "vaguely associate it with certain sounds" is (potentially) very high.

But if we are ever going to develop anything remotely approaching general AI, such a program has to exist, because general AI, by definition, doesn't exist without it.

General AI only requires the existence of an agent capable of creating the music-transcription program, not that the program itself actually exists. The fact that this very specific program doesn't exist yet is not very good evidence that we aren't approaching AGI. If you imagine an AI's intelligence as growing from the equivalent of a one-year-old infant to a 25-year-old programmer's over the course of a year, you don't get a smooth increase in the number of different useful programs the AI is capable of writing over the course of this year, you instead get basically zero useful programs until some critical point, and then an explosion as it becomes capable of coding all human programs.

In which case I'm even less worried. I'm unaware of any AI that has been able to completely change its functionality without additional programming. Something tells me that no matter how much I try to talk Chat GTP into becoming a MediaMonkey plugin, it's not going to happen.

Language models are fairly flexible, albeit not great because they operate at the level of language. For example you can give ChatGPT maths problems and it does adequately at them, though it sometimes makes startling errors.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats