New ways to see music (with color! and fire!)
-
0:00 - 0:01My passions
-
0:01 - 0:05are music, technology and making things.
-
0:05 - 0:08And it's the combination of these things
-
0:08 - 0:11that has led me to the hobby of sound visualization,
-
0:11 - 0:15and, on occasion, has led me to play with fire.
-
0:15 - 0:18This is a Rubens' tube. It's one of many I've made over the years,
-
0:18 - 0:19and I have one here tonight.
-
0:19 - 0:21It's about an 8-foot-long tube of metal,
-
0:21 - 0:22it's got a hundred or so holes on top,
-
0:22 - 0:24on that side is the speaker, and here
-
0:24 - 0:26is some lab tubing, and it's connected to this tank
-
0:26 - 0:28of propane.
-
0:29 - 0:32So, let's fire it up and see what it does.
-
0:38 - 0:40So let's play a 550-herz frequency
-
0:40 - 0:41and watch what happens.
-
0:41 - 0:50(Frequency)
-
0:50 - 0:53Thank you. (Applause)
-
0:53 - 0:55It's okay to applaud the laws of physics,
-
0:55 - 0:56but essentially what's happening here
-
0:56 - 0:58-- (Laughter) --
-
0:58 - 1:02is the energy from the sound via the air and gas molecules
-
1:02 - 1:04is influencing the combustion properties of propane,
-
1:04 - 1:06creating a visible waveform,
-
1:06 - 1:08and we can see the alternating regions of compression
-
1:08 - 1:11and rarefaction that we call frequency,
-
1:11 - 1:12and the height is showing us amplitude.
-
1:12 - 1:15So let's change the frequency of the sound,
-
1:15 - 1:16and watch what happens to the fire.
-
1:16 - 1:26(Higher frequency)
-
1:26 - 1:29So every time we hit a resonant frequency we get a standing wave
-
1:29 - 1:31and that emergent sine curve of fire.
-
1:31 - 1:33So let's turn that off. We're indoors.
-
1:33 - 1:38Thank you. (Applause)
-
1:38 - 1:41I also have with me a flame table.
-
1:41 - 1:42It's very similar to a Rubens' tube, and it's also used
-
1:42 - 1:44for visualizing the physical properties of sound,
-
1:44 - 1:46such as eigenmodes, so let's fire it up
-
1:46 - 1:49and see what it does.
-
1:52 - 1:57Ooh. (Laughter)
-
1:57 - 2:00Okay. Now, while the table comes up to pressure,
-
2:00 - 2:01let me note here that the sound is not traveling
-
2:01 - 2:04in perfect lines. It's actually traveling in all directions,
-
2:04 - 2:07and the Rubens' tube's a little like bisecting those waves
-
2:07 - 2:09with a line, and the flame table's a little like
-
2:09 - 2:11bisecting those waves with a plane,
-
2:11 - 2:15and it can show a little more subtle complexity, which is why
-
2:15 - 2:17I like to use it to watch Geoff Farina play guitar.
-
2:17 - 3:00(Music)
-
3:00 - 3:02All right, so it's a delicate dance.
-
3:02 - 3:04If you watch closely — (Applause)
-
3:04 - 3:07If you watch closely, you may have seen
-
3:07 - 3:09some of the eigenmodes, but also you may have seen
-
3:09 - 3:14that jazz music is better with fire.
-
3:14 - 3:16Actually, a lot of things are better with fire in my world,
-
3:16 - 3:18but the fire's just a foundation.
-
3:18 - 3:20It shows very well that eyes can hear,
-
3:20 - 3:21and this is interesting to me because
-
3:21 - 3:24technology allows us to present sound to the eyes
-
3:24 - 3:27in ways that accentuate the strength of the eyes
-
3:27 - 3:29for seeing sound, such as the removal of time.
-
3:29 - 3:33So here, I'm using a rendering algorithm to paint
-
3:33 - 3:35the frequencies of the song "Smells Like Teen Spirit"
-
3:35 - 3:37in a way that the eyes can take them in
-
3:37 - 3:39as a single visual impression, and the technique
-
3:39 - 3:41will also show the strengths of the visual cortex
-
3:41 - 3:43for pattern recognition.
-
3:43 - 3:45So if I show you another song off this album,
-
3:45 - 3:48and another, your eyes will easily pick out
-
3:48 - 3:51the use of repetition by the band Nirvana,
-
3:51 - 3:53and in the frequency distribution, the colors,
-
3:53 - 3:56you can see the clean-dirty-clean sound
-
3:56 - 3:57that they are famous for,
-
3:57 - 4:01and here is the entire album as a single visual impression,
-
4:01 - 4:03and I think this impression is pretty powerful.
-
4:03 - 4:05At least, it's powerful enough that
-
4:05 - 4:06if I show you these four songs,
-
4:06 - 4:09and I remind you that this is "Smells Like Teen Spirit,"
-
4:09 - 4:11you can probably correctly guess, without listening
-
4:11 - 4:13to any music at all, that the song
-
4:13 - 4:15a die hard Nirvana fan would enjoy is this song,
-
4:15 - 4:17"I'll Stick Around" by the Foo Fighters,
-
4:17 - 4:19whose lead singer is Dave Grohl,
-
4:19 - 4:23who was the drummer in Nirvana.
-
4:23 - 4:24The songs are a little similar, but mostly
-
4:24 - 4:26I'm just interested in the idea that someday maybe
-
4:26 - 4:30we'll buy a song because we like the way it looks.
-
4:30 - 4:31All right, now for some more sound data.
-
4:31 - 4:34This is data from a skate park,
-
4:34 - 4:36and this is Mabel Davis skate park
-
4:36 - 4:38in Austin, Texas. (Skateboard sounds)
-
4:38 - 4:40And the sounds you're hearing came from eight
-
4:40 - 4:42microphones attached to obstacles around the park,
-
4:42 - 4:44and it sounds like chaos, but actually
-
4:44 - 4:47all the tricks start with a very distinct slap,
-
4:47 - 4:49but successful tricks end with a pop,
-
4:49 - 4:51whereas unsuccessful tricks
-
4:51 - 4:53more of a scratch and a tumble,
-
4:53 - 4:57and tricks on the rail will ring out like a gong, and
-
4:57 - 4:59voices occupy very unique frequencies in the skate park.
-
4:59 - 5:01So if we were to render these sounds visually,
-
5:01 - 5:03we might end up with something like this.
-
5:03 - 5:05This is all 40 minutes of the recording,
-
5:05 - 5:07and right away the algorithm tells us
-
5:07 - 5:09a lot more tricks are missed than are made,
-
5:09 - 5:12and also a trick on the rails is a lot more likely
-
5:12 - 5:15to produce a cheer, and if you look really closely,
-
5:15 - 5:16we can tease out traffic patterns.
-
5:16 - 5:22You see the skaters often trick in this direction. The obstacles are easier.
-
5:22 - 5:24And in the middle of the recording, the mics pick this up,
-
5:24 - 5:27but later in the recording, this kid shows up,
-
5:27 - 5:30and he starts using a line at the top of the park
-
5:30 - 5:32to do some very advanced tricks on something
-
5:32 - 5:33called the tall rail.
-
5:33 - 5:35And it's fascinating. At this moment in time,
-
5:35 - 5:38all the rest of the skaters turn their lines 90 degrees
-
5:38 - 5:40to stay out of his way.
-
5:40 - 5:42You see, there's a subtle etiquette in the skate park,
-
5:42 - 5:44and it's led by key influencers,
-
5:44 - 5:47and they tend to be the kids who can do the best tricks,
-
5:47 - 5:50or wear red pants, and on this day the mics picked that up.
-
5:50 - 5:54All right, from skate physics to theoretical physics.
-
5:54 - 5:55I'm a big fan of Stephen Hawking,
-
5:55 - 5:57and I wanted to use all eight hours
-
5:57 - 5:59of his Cambridge lecture series to create an homage.
-
5:59 - 6:02Now, in this series he's speaking with the aid of a computer,
-
6:02 - 6:05which actually makes identifying the ends of sentences
-
6:05 - 6:09fairly easy. So I wrote a steering algorithm.
-
6:09 - 6:11It listens to the lecture, and then it uses
-
6:11 - 6:13the amplitude of each word to move a point on the x-axis,
-
6:13 - 6:16and it uses the inflection of sentences
-
6:16 - 6:18to move a same point up and down on the y-axis.
-
6:18 - 6:21And these trend lines, you can see, there's more questions
-
6:21 - 6:23than answers in the laws of physics,
-
6:23 - 6:25and when we reach the end of a sentence,
-
6:25 - 6:27we place a star at that location.
-
6:27 - 6:30So there's a lot of sentences, so a lot of stars,
-
6:30 - 6:32and after rendering all of the audio, this is what we get.
-
6:32 - 6:35This is Stephen Hawking's universe.
-
6:35 - 6:42(Applause)
-
6:42 - 6:45It's all eight hours of the Cambridge lecture series
-
6:45 - 6:47taken in as a single visual impression,
-
6:47 - 6:48and I really like this image,
-
6:48 - 6:50but a lot of people think it's fake.
-
6:50 - 6:52So I made a more interactive version,
-
6:52 - 6:58and the way I did that is I used their position in time
-
6:58 - 7:00in the lecture to place these stars into 3D space,
-
7:00 - 7:03and with some custom software and a Kinect,
-
7:03 - 7:05I can walk right into the lecture.
-
7:05 - 7:07I'm going to wave through the Kinect here
-
7:07 - 7:09and take control, and now I'm going to reach out
-
7:09 - 7:12and I'm going to touch a star, and when I do,
-
7:12 - 7:14it will play the sentence
-
7:14 - 7:16that generated that star.
-
7:16 - 7:19Stephen Hawking: There is one, and only one, arrangement
-
7:19 - 7:22in which the pieces make a complete picture.
-
7:22 - 7:26Jared Ficklin: Thank you. (Applause)
-
7:26 - 7:30There are 1,400 stars.
-
7:30 - 7:31It's a really fun way to explore the lecture,
-
7:31 - 7:33and, I hope, a fitting homage.
-
7:33 - 7:38All right. Let me close with a work in progress.
-
7:38 - 7:41I think, after 30 years, the opportunity exists
-
7:41 - 7:43to create an enhanced version of closed captioning.
-
7:43 - 7:45Now, we've all seen a lot of TEDTalks online,
-
7:45 - 7:48so let's watch one now with the sound turned off
-
7:48 - 7:52and the closed captioning turned on.
-
7:52 - 7:54There's no closed captioning for the TED theme song,
-
7:54 - 7:56and we're missing it, but if you've watched enough of these,
-
7:56 - 7:58you hear it in your mind's ear,
-
7:58 - 8:01and then applause starts.
-
8:01 - 8:03It usually begins here, and it grows and then it falls.
-
8:03 - 8:05Sometimes you get a little star applause,
-
8:05 - 8:07and then I think even Bill Gates takes a nervous breath,
-
8:07 - 8:09and the talk begins.
-
8:09 - 8:15All right, so let's watch this clip again.
-
8:15 - 8:16This time, I'm not going to talk at all.
-
8:16 - 8:17There's still going to be no audio,
-
8:17 - 8:19but what I am going to do is I'm going to render the sound
-
8:19 - 8:24visually in real time at the bottom of the screen.
-
8:24 - 8:26So watch closely and see what your eyes can hear.
-
8:48 - 8:50This is fairly amazing to me.
-
8:50 - 8:53Even on the first view, your eyes will successfully
-
8:53 - 8:56pick out patterns, but on repeated views,
-
8:56 - 8:58your brain actually gets better
-
8:58 - 9:00at turning these patterns into information.
-
9:00 - 9:01You can get the tone and the timbre
-
9:01 - 9:02and the pace of the speech,
-
9:02 - 9:04things that you can't get out of closed captioning.
-
9:04 - 9:07That famous scene in horror movies
-
9:07 - 9:09where someone is walking up from behind
-
9:09 - 9:11is something you can see,
-
9:11 - 9:14and I believe this information would be something
-
9:14 - 9:17that is useful at times when the audio is turned off
-
9:17 - 9:20or not heard at all, and I speculate that deaf audiences
-
9:20 - 9:21might actually even be better
-
9:21 - 9:23at seeing sound than hearing audiences.
-
9:23 - 9:24I don't know. It's a theory right now.
-
9:24 - 9:26Actually, it's all just an idea.
-
9:26 - 9:30And let me end by saying that sound moves in all directions,
-
9:30 - 9:32and so do ideas.
-
9:32 - 9:35Thank you. (Applause)
- Title:
- New ways to see music (with color! and fire!)
- Speaker:
- Jared Ficklin
- Description:
-
Designer Jared Ficklin creates wild visualizations that let us see music, using color and even fire (a first for the TED stage) to analyze how sound makes us feel. He takes a brief digression to analyze the sound of a skatepark -- and how audio can clue us in to developing creativity.
- Video Language:
- English
- Team:
- closed TED
- Project:
- TEDTalks
- Duration:
- 10:00
Morton Bast edited English subtitles for New ways to see music (with color! and fire!) | ||
Morton Bast edited English subtitles for New ways to see music (with color! and fire!) | ||
Jenny Zurawell edited English subtitles for New ways to see music (with color! and fire!) | ||
Thu-Huong Ha approved English subtitles for New ways to see music (with color! and fire!) | ||
Thu-Huong Ha accepted English subtitles for New ways to see music (with color! and fire!) | ||
Thu-Huong Ha edited English subtitles for New ways to see music (with color! and fire!) | ||
Morton Bast edited English subtitles for New ways to see music (with color! and fire!) | ||
Joseph Geni edited English subtitles for New ways to see music (with color! and fire!) |