1 00:00:00,012 --> 00:00:03,491 >> Welcome to module one of Digital Signal Processing. 2 00:00:03,491 --> 00:00:06,511 In this module we are going to see what signals actually are. 3 00:00:06,511 --> 00:00:11,274 We are going to go through a history, see the earliest example of these discrete-time signals. 4 00:00:11,274 --> 00:00:13,732 Actually it goes back to Egyptian times. 5 00:00:13,732 --> 00:00:17,276 Then through this history see how digital signals, 6 00:00:17,276 --> 00:00:21,094 for example with the telegraph signals, became important in communications. 7 00:00:21,094 --> 00:00:25,651 And today, how signals are pervasive in many applications, 8 00:00:25,651 --> 00:00:27,603 in every day life objects. 9 00:00:27,619 --> 00:00:29,914 For this we're going to see what the signal is, 10 00:00:29,914 --> 00:00:32,630 what a continuous time analog signal is, 11 00:00:32,646 --> 00:00:36,530 what a discrete-time, continuous-amplitude signal is 12 00:00:36,531 --> 00:00:42,327 and how these signals relate to each other and are used in communication devices. 13 00:00:42,327 --> 00:00:45,026 We are not going to have any math in this first module. 14 00:00:45,026 --> 00:00:49,361 It is more illustrative and the mathematics will come later in this class. 15 00:00:50,683 --> 00:00:54,722 This is an introduction to what digital signal processing is all about. 16 00:00:54,722 --> 00:00:57,952 Before getting going, let's give some background material. 17 00:00:57,952 --> 00:01:03,068 There is a textbook called Signal Processing for Communications by Paolo Prandoni and myself. 18 00:01:03,068 --> 00:01:08,170 You can have a paper version or you can get the free PDF or HTML version 19 00:01:08,170 --> 00:01:11,235 on the website here indicated on the slide. 20 00:01:11,235 --> 00:01:16,835 There will be quizzes, there will be homework sets, and there will be occasional complementary lectures. 21 00:01:16,835 --> 00:01:19,764 What is actually a signal? 22 00:01:19,764 --> 00:01:23,851 We talk about digital signal processing, so we need to define what the signal is. 23 00:01:23,851 --> 00:01:28,117 Typically, it's a description of the evolution over physical phenomenon. 24 00:01:28,117 --> 00:01:32,807 Quite simply, if I speak here, there is sound pressure waves going through the air 25 00:01:32,807 --> 00:01:36,429 that's a typical signal. When you listen to the speech, there is a 26 00:01:36,429 --> 00:01:40,350 loud speaker creating a sound pressure waves that reaches your ear. 27 00:01:40,351 --> 00:01:44,124 And that's another signal. However, in between is the world of 28 00:01:44,124 --> 00:01:48,952 digital signal processing because after the microphone it gets transformed in a 29 00:01:48,952 --> 00:01:51,876 set of members. It is processed in the computer. 30 00:01:51,876 --> 00:01:54,650 It is being transferred through the internet. 31 00:01:54,650 --> 00:01:59,236 Finally it is decoded to create the sound pressure wave to reach your ears. 32 00:01:59,237 --> 00:02:04,176 Other example are the temperature evolution over time, the magnetic 33 00:02:04,176 --> 00:02:09,405 deviation for example, L P recording , is a grey level on paper for a black and 34 00:02:09,405 --> 00:02:13,378 white photograph,some flickering colors on TV screen. 35 00:02:13,378 --> 00:02:17,504 Here we have a thermometer recording temperature over time. 36 00:02:17,504 --> 00:02:22,710 So you see the evolution And there are discrete ticks and you see how it changes 37 00:02:22,710 --> 00:02:26,266 over time. So what are the characteristics of digital 38 00:02:26,266 --> 00:02:28,973 signals. There are two key ingredients. 39 00:02:28,973 --> 00:02:33,322 First there is discrete time. As we have seen in the previous slide on 40 00:02:33,322 --> 00:02:38,354 the horizontal axis there are discrete Evenly spaced ticks and that corresponds 41 00:02:38,354 --> 00:02:42,582 to discretisation in time. There is also discrete amplitude because 42 00:02:42,582 --> 00:02:47,136 the numbers that are measured will be represented in a computer and cannot have 43 00:02:47,136 --> 00:02:51,570 some infinite precision. So what amount more sophisticated things, 44 00:02:51,570 --> 00:02:56,359 functions, derivative, and integrals. The question of discreet versus 45 00:02:56,359 --> 00:03:01,741 continuous, or analog versus discreet, goes probably back to the earliest time of 46 00:03:01,741 --> 00:03:04,742 science, for example, the school of Athens. 47 00:03:04,742 --> 00:03:09,676 There was a lot of debate between philosophers and mathematicians about the 48 00:03:09,676 --> 00:03:14,835 idea of continuum, or the difference between countable things and uncountable 49 00:03:14,835 --> 00:03:17,789 things. So in this picture, you see green are 50 00:03:17,789 --> 00:03:23,171 famous philosophers like Plato, in red, famous mathematicians like Pythagoras, 51 00:03:23,171 --> 00:03:28,007 somebody that we are going to meet again in this class, and there is a famous 52 00:03:28,007 --> 00:03:33,086 paradox which is called Zeno's paradox. So if you should narrow will it ever 53 00:03:33,086 --> 00:03:37,189 arrive in destination? We know that physics Allows us to verify 54 00:03:37,189 --> 00:03:42,301 this but mathematics have the problem with this and we can see this graphically. 55 00:03:42,301 --> 00:03:47,004 So you want to go from A to B, you cover half of the distance that is C, center 56 00:03:47,004 --> 00:03:52,240 quarters that's D and also eighth that's E etc will you ever get there and of course 57 00:03:52,240 --> 00:03:57,014 we know you gets there because the sum from 1 to infinity of 1 over 2 to the n is 58 00:03:57,014 --> 00:04:02,328 equal to 1, a beautiful formula that we'll see several times reappearing in this. 59 00:04:02,329 --> 00:04:07,286 Unfortunately during the middle ages in Europe, things were a bit lost. 60 00:04:07,286 --> 00:04:12,556 As you can see, people had other worries. In the 17th century things picked up 61 00:04:12,556 --> 00:04:15,694 again. Here we have a physicist and astronomer 62 00:04:15,694 --> 00:04:20,592 Galileo, and the philosopher Rene Descartes, and both contributed to the 63 00:04:20,592 --> 00:04:26,142 advancement of mathematics at that time. Descartes' idea was simple but powerful. 64 00:04:26,142 --> 00:04:30,468 Start with a point, put it into a co-ordinate system Then put more 65 00:04:30,468 --> 00:04:34,456 sophisticated things like lines, and you can use algebra. 66 00:04:34,456 --> 00:04:39,313 This led to the idea of calculus, which allowed to mathematically describe 67 00:04:39,313 --> 00:04:43,758 physical phenomenon. For example Galileo was able to describe 68 00:04:43,758 --> 00:04:49,806 the trajectory of a bullet, using infinite decimal variations in both horizontal and 69 00:04:49,806 --> 00:04:54,362 vertical direction. Calculus itself was formalized by Newton 70 00:04:54,362 --> 00:04:59,570 and Leibniz, and is one of the great advances of mathematics in the 17th and 71 00:04:59,570 --> 00:05:02,768 18th century. It is time to do some very simple 72 00:05:02,768 --> 00:05:07,810 continuous time signal processing. We have a function in blue here, between a 73 00:05:07,810 --> 00:05:10,796 and b, and we would like to compute it's average. 74 00:05:10,796 --> 00:05:15,464 As it is well known, this well be the integral of the function, divided by the 75 00:05:15,464 --> 00:05:19,033 length's of the interval, and it is shown here in red dots. 76 00:05:19,033 --> 00:05:23,487 What would be the equivalent in this discreet time symbol processing. 77 00:05:23,487 --> 00:05:27,170 We have a set of samples between say, 0 and capital N minus 1. 78 00:05:27,170 --> 00:05:32,441 The average is simply 1 over n, the sum Was the antidote terms x[n] between 0 and 79 00:05:32,441 --> 00:05:36,043 N minus 1. Again, it is shown in the red dotted line. 80 00:05:36,043 --> 00:05:41,543 In this case, because the signal is very smooth, the continuous time average and 81 00:05:41,543 --> 00:05:45,179 the discrete time average Are essentially the same. 82 00:05:45,179 --> 00:05:49,925 This was nice and easy but what if the signal is too fast, and we don't know 83 00:05:49,925 --> 00:05:54,349 exactly how to compute either the continuous time operations or an 84 00:05:54,349 --> 00:05:59,573 equivalent operation on samples. Enters Joseph Fourier, one of the greatest 85 00:05:59,573 --> 00:06:04,772 mathematicians of the nineteenth century. And the inventor of Fourier series, 86 00:06:04,772 --> 00:06:09,677 Fourier analysis which are essentially the ground tools of signal processing. 87 00:06:09,677 --> 00:06:13,470 We show simply a picture to give the idea of Fourier analysis. 88 00:06:13,470 --> 00:06:17,990 It is a local Fourier spectrum as you would see for example on an equalizer 89 00:06:17,990 --> 00:06:21,540 table in a disco. And it shows the distribution of power 90 00:06:21,540 --> 00:06:26,910 across frequencies, something we are going to understand in detail in this class. 91 00:06:26,910 --> 00:06:31,767 But to do this quick time processing of continuous time signals we need some 92 00:06:31,767 --> 00:06:35,498 further results. And these were derived by Harry Niquist 93 00:06:35,498 --> 00:06:38,776 and Claude Shannon, two researchers at Bell Labs. 94 00:06:38,776 --> 00:06:44,606 They derived the so-called sampling theorem, first appearing in 1920's and 95 00:06:44,606 --> 00:06:48,977 formalized in 1948. If the function X of T is sufficiently 96 00:06:48,977 --> 00:06:54,827 slow then there is a simple interpolation formula for X of T, it's the sum of the 97 00:06:54,827 --> 00:07:00,609 samples Xn, Interpolating with the function that is called sync function. 98 00:07:00,609 --> 00:07:05,707 It looks a little but complicated now, but it's something we're going to study in 99 00:07:05,707 --> 00:07:10,723 great detail because it's 1 of the fundamental formulas linking this discrete 100 00:07:10,723 --> 00:07:13,625 time and continuous time signal processing. 101 00:07:13,625 --> 00:07:18,720 Let us look at this sampling in action. So we have the blue curve, we take 102 00:07:18,720 --> 00:07:24,266 samples, the red dots from the samples. We use the same interpolation. 103 00:07:24,266 --> 00:07:28,720 We put one blue curve, second one, third one, fourth one, etc. 104 00:07:28,720 --> 00:07:33,488 When we sum them all together, we get back the original blue curve. 105 00:07:33,488 --> 00:07:37,000 It is magic. This interaction of continuous time and 106 00:07:37,000 --> 00:07:41,140 discrete time processing is summarized in these two pictures. 107 00:07:41,140 --> 00:07:44,474 On the left you have a picture of the analog world. 108 00:07:44,474 --> 00:07:49,394 On the right you have a picture of the discrete or digital world, as you would 109 00:07:49,394 --> 00:07:54,415 see in a Digital camera for example, and this is because the world is analog. 110 00:07:54,415 --> 00:07:58,693 It has continuous time continuous space, and the computer is digital. 111 00:07:58,693 --> 00:08:03,476 It is discreet time discreet temperature. When you look at an image taken with a 112 00:08:03,476 --> 00:08:06,919 digital camera, you may wonder what the resolution is. 113 00:08:06,919 --> 00:08:11,685 And here we have a picture of a bird. This bird happens to have very high visual 114 00:08:11,685 --> 00:08:16,479 acuity, probably much better than mine. Still, if you zoom into the digital 115 00:08:16,479 --> 00:08:21,693 picture, after a while, around the eye here, you see little squares appearing, 116 00:08:21,693 --> 00:08:27,381 showing indeed that the picture is digital Because discrete values over the domain of 117 00:08:27,381 --> 00:08:32,358 the image and it also has actually discrete amplitude which we cannot quite 118 00:08:32,358 --> 00:08:37,234 see here at this level of resolution. As we said the key ingredients are 119 00:08:37,234 --> 00:08:41,516 discrete time and discrete amplitude for digital signals. 120 00:08:41,516 --> 00:08:46,755 So, let us look at x of t here. It's a sinusoid, and investigate discrete 121 00:08:46,755 --> 00:08:49,926 time first. We see this with xn and discrete 122 00:08:49,926 --> 00:08:53,325 amplitude. We see this with these levels of the 123 00:08:53,325 --> 00:08:59,228 amplitudes which are also discrete ties. And so this signal looks very different 124 00:08:59,228 --> 00:09:02,722 from the original continuous time signal x of t. 125 00:09:02,722 --> 00:09:08,308 It has discrete values on the time axes and discrete values on the vertical 126 00:09:08,308 --> 00:09:11,903 amplitude axis. So why do we need digital amplitude? 127 00:09:11,903 --> 00:09:16,727 Well, because storage is digital, because processing is digital, and because 128 00:09:16,727 --> 00:09:20,787 transmission is digital. And you are going to see all of these in 129 00:09:20,787 --> 00:09:23,966 sequence. So data storage, which is of course very 130 00:09:23,966 --> 00:09:27,718 important, used to be purely analog. You had paper. 131 00:09:27,718 --> 00:09:30,764 You had wax cylinders. You had vinyl. 132 00:09:30,764 --> 00:09:37,736 You had compact cassettes, VHS, etcetera. In imagery you had Kodachrome, slides, 133 00:09:37,736 --> 00:09:42,748 Super 8, film etc. Very complicated, a whole biodiversity of 134 00:09:42,748 --> 00:09:46,195 analog storages. In digital, much simpler. 135 00:09:46,195 --> 00:09:51,271 There is only zeros and ones, so all digital storage, to some extent, looks the 136 00:09:51,271 --> 00:09:53,974 same. The storage medium might look very 137 00:09:53,974 --> 00:09:58,795 different, so here we have a collection of storage from the last 25 years. 138 00:09:58,796 --> 00:10:03,764 However, fundamentally there are only 0's and 1's on these storage devices. 139 00:10:03,764 --> 00:10:07,641 So in that sense, they are all compatible with each other. 140 00:10:07,641 --> 00:10:10,727 Processing also moved from analog to digital. 141 00:10:10,727 --> 00:10:15,825 On the left side, you have a few examples of analog processing devices, an analog 142 00:10:15,825 --> 00:10:19,975 watch, an analog amplifier. On the right side you have a piece of 143 00:10:19,975 --> 00:10:22,797 code. Now this piece of code could run on many 144 00:10:22,797 --> 00:10:27,076 different digital computers. It would be compatible with all these 145 00:10:27,076 --> 00:10:30,422 digital platforms. The analog processing devices Are 146 00:10:30,422 --> 00:10:35,216 essentially incompatible with each other. Data transmission has also gone from 147 00:10:35,216 --> 00:10:38,435 analog to digital. So lets look at the very simple model 148 00:10:38,435 --> 00:10:42,705 here, you've on the left side of the transmitter, you have a channel on the 149 00:10:42,705 --> 00:10:46,969 right side you have a receiver. What happens to analog signals when they 150 00:10:46,969 --> 00:10:50,574 are send over a channel. So x of t goes through the channel, its 151 00:10:50,574 --> 00:10:55,404 first multiplied by 1 over G because there is path loss and then there is noise added 152 00:10:55,404 --> 00:10:59,833 indicated here with the sigma of t. The output here is x hat of t. 153 00:10:59,833 --> 00:11:04,318 Let's start with some analog signal x of t. 154 00:11:04,318 --> 00:11:08,853 Multiply it by 1 over g, and add some noise. 155 00:11:08,853 --> 00:11:13,751 How do we recover a good reproduction of x of t? 156 00:11:13,751 --> 00:11:22,191 Well, we can compensate for the path loss, so we multiply by g, to get xhat 1 of t. 157 00:11:22,191 --> 00:11:26,833 But the problem is that x1 hat of t, is x of t. 158 00:11:26,834 --> 00:11:34,917 That's the good news plus g times sigma of t so the noise has been amplified. 159 00:11:34,917 --> 00:11:41,455 Let's see this in action. We start with x of t, we scale by G, we 160 00:11:41,455 --> 00:11:47,898 add some noise, we multiply by G. And indeed now, we have a very noisy 161 00:11:47,898 --> 00:11:51,605 signal. This was the idea behind trans-Atlantic 162 00:11:51,605 --> 00:11:57,924 cables which were laid in the 19th century and were essentially analog devices until 163 00:11:57,924 --> 00:12:02,575 telegraph signals were properly encoded as digital signals. 164 00:12:02,575 --> 00:12:08,225 As can be seen in this picture, this was quite an adventure to lay a cable across 165 00:12:08,225 --> 00:12:13,620 the Atlantic and then to try to transmit analog signals across these very long 166 00:12:13,620 --> 00:12:17,549 distances. For a long channel because the path loss 167 00:12:17,549 --> 00:12:23,317 is so big, you need to put repeaters. So the process we have just seen, would be 168 00:12:23,317 --> 00:12:27,717 repeated capital N times. Each time the paths loss would be 169 00:12:27,717 --> 00:12:32,307 compensated, but the noise will be amplified by a factor of n. 170 00:12:32,307 --> 00:12:38,108 Let us see this in action, so start with x of t, paths loss by g, added noise, 171 00:12:38,108 --> 00:12:44,663 amplification by G with the amplification the amplification of the noise, and the 172 00:12:44,663 --> 00:12:48,580 signal. For the second segment we have the pass 173 00:12:48,580 --> 00:12:54,629 loss again, so X hat 1 is divided by G. And added noise, then we amplify to get x 174 00:12:54,629 --> 00:12:59,751 hat 2 of t, which now has twice an amount of noise, 2 g times signal of t. 175 00:12:59,751 --> 00:13:05,151 So, if we do this n times, you can see that the analog signal, after repeated 176 00:13:05,151 --> 00:13:07,472 amplification. Is mostly noise. 177 00:13:07,472 --> 00:13:10,680 And that becomes problematic to transmit information. 178 00:13:10,680 --> 00:13:13,810 In digital communication, the physics do not change. 179 00:13:13,810 --> 00:13:16,606 We have the same path loss, we have added noise. 180 00:13:16,606 --> 00:13:20,299 However, two things change. One is that we don't send arbitrary 181 00:13:20,299 --> 00:13:23,840 signals but, for example, only signals that[INAUDIBLE]. 182 00:13:23,840 --> 00:13:29,936 Take values plus 1 and minus 1, and we do some specific processing to recover these 183 00:13:29,936 --> 00:13:33,211 signals. Specifically at the outward of the 184 00:13:33,211 --> 00:13:37,902 channel, we multiply by g, and then we take the signa operation. 185 00:13:37,902 --> 00:13:41,689 So x1hat, is signa of x of t, plug g times sigma of t. 186 00:13:41,689 --> 00:13:46,998 Let us again look at this in action. We start with the signal x of t that is 187 00:13:46,998 --> 00:13:49,362 easier, plus 5 or minus 5. 5. 188 00:13:49,362 --> 00:13:55,218 It goes through the channel, so it loses amplitude by a factor of g, and their is 189 00:13:55,218 --> 00:13:59,222 some noise added. We multiply by g, so we recover x of t 190 00:13:59,222 --> 00:14:04,639 plus g times the noise of sigma t. Then we apply the threshold operation. 191 00:14:04,639 --> 00:14:10,675 And true enough, we recover a plus 5 minus 5 signal, which is identical to the ones 192 00:14:10,675 --> 00:14:15,076 that was sent on the channel. Thanks to digital processing the 193 00:14:15,076 --> 00:14:18,881 transmission of information has made tremendous progress. 194 00:14:18,881 --> 00:14:23,549 In the mid nineteenth century a transatlantic cable would transmit 8 words 195 00:14:23,549 --> 00:14:26,361 per minute. That's about 5 bits per second. 196 00:14:26,361 --> 00:14:30,338 A hundred years later a coaxial cable with 48 voice channels. 197 00:14:30,339 --> 00:14:36,439 At already 3 megabits per second. In 2005, fiber optic technology allowed 10 198 00:14:36,439 --> 00:14:41,316 terabits per second. A terabit is 10 to the 12 bits per second. 199 00:14:41,316 --> 00:14:47,444 And today, in 2012, we have fiber cables with 60 terabits per second. 200 00:14:47,444 --> 00:14:52,862 On the voice channel, the one that is used for telephony, in 1950s you could send 201 00:14:52,862 --> 00:14:56,649 1200 bits per second. In the 1990's, that was already 56 202 00:14:56,649 --> 00:15:00,559 kilobits per second. Today, with ADSL technology, we are 203 00:15:00,559 --> 00:15:05,936 talking about 24 megabits per second. Please note that the last module in the 204 00:15:05,936 --> 00:15:11,716 class will actually explain how ADSL The works using all the tricks in the box that 205 00:15:11,716 --> 00:15:16,797 we are learning in this class. It is time to conclude this introductory 206 00:15:16,797 --> 00:15:19,661 module. And we conclude with a picture. 207 00:15:19,661 --> 00:15:25,035 If you zoom into this picture you see it's the motto of the class, signal is 208 00:15:25,035 --> 00:15:25,890 strength.