W01_L02_P05 - Signal detection and thresholding

And you might say, wait a minute.. Let's,
let's make this a little stronger. Let's

make the noise a little stronger and see
how well this does. Let's go up to height

like, if I went back to twenty, double the
noise. Every realization of noise by every

time I push that run button, gives me
something different. You guys all realize

that? Every time I do that, it generates a
new noise field. Here is another

realization. Look at that. Pretty clear.
What's here is still just a mess. You

wouldn't be able to make a detection. Here
it says, something is clearly above that

line. Now, you would say, if you'd just
looked at this, you say, well, wait a

minute, How do I know that if I don't just
move the filter over here I'm just not

going to grab this stuff and it's not
going to give me something, right? You

could say that cuz maybe you're
distrustful right now. You don't trust me.

You think I'm trying to pull something on
you like oh, that guy, he's a fraud. Look

at that, I could of done that with
anything. Maybe you're, hopefully you're

not thinking of that. But, but seriously,
if you look at this data in the spectrum,

it doesn't really look any different here
than here than here than here than here.

And it looks to me like if you filtered
there, you'd get the same thing, right?

What's the difference actually with a
signal versus noise? In a signal, the

information is coherent. The frequency
components are in phase okay? In white

noise, yes, this is lots of fluctuation,
lots of energy. All this information over

here, I don't apply the absolute value.
This stuff is incoherent. In other words,

it has random phase relationships. So,
when I, I'm going to show you, we're going

to move the filter over to here, and I'm
going to prove to you that I'm not fibbing

you, okay? I went to the side, looked at
the same plot, move the filter, okay? By

the way, the filtered signal, so in other
words, if I look at the frequency domain,

if I take this blue filter and move it
over here, you see that green line? It's

going to look pretty close to the same.
But the key is what happens in the time

domain. Alright. So, all you got to do to
move the filter let's go back over here

and let's maybe make it centered around
-ten.

That sound good? Or I don't know, fifteen?
Let's, let's make fifteen. Sure. Oh, there

we go. I just moved my filter. I can do
either one. I mean, I could just filter

anywhere I want. I could go to one side or
the other filter or both. But I'll just

show you what happen. What we want to look
at is a bi-filter around that frequency.

So, in other words, omit, where this is
zero is at a certain frequency, omega

zero.
And if you go to the one side, you're

going to higher frequencies. The other
side, lower frequencies. So now, I'm just

moving to lower frequencies. But zero
frequency is the middle, right? Yeah. And

so. Well it. From zero, that's the rangeof
frequency. So, if you go to the negative,

your proof is probably the same. Yeah. But
it's not really zero, its omega zero,

right, rescaled. You've centered out,
you've taken out the center frequency.

Okay. So, okay, here is what I get
alright. It's not as impressive as you

think. I mean, it is more impressive than
you think. . what's it go from? Sorry,

sorry. I know it's just that form here
that oops, to show you and take it away, I

mean, its like a tuple. Okay. zero, one.
There, I filtered over that. The reason I

did it, it rescales automatically. I
wanted you to see that threshold line. The

threshold line is there, 0.5. It's always
been there. If I take that same look at

here, the green under here still has, you
know, that's a spike. You think oh, a

spike lead to a spike here. The spike
there doesn't lead to a spike there. Spike

there leads to stuff. That's slow. You
would never launch a missile there, okay?

So, what this allowed us to do, right, is
a very simple algorithm where you just

say, look, I want to do data analysis. And
so, this gets back to the bigger picture

generically, which is, what we did here is
just said, look if I want to do data

analysis and I'm bringing in all this
informati on, it's contaminated with

noise. But I know something about what I'm
looking for. I knew here I was looking for

omega zero.
I'll just say okay, filter around omega

zero.
Take out a little Gaussian. And by the

way, you can do better, you can do worse.
This is why people work so hard in filter

design because I can make super fancy
filters that would do better and better

and better job without really making a
detection. Remember, radar actually, well,

I don't know if you, I didn't say anything
about it but, radar, advanced radar was

actually tremendous. It was at the end of
World War II, actually. And it was a

turning point in the war cuz Hitler was
bombing England into submission. And they

were actually probably a few months out
from basically an unconditional surrender.

But then, what was happening in technology
there is they actually built radar an all

of a sudden, Luftwaffe, which was
basically their dominant force coming in

over England. The English Air Forces
always, always knew where they were, how

did, could that happen, they could've
invented radar, the Germans didn't know

it, it totally changed the war. Pretty
important and since that time the

development of radar has been huge. And
people worked very hard at this. for

instance, MIT Lincoln Labs. It's, they do
radar stuff all the time, they have three

positions open if you need one right now.
so, for instance, this, so you can do a

lot of advanced design, filtering, taking
in account balancing of the signal all

around from things. I mean this is but
it's ultimately, it's just, it's just

this, this is the key idea right there.
This is the key idea. What they want to do

is track crap from real signal, okay?
That's the ultimate goal. And ultimately,

what you want to do, whatever your data
is, you'd like to take out what shouldn't

be there and you want an accurate,
statistically correct way to do that,

okay? So, this is the kind of mechanism we
start looking at. Now, there are other

ways to subtract out noise, I want to talk
about that on Monday, okay? We'll build

some more code. Thing I'd encourage you to
do. Take the code here. You can play with

filter width, right? There's a whole code
in the notes. You could just start playing

with filter width, you can start playing
with your filter design if you'd like,

right? It's very simple to do. How many
lines? We just did it in line here. It's

not that many lines. You just start
looking at what happens to your signal.

Play around with your noise. Play around
with your signal you put in. See how well

you can reconstruct it.