10.2 - Some research projects that use techniques you learned here

From the official description of 10.. videos:

Goodbye!

As a parting message, we prepared an extra Module (yes, a "bonus feature", just like in DVDs!) with the following purposes:

give you some pointers if you want to learn more about signal processing
show you some of the cool research topics in signal processing that are currently being pursued in our lab
show you how signal processing translates to the real world by introducing several startups founded by current and former members of our lab

Module 10.2.
Some research projects that use the

techniques you have learned in the
digital signal processing class.

We're going to talk about some current
research in the lab.

There is a whole slew of them, and it's a
selection here of interesting research

projects that we can briefly discuss.
The first one is, eFacsimile is a project

on art work acquisition.
The second one is about signal processing

in sensor networks.
Then there is a network science result on

source localization put in graphs.
Then we talk about sampling result, so

called finite rate of innovation
sampling.

Then we talk again about sampling, that
of physical fields, using some new

techniques for sampling.
Then, we have a project on image

acquisition where we change the sensors
used in acquiring images.

Then, an old classic, predicting the
stock market.

Then, we talk about inverse problems.
The three next projects are actually

inverse problems.
The first one is on the diffusion

equation, the second one is trying to
understand the nuclear fall out from

Fukushima, and last but not least is an
inverse problem in acoustics.

The eFacsimile project.
This is a project that we do together

with Google to try to improve how artwork
is represented on the internet.

it's lead by [INAUDIBLE] researcher Loic
Baboulaz and several PhD students are

involved in this.
The questions are how to capture,

represent, and render artwork as well as
possible.

And to do this, we need some advanced
techniques on relighting, manipulation

of, the so called, light fields that is
acquired, and potentially high resolution

solutions for mobile devices.
There are some demos online that I

encourage you to actually watch.
because this really doesn't show the

idea.
This is of course a static version.

But for example one of the demos is you
take a, an oil painting here.

And you acquire it in such a way that if
you show it on a tablet and you move it,

it will actually exactly look like the
original oil painting.

So you get the illusion that you have
actually the oil painting in the hand.

So if the light changes, the vis,
visualization will change.

If you turn the tablet, the visualization
will change.

so then, what is quite stunning and I
suggest you actually watch it, similarly,

there is another demo which deals with
stain glasses.

So stain glasses are very interesting art
objects, but very difficult to render on

the internet.
And so here we will stimulate the stained

glass, so if you have a tablet in your
hand and you move it, it looks like if

you had, stained glass in your hand.
So the tools we use is, we use

traditional cameras, but we also use
so-called light field cameras.

You might have heard of the light
[INAUDIBLE] for example.

That's a new generation of camera.
It's extremely interesting.

And so we need to fully understand light
transport theory and [INAUDIBLE].

Which uses sparse recovery methods or
compress sensing.

The website of the project is given here.
And as I indicated there is YouTube demo

that shows quite realistically the demos
that we're discussed just in a minute

ago.
So, next project is about wireless sensor

networks, in particular about monitoring
visually in a wireless sensor network.

So, sensor networks have deployed for
many years.

We have large projects here, in the lab,
on environmental monitoring.

And the current generation is actually
equipped with camera, and then you have a

problem of compression, of
representation.

So even though the trend is towards
smaller and smaller devices, they are

still power hungry and in particular if
you have a sophisticated camera, the

number of images or the number of pixels
that is generated might actually overwelm

the power budget of the system.
And so Dr.

Zichong Chen, who finished his PhD here,
and did his post doc, together with

Guillermo Barrenetxea, are looking at
creating large scale sensor networks

equipped with cameras that are energy
efficient.

So why do we want images?
Well, here are a few examples.

This is from I think a Berkeley project
about, monitoring birds' nest.

Unfortunately a snake is showing up an
he's actually eating all the eggs in the

nest.
So that's, monitoring for why life

protection.
Here is an example from the Swiss Alps

monitoring for for avalanche detection.
Here is also an example from the Swiss

Alps, its monitoring to see weather
conditions.

And, finally, here is monitoring networks
that is installed on the PFL campus.

In all these cases you have many cameras
using small communication devices.

And so compression and representation is
extremely critical.

So there are a number of results that you
can find in the thesis of Dr.

Chi Chong Chang, given here in this,
website, and essentially the idea is

that, cameras can help each other to
reduce the amount of information that

actually has to be sent to the base
station or into the cloud, for doing

efficient monitoring.
So a long with signal processing today,

actually it's moving to single processing
on graphs.

I don't have to explain to you the
importance, for example, of social

networks.
And so Pedro Pinto who was involved here

in the class and is a post doc in the
lab, together with Patrick Thiran, has

worked on the problem of source
localization.

So you have some graph here, let's say
social network and somebody launches a

rumor.
Here is the source and the rumor gets

forwarded along the edges of the graph at
different times and you have some

observers, say green nodes here, that
receives a rumor at some instant of time.

They know where the rumor comes from, you
know who told you the gossip, and you

know when you got the gossip information.
So, the question is, you know the

structure of the graph, or you have an
approximation of the structure of the

graphs.
You have these observations.

Can you figure out who actually, spreads
the rumor first.

It turns out this has, an interesting
solution.

And using only few observers, about 20%,
you can achieve a very high accuracy in

finding the source of a rumor on a large
scale network.

And there are many interesting questions
here, to pursue in this source

localization in social networks.
And there was a paper that came out last

year, Locating the Source of Diffusion in
Large-Scale Networks that had quite a bit

of impact.
The project is actually funded by the

Bill and Melinda Gates Foundation.
The reason is that one of the

applications is to monitor health
problems.

For example, here is a map of Cholera
outbreak in Africa, and the map shows the

river network.
Cholera is a water born disease, and so,

typically Cholera will actually diffuse
along waterways.

But you know when people fell sick at
certain locations and then you can infer

the source of the actual Cholera
outbreak.

There is another example here, which is a
simulation of, if you had to figure out

if there was some pollution or attack on
the New York subway, and if you could

figure out knowing the network of the New
York subway and when you start detecting

the problems where the source of the
problem actually was.

The next project is on sampling, so we
have worked on a new theory of sampling

here called Finite Rate of Innovation
Sampling, and it is used in

communications problems, and in
monitoring problems to reduce the number

of samples being transmitted or acquired.
Dr.

Freris, who is a senior scientist with
doctoral students and MS assistants, are

actually working on doing ECG monitoring
at very low power for wireless health

monitoring.
So here is a block diagram, it's

relatively complicated so let me not get
into this, but it uses some fairly

sophisticated techniques to reduce the
sampling rate so as to reduce the energy

consumption on these wireless devices.
So, this project is actually sponsored by

somebody well known, Qualcomm, interested
in the theory of sampling.

And the extension here for this
particular project has been

generalization of the initial finite rate
of innovation sampling methodology.

To get better compression, and better
modelization of the signals.

So here we have the ECG signal, and then
there is sophisticated models that

allows, to take very few parameters, to
model the ECG signal.

There are a number of papers here, the
initial paper on finite rate of

innovation sampling is this 2002 paper,
and the number of recent papers have done

extension to this theory.
So if you like sampling I welcome you to

actually read up on this stuff, it's one
of my favorite research topics.

When we talk about sampling already in
sensor networks we have mentioned that

placing a sensor is like taking a sample.
And so that spatial sampling, now if you

do spatial sampling, you can also use
mobile sensors and Dr.

Unnikrishnan here, a post doc in the lab,
has worked on this or generalization of

the theory of sampling when you have
mobile sensors that can actually go over

a field in an arbitrary fashion.
Then you maybe show this in an example.

It's again a temperature monitoring
example here on the EPFL campus, or you

have buildings.
You have that open space between

buildings.
Those buildings are, of course, hot.

The open space are cool.
And you would like to have monitoring of

this temperature field not with static
spatial sensors, but with people running

around, having a thermal meter let's say
on their mobile phone.

And the question is, how accurate can you
actually measure temperature using a

device like this?
And so, this is being done actually for

pollution monitoring in the city of
Lausanne so there's some equipment put on

buses to measure pollution parameters.
And what we do here is we try to develop

a theory of how good you can sample when
you have these mobile sensors going over

a surface and measuring a field.
The results are very mathematical but are

interesting because our non-trivial
extension of sampling theory through

multiple dimensions.
And a few papers are mentioned here if

you are interested in more detail.
The next project is about a new way of

doing image acquisition.
So in this class, we have seen sampling

and we have seen quantization.
And when we do quantization typically we

say, let's take [UNKNOWN] samples and
then take as many bits as possible.

Let's say eight bits for speech, 12 bits
for images 24 bits maybe for audio,

etcetera.
Now here we took the extreme other

example we said lets build an image
sensor that has many, many, many pixels

but the pixels only detect either a
enough light or not.

So the pixels are actually binary
detectors.

And so you have a light intensity here.
Which changes over space.

You have a lens that smooths the light
intensity.

So what reaches the camera is this smooth
curve here.

And this smooth curve you sample very,
very, very finely.

But you only decide if it's above or
below a certain threshold.

So the sensor only generates a sequence
of binary digits.

So that's the imaging model.
And this has been studied by Dr.

Feng Yang, did his PhD thesis on this, is
now a post-doc working on this project,

and a whole slew of other people.
This was a very extensive project.

And what is interesting is that this new
way of acquiring images, for example,

allows to do high dynamic range imaging.
Here is a simulation of a high dynamic

range image in a much easier way than
with conventional cameras.

That's one advantage.
Another one is that you can have very,

very cheap sensors.
So here's an example of one that was

built in the lab.
And then, you take many, many frames.

They are extremely noisy.
If they look noisy, they are simply

binary, so you only have zeroes and ones,
but you have enough of these, and you do

an optimal reconstruction method.
You actually can recognize here, the logo

of EPFL.
There are publications here that you are

welcome to look up.
And the thesis is online.

Last but not least Rambus silicon valley
company, actually works with us on this

and has acquired some of the technologies
that was developed in this project.

And old classic is trying to predict the
stock market.

So, we gave it another shot.
so Lionel Coulot did his PhD thesis, was

co-advised with Peter Bossaerts who is at
Caltech.

And we were trying to understand if
methods from information theory would

allow to predict models for the stock
market, and that requires statistical

models for what the stock market might
be.

And what is interesting is that you have
to decide between very sophisticated

models that might be overkill and are
hard to estimate, and very simple models

which might be too simplistic, but which
might be very robust to things that

happen in the stock market.
And, in the end we used coding theory and

classic algorithmic methods like dynamic
programming to come up with a method that

decides what is the correct model at
every time of, the observation of the

stock market.
So I'm just going to show a picture.

And the picture is, is a value on the
stock market.

And the question is, can you detect if
the stock market is in a bear market or a

bull market?
So when the stock market goes up it's,

called bull market.
If it goes down, it's a bear market.

What is very hard is to decide by
watching every day what's happening.

If currently the trend is going up or the
trend is going down and you need to do

this with an online algorithm.
Okay, you cannot look into the future and

this method developed by Lionel allows to
do a model fitting and to very quickly

detect when the stock market changes from
a bull market to a bar, bear market.

The thesis online and this was sponsored
by, as you may guess, by a bank.

And the results are interesting, but we
are still having a regular day job so you

can guess that the method is not
completely fool proof to predict the

stock market.
But the methods, the algorithms, and the

theory behind it is quite cool.
The next few projects are so called

inverse problems.
So inverse problems are problems where

you have some measurements but the
measurements do not describe the signal

you're interested in.
But some indirect measurement of the

signal, so you try to invert the system
to go back to the original signal.

You all know about computerized
tomography, a medical image method, where

you can see inside the body without
really going there.

And that's a typical inverse problem.
Here we are interested in inverse

problems in environmental monitoring.
So, the first example is diffusion

equation.
And we have a physical phenomena, for

example temperature has been discussed,
or atmospheric dispersal of pollution.

We want to measure the field at locations
where we can put sensors, and the goal is

to find where are the sources, for
example, of pollution.

Now this is a hard problem because, you
have to model how, for example, pollution

is being diffused.
That depends on weather patterns and so

on.
But the tools we are using are typical

signal processing tools, for analysis.
Sampling theory for exemplifying finite

rate of innovation sampling or
compressive sensing, that has also been

mentioned earlier.
Let's look at the picture.

That's a very simple example of this.
Assume you have two smokestacks and

inside a factory compound, and the
smokestacks produce pollution which

changes every day.
You don't know how much pollution is

being released, and you're working for an
environmental monitoring agency.

You put sensors outside of the compound
and you measure what arrives, in terms of

pollution, at these sensors.
And the goal is to figure out if what

came out of smoke stack was within the
bounds allowed, lets say by z,

Environmental Protection Agency.
So this is a interesting and non-trivial

problem but there are some interesting
results that were produced by Yuri

Ranieri, whom you all know because he was
the famous Master Chief assistant for the

BSB class.
So we are able to recover sparse sources

using this inversion method.
we use this finite rate of innovation

sampling techniques to actually do it.
And here we is a list of publications

that came out of this research.
This problem is also an inverse problem.

It's a Fukushima inverse problem.
It is a PhD project of Marta

Martinez-Camara, and a few other of us
are involved in this, and we collaborate

with a specialist Andreas Stohl.
Who is a specialist of monitoring of

radioactive diffusion.
So what we like to do is figure out how

much radionuclides were actually released
in Fukushima at the time of the of the

nuclear accident at Fukushima.
We have only very few sensors, they are

located around the world very far away
from Fukushima.

And the question is, is it possible from
these few measurements around the world

taken later, to invert the entire process
as I diffuse the initial release of

radioactive material into the atmosphere.
What tools are we using?

Sparse regularizations, so that's
compressed sensing.

And we need to using atmospheric
dispersion model to understand how

radioactive material from from Fukushima
was actually transported across the

world.
So one result that we have and which is

very interesting is we were able to
estimate the emission of Xenons, that's

radioactive gas that was released at the
time of explosions at Fukushima, went up

into the atmosphere, was transported by
weather patterns all over the world.

And from the measurements all over the
world, we were able to pinpoint exactly

when the Xenon was released, and how much
Xenon was released into the atmosphere.

And it turns out we actually know the
total amount of Xenon that was released,

because after the accident no Xenon was
actually left in the nuclear power plant.

Currently we're trying to go beyond this
and estimate the Cesium release, but that

turns out to be a harder problem.
The paper that describes this will be

published ICASSP this year and is
available online here in infoscience.

Last but not least is a project we call,
"Can One Hear the Shape of a Room?" It's

a PhD project of Ivan Dokmanic and
several other people in the lab, in

particular, Reza Parhizkar, Andreaz
Walther, have worked on this.

And also we have a collaboration with Yue
Lu.

He's now with Harvard.
Now you know about this problem because,

Ivan gave module 512 about gear
dereverberation, echo cancellation.

And, uh,the next step is to say, if I
listen to echoes, can I actually

understand what is a room shape?
So if I know the room shape, then I know

how to generate the echoes.
But if you give me the echoes, can I know

the room shape?
It's a classic inverse problem, very cute

one.
And we usually explain it by saying,

let's say you enter a room, you're
blindfolded.

And so you don't see the room at all.
You snap your finger.

You therefore elicit echoes, you listen
very carefully to the echoes.

Can you exactly see or hear the shape of
the room?

Now this has a beautiful theory, which we
won't have time to really explain, but

that you can read up about because it's
published material.

But if you have a source or receiver you
have a direct pass between the source and

the receiver, and you have echoes given
by the walls.

The echoes given by the walls correspond
to so called mirror or image sources, so

this is the same as if you had a source
here and the sound would have gone

straight here.
So if you can locate all these image

sources, then you can actually locate the
room.

The walls, therefore the room.
And this is, you know, in principal

do-able the question was is it always
true that this can be done?

And is it also realistic to do it in
practice?

So, here are examples of a system with
five microforms, you have one source five

microforms.
You have somebody snap his finger and you

have the echos related to the walls and
you see there is a complexity which is,

these echos come in random orders because
different walls are at different

distances of the microphone.
And the question is, can we find out the

shape from a set of measurements as we
see here?

How many measurements do we need?
Can we have a robust algorithm?

So the answer is summarized in, yes we
can.

And there are some experiments we did,
both at the labs.

So this is one of our seminar rooms we
created a, an artificial wall here to

have different shapes of rooms.
So this is a typical shape of room.

Then in this case, with five microphone
and one source, we were able to estimate

the size, shape of the room very
accurately to more, better than 1%.

And once we had this, we said, well,
let's see how robust this is.

We went to Lausanne Cathedral and that's
actually a foyer of the Lausanne

Cathedral, which is not at all needing
the assumptions of the algorithms that

I've described very briefly here.
And it was still possible to see the

major refractors, meaning the major walls
here in the Lausanne Cathedral.

And so the answer is yes, one can hear
the shape of a room.

And you can visit Ivan's web page to see
more details.

Now these were just a selection of
projects, of works that is being done by

PhDs and post-docs and senior researchers
in the lab.

Please go to the website, as that gives
the entire portfolio of research here of

what the lab is currently doing.