-
I feel very honored to be invited here.
-
Thank you very much.
-
I like to, I think I've seen one
-
maybe two other people with gray hair here
-
[audience laughter]
-
The last talk I gave a few weeks ago
-
was to a meeting of ophthalmologists
-
and that was a bunch of
-
much older people, okay, and
-
a homage here to the Fifth Elephant
-
This is the novel, the cover of the novel
-
from which it was taken and, well actually
-
I'm using this as a connection for a
-
little bit of boasting because
-
Terry Pratchett wrote the book
-
I am a co-author of a co-author of
Terry Pratchett
-
and I actually signed a publisher contract
-
on my 70th birthday
-
a few weeks ago to publish a
-
science-fiction novel with Ian Stewart
-
and I mention that not just
as boasting but,
-
ok, this is a Data Geeks meeting rather
-
than the Graphics Geeks meeting, but if
-
anybody has graphics enthusiasm, there is
-
all kinds of stuff that would be fun
-
to build for the website we are putting
-
together for that novel. strange things
-
happening on that planet, so do make
-
contact if you're interested in drawing
-
strange and beautiful things because
-
I have some strange and beautiful things
-
to draw and some to interact with.
-
What I don't have is a budget
-
you have to just like it
-
ok, that is pure digression
-
I was originally a mathematician and
-
that was my PhD back before almost anybody
-
here was born, and I've kinda wandered
-
around the world and the sciences and
-
I've turned into some sort of an engineer
-
But what I'm going to talk about here is
-
the power of particular mathematical
-
point of view, which is that numbers are
-
not just numbers
-
They belong together in shapes, so
-
What are data?
-
Mostly, they're numbers
-
I know there are fields and things
-
Not Synced
We've been hearing about that, but
-
Not Synced
then you keep counting
-
Not Synced
Lots and lots of it is numbers
-
Not Synced
But are numbers only numbers?
-
Not Synced
Well, no they gather together in things
-
Not Synced
They come in patterns
-
Not Synced
and really big data is all about
-
Not Synced
the arrangements those things make
-
Not Synced
just knowing the numbers, you don't
know anything
-
Not Synced
You got to know how they fit together
-
Not Synced
Patterns are shapes
-
Not Synced
So, studying shapes, data shapes,
any kind of shapes
-
Not Synced
Space-time shapes. That's Geometry
-
Not Synced
But not the kind that I was doing when I
was 13 or 14 years old
-
Not Synced
Mind you, I had some taste for it and it
was quite fun
-
Not Synced
but it was all flat in the sand,
just like that
-
Not Synced
and here is Euclid
-
Not Synced
Stuff we would write in little triangles
-
Not Synced
and fun things
-
Not Synced
This I remember as a remarkable theorem,
-
Not Synced
but I have never ever, ever, ever
seen a use for
-
Not Synced
[audience laughs]
-
Not Synced
It's weird, it's very much something
-
Not Synced
about the plane, it's strange
-
Not Synced
and I have never encountered it or
-
Not Synced
referred to it in anything useful since I
left school
-
Not Synced
It's a bizarre theorem,
which is occasionally useful
-
Not Synced
Everything is so much in the plane
-
Not Synced
Data shapes don't live mostly in the plane
-
Not Synced
Geometry doesn't mean that you replace
-
Not Synced
now this, by the way, is highly superior
pointer technology
-
Not Synced
Much better than those twinkling little
-
Not Synced
red things that you lose track of
where it's pointing to
-
Not Synced
and 10% of your audience can't see red
-
Not Synced
Now, here is something serious
-
Not Synced
Children think in 3D
-
Not Synced
They think brilliantly in 3D
-
Not Synced
They naturally work in 3D
-
Not Synced
They are connecting how their vision
-
Not Synced
is working with their hands
-
Not Synced
they can reach out and grab your nose
-
Not Synced
If you watch a small child, it's doing
a lot of practice at building
-
Not Synced
a 3D model of the world, and then,
-
Not Synced
and these days that continues into
primary school
-
Not Synced
a hundred years ago, ugh..
-
Not Synced
but now primary school's good
-
Not Synced
but their secondary schools suck rocks
-
Not Synced
It's still, if you get any geometry
-
Not Synced
it's flat, flat stuff.
-
Not Synced
It can get more and more complicated
-
Not Synced
yeah.. but,
-
Not Synced
I just grabbed that off the web
-
Not Synced
as one particular complicated 2D diagram
-
Not Synced
but fix your mind in 2D, you
get to the point where you can't think
-
Not Synced
Are the x- and y- axes this way
or this way?
-
Not Synced
I found my UCLA students
-
Not Synced
if I switched drawing on the blackboard
-
Not Synced
from this way to that way
-
Not Synced
because something could be seen better
that way
-
Not Synced
they couldn't turn it in their heads.
-
Not Synced
Well data doesn't live in the plane.
-
Not Synced
It's not flat.
-
Not Synced
If we have three variables, we have
three dimensions.
-
Not Synced
That might be how far this way,
-
Not Synced
how far this way, and how far up.
-
Not Synced
And if you're doing graphics, it's three
very directly spatial dimensions.
-
Not Synced
But if it's, if you've just got numbers
about people
-
Not Synced
I look at everybody here, I know their
-
Not Synced
height, well I don't know their height but
-
Not Synced
you guys do because you're big data guys.
-
Not Synced
Know the height, know the weight, know
the age.
-
Not Synced
Three numbers, that's a three
dimensional set.
-
Not Synced
And the pattern that you make, that's
-
Not Synced
three dimensional geometry.
-
Not Synced
But of course, you typically have
-
Not Synced
a lot more. So you've got 'n'
dimensions
-
Not Synced
and 'n' can be quite big.
-
Not Synced
So you need to think about 'n' dimensions.
-
Not Synced
And there's two ways to do it.
-
Not Synced
One is to turn it all into algebra, which
-
Not Synced
is what people spend a lot of their time
doing.
-
Not Synced
And in this talk I'm only going to talk
linear algebra
-
Not Synced
which doesn't mean it's the only kind
there is,
-
Not Synced
but I've only got a few minutes.
-
Not Synced
Or you can practice thinking in 3D and
-
Not Synced
build up insights that help you very
-
Not Synced
seriously in 'n'-dimensional thinking.
-
Not Synced
I took up carving things when I was a grad
-
Not Synced
student because I realized that my mind
-
Not Synced
had been flattened by my high school
-
Not Synced
and my undergraduate. I needed to loosen
up
-
Not Synced
my mind and think in 3D, so I started
-
Not Synced
using my hands. You got, this is the
-
Not Synced
visual part of the brain, this is the
-
Not Synced
motor part of the brain, and the motor
cortex
-
Not Synced
just has to be 3D because you've got to
-
Not Synced
pick things up, you got to twist them,
-
Not Synced
connect it all up. So seriously, for 3D
-
Not Synced
thinking, take up sculpture.
-
Not Synced
So practice thinking in 3D, and the more
-
Not Synced
3D you can think, the readier you are to
-
Not Synced
think in other dimensions, general
dimensions.
-
Not Synced
But 2D? Nah, it's not enough.
-
Not Synced
So, question for you guys.
-
Not Synced
Most people here have done things - are
-
Not Synced
doing things with matrices sometimes,
right?
-
Not Synced
What does a matrix even mean?
-
Not Synced
Whats it represent?
-
Not Synced
Blah, you have told me the data structure.
-
Not Synced
It's, yeyah, it's an array.
-
Not Synced
This is the data structure.
-
Not Synced
It's an array this way, and this way.
-
Not Synced
But, at the level of algebra, and geometry.
-
Not Synced
It's something a bit more.
-
Not Synced
It's something that operates on vectors.
-
Not Synced
Transforms vectors, and in particular
-
Not Synced
there was a rule that they taught me
-
Not Synced
yea back at first or second year of graduate
-
Not Synced
of which way you multiply the
matrix and the vector.
-
Not Synced
And, I swear to you it took
me a year to remember
-
Not Synced
When I'm multiplying 2 matrices
do I go along this way
-
Not Synced
or along that way. Because it
was a damn silly rule.
-
Not Synced
That came from the Algebra book.
-
Not Synced
But, trying to avoid spending
too much time on this.
-
Not Synced
You do know the rule most of you.
So, if you have this 3x3 matrix.
-
Not Synced
And, apply it to this vector.
[column of '1,0,0']
-
Not Synced
You get this. Which is this column.
[points to 'a,c,f' columns]
-
Not Synced
And, if you apply to this vector.
[column of '0,1,0']
-
Not Synced
You get this column.
[column of 'b,d,g']
-
Not Synced
And, if you apply to this vector.
[ column of '0,0,1']
-
Not Synced
You get the last column.
-
Not Synced
Now, '1,0,0' means lets suppose
this is the x direction.
-
Not Synced
X this way. It says anything
that is purely in the
-
Not Synced
X direction goes to 'a,c,f'.
-
Not Synced
Wherever that is.
Which is a 3 dimensional vector sum.
-
Not Synced
Anything that is in the Y direction.
Like, '0,1,0' goes to something else.
-
Not Synced
Specifically, it goes to 'b,d,g'.
-
Not Synced
And, anything that starts vertical
goes to 'c,e,h'.
-
Not Synced
So the matrix is actually a list of vectors.
-
Not Synced
It's saying, where does the first one go
-
Not Synced
where does the second one go,
and where does the third one go.
-
Not Synced
And, believe me if you're doing 3d
computer graphics;
-
Not Synced
understanding that point
will make it much easier
-
Not Synced
than anything I've ever seen
in an OpenGL manual
-
Not Synced
of what you ought to be do.
-
Not Synced
They don't explain matrices very well.
-
Not Synced
It's just a list of where
these 3 things go.
-
Not Synced
And, in the case of a rotation
that's particularly tidy.
-
Not Synced
Right angles, things at right angles
go to things at right angles.
-
Not Synced
And, so on. But, not every matrix
is doing something
-
Not Synced
as simple as a rotation. Unless you're as
-
Not Synced
simple-minded as an IQ theorist.
-
Not Synced
And, they rotate things they've
no justification doing.
-
Not Synced
If you remember that, you can always
clarify, see more definitely
-
Not Synced
what the algebra is doing, and
if you know what the algebra is doing.
-
Not Synced
You can make it drive better code.
-
Not Synced
What should the code be doing?
-
Not Synced
So, I'm just going to illustrate this point
of view.
-
Not Synced
With a very top down glimpse at
some of the things people do
-
Not Synced
when they got a lot of data. One of them
-
Not Synced
is Principal Component Analysis.
-
Not Synced
Now this very sketchy, very, very 2d.
-
Not Synced
I didn't have time to do wonderful 3d animations.
I'm sorry.
-
Not Synced
But, I got this variable, this variable.
-
Not Synced
Put them together I got datapoints.
Which are pairs of variables.
-
Not Synced
And, roughly speaking I can say.
Just looking at this.
-
Not Synced
That as this increases, that increases.
-
Not Synced
But, if I want to compress my data.
Than, I rotate my axis.
-
Not Synced
I put an axis along this way.
And, another axis along this way.
-
Not Synced
So, it's going to be a matrix that says
this guy, goes to there
-
Not Synced
and this guy, goes to there.
-
Not Synced
Finding that matrix; you had this chance
of algebra
-
Not Synced
not fitting into this time. But that's
the idea of Principal Component Analysis.
-
Not Synced
And, if you're applying it to a machine.
That does wobbles here, and here.
-
Not Synced
And, makes squeaks there, and
all sorts of things.
-
Not Synced
You've got a lot of numbers,
and you got to do the algebra
-
Not Synced
a bit more complicatedly than
that 2d picture represents.
-
Not Synced
But, that is really what's going on.
-
Not Synced
You're finding the way to move your axes.
-
Not Synced
And now, if you know where you are
on this axis.
-
Not Synced
You know most of what you want
to know about a datapoint.
-
Not Synced
Is it here or is here, one number.
How far along it is.
-
Not Synced
And, then you say well you can expect some
errors in that direction.
-
Not Synced
Where in the previous picture
you had to know
-
Not Synced
2 directions, 2 numbers.
So Principal Component Analysis
-
Not Synced
is beautiful technique for
information compression,
-
Not Synced
reducing the amount of arbitrariness;
handling all sorts of things
-
Not Synced
excellently; as long as things
are reasonably linear.
-
Not Synced
Which, is a very, very big if.
-
Not Synced
Small variations are more often linear.
That's the whole point of
-
Not Synced
the calculus, the calculus is a linear
approximation of small changes.
-
Not Synced
But, um, larger systems
with more variation.
-
Not Synced
Be wary, they're generally not linear.
-
Not Synced
There's an expression
"non-linear mathematics."
-
Not Synced
Which is a bit like
'non-elephant biology.'
-
Not Synced
You shouldn't be defining everything else
by what it isn't.
-
Not Synced
When what it isn't, is so special.
-
Not Synced
Mm, yea, you look to me like
an unusual elephant
-
Not Synced
with some missing teeth, and going
round on two back legs
-
Not Synced
for some reason.
[audience laughs]
-
Not Synced
That's not really a good starting
point for saying what you do.
-
Not Synced
So there's other mathematics
than linear
-
Not Synced
but linear is very powerful,
particularly when variation
-
Not Synced
is reasonably small.
-
Not Synced
And, that is the whole idea
-
Not Synced
of Principal Component Analysis.
-
Not Synced
Managing the machinery of it,
I know people who spent
-
Not Synced
their entire lives doing nothing but
crunching those matrices.
-
Not Synced
But, there's technique.
But, you need the idea
-
Not Synced
of how it's all working. Okay?
-
Not Synced
So, let's try another thing.
-
Not Synced
How many of you have done linear programming?
-
Not Synced
Okay... what is the geometry
in linear programming?
-
Not Synced
Actually the first course I ever
gave back when I was a graduate student.
-
Not Synced
I was told, "Teach these economists
from this book."
-
Not Synced
And one of the things they were supposed
to learn was linear programming.
-
Not Synced
It was all matrices, and you pivot this,
and you shquiggle that.
-
Not Synced
Is that the kind of linear programming
you had?
-
Not Synced
Right? What the matrices do.
-
Not Synced
Yeaa, but what's really going on
is what the geometry is doing.
-
Not Synced
So, first of all what does this mean?
-
Not Synced
Suppose this was just 'X, Y, and Zed'.
-
Not Synced
That if I put equal to 0; that's a plane.
-
Not Synced
I'm positive on one side, and
negative on the other.
-
Not Synced
I say I got to be positive.
-
Not Synced
Oh, how would I define a cube?
-
Not Synced
I'd say I'd got something
positive on this side of the cube,
-
Not Synced
something positive on
that side of the cube,
-
Not Synced
something positive as I go down
from the top,
-
Not Synced
something that is positive
as I go up from the bottom.
-
Not Synced
With 6 inequalities I've got a cube.
-
Not Synced
Same idea in N-dimensions.
-
Not Synced
But 3 is plenty for thinking
about this one.
-
Not Synced
2d figures don't give the magic at all
-
Not Synced
of what you need to do.
-
Not Synced
But, the 3d problem does.
-
Not Synced
You see here is where one
of those limiting planes is.
-
Not Synced
Here's where another one meets.
-
Not Synced
So they're doing all kinds
of things like this.
-
Not Synced
Inside this polyhedron you're
satisfying all those constraints.
-
Not Synced
Go outside, cross any one of
those planes; you're not.
-
Not Synced
And, the algebra problem is
that the planes meet
-
Not Synced
in a whole lot of places.
-
Not Synced
It looks like a sort of weird,
3-dimensional, hedgehoggy thing.
-
Not Synced
This plane, and this plane,
they don't meet on the surface;
-
Not Synced
but they meet somewhere here.
-
Not Synced
So you want to make sure you're
staying inside this region.
-
Not Synced
So about 1950 comes the
'Simplex Method'
-
Not Synced
That confused, because simplex is a word
-
Not Synced
that mathematicians use differently.