I feel very honored to be invited here.
Thank you very much.
I like to, I think I've seen one
maybe two other people with gray hair here
[audience laughter]
The last talk I gave a few weeks ago
was to a meeting of ophthalmologists
and that was a bunch of
much older people, okay, and
a homage here to the Fifth Elephant
This is the novel, the cover of the novel
from which it was taken and, well actually
I'm using this as a connection for a
little bit of boasting because
Terry Pratchett wrote the book
I am a co-author of a co-author of Terry Pratchett
and I actually signed a publisher contract
on my 70th birthday
a few weeks ago to publish a
science-fiction novel with Ian Stewart
and I mention that not just as boasting but,
ok, this is a Data Geeks meeting rather
than the Graphics Geeks meeting, but if
anybody has graphics enthusiasm, there is
all kinds of stuff that would be fun
to build for the website we are putting
together for that novel. strange things
happening on that planet, so do make
contact if you're interested in drawing
strange and beautiful things because
I have some strange and beautiful things
to draw and some to interact with.
What I don't have is a budget
you have to just like it
ok, that is pure digression
I was originally a mathematician and
that was my PhD back before almost anybody
here was born, and I've kinda wandered
around the world and the sciences and
I've turned into some sort of an engineer
But what I'm going to talk about here is
the power of particular mathematical
point of view, which is that numbers are
not just numbers
They belong together in shapes, so
What are data?
Mostly, they're numbers
I know there are fields and things
We've been hearing about that, but
then you keep counting
Lots and lots of it is numbers
But are numbers only numbers?
Well, no they gather together in things
They come in patterns
and really big data is all about
the arrangements those things make
just knowing the numbers, you don't know anything
You got to know how they fit together
Patterns are shapes
So, studying shapes, data shapes, any kind of shapes
Space-time shapes. That's Geometry
But not the kind that I was doing when I was 13 or 14 years old
Mind you, I had some taste for it and it was quite fun
but it was all flat in the sand, just like that
and here is Euclid
Stuff we would write in little triangles
and fun things
This I remember as a remarkable theorem,
but I have never ever, ever, ever seen a use for
[audience laughs]
It's weird, it's very much something
about the plane, it's strange
and I have never encountered it or
referred to it in anything useful since I left school
It's a bizarre theorem, which is occasionally useful
Everything is so much in the plane
Data shapes don't live mostly in the plane
Geometry doesn't mean that you replace
now this, by the way, is highly superior pointer technology
Much better than those twinkling little
red things that you lose track of where it's pointing to
and 10% of your audience can't see red
Now, here is something serious
Children think in 3D
They think brilliantly in 3D
They naturally work in 3D
They are connecting how their vision
is working with their hands
they can reach out and grab your nose
If you watch a small child, it's doing a lot of practice at building
a 3D model of the world, and then,
and these days that continues into primary school
a hundred years ago, ugh..
but now primary school's good
but their secondary schools suck rocks
It's still, if you get any geometry
it's flat, flat stuff.
It can get more and more complicated
yeah.. but,
I just grabbed that off the web
as one particular complicated 2D diagram
but fix your mind in 2D, you get to the point where you can't think
Are the x- and y- axes this way or this way?
I found my UCLA students
if I switched drawing on the blackboard
from this way to that way
because something could be seen better that way
they couldn't turn it in their heads.
Well data doesn't live in the plane.
It's not flat.
If we have three variables, we have three dimensions.
That might be how far this way,
how far this way, and how far up.
And if you're doing graphics, it's three very directly spatial dimensions.
But if it's, if you've just got numbers about people
I look at everybody here, I know their
height, well I don't know their height but
you guys do because you're big data guys.
Know the height, know the weight, know the age.
Three numbers, that's a three dimensional set.
And the pattern that you make, that's
three dimensional geometry.
But of course, you typically have
a lot more. So you've got 'n' dimensions
and 'n' can be quite big.
So you need to think about 'n' dimensions.
And there's two ways to do it.
One is to turn it all into algebra, which
is what people spend a lot of their time doing.
And in this talk I'm only going to talk linear algebra
which doesn't mean it's the only kind there is,
but I've only got a few minutes.
Or you can practice thinking in 3D and
build up insights that help you very
seriously in 'n'-dimensional thinking.
I took up carving things when I was a grad
student because I realized that my mind
had been flattened by my high school
and my undergraduate. I needed to loosen up
my mind and think in 3D, so I started
using my hands. You got, this is the
visual part of the brain, this is the
motor part of the brain, and the motor cortex
just has to be 3D because you've got to
pick things up, you got to twist them,
connect it all up. So seriously, for 3D
thinking, take up sculpture.
So practice thinking in 3D, and the more
3D you can think, the readier you are to
think in other dimensions, general dimensions.
But 2D? Nah, it's not enough.
So, question for you guys.
Most people here have done things - are
doing things with matrices sometimes, right?
What does a matrix even mean?
Whats it represent?
Blah, you have told me the data structure.
It's, yeyah, it's an array.
This is the data structure.
It's an array this way, and this way.
But, at the level of algebra, and geometry.
It's something a bit more.
It's something that operates on vectors.
Transforms vectors, and in particular
there was a rule that they taught me
yea back at first or second year of graduate
of which way you multiply the matrix and the vector.
And, I swear to you it took me a year to remember
When I'm multiplying 2 matrices do I go along this way
or along that way. Because it was a damn silly rule.
That came from the Algebra book.
But, trying to avoid spending too much time on this.
You do know the rule most of you. So, if you have this 3x3 matrix.
And, apply it to this vector. [column of '1,0,0']
You get this. Which is this column. [points to 'a,c,f' columns]
And, if you apply to this vector. [column of '0,1,0']
You get this column. [column of 'b,d,g']
And, if you apply to this vector. [ column of '0,0,1']
You get the last column.
Now, '1,0,0' means lets suppose this is the x direction.
X this way. It says anything that is purely in the
X direction goes to 'a,c,f'.
Wherever that is. Which is a 3 dimensional vector sum.
Anything that is in the Y direction. Like, '0,1,0' goes to something else.
Specifically, it goes to 'b,d,g'.
And, anything that starts vertical goes to 'c,e,h'.
So the matrix is actually a list of vectors.
It's saying, where does the first one go
where does the second one go, and where does the third one go.
And, believe me if you're doing 3d computer graphics;
understanding that point will make it much easier
than anything I've ever seen in an OpenGL manual
of what you ought to be do.
They don't explain matrices very well.
It's just a list of where these 3 things go.
And, in the case of a rotation that's particularly tidy.
Right angles, things at right angles go to things at right angles.
And, so on. But, not every matrix is doing something
as simple as a rotation. Unless you're as
simple-minded as an IQ theorist.
And, they rotate things they've no justification doing.
If you remember that, you can always clarify, see more definitely
what the algebra is doing, and if you know what the algebra is doing.
You can make it drive better code.
What should the code be doing?
So, I'm just going to illustrate this point of view.
With a very top down glimpse at some of the things people do
when they got a lot of data. One of them
is Principal Component Analysis.
Now this very sketchy, very, very 2d.
I didn't have time to do wonderful 3d animations. I'm sorry.
But, I got this variable, this variable.
Put them together I got datapoints. Which are pairs of variables.
And, roughly speaking I can say. Just looking at this.
That as this increases, that increases.
But, if I want to compress my data. Than, I rotate my axis.
I put an axis along this way. And, another axis along this way.
So, it's going to be a matrix that says this guy, goes to there
and this guy, goes to there.
Finding that matrix; you had this chance of algebra
not fitting into this time. But that's the idea of Principal Component Analysis.
And, if you're applying it to a machine. That does wobbles here, and here.
And, makes squeaks there, and all sorts of things.
You've got a lot of numbers, and you got to do the algebra
a bit more complicatedly than that 2d picture represents.
But, that is really what's going on.
You're finding the way to move your axes.
And now, if you know where you are on this axis.
You know most of what you want to know about a datapoint.
Is it here or is here, one number. How far along it is.
And, then you say well you can expect some errors in that direction.
Where in the previous picture you had to know
2 directions, 2 numbers. So Principal Component Analysis
is beautiful technique for information compression,
reducing the amount of arbitrariness; handling all sorts of things
excellently; as long as things are reasonably linear.
Which, is a very, very big if.
Small variations are more often linear. That's the whole point of
the calculus, the calculus is a linear approximation of small changes.
But, um, larger systems with more variation.
Be wary, they're generally not linear.
There's an expression "non-linear mathematics."
Which is a bit like 'non-elephant biology.'
You shouldn't be defining everything else by what it isn't.
When what it isn't, is so special.
Mm, yea, you look to me like an unusual elephant
with some missing teeth, and going round on two back legs
for some reason. [audience laughs]
That's not really a good starting point for saying what you do.
So there's other mathematics than linear
but linear is very powerful, particularly when variation
is reasonably small.
And, that is the whole idea
of Principal Component Analysis.
Managing the machinery of it, I know people who spent
their entire lives doing nothing but crunching those matrices.
But, there's technique. But, you need the idea
of how it's all working. Okay?
So, let's try another thing.
How many of you have done linear programming?
Okay... what is the geometry in linear programming?
Actually the first course I ever gave back when I was a graduate student.
I was told, "Teach these economists from this book."
And one of the things they were supposed to learn was linear programming.
It was all matrices, and you pivot this, and you shquiggle that.
Is that the kind of linear programming you had?
Right? What the matrices do.
Yeaa, but what's really going on is what the geometry is doing.
So, first of all what does this mean?
Suppose this was just 'X, Y, and Zed'.
That if I put equal to 0; that's a plane.
I'm positive on one side, and negative on the other.
I say I got to be positive.
Oh, how would I define a cube?
I'd say I'd got something positive on this side of the cube,
something positive on that side of the cube,
something positive as I go down from the top,
something that is positive as I go up from the bottom.
With 6 inequalities I've got a cube.
Same idea in N-dimensions.
But 3 is plenty for thinking about this one.
2d figures don't give the magic at all
of what you need to do.
But, the 3d problem does.
You see here is where one of those limiting planes is.
Here's where another one meets.
So they're doing all kinds of things like this.
Inside this polyhedron you're satisfying all those constraints.
Go outside, cross any one of those planes; you're not.
And, the algebra problem is that the planes meet
in a whole lot of places.
It looks like a sort of weird, 3-dimensional, hedgehoggy thing.
This plane, and this plane, they don't meet on the surface;
but they meet somewhere here.
So you want to make sure you're staying inside this region.
So about 1950 comes the 'Simplex Method'
That confused, because simplex is a word
that mathematicians use differently.