So, I think we are ready to start
So, next talk is from Martin, about Debian on ARM devices.
So, it's Debian on ARM devices, it's not about the Debian ARM port itself, or any of the various ARM ports
I would say it's sort of a mix of different topics
it's basically ---
I mean, basically I often get
it's basically ---
I mean, basically I often get
the question from users
"Well, Debian works on this particular ARM device,
I have a device which is like really really similar,
so surely Debian should work on it, right?"
And so far, the answer unfortunately was usually "no"
and - and it's really frustrating for users
because they don't see why.
You know, the device is very much the same, so why doesn't it just work.
And so, basically what I'm trying to do is
to show like a little bit behind the scenes
of how Debian on ARM works
And there has been a whole lot of progres
So I'm going to show how did it work,
what are the improvements, and
and how is it going to work in the future
And it's all more from a develeoper perspective
so, it's like, how do we support it in the Debian Installer
its' like a mix of stuff, and I should say,
I'm really looking at it from somebody who basically
adds support for devices in the installer
but I'm not like a kernel guy or anything, so
there are a lot of people in this room who know a bit more about the technical details
but I'm not going to really drill down really deep down anyway
But if I'm incorrect, I'm sure that
there are plenty of people that will correct me.
So, where do you find ARM? So it's pretty much everywhere these days, I mean...
Pretty much every phone has an ARM chip in it
All those "Internet of Things" gadgets are ARM-based
You have NAS devices, and that's really the area I used to focus on
those small boxes where you basically add a hard drive
and most people just use it for storage, but it's actually a
full PC where you can install Debian on it
so I always found that to be very interesting
and nowadays you have a lot of development boards
And I'm not sure whether "development boards" is the right word
Because when I hear "development boards" I think of it as really expensive,
Like 5K for boards or something
But nowadays you have those, maybe, a better board would be like bareboards
Like "Raspberry Pi", like basically it's just the board
it does not come with a case, you can buy those as well
and it's like 30 dollars or something, you can get
And that gets pretty much where most of the
excitement is going for Debian users at this moment
And that can cause a lot of frustration
And everybody says that ARM is going to be big on servers
and I think that's something we ar going to see
But right now, again, it's
a little bit frustrating
How does it work as a Deban Installer
There are basically two
ways of installing on those ARM Pi devices
So the, like, historic, that we normally
would be to install on, like, ascreen
can be a serial console,
and then you just download via the network
and then you just download via the network
from a CD image or wherever,
But the other method, which we actually use for those
net devices was a network console
where you basically ssh into the installer
and you perform the installation via ssh
and that was really the only way, because those mass devices
don't have any I/O, so I
even in there, you can
it's just a serial console, most of them
That's something we didn't want to require from the user
And the other thing that's coming through, I think that it's basically cool that was announce today,
screen support, so basically
so you can have multiple sessions with in vi,
which can be useful if you have an open shell to debug
And so, how -- And so, this is really looking back at
like running Debian on NAS devices
which is something that used to be really popular
And the way it worked was basically
that we provided an installer image
which was sort of a firmware image
as a lot of those NAS devices
have like a firmware upgrade mechanism
and so we would basically fake
we would create a firmware image
which the software would accept
but instead of a firmware update
it would actually contain the Debian installer
so you would install that upgrade
you would reboot, and
you could ssh to the installer
and obviously, in order to ssh to the installer
it needs to bring up the network
and there is a tool called oldsys-preseed
that basically reads the network configuration
from the device
and then sets up the network
and it can always recognize the DHCP
and then the user connects via ssh
again, there would be some indication, maybe
you know, there would be a "beep"
or maybe change the LED
to indicate that the installer is ready
and then, the user basically performs
a regular installation
with normal d-i; they don't have to
do anything differently.
And at the end, flash-kernel runs
to make the system bootable
flash-kernel is called that way
because it used to support
flash-kernel is called that way
because it used to support
like the initial device it supported
booted from flash
but it also generates bootable devices
bootable images of disk devices
so it really, flash-kernel really requires
understanding of each device
and our philosophy
in those cases, with those NAS devices
was really,
we don't touch anything
in the firmware, like
in the configuration,
so sometimes
instead of changing the root device
in the u-boot config we would actually
hard-code that in the ramdisk
just because we wanted people to go back
to the original firmware if they had to
if they had to send it in for repair or something
and
and that kind of approach really worked well
so I think we really had a lot of
people, lot of users running
in Debian in those kind of NAS devices
and it was really easy to do.
You get a firmware image,
you connect to the installer via ssh,
and it just works, it's a normal
debian-installer, the way
everybody knows it,
because some of the other distros
they basically provide, like,
tarballs and instructions
so you need to partition a disk,
need to un-tar it,
you need to change those files,
and even if it sounds simple,
it's so many steps that
you always... you often get something wrong
and then you put the drive into the
NAS device, and it doesn't boot, and you don't know why
where did you make that mistake,
which step
and you basically have to start from scratch
so Debian really provided something unique
by adding
the debian-installer support
but anyway, that's the way
it sort of used to work.
Nowadays, with a lot of those bareboard
it's much easier
I'll get to that.
So at the moment there are three different ARM ports:
There is the old armel
which used to be the "newer" ARM
and now it's old,
and one of the discussions
that we are probably going to have later today on the BoF
is about, should we remove that
after Stretch.
There is armhf and the arm64
and we get basically the question I hope to answer.
If, you know, device A works,
I have a device which is really similar,
but it doesn't work. Why is that?
So, heh,
There have been a lot of changes
in various upstream projects, specially
the kernel and U-boot
that really make things easier.
So, in the past
we basically had
a kernel image
for each platform,
where a platform is basically like
a SSE family
and there would be a different image
because ARM takes a long time
to compile, there was always some debate
about adding a new platform
because there would be a new
image flavor, which takes some time,
and that was just really like,
we couldn't just have one ARM kernel
which works everywhere.
And a lot of people didn't understand
why you had a different kernel
because different platforms,
although there has been a lot of progress upstream,
at least nowadays, with armhf
and with arm64
we just have one kernel.
And upstream basically
Maybe some of you remember the rant
by Niels about
the ARM people doing everything
in different ways,
and there has been a lot of standarization
over the years.
Basically, the other thing is
there used to be for each device
there was a board file
it was like a C file
to initialize the different components,
and the boot loader would
pass the machine ID to the kernel,
and it would load that boot file.
And nowadays,
there is basically a
device tree in the kernel,
which is a description of the hardware
and it will basically compile it
to, like, a binary blob, the .dtb.
And, so, basically you just need
the kernel image, and
the .dtb, which is hardware-specific,
and then it boots.
And obviously, for us in Debian
that makes it much much easier
to support a lot of devices.
The other thing that changed in U-boot,
when you install the Debian kernel image,
it creates
the vmlinux file
and also the RAM disk
but with U-boot,
you couldn't actually load those files
correctly. You basically
had to wrap them in a U-boot image
and it's not really hard
it's just the command initram,
but all of the different devices had
different load addresses,
and, again, that's hardware-specific knowledge
that flash-kernel needed to know.
And nowadays,
there is a command which
directly loads the kernel,
so that, again, was a step to make things easier.
And the last thing which really made
things easier is distro support
in U-boot.
So, basically, in the past
every U-boot, every
devices in U-boot,
booted in a different way,
just in terms on where would it load
the file from, or
what variables would it use, and
nowadays there is something called distro-support
which is basically a standardized way
to boot a Linux distro
with U-boot.
There are basically
two ways, either can
read a config file,
or it can basically run a
boot script, and that's what
we use in Debian, so we basically have
a generic boot script
which loads the kernel,
loads the ramdisk, the .dtb,
and boots Debian.
It can use the generic
bootscript in almost all
of the modern devices.
So nowadays, because of that,
it's much more standard, and
it's much easier to support
those devices.
So here are some examples of flash-kernel
So, basically, flash-kernel has like a database
of devices which it supports
so it's like the machine entry
in /proc/cpuinfo,
and then the kernel flavors
it can run
and, so, this one is
an old device, it still uses
a board file, and it
needs the U-boot wrapper,
So...
You are basically setting the machine ID
those are the flash partitions
where the kernel and the
ramdisk are stored, and
that's the load address for the
U-boot wrapper
And, now, that's a device
which used to have
a boot file,
and which then migrated
to a .dtb,
so basically, heh,
that was really really painful for Debian
so, basically, the kernel people said
"Oh, you are going to move to device tree,
so don't worry,
we are not gonna remove those old board files,
you can still use them."
And then, a few years later they realized
it's really hard to keep both alive,
and they got rid of the board files.
And that was really painful to us
because we had to migrate
So you can see,
here, a kernel version,
and so, from that kernel on,
you needed to use the new way.
And one of the reasons it was painful
specially on the QNAP devices
is because with
they actually have two different CPUs
so there are different barriers
and whilst the board files,
the same board file worked
regardless of the CPU,
because of the way the device tree worked
it actually needed two different device trees
depending on the CPU
so now we basically, you know, something that just worked,
you know, it used to work fine,
you just had one kernel
with the machine ID
at least would work,
and somehow with the device tree
we needed to figure out which one
we need, and so
[?] fortunately did all of that work.
So that's actually a script that runs
to figure out which device tree
that particular device needs.
So now, that's actually
the reason I want to show this is
how simpler things are these days, so
this is an example from
[?] platform, which uses distro-support,
and the only thing you really need is
a machine entry with the name,
and then
like the kernel-flavor,
but that's the same for...
you know, there is only one kernel flavor.
And then, the device tree ID
and, again, all of that stuff is generic,
so it just uses the generic bootscript,
and the generic boot path,
so basically
all it pretty much needs for
a new device is, like,
you know,
these two entries,
it's really really simple.
So here, I just wanted to
tell about the different ARM ports and...
So, for me it was really hard to structure this, because
those changes in the kernel
and U-boot
you know, happen independent of our ARM ports,
But at the same time, because
because the armhf world is much newer,
it works in a different way.
So, basically, the armel
we still have different flavors
but we have already combined the orion and the kirkwood
into one marvell flavor,
and we have the versatile one.
So one of the problems we have in armel is that
a lot of those NAS devices
boot from flash
and they only have, some of them,
3MB for the kernel,
which used to be a lot of space
but nowadays it's not.
So that really puts a lot of restrictions
so basically we disable some stuff
from armel
but I think that armel is really really widely used
because of those NAS devices,
but I think they are slowly getting old,
but there are still a lot of people that use them.
Like I said, it requires the U-boot image
Originally, we used board files
and device tree, most of them
have switched over to device tree
even if some of them still use board files
and adding a new device requires
a number of changes in the installer.
So basically, you needed to map
the device to
the complete image
and there are just there a couple of
things, so basically, any one device
you have to change like five different
places in the installer.
And it was quite confusing for people who wanted
to add new devices
Because it isn't really documented very well
So some examples are a list of three people
who are involved in that port
So, Roger is someone
who is quite new, and who really got
involved into porting those old devices.
And so, armhf is much nicer,
so the majority of
devices support distro-support.
And the other thing we do
is, for some of the devices
we provide SD card images,
which contain
U-boot and the Debian installer
So basically Vagrant
maintains U-boot in Debian, and
a lot of those devices are supported
in U-boot in Debian.
So we just
provide an SD image, so you can just
store it in the SD card, you put it in
and, because of the distro-support,
it just loads debian-installer,
you do the installation, reboot,
and things just work.
So I think it really has gone
...Come a long way.
So nowadays,
adding a new device
requires much fewer changes
than it used to be.
Because as everything uses the same, like, one kernel flavor,
you don't need to patch all those different places anymore,
and because of distro-support, it just works.
You can pretty much use the generic bootscript.
So, arm64.
So, the problem until recently
is that there simply wasn't any hardware
that people could buy.
And that's changing rapidly now
[?]
We have for example the Raspberry Pi 3
which uses a 64 bit CPU
but the software it ships is only 32 bits,
but most of the work
is now
upstreamed to run 64 bit on it
and there are a lot of
devices which sort of work but not quite
but anyway, so the idea for arm64
is that a lot of the
new hardware
specially on the server side
will use UEFI,
and so you basically just
it just works like a PC.
So you have Grub,
you can run debian-installer from Grub
and you get Grub afterwards.
And just boots like a PC,
and Steve has done a lot of work in that area,
And in theory,
it should just work out of the box.
So, if it uses
UEFI, we don't need to add anything.
It just works.
There's nothing to do.
At least, that's the theory.
So, what I found is
...I made a disclaimer, so
...Assuming the kernel
you know, kernel support and stuff...
...To be honest,
right now, that's a big assumption.
So I've been playing with a few
64 bit ARM boards
and basically, what I found is
there is some support upstream,
so I can boot a kernel,
but, oh! There is no USB support!
there is no Wifi support!
So I cannot actually do anything with it.
But I can get to something that
because arm64 is still farily new.
And there is a lot of work going on upstream
to support the various platforms.
So, I think over time that's really going to be much better.
Even though that UEFI idea is there
in reality,
we are going to see different solutions
for arm64.
So we are going to see UEFI, in particular on servers,
but we will also see U-boot.
So a lot of those bare boards
they have U-boot.
And, so, right now
we can use the distro support
and I think that it works pretty well.
But one thing that SuSE has done
so, they just had the same issue,
they want to support all those devices
but they just want to use the same mechanism everywhere,
so they have actually implemented
UEFI on top of U-boot,
so you can basically use U-boot to load Grub
and then you have Grub
so I think that's something we need to figure out;
do we want to stay with distro-support?
do we want to use
UEFI on top of U-boot?
is that something we want to
give users the option?
And then, Fastboot
is something used
in the Android world
and a lot of those, like, I see a lot of
both bare boards, but also
game consoles and stuff, which
which are sort of Android-oriented,
but which can also run normal Linux,
and they will use like Fastboot or something.
And there are tons of other bootloaders out there
But I would definitively say, like, the good ones
are UEFI and U-boot
So, I gave a similar talk a few weeks ago
and turns out, a lot of non-free
firmware, like, "can I
run that stuff, you know,
purely with free software?"
...And...
So, the thing with ARM is that
there are a lot of different platforms
I'm not sure about all of them,
but some of the platforms
I looked at, yes, you
do need some... There is always something propietary.
So in a lot of cases
I see where you have
you use the proprietary first-stage boot loader,
so the Raspberry Pi is an example,
so, I don't actually have a Raspberry Pi, but
the way I understand it is
you basically put some boot files
in an SD card,
and they are proprietary, and
then they load the kernel.
Or in case of [?]
supported in Debian,
the first stage boot loader would basically
be used to run U-boot,
and then we could use U-boot, and
the U-boot support all of it is free software,
but the first stage boot loader isn't.
I have had, there are some people
working on a free replacement for
the Raspberry Pi, though.
With nVidia Tegra,
which is
something I'm working on at the moment
you also have a tiny first stage boot loader,
but then again, you have U-boot, which is free,
and in that case, you also have some firmware
images for the GPU
and for other stuff.
For the Dragon board,
The Dragon board sounded really interesting
because it actually has a
a graphics chip
which can be used with free software,
so that sounded pretty cool,
but again, it has a
proprietary first
stage boot loader,
and then there is a second stage boot loader
which is actually open source
and then there's actually a U-boot of that,
and then you have some
binary blobs which
also need to be installed in flash to work properly
On the Marvell side, I'm
not really aware of anything.
I have never needed to flash anything
proprietary, but
I'm not sure how U-boot gets started
on Marvell, so maybe there is something.
So the future is all... So NAS devices are
like I said, that's something that's
really been popular in Debian.
Specially on the QNAP devices,
but the problem is, so, QNAP...
so the devices we currently support
are pretty old
nowadays,
and they have some newer devices,
but they are not properly supported
in the upstream kernel.
So, I have no plans
to support Debian on those.
I recently did some work on
Seagate NAS devices
which are actually pretty interesting
but again, they will go out of date
already,
and then there's the whole 64 bit ARM
so I think people are really waiting
for arm64 servers
I think there's going to be a whole lot of
work on that area.
And then, there are all of those development boards
Raspberry Pi,
Pine64,
Allwinder,
Basically, all of them
sound exciting, but if you
look at them, all of them
have some upstream issues, so it's...
It is really quite frustrating at the moment.
But I think things are really
moving [?]
And then there's this 96Board initiative,
which is actually done by Linaro,
and, so, Linaro supports
Linux on ARM, so you'd think, "wow,
there are some really nice boards coming out!"
and they differentiated
between the consumer edition and the enterprise edition
beside, I bought some consumer-edition boards,
and it's really horrible.
So first of all, I had to spend like half a day
just putting the components
because it uses, like, a nonstandard power supply,
a nonstandard serial console,
so finally I found
all the pieces I needed, and then
I was expecting, well, it's from Linaro!
Surely everything just works upstream!
But it doesn't. It's like
yes, I can boot the Linux kernel,
but there's no USB, there's no
Wifi, no nothing...
So, yes, that's a little bit frustrating.
On the enterprise edition,
I think that looks more interesting.
You know, that's more standardized.
But again, there have been some delays.
[audience; unintellegible]
Yes, OK.
So that would be interesting to see.
So, the questions that I actually have nowadays
is, so...
Because of those changes,
because of having, you know,
a mode of U-boot, most of those
new devices having a U-boot
with distro-support,
having a kernel which, you know,
one kernel image that works on all of those
devices which are supported,
it's really easy to support
a new device?
As long as it's supported by the kernel.
But, at the same time, I think it's a big
challenge for Debian.
Because right now, if you look at armhf,
we basically say, well,
hint the installer,
and if there's a device tree,
it's probably going to work.
But, what does that mean, right?
And...
which really work, which means,
so, we have Vagrant
having U-boot support
in Debian, we have
people testing debian-installer,
testing the Debian kernel, so we have
devices which really work, well supported,
and then we have some devices
on the other hand where, well, yes,
there is a device tree, but no one
has ever tried it. And
at the moment, we have no way
for users to differentiate
those use case... Those
support levels.
So I'm wondering if we need
like a table,
somewhere where maybe
some support levels, like green, yellow...
red?
Where green is, "yes, we have Debian
people who have
testing that stuff",
yellow would be, "well, we have heard some report
that it might work", and
green is, you know, doesn't work,
or we haven't tried that it works.
Maybe we need something like that.
But right now,
I see from users
"I want to run Debian on my ARM device", and
they don't know if it's going to work.
Yes, then...
[audience; unintellegible]
Yes, so Ben is saying we also need
to track what the earliest and the latest
kernel versions that there have been tested.
So I think we really need to
come up with some criteria
to have, you know, define those kernel support levels that indicate that
And yes, the other question is related
to, with all those boards,
how do we actually test them?
So, I keep --
it's really... remember [?]
which is great, but also acknowledge
those boards, they are so cheap!
so it's like, "oh, there's a new board!
It's £30! I'll just get it!"
And then we have, like, those piles of boards
and realize, well, I don't have time
for testing all of that stuff!
I think that's going to be a real challenge.
hundreds -- I don't know,
it's really hundreds of boards coming out.
How do we support all of that?
So, I think that's the question I wanted to raise,
and maybe something we can talk about in the BoF
But yes, I'm obviously
open for questions now.
[James / purpleidea:] Hello.
So, don't quote me exactly,
so, I believe
RedHat has this problem as well, obviously,
I mean, they are obviously more interested in the
ARM64-only server stuff,
but I believe the way they are going about it
maybe it's something could collaborate on
is, they are trying to make sure and push
all the vendors to be standardized
because, as you pointed out,
it's just not, it's crazy
with all the different device tree differences and so on,
so I believe their strategy is to
work with the vendors, and
require everything to be upstreamed, and
no device tree
specifically, to push everything to just boot
with one kernel, and so on.
So maybe that could be
sacrifice a few of the
shitty boards, but work with the vendors
that make the ones that are upstreamed.
[Martin:] Yes, so I may be mistaken, but
as far as I know, RedHat
basically says, "we only support
UEFI and ACPI"
and that's fair enough,
and I think that works for them, because
it's just the server world they care about,
but I think in the case of Debian
there are so many boards out there which
don't need those specs,
and we sort of live, like, in the "real world"
so I don't think we can
Well -- I know it's different. I mean,
if RedHat wants to target
the server people, like, the people
which give money
that's fair enough, that makes sense for them
but Debian, we run everywhere
so I think we need to support
all those weird
cases as well.
And maybe if it gets too weird,
I mean,
I put in some work in the Dragonboard,
and then I realized,
am I actually spending the time because
no one, like, I have heard
from no one that they have
that board, I mean, like [?]
for the Raspberry Pi, but I have heard
like, pretty much no one has the
Dragonboard, so maybe,
...it's not worth my while,
but if there's a board
that people want to use it,
I think we should support it in Debian.
And we do have the infrastructure
so we, you know, it doesn't have to be
UEFI and ACPI,
we can support other devices, because
we have done it before.
But I definitively agree with
the point about getting more standardization
and that stuff, so
I agree, yes.
And the whole distro-support
in U-boot, that has really made things easier for us.
[Phil:] I may begin to say some of the same things Steve has, anyway.
Steve]: So, yes, on the
ACPI versus DT thing,
it's more a question
of the quality of the implementation of the
of the firmware,
and the upstream support that is...
DT isn't fundamentally
worse for upstream support
than ACPI is,
RedHat are
have some reasons
[?] of sensibility,
but then, it's not
fundamentally, it doesn't fundamentally stop it
people supporting it, specially
a community distro like Debian.
I was also going to ask Martin,
have you talked to
people like kernel CI
about testing and coverage stuff?
[Martin:] Yes, not really, but
I think there are some things we should look into.
[Steve:] Yes, because the whole --
Obviously, making sure the kernel boots
on random hardware
stuff, is exactly what that's doing
and there's at least infrastructure there that might be
nice to play with.
[Martin:] Yes. I think that's a good idea.
[sledge:] So, on the whole 96Boards thing,
sorry!
Most of the engineers involved
are totally aware of
how [?] it really is.
We tried to tell management
they would --
They don't want to listen.
Umm.. So,
there has been a huge amount of pushback
saying, you know, "we're Linaro
we should make sure this is all upstreamed".
[Martin:] Well, I am aware
it's just, Linaro
has a really good brand,
so people expect, you know, when I saw
96Boards it's Linaro,
obviously that stuff it's going to work!
and that's why I was really disappointed.
[Steve:] One positive thing there
is that the first DE board
the SoC does basically work upstream
already with current upstream kernel
it's not like there's some patches still
to go in. Apart from the PCI, it should basically
work... So, hopefully,
hopefully, that's going to be a much
less spectacular
story than the
consumer-edition boards have been.
As they are unfortunate.
[Sledge:] The first EE board
the [?]board, is due to start shipping
really soon now.
There are examples already out there
with engineers for validation.
Literally, we are talking the next couple of weeks
is what I have been told.
That's the first 96Boards I would
actually spend any time on at all personally,
the CE boards are a joke.
I forgot what else I was going to say. Oh!
Yes! Then...
I guess we will go through a lot of this again
in the ARM BoF later on
in terms of
which things we support.
We will carry on with the discussion.
[Vagrant:] Um... Is this working?
Yes, so...
We have got all this great support for
distro-boot command and U-boot, and
now we are starting to get
to UEFI emulation
in U-boot, so even for boards
that don't have UEFI emulation--
UEFI support, if they support
U-boot, it might work
although that means all the standarization
we worked towards, we throw it away.,
Meh!
[Martin:] Yes, I think that's something we need to figure out,
which one do we want to standardize on,
or is that something we want to
leave up to the user.
but yes,
I mean, I think one
I mean, I guess it was
so confusing, because there are like
all those different ways, but
what I really wanted to show is that
things are getting much more
standardized and much better.
So, it used to be really
pretty horrible, and nowadays,
you know, if a platform
uses a modern
U-boot, if it has
upstream support, then it is
actually pretty trivial to add
Debian support, and
I think that wasn't the case
in the past.
I think Ben had another one?
[Ben:] So, I'm
[laughs]
I'm interested to know how much
how much interest there is among the ARM porters
in graphics support.
GPU support, rather
than settling for dumb framebuffers.
How many...
How many of these boards...
Quite a lot of these boards do have HDMI
and I don't know
to what degree they are actually supported.
Are any current projected work
which definitively is using the
GPU on ARM system, and
what's in Debian,
is the Debian base system
does Debian... What's in Debian does not
support GPU.
[Martin:] So, I can't speak for Vagrant...
I have personally done some work
on the Tegra recently,
and nVidia is actually
working upstream on [?]
so I'm really interested in it
but I think that a lot of
the board used, there's money
GPUs I'm not sure would be
...you can get a [laughs]
can we get a microphone for Steve?
[Sledge:] So, on the Mali front,
there has been a lot of haste where we
have been scared of releasing any of this
in any way, shape or form,
without huge
[?]-relays and all that kind of stuff.
So it's not really distributable.
Even, at all.
We are a long way of it being releasable free,
There are packages coming that we
should be getting shippable in non-free
at least if you want to get full acceleration
on your Mali stuff.
Hopefully, that will solve some of the problems
There is a lot, there are a lot of people
in [?] who are very very keen on
supporting it properly, free and upstreamed.
It's a long fight.
It will take a while.
I mean, most of the Mali driver developers
themselves I have spoken to, would love to
make it free. It just comes down
to legal people
being scared.
[Tobi:] Are there some questions left?
[Martin:] Well, thanks for coming!
And there is going to be the ARM BoF
later today, and
Vagrant is also going to talk about his experience
about running a build network
on armhf boards..
Thanks!
[audience clapping]