meetings-archive.debian.net/.../bugsdebianorg_Database_Ho.webm

maintains still some of the flat files in our system

that should increase speed

the flat files of our system

that my primary motivation

for giving a talk was

to try to force myself

into actually doing
to try to force myself

into actually doing

the work that I'm going

talk about in this talk [laughts]

????

lightly successful

so I may introduce you some of them, but???

become too much

of the professional procrastinator

to being influenced by even giving a talk

Let's see how this work out

The first couple of things I'm going to talk about

are just some general bugs statistics

I'll enjoy showing some plots

I'm going to introduce the basic arquitecture of debbugs

just to backup, debbug is

the system behind bugs are debian debug

so if you file the bug

you fix the bug

you wonder where bugs exists

you interact with that bugs

????

I'm going to talk about some of the new features

I'm going to talk about some planned features

specially features that won't happen If I suddenly

have two or three more people would help

and I hopefully would pleased with

of you or people who are missing

this talk online

or who may watch this talk recording later

to assist me in implementing some of these features

I'm a nice person

we are nice people

and I'd like to see any pro hackers

to see a css hacker or javascript hackers

or people who writes documentation helps

the goal of the BTS is to

report bugs

track the evolution of bugs

fix bugs

and hopefully reduce the impact of bugs

???

this is how many bugs do we have versus time

as you can see our bug growth is

roughly linear over time

and actually decreasing slowly but

that a huge number of bugs

people like to track exactly how many bugs we have

???

fun contest

guessing when the ticket bug will be filed fo ar example

the 760 000

bug will be filed

at I think

on September 2

and the 800 000 will be filed almost a year from now

on September 13

observing the linear progression

maintains itself

??? enjoys that but he is not here

??

Anyway that's to show the bug reporting rate

We average roughly have a 42 bugs found by day

as you see, that's a huge number bugs

This is the bug closing graph

It's actually taking not bug closures. It's actually bugs being archetyped ¿?

but for the most part it's aproximated

bugs closure rate ???

So, we close roughly 95 bugs per day

So from that you could imagine that the bug system is gaining

50 or so much bugs everyday that are not going to be fixed

Further than fortunately in this graph you can see that

the bug closing rate is decreasing

in contrast with the bug reporting rate awesome increasing

something that i've seen

in previous post I'v made on my blog

this is actually kind of disturbing

I'm not sure what that means for Debian as a whole

what means anything but

I'd much rather seen the overall rating increasing

than decreasing

In this graph you are familiar with

this is RC bugs from the bug ??

lukely the RC bugs are those that matter for the next relase are decreasing

getting in line for a new release there

??

Ok. So those is enough on graphs.

Now I'm going to talk about

debbugs system and how it works

debbugs two main components

there is a mail backend

wich is what interact with any email

submits bugs ??

or a bug number in bugs at ??

Next system on deb ??

wich has and old files and process you email ¿?

The other aspect in debbugs is a web project

That is what displays information on what bugs are on wich packets

and the bug ??

and it's also here

on to another machine

on beach so that ?? faster ??

that bugs interact with dak

wich is a software wich is responsable from maintaining the ??

so duk tells debbugs

wich who maintains wich packages

so debbugs know who is ??

??

It also tells debbugs wich packages are in wich suite

and wich arquitecture

so debbugs can calculate wether a bug is present in a particular suite

for example if the bug is fixed on stable or whether it's fixed in testing oin r stable

what was calculated by previous ??

britney also

is the testing migration

software that migrates software from unstable

to testing

it uses information from the ¿buxtehude? as well

it regards to wether a package is becoming more buggy

or less buggy

by operating

the actual ?? that does that is

???

and it provides a list of bugs

and also does the RC bug graphs

but ir provides ????

debbugs ?? is like this

mail comes in

there is spam processing that happens on the first step ??

??

???

is primary responsable from keeping bugtrack system free of spam

and these spamming

?? the little bit spam that actually make its way ??

??

is not a great job

after that mails been despammed

then, it goes to processall

and sub process

is responsable from handleing

emails that ? submit

and emails that ?? bug numbers

like this ???

service is responsable from handleing ??

so any email you send ??

now with the advent of

control at submit¿?

????

this diagram is getting a little bit bluried. It's actually an abstraction ??

???

then all of the information is stored in flat files

and a db-h ? wich has

small hash functions split ??

and is indexed with a couple of flat files indexes

and then a cgi scripts use both the indexes and the flat files system to display bugs

so that's how

debbugs worked before I started working on it

The current plan is to add on and basically replace this indexes

with a databse layer

and so I'm going to keep part of the flat files just because that's a well tested system

there are lots of things already parts of flat files ¿?

and I don't know how to deal with them ¿?

and add on top a postgresql based database

that, the cgi scripts ?? in order to display information ??

this will help both, increase the speed you get results back

you are looking and ??

and also enable you to do more complicated things like

?? bugs that actually affects a particular version

without waiting for huge amounts of time

for a query to complete

so for example, you want to look at all security bugs

wich affects unstable

well that's actually a really hard query to do

without a database layer

so that's one of the major things a database to do ??

so the script that actually handles loading things in the database is called dbugloader.sql

Debbugs is run in parallel

but perl has recently come quite a wise

in handling databases

Most everybody has adopted

the perl ?? by using DBIx

in order to talk to databases

very successful database abstraction

Anybody who has ever written code in DBIx

knows that it is extremely tedious

to do joing and complicated statements

?? writing sql and scaping and etc.

using place orders

?? you have to keep track them

DBA has classes and extensions

that bloms together a huge number of

?? into a really concurrent database abstraction service

where if you give it your schema

It would build classes and enable you to talk to ?? result groups

from your database

It's a complete system

you can actually write a schema entirely in DBIx class

that you can then to SQL alike

You can convert into a PostgreSQL, mySQL, etc

In this case I'm interested in

In write for PostgreSQL

that actually will be the database package?

there may be an option eventually to use SQL Lite for testing but

my primary goal is to deploy PostgreSQL

I'm also in using

a bunch of classes that are specific to Debian

for example the Debian Version Extension to PostgreSQL

that enables you to sort by Debian's version

That's extremely important to DBI ¿?

And that is something that handles very well in PostgreSQL

What I've actually done is rewrite ?? directing SQL

and DBIx class has an extension called Schema Loader

which handles converting the SQL schema

into the class decorations for DBIx class automatically

so you just write plain old SQL like you used to

and it automatically creates all the database related

Plurar classes that you used to talk results from database

There is another module which handles deployment

it can do automated upgrades from

different schema revisions

so as you change your schema

it handles doing both upgrades

and new installs of the new schema

wich you can also do downgrades

and yo can do others things

in addition to just execute SQL alter statements you can also run

¿pro code? or anything else you want at the database

IH operations¿?

so that enables much easier changes to the schema in the future

and finally the actual module in Debbugs ???

is DebBugs DB

and so all of the database interactions classes in Debbugs ¿?

and so this is the .....

It's kind of complicated but this basically tracks

all of the bugs relationships

It tracks who correspond with the DBs

has all this source package versions imaginary packae versions

version depences show for example when you uploaded version from a previous version

this ables all that to be tracked

Taken the DQL schema as

as inspiration but unfortunately

the debbugs ????

classes but I think it makes sense so

Anyway if somebody uses PostgreSQL genius

or an SQL hacker and is interested in

maybe offering suggestions right ???

¿talk about?

Is actually pretty easy

just called debbugs SQL bugs

uploads them

There are two different part of bugs in debbugs

There is the one that you can actually modify

and there are the archive bugs

so this handles join both with both sets of bugs

can also load versioning information

this logs wich packages are depending on the ones

and de Debian ¿? the architecture

and source version

So, the SQL is actually working

?? sql query

which you also write this easy in a DBIx class

let me show you that this actually works

???

I can run the SELECT statement wich is selecting the count of bugs WHERE

which has been modify since June or July

which are not done and

and which are

which are done and which there are an outer set

and the answer is

521

as you can see that is a full load of all the bugs in Debian

The actual SQL query executes very quick

??

I hopped to have more of this done by the time of this talk but

There still is a lot of work that's needed

The bugs file currently are not loaded

the bugs files are ??

That's needed to enable full text searching index

It also currently doesn't do ??

thats ?? faster loading ??