Re: [hfdb] Scope (Was Re: Grand Unified Hardware Database)

[David Zeuthen]

Hi,

On Thu, 2004-07-22 at 10:19 +1000, Zenaan Harkness wrote:
> On Wed, 2004-07-21 at 16:54, James K. Lowden wrote:
> > On Wed, 21 Jul 2004 Zenaan Harkness <address@hidden> wrote:
> > > I have a device - a "Microsoft Natural Keyboard". I'm using it right
> > > now. How do I add it to the database? 
> > 
> > That's really two questions: What data go in what tables, and how does one
> > move the data into said tables.  
> > 
> > What do you know about this keyboard?  Then let's walk through where each
> > piece belongs.  
> 

My hunch feeling is that we need to consider each device type (printer,
scanner, storage, camera, input, usb hub) in detail to look at what
information is needed to store. 

Second, I think it's good to partition this data into three categories:

 1. Data we extract from the hardware itself in order to uniquely
    identify what we're dealing with; and

 2. Data that is useful for software using the hardware but cannot be
    extracted from the hardware proper; and

 3. Metadata about the hardware, e.g. textual content consumed only by
    human beings for the purpose of giving more information about the
    keyboard

Note that some device types like USB hubs are pretty uninteresting for
the HFDB to deal with since a) they always work with any OS with a
working USB stack; and b) all interesting data can be extracted for the
hardware itself. 

That is not to say they shouldn't be included, we could keep them in the
HFDB and track data e.g. what colour the hub is, but that might not be
terribly useful :-)

So, looking at my keyboard, I'd say:

Data that can be extracted:
===========================
The USB product id is 0x058f, the USB product id is 0x9410 and the USB
device revision is 0x0101. This uniquely identify my device. 

(In some cases, like printers, we need a bit more, like the IEEE1284 ID
string, since vendors erroneously reuse the supposed to be unique keys I
gave above.)

Data that is useful for software:
=================================

My laptop got a nonstandard key on the keyboard is a button with a
'power off' symbol and the keycode is <some_kay_code_value> (I haven't
looked it up yet). So, we might be interested in storing this and one
day export it to HAL in this fashion

 input.keyboard.poweroff_button_keycode =  <some_key_code_value>

(Ideally we need to go through all available software and see where the
effort of proving additional data that cannot be extracted from the
hardware helps.)

(One example is the ACME applet for GNOME that enables the user to
associate actions with multimedia keys on the keyboard - perhaps if the
HFDI exposes data that says keycode 0x123 corresponds to the 'volume-'
we can enhance that piece of software, ACME, so it 'just works' when
plugging in the keyboard.)

Metadata
========

"The keyboard is a from Microconnectors and looks like this

 http://www.microconnectors.com/mackey.html
 (Google cache: 
http://66.102.9.104/search?q=cache:xtSLPOPDKrQJ:www.microconnectors.com/mackey.html++site:www.microconnectors.com+microconnectors&hl=en)

It also features two-port USB HUB."

> OK, I can say the following from my experience:
> 
> * It's a Microsoft Natural Keyboard (may be "... Pro").

This we can usually probe from the hardware.

> * It's a US-English style layout, I think 104 keys (Windows + Menu keys)

Specifically for keyboards, this is difficult to probe from the
hardware; users can also switch language layouts so I think this is not
important to represent (of course, if we can represent it, great, let's
do it).

> * It's got 16 special-function keys across the top, above the Fn keys:
>  - back
>  - forward
>  - stop
>  - refresh
>  - search
>  - favourites
>  - web/home
>  - mail
>  - mute
>  - volume -
>  - volume +
>  - play/pause
>  - stop
>  - prev track
>  - next track
>  - media
> * It's got 3 special function keys above the keypad:
>  - my computer
>  - calculator
>  - sleep

We should store these keycodes as category 2 data.

> * It has three led lights for num-lock, caps-lock and scroll-lock.

Pretty standard for a keyboard, but might be useful to export as
category 2 data.

> * It has a cable that splits near the computer end, with PS/2 and USB
> connectors.
> * It has a 2 port USB hub built in.

This is category 3 data.

> * It has a serial number: 5157703025042, and other stickers underneath.

Probably not generic enough to be useful :-)

> * I've been able to make Linux consoles and X work with it.
> * The PS connector works under Linux kernels 2.4.[246].
> * I've been able to make it work using the USB connector with 2.4.4.

Specifically USB HID and PS2 keyboard/mouse works with virtually every
operating with USB respectively PS2 support. However, for the sake of
argument (since it sadly applies to other hardware) we should store this
somehow.

> * I haven't tried to use the special "multimedia keys".

ACME or xev would be helpful here :-)

I think what we should next is to come up with a schema design for the
database. Perhaps it need not to be more complex that this back-of-the-
envelope design?

We have a table for each physical bus type that contains the properties
we can extract from the hardware. 

 USBDevice

 IEEE1394Device

 ...

The set of properties that make the device unique constitute the PRIMARY
key. This is category 1 data, there shouldn't be any NULL elements.

For each device type we have tables specific to each device type

 Keyboard

 Mouse

 Storage

 Printer

 PortableAudioPlayer

 ...

with category 2 data. Each table got FOREIGN keys into the tables from
a. The PRIMARY key is a combination of said FOREIGN keys and other
useful identifying data (such as IEEE1284 ID for printers).

Many values here can be NULL; for Keyboard we probably want an elements
for each keycode for multimedia keys but for my example above only one
of them will contain useful data.

Finally we tie everything together in a Device table where that
references rows from the tables above (Plural since a device may have
several capabilities (multi-function printers) e.g. three USB
interfaces, one for printing, scanning and storage (card reader)). In
the Device table we also store category 3 data.

Now, this is by design extensible - we can go ahead and add an element
to the Keyboard table if some vendor comes out with a keyboard with a
new multimedia key.

The design also lends itself to data submission; I mean once we have the
design in place we can easily specify what format we like data in. How
this data is submitted doesn't really matter right now - initially we
can write this in emails to Zenaan or define a really simple XML format
or something.

I'm not a big database design myself, it looks like James is, so any
comments? How should we go about specifying the database schema?

Eventually I'd like to automate this as part of the Project Utopia bits
we're working on for GNOME. The thing is, the HAL project was
specifically designed this way - we basically got all the necessary
category 1 bits in place and (very few) HAL device information files for
the category 2 bits 

(such as this particular card reader is for compact flash so we export a
property called storage.media=compact_flash and actually GNOME-VFS takes
advantage of this and renders a nice icon, see 
http://freedesktop.org/~david/gnome-vfs-take3-2.png).

We can also start really slow and look at one device at a time, I
suggest we start looking at necessary category 1 and 2 data for storage
devices and portable audio players.

Once we have successfully shown that we can do this for the ''soft
properties'' e.g. data used only in the desktop to e.g. render icons
(Compact Flash) or functionaly (e.g. Portable Audio Player) we can move
into the more hardcore things such as 'Does this device work on the
Linux kernel X.Y.Z' or use the 'FooBar' driver for FreeBSD 4.x and
'BazBat' driver on GNU/Hurd.

With regards to the concerns raised by Richard of aiming too high and
starting slow, I don't know. Thing is, we really need all the salient
features on an RDBMS system as James and Zenaan have pointed out. And if
my back-of-the-envelope database design above is just somewhat correct
it shouldn't be that difficult to implement IMHO.

Another thing, is that I think there is also a demand for the HFDB since
GNOME have stated in their roadmap that they are interested in utilizing
HAL (as an optional dependency) and thus the information that the HFDB
can provide. And I think that is great, so let's go ahead and build it!

Cheers,
David

Note: I'm away all weekend so I can't respond to mail before late sunday
CET.