[hfdb] Scope (Was Re: Grand Unified Hardware Database)

[David Zeuthen]

Hi,

I just subscribed to this mailing list, so I'm a bit late into the
discussions.

Note, my angle on this is related to the freedesktop.org HAL work I'm
doing and somewhat the Project Utopia bits you might have read about
in various blogs. For some starting points see

 http://freedesktop.org/Software/hal
 http://freedesktop.org/~david/guadec2004-hal-and-gnome.pdf
 http://freedesktop.org/~david/HAL-GUADEC2004.sxi

On Sun, 18 Jul 2004, James K. Lowden wrote:
> 1.  Data arrive.  Someone sends mail, fills out a form, contributes an
> existing data store, whatever.  Now it's "here".  

In the future, this might even be automated from GUI bits somewhat in
the fashion that GNOME bug reports are automated through bug-buddy.

> 2.  Zen & Co. massage data into a form consistent with what we already
> have, and consistent with a data model we devise.  (The devising is my
> primary contribution.)  
>
> 3.  Massaged data are loaded in the RDBMS.  

Right. As I see it the universe we're trying to model is so complex
that it actually requires many of the features traditionally used in
RDBMS. But more on that below.

> 4.  Reports are issued in response to queries.  These may be static web
> pages regenerated periodically, compatibility lists included with a given
> distribution (of an OS, driver, application, or hardware), or on-line
> web-based queries.  And others, of course. 

Exactly. Ideally, if I'm ParanoidFreeOSVendor Inc, I can go ahead and
download a dump of the entire database and process it as I see fit. OTOH,
if I'm CasualFreeOSVendor Inc, I might want to access the database (or my
own replica of it) through a web service from the computer system of one
of my customers. This is useful when the customer attach a device that is
unknown to the local system, e.g. the device has shipped after the OS has
shipped.

> I'm guessing our little RDBMS/XML debate is really just a proxy for a more
> basic one: What is the scope of the project?  What, exactly, are we trying
> to track, and for whom?  I have my ideas, some of which are outlined
> above, and quite a few others have been tossed around just prior to our ML
> coming online.  
>
> Maybe a concrete example would help.  I suggest this:  Show me a mock
> tabular report you'd like to see, something you imagine would benefit
> someone we're trying to help, and tell me who that person is.  Just a half
> dozen lines should do.  Or point me to something online and tell me what's
> missing or overmuch in it.  Maybe if we can converge on what we're doing,
> we'll agree on how.  

I think, at least from the HAL projects point of view, we want a lot
of data, indulge some rambling :-)

One thing that we're working on right now are the ''soft properties'' of
hardware; e.g. today we can merge a HAL Free Device Information (.fdi file)
like this one

 <?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- --> 
 
 <deviceinfo version="0.2">
   <device>
     <match key="info.bus" string="usb">
       <match key="usb.vendor_id" int="0x066f">
         <match key="usb.product_id" int="0x8000">
           <merge key="info.category" 
type="string">portable_audio_player</merge>
           <merge key="info.capabilities" 
type="string">portable_audio_player</merge>
           <merge key="portable_audio_player.output" type="string">mp3 
wma</merge>
           <merge key="portable_audio_player.input" type="string">wav</merge>
           <merge key="portable_audio_player.localdb" type="bool">false</merge>
         </match>
       </match>
     </match>
   </device>
 </deviceinfo>

which says that the USB device with VendorID 0x66f and ProductID 0x8000 
is in fact not just a USB storage device, but it's actually a MP3 player.

Now, this is important as the OS sees it the device there's no chance,
apart from unreliable heuristics, that the OS can identify it as a MP3
player capable of playback of MP3 and WMA and recording of WAV. With
this, today, we can actually display a nice icon on the desktop like
this one

 http://freedesktop.org/~david/gnome-vfs-take3-1.png

instead of a dull grey icon :-). We're still working out to define the
names of the various properties and integrating this into the desktop
and applications. There's a few other devices that needs ''soft
properties'', for instance cameras.

All this is important if we want Free Software OS's to meet, and exceed, 
the level of hardware handling that is present in the commercial operating
systems available today such as Mac OS X and Windows XP. It's important
to note that such properties are completely OS independent; it's also
important to note that while this give some impression of ''Making hardware
just work'' it really requires operational and working driver software
on the system to which the device is attached.

In order to solve the second part of the puzzle, e.g. how do we somehow
intelligently find out how to obtain driver software, I think this can
also be partially solved through this proposed database.

Now, one problem of free software OS'es is that while there is driver
software available it's, for one reason or another, not available in
the installed OS. We've probably all tried this; we have to Google for
the driver and most of the time there is one or two projects on sf.net
where we can download the tarball and install it :-). This is not my
idea of a usable operating system.

My proposal for this would be have a OS-dependent name of the driver
in the database. E.g. so from the database I might be able to generate
this HAL .fdi file

 <?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- --> 
 
 <deviceinfo version="0.2">
   <device>
     <match key="info.bus" string="pci">
       <match key="pci.vendor_id" int="0x109e">
         <match key="pci.product_id" int="0x0369">
           <match key="pci.subsystem_vendor_id" int="0x1002">
             <match key="pci.subsystem_product_id" int="0x0003">
               <merge key="linux.driver" type="string">bttv</merge>
               <merge key="freebsd.driver" type="string">bktr</merge>
             </match>
           </match>
         </match>
       </match>
     </match>
   </device>
 </deviceinfo>

and from the OS will be able to know that it needs this driver. Granted,
this example covers a device that is probably available in most Free OS
kernels, but think of other devices (the above is only an example).

So, basically, I think that it would be perfect if the database is
designed in a way so all this data is stored in a convenient extensible
way. It seems to me that a RDBMS with normalised tables is the best 
option, because if a new device type emerges, say a portable video player
we could just introduce a new table PortableVideoPlayer and link that
to the Device table.

Though I've spoke of HAL above, the database should definitely be indepedent
of HAL, but I'd really like to see it support all the features that HAL
can use :-)

I hope I wasn't rambling too much :-)

Cheers,
David