[RFC] Detect other software using embedding area

[Colin Watson]

Every so often people report that booting Windows renders GRUB
unbootable.  This is usually because some Windows application writes to
a sector in the "embedding area" (or boot track), so that application
and GRUB end up fighting with each other on alternate boots.  I've begun
to gather information about these applications, as there don't seem to
be very many of them, and I propose that we do something like the
following to defend ourselves against problems and to try to avoid
causing problems in our turn.

On the applications in question: some of them are "licence managers",
and I'm unsympathetic towards these, since their use of the boot track
is solely to try to enforce their proprietary licences in a way that
can't easily be worked around by uninstallation tools, but the effect of
this is also to impinge upon users' rights to back up and restore their
system without unreasonable amounts of effort.  However, some of the
applications are backup and recovery utilities, and may well have a
better justification for storing data outside a partition, although I
can't easily tell exactly what they're doing.


When I blogged about this recently (and rather unexpectedly ended up on
Slashdot), several people followed up to say that GRUB shouldn't be
using the embedding area because it was never defined to be used for
anything in particular and has no protocol for arbitrating among
conflicts like this.  Sometimes I even replied to them, depending on how
polite they were about it ...  They're not entirely wrong; GRUB is
skating close to the edge of the lack-of-specification here.  However,
all the alternatives have severe problems too, and you get to choose
what set of things you'd like to break:

 * Simply loading core.img from a filesystem using blocklists has
   obvious problems when the filesystem decides to rearrange blocks
   during fsck, defragmentation, etc.

 * Most filesystems don't reserve enough space near the start for us to
   be able to insert core.img safely there (although we should make
   better use of the ones that do; but grub-devel is not a filesystem
   development list and there's only so much we can fix).

 * Most people who suffer from problems like this are on dual-boot
   systems, where the free OS is not the first one to be installed, and
   in that situation it's not unusual for the BIOS to be unable to
   address disk space after that occupied by the first OS.  Automatic
   partitioners can't safely move the first OS to make room for a boot
   partition at the start, so many people will in practice need GRUB to
   be placed before the first partition.

I'd like to avoid having another argument about that here.  This patch
is solely to try to work around the problems with our current setup in a
way that makes as few changes as possible.  Maybe in ten years we'll all
be using GPT and won't have to worry about it.


Some people have also suggested that GRUB should use an error-correcting
code such as Reed-Solomon when embedding core.img.  This is an
interesting idea and it might be worth doing in the future.  However, it
doesn't fully solve the problem at hand because GRUB would still be
overwriting sectors used by these other applications, some of which are
very popular and people actually do want to use.  We won't do ourselves
any favours by appearing to be a poor citizen.


2010-08-31  Colin Watson  <address@hidden>

        * util/i386/pc/grub-setup.c (embed_signatures): New array.
        (setup): When embedding the core image in a post-MBR gap, check
        for and avoid sectors matching any of the signatures in
        embed_signatures.

=== modified file 'util/i386/pc/grub-setup.c'
--- util/i386/pc/grub-setup.c   2010-08-19 11:24:00 +0000
+++ util/i386/pc/grub-setup.c   2010-08-31 10:19:46 +0000
@@ -63,6 +63,33 @@ static const grub_gpt_part_type_t grub_g
 #define grub_host_to_target32(x)       grub_cpu_to_le32(x)
 #define grub_host_to_target64(x)       grub_cpu_to_le64(x)
 
+struct embed_signature
+{
+  const char *signature;
+  int signature_len;
+};
+
+/* Signatures of other software that may be using sectors in the embedding
+   area.  */
+struct embed_signature embed_signatures[] =
+  {
+    {
+      /* ZISD */
+      .signature = "ZISD",
+      .signature_len = 4
+    },
+    {
+      /* FlexNet */
+      .signature = "\xd4\x41\xa0\xf5\x03\x00\x03\x00",
+      .signature_len = 8
+    },
+    {
+      /* FlexNet */
+      .signature = "\xd8\x41\xa0\xf5\x02\x00\x02\x00",
+      .signature_len = 8
+    }
+  };
+
 static void
 setup (const char *dir,
        const char *boot_file, const char *core_file,
@@ -85,7 +112,7 @@ setup (const char *dir,
   char *prefix = NULL;
   char *tmp_img;
   int i;
-  grub_disk_addr_t first_sector;
+  grub_disk_addr_t first_sector, es;
   grub_uint16_t current_segment
     = GRUB_BOOT_MACHINE_KERNEL_SEG + (GRUB_DISK_SECTOR_SIZE >> 4);
   grub_uint16_t last_length = GRUB_DISK_SECTOR_SIZE;
@@ -400,15 +427,95 @@ setup (const char *dir,
   assert (first_block->segment == grub_host_to_target16 
(GRUB_BOOT_MACHINE_KERNEL_SEG
                                                    + (GRUB_DISK_SECTOR_SIZE >> 
4)));
 
-  /* Make sure that the second blocklist is a terminator.  */
   block = first_block - 1;
+
+  if (strcmp (dest_partmap, "msdos") == 0)
+    {
+      /* Check for software that is already using parts of the MBR gap.  */
+      char *embed_signature_check = xmalloc (GRUB_DISK_SECTOR_SIZE);
+      grub_uint16_t extra_sectors = 0;
+
+      for (es = embed_region.start + 1; es < embed_region.start + core_sectors;
+          es++)
+       {
+         struct grub_boot_blocklist *prev_block;
+         grub_uint64_t prev_start;
+         grub_uint16_t prev_len;
+         grub_uint16_t prev_segment;
+
+         if (grub_disk_read (dest_dev->disk, es, 0, GRUB_DISK_SECTOR_SIZE,
+                             embed_signature_check))
+           continue;
+
+         for (i = 0; i < ARRAY_SIZE (embed_signatures); i++)
+           if (! memcmp (embed_signatures[i].signature, embed_signature_check,
+                         embed_signatures[i].signature_len))
+             break;
+         if (i == ARRAY_SIZE (embed_signatures))
+           continue;
+         grub_util_info ("sector %llu is already in use; avoiding it", es);
+         extra_sectors++;
+
+         /* Adjust the last blocklist, splitting it in two if necessary, to
+            avoid this sector.  */
+         prev_block = block + 1;
+         prev_start = grub_target_to_host64 (prev_block->start);
+         prev_len = grub_target_to_host16 (prev_block->len);
+         prev_segment = grub_target_to_host16 (prev_block->segment);
+
+         if (es == prev_start)
+           prev_block->start = grub_host_to_target64 (prev_start + 1);
+         else
+           {
+             prev_block->len = grub_host_to_target16 (es - prev_start);
+
+             block->start = grub_host_to_target64 (es + 1);
+             block->len = grub_host_to_target16
+               (prev_len - (es - prev_start));
+             block->segment = grub_host_to_target16
+               (prev_segment +
+                (GRUB_DISK_SECTOR_SIZE >> 4) * (es - prev_start));
+
+             block--;
+             if (block->len)
+               grub_util_error (_("the post-MBR gap is too fragmented by 
sectors used by other software"));
+           }
+       }
+      free (embed_signature_check);
+
+      if ((unsigned long) (core_sectors + extra_sectors)
+         > embed_region.end - embed_region.start)
+       {
+         grub_util_warn (_("Your embedding area is unusually small.  core.img 
won't fit in it."));
+         goto unable_to_embed;
+       }
+    }
+
+  /* Make sure that the last blocklist is a terminator.  */
   block->start = 0;
   block->len = 0;
   block->segment = 0;
 
   /* Write the core image onto the disk.  */
-  if (grub_disk_write (dest_dev->disk, embed_region.start, 0, core_size, 
core_img))
+  if (grub_disk_write (dest_dev->disk, embed_region.start, 0,
+                      GRUB_DISK_SECTOR_SIZE, core_img))
     grub_util_error ("%s", grub_errmsg);
+  tmp_img = core_img + GRUB_DISK_SECTOR_SIZE;
+  for (block = first_block; block->start; block--)
+    {
+      grub_uint64_t start;
+      size_t size;
+
+      start = grub_target_to_host64 (block->start);
+      size = grub_target_to_host16 (block->len) << GRUB_DISK_SECTOR_BITS;
+      /* The last sector may be incomplete.  */
+      if (! (block + 1)->start && (core_size % GRUB_DISK_SECTOR_SIZE))
+       size -= GRUB_DISK_SECTOR_SIZE - (core_size % GRUB_DISK_SECTOR_SIZE);
+
+      if (grub_disk_write (dest_dev->disk, start, 0, size, tmp_img))
+       grub_util_error ("%s", grub_errmsg);
+      tmp_img += size;
+    }
 
   /* FIXME: can this be skipped?  */
   *boot_drive = 0xFF;


Other comments welcome.

Thanks,

-- 
Colin Watson                                       address@hidden