libcdio-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Libcdio-devel] Invalid/bad directory errors with openSUSE Leap 15.0


From: Pete Batard
Subject: Re: [Libcdio-devel] Invalid/bad directory errors with openSUSE Leap 15.0 ISO
Date: Sat, 26 May 2018 19:17:10 +0100
User-agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0

Hi Thomas,

That's a great analysis!

I'm certainly learning a lot about the origin of these from_733 byte order warnings, which I always wondered about... So thanks a lot for spending time investigating this.

Since I have an account with openSUSE's bugzilla, I'll report the bug to them when I get a chance.

Now, with regards to your question on how libcdio bails out in my environment, please be mindful that I got 2 different types of bailing out depending on the example application I used.

When using 'example/isolist' I got "WARN: Invalid directory stat at offset 510", just like you, and it's only when I started playing with 'example/isofile' that I got "WARN: Bad directory information for docu".

So the way libcdio bails out depends on how you are trying to access the content.

Finally, if one replaces the calls to from_733_with_err() to from_733() in iso9660_fs.c then both the issue with isolist and isofile go away, so that's what I would advocate as a patch. You will still get the warning about the broken byte order, but I don't think we should bail out when that happens:

-----------------------------------------------------------------------
$ ./isolist /d/ISOs/openSUSE-Leap-15.0-DVD-x86_64.iso
Application: openSUSE-Leap-15.0-DVD-x86_64-Build267.2-Media
Preparer   : KIWI - http://opensuse.github.com/kiwi
Publisher  : SUSE LINUX GmbH
System     : LINUX
Volume     : openSUSE-Leap-15.0-DVD-x86_64267
++ WARN: from_733: broken byte order
d [LSN     29]     4096 /.
d [LSN     29]     4096 /..
d [LSN     31]     2048 /boot
- [LSN  58362]    21828 /checksums
- [LSN  58373]      481 /checksums.asc
- [LSN  58374]    65639 /control.xml
d [LSN     53]     6144 /docu
d [LSN     56]     2048 /efi
- [LSN  58407]      615 /gpg-pubkey-307e3d54-5aaa90a5.asc
- [LSN  58408]      972 /gpg-pubkey-39db7c82-5847eb1f.asc
- [LSN  58409]     1059 /gpg-pubkey-3dbdc284-53674dd4.asc
- [LSN  58410]    18092 /gplv2.txt
- [LSN  58419]    35147 /gplv3.txt
d [LSN    397]     2048 /media.1
d [LSN    295]   206848 /noarch
- [LSN  58437]     1250 /readme
d [LSN     49]     6144 /repodata
- [LSN  58438]    99678 /susego.ico
- [LSN  58487]    10247 /susego.png
- [LSN  58493]    11612 /susego.svg
d [LSN     59]   481280 /x86_64
- [LSN  58499]      351 /.treeinfo
-----------------------------------------------------------------------

Regards,

/Pete

On 2018.05.26 17:44, Thomas Schmitt wrote:
Hi,

the disease seems not new. I get similar errors with:
   openSUSE-Leap-42.1-NET-x86_64.iso
   openSUSE-13.1-NET-x86_64.iso

To blame is

   Preparer   : KIWI - http://opensuse.github.com/kiwi

which obviously fiddles with the ISOs after they were produced by the
SuSE version of genisoimage. (Among other peculiarities, the ISOs have
two heads and an EFI System partition between them.)

The damaged directory record in
   openSUSE-Leap-15.0-DVD-x86_64.iso
belongs to file

   /CHECKSUMS.asc

In the older two ISOs it is file /content.asc .

One would have to file a bug to SuSE/Kiwi pointing out ECMA-119, 7.3.3
"Both-byte orders" and 9.1.4 "Data Length (BP11 to BP18)", which prescribe
that the byte count of a data file is to be recorded as first a
little-endian 32-bit number and then a big-endian 32-bit number.

Kiwi obviously only patches the little-endian bytes into the ISO when
it sets the final size of the affected files (which are GPG signatures
probably made after the ISO was finished).
The file in openSUSE-Leap-15.0 bears as size:

   e1  01  00  00  00  00  08  00

where the first four bytes are correct and the next four are wrong.
Probably because only the first four were overwritten by Kiwi, when it
patched in the new size after manipulating the file content.

Since the error is noticed during reading of the file's directory record,
libcdio has not yet learned its name but only the location of the damaged
record.

Currently libcdio reacts quite ungraceful on the situation.
Linux and libisofs tolerate it:

   # mount openSUSE-Leap-15.0-DVD-x86_64.iso /mnt/iso
   $ ls -l /mnt/iso/CHECKSUMS.asc
   -r--r--r-- 1 root root 481 May 16 19:45 /mnt/iso/CHECKSUMS.asc

and

   $ xorriso -indev openSUSE-Leap-15.0-DVD-x86_64.iso -lsdl /CHECKSUMS.asc --
   ...
   -r--r--r--    1 0        0             481 May 16 19:45 '/CHECKSUMS.asc'


Tolerance would probably have to be implemented deep down in
   from_733_with_err (uint64_t p, bool *err)
in
   include/cdio/bytesex.h
which detects the situation and issues the first warning message.


---------------------------------------------------------------------------
We should try to fid out why Pete's libcdio clone bails out with ++ WARN: Bad directory information for docu

whereas mine (quite fresh) reports

   ++ WARN: Invalid directory stat at offset 510

---------------------------------------------------------------------------
Here's he long story how i came to above conclusion

Pete Batard wrote:
++ WARN: from_733: broken byte order

This comes from inline functioni
   from_733_with_err (uint64_t p, bool *err)
in
   include/cdio/bytesex.h
if
   if (uint64_swap_le_be (p) != p) {
which effectively tests whether p as 64 bit byte string complies to
ECMA-119, 7.3.3 "Both-byte orders", which prescribes to store a 32-bit
unsigned integer first as little-endian and then as big-endian byte string.


++ WARN: Bad directory information for docu

Function _fs_iso_stat_traverse() in
   lib/iso9660/iso9660_fs.c
reports this if _iso9660_dir_to_statbuf() in the same file returns NULL.
_iso9660_dir_to_statbuf() calls from_733_with_err() on two occasions
when it looks for the block address and for the size of the file.
Those are the bytes from offsets 2 to 9 and and 10 to 17 (BP3 to BP10,
BP 11 to BP18 in ECMA-119 speak).


docu/RELEASE-NOTES.en.txt

I downloaded the ISO and now look at the directory record of that file
at offset 112228, i see

    cc  f4  02  00  00  02  f4  cc  53  3c  00  00  00  00  3c  53
   204 244   2   0   0   2 244 204  83  60   0   0   0   0  60  83

which says block address 193740 and size 24723 bytes, and looks exactly
like the prescribed byte waste by mirroring.
So it's not that file but probably one of its siblings or a sibling of
its boss directory.

Hrmpf. My clone does not get there. It rather reports:

   ++ WARN: Invalid directory stat at offset 510

Well, "offset" is welcome. But the directory's LBA would have been nice.
The message stems from iso9660_ifs_readdir() in lib/iso9660/iso9660_fs.c .
I augment it by:

-          cdio_warn("Invalid directory stat at offset %lu", (unsigned 
long)offset);
+          cdio_warn("Invalid directory stat at LBA %u , offset %lu",
+                    p_stat->lsn, (unsigned long)offset);

This yields:

   ++ WARN: Invalid directory stat at LBA 29 , offset 510

That would be absolute byte offset 59902. Two bytes further i see

   05  e4  00  00  00  00  e4  05  e1  01  00  00  00  00  08  00

So the specs violation is in the file size field: 0x1e1 != 0x800.

The file goes by the ISO 9660 name "CHECKSUM.ASC;1".
Rock Ridge name is "CHECKSUMS.asc".

xorriso lists it with size 481 = 0x1e1. So the 0x800 alternative is wrong.
The content as extracted by xorriso looks plausible:

   -----BEGIN PGP SIGNATURE-----
   Version: GnuPG v1.0.7 (GNU/Linux)

   iQEVAwUAWvxud7iLL9Q9vcKEAQgmPggAwnGfRxlVpfjjBF07FQS7jQqeOQ47/fcS
   KwHWsxab0uoNNKvWV8QtGOLKSfKxdwDioE+cJU+7ZiPXjrHI7KsCyy+xV6GE75l+
   BEF+Ud4xuKxV7KF1jnG/8eHeVXywvA4Q4i2AyBGcIjrr1a10p0g+GyIt7ueaJJ1i
   1BGGn5AGyi6chE2MAGsBoDk4AxS3yUXKWgF4lGU7H2ZBNiNSFYy3OeRfKzmrLJPG
   C0orEG75pm3hM9/7v3AbnMHN0pGr3NRks/RJREBYGFTsiu7sDQ7RMNLdpa/0xpDn
   hEFI5t+0p/KIuKTISOfiJZDUGtooIF331K37P36hi9q3lvfmQIlTtg==
   =o7aX
   -----END PGP SIGNATURE-----

---------------------------------------------------------------------------

Have a nice day :)

Thomas





reply via email to

[Prev in Thread] Current Thread [Next in Thread]