Re: "Calculated and written ECMA-119 tree end differ" under very specific circumstances

[Thomas Schmitt]

Hi,

Valtteri Vuorikoski wrote:
> I'm hitting a "Calculated and written ECMA-119 tree end differ"

Ouch. Just when i am in the process of making a new release.
There is already a new GNU xorriso tarball uploaded.
(The libraries are not yet uploaded because the website makes problems ...)

There was some work done on zisofs during the last development cycle.
A bug was fixed on big-endian machines.
Can you please download, build, and test

  http://www.gnu.org/software/xorriso/xorriso-1.5.4.tar.gz

Do

  tar xzf xorriso-1.5.4.tar.gz
  cd xorriso-1.5.4
  ./configure --prefix=/usr
  make

and use the resulting binary without installation as

  xorriso/xorriso


> Any ideas how I could debug this further?

I will dive into the code to learn more about the error. As the message
says, there must be some difference between planning the ISO layout and
writing the ISO data stream.


> A specific directory containing 101 MPEG-1 files with all-numbers
> names, mounted over NFS.

Ouch again. I haven't used NFS since a decade. I will have to learn again
how to set it up.

(As a side note i wonder whether zisofs can compress any gain out of MPEG.)


> Either disabling zisofs or removing any of
> hardlinks/xattr/md5 makes the operation succeed (or -for_backup,
> but "-acl on" can be removed and the error is still triggered)

That's a peculiar observation. "hardlinks/xattr/md5" trigger the production
of AAIP records. But "-acl" does this too.


> * As far as I can tell, the directory contains neither hardlinks, xattrs
> nor ACLs.

This kills the theory that "-acl" had nothing to work on, but that the
others had some work.

What do you get if you replace the final -commit command by

  -find / -has_xattr -exec get_any_xattr -- -rollback

This should list the files which will get AAIP records attached for
hardlink detection. Like
  ...
  # file: n
  isofs.di="\002\010\003\0036\020\214"
  ...

"isofs.di" records device and inode number of the file in the original
filesystem.

Most AAIP info gets attached only when the ISO is written.
Is the written ISO loadable by xorriso and if so, does it tell something
about the files when you do:

  xorriso -xattr any -indev foo3.iso -find / -has_xattr -exec get_any_xattr --

I see with my single file test:

  # file: .
  isofs.st="1612082097"
  isofs.nt="\001\001\001\377"
  
isofs.ca="\004\000\000\000\040\004\000\000\000u\004\000\000\000\003\001\020MD5"

  # file: n
  isofs.di="\002\010\003\0036\020\214"
  isofs.cx="\000\000\000\001"

The meaning of "isofs." attributes is documented in
  
https://dev.lovelyhq.com/libburnia/libisofs/raw/branch/master/doc/susp_aaip_isofs_names.txt
".st" is a timestamp before image composition. ".nt" records the name
truncation length (here: 255). ".ca" tells where the checksum list is.
".cx" tells the index in the checksum list.


> I can provide the produced .iso file if needed.

This might be of help. Send it directly to  scdbackup@gmx.net .

It would be nice if it was smaller than 119 MB.
Do you need all 101 files to reproduce the failure ?
Do the files have to be large MPEGs (or do small dummy files suffice) ?


Well, it is time for breakfast and then i'll try to find out more about
the warning message and the ways how to provoke it.


Have a nice day :)

Thomas