[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?
From: |
Benoit Leveau |
Subject: |
Re: [Openexr-devel] OpenEXR 1.7 not backward compatible? |
Date: |
Tue, 24 Aug 2010 11:08:21 +0100 |
User-agent: |
Thunderbird 2.0.0.12 (X11/20080213) |
Is it not possible to avoid using long channel or attribute
names in the first place when backward compatibility is an issue?
Somewhere in your code you are assigning a name longer than 31
characters to an attribute or a channel. With OpenEXR 1.6 and
earlier that name gets truncated to 31 characters plus a trailing
null. If you can shorten that name so that it doesn't get
truncated, then the on-disk files will be backward-compatible
without any library changes.
I'm aware that some of our code is using names longer than 31 characters, but
this code is possibly spread on a lot
of projects/tools, and because this has never been an issue until now we've
never tracked all these places.
If we were to manually search for these usage, there is no guarantee we'll get
all of them and if only one channel has a long name,
the exr file won't be readable by any application, breaking the pipeline. It is
safer for us to patch the library to make sure this
never happens.
This is also stated in the ChangeLog, which I would hope is required
reading for any developer switching to 1.7.
I read the changelog while doing the upgrade, but dealing with a lot of legacy
code for different developers I wasn't aware we
were using such long names in a few places. And honestly, it didn't occur to me
that the library was behaving differently until
I really dig into the issue. Finally, OpenEXR was just one among dozen of
third-party libraries we upgraded. Tracking each individual change
wasn't an option, especially since we skipped a few versions for each library.
Adding a backwards compatibility mode goes against the whole point of the
change.
As I said, OpenEXR was just one of the many libraries we upgraded. Luckily,
they didn't apply this motto, and backward compatibility was ensured by most of
them.
We didn't upgrade the ones which required major changes in our code, and may do
the same with OpenEXR (stick with 1.6) if we can't patch 1.7.
Jonathan Litt wrote:
This is also stated in the ChangeLog, which I would hope is required
reading for any developer switching to 1.7.
I'm a bit curious about the fact that this entire time you've been
creating exr files with truncated attribute names
"prmanExrDriverAddDisplayWindowO" and "prmanExrDriverDisplayWindowOffs",
and that has never been a concern? Did those just not get noticed before
they became the de facto names instead of the un-truncated ones in all
of your facility files? To be fair, channel names rather than attribute
names are prime candidates to end up truncated since they often come
from user input (e.g., AOV names) rather than, say, a driver developer.
(Although the new metadata stuff in Nuke makes creating attributes more
accessible to everyone.)
Your situation is a perfect example of why long attribute names are so
badly needed in the first place. Adding a backwards compatibility mode
goes against the whole point of the change. The only way long name
support is going to work is if everyone starts using it, and nothing is
stopping apps from doing their own truncation to prevent the flag. IF a
31-char truncation mode is added to the standard distribution it should
be a runtime API option that defaults to off, and developers should be
strongly encouraged to only enable it upon explicit user request as
opposed to compiling it in. If Nuke or prman other apps/drivers wish to
pass the runtime truncation option to the user, that's great. On the
other hand suggesting that it's ok for developers to turn it off by
making it be an official static option is not in the best interest of
anyone. Of course I'm not talking about companies that use OpenEXR
internally -- I'm talking about companies that ship products with
OpenEXR support.
All that said, it couldn't hurt for this change to be publicized more
loudly in any further 1.7 announcements. :)
-Jonathan
On Aug 23, 2010, at 8:11 AM, Benoit Leveau wrote:
I saw that in ImfHeader.cpp, line 117 (version 1.7.0):
// If an OpenEXR file contains any attribute names, attribute type names
// or channel names longer than 31 characters, then the file cannot be
// read by older versions of the IlmImf library (up to OpenEXR 1.6.1).
// Before writing the file header, we check if the header contains
// any names longer than 31 characters; if it does, then we set the
// LONG_NAMES_FLAG in the file version number. Older versions of the
// IlmImf library will refuse to read files that have the LONG_NAMES_FLAG
// set. Without the flag, older versions of the library would mis-
// interpret the file as broken.
This means that if you're writing an image with a channel name longer
than 31 characters,
- the previous behavior was to truncate the name (readable in all
versions),
- the new behavior is to set the flag and write the long names
(readable in 1.7+ only).
This will require us to either truncate the channel names manually
everywhere we use OpenEXr,
or edit the OpenEXR library so it never sets the flag and truncates
the names manually.
Benoit
Jeff Clifford wrote:
Ah that would explain it. Glad you found the difference.
Cheers,
Jeff.
Benoit Leveau wrote:
Hi,
That 0x400 value comes from the following in ImfVersion.h:
const int LONG_NAMES_FLAG = 0x00000400; // File contains long
// attribute or channel
// names
which also explains the rest of the diff (i.e., long names).
I guess images written with that flag are not supported by old
versions of the library.
I'll try to see which option controls that flag when writing images,
should be easy now.
Cheers,
Benoit
Benoit Leveau wrote:
Hi Jeff,
Thanks for the feedback! It's good to know it works on your side.
We used the configure script that comes with the library, and
didn't use any specific option.
I just tried to use exrheader to compare files generated with 1.6
and 1.7.
A diff of the output gives that interesting result:
4c4
< file format version: 2, flags 0x0
---
file format version: 2, flags 0x400
20c20
< prmanExrDriverAddDisplayWindowO (type int): 0
---
prmanExrDriverAddDisplayWindowOffset (type int): 0
22,23c22,23
< prmanExrDriverDisplayWindowOffs (type box2i): (0 0) - (0 0)
< rmanXMLStatisticsFileName (type string): "x+g"
---
prmanExrDriverDisplayWindowOffsetValues (type box2i): (0 0) - (0 0)
rmanXMLStatisticsFileName (type string): "(XXX" <= 'X' for some
non-printable characters
There's probably something in our code that doesn't initialize that
version/flag.
Benoit
Jeff Clifford wrote:
Hi Benoit,
It may be worth stating what your configure options are?
The reason I say that is that we too rebuilt our internal apps and
PRMan driver against 1.7.0 under Linux64. However we've not hit
upon that message. We're able to use older OpenEXR lib apps (like
Nuke) to read our 1.7.0 images fine.
Jeff.
Benoit Leveau wrote:
Hi,
After re-compiling our renderman display driver with OpenEXR 1.7
libraries on linux 64bit, none of the images we generate
can be loaded into applications that link with older versions of
OpenEXR (e.g. Nuke, in-house tools).
The error I get with our in-house player is:
OpenEXR input exception:: >> Cannot read image file "xxxxx.exr".
The file format version number's flag field contains unrecognized
flags. <<
I get the same exact error message in Nuke 6.0.
If I compare a file generated by 1.7 and 1.6.1 (or 1.4.0), the
two files indeed differ, even in the first few bytes. It seems
that the 1.7 libraries add new header flags,
and this is not safely ignored by old versions.
IMHO, if files generated by 1.7 can't be loaded by current
versions of Nuke, that may greatly affect its widespread usage.
Thanks for your help,
Benoit
_______________________________________________
Openexr-devel mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/openexr-devel
_______________________________________________
Openexr-devel mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/openexr-devel
_______________________________________________
Openexr-devel mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/openexr-devel
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, (continued)
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Benoit Leveau, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Benoit Leveau, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Jeff Clifford, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Benoit Leveau, 2010/08/23
- Message not available
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Benoit Leveau, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Piotr Stanczyk, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Benoit Leveau, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Piotr Stanczyk, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Florian Kainz, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Jonathan Litt, 2010/08/23
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?,
Benoit Leveau <=
- Re: [Openexr-devel] OpenEXR 1.7 not backward compatible?, Benoit Leveau, 2010/08/25