[Octave-bug-tracker] [bug #33420] Wrong constants in wavread.m

[James D. Johnston]

URL:
  <http://savannah.gnu.org/bugs/?33420>

                 Summary: Wrong constants in wavread.m
                 Project: GNU Octave
            Submitted by: j_j
            Submitted on: Sun 29 May 2011 06:19:25 AM GMT
                Category: Libraries
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: j_j
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 3.2.4
        Operating System: Microsoft Windows

    _______________________________________________________

Details:

Wavread.m uses the constant of 32767 to divide the 16 bit signed integer. It
should use 32768. For converting any sized integer to float, you should use a
multiplicative constant of 2^(nbits-1) not 2^(nbits-1)-1  

The correct output range for wavread with any PCM file should be from -1 to
1-2^(nbits-1), not -1-2^(nbits01) to 1.

This likely requires changes for 24 bit as well, I didn't check.

This manifested when trying to take histograms of 16 bit PCM files, and I
caught some negative 1 addresses.




    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?33420>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/