Re: expand/unexpand: add tests, refactor common code

[Assaf Gordon]

Hello Pádraig and all,

> On Jun 25, 2016, at 07:20, Pádraig Brady <address@hidden> wrote:
> 
> As part of this, or at least before looking at multibyte changes,
> it would be worth considering this proposal for changing the
> unexpand algorithm: http://bugs.gnu.org/23335

The above bug-report addresses this TODO item:
===
unexpand: [http://www.opengroup.org/onlinepubs/007908799/xcu/unexpand.html]
  printf 'x\t \t y\n'|unexpand -t 8,9 should print its input, unmodified.
  printf 'x\t \t y\n'|unexpand -t 5,8 should print "x\ty\n"
===

I haven't yet carefully read the POSIX description (it seems misleadingly 
short),
but a quick test shows many different implementations struggle with this input 
(see below).

So before starting to work on changing the algorithm, I'd like to verify the 
stated expected output is indeed what is expected by posix (or perhaps, as 
there is simply no consensus, this is not a commonly used case and is better 
left as is?).

Thanks,
 - assaf


===

System: AIX
which unexpand: /usr/bin/unexpand
input: 
          78  09  20  09  20  79  0a
           x  \t      \t       y  \n
    

expand -t 8,9 :
          78  09  09  09  20  79  0a
           x  \t  \t  \t       y  \n
    

expand -t 5,8 :
          78  09  20  09  20  79  0a
           x  \t      \t       y  \n

===

System: SunOS 5.11
which unexpand: /usr/bin/unexpand
input: 
          78  09  20  09  20  79  0a
           x  \t      \t       y  \n
    

expand -t 8,9 :
          78  09  20  09  20  79  0a
           x  \t      \t       y  \n
    

expand -t 5,8 :
          78  09  09  20  79  0a
           x  \t  \t       y  \n

===

System: Linux 3.16.0-4-amd64 (Debian 8)
which unexpand: /usr/local/bin/unexpand (coreutils 8.25)
input:  
      78  09  20  09  20  79  0a
       x  \t      \t       y  \n

expand -t 8,9 :
      78  09  09  09  20  79  0a
       x  \t  \t  \t       y  \n

expand -t 5,8 :
      78  09  09  20  79  0a
       x  \t  \t       y  \n

===

System: FreeBSD 10.1-RELEASE
which unexpand: /usr/bin/unexpand
input: 
               78  09  20  09  20  79  0a                                    
               x  \t      \t       y  \n                                    

expand -t 8,9 :
               78  09  09  20  79  0a                                        
               x  \t  \t       y  \n                                        

expand -t 5,8 :
               78  09  09  09  79  0a                                        
               x  \t  \t  \t   y  \n                                        

===

(test code available in http://files.housegordon.org/tmp/test-unexpand.sh)