Re: expand/unexpand: add tests, refactor common code

[Pádraig Brady]

On 27/06/16 06:17, Assaf Gordon wrote:
> Hello Pádraig and all,
> 
>> On Jun 25, 2016, at 07:20, Pádraig Brady <address@hidden> wrote:
>>
>> As part of this, or at least before looking at multibyte changes,
>> it would be worth considering this proposal for changing the
>> unexpand algorithm: http://bugs.gnu.org/23335
> 
> The above bug-report addresses this TODO item:
> ===
> unexpand: [http://www.opengroup.org/onlinepubs/007908799/xcu/unexpand.html]
>   printf 'x\t \t y\n'|unexpand -t 8,9 should print its input, unmodified.
>   printf 'x\t \t y\n'|unexpand -t 5,8 should print "x\ty\n"
> ===

I think the second command is wrong there actually?
Surely it should print "x\t\t y\n"

> I haven't yet carefully read the POSIX description (it seems misleadingly 
> short),
> but a quick test shows many different implementations struggle with this 
> input (see below).
> 
> So before starting to work on changing the algorithm, I'd like to verify the 
> stated expected output is indeed what is expected by posix (or perhaps, as 
> there is simply no consensus, this is not a commonly used case and is better 
> left as is?).

Interesting.
To me the Solaris output is most correct since it does minimal translation,
however the coreutils output will behave the same with tabstops at 8 and 9.
I.E. we use a tab rather than the existing space to fill the single width space.
So ideally we'd have the Solaris output, but I wouldn't jump through any hoops,
or slow down the code to achieve it.

thanks!
Pádraig.

> ===
> 
> System: AIX
> which unexpand: /usr/bin/unexpand
> input: 
>           78  09  20  09  20  79  0a
>            x  \t      \t       y  \n
>     
> 
> expand -t 8,9 :
>           78  09  09  09  20  79  0a
>            x  \t  \t  \t       y  \n
>     
> 
> expand -t 5,8 :
>           78  09  20  09  20  79  0a
>            x  \t      \t       y  \n
> 
> ===
> 
> System: SunOS 5.11
> which unexpand: /usr/bin/unexpand
> input: 
>           78  09  20  09  20  79  0a
>            x  \t      \t       y  \n
>     
> 
> expand -t 8,9 :
>           78  09  20  09  20  79  0a
>            x  \t      \t       y  \n
>     
> 
> expand -t 5,8 :
>           78  09  09  20  79  0a
>            x  \t  \t       y  \n
> 
> ===
> 
> System: Linux 3.16.0-4-amd64 (Debian 8)
> which unexpand: /usr/local/bin/unexpand (coreutils 8.25)
> input:  
>       78  09  20  09  20  79  0a
>        x  \t      \t       y  \n
> 
> expand -t 8,9 :
>       78  09  09  09  20  79  0a
>        x  \t  \t  \t       y  \n
> 
> expand -t 5,8 :
>       78  09  09  20  79  0a
>        x  \t  \t       y  \n
> 
> ===
> 
> System: FreeBSD 10.1-RELEASE
> which unexpand: /usr/bin/unexpand
> input: 
>                78  09  20  09  20  79  0a                                    
>                x  \t      \t       y  \n                                    
> 
> expand -t 8,9 :
>                78  09  09  20  79  0a                                        
>                x  \t  \t       y  \n                                        
> 
> expand -t 5,8 :
>                78  09  09  09  79  0a                                        
>                x  \t  \t  \t   y  \n                                        
> 
> ===
> 
> (test code available in http://files.housegordon.org/tmp/test-unexpand.sh)