Re: some concern about the fix of " tail: consistently output all data for truncated files"

[Pádraig Brady]

On 07/11/16 07:18, Lian, George (Nokia - CN/Hangzhou) wrote:
> From: Lian, George (Nokia - CN/Hangzhou)
> Sent: Monday, November 07, 2016 1:17 PM
> To: 'address@hidden' <address@hidden>
> Cc: Zizka, Jan (Nokia - CZ/Prague) <address@hidden>; Zhang, Bingxuan (Nokia - 
> CN/Hangzhou) <address@hidden>; Li, Deqian (Nokia - CN/Hangzhou) 
> <address@hidden>; Bao, Xiaohui (Nokia - CN/Hangzhou) <address@hidden>
> Subject: some concern about the fix of " tail: consistently output all data 
> for truncated files"
> 
> 
> Hi,
> 
> There are a fix about truncated for tail application of coreutils,
> 
> The change messages is "tail: consistently output all data for truncated 
> files",
> 
> Now I have some concern for this case
> 
> 1)      I suppose Truncate maybe not always truncate to ZERO, for example, 
> sometimes a file maybe only has been truncate the bottom 100 lines of 1M 
> lines , in this case , SEEK from head will be confused by the users.
> 2)      Sometimes truncated maybe not really happen even "stats.st_size < 
> fspec->size" is meet,  for example, a network based filesystems, for it's 
> current implementation, it will active the read data-cache first and then 
> update the meta-data cache, it is not an atomic operation, I suppose it is 
> reasonable that the "tail application" will read the data-cache with new 
> updated data, but get stat size with old value, in this case, the tail assume 
> it is truncated and dump the data from the begin, it also make more confuse 
> to users.
> 3)      As the comments in your changes, the stat size is bigger thand the 
> length has read, it also maybe happ truncate.
> 4)      Why can't get size of fstat first, and read not more than the size of 
> fstat during the tail_forever loop?
> 
> Could you please share your comments for the above concerns?
> Hi,
> 
> Add one more suggestion, if we have not a perfect solution to consider all 
> the case of truncate, could we add an option to tail, such like tail 
> -no-truncate
> If tail run with this option, than application not consider any truncate case.
> 
> For example, I suppose syslog output file will not have any truncate case in 
> our environment, then the tail could use the option to avoid the 
> mis-truncated case?

Note for case 2) above, we only update fspec->size _after_ the read,
so I'm not sure how practical the race with reading a _smaller_ st_size after 
that is?
I.E. the heuristic is fairly good I think,
so an option may be overkill.
We'd have to see a demonstratable issue to consider such an option.

thanks,
Pádraig