[Info-mtools] Bug: mformat does not format FAT32 correctly

[Alain Knaff]

Hi,

First, thanks for your patch, and sorry for the slow response. When I
got your mails, I was rather busy, and eventually they ended up below
the pile...

On 11/08/18 12:45, Pali Rohár wrote:
[...]
> On Tuesday 12 December 2017 10:41:26 Pali Rohár wrote:
>> On Monday 04 December 2017 19:29:53 Pali Rohár wrote:
>>> 2) mformat refuse to create FAT32 if disk size is less then 257 MB.
>>>
>>> Test to reproduce (-F means to format as FAT32):
>>>
>>>   $ truncate -s 100M fat.img
>>>   $ mformat -F -h 64 -s 32 -t 100 -m 0xf8 :: -i fat.img
>>>
>>> mformay show just error message:
>>>
>>> Too few clusters for this fat size. Please choose a 16-bit fat in your
>>> /etc/mtools.conf or .mtoolsrc file
>>>
>>> Fix is to set size of cluster (in number of sectors) to 1. So mformat
>>> parameter: -c 1

Exactly.

>>>
>>> It is pity that mformat cannot calculate cluster size for smaller FAT32
>>> disks (when size is less then 257 MB). Note that mformat can calculate
>>> it correctly for FAT16 (when -F is not used).

As some Microsoft documents (the cited knowledge base article) claimed
that 4KB clusters where the default minimum size, this was used by
default (... while still allowing to override it using -c if wanted).

[...]
>>>     switch(abs(fat_bits)) {
>>> ...
>>>             case 32:
>>>                     Fs->cluster_size = 8;
>>>                     /* According to
>>>                      * 
>>> http://support.microsoft.com/support/kb/articles/q154/9/97.asp
>>>                      * Micro$oft does not support FAT32 with less than 4K
>>>                      */
>>>                     return;
>>> ...
>>>
>>> It means that cluster size is hardcoded to 8 sectors for FAT32. This is


>>> also reason for problem 3). For FAT16 there is autodetection code. I do
>>> not know what that comment mean

Really? Or are you joking? :-)

It gives the web address of a document published at Microsoft's support
site. This document contains a sectence which could be understood to
mean that FAT32 filesystems should not have cluster sizes less than 4K
(i.e. 8 sectors).


> or what it try to refer as link is dead.

Indeed, they moved it to a different address:
https://support.microsoft.com/en-us/help/154997/description-of-the-fat32-file-system

The relevant sentence is:
"The FAT32 file system allows for a default cluster size as small as 4 KB"

Ok, so this is the _default_ size, but smaller sizes are apparently
permissible.


>>>
>>> So I would like to know, why is hardcoded 8 sectors as cluster size for
>>> FAT32?

Actually, in mtools 4.0.18 this was not hardcoded, but overrideable
using the -c option, as you noticed yourself in your earlier comment.

I'm not sure why you disabled this possibility in your patch, or maybe
that was just some leftover from earlier testing.

>>> I suspect this is a bug which needs fixing. At least to have
>>> working mformat in FAT32 mode for small disk (or disk images).

I've now implemented the table you suggested (starting with 1 and going
to 64), but I still kept the possibility to override with -c

>>
>> Also please note that current versions of Windows (probably since 2000)
>> allow user to specify cluster size in Format dialog also for FAT32. And

Exactly. And that's why I re-enabled the -c option.

>> default value depends on disk size, it is not hardcoded to 8 (=4kB).
>>
>> See also:
>> https://support.microsoft.com/en-us/help/192322/description-of-default-cluster-sizes-for-fat32-file-system
>> https://support.microsoft.com/en-us/help/140365/default-cluster-size-for-ntfs--fat--and-exfat

Oddly, these leave out the step with cluster_size = 1 but start with 4K
(8 sectors)

>>
>> So I would really suggest to change above hardcoded value 8 to either
>> mapping table or use similar algorithm as is already present for FAT16
>> in mformat.c
>>
>>> Because, in Microsoft FAT32 specification (fatgen103.doc) on page 20 for
>>> FAT32 is written calculation table. For disk size there is cluster size
>>> (in number of sectors), see table below:
>>>
>>> 32.5 MB - 260 MB   cluster_size =  1
>>>  260 MB -   8 GB   cluster_size =  8
>>>    8 GB -  16 GB   cluster_size = 16
>>>   16 GB -  32 GB   cluster_size = 32
>>>   32 GB -   2 TB   cluster_size = 64

Indeed, fatgen103.doc starts with cluster_size = 1.

>>>
>>> Note that in that Microsoft FAT specification is written: "The rest of
>>> this section is totally specific to drives that have 512 bytes per
>>> sector.". So there is need to recalculate values when (logical) sector
>>> size is not 512 bytes.
> 
> In attachment is a patch for mtools which calculates FAT32 cluster size
> according to that Microsoft specification. With that patch I'm able to
> correctly format small FAT32 disks (below 100M) and also big disks (2TB)
> and even also very huge disks 16TB with 4k sectors.
> 
> Examples:
> 
> small 100 MB FAT32 disk image
> $ truncate -s 100MB /tmp/fat
> $ mformat -F -h 64 -s 32 -t $((100-1)) :: -i /tmp/fat
> 
> big 2 TB FAT32 disk image (this is maximal size for 512 byte sectors)
> $ truncate -s 2TB /tmp/fat
> $ mformat -F -h 64 -s 32 -t $((2*1024*1024-1)) :: -i /tmp/fat
> 
> very huge 16 TB FAT32 disk image with 4k sectors
> $ truncate -s 16TB /tmp/fat
> $ mformat -F -M 4096 -h 8 -s 32 -t $((16*1024*1024-1)) :: -i /tmp/fat
> 
> Everything is correctly detected and mounted on Linux.
> 

This is now part of the development version of mtools, to be released in
a week or so.

Thanks,

Alain