[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#22517: dd byte count report does not correlate with df byte count re
From: |
Bernhard Voelker |
Subject: |
bug#22517: dd byte count report does not correlate with df byte count report |
Date: |
Tue, 2 Feb 2016 00:20:36 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 |
On 02/01/2016 05:53 AM, Bob Gustafson wrote:
> Description of problem:
>
> I am trying to copy /root, /home from a single disk (running the os) to a
> bigger raid 1 disk pair running in the same chassis.
>
> fedora is the lvm vg_group of the running system, fedora23 is the lvm
> vg_group of the new disks (raid 1, 2TB disks).
>
> address@hidden user1]# lvscan
> ACTIVE '/dev/fedora/swap' [15.57 GiB] inherit
> ACTIVE '/dev/fedora/home' [865.45 GiB] inherit
> ACTIVE '/dev/fedora/root' [50.00 GiB] inherit
> ACTIVE '/dev/fedora23/swap' [15.60 GiB] inherit
> ACTIVE '/dev/fedora23/root' [50.00 GiB] inherit
> ACTIVE '/dev/fedora23/home' [1.75 TiB] inherit
>
>
> Below is the command I'm using to copy over one partition to the raid pair:
>
> address@hidden user1]# dd if=/dev/fedora/root of=/dev/fedora23/root
> 104857600+0 records in
> 104857600+0 records out
> 53687091200 bytes (54 GB) copied, 5639.3 s, 9.5 MB/s
> address@hidden user1]#
>
>
> address@hidden user1]# df -h
> Filesystem Size Used Avail Use% Mounted on
> devtmpfs 16G 0 16G 0% /dev
> tmpfs 16G 392K 16G 1% /dev/shm
> tmpfs 16G 1.8M 16G 1% /run
> tmpfs 16G 0 16G 0% /sys/fs/cgroup
> /dev/mapper/fedora-root 50G 27G 20G 58% /
> tmpfs 16G 84K 16G 1% /tmp
> /dev/loop0 1.9G 3.0M 1.7G 1% /srv/node/swift_loopback
> /dev/mapper/fedora-home 852G 57G 752G 7% /home
> /dev/sdc1 477M 215M 233M 48% /boot
> tmpfs 3.2G 8.0K 3.2G 1% /run/user/42
> tmpfs 3.2G 24K 3.2G 1% /run/user/1000
>
> Note that dd says 54GB were copied over into a disk partition of size only
> 50GB. No errors during the copy. And the results show that only 27G was in
> the fedora partition to start with. (54GB is double 27GB - this may be a clue)
>
> Another example:
>
> address@hidden user1]# dd if=/dev/fedora/home of=/dev/fedora23/home
> 1814978560+0 records in
> 1814978560+0 records out
> 929269022720 bytes (929 GB) copied, 32437.8 s, 28.6 MB/s
> address@hidden user1]#
>
> Taking a look at the data size on the target partitions:
>
> address@hidden user1]# mount /dev/fedora23/root /mnt/rootn
> address@hidden user1]# mount /dev/fedora23/home /mnt/homen
>
> address@hidden user1]# du -h /mnt/rootn | tail -1
> 26G /mnt/rootn
>
> address@hidden user1]# du -h /mnt/homen | tail -1
> 57G /mnt/homen
>
> for the rootn partition, the 26G copied size is consistent with the source
> size.
>
> For the homen partition, the 57G is consistent with the source size.
>
> However, the count given by the dd copies don't match:
>
> In the case of the root->rootn copy it is 54G instead of 27G
> in the case of the home->homen copy it is 942G instead of 57G
>
>
> Version-Release number of selected component (if applicable):
>
> dd is (coreutils) 8.24
>
> How reproducible:
>
> I did two tests - as shown
>
> Steps to Reproduce:
> 1. create raid 1 pair
> 2. copy a 3rd single disk partition to a raid1 pair lvm partition
> 3. do df and compare results.
>
> Actual results:
>
> As shown above in problem narative.
>
> Expected results:
>
> I would think that the sizes reported by dd and df would be comparable, but
> they are not.
>
> Additional info:
>
> See also https://bugzilla.redhat.com/show_bug.cgi?id=1303456
I think there are several misunderstandings here ... which is like
comparing apples and oranges.
a) the MiB numbers are 1024-based, while MB are 1000-based.
b) df(1) and du(1) show statistics or summaries of the data *inside*
a file system, while the file system itself may need some space to
manage the data. And even df/du report different sizes:
http://www.gnu.org/software/coreutils/faq/coreutils-faq.html#df-and-du-report-different-information
Starting with BTRFS, the situation may even be worse (df showing
lots of free space while BTRFS doesn't allow to create - or even
delete - another file).
Thus said: the size of the underlying device is not 100% related to
the sizes inside the file system on that device.
You may get the real size using e.g.
$ wc -c < /dev/sda1
2147483648
and dd(1) will report the same size:
$ dd if=/dev/sda1 of=/dev/null
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB) copied, 1.54335 s, 1.4 GB/s
Have a nice day,
Berny