From MAILER-DAEMON Mon Oct 05 09:55:23 2020 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1kPQxQ-0006eb-S0 for mharc-bug-ddrescue@gnu.org; Mon, 05 Oct 2020 09:55:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56272) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPD2Y-00076s-TB for bug-ddrescue@gnu.org; Sun, 04 Oct 2020 19:03:38 -0400 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21118) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPD2V-0006Of-C8 for bug-ddrescue@gnu.org; Sun, 04 Oct 2020 19:03:38 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1601852609; cv=none; d=zohomail.com; s=zohoarc; b=VBnkDhGj5KDTl70WVdyqcmHpKZdH3O73VVdqwLoCGWrHwTMQHENOVIPMExcL3THHFgVxYSWZtcHPFK6Uybnp6918Nttjxix0awuiLitxFlBN55kn9DS502dQ3tZoXhocySnRx/qTvC63Fob4oWJ7uxOwHKfma4xOOWJ5jDd3R1Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601852609; h=Content-Type:Date:From:MIME-Version:Message-ID:Subject:To; bh=mxaFDh8/xfjlJCJvD/lPgT4rHeWRdNLv3sV9hbLSoY4=; b=VrJlagq6hUE1gHruID3Zm0SK09b+U4FahScOy3dKqyhtm5kj4G6PYAsuwNtqUA8qWOwlsWbg+SzxbbznpZQlyr2LGGbTo3oj4sdOLKcC9eYwBVbRHEkm3mHihqlibpCWADP0p1m3gJb/Lu47G/pXCKHlsWNR058zJCPvRCW/ODk= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass smtp.mailfrom=toddb@nvr.com; dmarc=pass header.from= header.from= Received: from archive2.local (c-71-193-200-84.hsd1.or.comcast.net [71.193.200.84]) by mx.zohomail.com with SMTPS id 160185260764079.25521889173706; Sun, 4 Oct 2020 16:03:27 -0700 (PDT) To: bug-ddrescue@gnu.org From: Todd Brunhoff Message-ID: <83a0d257-e3aa-58a2-f047-f7ea1fcd201c@nvr.com> Subject: a story of a ddrescue failure Date: Sun, 4 Oct 2020 16:03:27 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 Content-Language: en-US X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=toddb@nvr.com; helo=sender4-of-o51.zoho.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/04 19:03:31 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 05 Oct 2020 09:55:10 -0400 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: bug-ddrescue@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Bug reports for ddrescue, data recovery tool." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Oct 2020 23:03:39 -0000 This weekend I spent my time recovering files from my brother's My Passport= Ultra, a 2TB spinning disk that failed on his windows laptop after about 4 years of travel around= the world. I tried four attempts at recovery, the second of which was ddrescue which, sadly, made t= hings worse.=C2=A0 I'll describe the four here in hopes of improving things for the future in ddres= cue. Attempt #1: I have long used dd to recover file systems, mostly linux, by setting bs=3D= 512 which is generally the sector size. This usually copies the most data because nothing is lost = when a large read fails because one sector went bad. This was able to copy the partition tabl= es amidst lots of errors.=C2=A0 After that initial 1.5MB it appeared to be copying the rest o= f the disk successfully showing only 2256 bad sectors out of 1.5TB over 20 hours or so. When plugging in the disk, the erros began with this. Oct 02 13:21:28 archive1 kernel: sd 6:0:0:0: [sdg] tag#0 FAILED Result: hos= tbyte=3DDID_ERROR driverbyte=3DDRIVER_OK cmd_age=3D0s Oct 02 13:21:28 archive1 kernel: sd 6:0:0:0: [sdg] tag#0 CDB: Read(10) 28 0= 0 00 00 00 00 00 00 08 00 Oct 02 13:21:28 archive1 kernel: blk_update_request: I/O error, dev sdg, se= ctor 0 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0 Oct 02 13:21:28 archive1 kernel: usb 4-3: reset SuperSpeed Gen 1 USB device= number 2 using xhci_hcd As the copy progressed, this became the pattern every 5 seconds: Oct 02 13:41:47 archive1 kernel: buffer_io_error: 159019 callbacks suppress= ed Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288589, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288590, async page read Oct 02 13:41:47 archive1 kernel: Buffer I/O error on dev sdg1, logical bloc= k 288591, async page read But at that point I had begun to read other articles about what not to do (= including not copying 512 bytes at a time), and the manual for ddrescue, and started extracting 10MB = every 10GB and piping it into 'od -Ad -tx1z'. This showed that while dd was happy with the reads, t= he entire 1.5TB read so far was all zeros. So I killed that, and started ddrescue. Attempt #2 Ddrescue seems to be well written with lots of strategy and options. The do= c reads a bit like rsync, where you end up starting with the examples in the middle of the doc. After= doing a power cycle on the drive, which started out with the same errors as the first ones above. = I started with this ddrescue --sparse -i30GiB /dev/sdg sdg.bin sdg.mapfile ... so that I could avoid the start of the disk, which is where I thought m= ost of the errors fell. ddrescue made some progress, but continuous errors looked like this: Oct 03 16:01:26 archive1 kernel: usb 4-3: reset SuperSpeed Gen 1 USB device= number 14 using xhci_hcd Oct 03 16:01:26 archive1 kernel: sd 7:0:0:0: [sdg] tag#0 FAILED Result: hos= tbyte=3DDID_ERROR driverbyte=3DDRIVER_OK cmd_age=3D0s Oct 03 16:01:26 archive1 kernel: sd 7:0:0:0: [sdg] tag#0 CDB: Read(10) 28 0= 0 e8 df 86 78 00 00 80 00 Oct 03 16:01:26 archive1 kernel: blk_update_request: I/O error, dev sdg, se= ctor 3906963064 op 0x0:(READ) flags 0x80700 phys_seg 16 prio class 0 Oct 03 16:01:26 archive1 kernel: usb 4-3: reset SuperSpeed Gen 1 USB device= number 14 using xhci_hcd Oct 03 16:01:26 archive1 kernel: sd 7:0:0:0: [sdg] tag#0 FAILED Result: hos= tbyte=3DDID_ERROR driverbyte=3DDRIVER_OK cmd_age=3D0s Oct 03 16:01:26 archive1 kernel: sd 7:0:0:0: [sdg] tag#0 CDB: Read(10) 28 0= 0 e8 df 87 08 00 00 78 00 Oct 03 16:01:26 archive1 kernel: blk_update_request: I/O error, dev sdg, se= ctor 3906963208 op 0x0:(READ) flags 0x80700 phys_seg 15 prio class 0 but eventually ended with this after about 15 minutes. Oct 03 16:01:15 archive1 kernel: usb 4-3: reset SuperSpeed Gen 1 USB device= number 13 using xhci_hcd Oct 03 16:01:20 archive1 kernel: usb 4-3: device firmware changed Oct 03 16:01:20 archive1 kernel: usb 4-3: USB disconnect, device number 13 Oct 03 16:01:20 archive1 kernel: scsi 6:0:0:0: rejecting I/O to dead device Oct 03 16:01:20 archive1 kernel: blk_update_request: I/O error, dev sdg, se= ctor 62934352 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 I tried power cycle and the same ddrescue command (using the same map file)= it failed in about 3 minutes with this: Oct 03 16:23:27 archive1 kernel: usb 4-3: new SuperSpeed Gen 1 USB device n= umber 26 using xhci_hcd Oct 03 16:23:33 archive1 kernel: usb 4-3: device descriptor read/8, error -= 110 Oct 03 16:23:33 archive1 kernel: usb 4-3: new SuperSpeed Gen 1 USB device n= umber 26 using xhci_hcd Oct 03 16:23:38 archive1 kernel: usb 4-3: device descriptor read/8, error -= 110 Oct 03 16:23:38 archive1 kernel: usb usb4-port3: unable to enumerate USB de= vice So I gave up on that, since the drive/driver was giving up the ghost. Attempt #3 I had to switch to win10. Popular mechanics had a pretty good article (https://www.popularmechanics.c= om/technology/gadgets/how-to/a3086/hard-drive-recovery/) which, for software, pointed at EaseUS, Recuva and Prosoft Data Rescue. I f= irst tried Prosoft, which was immediately able to find files. The price of $19 seemed good but after poking around and findin= g that the pro version was $399, it felt like an upsell. Attempt #4 The last effort was with EaseUS. This was pretty good, found files immediat= ely, and had a clear policy of $99/year and $67/mo, cancel anytime. So I grit my teeth, and signed up, with an entry on my cal= endar to cancel in 3 weeks. The UI was very bare. It found all of the files plus a huge number of delet= ed files over a period of about 12 hours. I started the restore, going from a usb->sata to a usb->sata 1TB extra drive I had. It i= s still running and look like it will finish in about 12 hours. Conclusion: There's something about both prosoft and easus where they know how to avoid= or suppress errors from the drive and still extract the files, clearly losing much of the directory structure. If ddrescue coul= d figure out that algorythm of avoiding the errors, I would it as my first choice. Todd Brunhoff Media Architect Portland, OR From MAILER-DAEMON Mon Oct 12 14:11:46 2020 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1kS2IU-0000dc-MN for mharc-bug-ddrescue@gnu.org; Mon, 12 Oct 2020 14:11:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46956) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kROiw-0004d6-9X for bug-ddrescue@gnu.org; Sat, 10 Oct 2020 19:56:26 -0400 Received: from smtp3-g21.free.fr ([2a01:e0c:1:1599::12]:10655) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kROiu-0001OY-ES for bug-ddrescue@gnu.org; Sat, 10 Oct 2020 19:56:26 -0400 Received: from [10.137.0.11] (unknown [90.3.152.142]) (Authenticated sender: villette.fr@free.fr) by smtp3-g21.free.fr (Postfix) with ESMTPSA id A990913F86F for ; Sun, 11 Oct 2020 01:56:19 +0200 (CEST) To: bug-ddrescue@gnu.org From: =?UTF-8?Q?Fran=c3=a7ois_Villette?= Subject: gddrescue ssd/nvme questions Message-ID: <222fba1f-c861-5c56-eacd-b32a86a5070d@free.fr> Date: Sun, 11 Oct 2020 01:56:16 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Received-SPF: none client-ip=2a01:e0c:1:1599::12; envelope-from=villette.fr@free.fr; helo=smtp3-g21.free.fr X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: 1 X-Spam_score: 0.1 X-Spam_bar: / X-Spam_report: (0.1 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, SPOOFED_FREEMAIL=1.999 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 12 Oct 2020 14:11:45 -0400 X-BeenThere: bug-ddrescue@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Bug reports for ddrescue, data recovery tool." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Oct 2020 23:56:26 -0000 Hello, There's a talk on Xigmanas forums (custom FreeBSD OS NAS) about SSD with ddrescue and dd: https://www.xigmanas.com/forums/viewtopic.php?f=15&t=15441 (see #10 & 11). I've successfully rescued HDDs, but never had yet the opportunity for SSD nor NVMe. Does ddrescue work on SSD & NVMe ? I've done clones of SSDs with ddrescue but they were brand new, so can't say for sure if ddrescue detects bad cells on them to try to rescue them. Also, is there a way, on SSD or NVMe, to get the files' names lying in bad sectors with ddrescue ? Best regards, François From MAILER-DAEMON Tue Oct 13 11:39:13 2020 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1kSMOP-0001OO-0x for mharc-bug-ddrescue@gnu.org; Tue, 13 Oct 2020 11:39:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34332) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kSMOO-0001NQ-84 for bug-ddrescue@gnu.org; Tue, 13 Oct 2020 11:39:12 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37536) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kSMON-0003zt-Gt; Tue, 13 Oct 2020 11:39:11 -0400 Received: from 59.red-88-16-13.dynamicip.rima-tde.net ([88.16.13.59]:49356 helo=[192.168.1.2]) by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_CAMELLIA_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kSMOM-0007Hw-Ik; Tue, 13 Oct 2020 11:39:10 -0400 Message-ID: <5F85CA75.1000401@gnu.org> Date: Tue, 13 Oct 2020 17:40:37 +0200 From: Antonio Diaz Diaz User-Agent: Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.9.1.19) Gecko/20110420 SeaMonkey/2.0.14 MIME-Version: 1.0 To: =?ISO-8859-15?Q?Fran=E7ois_Villette?= CC: bug-ddrescue@gnu.org Subject: Re: gddrescue ssd/nvme questions References: <222fba1f-c861-5c56-eacd-b32a86a5070d@free.fr> In-Reply-To: <222fba1f-c861-5c56-eacd-b32a86a5070d@free.fr> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: bug-ddrescue@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Bug reports for ddrescue, data recovery tool." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2020 15:39:12 -0000 Hello Fran=E7ois, Fran=E7ois Villette wrote: > There's a talk on Xigmanas forums (custom FreeBSD OS NAS) about SSD wit= h > ddrescue and dd: > https://www.xigmanas.com/forums/viewtopic.php?f=3D15&t=3D15441 > (see #10 & 11). I get an "Internal server error" when trying to access that page. > Does ddrescue work on SSD & NVMe? Ddrescue should work with any device for which the kernel provides a bloc= k=20 device name. > I've done clones of SSDs with ddrescue but they were brand new, so can'= t > say for sure if ddrescue detects bad cells on them to try to rescue the= m. Ddrescue does not "detect bad cells". It is the device driver the one tha= t=20 detects them and returns a "read error" to ddrescue. > Also, is there a way, on SSD or NVMe, to get the files' names lying in > bad sectors with ddrescue? The same methods used for hard discs should work because they are perform= ed=20 on the copy. See=20 http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Fill-mod= e Best regards, Antonio. From MAILER-DAEMON Mon Oct 19 11:27:23 2020 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1kUX4F-0007N2-RA for mharc-bug-ddrescue@gnu.org; Mon, 19 Oct 2020 11:27:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59520) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUBdq-0005lV-5e for bug-ddrescue@gnu.org; Sun, 18 Oct 2020 12:34:42 -0400 Received: from smtp4-g21.free.fr ([212.27.42.4]:45680) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUBdo-0008OV-6N; Sun, 18 Oct 2020 12:34:41 -0400 Received: from [10.137.0.11] (unknown [81.249.156.114]) (Authenticated sender: villette.fr@free.fr) by smtp4-g21.free.fr (Postfix) with ESMTPSA id 5EB3819F5BD; Sun, 18 Oct 2020 18:34:33 +0200 (CEST) Subject: Re: gddrescue ssd/nvme questions To: Antonio Diaz Diaz Cc: bug-ddrescue@gnu.org References: <222fba1f-c861-5c56-eacd-b32a86a5070d@free.fr> <5F85CA75.1000401@gnu.org> From: =?UTF-8?Q?Fran=c3=a7ois_Villette?= Message-ID: Date: Sun, 18 Oct 2020 18:34:32 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <5F85CA75.1000401@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Received-SPF: none client-ip=212.27.42.4; envelope-from=villette.fr@free.fr; helo=smtp4-g21.free.fr X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/18 12:34:36 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FROM=0.001, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 19 Oct 2020 11:27:22 -0400 X-BeenThere: bug-ddrescue@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Bug reports for ddrescue, data recovery tool." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2020 16:34:42 -0000 Hello Antonio, Thank you for your answer. On 2020-10-13 17:40, Antonio Diaz Diaz wrote: > Hello François, > > François Villette wrote: >> There's a talk on Xigmanas forums (custom FreeBSD OS NAS) about SSD with >> ddrescue and dd: >> https://www.xigmanas.com/forums/viewtopic.php?f=15&t=15441 >> (see #10 & 11). > > I get an "Internal server error" when trying to access that page. It happens on this forum. I've copy/paste the link from this email and was able to access the page... I'll post your answer there. >> Does ddrescue work on SSD & NVMe? > > Ddrescue should work with any device for which the kernel provides a > block device name. > >> I've done clones of SSDs with ddrescue but they were brand new, so can't >> say for sure if ddrescue detects bad cells on them to try to rescue them. > > Ddrescue does not "detect bad cells". It is the device driver the one > that detects them and returns a "read error" to ddrescue. > >> Also, is there a way, on SSD or NVMe, to get the files' names lying in >> bad sectors with ddrescue? > > The same methods used for hard discs should work because they are > performed on the copy. See > http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Fill-mode Sure I didn't read thoughtfully this part (and not only this one)... I'll get a bad drive to try that out. > > Best regards, > Antonio. > Best regards, François