[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] Some URLs are saved as gzip-compressed data

From: Darshit Shah
Subject: Re: [Bug-wget] Some URLs are saved as gzip-compressed data
Date: Fri, 8 Mar 2019 23:35:56 +0100
User-agent: NeoMutt/20180716

Hi Mark,

The server in this case seems to be badly configured. Wget's request to the
server looks like this:

> ---request begin---
> GET /wcsstore/CrucialSAS/firmware/mx100/MU03/MX100_MU03_Update.zip HTTP/1.1
> User-Agent: Wget/1.20.1 (linux-gnu)
> Accept: */*
> Accept-Encoding: identity
> Host: assets.crucial.com
> Connection: Keep-Alive

If you notice here, the Encoding requested by Wget is "indentity", that is,
please don't make any changes to the file. And yet, the server replies back
with a "Content-Encoding: gzip". The server is sending it in a format that Wget
did not request. Firefox on the other hand, does have an "Accept-Encoding"
header that contains gzip. So it is able to deal with the resulting data

One thing you can do it, use the `--compression=gzip` option. This will make
Wget request a gzip encoding, causing the resulting file to be correct.
However, remember this is only a workaround for a broken server. And the
`--compression=gzip` is not a default since it doesn't always work as expected.
Sadly, there are too many servers out there that break the spec in weird ways
and we are unable to support all those cases at this point.

* address@hidden <address@hidden> [190308 23:20]:
> I'm testing wget 1.20.1 on Lubuntu 16.04 x86-64.
> I noticed an issue when trying to download a certain URL which is
> a .zip archive. The file created by wget is gzip-compressed. But
> downloading using e.g. Firefox the URL is saved as a .zip file.
> Example:
> $ wget 
> https://assets.crucial.com/wcsstore/CrucialSAS/firmware/mx100/MU03/MX100_MU03_Update.zip
> --2019-03-08 20:24:31--  
> https://assets.crucial.com/wcsstore/CrucialSAS/firmware/mx100/MU03/MX100_MU03_Update.zip
> Resolving assets.crucial.com...
> Connecting to assets.crucial.com||:443... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 12408650 (12M) [application/zip]
> Saving to: ‘MX100_MU03_Update.zip’
> MX100_MU03_Update.zip                  
> 100%[==========================================================================>]
>   11.83M   738KB/s    in 18s     
> 2019-03-08 20:24:49 (680 KB/s) - ‘MX100_MU03_Update.zip’ saved 
> [12408650/12408650]
> $ file MX100_MU03_Update.zip 
> MX100_MU03_Update.zip: gzip compressed data, from Unix
> Mark

Thanking You,
Darshit Shah
PGP Fingerprint: 7845 120B 07CB D8D6 ECE5 FF2B 2A17 43ED A91A 35B6

Attachment: signature.asc
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]