Re: [Bug-wget] wget --spider doesn't get file size

[Yiwei Yang]

Thanks, Tim.


On Tue, Jun 25, 2013 at 2:33 AM, Tim Ruehsen <address@hidden> wrote:

> Hi.
>
> Am Monday 24 June 2013 schrieb Yiwei Yang:
> > I want to get some file size. Some people recommend wget --spider.
> However,
> > when I run it on some links, like http://autos.cn.yahoo.com/,
> >
> http://www.cbsa.gc.ca/menu-eng.html,http://oprofile.sourceforge.net/mail/an
> > d so on,  then it said Length: unspecified [text/html]. Is there way to
> > solve this or I could use another way to get file size? Thank you!
>
> With many dynamically composed pages the web server doesn't know about the
> page length when starting the transmission. That means you have to download
> the whole page to determine the length. --spider only works when the server
> sends a Content-Length: header (regularly used with static pages).
>
> Keep in mind that dynamically composed pages (as the name says) do not
> have a
> fixed size. It might change from download to download or from time to time.
>
> Regards
>
>      Tim
>