Re: [Bug-wget] [PATCH] Update HSTS info

[Darshit Shah]

Hi Ander,

Could you please take a look at dkg's suggestions and recreate a patch?

On Mon, Aug 10, 2015 at 9:36 PM, Daniel Kahn Gillmor
<address@hidden> wrote:
> Hi Ander--
>
> Thanks for this!  having the concise, human-readable description of the
> intended behavior is really useful.
>
> a couple comments on the proposed documentation below:
>
> On Sat 2015-08-08 13:50:17 -0400, Ander Juaristi wrote:
>
>> Those two patches add two extra debug traces for HSTS, and most importantly, 
>> update the documentation to include information about HSTS.
>  [...]
>> address@hidden address@hidden
>> +By default, Wget stores its HSTS database in @file{~/.wget-hsts}.
>> +You can use @samp{--hsts-file} to override this. Wget will use
>> +the supplied file as the HSTS database. Such file must conform to the
>> +correct HSTS database format used by Wget. If Wget cannot parse the provided
>> +file, the behaviour is unspecified.
>
> There is no pointer or reference here to what the "correct HSTS database
> format used by Wget" is.  providing a brief pointer (e.g. "look at file
> doc/FOO in the wget source" or "see https://example.org/wget-hsts-db";)
> would be useful.

I agree. We should supply a sample hsts config file for the users,
similar to the wgetrc file that is available in the tarball.

>
>> +Be aware though, that Wget may modify the provided file if any change occurs
>> +between the HSTS policies requested by the remote servers and those in the
>> +file. When Wget exists, it effectively updates the HSTS database by 
>> rewriting
>> +the database file with the new entries.
>
> Is it possible that a user would want to decouple reading from
> (respecting) an HSTS file and writing to it?  some examples:
>
>  * A wget process might have no ability to modify the filesystem (e.g.
>    a constrained wget -O- in a pipeline), but wants to respect a
>    system-provided HSTS ruleset.  This might want to read an hsts file
>    without trying to write to it (or at least without raising an error
>    on an attempted modification).
>
This also allows system admins to create a hsts config file in a
location where the current user does not have write-access.

>  * A wget process used as part of network testing suite might want to
>    disobey any system-provided HSTS rules (it needs to emulate behavior
>    of "ignorant" clients), but might also want to collect the HSTS rules
>    it sees for later review.  This might want to update an hsts file
>    without trying to respect its contents.
>
> I don't know if these use cases warrant the additional option complexity
> of accomodating them, but it'd be good to make that decision explicitly
> (sorry if that's already been done and i've just missed the discussion.
>
>> +Care is taken not to override possible changes made by other Wget processes 
>> at
>> +the same time over the HSTS database. Before dumping the updated HSTS 
>> entries
>> +on the file, Wget will re-read it and merge the changes.
>
> This sounds like there might still be a race condition where one
> process's changes get clobbered.  can we make any atomicity guarantees
> for users who might be concerned about this?
>
>> +Using a custom HSTS database and/or modifying an existing one is 
>> discouraged.
>> +For more information about the potential security threats arised from such 
>> practice,
>> +see section 14 "Security Considerations" of RFC 6797, specially section 14.9
>> +"Creative Manipulation of HSTS Policy Store".
>
> I'd normally characterize these threats as "privacy concerns", not
> necessarily "security concerns".  users of wget might understand them
> most closely as offering "cookie-equivalent" mechanisms for some
> http/https clients.
>
> sorry these questions and considerations are in text form and not as
> patches.  feel free to take from them whatever you might find useful.
>
> Regards,
>
>         --dkg
>



-- 
Thanking You,
Darshit Shah