Re: [libmicrohttpd] Built-In Post Processor With Spaces

[Christian Grothoff]

Dear Kenneth,

The issue is a bit more complex (see in particular
https://gnunet.org/bugs/view.php?id=3371).  We used to
convert '+' to space in the MHD_unescape, but then it
turned out that in some places that's not legal (#3371).
So we now call MHD_unescape_plus() first in cases where
we need to unescape '+' as well as the %-encodings.

However, I forgot to insert the respective call in the
post processor logic for URI-encoding, and somehow nobody
reported that.

I'm sorry it took me a while, I wanted to double check
that this is really a place where that is needed and not
one where your client does something odd, and as you noticed
I was moving.

Anyway, the fix is in SVN 34310 and I hope to make the next
release in about a week.


Happy hacking!

Christian


On 09/29/14 14:23, Kenneth Mastro wrote:
> Christian (et all),
> 
> Do you (or anybody) have any thoughts on adding support for the standard
> '+' sign escaping to the 'unescape' code used by the built-in post
> processor for 'application/x-www-form-urlencoded' data?  I started a
> thread about this a couple weeks ago (titled 'Post Processing Wtih
> Spaces', I think).
> 
> As a reminder, the jist is that the built-in post-processor has unescape
> code that can't be circumvented with your own function, but it does not
> decode the '+' sign used for spaces.  I'm not sure if the '+' is
> considered spec or not, but it's been used for a very long time and is
> incredibly common.  jQuery is inserting it in an ajax call I'm using,
> and I think my browser (usually Firefox) is doing it on forms submitted
> as well.  I can avoid the problem on forms by setting the encoding type
> to multipart/form-data (a hack-ish workaround), but that doesn't work
> with the ajax call.  So...
> 
> I'm proposing adding a few lines of code to MHD's internal.c's
> MHD_http_unescape function to do this - to look for the '+' character
> and substitute a space.  Right now, it only looks for the %-based
> encoding of spaces - thereby only allowing '%20' for spaces.  I solved
> this in my local copy with 5 lines of code - and it could have been less
> if I reorganized the switch statement slightly.  I don't see how this
> could break anything, since +'s themselves are encoded to '%2B'.  I.e.,
> if you see a '+' it should ALWAYS mean a space.
> 
> If there's any risk of a backward compatibility problem, I guess this
> could also be solved by allowing a custom escape function for the
> built-in post-processor, but that's more work and seems a bit excessive
> to me.
> 
> I can't imagine I'm the only person who has encountered this - maybe
> others just bite the bullet and write their own post-processor to get
> around it?  No idea.  I can certainly do that, but it seems a bit silly
> for such a simple/common problem.  As it stands, the built-in
> post-processor doesn't seem useable for text-based input with
> 'application/x-www-form-urlencoded' encoding.
> 
> I think I saw you (Christian) were busy moving, so I don't want to nag
> or anything.  But, I imagine a 0.9.38 release is coming up soon and
> would love to see this in there if you think it's appropriate.
> 
> 
> Thanks much,
> Ken
> 
> PS: Still loving MHD after using it for many months.  It has been
> enormously useful and incredibly reliable.  I appreciate it greatly, and
> I'm so happy I went down this path.  So - thanks again for
> writing/maintaining it.
> 
>