Re: [Lynx-dev] Alternative User-Agent header

[Travis Siegel]

fanfiction.net is using cloudflare to help distribute the load of web 
requests.  Cloudflare apparently requires javascript, cookies, and 
something in the browser string that identifies the browser as a 
standard browser.  Two of those you can fix with lynx, but you're never 
going to solve the javascript issue just using lynx. I've not tried 
hitting fanfiction.net using one of the various encarnations that can 
handle javascript, it may or may not work. I also haven't tried changing 
my identification string, though I doubt highly  that will solve the 
problem.  I'm guessing that if you could find the ip address of the 
fanfiction.net actual servers, and not the cloudflare servers they use 
to spread the workload, then you could bypass all these checks, and then 
the site may or may not work properly, but at least then you'd have a 
fighting chance.  I'm not convinced cloudflare will let lynx through no 
matter what you do try to get around it.


On 6/24/2021 6:31 PM, Karen Lewellen wrote:
> Do you mind testing that by visiting
> Www.fanfiction.net
> Since you  say you can browse  everywhere?
>
>
>
> On Thu, 24 Jun 2021, Stef Caunter wrote:
>
> > I can browse anywhere using this alias for 'lynx'
> >
> > alias lynx='/usr/local/bin/lynx -nopause -tna -useragent=IE . '
> >
> > "IE" seems to be the most innocuous choice for the major news sites. I
> > have not tried "googlebot" ;)
> >
> > /stef
> >
> > _______________________________________________
> > Lynx-dev mailing list
> > Lynx-dev@nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/lynx-dev
>
> _______________________________________________
> Lynx-dev mailing list
> Lynx-dev@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lynx-dev