Re: DNS issue affecting gnu.org (and subdomains)

[Sebastian Tennant]

Quoth Ar Rakin <rakinar2@gmail.com>
on Sat, 25 Mar 2023 23:38:29 +0600:
> Hello Sebastian,
>
> I'm also unable to access gnu.org. I don't exactly know why this is
> happening, though I've encountered this type of issues before, with
> my own domains. But it got fixed automatically after a few
> days. Hopefully, it will be fixed automatically after some time.
>
> What I can see:
>
> $ host gnu.org
> ;; connection timed out; no servers could be reached
>
> $ ping gnu.org
> ping: gnu.org: Temporary failure in name resolution
>
> Thanks,
>
> Rakin

Hello Rakin.  Thanks for the corroboration.


Quoth Eli Zaretskii <eliz@gnu.org>
on Sat, 25 Mar 2023 20:52:00 +0300:
>> […]
>
> You will find the information here:
>
>   https://hostux.social/@fsfstatus
>
> That place is always good to look at when such issues occur.

Hello Eli.  Thanks for the link.  Noted.


Quoth Bob Proulx <bob@proulx.com>
on Sat, 25 Mar 2023 15:05:44 -0600:
>> […]
>
> +1 for the https://hostux.social/@fsfstatus status page.  The FSF
> sysadmins post information there (sometimes terse) when there are
> problems seen that affect systems.  It's something everyone should
> bookmark where they can find it easily.
>
>>  $ host gnu.org 8.8.8.8
>>  [...]
>>  Host gnu.org not found: 2(SERVFAIL)
>>
>> Nope, Google's resolver can't resolve gnu.org either.
>
> The authoritative nameservers (a fancy title for the upstream ones)
> are getting DDoS'd off the net.  Which means that all resolution by
> downstream nameservers, even Google ones, are timing out.

Hello Bob.  A DDoS attack.  I see.

> Compounded by the very short 300 second TTL on the gnu.org records
> mean that even if a lookup is successful that it can only be cached
> for five minutes and then discarded.  Upon which then it needs to be
> looked up again and the query will have to fight its way through the
> DDoS in a mixed martial arts cage fight arena to get the data again.

What's the thinking behind the short TTL?

>> […]
>
> The nameservers are overwhelmed making them slow to respond.  And
> then additionally I am seeing a very high packet loss across the
> network into the Boston machines.  That high packet loss means
> retries at the network protocol level making things slow.  I have
> seen 30-45 seconds on average here looking up DNS for a while.

Understood.

>> […]
>
> There is really nothing special about the Google resolver.  If the
> upstream ns*.gnu.org nameservers can't receive and can't send data
> then gnu.org names cannot be resolved.

Yup.  Understood.  I know there's nothing special about Google's
nameservers.  They have an easy-to-remember IP address, that's all.

>> I fetch from git.sv.gnu.org every 30 minutes and the fetch beagn to
>> fail two days ago (on 23rd March) at around 10pm GMT.  It has been
>> failing much more often than not since then.

> Yes.  That's about when the attack started.  I assume it is an
> attack.  That's what sysadmin said about it.  I have no special
> ability to observe this particular attack and am suffering through
> the packet loss of it along with the rest of you.

:-) It seems the worst is over now.  (Until the next time, in any
case).


Quoth Ian Kelling <iank@fsf.org>
on Sat, 25 Mar 2023 18:51:48 -0400:
> Update: We think we've got things working now.

Hello Ian.  Thanks for the update.  And for your efforts restoring
normal service.