bug-grub
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[bug #56391] Easily-triggered TCP bug(s)

From: Faidon Liambotis
Subject: [bug #56391] Easily-triggered TCP bug(s)
Date: Sat, 25 May 2019 22:59:13 -0400 (EDT)
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0 
URL:
  <https://savannah.gnu.org/bugs/?56391>

                 Summary: Easily-triggered TCP bug(s)
                 Project: GNU GRUB
            Submitted by: paravoid
            Submitted on: Sun 26 May 2019 02:59:11 AM UTC
                Category: Network
                Severity: Major
                Priority: 5 - Normal
              Item Group: Software Error
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 
                 Release: Git master
         Reproducibility: Every Time
         Planned Release: None

    _______________________________________________________

Details:

GRUB (git master @ 53e70d30cf0d18e6c28bab0ab8d223a90d3e1b46), PXE booted, with
this configuration:
  set prefix=(http)/grub
  menuentry 'memtest86+' --id memtest {
    linux16 /boot/memtest86+.bin
  }

Boot the host (say, a QEMU). Reboot and try it again, relatively quickly.
Boot gets stuck at "Welcome to GRUB" and cannot fetch files for a while. This
is the simplified version -- I started debugging this when in a complicated
menu I was getting a "connection timed out" every now and then when selecting
an entry like memtest.

The HTTP server is a Linux 4.9, with nginx, in the same LAN (no firewalls or
anything like that). Nothing out of the ordinary there.

It appears that GRUB has a fixed local port that it starts with (21550); the
(src IP, src port, dst IP, dst port) tuple remains the same in the two fetches
across reboots, so the socket has remained in TIME_WAIT on the server. Thus,
the server attempts to reuse the same ack nr, per RFC. All hell breaks loose
after that. Multiple bugs I think -- GRUB gets something unexpected but
doesn't notice, doesn't RST, and keeps sending the same ACKs over and over.
Plus, Wireshark doesn't seem to be happy with the FIN/ACK of the first
connection either.

(I took a look at the source, which seems ridden with FIXMEs. Have you thought
about using a third-party TCP/IP stack like lwIP or PicoTCP? No idea how easy
it would be to integrate this, apologies if this is a stupid idea :)

Packet capture from tshark (tshark -n -o tcp.relative_sequence_numbers:FALSE
-r grub.pcap 'tcp.port == 21550'):

<first boot>
    1   0.000000 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [SYN]
Seq=96 Win=8192 Len=0
    2   0.000290 192.168.24.1 → 192.168.24.104 TCP 60 80 → 21550 [SYN,
ACK] Seq=150886417 Ack=97 Win=29200 Len=0 MSS=1460
    3   0.000689 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=97 Ack=150886418 Win=8192 Len=0
    4   0.001023 192.168.24.104 → 192.168.24.1 HTTP 162 GET
/grub/i386-pc/command.lst HTTP/1.1 
    5   0.001175 192.168.24.104 → 192.168.24.1 TCP 162 [TCP Retransmission]
21550 → 80 [PSH, ACK] Seq=97 Ack=150886418 Win=8192 Len=108
    6   0.001208 192.168.24.1 → 192.168.24.104 TCP 60 80 → 21550 [ACK]
Seq=150886418 Ack=205 Win=29200 Len=0
    7   0.001335 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 6#1] 80
→ 21550 [ACK] Seq=150886418 Ack=205 Win=29200 Len=0
    8   0.001576 192.168.24.1 → 192.168.24.104 TCP 590 HTTP/1.1 200 OK  [TCP
segment of a reassembled PDU]
    9   0.001581 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK]
Seq=150886954 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   10   0.001583 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK]
Seq=150887490 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   11   0.001585 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK]
Seq=150888026 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   12   0.001587 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK]
Seq=150888562 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   13   0.001646 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK]
Seq=150889098 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   14   0.001651 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK]
Seq=150889634 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   15   0.001653 192.168.24.1 → 192.168.24.104 HTTP 393 HTTP/1.1 200 OK 
   16   0.001894 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150886954 Win=8192 Len=0
   17   0.002365 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150887490 Win=8192 Len=0
   18   0.002554 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150888026 Win=8192 Len=0
   19   0.002721 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150888562 Win=8192 Len=0
   20   0.002885 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150889098 Win=8192 Len=0
   21   0.003047 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150889634 Win=8192 Len=0
   22   0.003213 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150890170 Win=8192 Len=0
   23   0.003393 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK]
Seq=205 Ack=150890510 Win=8192 Len=0
   24   0.003566 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [FIN,
ACK] Seq=205 Ack=150890510 Win=0 Len=0
   25   0.003770 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Out-Of-Order]
21550 → 80 [FIN, ACK] Seq=205 Ack=150890510 Win=0 Len=0
   26   0.003798 192.168.24.1 → 192.168.24.104 TCP 60 80 → 21550 [ACK]
Seq=150890510 Ack=206 Win=29200 Len=0
   27   0.003945 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 26#1] 80
→ 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
<reboot here>
   71  36.235831 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Port numbers
reused] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   72  36.236082 192.168.24.1 → 192.168.24.104 TCP 60 [TCP ACKed unseen
segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
   73  36.236502 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#1]
21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   74  36.635996 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   75  37.035883 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   76  37.036117 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#1]
[TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200
Len=0
   77  37.036428 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#2]
21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   78  37.435783 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   79  37.835738 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   80  37.835972 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#2]
[TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200
Len=0
   81  37.836155 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#3]
21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   82  38.235704 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   83  38.635605 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   84  38.635810 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#3]
[TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200
Len=0
   85  38.635962 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#4]
21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   86  39.035594 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   87  39.435549 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission]
21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   88  39.435754 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#4]
[TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200
Len=0
<etc. etc. etc., keeps repeating for a while>




    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?56391>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/
reply via email to
[Prev in Thread] Current Thread [Next in Thread]