[bug #63245] PXE boot with GRUB 2.06 sometimes fail on generation 11 Intel NUCs

[Matttias Andersson]

URL:
  <https://savannah.gnu.org/bugs/?63245>

                 Summary: PXE boot with GRUB 2.06 sometimes fail on generation
11 Intel NUCs 
                 Project: GNU GRUB
               Submitter: jerrymattias
               Submitted: Thu 20 Oct 2022 06:30:08 AM UTC
                Category: None
                Severity: Major
                Priority: 5 - Normal
              Item Group: Hardware-specific
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
                 Release: other
                 Release: 
         Discussion Lock: Any
         Reproducibility: Intermittent
         Planned Release: None


    _______________________________________________________

Follow-up Comments:


-------------------------------------------------------
Date: Thu 20 Oct 2022 06:30:08 AM UTC By: Matttias Andersson <jerrymattias>
Short background: 
We sell a commercial server system that is made to handle large Audi/Video
installations handling both device control and media management and
interaction. The media players (Intel NUC in general) runs Porteus Kiosk
https://porteus-kiosk.org/. That system uses grub to PXE boot from our server.
It has always worked very well in the past. The OS runs as read only on
ramdisk and the boot files are kept on our server. 

Problem:
With the 11th generation Intel NUC, all flavours i3 to i7 we started to get
reports from installations where they saw the kiosks sometimes does not finish
the boot process but instead ends up showing the grub prompt. Since
installations may be with 100 or more computers all running without a human
user that can reboot, it is a problem.

Way to reproduce: 
Since our system (pixilab.se/blocks) also can easily be programmed doing
different tasks I set up a task that as soon as the kiosk browser connects to
our web-server we send a power down, wait for a bit then send a WakeOnLan to
cold boot again. With this loop it is pretty easy to reproduce the issue and
there is a counter and a log setup that keep track on how many successful
loops we made and how long any boot process took. 

 
Error frequency: Around 1 in 1000 attempts. It may happen at the third
attempt; it may run up so 700 attempts then fail. It seems to be something
that can happen at every boot but the probability is pretty low. 

What I found:
I have made wireshark recordings from successful and unsuccessful attempts.
The fetch and execution of the grup  packageThe only thing found there was
that when it fails, there is never any request to fetch the grub.cfg file from
server. 

If I try to fetch the grub.cfg from manually from grub command line the screen
goes black for a second or two then the grub prompt returns again. ( I have
this on video but it seem to be problem to submit with that attached)

It seem grub is crippled in the way it cannot sent anything out on the network
at all. Not tftp and not http.  

>From what I can see Grub seem to have a network device that is up with a valid
IP.

Who is affected: 
Probably anyone running PXE boot with GRUB 2.06 or 2.16 but mainly userless
applications such as IoT and Signage and cluster applications running on Intel
NUC gen 11 hardware. (Perhaps any Intel device gen 11, NUC is what I have
here)   

I am more than happy to provide any data that I can,but will need help from
someone to provide some syntax that may help to shine a light in what goes
wrong. 

I have raised this concern with Intel Custome Service case No 05585598. I have
had some rather good response, but since they do not have the facilities to
create the loop they gave up after just 20 attempts. They even provided some
test firmware to see if it helped. But I figure, perhaps someone that is
fluent in GRUB 2 may be of assistance to try to figure out why and what
happens. 


I believe this problem may affect any future product from Intel unless fixed
by Intel or Grub. 
I am happy to assist since I can reproduce without too much effort.



 







    _______________________________________________________
File Attachments:


-------------------------------------------------------
Date: Thu 20 Oct 2022 06:30:08 AM UTC  Name: grubnetx64.efi.signed  Size: 1MiB
  By: jerrymattias

<http://savannah.gnu.org/bugs/download.php?file_id=53885>

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?63245>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/