Overview
Here, the SRCF DRP machines are used as an example. These machines are from SuperMicro and have one of their versions of screens and interfaces. Many of our machines have their IPMI interface connected on the network. Point a browser at https://<node name>-ipmi.pcdsn and a page comes up on which to enter a username and password (contact pcds-it for good values). After successful login, the IPMI page comes up and shows many capabilities, including:
- a remote interface to the system console
- interacting with system power
- updating the BIOS
IT Utilities For Managing BIOS Code/Settings
(courtesy of Simon Elmir on Feb. 7, 2024)
This approach promises to be significantly faster than the manual approach Ric describes below.
The one we have covers Supermicro servers: "Supermicro Update Manager (SUM)"
https://www.supermicro.com/en/solutions/management-software/supermicro-update-manager
The utility should be available at one of these two paths, depending on which environment you're in:
* PCDS: /cds/sw/package/supermicro/sum_latest_Linux_x86_64/sum
* S3DF: /sdf/group/scs/tools/supermicro/sum_latest_Linux_x86_64/sum
SUM can be run locally on the target server, or it can interact remotely using IPMI credentials. Changes to BIOS settings get applied on next reboot.
Users Guide: https://www.supermicro.com/Bios/sw_download/636/SUM_UserGuide.pdf
Remote Console
The remote console can be run with either a Java plug-in or HTML5 via selection through the 'settings' link on the System tab. If the Java method gives trouble, switch to HTML5, which is slower but reliably provides readable screens. Click on the preview panel to bring up the console screen.
Getting to the BIOS screens
To get to the BIOS screens, the machine must be power cycled. There are several methods with which this can be achieved, including the Reset button on the Power Control panel on the System tab. After the power cycle takes effect, the console window will go blank for a while, then show some stuff, go blank again, etc., before finally showing a screen giving some options to enter. In the case of these machines, the BIOS parameters are eventually accessible by going into Setup mode using the Delete key (or fn-delete on a Mac) during the short time the options are available. In at least some cases, the machine goes through the boot cycle twice. Wait too long and the machine boots the OS.
Noting BIOS parameters
It may be important to note all the current BIOS parameter states before updating the firmware because some values may not be obvious or intuitive to re-configure after they are reset by the update process. Here are some screenshots from the v1.0 (old) and v2.3 (new) BIOSes on the drp-srcf-cmpNNN machines:
Main tab
Advanced tab
Boot Feature, cmp003
Trusted Computing
PSP fw ver
ACPI Settings
Super IO
Super IO Ser Port 1 Cfg
Super IO SOL Cfg
Ser Port Console Redirect
CPU Cfg
CPU1 Info
NB Cfg
Mem Cfg
CPU1 Mem Info
AMI Graphic Output Proto
PCIe Cfg top, PCIe Cfg bottom
PCIe Network Stack Cfg
PCIe Hot-Plug Settings
USB Cfg
SATA Cfg
ASMedia SATA Ctrl
ASMedia SATA Info
SATA Info
SATA0
SATA1
HTTP Boot cfg
Network cfg
iSCSI Cfg
10GBase-T A
FWImgMenuA
DevCfgMenuA
MBAcfgMenuA
iSCSI BootCfgMenuA
iSCSI GenParA
iSCSI InitParA
iSCSI 1stTarParA
iSCSI 2ndTarParA
2ndDevA
10GBase-T B
FWImgMenuB
DevCfgMenuB
MBAcfgMenuB
iSCSI BootCfgMenuB
iSCSI GenParB
iSCSI InitParB
iSCSI 1stTarParB
iSCSI 2ndTarParB
2ndDevB
TLSAuthCfg
ServerCACfg
EnrCert
DelCert
RamdiskCfg
DriverHealth
Mlx
Broadcom1
Broadcom2
IPMI tab
IPMI SEL
IPMI BMC
EvtLog tab
EvtLog SMBIOS settings
EvtLog SMBIOS evt log
Security tab top, bottom
Secure Boot
SecBoot Key Mgmt
Boot tab old cmp001, old cmp002, new cmp001
SaveAndExit tab old cmp002, new cmp001
Updating the BIOS
Place the firmware update file somewhere in the pcds file system. It may need to be unpacked from a .zip file first. The filename will look something like BIOS_H12SSW-1B2B_20211020_2.3_STDsp.bin which you can find here:
BIOS_H12SSW-1B2B_20211020_2.3_STDsp.bin
It may be best to issue 'sudo halt' on the machine that is about to be updated. I'm not sure this is strictly necessary, but there was a case where a machine didn't survive the BIOS updated and was effectively 'bricked'. Making sure nothing else is going on during the update seems prudent.
On the IPMI home page for the machine being updated, find the 'BIOS Update' menu item. For the drp-srcf-cmp* machines, it is in the Maintenance menu, as shown in IPMI page. This allows you to browse to the BIOS file and initiate its loading.
After selecting the BIOS file to load and clicking on 'Upload BIOS', the system presents a screen on which one can select to 'Preserve NVRAM' (deselected) and 'Preserve SMBIOS' (selected). It's not clear what's correct, so go with the defaults. Also check that 'Existing Date' and 'New Date' are as expected before clicking on 'Start Upgrade'.
After a while the loading will complete and you will be asked whether the system should reset. Go ahead and do so, being prepared to stop the boot process at the BIOS setup stage as described in Getting to the BIOS screens, above. Since a number of the BIOS parameters will have been reset to their default values, they will now need to be reconfigured to their old settings. Note that in the new BIOS version, some parameters may have disappeared, moved to different subsections, renamed or reordered with respect to the old version.
For BIOS v2.3, the parameters on the Advanced tab that needed restoring were:
- Boot Feature:
- Restore on AC Power Loss: Stay Off
- Power Button Function: 4 Seconds Override
- CPU Configuration:
- SMT Control: Disabled
- PCIe/PCI/PnP Config.:
- ... Slot 1 ... Bifurcation: x8x8
- (note: "page down"/Up-Arrow to see this)
- ... Slot 1 ... Bifurcation: x8x8
- Supermicro ... Enet Ctrlr A:
- DevCfgMenu:
- Number of VFs per PF: 8
- Energy Eff. Enet: Disabled
- DCB Protocol: Enabled (IEEE Only)
- MBACfgMenu:
- Legacy Boot Protocol: PXE
- Pre-boot Wake On LAN: Enabled
- DevCfgMenu:
- Supermicro ... Enet Ctrlr B:
- DevCfgMenu:
- Number of VFs per PF: 8
- Energy Eff. Enet: Disabled
- DCB Protocol: Enabled (IEEE Only)
- MBACfgMenu:
- Legacy Boot Protocol: PXE
- Pre-boot Wake On LAN: Enabled
- DevCfgMenu:
In addition, the boot options on the Boot tab need to be modified. First go into UEFI NETWORK Drive BBS Priorities (scroll to the bottom):
- Move the IB option to be last
Next, go back to the boot options and rearrange the options to:
- UEFI USB Key
- UEFI Hard Disk: RedHat Boot Manager
- UEFI Network:(B198/D0/F0): UEFI PXE IPv4 Supermicro (First Ethernet Controller)
- UEFI AP:UEFI: Built-in EFI Shell
- Set remaining options to
Disabled
Save the changes by going to the Save & Exit tab and selecting Save Changes and Reset, which also reboots the machine.
Supermicro Command Line Tool
From demo meeting on Feb. 21, 2024 with the IT team (Simon Elmir presenting)
https://www.supermicro.com/en/support/resources/downloadcenter/smsdownload?category=SUM
Supermicro Update Manager ("sum")
different path on PCDS machines vs S3DF
output current bios settings to file: sudo /sdf/group/scs/tools/supermicro/sum_latest_Linux_x86_64/sum -c GetCurrentBiosCfg --file sdfmilan001.biosCfg.xml
edit the .xml file then use ChangeBiosCfg option to update settings (change is queued up to be applied when server is next rebooted)
There Is a "--tui" option with GetCurrentBiosCfg which brings up a simulation of the usual crash-cart bios configuration utility (F4 to save and exit). Changes are saved to the .xml option.
Any setting that is removed from the .xml is left unchanged, so can have a minimal .xml file with only settings that need to be changed. see /sdf/group/scs/tools/supermicro for example .xml files.
"-i" option going over ipmi interface (e.g. sdfmilan001-oob, which is the ipmi interface?). Needs -u "username" and -p "password" arguments
On PCDS machines path is /cds/sw/package/supermicro/sum_latest_Linux_x86_64/sum
GetBiosInfo is another command
UpdateBios is another command (takes a --file option too, I believe)
The is a SUM_UserGuide.pdf here: /cds/sw/package/supermicro/sum_latest_Linux_x86_64/SUM_UserGuide.pdf
Overview
Content Tools