Finally hooked up the L2 and NUMALink Router I bought a while back with the two Origin 300 bricks. Cabled everything up, powered up the L2, then each O300 brick...
Interesting to see that each brick seems to have it's serial number cleared whenever the L2 first detects it. I knew the L2 would push/enforce it's serial number, but interesting to see it in action. After I checked the serial number at the start of this, I plugged in the power for the NUMALink Router.
Code:
Select all
M2100887-001-L2>* serial
L2 system serial number: M2100887.
M2100887-001-L2>hub.c: USB new device connect on bus1/1, assigned device number 3
hub.c: USB hub found
hub.c: 7 ports detected
hub.c: USB new device connect on bus1/1/1, assigned device number 4
Manufacturer: Silicon Graphics, Inc.
Product: SN1 L1 System Controller
SerialNumber: 00000000
sgil1.c: SGI L1 connected, minor: 64 device: 1.4
INFO: opened USB device at b1;p1/1;d4 (/dev/sgil1_0)
M2100887-001-L2>env
001r08:
Environmental monitoring is enabled and running.
Description State Warning Limits Fault Limits Current
-------------- ---------- ----------------- ----------------- -------
12V bias Wait Pwr 10% 10.80/ 13.20 20% 9.60/ 14.40 0.00
2.5V Wait Pwr 10% 2.25/ 2.75 20% 2.00/ 3.00 0.10
3.3V aux Wait Pwr 10% 2.97/ 3.63 20% 2.64/ 3.96 3.29
5V aux Wait Pwr 10% 4.50/ 5.50 20% 4.00/ 6.00 5.12
Description State Warning RPM Current RPM
-------------- ---------- ----------- -----------
FAN 0 LEFT Wait Pwr 2160 0
FAN 1 RIGHT Wait Pwr 2160 0
Advisory Critical Fault Current
Description State Temp Temp Temp Temp
----------------- ---------- --------- --------- --------- ---------
0 POWER Wait Pwr 30C/ 86F 40C/104F 50C/122F 11C/ 51F
M2100887-001-L2>
M2100887-001-L2>001r08 pwr up
M2100887-001-L2>
M2100887-001-L2>* env
001r08:
Environmental monitoring is enabled and running.
Description State Warning Limits Fault Limits Current
-------------- ---------- ----------------- ----------------- -------
12V bias Enabled 10% 10.80/ 13.20 20% 9.60/ 14.40 11.88
2.5V Enabled 10% 2.25/ 2.75 20% 2.00/ 3.00 2.51
3.3V aux Enabled 10% 2.97/ 3.63 20% 2.64/ 3.96 3.29
5V aux Enabled 10% 4.50/ 5.50 20% 4.00/ 6.00 5.10
Description State Warning RPM Current RPM
-------------- ---------- ----------- -----------
FAN 0 LEFT Enabled 2160 5720
FAN 1 RIGHT Enabled 2160 5625
Advisory Critical Fault Current
Description State Temp Temp Temp Temp
----------------- ---------- --------- --------- --------- ---------
0 POWER Enabled 30C/ 86F 40C/104F 50C/122F 12C/ 53F
M2100887-001-L2> 001r08 l1
001r08-L1>
001r08-L1>serial
BSN: NDW543 SSN: M2100887 Time: 03/22/2012 12:08:25 PST
001r08-L1>version
L1 1.28.3 (Image B), Built 03/20/2004 00:00:51 [Base 1MB image]
001r08-L1>
001r08-L1>let the carnage begin
001r08-L1>serial
BSN: NDW543 SSN: M2100887 Time: 03/22/2012 12:09:01 PST Security: OFF
001r08-L1>
001r08-L1>reboot_l1usb.c: USB disconnect on device 4
sgil1.c: SGI L1 disconnected, minor: 64 device: 1.4
hub.c: USB new device connect on bus1/1/1, assigned device number 5
Manufacturer: Silicon Graphics, Inc.
Product: SN1 L1 System Controller
SerialNumber: 00000000
sgil1.c: SGI L1 connected, minor: 65 device: 1.5
INFO: closed connection to 001r08
INFO: opened USB device at b1;p1/1;d5 (/dev/sgil1_1)
ERROR: no response from 001r08
001r08-L1>
001r08-L1>link v
port '1' appears okay (1)
port '2' appears okay (1)
port '3' appears okay (0) // Have two O300 bricks on
port '4' appears okay (0) // ports 3 & 4, not plugged in
port '5' appears okay (1)
port '6' appears okay (1)
port '7' appears okay (1)
port '8' appears okay (1)
001r08-L1>
001r08-L1>rtr
3400 Series: router type is ordinary
001r08-L1>
// Hmm, "3400" - maybe this didn't come from a Prism...?
001r08-L1>log
11/09/05 11:44:02 power down (COMMAND)
11/09/05 11:44:46 power up hold (COMMAND)
11/09/05 11:44:57 reset (COMMAND)
11/23/05 11:32:05 power down (COMMAND)
11/23/05 11:53:41 power down (COMMAND)
11/28/05 09:15:35 L1 booting 1.28.3
11/28/05 09:15:37 USB0: waiting on open
11/28/05 09:16:24 USB0: opened
11/28/05 09:16:24 USB0: registered for events
11/28/05 09:16:46 power up hold (COMMAND)
11/28/05 09:16:57 reset (COMMAND)
12/19/05 08:37:11 power down (COMMAND)
12/19/05 08:37:37 USB-R: USB:bus was reset
12/19/05 08:37:37 UNREG: 30004c80 0 7
12/19/05 08:37:37 USB0: unregistered
01/03/06 12:04:05 L1 booting 1.28.3
01/03/06 12:04:07 USB0: waiting on open
01/03/06 12:04:53 USB0: opened
01/03/06 12:04:53 USB0: registered for events
01/03/06 12:05:10 power up hold (COMMAND)
01/03/06 12:05:21 reset (COMMAND)
01/14/06 02:44:20 48V power fail warning
01/14/06 02:44:20 VRM not OK: type 2.5V, number 1
01/14/06 11:37:43 L1 booting 1.28.3
01/14/06 11:37:45 USB0: waiting on open
01/14/06 11:38:31 USB0: opened
01/14/06 11:38:31 USB0: registered for events
01/14/06 11:38:45 USB-R: USB:connection lost
01/14/06 11:38:45 UNREG: 30004c80 0 7
01/14/06 11:38:45 USB0: unregistered
01/14/06 11:38:46 USB0-R: IRouter:read failed - read error
01/14/06 11:38:46 USB0: waiting on open
01/14/06 11:39:31 USB0: opened
01/14/06 11:39:31 USB0: registered for events
01/15/06 10:36:09 power down (COMMAND)
01/15/06 10:36:14 power up hold (COMMAND)
01/15/06 10:36:26 reset (COMMAND)
04/27/06 12:03:19 power down (COMMAND)
06/21/06 10:39:37 L1 booting 1.28.3
06/21/06 10:39:39 USB0: waiting on open
12/16/06 17:39:10 L1 booting 1.28.3
12/16/06 17:39:12 USB0: waiting on open // Date was current when I powered it up, so it's been sitting...
03/22/12 12:06:31 L1 booting 1.28.3
03/22/12 12:11:02 L1 booting 1.28.3
03/22/12 12:11:04 USB0: waiting on open
03/22/12 12:11:04 USB0: opened
03/22/12 12:11:04 USB0: registered for events
001r08-L1>
001r08-L1>port
Port Name Stat Remote Pwr Local Pwr Link LED SW LED
--------- ---- ---------- ---------- -------- --------
1 0x02 none okay off off
2 0x02 none okay off off
3 0x00 none none off off
4 0x00 none none off off
5 0x02 none okay off off
6 0x02 none okay off off
7 0x02 none okay off off
8 0x02 none okay off off
001r08-L1>
001r08-L1>^T
escaping to L2 system controller, <CTRL_T> to send escape to L1
M2100887-001-L2>l2
L2 command processor engaged, <CTRL-D> for console mode.
M2100887-001-L2>
Okay, let's plug in the first O300 brick. Note that this first brick temporarily shows as 001?01 while being initialized by the L2 controller.
Code:
Select all
M2100887-001-L2>hub.c: USB new device connect on bus1/1/4, assigned device number 6
usb.c: USB device not accepting new address=6 (error=-110)
hub.c: USB new device connect on bus1/1/4, assigned device number 7
usb.c: USB device not accepting new address=7 (error=-110)
hub.c: USB new device connect on bus1/1/4, assigned device number 8
Manufacturer: Silicon Graphics, Inc.
Product: SN1 L1 System Controller
SerialNumber: 00000000
sgil1.c: SGI L1 connected, minor: 64 device: 1.8
INFO: opened USB device at b1;p1/4;d8 (/dev/sgil1_0)
001?01 INFO: System serial number reassigned (8000) to M2100887 from attached L2.
M2100887-001-L2>
M2100887-001-L2>cfg
L2 10.10.10.107: - 001 (LOCAL)
L1 10.10.10.107:1:0 - 001r08
L1 10.10.10.107:0:0 - 001c01
M2100887-001-L2>
M2100887-001-L2>001c01 l1
entering L1 mode 001c01, <CTRL-T> to escape to L2
001c01-L1>serial
BSN: MYS631 SSN: M2100887 Time: 03/22/2012 15:24:55 EST
001c01-L1>
001c01-L1>^T
escaping to L2 system controller, <CTRL_T> to send escape to L1
M2100887-001-L2>l2
L2 command processor engaged, <CTRL-D> for console mode.
M2100887-001-L2>
Plugging in the second O300 brick...
Code:
Select all
M2100887-001-L2>
M2100887-001-L2>hub.c: USB new device connect on bus1/1/3, assigned device number 9
Manufacturer: Silicon Graphics, Inc.
Product: SN1 L1 System Controller
SerialNumber: 00000000
sgil1.c: SGI L1 connected, minor: 66 device: 1.9
INFO: opened USB device at b1;p1/3;d9 (/dev/sgil1_2)
001?02 INFO: System serial number reassigned (8000) to M2100887 from attached L2.
M2100887-001-L2>
M2100887-001-L2>cfg
L2 10.10.10.107: - 001 (LOCAL)
L1 10.10.10.107:1:0 - 001r08
L1 10.10.10.107:2:0 - 001c02
L1 10.10.10.107:0:0 - 001c01
M2100887-001-L2>001c02 l1
entering L1 mode 001c02, <CTRL-T> to escape to L2
001c02-L1>serial
BSN: MMH445 SSN: M2100887 Time: 03/22/2012 14:16:37 CST
001c02-L1>
001c02-L1>dsp
revision: 2.0
line 1: "001c02 /"
line 2: "L1 running "
leds: PWR=off SERVICE=off FAULT=off
001c02-L1>
Initially, however, the second brick was not being seen as part of the system at the PROM level:
Code:
Select all
M2100887-001-L2>^D
entering system console mode (001c01 console), <CTRL_T> to escape to L2
System Maintenance Menu
1) Start System
2) Install System Software
3) Run Diagnostics
4) Recover System
5) Enter Command Monitor
Option? 1
Starting up the system...
Unable to execute dksc(0,1,8)/sash: no such device
Unable to load bootfile: no such device
Autoboot failed
Unable to continue; press <enter> to return to the menu:
Automatic update of PROM environment disabled
Installing PROM Device drivers ............
Base I/O Ethernet set to /dev/ethernet/ef0
Walking SCSI Adapter 0, (pci id 1)
1- 2- 3- 4- 5- 6- 7- 8- 9- 10- 11- 12- 13- 14- 15- = 0 device(s)
Walking SCSI Adapter 1, (pci id 1)
1- 2- 3- 4- 5- 6- 7- 8- 9- 10- 11- 12- 13- 14- 15- = 0 device(s)
Initializing PROM Device drivers .......... DONE
Checking hardware inventory ...............
[color=#0040FF]***Warning: Board in module 001c02 is missing or disabled
It previously contained a node-board, barcode MMH445 laser 518a1d15
***Warning: Board in module 001c02 is missing or disabled
It previously contained a IBRICK board, barcode MJS307 laser 51450937
***Warning: Board in module 001c02 is missing or disabled
It previously contained a IBRICK board, barcode MJS307 laser 51450937
[/color]DONE
System Maintenance Menu
1) Start System
2) Install System Software
3) Run Diagnostics
4) Recover System
5) Enter Command Monitor
Option?
This persisted across some resetenv and reboot_l1 commands on 001c02. When the PROM initialized, it would still only see 1 object:
Code:
Select all
>> reset
Resetting the system...
Starting PROM Boot process
IP35 PROM SGI Version 6.210 built 02:33:51 PM Aug 26, 2004
Testing/Initializing memory ............... DONE
Copying PROM code to memory ............... DONE
Discovering local IO ...................... DONE
Discovering NUMAlink connectivity .........
Local hub NUMAlink is down.
*** Local network link down
DONE
Found 1 objects (1 hubs, 0 routers) in 5895 usec
Waiting for peers to complete discovery.... DONE
No other nodes present; becoming global master
I powered everything down, un/plugged 001c02, and brought it all back up and that apparently did it:
Code:
Select all
M2100887-001-L2>* power down
M2100887-001-L2>usb.c: USB disconnect on device 10
sgil1.c: SGI L1 disconnected, minor: 67 device: 1.10
INFO: closed connection to 001c02
hub.c: USB new device connect on bus1/1/3, assigned device number 11
Manufacturer: Silicon Graphics, Inc.
Product: SN1 L1 System Controller
SerialNumber: 00000000
sgil1.c: SGI L1 connected, minor: 66 device: 1.11
INFO: opened USB device at b1;p1/3;d11 (/dev/sgil1_2)
M2100887-001-L2>001r08 power up
M2100887-001-L2>001c02 pwr up
M2100887-001-L2>M2100887-001-L2>001c01 pwr up
entering system console mode (001c01 CPU0), <CTRL_T> to escape to L2
Starting PROM Boot process
Starting PROM Boot process
IP35 PROM SGI Version 6.210 built 02:33:51 PM Aug 26, 2004
IP35 PROM SGI Version 6.210 built 02:33:51 PM Aug 26, 2004
Testing/Initializing memory ..........Testing/Initializing memory ............D.
ONE
Copying PROM code to memory ............... DONE
..Discovering local IO ...................... DONE
Discovering NUMAlink connectivity ......... DONE
Found 3 objects (2 hubs, 1 routers) in 106643 usec
Waiting for peers to complete discovery.... .... DONE
Copying PROM code to memory ............... DONE
Discovering local IO ...................... DONE
Discovering NUMAlink connectivity ......... DONE
Found 3 objects (2 hubs, 1 routers) in 22785 usec
Waiting for peers to complete discovery.... DONE
Global master is /hw/rack/001/bay/01
DONE
Global master is /hw/rack/001/bay/01
Intializing any CPUless nodes.............. DONE
Checking partitioning information ......... Checking partitioning in
formation ......... DONE
DONE
Local slave entering slave loop
Local slave entering slave loop
Local slave entering slave loop
Local master entering slave loop
Loading BASEIO prom ....................... Local slave entering sla
ve loop
Local slave entering slave loop
Local slave entering slave loop
DONE
BASEIO PROM Monitor SGI Version 6.210 built 02:30:38 PM Aug 26, 2004 (BE64)
8 CPUs on 2 nodes found.
Automatic update of PROM environment disabled
Installing PROM Device drivers ............
Base I/O Ethernet set to /dev/ethernet/ef0
Walking SCSI Adapter 0, (pci id 1)
1+ Device Vendor Product: IBM-SSG S53D073
2- 3- 4- 5- 6- 7- 8- 9- 10- 11- 12- 13- 14- 15- = 1 device(s)
Walking SCSI Adapter 1, (pci id 1)
1- 2- 3- 4- 5- 6- 7- 8- 9- 10- 11- 12- 13- 14- 15- = 0 device(s)
Initializing PROM Device drivers .......... DONE
Checking hardware inventory ............... DONE
**** System Configuration and Diagnostics Summary ****
CONFIG:
No. of NODEs enabled = 2
No. of NODEs disabled = 0
No. of CPUs enabled = 8
No. of CPUs disabled = 0
Mem enabled = 8192 MB
Mem disabled = 0 MB
No. of RTRs enabled = 1
No. of RTRs disabled = 0
DIAG RESULTS:
ALL DIAGS PASSED.
**** End System Configuration and Diagnostics Summary ****
System Maintenance Menu
1) Start System
2) Install System Software
3) Run Diagnostics
4) Recover System
5) Enter Command Monitor
Option? 5
Command Monitor. Type "exit" to return to the menu.
>> hinv -v
IP35 Node Board, Module 001c01
ASIC BEDROCK Rev 2, 200 MHz, (nasid 1)
Processor A: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 0)
R14010FPC Rev 2.4
Processor B: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 1)
R14010FPC Rev 2.4
Processor C: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 2)
R14010FPC Rev 2.4
Processor D: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 3)
R14010FPC Rev 2.4
Memory on board, 4096 MBytes (Standard)
Bank 0, 1024 MBytes (Premium) <-- (Software Bank 0)
Bank 1, 1024 MBytes (Premium)
Bank 2, 1024 MBytes (Premium)
Bank 3, 1024 MBytes (Premium)
IP35 Node Board, Module 001c02
ASIC BEDROCK Rev 2, 200 MHz, (nasid 0)
Processor A: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 4)
R14010FPC Rev 2.4
Processor B: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 5)
R14010FPC Rev 2.4
Processor C: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 6)
R14010FPC Rev 2.4
Processor D: 600 MHz R14000 Rev 2.4
Secondary Cache 4MB 300MHz Tap 0xa , (cpu 7)
R14010FPC Rev 2.4
Memory on board, 4096 MBytes (Standard)
Bank 0, 1024 MBytes (Premium) <-- (Software Bank 0)
Bank 1, 1024 MBytes (Premium)
Bank 2, 1024 MBytes (Premium)
Bank 3, 1024 MBytes (Premium)
IBRICK Bridge, Module 001c01
ASIC BRIDGE Rev 2, (widget 14)
adapter ID (Vendor 1000 Device 640 class c subclass c)
(pci id 1 function 0)
adapter ID (Vendor 1000 Device 640 class c subclass c)
(pci id 1 function 1)
IBRICK Bridge, Module 001c01
ASIC BRIDGE Rev 2, (widget 15)
adapter PCI (SCSI interface) Rev 6
(pci id 1)
peripheral DISK, BUS 0, ID 1, IBM-SSG S53D073
adapter IOC3 Rev 1
(pci id 4)
controller multi function SuperIO
controller Ethernet Rev 1
adapter USB (OHCI interface)
(pci id 5)
IBRICK Bridge, Module 001c02
ASIC BRIDGE Rev 2, (widget 14)
adapter ID (Vendor 1000 Device 640 class c subclass c)
(pci id 1 function 0)
adapter ID (Vendor 1000 Device 640 class c subclass c)
(pci id 1 function 1)
IBRICK Bridge, Module 001c02
ASIC BRIDGE Rev 2, (widget 15)
adapter PCI (SCSI interface) Rev 6
(pci id 1)
adapter IOC3 Rev 1
(pci id 4)
controller multi function SuperIO
controller Ethernet Rev 1
adapter USB (OHCI interface)
(pci id 5)
ASIC ROUTER , Module 001r08, (nasid 0)
ASIC XBOW Rev 2, on CBrick, Module 001c01
ASIC XBOW Rev 2, on CBrick, Module 001c02
>>
>> update
Writing 6 records...... DONE
Updated new configuration. Wrote 6 records.
>>
>> modnum
001c01 001c02
>>
>> boot
896+111372+16725+3848 entry: 0xa8000002012a6d5c
Standalone Shell SGI Version 6.5 ARCS Oct 6, 2003 (64 Bit)
sash: boot
5172875+1102192+1061824 entry: 0xa8000002000418e0
IRIX Release 6.5 IP35 Version 10070055 System V - 64 Bit
Copyright 1987-2003 Silicon Graphics, Inc.
All Rights Reserved.
mem_alloc: pagesize is 1048576
Inside mem_alloc_init, total pages is 200
mem_alloc: path to MA device is /hw/mem_alloc
mem_alloc: path /hw/mem_alloc added
mem_alloc: name of MA device 0 is 0
mem_alloc: device 0 added under /hw/mem_alloc
mem_alloc: 0 pgs allocated, each 1048576 bytes
priority_lists initialized
Returning 0 from mem_alloc_init
QLFC: running as interrupt thread.
QLFC: using spinlocks.
NOTICE: pcibr_attach: /hw/module/001c01/Ibrick/xtalk/15/pci Bus holds a usb part
- settingbridge PCI_RETRY_HLD to 4
Setting hub ixtt.rrsp_ps field to 0x4e20
NOTICE: pcibr_attach: /hw/module/001c02/Ibrick/xtalk/15/pci Bus holds a usb part
- settingbridge PCI_RETRY_HLD to 4
Setting hub ixtt.rrsp_ps field to 0x4e20
Selecting Default Server
NOTICE: XVM mirrors disabled
NOTICE: XVM snapshot disabled
xvminit complete
The system is coming up.
network: WARNING: Failed to configure ef1 as gate-origin.
Warning: Internet Gateway web server running as root.
Use "chkconfig webface_apache off" to disable.
inst:
inst: Software installation has installed new configuration files and/or saved
inst: the previous version in some cases. You may need to update or merge
inst: old configuration files with the newer versions. See the "Updating
inst: Configuration Files" section in the versions(1M) manual page for details.
inst: The shell command "versions changed" will list the affected files.
inst:
inst: These directories were unable to be moved properly during the
inst: installation process. Check for any user-modified files, then
inst: delete the directories.
inst: /usr/include/Vk.O
origin console login: root
Password:
IRIX Release 6.5 IP35 origin
Copyright 1987-2003 Silicon Graphics, Inc. All Rights Reserved.
Last login: Tue Nov 16 00:28:58 PST 2010 by [email protected]
TERM = (vt100)
origin 1# uname -aR
IRIX64 origin 6.5 6.5.22f 10070055 IP35
origin 2#
The OS install was left over from a test run so I'll scratch that, decide on .22 or .30, and load some Nekoware so I can put these CPUs and the Router through their paces.