Nekonomicon - Changed RMs - Now Onyx2 Deskside Won't Boot

Thomas W.
From Germany/NRW/Bonn
Who joined Sept. 9, 2003, 11:54 p.m.

Wrote the following at June 16, 2010, 2:11 a.m...

I have replaced my two RM9 with two RM10...

and now my Onyx2 will not boot

System controller says "P 0 M 1C"

and the LEDs on the node boards giving the following code:

N1...................N2
CPU A..CPU B......CPU A..CPU B
:idea:

.....

.........

.....

.....

.........

.....

.........

.....

.........

.....

.........

.....

.........

.....

.........

.....

.........

.....

= LED is ON
:arrow:

= LED is OFF
:oops:

= LED is blinking

What is going wrong with my Onyx2??

recondas
Moderator
From NC - USA
Who joined June 6, 2004, 5:55 p.m.

Wrote the following at June 16, 2010, 4:07 a.m...

Thomas - To make an evaluation of the Nodeboard LED codes meaningful, you should record the LEDs as either off or on. If they blink record them as on - if they blink in a repeatable pattern, you could also record both the 'blink' and 'non-blink' states for each CPU <see below for explaination>.

A repeating pattern of blinking LEDs is actually a good thing. Static LEDs after CPU initialization is almost always a bad sign, a repeated pattern of blinking LEDs after CPU initialization is expected. Once the CPUs have initialized <but prior to IRIX booting>, the expected behavior is that the Nodeboard LEDs will alternate <blink> "00" and "55". I interpreted your smiley face rendition of the LEDs to read:

That and the display of POM1C on the MSC suggest to me that there's nothing wrong with the compute-side hardware in your Onyx2.

I'd suggest connecting a serial terminal to check the output of the power on diagnostics. If you can get to the PROM, you might check an hinv to make sure all looks well <running 'update' probably wouldn't be a bad idea either>. If all looks well at the PROM level, try booting the system with the serial terminal connected.

If it boots, I'd suggest testing the graphics hardware with irsaudit. http://www.nekochan.net/wiki/ ... h_irsaudit

The proximity of the your newly developed problem and the addition of your newly acquired RM10s would make failure to initialize graphics a more likely suspect. Your Onyx2 might be halting at the PROM because there aren't any graphics, or might even be booting but not generating a display.

***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************

Ryan Fox
From Cape Town, South Africa
Who joined Dec. 20, 2004, 11:03 a.m.

Wrote the following at June 21, 2010, 12:19 a.m...

As recondas mentioned: Whip out a serial terminal or fire up HyperTerm and go through the usual PROM clearing and updating procedures. Saved me many hours of irritation. Also if you wait I can provide information on how to disable extensive memory
testing reducing the POST-to-boot to a few minutes

MAYA, nut-
:Octane2:

Octane 2 R14k 600 V12 4GB, Octane2 R14K 600 V10 1GB ,
:Onyx2:

Onyx2 IR3 4GB Quad R14K 500 DIVO, Onyx2 IR Quad R12K 400 2GB,
:Indigo2:

SGI Indigo 2 R8K75 TEAL Extreme 256MB,
:Indigo2IMP:

SGI Indigo 2 R10K 195 Solid Impact 256MB, MAX Impact Pending ,
Apple G5 Quad, NV Quadro 4500 + 7800GT, 12GB RAM
Sun Blade 1000 Dual 900 XVR 1000 4GB
Sun Blade 2000 Dual 1200 XVR 1200 8GB

Thomas W.
From Germany/NRW/Bonn
Who joined Sept. 9, 2003, 11:54 p.m.

Wrote the following at June 23, 2010, 8:12 a.m...

I have fired up HyperTerm...

Code: Select all


   IP27 PROM SGI Version 6.94  built 04:00:43 PM Dec  5, 2001
   

   Testing/Initializing memory ...............             DONE
   

   Copying PROM code to memory ...............             DONE
   

   Discovering local IO ......................             DONE
   

   Discovering NUMAlink connectivity .........             DONE
   

   Found 2 objects (2 hubs, 0 routers) in 10464 usec
   

   Waiting for peers to complete discovery....             DONE
   

   Global master is /hw/module/1/slot/n2
   

   Testing/Initializing all memory ...........             DONE
   

   Checking partitioning information .........             DONE
   

   Loading BASEIO prom .......................             DONE
   

   

   BASEIO PROM Monitor SGI Version 6.156  built 11:26:28 AM Nov 18, 2003 (BE64)
   

   4 CPUs on 2 nodes found.
   

   Installing PROM Device drivers ............
   

   Base I/O Ethernet set to /dev/ethernet/ef0
   

   Installing Graphics Console...
   

   graphics install: searching for pipe 0
   

   

   Walking SCSI Adapter 0 (/hw/module/1/slot/io1), (pci id 0)
   

   1+ 2- 3- 4- 5- 6+ 7- 8- 9- 10- 11- 12- 13- 14- 15- = 2 device(s)
   

   

   

   Walking SCSI Adapter 1 (/hw/module/1/slot/io1), (pci id 1)
   

   1- 2- 3- 4- 5- 6- 7- 8- 9- 10- 11- 12- 13- 14- 15- = 0 device(s)
   

   

   Initializing PROM Device drivers ..........             DONE
   

   Cannot connect to keyboard -- check the cable.
   

   Cannot open /dev/input/ioc3pckm0 for input
   

   Cannot connect to keyboard -- check the cable.
   

   Cannot open /dev/input/ioc3pckm0 for input
   

   Checking hardware inventory ...............              /hw/module/1/slot/n1:ME
   

   M BANK 0 missing or disabled
   

   Found new or re-enabled component MEM BANK 2 3 4 5 6 7
   

   ***Warning: Found a new IP27 board in module 1, slot n2, serial MHV535
   

   Please use the 'update' command from the PROM Monitor to update the inventory
   

   ***Warning: Found a new PCI_XIO board in module 1, slot io2, serial GLY410
   

   Please use the 'update' command from the PROM Monitor to update the inventory
   

   ***Warning: Found a new MSCSI board in module 1, slot io3, serial HPP125
   

   Please use the 'update' command from the PROM Monitor to update the inventory
   

   DONE
   

   

   **** System Configuration and Diagnostics Summary ****
   

   CONFIG:
   

   No. of NODEs enabled    = 2
   

   No. of NODEs disabled   = 0
   

   No. of CPUs enabled     = 4
   

   No. of CPUs disabled    = 0
   

   Mem enabled             = 3456 MB
   

   Mem disabled            = 768 MB
   

   No. of RTRs enabled     = 0
   

   No. of RTRs disabled    = 0
   

   

   DIAG RESULTS:
   

   /hw/module/1/slot/n2/node/mem: MEMBANK(S) 2  disabled
   

   Reason:
   

   Bank 2: Some DIMMs failed mem test.
   

   /hw/module/1/slot/n1/node/mem: MEMBANK(S) 0  disabled
   

   Reason:
   

   Bank 0: Some DIMMs failed mem test.
   

   **** End System Configuration and Diagnostics Summary ****
   

   

   

   System Maintenance Menu
   

   

   1) Start System
   

   2) Install System Software
   

   3) Run Diagnostics
   

   4) Recover System
   

   5) Enter Command Monitor
   

   

   Option?

"hinv" from within HyperTerm...

Code: Select all


   >> hinv
   

   System  SGI-IP27
   

   4 400 MHz IP27 Processors
   

   Main memory size: 3456 Mbytes, (768 Mbytes disabled)
   

   Graphics Controller
   

   Integral SCSI controller
   

   Integral SCSI controller
   

   Integral SCSI controller
   

   Integral SCSI controller
   

   Integral SCSI controller 0
   

   Integral SCSI controller 1
   

   Integral Fast Ethernet
   

   IOC3 serial port, (adapter 0)
   

   Integral Fast Ethernet
   

   IOC3 serial port, (adapter 0)
   

   Disk drive: unit 1 on SCSI Controller 0, (dksc(0,1,0))
   

   CDROM: unit 6 on SCSI Controller 0, (cdrom(0,6,7))

Where are the RM10??

Thomas W.
From Germany/NRW/Bonn
Who joined Sept. 9, 2003, 11:54 p.m.

Wrote the following at June 23, 2010, 8:20 a.m...

Code: Select all


   System Maintenance Menu
   

   

   1) Start System
   

   2) Install System Software
   

   3) Run Diagnostics
   

   4) Recover System
   

   5) Enter Command Monitor
   

   

   Option? 1
   

   

   

   Starting up the system...
   

   

   IRIX Release 6.5 IP27 Version 10060437 System V - 64 Bit
   

   Copyright 1987-2004 Silicon Graphics, Inc.
   

   All Rights Reserved.
   

   

   Setting rbaud to 19200
   

   The system is coming up.
   

   

   Moving /core to /usr/tmp/core.postinst_detected
   

   An old core file has been found at /core. To reduce
   

   the chance of running out of disk space on the root
   

   filesystem during postinst processing, the file /core
   

   is being moved to /usr/tmp/core.postinst_detected.
   

   WARNING: ef0: link fail - check ethernet cable
   

   ONYX2_IR2E: Unknown host
   

   routed: Send bcast sendto(ef0, 192.168.2.255.520): Network is down
   

   ESP has already been started
   

   inst:
   

   inst: Software installation has installed new configuration files and/or saved
   

   inst: the previous version in some cases.  You may need to update or merge
   

   inst: old configuration files with the newer versions.  See the "Updating
   

   inst: Configuration Files" section in the versions(1M) manual page for details.
   

   inst: The shell command "versions changed" will list the affected files.
   

   inst:
   

   inst: These directories were unable to be moved properly during the
   

   inst: installation process.  Check for any user-modified files, then
   

   inst: delete the directories.
   

   inst:    /usr/include/Vk.O
   

   IR0: ARM: Welcome to ARMLand - 0/0x0d00
   

   IR0: ARM: running...(sherwood-root 0410060304)
   

   IR0: ARM: ******************************************************
   

   IR0: ARM: * InfiniteReality/Reality Software, IRIX 6.5 release *
   

   IR0: ARM: ******************************************************
   

   

   

   

   ONYX2_IR2E console login: IR0: GE0: Welcome to GELand!
   

   IR0: GE1: Welcome to GELand!
   

   IR0: GE2: Welcome to GELand!
   

   IR0: GE3: Welcome to GELand!

bigD
From Socal
Who joined June 17, 2008, 8:38 p.m.

Wrote the following at June 23, 2010, 8:22 a.m...

Thomas W. wrote: Where are the RM10??

Do this:

Code: Select all


   /usr/gfx/gfxinfo -v

recondas
Moderator
From NC - USA
Who joined June 6, 2004, 5:55 p.m.

Wrote the following at June 23, 2010, 8:30 a.m...

Thomas W. wrote: Where are the RM10??

Thomas - SGI provided a program, "irsaudit", that will do in-depth testing of InfiniteReality graphics boards. I'd strongly suggest you run the program and report back the results section. I wrote a wiki that might prove helpful: http://www.nekochan.net/wiki/ ... h_irsaudit There's an example of an irsaudit run and a screen capture of the results section in this post: viewtopic.php?f=3&t=16922&p=132650&#p132650

You might also want to stop in the PROM, select option 5, and run "enableall" and "update" at the PROM command line.

If the the disabled memory banks don't come back up, you'll probably need to drop into POD mode and clear/reset the power-on diagnostic logs. viewtopic.php?f=3&t=16720907&p=7299835&#p7299835

***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************

Thomas W.
From Germany/NRW/Bonn
Who joined Sept. 9, 2003, 11:54 p.m.

Wrote the following at June 23, 2010, 11 a.m...

recondas wrote: You might also want to stop in the PROM, select option 5, and run "enableall" and "update" at the PROM command line.

done that... now 2 CPUs are gone...

Code: Select all


   IP27 PROM SGI Version 6.94  built 04:00:43 PM Dec  5, 2001
   

   Testing/Initializing memory ...............             DONE
   

   Copying PROM code to memory ...............             DONE
   

   Discovering local IO ......................             DONE
   

   Discovering NUMAlink connectivity .........             DONE
   

   Found 2 objects (2 hubs, 0 routers) in 10467 usec
   

   Waiting for peers to complete discovery....             DONE
   

   Global master is /hw/module/1/slot/n2
   

   Testing/Initializing all memory ...........             DONE
   

   *** Barrier sync warning: local=5090, NIC 0x59a385=4963 (promrev mismatch?)
   

   Checking partitioning information .........             DONE
   

   Loading BASEIO prom .......................             DONE
   

   

   BASEIO PROM Monitor SGI Version 6.156  built 11:26:28 AM Nov 18, 2003 (BE64)
   

   2 CPUs on 1 nodes found.
   

   Installing PROM Device drivers ............
   

   Base I/O Ethernet set to /dev/ethernet/ef0
   

   Installing Graphics Console...
   

   graphics install: searching for pipe 0
   

   

   Walking SCSI Adapter 0 (/hw/module/1/slot/io1), (pci id 0)
   

   1+ 2- 3- 4- 5- 6+ 7- 8- 9- 10- 11- 12- 13- 14- 15- = 2 device(s)
   

   

   

   Walking SCSI Adapter 1 (/hw/module/1/slot/io1), (pci id 1)
   

   1- 2- 3- 4- 5- 6- 7- 8- 9- 10- 11- 12- 13- 14- 15- = 0 device(s)
   

   

   Initializing PROM Device drivers ..........             DONE
   

   Cannot connect to keyboard -- check the cable.
   

   Cannot open /dev/input/ioc3pckm0 for input
   

   Cannot connect to keyboard -- check the cable.
   

   Cannot open /dev/input/ioc3pckm0 for input
   

   Checking hardware inventory ...............
   

   Warning: Inventory table ID value is 0. Check Midplane NIC
   

   Writing 4 records.... DONE
   

   Updated new configuration. Wrote 4 records.
   

   

   **** System Configuration and Diagnostics Summary ****
   

   CONFIG:
   

   No. of NODEs enabled    = 1
   

   No. of NODEs disabled   = 0
   

   No. of CPUs enabled     = 2
   

   No. of CPUs disabled    = 0
   

   Mem enabled             = 2112 MB
   

   Mem disabled            = 0 MB
   

   No. of RTRs enabled     = 0
   

   No. of RTRs disabled    = 0
   

   

   DIAG RESULTS:
   

   ALL DIAGS PASSED.
   

   **** End System Configuration and Diagnostics Summary ****
   

   

   

   System Maintenance Menu
   

   

   1) Start System
   

   2) Install System Software
   

   3) Run Diagnostics
   

   4) Recover System
   

   5) Enter Command Monitor
   

   

   Option?

recondas
Moderator
From NC - USA
Who joined June 6, 2004, 5:55 p.m.

Wrote the following at June 23, 2010, 11:20 a.m...

Thomas W. wrote: done that... now 2 CPUs are gone...

Have you tried entering POD mode to clear and reinitialize the system diagnostic logs? <if you need it, there's a how-to linked in my previous post>

Thomas W. wrote:

Code: Select all
Checking hardware inventory ............... Warning: Inventory table ID value is 0. Check Midplane NIC

That doesn't sound good. If you pull any boards you might want to visualize the midplane to make sure the NIC hasn't been knocked loose. Never had the deskside version of an Onyx2, so I'm not sure of it's location, but it should be similar in appearance to a coin cell battery.

If the NIC hasn't been physically disturbed <or if you just want to play it safe>, you might pull the RM10s and re-try things with your original configuration - just in case one or both of the RM10s have an electrical malfunction or some other issue that's causing your sudden rash of problems.

***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************

rusti
From Brussels, Belgium
Who joined Nov. 29, 2009, 2:20 a.m.

Wrote the following at June 24, 2010, 1:39 p.m...

I just took a picture of an Onyx2 Deskside midplane. The NIC is marked. So if you look for it check under nodeboard N1 near the bottom.

Current inventory: :O2:

Former members: :PI:

bigD
From Socal
Who joined June 17, 2008, 8:38 p.m.

Wrote the following at June 24, 2010, 7:16 p.m...

I can't believe I suggested gfxinfo when the poor guy can't get his Onyx2 to boot. That's what you get for drinking and posting! :oops:

I'm interested to see how this turns out. I dropped RM10s into my Onyx2 without any trouble. If the NIC seems fine, I'd do what recondas mentioned and roll back to the RM9s. And definitely clear the logs via POD - that'll probably get your other node back online.

hamei
From over the rainbow
Who joined Feb. 24, 2004, 4:10 p.m.

Wrote the following at June 24, 2010, 8:43 p.m...

bigD wrote: ... . If the NIC seems fine ...

Tangential to the problem here but that NIC ... looks like the NIC from an Octane. Makes me wonder if it's possible to transfer NIC's between an Octane and an Onyx2 ? Or vicee versee ?

rusti
From Brussels, Belgium
Who joined Nov. 29, 2009, 2:20 a.m.

Wrote the following at July 2, 2010, 1:01 a.m...

Thomas: Did you make any progress? If the NIC is the problem and a replacement NIC would be helpful, let me know because the one on the photo would be available.

Current inventory: :O2:

Former members: :PI:

Thomas W.
From Germany/NRW/Bonn
Who joined Sept. 9, 2003, 11:54 p.m.

Wrote the following at July 2, 2010, 2:26 a.m...

at the moment i have not the time to tinker around with the Onyx2

maybe next week... i hope

recondas
Moderator
From NC - USA
Who joined June 6, 2004, 5:55 p.m.

Wrote the following at July 24, 2010, 2:08 p.m...

So Thomas W. - is your Onyx2 still floating belly up <or did you figure out what was wrong>?

***********************************************************************
Welcome to ARMLand - 0/0x0d00
running...(sherwood-root 0607201829)
* InfiniteReality/Reality Software, IRIX 6.5 Release *
***********************************************************************

SGI: Hardware

Changed RMs - Now Onyx2 Deskside Won't Boot