SGI: Hardware

So I'm using this old D9 Stone with 8 FC-AL 300gb HDDs to store various bits and blobs (family pics, mp3, home recordings etc.). One problem I have is that the HDDs mount in slightly different order every time I reboot. I cannot figure out why. Is there a simple way to get the HDDs to mount the same way every time so it doesn't wreck havoc in my symbolic links ?

No Fiber Channel or IRIX automount expert around here ?? Someone must have encountered this problem before...

That D9 probably came with different disks originally.

What you want to do is control the order in which the disks spin up (staggered spinup). Disks often have a jumper to choose between spin-up when power is applied and spin-up on command. In the former case, the disks will all spin up at the same time and claim an ID from the SES more or less random. It also put a big strain on the PSU of the D9 when all disks spin up.

If you can make them spin up on demand, or after a certain delay, the outcome should be more predictable.

Thanks for your reply. The disks have no jumper whatsoever. Is there a software way to control disk spin-up ?

Doesn't the Discreet (Autodesk) software that comes with those things have a way of taking care of that for you?

I do not use the Stone+Wire Discreet driver as the Stone comes from another system. I only use it with as a standard storage array with the standard Irix driver. I clicked "OK" when Irix proposed me to automount the drives at startup. I haven't seen any option anywhere to delay the spindle of the drives individually or anything else for that matter. Hence the situation...

Disclaimer : I don't know anything about fiber channel. But I have a couple of sata disks that do something similar to what you describe ..

Does each disk show up individually ? /dev/dksc/dsk3d2s7 etc etc ? If so, I am guessing you have your links interlaced ? Because on my system, there's a mount point for each disk on the boot filesystem. There's no problem with this setup, it doesn't care when each disk comes online. The boot disk comes up first, then the subdisks can mount in any order.

I'm not sure they would ever have done what you are talking about because in a big installation, what if disk #3 with mountpoints for disks 4 and 5 failed ? Then everything dependant on it would also fail ...

What about using XVM and making a raided drive?

hamei wrote: Disclaimer : I don't know anything about fiber channel. But I have a couple of sata disks that do something similar to what you describe ..

Does each disk show up individually ? /dev/dksc/dsk3d2s7 etc etc ? If so, I am guessing you have your links interlaced ? Because on my system, there's a mount point for each disk on the boot filesystem. There's no problem with this setup, it doesn't care when each disk comes online. The boot disk comes up first, then the subdisks can mount in any order.

Yes, each disk show up individually. Yes my links are all mixed up. Precisely, I devised a small website for my own usage with data stored on the arrray and symbolic links pointing to paths within the array are all mixed up everytime I reboot the system.

Basically, my question is :

How do you maintain correct and stable symbolic links from the root to the content of the FC array thoughout several reboots of the whole system ??? There must be a simple way as it seems to me a really basic functionality. I'm no Unix specialist, although I used Irix, SunOs and Aix in various dev environment from 1994 to 2002. But I was a software developer, not a sysadmin. I have no idea how such common problem is handled in a standard environment.

hamei wrote: I'm not sure they would ever have done what you are talking about because in a big installation, what if disk #3 with mountpoints for disks 4 and 5 failed ? Then everything dependant on it would also fail ...

What you're describing is an entirely different problem concerning mountpoints. For example, in a SCSI array, each disk has a fixed SCSI id and show up every time in the same device in the filesystem (usually with a naming scheme related to the SCSI id), regardless of spindle time or any other consideration.

uunix wrote: What about using XVM and making a raided drive?

Doesn't XVM require some kind of license ? I do not trust RAID arrangements for my own small tasks anyway. I prefer having the simplest possible scheme (JBOD wthout interleaving of the data across the disks, much too risky).

You can use XLV to create stripes and concatenated volumes without a license, it only requires one for plexes (mirroring). Not sure if the situation is the same with XVM, but perusing techpubs it seems to be the same.

From your symptoms it seems like you're using automounting/mediad. If you just turn that off, and add entries to fstab manually, I don't think you will still see things breaking.

robespierre wrote: You can use XLV to create stripes and concatenated volumes without a license, it only requires one for plexes (mirroring). Not sure if the situation is the same with XVM, but perusing techpubs it seems to be the same.

I don't want to create stripes or concatenation as it weakens the integrity of the data greatly. I just want to use my disks in the simplest way.

robespierre wrote: From your symptoms it seems like you're using automounting/mediad. If you just turn that off, and add entries to fstab manually, I don't think you will still see things breaking.

I guess this is what I'm using. I'll look into fstab and report the result. Thanks for you help !

Jack's original comments and J-J's follow-up seemed to indicate that it's normal for these systems to dynamically assign IDs to the drives. Seems pathological to me but as others have indicated, I've avoided Fibre Channel and SANs for the most part.

A close examination of the drive PCBs may reveal jumpers somewhere on the underside, if you don't find them on one end or the other. Take the model number and try to find the spec sheet on the web, that should tell you where to find them. Then get some jumpers and fix the addresses - that should stop the madness of them popping up in different places every power cycle, if I understand what's happening.

Good luck! All this tells me is, stay the heck away from these "stones" - they'll drag you under!

This is helpful

http://www.futuretech.blinkenlights.nl/fc.html

And Chris Kalisiak used to post here a lot, so he is acquainted with Irix and Fiber Channel both. He had a business dealing with Fiber Channel so might still be available if you get stuck.

But this sentence :

"Note that the drive numbers attached to controller 2 are sequential starting with zero, and the drives attached to controller 3 are somewhat random. This is because the eight are in a commercial array with predetermined ID's, and the six are in a configuration known as a JBOD, or "Just a Bunch of Disks", using my T-Cards with somewhat random drive ID jumpers"

indicates that fiber channel disks should have i.d. jumpers ?

Here's his store :

http://ckcomputersystems.com/

Nice guy, he doesn't bite, likes SGI's ...

hamei: he means that the numbers are non-consecutive, not that they constantly change. he says that the jumpers are on the test card.

The loop ID of a fibre channel disk is set by seven pins on its connector, they do not have jumpers.
Normally they would be fixed by the backplane based on location, with the high bits variable for chaining multiple enclosures together. That assignment can be changed at runtime if the enclosure has "intelligent services", an embedded microcontroller for configuring the array. That is different from a RAID processor. The way you communicate with the services is either as a target (it has its own bus/loop interface), or as a slave to a designated FC disk using ESI/SFF-8067. You should be able to tell them apart by a scsi bus probe. One of the functions could be assigning loop IDs to drives based on location (e.g. so you can provide a hot spare disk).

looking for info about these D9 arrays, they seem to use SES (a dedicated target that provides services).
http://download.autodesk.com/us/systemd ... _guide.pdf

The drive does not "grab" an address based on its spinup order, since the address is what determines the spinup order when staggered start is enabled. All these options are controlled by SCA connector pins, none of it is managed by the drive.

robespierre: What you say makes sense, thanks. Looks like my original impulse to resist opening my mouth was correct - I don't know about this, and am not helping...

I'll tell you something though - I believe I've got Storage Enclosure Services in my SE3016 JBOD. If it ever starts changing what IDs the drives get without my having moved them around, it's getting replaced...

Thanks everyone for your insights. I guess the only solution left for me is to write some kind of script that will mount the drives in some temp mountpoints and then identify each of them by checking the existence of a id file in them and then remount them at the right locations based on that. Ugly solution I hoped to avoid !

So when you do a hinv from winterm the disk id's are different each time you boot? what if the array stays powered on across reboots?

Yes, the ids are slightly different every time. Not totally jumbled, but the 1 and 2 are swapped etc. Just slightly different every time. The 7th never becomes the first but it might become the 6th, you get the picture. I never tried to let the stone powered on since it burns about 150 watt/h (the octane burns about 250 watt/h) so for the time being I only power them when I feel like toying with old computers. The long term plan is to let them powered up 24h but I still don't have an upload rate worhty enough of it (my street is only DSL, not optical fiber so the rate is really low).

Howdy. Very interesting timing, that I happened to stop back for a visit while this thread is relatively fresh.

I'll need to digest the discussions, and I'll stop back later, but what I can offer for sure is that there are no jumpers on an FC drive of any consequence. There are a few, but they're mostly for the manufacturer.

The FC-AL Loop ID is assigned by the backplane the drive connects to. Our T-Cards have jumpers on them that let you configure them to specific FC-AL addresses. The DS9 probably does assign the same FC-AL addresses every time, but it might be a different discovery order by the SGI box that's causing them to be mounted at different mount points.

I'll stop back again later today.

Chris