Everything Else

So unbelievably jealous.

jan-jaap wrote: I like it. What's that storage you've got there

In the aforementioned rack, there is a TP900 with 8x300, a Netapp 3040 running Ontap 8.1.2, a LTO6 drive and 2 netapp shelves with 14x1TB each (they run an aggregate mirrored between the two shelves), the brocade is on the other side of the rack, behind the soekris(es).

Pontus wrote: I've been eyeing the "broder" shelf system

Some things worth sharing about the Broder series. What I wanted from a shelving system was:

* Something that would last me a long time. So nothing with chip wood shelves for example.
* Not something with legs on all four sides because the back legs would rest on my cable gutter.
* Something 60cm deep. Most shelving systems are designed with archiving in mind and are 30 - 40cm deep.
* Something with a 'metallic', 'industrial' look to make the colors of the workstations stand out.

Enter the Broder series. That was ~ 1 year before I started construction of the annex and I didn't want to buy it back then because I didn't see the point of spending money on something, only to have to spend more to put it in storage for another year. Of course by the time I went to the Ikea to buy one they had stopped selling it. A quick search on the internet will show that many people are unhappy about this, and second hand prices are close to or sometimes even higher than the new price. It sells quickly second hand, and due to the wide range of configurations chances of finding exactly what you need are not good. I started looking for alternatives but there aren't many. You quickly end up in shops selling store inventory and price tags are *not* hobbyist friendly...

In the end I found someone who was selling a large lot of Broder parts, and had more or less mislabeled them so they wouldn't show up in the searches. I bought the lot. My car (VW Passat Station) was packed front to back, and from the floor to the roof I will eventually sell off the parts I don't need and hopefully reclaim some of the money I spent.

I'm using the 60x120cm grated shelves, and the 60cm 'T' legs. The legs (optionally!) came with adjustable, spring loaded extension to bolt them to the ceiling. I highly recommend these to keep the rack from tipping over.

One thing I had not anticipated was that the tolerances of the parts are rather 'loose'. The extensions I mentioned will rattle if you shake the legs for example. I was afraid that a running computer system would cause nasty vibrations so I applied duck tape in the right places.

Another place where the tolerances are annoying is the brackets that attach the shelves to the legs:


You can imagine that if the hooks are even slightly too far apart it will allow the bracket to 'rotate' on the vertical stand. A rotation of even 1 or 2 degrees will result in the shelf moving 1 cm up or down at the edge -- unacceptable. Judging by the tape remnants everywhere, the previous owner had struggled with this as well. Now I'm happy I've got this ridiculous amount of parts because I can select the best, and then, using hammer and pliers make them just right. I hope it will stay solid for years to come, but only time will tell...

Oh, and yes, one of the three feet of each 'T' leg is still resting on my cable gutter. I think the bulk of the weight is on the center foot and the outer feet are mostly for balance. But I've added some reinforcements inside the cable gutter anyway. What's more annoying is I can't open the gutter without removing the rack first. I will probably cut the lid of the gutter in a couple of places to be able to access the gutter under / behind the rack. That's why this is still not fully assembled, or bolted to the ceiling. I have to be absolutely sure before I cut the floor panels on the cable gutter ...

jan-jaap wrote: Now if only I could shut up that bloody Silkworm, it is absolutely intolerable....

Noise canceling headphones?

jan-jaap wrote: Now if only I could shut up that bloody Silkworm, it is absolutely intolerable....

What model of Silkworm? I recently acquired a pair of 32-port Silkworm 4100s (well, actually, the IBM-badged equivalent) that are intolerably noisy for about 2 minutes after powering them on. Then once their embedded operating system has booted (and the fans are under control of the environmental monitoring software) they quiet down to a much more reasonable level. Maybe a newer Silkworm (or even a firmware update for yours) would help your situation?

BTW, I love that row of desksides with their monitors lined up on top in one of your earlier photos. I don't have anything that large, so to see several together is really impressive!

I really like your lighting. When I did my room, I considered putting in similar recessed lights to supplement the large fluorescent fixtures I installed, putting them on a separate circuit for more control of the "mood," but in the end left them out to save a bit on cost. After seeing how they look in your room, I'm wishing I'd installed them.

Your whole room looks spectacular! I would definitely be happy spending my free time there.

jpstewart wrote: What model of Silkworm? I recently acquired a pair of 32-port Silkworm 4100s (well, actually, the IBM-badged equivalent) that are intolerably noisy for about 2 minutes after powering them on. Then once their embedded operating system has booted (and the fans are under control of the environmental monitoring software) they quiet down to a much more reasonable level. Maybe a newer Silkworm (or even a firmware update for yours) would help your situation?

Mine's a HP branded Silkworm 3800. 2Gbit generation, fans always in lift-off mode.

I guess I should start looking for a Silkworm 4100 or maybe even a 5000 if I can get it for the right price. The 5000 uses only 50W -- should be silent *cough*. They are tricky to pick up 2nd hand though, because if you get one with a boot and FOS password set you need a Brocade contract to unlock it, and you have to be sure nobody wiped the license codes or it's a door stop as well.

I've got dual 4GBit adapters in several systems (Tezro, O350, my server etc). If I get a 32port FC switch and partition it into two zones I won't need a second switch to get dual paths to storage.

jpstewart wrote: BTW, I love that row of desksides with their monitors lined up on top in one of your earlier photos. I don't have anything that large, so to see several together is really impressive!

I think the monitors will go away in the end. I want to concentrate everything on a single keyboard/mouse/monitor on the desk.

I got myself a cupboard for the books, and assembled the rest of the Broder rack. This is how it looks now:





Of course I ran into constraints I didn't see coming with the Broder rack. I would have liked the shelf above the Octanes a little lower, but the clips that keep the vertical stand extensions in place make this impossible. There's a horizontal bar under the shelf above the O2; in the center there's a screw which is going to kill the skins of the O2 one day. The bar used to be above the Indigo's but it doesn't fit there either. I would like to be able to access the power switches of the Indigo's, but like this I can't reach them, at least the center one. OTOH, I have this PDU under the floor with 24 switchable outlets, so what the heck.

I'm not sure the Discreet disk box will stay, it's not *that* pretty, and I have a TP9100 JBOD in the 19" rack. Also, there's a total of 16 systems (17 if I hook up the 4D/380 as well) and 16 ports on the KVM switch (to the left of the O2). I may move the Octane2 to the other side of the room. It's V12+DCD is rather wasted on an analog KVM.

There's no cables anywhere yet, so this is still more or less a mockup. But now at least I know how long the cables have to be.

The pictures are not the best ever. I should be using a tripod or (better) take them during the day

what's on top of the indy, a scsi cdrom?

Whooee, comin' along!

mia wrote: what's on top of the indy, a scsi cdrom?

Yup, a Toshiba XM-3501B, a 4x speed external SCSI CD-ROM. Caddy loading. Pretty sure that was what you got if you ordered the CD-ROM option with an Indigo or Indy. This one is from 1995.

It could use a retr0bright treatment

jan-jaap wrote: I may move the Octane2 to the other side of the room. It's V12+DCD is rather wasted on an analog KVM.

Another attempt. First a better pic of yesterdays setup:

Next: removed the Octane2, and shuffled the rest around. Added 1600SW + keyboard, mouse:

I think this is an improvement, but I haven't made up my mind completely.

This would free up two slots on the KVM, bringing the total to 15 (incl. the 4D/380), and I can reach the power switches of the Indigos. The O2 would be standalone; the KVM will be connected to a more standard (1280x1024) monitor on my desk. More confortable to work with, and most old SGI's don't deal well with the 1600x1024 resolution of the 1600SW. Plus I'm not blown away with the picture quality of the 1600SW. I'm sure it was good back in the day, but these days even a decent (old) CRT will look better.

jan-jaap wrote: If I get a 32port FC switch and partition it into two zones I won't need a second switch to get dual paths to storage.

I work with FC-SANs on a regular basis and a rule of thumb is to *always* make seperate zones between each initiator (host) and target (storage) - if you don't intend to use a HA-config with shared storage, which would keep SCSI reset/reservations within the cluster.
That way you don't confuse other initiators (hosts) when doing SCSI polls and keep SCSI reset conflicts down to a minimum.

jan-jaap wrote:

jan-jaap wrote:

What an absolutely gorgeous eye-candy photograph! Congratulations on such a spectacular looking computer room! I am waiting with baited breath to see future pictures of the completed setup and of the environment you have running on individual systems. Will you have a couple of work areas/workstations where you will temporarily move machines in when using them? I have often wondered about a few desks each with a single KVM setup tied to several machines so that all or most machines can always be connected and ready to go. But then KVMs are a pain in the rear, they don't quite work with everything and for me, there's something a lot more appealing about a workstation set-up for a single computer, with peripherals, display & keyboard all arranged together. The many systems/one display/KB visually steals the beauty from the machine because no one computer is arranged together with all its peripherals. The physical aesthetics of an SGI setup are a big part of the experience. Good lighting to show the machine off, environs that focus attention on the system and a display/keyboard/mouse setup that allows you to see the whole system as "one" are all elements to consider in a computer room as beautiful as yours.

The glam pics that got me into SGI as a kid were the famous BYTE Indigo cover, the (also BYTE) Indigo2 with the huge CRT on top showing a cow (!) render, as well as that awesome Indy article with the SGI CRT and video cam all installed and ready to go. The peripherals etc. made at least two of those pics far more beautiful.

Arghhhhhh! So spacious.

I'm living in Hong Kong and can only dream...

jpstewart wrote:

jan-jaap wrote: Now if only I could shut up that bloody Silkworm, it is absolutely intolerable....

What model of Silkworm? I recently acquired a pair of 32-port Silkworm 4100s (well, actually, the IBM-badged equivalent) that are intolerably noisy for about 2 minutes after powering them on. Then once their embedded operating system has booted (and the fans are under control of the environmental monitoring software) they quiet down to a much more reasonable level. Maybe a newer Silkworm (or even a firmware update for yours) would help your situation?

I'm happy to report that the HP SANswitch 2-16 (a.k.a. Silkworm 3800) has been replaced with a shiny new^H^H^H second hand EMC DS-5000B (a.k.a. Brocade 5000). The 5000 is the successor to the 4100. It's also noisy at start-up (though less than the 3800), and then settles down. It's still an 1U unit and those fans still run at ~ 7500RPM, but it's much much more agreeable.

Strangely, the airflow is back-to-front. Ideally I should mount it in the back of the rack, but that's not very convenient. Think I'll ignore that for now.

As a side effect of this noise canceling effort I now have 32*4Gb connectivity instead of 16*2Gb Took me a while to find one with 32 (licensed!) ports for an acceptable price, and on the right continent. As a bonus, it also come with Extended Fabric, Fabric Watch, Performance Monitor and Trunking licenses. That should keep me busy for a while

What kind of storage are you going to feed with that beast?

jan-jaap wrote: Strangely, the airflow is back-to-front. Ideally I should mount it in the back of the rack, but that's not very convenient.

That seems to be somewhat common on switches. I guess in a data centre environment it makes sense to mount the switch at the back of the rack. Putting the switch ports at the back where the servers' ports are can simplify cabling --- no need to pull cables through to the front of the rack. I'll have to check the airflow on my 4100s now. I never thought about that until you mentioned it.

jan-japp wrote: As a bonus, it also come with Extended Fabric, Fabric Watch, Performance Monitor and Trunking licenses. That should keep me busy for a while

Congratulations on a nice score! Those licenses cost a fortune on their own. Have fun with it.

vishnu wrote: What kind of storage are you going to feed with that beast?

Uhm, I've got an IBM DS4300 SAN server (with an extra EXP710 disk tray) and an SGI TP9100 JBOD. Both have dual 2Gb attachments.

I'm gradually upgrading my infrastructure to 4Gb. Several of my SGIs have dual 4Gb cards already. I don't expect to upgrade to 'real' 4Gb FC SAN hardware anytime soon. All disks are 2Gb as well, for example so I'd have to throw out everything. A more likely upgrade path would be that my next file server will run an FC adapter in target mode. With a half decent RAID adapter and SSD caching that should be plenty fast. And I wouldn't need 28 spindles to get 8TB of storage capacity

jpstewart wrote:

jan-jaap wrote: Strangely, the airflow is back-to-front. Ideally I should mount it in the back of the rack, but that's not very convenient.

That seems to be somewhat common on switches. I guess in a data centre environment it makes sense to mount the switch at the back of the rack. Putting the switch ports at the back where the servers' ports are can simplify cabling --- no need to pull cables through to the front of the rack. I'll have to check the airflow on my 4100s now. I never thought about that until you mentioned it.

There's another possibility: the only place where this thing generates any significant amount of heat, is the cluster of GBICs at the front. Airflow front to back would draw the hot air through the chassis. Back to front probably results in lower chassis temperature.

jan-jaap wrote:

vishnu wrote: What kind of storage are you going to feed with that beast?

Uhm, I've got an IBM DS4300 SAN server (with an extra EXP710 disk tray) and an SGI TP9100 JBOD. Both have dual 2Gb attachments.

And your main usage of that enterprise-class storage is for photos of the wife and kids, right?