Getting Started, Documentation, Tips & Tricks

Ive gotten a few sgi machines in the past months and have been debating on getting an altix 350, they look like really interesting machines, and the price seems to be low enough that I can build up a respectable system in a few months, however I dont know anything about them, so im hoping someone here can fill in some of the blanks for me. Ive been digging through some of the manuals and documentation, but there is quite a bit to go through, i was hoping someone here could give me some clear answers. I really dont want to drop a lot of mony and end up with something i cant use.

1- I see you can connect multiple machines together with numalink cables, which is kinda one of the main selling points for me, but im unclear on how exactly it works. If it like a fast cluster, or does the memory and cpu's across all the machines appear to all be on the same machine? Do I need software written specifically to take advantage of this, or can i just run any old program and it will be none the wiser that its all on seperate machines.

2 - Can I use just a standalone altix 350? As in just buy one and connect more later as i need? I dont see much in terms of io on the back of them, just a console port, numa link, power. Can I just add any old pci card in there or do i need a special SGI ones? I suppose my end goal here is to get a system with enthernet I can ssh into. Possibly vga too if its posible.

3 lastly, i see there appear to be several different versions of the Altix 350, different outer appearances. Links to different ones ive seen on ebay at end of this post. Can i use these different ones together or do they all need to be the same. Does one of these have better features than the others that I should be more inclined to buy?

Again, any advice is much appriciated. I really rushed into buying my Indigo, and the unseen costs of getting the extra proprietary keyboard, mouse and memory really added up, I dont want to make that mistake agian, and really be clear on what im planning on getting this time

--thanks

Links to different Altix 350 models :
http://www.ebay.com/itm/350868875661?ss ... 1423.l2649

http://www.ebay.com/itm/190914296364?ss ... 1423.l2649

MrBill wrote: Ive gotten a few sgi machines in the past months and have been debating on getting an altix 350, they look like really interesting machines, and the price seems to be low enough that I can build up a respectable system in a few months, however I dont know anything about them, so im hoping someone here can fill in some of the blanks for me. Ive been digging through some of the manuals and documentation, but there is quite a bit to go through, i was hoping someone here could give me some clear answers. I really dont want to drop a lot of mony and end up with something i cant use.

1- I see you can connect multiple machines together with numalink cables, which is kinda one of the main selling points for me, but im unclear on how exactly it works. If it like a fast cluster, or does the memory and cpu's across all the machines appear to all be on the same machine (yes) ? Do I need software written specifically to take advantage of this (no) , or can i just run any old program and it will be none the wiser that its all on seperate machines.

It looks like a single system image, same as MIPS stuff.
I say no above, but you sort of do need special software, it need to be multi-threaded and it needs to scale well.

or you can do large parallel builds eg: gmake -j(num-cpus - 1)

MrBill wrote: 2 - Can I use just a standalone altix 350? As in just buy one and connect more later as i need? (yes, you can upgrade) I dont see much in terms of io on the back of them, just a console port, numa link, power. Can I just add any old pci card in there or do i need a special SGI ones? (There is a limited list of cards supported, what are you expecting graphics?, these are run headless) I suppose my end goal here is to get a system with enthernet I can ssh into. Possibly vga too if its posible.

R.

I've long been tempted to do what you're contemplating but I'd do it with Origins not Altixs. My reason is simple; IRIX and MIPS vs. Linux and Intel. Yes the Origin loses the almighty PETAFLOP bragging rights to Altix, but then Altix loses that (all the time) to Big Blue,

Plus, I think the Origin's colors are more decorative...

Yeah the origins look much more stylish, especially in that fancy rack. You can numa link them together like the altix ones?

Ill be honest, I would kill for a deskside or rackmount origin server. The problem is that I just cant feasibly ship them to me, and driving to get them is equally expensive.

Heres a bunch of complete racks full of 16 Altix 350's on ebay, loaded and ready to go. $800 per rack , again, just no way of getting it here, so im going to have to build one up peice by peice, probably paying more for it in the end. If anything is a disappointment its going to be having a load of these servers and not having that fancy rack to go along with it.

http://www.ebay.com/itm/Server-Rack-w-S ... 2eca9d750b

In response to PymbleSoftware,

Thank you for your response. Its good to know that I wont need specially written software.

(There is a limited list of cards supported, what are you expecting graphics?, these are run headless)

Alright, good to know, at least i know now that not every pci card is supported. As for graphics, i did not know weather they supported it or not. Not a setback, i actually will prefer it that way.

Again, the info is very much appriciated.

Hi

You can run a standalone machine (the one with the IO9 or IO10 card, which included Ethernet by the way), then power it off, connect as many as you want (up to 8 blades) via Numalink cables, then restart the system. As long as the SAL (similar to PROM or BIOS) versions are the same and some other details (serial numbers, etc.) the same system will power up, except that it will have more processors and memory. If you connect via ssh -X for instance, you'll find exactly the same system parameters, desktop, etc. This is actually how I "play" with mine: most of the time I power on only the Base Compute Module and sometimes connect the other ones.

You don't need special softwares... that's more or less right. Don't forget that those systems are built around Itanium processors (IA-64). Therefore you will not find so many soft for them. However, you can compile them from source.

Regarding the PCI video card, search the forum. This topic has already been discussed more than once.

You can also search for more information about the OS that could be installed. RHEL and SuSE were officially supported, and some people successfully installed others as well.

The big thing is to get one with an IO9/IO10 controller inside it. Those typically go for more than the expansion modules, though in recent months I was able to pick up a few spare IO9/IO10s for my Prism and A350s.

Another thing to note is the varying CPU speeds 1.3ghz to 1.6ghz (with some difference in cache mixed in there), if you care, the 1.6ghz models are a bit more rare, especially the 9mb cache versions. But they do tend to show up individually on eBay from time to time, and it is just a drop in.

As smi and I (among others) noted: http://forums.nekochan.net/viewtopic.php?f=3&t=16727949 , ram is very picky. So if you find one with > 2gb of ram, consider yourself lucky. I've got 60gb+ of 2gb dimms that aren't compatible.

They are fun with the SSI support, but I get a lot more out of playing with my O300s. In hindsight, having 12 A350s and a Prism is kind of silly

If you don't already have some altix-specific software to run on this, it's not worth it.
we use it for quicktransit.

mia wrote: If you don't already have some altix-specific software to run on this, it's not worth it.
we use it for quicktransit.

Like BetXen said, if you have open source software and don't mind compiling yourself it's not that bad. It took a Saturday to get SVN built from source (along with the dependencies) along with a complete C++ build environment up and running with the Intel Compiler. But then again to each his (or her) own. Since retiring my ESXi server I needed a Linux box for work, so it filled that void.

There is something to be said about stuff simply not being ready to go from the opening of the box, the level of satisfaction with getting these machines up and running is well worth it in my opinion.

MrBill wrote: Yeah the origins look much more stylish, especially in that fancy rack. You can numa link them together like the altix ones?

Yes you can numalink Origins, for example the O350 scales to 32 processors and 64 gig memory, and the graphics options are vastly superior to Altix because Onyx InfinitePerformance and InfiniteReality are both compatible with Origin. You can start small because a single compute module works just fine all by itself...

If your intend is to bang your head on the wall porting code, then please do it on Irix, not Linux/IA64.

I realize that It is not X86 and software is going to need to be ported. Im fine with that. Im all for IRIX and mips stuff, i just haven't seen much of it on ebay, and what I do find generally tends to be extremely expensive. If it crosses my path at the right price im more than willing to throw together an origin 350 rack as well.


I think im going to grab a couple of these and pray I can find A base system later, as I dont want to be paying a fortune finding all the hard drive caddys, getting the rails,and numa link cables.

won't be as expensive as your electric bill.

won't be as expensive as your electric bill.

To be honest I completely forgot to think about what it will take to power this. oh well, worst case scenario I can just run the base node like BetXen said, and run all of them when they are needed. I had 3 dell quad xeon rackmount servers in the past, if power usage is similar to them with just these 3 nodes i can manage. Ive actually got a watt meter here, ive been meaning to post a listing of the power usage of my octane and indigo with different cpu / disk / power supply options. Ill post how much it uses once it gets here.

I ended up buying 3 of these. I got everything needed to run the system, and the base node is coming with a debian installation ready to use.
The base node also comes with a fibre channel controller installed, the guy I bought it off of said there is a fibre channel disk array that goes with it, im might grab that too.

I want to thank everyone here for the insightful info.

I guess I should start reading over all the documentation in detail before it gets here

about 225W per node btw.