Hardware Wanted

I am in the market for these for my Octane2. I am located in the US, east coast. PM me if you have these.

TeamBlackFox wrote: I am in the market for these for my Octane2. I am located in the US, east coast. PM me if you have these.

These things are bloody rare. I finally found a pair on eBay a few days ago, and with 6x 128MB modules, it gave my Octane a nice bump to 3.5GB of memory, but Linux seems to have issue with that much memory on this platform. Hard lockup (not even a panic/oops message) on any heavy disk I/O activity, which points the finger at XFS, MD-RAID, qla1280, generic SCSI, the I/O scheduler (though I tried two of them), or the block layer (or any combination thereof). Not really in the mood to debug that by putting my filesystem at risk, so I pulled them for now until I can attempt to figure out what's wrong w/ the IP30 memory code.

That said, if you're wondering what these chips look like for a visual reference, look for memory modules that have "stacked" chips. I don't mean the dual PCB 128MB modules. 1GB modules literally have two memory chip stacked on one another on a single PCB module.

Ian has them listed on his site as in-stock, but at £225 for two 1GB modules, they aren't cheap!

Trippynet wrote: Ian has them listed on his site as in-stock, but at £225 for two 1GB modules, they aren't cheap!

I got mine at a steal then. $130 for the pair.

Still looking.

let me say "sorry", really "sorry".

A lot of time ago (2009) I was given an IP30 equipped with 8 modules of 1Gbyte each, for a total of 8Gbyte of ram (1), but, as the fact the Xbov v1.4 is not well known in linux (i am afraid its documentation is still under NOX), i had to remove 6 modules for a total of 2Gbyte of ram. It was fine without SMP and without PCI, but when i added the PCI cadge i had an other issue, and in order to open a window to the XIO-PCI (in order to have a PCI_USB NEC EHCI chip on) i had to

- get out all my 1Gbyte ram modules
- get in 8x256Mbyte modules


in the exchange i wanted also a SMP 2xR12K@400Mhz module, so my 8Gbyte kit for the 2Gbyte ram kit + SMP, that was the deal, and this to achieve a total of 1.5Gbyte of ram (256Mbyte x 6)

(1) that dude exchanged his SGI Octane2 8Gbyte ram kit for an HP C3600 workstation (PA Risc v2 by HP @ 550Mhz) equipped with 4Gbyte of ram. At the beginning i was a gentoo/HPPA fan, then i switched to gentoo/MIPS-SGI


as far as i understand, the 256Mbyte memory kit comes from Octane1, which has an older Xbow chip, with a limitation in the maximal amount of ram. It was good for me, i was able to exchange the 1Gbyte-ram-module for a very expensive SMP module.

ivelegacy wrote: let me say "sorry", really "sorry".

A lot of time ago (2009) I was given an IP30 equipped with 8 modules of 1Gbyte each, for a total of 8Gbyte of ram (1), but, as the fact the Xbov v1.4 is not well known in linux (i am afraid its documentation is still under NOX), i had to remove 6 modules for a total of 2Gbyte of ram. It was fine without SMP and without PCI, but when i added the PCI cadge i had an other issue, and in order to open a window to the XIO-PCI (in order to have a PCI_USB NEC EHCI chip on) i had to

- get out all my 1Gbyte ram modules
- get in 8x256Mbyte modules

Hmm, in other words, any memory >2GB of RAM on an XBow 1.4 had issues in Linux? I may have ran into this trying to run 3.5GB. I guess I'll have to hunt for XBow information/documentation. I wonder if any of the old Altix code might shed some light. I don't suppose you've come across any XBow documentation?

As for the PCI card cage, that issue might have been the Linux/MIPS PCI scanning code detecting the cardcage first in its bus probe and stopping after that. I have a fix in the mips-sources ebuild now, but it hasn't been accepted upstream yet.

hi Kumba
nice to see you here

i am the guy who has emailed you for the IP28 memory layout, the same guy of the gentoo discussion about 2.6.17 vs 3.* line.

Kumba wrote: Hmm, in other words, any memory >2GB of RAM on an XBow 1.4 had issues in Linux?

everything > 1.5Gbyte has issues with the XIO_PCI-EHCI, no chances about that, unfortunately.

Kumba wrote: I guess I'll have to hunt for XBow information/documentation

I don't suppose you've come across any XBow documentation?

unfortunately no, i have no documentation, just your pdf (which i still have to study and understand more deeply)

Kumba wrote: As for the PCI card cage, that issue might have been the Linux/MIPS PCI scanning code detecting the cardcage first in its bus probe and stopping after that.

no, with 2Gbyte of ram the XIO_PCI has success in its probe but the PCI_EHCI NEC chip does not respond correctly, while if i remove 2 memory modules (each one is 256Mbyte, 8x256Mbyte=2Gbyte, 6x256Mbyte=1.5Gbyte) then the issue disappears and i am able to have a working PCI_EHCI.

We are talking about this configuration (i am the co-author of this ewiki page), so … a pretty old kernel 2.6.17-rc4+hack, you can see the dmesg here .

Kumba wrote: I have a fix in the mips-sources ebuild now, but it hasn't been accepted upstream yet

good, it may be your patch is good to be back ported to 2.6.17 in order to fix the 2Gbyte memory problem.

ivelegacy wrote: everything > 1.5Gbyte has issues with the XIO_PCI-EHCI, no chances about that, unfortunately.

ivelegacy wrote: no, with 2Gbyte of ram the XIO_PCI has success in its probe but the PCI_EHCI NEC chip does not respond correctly, while if i remove 2 memory modules (each one is 256Mbyte, 8x256Mbyte=2Gbyte, 6x256Mbyte=1.5Gbyte) then the issue disappears and i am able to have a working PCI_EHCI.

Oh, that issue. That's the old problem with PCI-to-PCI bridges and other PCI devices not playing really nice with DMA when >2G of RAM. Only way to work around that is likely going to require finding Octane system documentation at some point. Even Stan didn't have a lot of luck in figuring that one out.

ivelegacy wrote: good, it may be your patch is good to be back ported to 2.6.17 in order to fix the 2Gbyte memory problem.

I'd recommend moving on from 2.6.17 and focus on 3.19+. My patch only addressed the PCI bus enumeration problem, and that seems to have been something that came after 2.6.17. There's several known memory issues that I haven't figured out yet (namely why CONFIG_SLUB screws up and the whole R14K TLB problems).

Kumba wrote: Oh, that issue. That's the old problem with PCI-to-PCI bridges and other PCI devices not playing really nice with DMA when >2G of RAM. Only way to work around that is likely going to require finding Octane system documentation at some point. Even Stan didn't have a lot of luck in figuring that one out.

I don't understand the problem, here. The XBridge has a 2GB direct window for trivial DMA address computations, allowing PCI bus masters to access the low 2GB of cpu memory (only 1.5GB on Octane, because physical memory starts at 512MB on this platform).

So either your bus master device can use 64-bit addresses for DMA, and you can put any valid XIO address and reach all your node's physical memory, or it is limited to 32-bit addresses, and you have to make sure it will only attempt DMA to addresses reachable through the direct window.

Alternatively, for small DMA areas, you can setup Address Translation Entries, and direct your 32-bit DMA request to the matching area in the translated window, but ATE are a scarce resource.

@miod
let me understand, are you using gentoo on IP30 ?
if so, which kernel, and which configuration ?

ivelegacy wrote: @miod
let me understand, are you using gentoo on IP30 ?

I am not using gentoo (or any form of Linux) on IP30. I am just stating how the hardware works.

Ivelegacy, Miod is our resident OpenBSD dev.

nice to meet him

Last call, anyone? Before I give up hope and buy new 256MB modules, because my Octane fried the old ones I had >.>

I might have some Octane ram I can spare/sell but I'm out of town until late Friday.

I'm pretty sure I have 6GB in one my Octanes.

thegoldbug

TeamBlackFox wrote: Last call, anyone? Before I give up hope and buy new 256MB modules, because my Octane fried the old ones I had >.>

Did it include some fava beans and a nice chanti?

No, my Octane is Sakuya and not Hannibal Lecter for a reason.