The collected works of Kumba - Page 3

ivelegacy wrote: built in 2008

Okay, about as old as the last one I built (2007).

ivelegacy wrote: it works with SMP=disabled

I figured as much. Although I have not had any severe problems with SMP once I fixed the IRQ setup, I can't rule out that all SMP-related problems have been solved, though.

ivelegacy wrote:

Kumba wrote: and those were solved once I learned to use the correct handle_*_irq() function

isn't kernel matter?

Not following what you're asking here?

ivelegacy wrote:

Code: Select all

kika # uname -r
2.6.39-flesh-eating-bats


she has

Code: Select all

kika # gcc-config -l
[1] mips-unknown-linux-gnu-4.1.2 *
[2] mips-unknown-linux-gnu-4.5.2
[3] mips-unknown-linux-gnu-4.6.3


I used gcc- v4.1.2, uclibc profiled (cross compiler, mips3-be-glibc ---> mips3-be-uclibc)
and I can reuse my OpenWrt builder (kernel v4.2.*, gcc v4.6.*)

do you have a "demo" version ?
do you have a good ram-rootfs ?

I don't have a current initramfs at the moment. This is the thing that has been irritating me to no end over the weekend, is trying to get a uclibc-based mips3 chroot working so that I can create a new initramfs, but there's something jacked up with this uclibc root and the ncurses library. If I recompile ncurses, anything linked to libncurses.so.5.9 starts segfaulting, and I am stumped as to why. I am trying one last nutty idea before I call it quits and just focus instead on making a glibc-based initramfs.

Although, using OpenWRT to kickstart a new uclibc root is an interesting idea. I tried this with buildroot a while back, but they sadly chose to drop support for the old MIPS ISAs prior to mips32r1/mips64r1. When I asked how to undo that, the answer basically involved inserting lots of #ifdef macros in their code, and they didn't seem inclined to specify where. So I abandoned that idea entirely.

As for a demo kernel, I haven't had time to spin you one yet, since I've got the Octane tied up trying to get this uclibc idea working.

My thinking at the moment is, you've just got an incompatibility between the really old code of your initramfs, likely built against 2.6.x kernel headers or such, and the 4.3.x nature of the current kernel. If you were getting the same problems I was, you'd be seeing Data Bus Errors, not Instruction Bus Errors. An IBE means that the CPU attempted to decode a CPU opcode that it doesn't understand, so do make sure your binaries are compiled for MIPS-III or MIPS-IV at a minimum, and that no mips32rX/mips64rX stuff snuck in by accident.

ivelegacy wrote: I haven't understood if "handle_*_irq() function" is related only to the kernel
or if it has a relation with the userspace, and in case where/what?

is it related to "something" in libc? some kernel-helper?

Oh, sorry, it's a kernel function. In the IP30's IRQ initialization code, I was assigning the "handle_level_irq" to all IRQ numbers (HEART has 64 interrupt vectors available). handle_level_irq() isn't percpu-aware, and so it takes a longer code path, which adds latency, when handling a specific interrupt. This is fine for normal IRQs, but not SMP IPI's. So a quick-rejig to assign "handle_percpu_irq" for SMP IPI's was the final piece of the SMP puzzle, and that's how I got my Octane to boot in SMP mode and actually work well.

ivelegacy wrote: they are mips3

So in theory, it should work fine. What PAGE_SIZE is set in your kernel? 4KB should be universal and work with everything. If I get this uclibc-based netboot working, though, I'll find out if there are PAGE_SIZE incompatibilities there, as there were in the musl-based netboot I built. I run 64KB PAGE_SIZE on the Octane right now, as it helps speed compiles up by reducing TLB pressure. musl couldn't handle that well, though.

ivelegacy wrote:

nyef wrote: MIPS-II is MIPS-I plus a handful of new instructions plus pipeline interlocks to eliminate some of the hazards

I wonder if these new instructions & pipeline interlocks introduced by MIPS-II && MIPS-III && MIPS-IV to eliminate some of the hazards, are exposing a "weakness" within the spin-lock in the kernel-SMP, as I still and randomly get Instruction Bus Error on CPU1.

There is also this problem about stages: no man power to build them!

I'm making progress on the MIPS-II uClibc netboot thing. Had to rebuild my three stages all over again after running into some oddities with a few options I had in uclibc's config. Setting up a chroot now to built the netboot by hand in, while mapping out file/libs and looking into scripting the process for the future.

That said, your problem is not tied to spinlocks or anything. If it was a spinlock issue, you'd get a proper oops message and the kernel would likely die pretty fast due to a race condition, sleeping while atomic, etc. I still think it's something really weird about your hardware...maybe there's a hardware bug that simply isn't known about and thus, not being worked around? Hard to know right now. If I can get this netboot actually built and working without any more screwy issues popping up (uClibc apparently has quite a lot of them), then you'll be one of the first to test it to see if you're still having problems. I'll update the Gentoo bug you opened with further info when I have it. Only have time on the weekends to really get stuff done.

Additionally, the R10000 family handles *all* hazards in hardware, I think the only MIPS CPU to do so. Ralf's indicated that's one of the nicest features about this class of CPU.

All that said, stay away from the 4.4.x kernel on Octane for now. Something changed with (I think) Device DMA memory setups and the contiguous memory allocator. You'll get a Oops message when pci-ip30.c tries probing something, then updates devtmpfs (via kdevtmpfsd), and hits an unaligned access bug. So that's on my list to fix now. Cannot reproduce the problem on the Onyx2 and pci-ip27.c, so it's probably Octane-specific. The other possibility is you'll get ~27+ Oops messages in a row, if you have a graphics board installed and booting with the early printk hack enabled. Have to figure that one out, too...

Almost forgot to add, the MIPS-I, MIPS-II, etc, are Instruction Set Architecture (ISA) references. They don't really have much to do with the actual CPU hardware, in a way. Instead, they specify features a CPU should support, such as register size (32bits, 64bits), specific opcodes (MIPS-IV added square root), etc.

The way it works in MIPS is kinda like this:

• MIPS-I (32-bit)
  • MIPS-II (32-bit)
    • MIPS32R1
      • MIPS32R2 ... MIPS32R6
    • MIPS-III (64-bit)
      • MIPS-IV (64-bit)
        • MIPS64R1
          • MIPS64R2 ... MIPS64R6

You can see how ugly it looks when visualized in a tree-like layout, but that shows the compatibility chain. In other words, MIPS32r* branches off of MIPS-II, with some 32-bit backports of 64-bit opcodes added in various MIPS32R* releases (the latest being R6; there was no R4 due to some Asian cultural fear of the number 4). MIPS64R1 was spun off of what WAS going to be MIPS-V, but since MIPS-V never made it into actual silicon (it was supposed to be instituted by the aborted R18000 CPU), it technically doesn't exist. So MIPS64R* can instead be visualized as branching off of MIPS-IV.

ivelegacy wrote:

Kumba wrote: I'm making progress on the MIPS-II uClibc net boot thing

mips-3-uclibC-be was completed them by the end of the year, it's working with kernel 4.*

I can't use your netboot for Gentoo purposes, though. Stuff we support has to be built through our release tools (which is where all of the fun is at...). Glad your build is working for you (would still be curious to know if the problem really was old rootfs + new kernel or something else). But in the end, I still have to make this MIPS-II netboot work for (theoretically) anyone that has an Octane.

ivelegacy wrote:

nyef wrote: See MIPS Run (first edition) says that the R10k has an R4000-style cache

I do not have a copy on hand, I have an hardcover (paper) copy but it's not on hand (250 Km far from where I am) so I am going to buy an ebook for my kindle, this way you always have a copy on hand when you need a lecture, example, or MIPSfun :D

R10K family is an R4K-style cache. In fact, a *lot* of MIPS processors, including many modern-day ones, still have this R4K-style cache layout. If it ain't broke, don't fix it.

ivelegacy wrote: Linux is too bloated, up to 60 Mbyte of sources, I find myself drowning in all their clutter and a bit tired of having to dig through a lot of sources with a lot of #IFDEF to find the article of good source I need!

In the old days, it apparently wasn't too difficult to write assembler that was better than what a compiler can generate. But nowadays, gcc is really, really difficult to beat. So, a lot of the old assembler code that used to power the critical bits of Linux/MIPS code have been replaced by C-equivalents for better readability and maintainability.

That said, yes, certain parts of the Linux tree can be quite a mess (like, why is sound/ at the root instead of under drivers/ ?), but the arch/mips/* folder is pretty well organized:

R4K-cache core code:
http://lxr.free-electrons.com/source/ar ... mm/c-r4k.c

The generic cache interface definitions are here:
http://lxr.free-electrons.com/source/ar ... mm/cache.c

And the relevant header files:
http://lxr.free-electrons.com/source/ar ... sm/cache.h
http://lxr.free-electrons.com/source/ar ... cheflush.h
http://lxr.free-electrons.com/source/ar ... cacheops.h
http://lxr.free-electrons.com/source/ar ... r4kcache.h (this one has templated asm for R4K caches)

Most cache-related and TLB-related code is under the 'mm' directory in the MIPS' arch folder, if you want to look at what other CPUs do. Maybe you'll have better luck than me in reading and understanding it!

ivelegacy wrote: kernel FPU emulation, my RS/P(1) has no FPU, and I am sometimes (too often) using it to compile things for my IP30
to compile the kernel it takes 5 hours, while my IP30 2xR12K@400 takes 40 minutes, just to compare the cpupower
but RS/P eats less than 10Watt of electricity, while my IP30 eats up to 400Watt :shock: :shock: :shock:

How is your Octane pulling 400W? You said you run it headless with no graphics, so it should be well under 300W. Mine has the dual R14K, an Impact board with a PCI shoebox, and 3x 10k RPM 2.5" SCA drives, and it clocks in right around 303W average. These systems don't do power scaling, either, so regardless if it's compiling code or contributing to the ambient room temperature, it'll run at a fairly fixed wattage. And aren't you on 240V mains? That should be more efficient than our silly 120V setup over here in the states.

ivelegacy wrote: measured on my friend machine
- 2xR14K@600Mhz
- 2Gbyte of ram (currently downgraded to 1.7Gbyte)
- V12 + Mgras installed {SlotA,SlotB,SlotD}
- PCI ShoeBox with a dual SCSI controller and PCI_USB
- 2x40Gbyte SCA hard drives

we measured ~390Watt average

Okay, Impact AND Odyssey in the same system, pulling 390W sounds closer to what I'd expect. Your Octane should, however, be around 250W or less, since you removed all of your graphics options. That's actually not a bad draw, and while I don't know what electric rates are like in the EU, it shouldn't be that expensive on a per-month basis, especially if you're only running the Octane a couple hours a day.

ivelegacy wrote: yes || no

Yes : in Europe.Italy we have 240V
No : but my IP30 system is powered by a diesel generator (2), which comes with 120V setup
so I modified the IP30 PSU instead of buying a more expensive converter for the generator (1)
it's 5Kwatt diesel generator, paid 200 euro for the whole as second hand

Okay, I was not expecting that as an answer. Why can't you run the Octane on the 240V system? The power supply is auto-sensing and should handle either 120 or 240 just fine. And what did you modify on the PSU?

ivelegacy wrote: btw, things within the range { 200, 300, 400 } Watt are greater than things within the range { 20, 30, 40 } watt
corollary: RSP & Cavium Octeon eat less electricity -> they are 10x better dogs :D !

Maybe, maybe not. SGI built some wicked gear that has stood the test of time a lot longer than most computer gear normally does. Especially with dual R14K's. What does Cavium have for L2 caches on those chips?

ivelegacy wrote: the House where I live is property of my girlfriend, and even if I pay half the electricity she doesn't like the IP30, in first place

You need a new girlfriend.

If you Google for the right search terms, you'll find the BRIDGE, XBRIDGE, and XBOW manuals. Apparently been sitting out there on a NetBSD dev's site since 2009.

foetz wrote: iirc a quad 300 with an ir2 was around 750w

My 4x R14000/500 Onyx2 pulls about ~708W idle with just one SCSI disk. Graphics is not spun up because I'm booting Linux on the thing. I do have the PCI shoebox installed, but pulled all of the cards. Seems that for each PCI card, you raise the power consumption a couple of watts. I can't measure a similar effect on my Octane. The Onyx2 HD Video XIO board jumps the power draw by 50W easily....probably because that thing has 4-5 Xilinx FPGAs on it.

Highest I got it was ~768W. This is all on a standard US 120V system plugged into a NEMA-15 outlet and measured with a Kill-A-Watt. Wish I had a 240V socket and they made a 240V Kill-A-Watt...

jan-jaap wrote:

ivelegacy wrote: oh, and I also wonder how the CD-drive in the picture can work when its feets are standing up vertically :shock: :shock: :shock:

No problem, there are little catches in the CDROM tray which prevent the disc from falling out. Also quite handy if you want to place an Indigo2 on it's feet.

These catches have been found in CD-ROM drives for a long time. Some require you to flip them out manually (especially in older drives), while newer drives have the tray designed so that little, almost unnoticeable knubs, are built into the outer edge of the tray depression.

Does anyone know of a good source to acquire the 13w3 DIY adapter kits? I've looked on Monoprice, Pccables, Amazon, and eBay. All I can find are the premade cables that are predominantly the Sun-to-PC versions (despite many of them claiming SGI compatibility).

jan-jaap wrote: You can put up to 3 MGRAS graphics boards in an Octane, but I have never heard of the triple CPU option :mrgreen:

Octane's HEART chip was built to support up to four CPUs (and 16 DIMMs), but SGI never expanded the system out to actually use that. I've always wondered if the ramp-design of the HEART/BRIDGE heatsink was to allow for a quad module of sorts down the road.

maxxi.desktop wrote: he's baaaaaack!

Sorry for the lack of updates. My health has been up and down for quite a while, but I think we've figured it out :)

Updates:
1) HP acquired SGI... I've reopened the discussion and waiting for a reply

If you could ask them to dump hardware documentation/specifications for "ISD" or "Godzilla" (Octane system design using HEART), plus the user programming manual for the R14000 and R16000 CPU's, that'd be awesome. HP was using R14K's on their NonStop-series of servers, so I know there's gotta be a manual or errata sheet lying around that details information on the Status and Config registers of those CPUS, just in case any new features or capabilities were added after the R12K. And knowing how HEART works better on Octane would let me solve the damn DMA bug with BRIDGE.

Raion-Fox wrote:

Dodoid wrote: Certainly sounds problematic, but do you believe it is enough of an issue that Qubes use of systemd makes it less secure (even with virtualization and all the security stuff) than something like Devuan? Of course, no code is perfect, but I would think Qubes still has the advantage, even with systemd?

I wouldn't say anything that uses such an insecure architecture like systemd would be more fundamentally secure because of it. I think you're comparing apples and oranges. Gentoo-Hardened vs, say Qubes I'd say that Gentoo-hardened is a great deal more secure. But don't take it from me, we do have a Gentoo developer who lurks. Paging Kumba.

I don't think this board has a paging notifier like on some IRC clients. Luckily, I just randomly click links of interest and stumbled upon this.

I can't speak for Qubes or Devuan and what not. As far as systemd goes, I personally don't use it, and from a strictly technical point of view, I don't think the base idea, in principle, of replacing SysV Init, is itself bad. That said, I personally don't care for it or use it, and I take umbrage at its fan base which has a penchant to declare anything not using systemd to be "broken" in unspecified ways. Kinda like that homeopathic malarkey (butt candles!) that claims to purge multiple unnamed toxins from your body and such.

From the perspective of a Gentoo dev, we aim to let people choose how to best run their system. A standard stage3 install tarball starts you off on OpenRC, but you can switch out to systemd rather easily (I think we have a guide on our wiki; though I believe we have systemd stage3 tarballs now), so you're free to run your system however the hell you want. That, I am totally cool with and is why I remain a dev to this day. How much longer we can continue to do so is an open question. My perspective is the systemd community's main goal is to make themselves the only option on Linux, and right now, side-projects like eudev are tolerated. I've heard rumblings that eventually, udev will become wholly inseparable from systemd (something tied to eliminating "libudev"), and at that point, it's unknown how the game will change. We'll just have to see.

Raion-Fox wrote: I know paging doesn't work here Kumba. I was just hoping you'd encounter it eventually :p.

I was hoping you may be able to offer insight into Gentoo-hardened vs something like Qubes or comment on the Linux policy of burying broken shit under less-broke shit (ALSA being unable to handle multiple sound outputs due to blocking i/o and thus being assimilated by PolishAudio, hiding potential security issues in userspace behind namespaces and the like)

Gentoo Hardened is a bit of an umbrella project. It's known more for some of the hardened options enabled in the compiler (via a different gcc "specs" file we load). These options include things like stack-smashing (-fstack-protector), position-independent executables (-fPIE), position-independent code (-fPIC), something called "RELRO" (binutils/linker thing, too lazy to look it up right now), fortify-source (gcc), and such. Some of these compiler switches have become standard now (-fPIc is the big one, I think, but it was done for reasons beyond hardening -- but don't quote me on that). Glibc, as of its most recent release (2.25), fully supports -fstack-protector's strongest mode ('all').

There's also a kernel package called "hardened-sources" that includes the latest grsecurity patches for ASLR and PaX support, along with all of grsec's goodies. It tends to lag behind mainline kernel, however, because of a decision made by grsec upstream which sees patches for the latest release delayed by some number of days.

You can go a step further and also enable SELinux support under a Gentoo Hardened build if you really want to lock things down. That's beyond my experience, however, but we were one of the first distributions back in the day to support SELinux hardening.

All that said, I wouldn't try to characterize Gentoo's way of hardening as things like "hiding potential security issues in userspace behind namespaces and the like". Mostly, we're just flipping on additional compiler and linker flags that either are more strict in standards enforcement, or add checks to catch buffer overflows and stack smashing, plus a security-enhanced kernel package and SELinux policy support. Real security comes down to the end user. Running a Gentoo Hardened setup ain't gonna do squat if you allow root on TCP/23 with no password.

ibmfiles wrote: #4 Locked-down interface options so you're stuck with a mismatched contrasted colour scheme (while I can make it workable, it's still stupid). Obviously the the new Microsoft generation forgotten what Brad Wardell told MS to do in XP with at least making the clown interface customizable .

You can thank the generation of these minimalist-obsessed user-interface idiots for this. It's the latest fad in MacOS, KDE Plasma 5, etc. Bevels and color are out, simple lines and soulless black/white is in. If we're all lucky, Motif will be the next fad.

ibmfiles wrote: #7 No disclosure of ALL of the information Microsoft inhales through their "vortex" server and many others

Actually, MS came clean on that, kinda:
https://technet.microsoft.com/itpro/win ... and-fields