IBM

POWER6 Linux and AIX - Page 2

Sorry, a little late to the party, but I only recently got my hands on a POWER5+ based machine (yes I know, no POWER6 like in the thread title but most of the thread is about POWER5 anyhow :) ) and thought I could add some info about support in newer Debian releases:

  • Debian 8 (ppc userland with ppc64 kernel; tested with on disk installation and debootstrapped NFS root FS) and
  • Debian Sid (9) (ppc64 userland and kernel; tested with debootstrapped NFS root FS)
...both work on my p5-520Q (9131-52A).

The disk installation of Debian 8 works similar to Shiunbird's guide on Youtube with the following changes:

Shiunbird wrote:
astouffer wrote: Just a quick note for anyone else with this hardware, Debian 7 ppc installs fine.


I run Debian on my Power5. It was a pain to get it booting however.
After months trying to sort things out, I found out that the installer was reading the wrong devalias from the OpenFirmware (or the OpenFirmware was sending it wrong to the installer). This at the time of configuring the boot loader. So what was happening instead was that I was installing to the disk with SCSI ID 5 (listed by the installer as my first SCSI disk), but the boot loader configurator would detect my first disk as being SCSI ID 8 and not configure itself correctly.

I only used a single disk in the machine during installation (just like astouffer) to avoid this issue.

Shiunbird wrote: Edit: another thing... I found out that the partitioning layout matters a lot. I had to create the prep partition, and have /boot separated as ext2. Then the rest could be ext4 or whatever.

I was digging out a lot and found at the time some document from IBM recommending to use ext2 for /boot.

As per your guide I first tried to create a partition for /boot with EXT2, but this always failed with strange hangs both in the installer and on the command line. I succeeded in creating an EXT2 formatted partition by using a Debian 7 installer CDROM. IIRC this could be used by the Debian 8 installer later on if you do not try to reformat it, but in the end I read somewhere on the web (sorry, I don't remember the URL) that yaboot should "now" support EXT4, too, so I reinstalled Debian 8 using only a single EXT4 formatted partition for both / and /boot. And this also worked fine and is bootable.

Still important - but no change to Shiunbird's howto video - when creating an initrd use the targeted approach instead of generic , as yaboot seems to have a file size limit for the intrd it loads or it does not reserve enough memory for both kernel and initrd:

Code: Select all

Config file read, 4096 bytes
Welcome to yaboot version 1.3.16
Enter "help" to get some basic usage information
boot:
Linux                      old
boot:
Please wait, loading kernel...
Elf64 kernel loaded...
Loading ramdisk...
ext2: i/o error 2133571364 in read
ramdisk loaded at 01a00000, size: 5120 Kbytes
[...]
[    0.880975] List of all partitions:
[    0.880984] No filesystem could mount root, tried:
[    0.880997] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    0.881014] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.16.0-4-powerpc64 #1 Debian 3.16.39-1+deb8u2
[    0.881027] Call Trace:
[...]
[    0.888495] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

...and the driver for the disk controller is a module, so not compiled into the Linux kernel and hence unavailable for use if it cannot be loaded from an initrd.
:Indy: :O2: :Octane: :Octane2: :O200: = :O200: - :O200: = :O200: (O200 cluster w/2 GIGAchannel cabinets)
[ ( hp ) ] 712/80 c3000 (dead) :hpserv: (J5600) c3700 c3750 c8000 rp2470 :rx2600: (rx2620) rx4640
| d | i | g | i | t | a | l | AXPpci33 AlphaStation 200 AlphaStation 255 PWS 500au AlphaServer DS20E AlphaServer DS25
C O B A L T Qube 2 Qube 3 RaQ RaQ 2 RaQ 4r RaQ XTR
I want to point out three things
1) the IBM firmware pre-initializes the PCI in the proper way, according to what the linux kernel is expcted to find
2) the IBM firmware passes the 'device tree' to the kernel, which is what modern kernels need to find
3) PPC64 is more supported than PPC32, especially for the PCI part

Conclusion: POWER5 and POWER6 machines are the ideal platform for linux.
Head Full of Snow. Lemon Scented You
johnnym wrote: Still important - but no change to Shiunbird's howto video - when creating an initrd use the targeted approach instead of generic , as yaboot seems to have a file size limit for the intrd it loads or it does not reserve enough memory for both kernel and initrd:


Should have mentioned that yaboot gave me problems too. Grub2 works much much better.
Y888099 wrote: I want to point out three things
1) the IBM firmware pre-initializes the PCI in the proper way, according to what the linux kernel is expcted to find
2) the IBM firmware passes the 'device tree' to the kernel, which is what modern kernels need to find
3) PPC64 is more supported than PPC32, especially for the PCI part

Conclusion: POWER5 and POWER6 machines are the ideal platform for linux.


I have no experiences with later Powers, but I can agree POWER5 and POWER6 are very smooth for Linux.

However, on point 1, I'm not entirely sure of this. I have problems with graphic drivers that should in theory work with Linux. After a lot of digging out, it seems that the matrox drivers expect a different memory address and I'm not sure that's related to the way PCI initializes. It runs well with framebuffer.

On Debian 8, I can't even go with the framebuffer. Video goes via OF somehow and even during the installer the screen refreshes are painfully slow. this is probably related to the kernel page size (is that the correct term?) changing from 32K to 8K on Debian 8. This affects Power Macs as well.
I've heard you can recompile the kernel, but I haven't had success. In Power Macs, this still results in no 3D acceleration.

I still have a few uses for a graphical interface on my IntelliStation, so I went back to Debian 7 for the time being and will probably stay there as long as patches are still coming.

However, I have not tried Debian 8 with my PPC-flashed Radeon 7000 PCI. I'm confident it would work. However, AIX doesn't support it and swapping graphics cards to boot to AIX is too much of a hassle. I'd rather have the GXT135p working well.

PS: I'm going off-topic here. But in short, PCI is 99% OK. I've tried USB 2 cards, PCI FireWire 400 cards, extra network card, the Radeon 7000 PCI, and they all worked perfectly fine under Linux.

johnnym wrote: I was digging out a lot and found at the time some document from IBM recommending to use ext2 for /boot.

As per your guide I first tried to create a partition for /boot with EXT2, but this always failed with strange hangs both in the installer and on the command line. I succeeded in creating an EXT2 formatted partition by using a Debian 7 installer CDROM. IIRC this could be used by the Debian 8 installer later on if you do not try to reformat it, but in the end I read somewhere on the web (sorry, I don't remember the URL) that yaboot should "now" support EXT4, too, so I reinstalled Debian 8 using only a single EXT4 formatted partition for both / and /root. And this also worked fine and is bootable.[/quote]

I've found something new last week.
When you are installing Linux, it's not good to have an AIX bootable hard drive installed.

The yaboot installer detects the AIX bootable disk (probably via an OpenFirmware call), and then pops up an error complaining that there's no PReP boot partition detected. Installation goes smoothly if there's no AIX hard drive installed.

I'm considering to get a spare hard drive and update my guide to Debian 8, re-validating all the comments on this thread.
Image Image
Shiunbird wrote: However, on point 1, I'm not entirely sure of this. I have problems with graphic drivers that should in theory work with Linux. After a lot of digging out, it seems that the matrox drivers expect a different memory address and I'm not sure that's related to the way PCI initializes. It runs well with framebuffer.


well, with IBM OF ... at least you won't see this

Code: Select all

PCI: Probing PCI hardware
PCI host bridge to bus 0008:00
pci_bus 0008:00: root bus resource [io  0x2000-0x2fff]
pci_bus 0008:00: root bus resource [mem 0x80000000-0x9fffffff]
pci_bus 0008:00: root bus resource [bus 00-ff]
PCI: Hiding 4xx host bridge resources 0008:00:00.0
pci 0008:00:04.0: [Firmware Bug]: reg 0x10: invalid BAR (can't size)
pci 0008:00:04.0: [Firmware Bug]: reg 0x14: invalid BAR (can't size)
pci 0008:00:04.0: [Firmware Bug]: reg 0x18: invalid BAR (can't size)
pci 0008:00:04.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size)
pci 0008:00:00.0: of_irq_parse_pci: failed with rc=-22
pci 0008:00:03.0: BAR 1: assigned [mem 0x80000000-0x807fffff pref]
pci 0008:00:04.0: BAR 5: assigned [mem 0x80800000-0x8081ffff]
pci 0008:00:03.0: BAR 6: assigned [mem 0x80820000-0x8082ffff pref]
pci 0008:00:03.0: BAR 0: assigned [mem 0x80830000-0x80833fff]
pci 0008:00:04.0: BAR 6: assigned [mem 0x80834000-0x80837fff pref]
pci 0008:00:02.0: BAR 0: assigned [mem 0x80838000-0x80838fff]
pci 0008:00:02.1: BAR 0: assigned [mem 0x80839000-0x808390ff]
pci 0008:00:04.0: BAR 4: assigned [io  0x2000-0x203f], invalid, disabled


For two reasons
  • IBM moved the PCI stuff into PCI64, so if you have a 64bit IBM machine you are already aligned
  • IBM released devicetree configurations for their machines (it's "in-Trunk", a git-term which means you can find them in linux kernel source, somewere), so both the firmware and the kernel shouldn't have problems in probing since things are (should be?) already properly described


In my case .. I am on a 32bit machine, dead since 2.6.25, with a strange DMA engine on where I need to invalidate data cache to enforce coherency. And it uses an old firmware (uboot v1.2 ~ 2008) which neither understand nor provides support for dts blob.

It's unaligned with modern kernels, which expect some job done by the firmware, e.g. uboot doesn't initialize the PCI, and it doesn't understand the dts description encoded into the kernel, so you see Quirks :D

Quirks means ... you have a problem with PCI_IO, there is a conflict with address translation, things are NOT correctly pre-initialized by the firmware (OF is much more advanced, especially in PCI-auto-remapping), and the kernel is NOT able to handle them, resulting a failure.

To solve it, I had to add a boot wrapper between the bootloader and the kernel: it does the properly initialization, it understands the device tree, it passes the information to the kernel, making it happy. So it can bootstrap without Quirks.

Video boards like Ati Radeon, Voodoo3-4-5, use both PCI_IO and PCI_MEM. Before fixing the Quirks problem I was using Matrox M1 (MGA first generation), which is PCI_MEM only, therefore it only requires a windows in the PCI memory

and here it is

Code: Select all

matroxfb: framebuffer at 0x80000000, mapped to 0xc4080000, size 4194304


4Mbyte mapped in the PCI_MEM space without a problem!

But I have a lot of issues with vgaarb, as some "legacy" VGA devices implemented on PCI typically have the same hard-decoded addresses as they did on ISA. When multiple PCI devices are accessed at same time they need some kind of coordination.

I don't need confb (text console, directly offered by kernel), therefore I simply removed this module, enjoying a pure framebuffer on which I can attach (/dev/fb0, in userspace) an homemade directfb-vt200-terminal.

It works with less problems, even if it's a bit slower, but not so bad.

Shiunbird wrote: This affects Power Macs as well.


You could try to reprogram the Matrox's bios into OF-BIOS. I know there are Matrox with PC-bios, and Matrox with OF-bios. Some hardware, different firmware. It might help.

Shiunbird wrote: 3D acceleration.


3D acceleration, as well as 2D acceleration, are X11's matter, and they go more complex. They involve DRI and DRM. Direct Rendering Manager (XFree86 4.1.0) and higher DRI support. They both need kernel support as well as X11 drivers support

From my point of view, they are "userspace modules", /usr/lib/xorg/modules/*, therefore I handle them as "catalyst" problems if any. It's not kernel matter, I mean, once you have provided the proper kernel support.

Also, I don't know if Debian uses "fbdev" or a dedicated framebuffer, e.g. VIDEO="mga, ati, stl1, ...."

This is what I am using in a product I am developing. It's an X11-terminal made around an embedded CPU, PowerPC405GP, attached to a MGA video card. There is also an fpga for other PCI stuff, but it doesn't matter


/etc/X11/xorg.conf.d/43-graphic-card.conf

Code: Select all

Section "Device"
Identifier      "graphic0"
Driver          "MGA"
BusId           "PCI:0:3:0"
MatchSeat       "seat-0"
#----------------------------------------
Option          "NoAccel"    "True"
Option          "DRI"        "false"
Option          "MergedFB"   "False"
Option          "MetaModes"  "1024x768 800x600"
EndSection


# X11 can also use
#
#       Driver         "FBDev"
#
# but it uses a lot of system ram, and it's 20% slower





p.s.
I am not planning to add a video card to my C3600 workstation, it's already happy without it and I don't want to put Xorg stuff on the harddirve (because it costs a lot of time and effort with catalyst. HPPA is ... experimental)

So, I'd prefer to use a dedicated X11-term, which can be a laptop loaded with MobaXterm (you can download a free copy, you can buy a full featured commercial version for 60 euro). The HP box is more happy of being a server, and I am more happy if I can put my effort only on one machine (e.g. the PowerPC405GP) instead of supporting X11 everywhere.

You might consider to do the same: just assure you have a good lan connectivity (100Mbps), and no bottleneck in the network.

Oh, about tektronix and HP envizex xterminals: as far as I experimented, they are all pseudocolor (except NCD900 which costs a leg and an arm, plus an eye of your head), you can used them but only if you can forget gtk, kde, qt. They work good only with motif.
Head Full of Snow. Lemon Scented You
Y888099 wrote: It's unaligned with modern kernels, which expect some job done by the firmware, e.g. uboot doesn't initialize the PCI, and it doesn't understand the dts description encoded into the kernel, so you see Quirks :D

Quirks means ... you have a problem with PCI_IO, there is a conflict with address translation, things are NOT correctly pre-initialized by the firmware (OF is much more advanced, especially in PCI-auto-remapping), and the kernel is NOT able to handle them, resulting a failure.


That should be exactly what is happening. The weird thing is... on Debian 7 I can use fbdev. Matrox or mca in theory should work, but it doesn't because of what you described. Observation: the installer fails to detect proper modes, I need to edit X.conf manually.

On Debian 8, even fbdev fails. The installer tries fbdev, fails, and goes into some other mode (that I think routes through OF somehow, I don't even know how to explain my hypo given that I don't know well enough to understand what goes on), and the refreshes are painfully slow.

Y888099 wrote: You could try to reprogram the Matrox's bios into OF-BIOS. I know there are Matrox with PC-bios, and Matrox with OF-bios. Some hardware, different firmware. It might help.


As far as I know, the IBM GXT135p is a Matrox!
https://www.ibm.com/support/knowledgecenter/beta/8231-E2C/p7hcd/fc2849.htm
http://forums.nekochan.net/viewtopic.php?t=16721690
I remember reading in the past that if you configure via X.org the system to use Matrox, it goes well if plugged to a PCI Power Mac G5 (I have only PCI Express so I can't test), but fails if installed on the POWER5. Sadly, I can't find the reference now.

Y888099 wrote: Also, I don't know if Debian uses "fbdev" or a dedicated framebuffer, e.g. VIDEO="mga, ati, stl1, ...."


I managed to get it working with fbdev. No success with mga or matrox. My setup was not very different from the one you mentioned here:

Y888099 wrote: /etc/X11/xorg.conf.d/43-graphic-card.conf
[code]
Section "Device"
Identifier "graphic0"
Driver "MGA"
BusId "PCI:0:3:0"
MatchSeat "seat-0"
#----------------------------------------
Option "NoAccel" "True"
Option "DRI" "false"
Option "MergedFB" "False"
Option "MetaModes" "1024x768 800x600"
EndSection


I think I will try using X remotely as you suggest and see how it performs. I could even then get a GXT6500 installed for my AIX playing around and not have to bother with graphics card swapping ordeal. Then maybe I will be able to make the move to Debian 8.

@Everybody: Thanks for all the ideas and suggestions. I'll test everything and document how it goes. Once I have a "final" guide for Debian 8, I will publish it here and elsewhere for the benefit of others. Most of the folks with IBM boxes also want to have AIX, so I will see what I can put together for smooth dual booting.
Image Image