Your description of the machine is a little confusing, but it sounds like the machine is a dual processor MDD, not a "dual core" system.
You'll probably want to make sure that the machine has been upgraded to the latest firmware. If it hasn't already been upgraded, you'll need to boot it from a writable MacOS 9 filesystem in order to run the firmware update.
I don't have any experience running Linux or any of the non-Apple BSDs on this particular hardware, so I'll limit my comments to versions of MacOS.
The first question to ask is whether or not you want to boot MacOS 9 on this machine or if you want to run Classic apps under OS X. If neither, then I'd recommend installing Leopard (MacOS X 10.5.x). Leopard will perform quite nicely on this hardware, and it still receives security updates, etc. from Apple. Aside from a slight performance decrease compared to Tiger (10.4.x), the only significant disadvantage of Leopard on this machine is that Leopard doesn't support Classic.
If you want to run Classic apps or boot into MacOS 9, then install OS 9 first, and then install Tiger on top of it, unless you have software which specifically requires an older version of OS X. Tiger will give a nice balance between being reasonably up to date while having very good performance. Unfortunately, Apple no longer provides updates for Tiger, and some third party software packages are beginning to drop support for Tiger, too.
If this were my machine, I'd probably either create two partitions on the existing hard drive, or I'd add a second internal IDE/ATA hard drive. On one partition/drive, I'd install a bootable OS 9.2.x first, followed by installing Tiger on the same partition/drive. On the other partition/drive, I'd install Leopard. This configuration would let you boot natively into MacOS 9, MacOS 10.4.x, or MacOS 10.5.x.
As far as hardware upgrades go, it's reasonable to add more RAM, particularly if you plan on running Leopard, though it's not critical. If you boot into MacOS 9, it'll only see 1.5 GB of RAM, IIRC. Adding USB 2 ports is a good idea, but keep in mind that they will operate in USB 1.1 mode when booting under OS 9. If you need high speed external storage under MacOS 9, Firewire or SCSI are the only real options. The only thing likely to have a major impact on performance would be getting a new video card if it currently has a low spec card installed. Just be sure to get a card that has native support for Leopard technologies like Core Image. There's a bunch of info about that on the net. Also, some people have had good luck
flashing less expensive Windows versions of cards into Mac versions
. I wouldn't bother with a SATA upgrade.
Good luck -- it sounds like you're getting a very nice machine.