The collected works of rooprob

Netbsd on O2: good for you. The partially accelerated X is useful and because of it the browser you get does much better than the old sea monkey we're stuck with in irix. No 3d accel though. I do have a line to the author of the patches but hes busy hacking on powerpc, or growing up. I have tried to explore the poss for unlocking the ICE chip for more accel (there is a reason it's not been done).

Anyway it will run for a bit, then kernel fault. Irix is the only sensible choice.

What do I use my sgi for: nothing. It serves as an icon of mid90s design.
I would like to use it but every project I dream up either exceeds my featureset or budget.

I made it silent running, run at 1600x1000, and a little opengl in glut/C. I wont light it in blue LEDs, install a mini-itx or get rid of it
R12 300 O2, nice case. Not giving up yet!

I am up helping out. My O2 is primed for action.

I suggest we pick up each package in beta, check it against the current source and push it to prod.
Might be an idea to take a look at http://mxcl.github.com/homebrew .

I guess my biggest hurdle is the compiler options for the various Abis, a refresher on what the base level of support necessary and some experience on Linux assumption in some these packages. Some of it is documented on this twiki somewhere. Is my sgi compiler fully functional without a license, or do I have to use gcc? I know historically gcc has been not so good on MIPS. I'm sure many here would never dream of using it.

Found this post while I was just compiling fltk 1.3 and dillo 3

Fltk compiled ok. Same issues with ellipsis and c++. Ran out time to learn why. How's the patching? Got it to compile yet?

Did you run the fltk fluid? It works for a bit but the I managed to choose a menu item that made my O2 clear screen and play the down tune before offering me a chance to restart. No crash dump no log (savecore is on).
Second time I quickly made the fluid windows corrupt in redraw before I quit it.
Doesn't bode well for running fltk app like dillo. O2 is stable with native apps so I don't believe it's hardware issue.

Temperamental indeed.

I find myself with this problem - after several successful records of many minutes of footage, the next and subsequent invocations just will not capture. A reboot fixes the problem.

OK so something less invasive: reset the video system, sudo ml unld mvp driver, as suggested in the man O2Video. We see the slight pause on the next invocation as the kernel modules are reloaded, but we still cannot capture.

I did try recording as root and my system froze immediately. Pah! That was with logging as my user and using sudo. I tried logging in as root instead, and successfully recorded a few times until a total freeze on invocation. I the initiated reader would immediately point to h/w failure, like memory, but I'm not convinced.

Lets go back to just being a normal user and using dmrecord.

So what's going on ? top, par and gr_osview are my current observation points. In normal "working" operation we can see some, but not excessive CPU user and sys percentage, but mostly idle, as we'd expect as the video is being processed by ICE. In borked operation, we see a hugely increased amount of sys, which is kernel sys call activity. There are no other background tasks stealing cpu.

Why would this be? I'm still trying to find out. dmrecord or mediarecorder both fork a child to do the recording, and it's the combined CPU time consumed by the pair that breaches what the system can do, introduces the hiccup and as I'm cancelling the capture on a drop, dropping the session. In my par output of mediarecorder I have gazillions of select and ioctl calls, but my skills in this area make the system a bit opaque. Oddly I cannot get par and dmrecord to give me any data on the child process (FIX you have to run par as root).

You might thing par has something to do with it too, but when we're in a working state, I have enough headroom to have par collect stats.

oo@mapleleaf:~/MovieTest dmrecord -C -v -p video -p audio dmrec0.mv
Options:
Audio channels: 2
Video device: mvp
Input port: 'Camera Video Input' (1)
Compression scheme: jpeg
Compression engine: ice
Quality: 75
Audio rate: 44100hz
Image (field) height: 240
Image width: 640
video timing = VL_TIMING_525_SQ_PIX
DM_IMAGE_PIXEL_ASPECT = 1.000000

Hit the Enter key to begin recording...
Hit <ctrl>-c to stop recording...

Expecting buffer 11 from video, got buffer 23 instead


Warning: Expecting buffer 11 from compressor, got buffer 23 instead

Error capturing image frame 5
Post-processing output file 'dmrec0.mv' ...
Timing information:
5 image frames = 0.17 seconds of video captured.

Compression information:
Average compressed frame size: 20062.4 bytes
Average compression ratio: 30.6 : 1
Bit rate achieved: 4810161.0 bits/sec

Tried a few things: sudo ml unload the mvp and ice modules and retry - no joy. Tried systune mvpearly_frontiemsc = 0 (was 1) no change.
The only thing that reliably works is a full reboot. Then I can get 4 successful captures, then no more.
Actually sometimes it fixes itself and I can get a few more, but it's not reliable.

And then - just like that, the time it took me to write this out, it's working. Argh!

oo@mapleleaf:~/MovieTest sudo par -e ioctl -e select -i -o dmrec.par dmrecord -t 30.0 -C -v -p video -p audio dmrec0.mv
Options:
Audio channels: 2
Video device: mvp
Input port: 'Camera Video Input' (1)
Compression scheme: jpeg
Compression engine: ice
Quality: 75
Audio rate: 44100hz
Image (field) height: 240
Image width: 640
video timing = VL_TIMING_525_SQ_PIX
DM_IMAGE_PIXEL_ASPECT = 1.000000

Hit the Enter key to begin recording...

Recording was done in real time successfully.
Post-processing output file 'dmrec0.mv' ...
Timing information:
899 image frames = 30.00 seconds of video captured.

Compression information:
Average compressed frame size: 19717.6 bytes
Average compression ratio: 31.2 : 1
Bit rate achieved: 4727497.5 bits/sec

and it works...

oo@mapleleaf:~/MovieTest time dmrecord -t 30.0 -C -v -p video -p audio dmrec0.mv
Options:
Audio channels: 2
Video device: mvp
Input port: 'Camera Video Input' (1)
Compression scheme: jpeg
Compression engine: ice
Quality: 75
Audio rate: 44100hz
Image (field) height: 240
Image width: 640
video timing = VL_TIMING_525_SQ_PIX
DM_IMAGE_PIXEL_ASPECT = 1.000000

Hit the Enter key to begin recording...
Recording was done in real time successfully.
Post-processing output file 'dmrec0.mv' ...
Timing information:
899 image frames = 30.00 seconds of video captured.

Compression information:
Average compressed frame size: 17976.1 bytes
Average compression ratio: 34.2 : 1
Bit rate achieved: 4309947.1 bits/sec
0.698u 3.677s 0:32.16 13.5% 0+0k 2+4io 1pf+0w

and it doesn't work... meh It sucks being an SGI aficionado...

oo@mapleleaf:~/MovieTest time dmrecord -t 30.0 -C -v -p video -p audio dmrec0.mv
Options:
Audio channels: 2
Video device: mvp
Input port: 'Camera Video Input' (1)
Compression scheme: jpeg
Compression engine: ice
Quality: 75
Audio rate: 44100hz
Image (field) height: 240
Image width: 640
video timing = VL_TIMING_525_SQ_PIX
DM_IMAGE_PIXEL_ASPECT = 1.000000

Hit the Enter key to begin recording...

Expecting buffer 11 from video, got buffer 13 instead


Expecting buffer 15 from video, got buffer 17 instead


Warning: Expecting buffer 11 from compressor, got buffer 13 instead

Error capturing image frame 5
Post-processing output file 'dmrec0.mv' ...
Timing information:
5 image frames = 0.17 seconds of video captured.

Compression information:
Average compressed frame size: 17068.8 bytes
Average compression ratio: 36.0 : 1
Bit rate achieved: 4092415.5 bits/sec
0.027u 0.272s 0:01.94 14.9% 0+0k 2+5io 1pf+0w

CPU: MIPS R12000 Processor Chip Revision: 2.3
FPU: MIPS R12010 Floating Point Chip Revision: 0.0
1 300 MHZ IP32 Processor
Main memory size: 768 Mbytes
Secondary unified instruction/data cache size: 1 Mbyte on Processor 0
Instruction cache size: 32 Kbytes
Data cache size: 32 Kbytes
FLASH PROM version 4.18
Integral SCSI controller 0: Version ADAPTEC 7880
Disk drive: unit 2 on SCSI controller 0
CDROM: unit 4 on SCSI controller 0
Integral SCSI controller 1: Version ADAPTEC 7880
On-board serial ports: tty1
On-board serial ports: tty2
On-board EPP/ECP parallel port
CRM graphics installed
Integral Ethernet: ec0, version 1
Iris Audio Processor: version A3 revision 3
Video: MVP unit 0 version 1.4
AV: AV1 Card version 1, O2Cam type 1 version 0 connected.
Vice: TRE

oo@mapleleaf:~/MovieTest /sbin/uname -R
6.5 6.5.26m

... and after *five minutes* it works....

Interesting that IRIX has a unload delay of 5 minutes for many of it's modules....
I did say I tried unloading both mvp and vice myself, and it didn't cleanup the problem for me. However leaving the machine idle for 5 minutes and returning to the test, we see a successful recording...

Now how' to script this.... Wait another 5 minutes and see what modules are still loaded.

oo@mapleleaf:~/MovieTest sudo ml list

Loaded Modules:
Id: 6 Character device driver: prefix plp, major 38, unload delay 5 minutes, filename /var/sysgen/boot/ecplp.o
Id: 4020 File system: filename /usr/vice/etc/sgiload/libafs.IP32.nonfs.o
Id: 6000 Symbol Table module: unload delay 5 minutes, filename /unix


Registered Modules:
Id: 2 Character device driver: prefix pciba_, unload delay 5 minutes, filename /var/sysgen/boot/pciba.o
Id: 1013 Streams module: prefix kbd, fmodsw name kbd, unload delay 5 minutes, filename /var/sysgen/boot/kbd.o
Id: 3 Character/Block device driver: prefix smfd, unload delay 5 minutes, filename /var/sysgen/boot/smfd.o
Id: 4 Character device driver: prefix ts_, unload delay 5 minutes, filename /var/sysgen/boot/ts.o
Id: 5 Character device driver: prefix tpsc, unload delay 5 minutes, filename /var/sysgen/boot/tpsc.o
Id: 1014 Streams module: prefix ptrmap, fmodsw name ptrmap, unload delay 5 minutes, filename /var/sysgen/boot/ptrmap.o
Id: 1015 Streams module: prefix wacom, fmodsw name wacom, unload delay 5 minutes, filename /var/sysgen/boot/wacom.o
Id: 1016 Streams module: prefix magellan, fmodsw name magellan, unload delay 5 minutes, filename /var/sysgen/boot/magellan.o
Id: 1017 Streams module: prefix imp, fmodsw name imp, unload delay 5 minutes, filename /var/sysgen/boot/imp.o
Id: 1018 Streams module: prefix hitachi, fmodsw name hitachi, unload delay 5 minutes, filename /var/sysgen/boot/hitachi.o
Id: 1019 Streams module: prefix calcomp, fmodsw name calcomp, unload delay 5 minutes, filename /var/sysgen/boot/calcomp.o
Id: 1020 Streams module: prefix tablet, fmodsw name tablet, unload delay 5 minutes, filename /var/sysgen/boot/tablet.o
Id: 1021 Streams module: prefix sball, fmodsw name sball, unload delay 5 minutes, filename /var/sysgen/boot/sball.o
Id: 1022 Streams module: prefix dialbox, fmodsw name dialbox, unload delay 5 minutes, filename /var/sysgen/boot/dialbox.o
Id: 1023 Streams module: prefix dial, fmodsw name dial, unload delay 5 minutes, filename /var/sysgen/boot/dial.o


Registered and Currently Loaded Modules:
Id: 0 Character device driver: prefix vice, major 17, unload delay 5 minutes, filename /var/sysgen/boot/vice.o
Id: 1 Character device driver: prefix mvp, major 9, unload delay 5 minutes, filename /var/sysgen/boot/mvp.o

This isn't it either. Left system for 30 mins, first two tries failed, the next worked.

This smells more like some kind of software bug in either the mvp kernel module or the dm library interaction in these tools. Time to battle with the dm code directly....

Would be interested in any fellow O2 user with an O2cam, if they wouldn't mind wasting a couple of minutes:

With mediarecorder, select Tasks -> Movie -> Video Production JPEG and try and record more than 6 individual 20second clips one after another without it dropping a frame. I get to the 6th and they it wont record anymore without resorting to waiting for more than 5 minutes or rebooting.

Tell me what OS rev you are, and the CPU type.
Thanks a bunch,

Tried 6.5.30m. Basically the same results at recording. Didn't fix this sys storm.

Removed all the RAM, started reinstalling in pairs. I have two pairs of 128 and two pairs or 64.

Observation. It's ok with 256MB ram, seems to record reliably all the time. Doesn't matter whether it's 2x 128 or 4x64. So discounting the slots or the sticks.

Put in 384 (256 +128) and it's records and fails to record in even amounts. When it fails to record, it often only does twice

Found out about timex -p

Good:
18:08:34 vflt/s dfill/s cache/s pgswp/s pgfil/s pflt/s cpyw/s steal/s rclm/s
18:10:13 8.86 3.01 5.80 0.00 0.01 0.76 0.59 3.18 0.00
18:08:34 physmem kernel user fsctl fsdelwr fsdata freedat empty
18:10:13 98304 13870 12881 863 1691 13969 1117 53913

Bad:
17:54:59 vflt/s dfill/s cache/s pgswp/s pgfil/s pflt/s cpyw/s steal/s rclm/s
17:55:03 182.26 32.90 148.33 0.00 2.83 19.28 15.17 37.02 0.00
17:54:59 physmem kernel user fsctl fsdelwr fsdata freedat empty
17:55:03 98304 12919 12504 356 28 1716 464 70317

More faults (vflts/s) and less pages of free memory that may be reclaimable (freedat), this is the %sys activity that kills it. So what's all this about? what's the relationship between this and the installed ram, and is it more a systune? I know MIPS uses two memory segments for kernel data structures.

man sar(1)
-p Report paging activities:
vflt/s - address translation page faults (valid page not in memory);
dfill/s - address translation fault on demand fill or demand zero
page;
cache/s - address translation fault page reclaimed from page cache;
pgswp/s - address translation fault page reclaimed from swap space;
pgfil/s - address translation fault page reclaimed from filesystem;
pflt/s - (hardware) protection faults -- including illegal access to
page and writes to (software) writable pages;
cpyw/s - protection fault on shared copy-on-write page;
steal/s - protection fault on unshared writable page;
rclm/s - pages reclaimed by paging daemon.

Posting for my own amusement. I'm not expecting anyone to be able to fix this, or there to be any IRIX virtual memory authors haunting this list. Live in hope.

Yeah I'm on 6.5.30.

So I've read this, particularly the section Checking for Excessive Paging and Swapping.
http://techpubs.sgi.com/library/tpl/cgi ... /ch10.html

Particularly


I have developed a little test harness to gather par and sar output and tested as root with different RAM configurations.



I am seeing increased vflt/s - page faults (valid page not in memory)

Tested with 256MB, all good. 171 vflt/s (this is a common average result of a few goes)


Test with 384 MB (results from when it fails to record 10s of video, after several successful attempts)



Very much higher vflt/s, but zero pgswp and pgfil, which are pages retreived from disk.

So what I have is a "non-disk page fault" condition when I *add* more RAM to the box. And it doesn't seem to matter what arrangement of RAM is added (64MB or 128MB sticks) just that it goes wrong above 256MB of RAM.

So something is getting too big for something. Could just be a poorly written dmrecord. It came from 6.3 and the age of 256MB of RAM. People have talked about it being crappy - perhaps it's failing to localize it's resources effectively once RAM gets to the crazy heights of 384 MB and beyond. You can watch RAM deplete the longer you run it, and since it's spooling to disk through ICE it shouldn't be really consuming anything more than a static pool- not actually consuming RAM indefinitely. I'm going to write my own.

Great job on this hamei !
I know I said I'd help but I've been obsessing over my dmrecord/virtual memory problem.
Am learning lots about C and unix memory though.
Really enjoy reading sgi tech pubs documentation.

Much appreciate the TLC the package archive is getting. A platform without TLC is a dead one.

Best

I like bronze

Played around with the systunes, even setting them all the way to 256MB ram settings and nada - cannot stop the vm paging from saturating the capture and preventing from working.

So plan B. Poured over the man pages and lurkers guide while the wife has been doing coursework. I have a prototype in C. Uses the "cross platform" (meaning 6.x) 6.5 dmedia libraries. Architecturally it's using the event model on the vlPath which triggers callbacks - instead of forking/select polling. That much works. It could still go either way as I'm doing something wrong - it's running at something like 4 frames per second. And it doesn't actually save anything yet. Not fully understanding something about dmbuffers and the dmIC. The point of O2 is that only data pointers are passed and devices can all see the same RAM thanks to the UMA design. Perhaps the call overhead in the event model when handling video is too high - the docs says it's really supposed to be for application events. Don't know yet.

Who knows. Anyway, more later in the week.

May as well complete this thread.

I found the reason in the source for dmrecord.dmic (/usr/share/src/dmedia), which it seems to share behaviour of the stock utility /usr/sbin/dmrecord.

capture.c
DC(dmBufferSetPoolDefaults(p,options.outbufs,video.xferbytes,DM_TRUE,DM_TRUE));

The establishment of the buffer for the video -> compressor sets both cacheable and mapped to TRUE. This is actually a cache coherency overhead when passing data between the two pools which seems to drive the system into a problem state. When I set to FALSE, I observe consistent realtime compression without any issues over and over.

That was a bit of a rat hole. Learned about the dmedia libraries, written three different versions of dmrecord (forking and event driven), a par parser, some graphviz transformations of the call trace data. Pretty pictures too

Educational, but it was finding jrecvid.c on ftp.sgi.com (amazingly still functional) and on a hunch that led me to the apparent fix. Learning is fun.

I'm transitioning jobs at the moment which is why I have a stupid amount of free time right now to expend on this puzzle. (I'm not allowed to touch much at my current job, just hang around to answer questions until the end of the month). I may consider learning motif to make a little UI for my doings.

Hi,

I'm experimenting with ARS-2160H SCSI SCA to SATA interface in my O2.

I can *sometimes* get the PROM hinv to see the SCSI device, however IRIX hinv doesn't subsequently see it when it boots, and theres no entry for it in /dev.
On reboot most of the time the prom ceases to see the SCSI device again until I've pulled it and changed some jumpers on the ARS.

ARS-2160H
Has ID jumpers on the front for SCSI IDs 0 through 15, delayed init, SE to LVD, write protect.

Tried (with no real difference)
Set to SCSI ID 1, 3, 15.
Explicitly closed "delayed init", (not sure exactly what that does)

Doesn't matter which SCSI slot I put the disk into either or if the boot disk is present.

(already on the bus: internal disk (2), internal dvd rom (4)

Currently the machine boots just fine with my usual 10K SCSI disk.

I've also noticed (but not fully recorded) there is a sequence to when it will detect it and when it wont.
When I plug the power cable in it immediately starts (without my presseing the power button on the front). In this case I know it wont detect it, and sure enough it's not listed in PROM hinv.
When I plug the power in and it remains off, when it boots prom sometimes sees it.
Seems to depend on when it type "OFF" at the prom or just pull the power out.

Thanks for the infos. Diegel was right. It needed the SE jumper closed.

Two hours, job done. Cloned 70GB SCSI 10K disk to 120GB SSD, rebooted and all is well.
£50 for the SSD, and (cough) £more for the ARS.

Incidentally, for those feeling daring, you _can_ get two disks in an R10K O2 without breaking anything. It's a little precarious but the SSD on a naked ARS card (sans chassis) will hang in the otherwise obscured SCA socket. You can remove the PCI cage so long as you put the PCI card back in and leave enough space. I didn't have any other means to clone the disk.

That's fair concern about TRIM and getting a definitive story around support, especially when masquerading as a SCSI disk.

I picked the old line of OCZ Vertex2 128GB as a) SATA 2 IO rate still exceeds the SCSI2 bus, b) end of line is cheap and c) came out when no OS natively supported TRIM. Although I'm no expert, the controller onboard SandForce SF-1200 takes care of marking deleted blocks via garbage collection routine obviating the need for TRIM (apparently).

The ARS manual had screenshots of a Windows device tree clearly displaying an ARS, yet on my test rig the SCSI disk was reported OCZ Vertex clearly from the drive itself. With the right knowhow it may be possible to probe more information from the drive itself.

My O2 with a real SCSI disk clearly makes write ops every few seconds. I have never figured out which service this was. With an SSD only time will tell if it eventually kills it just by being on.

Thinking about it I may buy a second one given built-in garbage collection is something they'll eventually remove as a "cost saving" as more operating systems take care of it natively, in a kind of race-to-the-bottom to winmodems. Indeed newer Samsung 830 I added to my Macbook benefits from TRIM support (apparently) which isn't available in Snow Leopard (whoops!).

This is interesting. The O2 has a Ultra Wide SCSI 40MB/sec bus, so should I expect to see read performance at that rate?

diskperf seems to max out at reads 26MB/sec

Interesting info's, thanks.

My O2 is R12K 400, so as good as it gets. During diskperf osview was telling me 50% idle, with the rest in %sys time and some %iowait. So it's not all CPU, but it is a lot of kernel time which all makes sense.

I can do some experiments with NetBSD to get another baseline. Or I could do some work.

Anecdotally when I performed the xfsdump/xfsrestore from the 10K SCSI disk to the SSD it was all %sys time to almost 100% for the whole 2 hours or so. gr_osview was a solid red bar Back of an envelop tells me that was about 11-15MB/second, which seems a little low for two disks.

Ah well. With the filesystem IO cache it tells me I get 50MB/sec and it's all totally fine for what I use it for. If I squeeze any more out of it I'll let you know. I seem to remember from way back that fx lets you set lowlevel caching options on the disk. I wont disable fsync.


Nice guys at this outfit Active Network Systems Ltd | Tel: +44 (0)1480 437997 | Fax: +44 (0)1480 436031 | [email protected] | www.ans-ltd.co.uk

Pricey at £150 inc VAT and delivery for one but they did say they can get a price on a bulk order. If we can get the price down to £100 with others here I'll pitch in for one more.

Just an idea but is there a jumper on the disk that lets you pick SE over LVD ?
O2 is an SE system.