ShadeOfBlue wrote:
I think we're mixing up the current situation
on UNIX systems
and
elsewhere
.
Ja, the obvious conclusion is that Unix is not very well designed. It's not good for desktops and it's not good for mainframes.
It's just not very good
Quote:
What I said before was that
on a UNIX system
, if a CPU-bound program knows how many CPUs the system has, it can run better than if it just spawns hundreds of threads willy-nilly.
Ah ! Maybe there is our misunderstnanding. I would never say "spawn a thousand threads willy-nilly." My point is that in any activity there will be an optimum number of threads. For an exaggerated example, take pbzip. If you have 128 processors and a 256k zipfile, spawning 128 threads to unzip that file would be ridiculous. There is going to be an optimum number of threads
for the task at hand
. In general, the task at hand is not going to be determined by how many processors you have !
Figuring out the best way to structure a program is not so easy eaither, but I'm pretty certain that the count-the-processors approach is less than optimum.
Quote:
The responsiveness issues you describe can be fixed by running the CPU-intensive stuff at a lower priority (higher
nice(1)
value). If you run the rendering job at a nice value of 20, then every UI app will get priority over the rendering job and there shouldn't be any sluggishness. You could also try boosting the priority of UI apps (lowering their "nice" value with
renice(1)
).
This is a pretty crappy solution. Adjusting priority by which application has the focus is way better. Fixed priorities themselves are also fairly shitty. The OS/2 scheduler has 32 priority levels and they are dynamic.
This gives a responsive workstation.
People are so funny. There's a contingent of people in the OS/2 camp who are trying (or have tried) to create a layer that will allow OS/2 apps to run on top of Linux. The concept makes me laugh. That's exactly backwards. Do it that way and you have a reverse-rotation shaft-drive Yamaha pretending to be a Harley
Quote:
However, you must set the nice value manually, since the kernel has no clue which program does what.
Pretty crappy system.
Quote:
And this actually works, people use it this way.
People once built the pyramids using log rollers and thousands of slaves, too
Quote:
If you're compiling something in the background and you don't use "nice" to start those jobs at a much lower priority, they will run at the same priority as nedit and firefox and the rest of the stuff you've started. As far as the kernel is concerned, these tasks are equally important, so it won't favour nedit and firefox.
Yup. Crappy system.
Quote:
Now if a renderer starts 2160 threads (one for each line in a typical 4K image), it will burden the system needlessly, especially if you run it at the same priority as your GUI apps.
And I would never suggest that a program should do that. It makes no sense. This is the kind of thing a gnu "parallellizing libray" would do, choose a random number of threads based on what color the case of the computer is, rather than looking at
what is most efficient for the task.
Let the application decide what tasks need what threads (this does not mean a program should say "hoo cool, there's 64 processors so i'm going to grab all of them !" - a thread should have a distinct job to do, suitable for one task) then let the damned scheduler figure out what's the best way to apportion resources in a dynamic world. And if the user is writing a report, then guess what ? getting that report done with no obstacles is a hell of a lot more important than losing a few milliseconds to context switches. Even a few seconds. Heck, if I have to go back and redo an edit I just made because the stupid computer hung up my interface for 1/3 second and I clicked three times so now have a butchered text, I'll sacrifice two minutes to context switches !
Foregound
tasks are what count !
You're looking at this too much from a programmer's view. Look at it as a computer
user
for a change. We really don't give a shit if it takes ten seconds longer for the background task to finish. What is important to us is what
we
are doing.
NOW !
Quote:
There is much less overhead if the program spawns only as many threads as there are CPUs and then distributes work between them.
I do not think this is a good approach.
Quote:
Every scientific app using OpenMP works like this. The OpenMP library checks the number of cores available and then spawns the appropriate number of worker threads.
And OpenMP sucks dead donkey balls for workstation loads. It's just not optimized for what a workstation should be doing. For some dedicated render machine or a box simulating nuclear explosions, fine. But
not for a workstation.
It's got the wrong priorities.
Quote:
Origins are not desktop machines.
Sorry. I should have mentioned that this is a rackmount Tezro
Quote:
Having one CPU loaded more that the others is not an indication of a crappy scheduler.
On a 2p O350, I have to disagree. If the point of having more than one processor is to actually use them, then having one do 70% of the work while the other sits idle
and
my desktop goes unresponsive, is just bad design.
If you'll remember, Unix resisted the very idea of threads for years ! Both OS/2 and even Windows were threaded while Unix was standing proud with processes. A few people almost had to shove pthreads down Unix' throat. And Linus himself said that "Linux will never run on more than one processor." (That's when I lost interest in Linux. Talk about short-sighted ... Torvalds, not me)
Unix has never really figured it 0ut. They are latecomers to the party and they don't drink or have fun.
Of course now that 1984 has come and gone everyone revises history to suit the marketing needs, but this is the truth. Unix has got some honkin' big structural weaknesses.
Quote:
If you have lots of short-lived tasks, it makes no sense to spread them out to other CPUs and thrash their caches.
It makes a
lot
of sense if it means that my editing or browsing or whatever does not freeze at awkward moments. I'm not being obstinate for fun (at least not this time
) ... use OS/2 for a while. OS/2 has plenty of problems but it does
not
do this shit. Interface hangups happen way too often on this 2p machine.
Waaaay
too often. Once a month I could tolerate. Once a day is ridiculous.
Quote:
IRIX will actually distribute threads to other CPUs once it figures out they will run for more than a few seconds (or if the main CPU is very busy).
Nice. It should distribute them to other processors
now
, not in fifteen minutes.
Quote:
Also, if you re-run a single program many times, IRIX will try to run it on the same CPU, because that CPU already has the data that program needs in its cache, which is why it would be inefficient to run it on a different CPU.
I don't care if it's inefficient, I want it to do that because I goddamned well want that application running NOW because that's what I am doing NOW. I didn't pay thousands of dollars for a computer to tell me, "Oh no, you'll have to wait a second because it's not efficient for me to run your program on that other processor."
Dear computer, stfu and do what I say, thank you very much.
Quote:
If the kernel didn't care about cache locality and just tried to keep every CPU busy, it would be similar to one of those parts boxes with multiple drawers and having one of each screw type in every single drawer, because it's a waste to keep drawers empty. Finding a specific screw would be very slow
Actually, in real life this is not the case. Grab an old dually something and put OS/2 or BeOS on it.
(BeOS is, unfrotunately, not a good test. All you can do is spin the teapot while typing in a terminal. If you had Framemaker to work in while it was doing something else, then you could get pissed if the system hung you up. But since you can't do anything in BeOS, it's hard to get pissed off at your work getting screwed up by the computer. Or not get pissed off because the scheduler actually works, which would be my point ... Haiku is very pretty tho. If there were any decent gnu applications that you could compile it might be a nice system.)
Quote:
What you want is something like BeOS, a single-user desktop OS.
Agreed. Been there, done that, it works well. Alas, none of the programs I needed. OS/2 is also superior to Unix in this respect but same situation, no CAD programs
Neither of them have much OpenGL support either. Okay, BeOS does but no applications that use it. For general-purpose computing there's no way you'd catch me running Irix. And Linux is worse. I haven't tried other Unixes enough to render a judgement (but I'd use a paper and pencil before I'd run Solaris on a desktop. Nice from a command line but holyshit awful as a desktop.)
And ja 2, a single-user workstation is looking more and more attractive. What the hell do I really want all these users and groups for ? For most people it's pointless additional complexity. How many users do you really run your desktop as ?
Sometimes, just because everybody does it this way, doesn't mean it's a good way.
Quote:
IRIX was made to run on everything from workstations with a single CPU and one user to servers with 1024 CPUs and thousands of users;
Disagree. Remember the Personal Iris ? Indigo ? Indy ? Even the Crimson and Onyxes were "desksides", meant for one or two users, not a cast of thousands. They got into the hpc server schtick later on.
The underlying problem here is that Unix does not make a good workstation
and
it's not good on a mainframe, either. It's primitive. It's not well thought-out. In fact, it's not very good.
It's just better than Windows.
IBM uses Linux -- they virtualize thousands of Linux sessions on a mainframe like a herd of DOS boxes, kind of funny
Quote:
I'm not saying it's perfect, it could use improvements, but it's not a complete disaster
Ja, unless we build our own computers from scratch, all we can do is choose between evils
Irix so far is the best compromise for me so I'll stick with it ... but it sure ain't the Immaculate Collection.
The fact is, in many ways Yewnix is one huge kluge .... so we get new toolkits by the dozens while the underlying structural problems are wallpapered over. Post-modern society, whoopee
Quote:
I hope it's clearer now why checking the number of available CPUs is a perfectly normal thing for a CPU-bound program to do on a UNIX system
It was always clear - it's just that I disagree that that's a smart thing to do
) That and the ubiquitous MS office...