hm, is this only about the looks or interaction as well? the most annoying UI i got to know in recent times was/is Windows XP - not because of the looks, but because of how the user is being constantly bugged, reminded, has his focus stolen from him, has files hidden from him, well he's almost being terrorised while using that thing.
recently had to install it on my new machine after a long happy life with win2000 because the latter doesn't perform well with hyperthreading. my, what a different experience...
apples OSX would be a rather close second though - not only are the looks highly overdone to a point where it's no longer enhancing but degrading the whole experience, the thing is also rather restricting, won't adapt well to a user's needs and steals alot of system resources for it's polished cr*p. i'd rather put the mouse aside and use the shell on that one.
third place goes to the standard unix app filebrowser dialog. yes, damn inconvenient, yet a common guest on my screen. go away, thing.
summary: let the user customize your gui, design it with that in mind. different users = different tastes. make it so, that wizards and notifiers and similar annoyances can be turned off permanently.
a pretty gui might attract some, but in the long run, only functionality counts. for those who pretend to work, at least
don't overdo it and save up those precious resources.
good GUI example (in my humble opinion): apple (in fact, nothingreal) shake - flexible, attractive, fast (not on o2, though), compact, not cluttered, open and easily customizeable. does a good joob at vanishing into the background and leaving the focus to the actual work. no spacewaster either.
good but somewhat strange GUI example: nichimen mirai - minimalistic, puts total focus on the work, very generalised - once you have learned it, you won't feel lost in any place in that program. but since it'S so different, it takes a lot of time getting used to.
mediocre GUI example #1: alias maya - open and customizeable but also quite slow because of that, cluttered to the extreme because of all the legacy it carries with it, mix of some different usability concepts - why?
mediocre GUI example #2: 3ds max - very customizeable and reasonably fast. compact screen size. quite cluttered plugin-host: know one area of the program, have no idea what to do in another, a lot of different concepts (read: plugins from different vendors) in that one.
confusing GUI example: pixologic zbrush - awesome program with one of the strangest UI's on earth. fast, refresh-issues, lot's of tiny text and hard-to-grab micro-sliders, uses a lot of it's own non-standard terminology to confuse you even further. has gotten better in recent versions. for a really strange experience check out versions 1.23 or 1.55.