SGI: Development

From the dutch Tweakers newssite i've read an article about a HPC speed challenge from NASA to attempt to modernize their CFD FORTRAN code, up to a factor of 10000 (!)

The story is here and the page with details of the challenge is here

Apparently the PLEIADES cluster they want the code to run on has been built by SGI (in its later incarnations obviously).

Challenge is split into an Ideation part and an Architecture part, the former being the generation of ideas about a faster approach to the flow analysis, the latter doing algorithm improvements in individual modules.
The compiler environment is Intel/GNU C++/C/FORTRAN and SGI MPT.

The whole trick lies in integrating decade old FORTRAN CFD code onto a modern hybrid CPU/GPU platform with Infiniband interconnect hardware. This cannot be done easily in a pure code-algorithmic way since the speed will depend on the PBS queue system, placement of data in memory or on disks using the LUSTRE file system.

(Maybe not so) Remarkable is that NASA apparently either does not have the resources to do it themselves or that they think this project is too complex to being handled by a single ICT software company, the reseller (SGI, now HPE), or themselves. And there is an import restriction on the FUN3D software, limiting the code challenge to US citizens.

Well, it does look like the total prize money of USD 55.000 is a damn cheap crowdfundingly steal considering what NASA could get in return

On the subject of old SGIs I've been trying to learn UNICOS/mp so the few people who still use it could hire me. Of course all I have is documentation I can find, but UNICOS/mp is just IRIX with mods to run on a Cray X1 or X1e

dexter1 wrote: prize money of USD 55.000 is a damn cheap crowdfundingly steal

the us government had similar "challenges" not so long ago and the prize money there was not much more. compared to their budget it's a joke

A buddy of mine works at Ames and is a sysadmin for Pleiades. Right now all their developers are doing other things. If I knew Fortran, this would be alot of fun!

Sorry, is that a typo? $55 DOT 000 ? I have my long range glasses on and can't see the screen!

When I worked for large organizations, I spent a lot of time trying to get vendors to understand that having a "big" budget was not the same as having "available" budget for what they were selling, especially at the prices they assumed I could pay, even if I actually wanted to buy it. Once you've allocated a few million for one thing, it can be almost impossible to find 55 thousand dollars to spend on something else. No matter how big of a budget I ever had, there always were important things I wanted to do that I simply couldn't do because other important things were using up everything I had.

foetz wrote:

dexter1 wrote: prize money of USD 55.000 is a damn cheap crowdfundingly steal

the us government had similar "challenges" not so long ago and the prize money there was not much more. compared to their budget it's a joke

uunix wrote: Sorry, is that a typo? $55 DOT 000 ? I have my long range glasses on and can't see the screen!

Fifty-five thousand dollars in total : We use dots for thousand delimiter marks, the US folks use commas IIRC.

Foetz is right, on a 17 billion USD budget those 55000 USD hardly make a dent in their expenses. You could possibly recruit a programmer fulltime for 6-8 months doing the job.

praetor242 wrote: A buddy of mine works at Ames and is a sysadmin for Pleiades. Right now all their developers are doing other things. If I knew Fortran, this would be alot of fun!

It is but interfacing with C and C++ libraries (probably only GPU, but maybe Lustre/Infiniband) from a FORTRAN program means you have to use Swig or write wrappers which does argument type conversion for each function you use.
Most default types are easy, but complex and zomplex and string types are a different matter: they need careful testing.

Furthermore there is this matter of possible different name mangling between Intel and GNU compilers, which necessitates multiple compiler specific interfaces: Although that can be tested and adjusted if necessary.

Regarding Josehill's comment which crossed my post:

It is true that allocating project-millions for HPC machinery is actually easier than recruiting support and programmers on a project after that initial big budget spend.
For years I am trying to get faculty heads and ICT bosses around the fact that they should change the way projects are budgeted: Not prioritizing acquisition funds of hardware and software from whoever gives you the best deal, but also allocate money for developers who can teach researchers new skills in writing code and new ways of performing computation.

But man, that is frustratingly hard

I'm actually impressed that NASA was able to cut through the red tape to try and see if anyone would try this. It's not unusual for a hungry vendor or someone with more skills than credentials might try to solve a problem like this for little or no compensation in the hope that it will lead to a larger, better funded project.

For example, I gave a public talk once where I described a problem I was having, and a year later, I got a call from a software developer who was in the audience and decided to try to solve the problem. He said, "I have a prototype. I'll give it to you for free along with future upgrades if you give me feedback so I can refine it into a product and sell it to the industry." He had, in fact, solved my problem with the prototype, so I found a way to scrape together around a few thousand dollars to cover his immediate expenses, and he ended up doing well delivering the product to others, in part by saying that "Josehill at GlobalMegaBigPharmaCo is an early adopter and loves it!" Win-win for everyone.

dexter1 wrote: It is true that allocating project-millions for HPC machinery is actually easier than recruiting support and programmers on a project after that initial big budget spend.
For years I am trying to get faculty heads and ICT bosses around the fact that they should change the way projects are budgeted: Not prioritizing acquisition funds of hardware and software from whoever gives you the best deal, but also allocate money for developers who can teach researchers new skills in writing code and new ways of performing computation.

But man, that is frustratingly hard

Indeed. Proper planning for the full cost of running a project effectively is surprisingly rare and difficult in the wild. Expertise, training, and long term support somehow seem to be assumed to be freely available, yet they usually are the things in shortest supply!