Everything Else

Inventory/Catalog

I was wondering if anyone has a system, or is considering some kind of catalog or inventory for their hardware (or not so vintage, or other hobbies, or ...).

I have been looking to have spare parts or a second machine for most of my collection. Many of the spares are getting packed up and stored in a storage unit, as are systems that aren't being used often. For insurance purposes having an inventory would be good, but also just logistically. If I know what Origin 350/Tezro parts are in boxes in storage I don't have to go to the unit, dig out the box, etc.

I have had my eyes out for the follett library software that ran in DOS on the library computers in my high school, but I haven't found it. I'm beginning to consider building something - or just use one of the many home inventory programs? I don't know yet.

Anyone doing anything awesome?
:O3000: :Fuel: :Tezro: :Tezro: :Octane2: :Octane: :Indigo: :Indigo: :Indigo: :Indigo: :O2: :1600SW: :O2: :1600SW: :1600SW: :Indigo2: :Indigo2: :Indigo2: :Indigo2: :Indigo2IMP: :Indy: :Indy: :Indy: :Indy: :O3x0: :O3x02L: :O3x02L:
I have an Excel sheet of parts, with part#, name, status (untested/good/broken), test date, purpose (spare/sell/junk). I use it to print stickers to put on the ESD bags with the parts, to decide whether to buy something that shows up on eBay (do I have that already?), or decide whether I can sell something.

It happened too often that I looked at a box and thought (1) what's in there and (2) does it even work.

Achilles heel: you have to keep it up to date.
:PI: :Indigo: :Indigo: :Indy: :Indy: :Indy: :Indigo2: :Indigo2: :Indigo2IMP: :Octane: :Octane2: :O2: :O2+: Image :Fuel: :Tezro: :4D70G: :Skywriter: :PWRSeries: :Crimson: :ChallengeL: :Onyx: :O200: :Onyx2: :O3x02L:
To accentuate the special identity of the IRIS 4D/70, Silicon Graphics' designers selected a new color palette. The machine's coating blends dark grey, raspberry and beige colors into a pleasing harmony. ( IRIS 4D/70 Superworkstation Technical Report )
japes wrote: I was wondering if anyone has a system, or is considering some kind of catalog or inventory for their hardware (or not so vintage, or other hobbies, or ...).

I had some thoughts about this - also because my inventory is slowly getting out of control and I should start with something similar to stay in control (most things are still in my head, but I start to forget specific inventory items). And I also recogized those useful stickers jan-jaap had applied to the sse/se combination package I purchased from him late last year. What a cool idea is that!

I develop a collection of scripts to remote control machines and other gear (like PDUs, lights, etc.) and there I already organize inventory items (directories) in a directory structure. Each item can contain further items or so-called controls (on, off, reset, nmi/xir, etc.). To organize things in different machine halls (or locations) or for different purposes you can group items in super-items like machine-room-1, machine-room-2 or dns-servers, gluster-peers, etc. by simply adding symlinks:

Code: Select all

~/inventory$ tree
.
├── dns-servers
│   ├── machine-3 -> ../machine-room-1/machine-3
│   └── machine-7 -> ../machine-room-2/machine-7
├── gluster-peers
│   ├── machine-1 -> ../machine-room-1/machine-1
│   ├── machine-2 -> ../machine-room-1/machine-2
│   ├── machine-5 -> ../machine-room-2/machine-5
│   └── machine-6 -> ../machine-room-2/machine-6
├── machine-room-1
│   ├── machine-1
│   │   ├── config
│   │   └── controls
│   ├── machine-2
│   │   ├── config
│   │   └── controls
│   ├── machine-3
│   │   ├── config
│   │   └── controls
│   └── machine-4
│       ├── config
│       └── controls
└── machine-room-2
├── machine-5
│   ├── config
│   └── controls
├── machine-6
│   ├── config
│   └── controls
└── machine-7
├── config
└── controls


Hence I'm biased here a little bit, but still, a box full of spare parts could also be described as a directory containing other items. You can still group boxes inside of other boxes or lockers and even in different locations (cellar, attic, parent's home, etc.). Moving things around can be done with mv . Finding things could be done with find or grep depending on how you implement leaf items.

Code: Select all

~/inventory$ tree
.
├── sgi-graphics-cards
│   ├── mxe-1 -> ../cellar/box-2/mxe-1
│   ├── mxe-2 -> ../cellar/box-2/mxe-2
│   └── sse -> ../attic/box-3/sse
├── ram-modules
│   ├── box-1 -> ../cellar/box-1
│   ├── [...]
│   ├── [...]
│   └── [...]
├── cellar
│   ├── box-1
│   │   ├── pc2-5300-ecc-reg
│   │   └── [...]
│   ├── box-2
│   │   ├── mxe-1
│   │   └── mxe-2
│   ├── [...]
│   │   ├── [...]
│   │   └── [...]
│   └── [...]
│       ├── [...]
│       └── [...]
└── attic
├── box-3
│   ├── sse
│   └── o2-system-board
├── box-4
│   ├── single-r12k-400-o2
│   └── dual-r12k-400-octane2
└── [...]
├── [...]
└── [...]


And now assume you could use fsn for managing it? :D

So why not use a directory structure like an inventory?

A little rough and primitive and additional tools (for e.g. generating unique item numbers, creating summary lists) would still be helpful. E.g. in the future you could also make use of barcode scanners and preconfigured stickers with unique item numbers. So whenever you put something in a box, you scan the box, then choose the add function (implementation needed) and then scan the new item's sticker to add it. Similar actions could be done on removal or movement of a specifc item. And one could also think of barcodes for the state of the item (dead, untested, working), i.e. if an item is tested OK, select the state function for the item and scan the "working" barcode. You might even be able to start specific actions by scanning a barcode for add, remove, state, etc.. The company where I made my apprenticeship long ago had a system similar than that (not only for inventory but also for managing production, final testing, service, etc.) running on (Open)VMS. I think they started with VAXen and later moved to Alpha, not sure if its still in use today, maybe on Itanium.

jan-jaap wrote: Achilles heel: you have to keep it up to date.

Same here, but I think you always need to keep an inventory up to date, regardless what tool you use, except maybe where you have an automatic warehouse which takes care of that and you simply insert or (re)move items (from) there. My father once had a small automatic shelf store (with only a few shelves though) where you could select each shelf by a button press and the machine rotated until the desired shelf was available for access. Not sure if this could be also controlled remotely. He used it for tools but sadly gave it away some time ago... :(
:Indy: :O2: :Octane: :Octane2: :O200: = :O200: - :O200: = :O200: (O200 cluster w/2 GIGAchannel cabinets)
[ ( hp ) ] 712/80 c3000 (dead) :hpserv: (J5600) c3700 c3750 c8000 rp2470 :rx2600: (rx2620) rx4640
| d | i | g | i | t | a | l | AXPpci33 AlphaStation 200 AlphaStation 255 PWS 500au AlphaServer DS20E AlphaServer DS25
C O B A L T Qube 2 Qube 3 RaQ RaQ 2 RaQ 4r RaQ XTR
johnnym wrote:
jan-jaap wrote: Achilles heel: you have to keep it up to date.

Same here, but I think you always need to keep an inventory up to date, regardless what tool you use


By far, the most important point. An inventory that is kept up to date in a paper notebook is infinitely more valuable than a sophisticated, semi-automated, barcode/RFID-enabled system that is not kept up to date.
I may or may not be working on a worldwide SGI part inventory, collection showcase, and offer making site, and there may or may not be an almost complete version running on Apache on my Mini Indigo ;) .
:Onyx: :O2000: :Fuel: :Octane: :Octane: :Octane: :O2: :O2: :Indigo2: :Indigo2: :Indy: :Indy:
and a small army of Image
johnnym wrote: So why not use a directory structure like an inventory?

This focuses very much on *where* a part is. As if things stay in the boxes they arrive in, in chronological order. I try to keep related parts together, i.e. I have a place for Octane/Onyx2 related parts, ditto for Challenge/Onyx1, 4D series etc etc. Only generic parts (SCSI disks, cables, books, keyboard & mice) stay together rather than with the "system" category. The purpose of my inventory is not to know where things are. This should be obvious.

Like many of us (I suspect) I have amassed a lot of parts over the years. Many of those came in large batches. Many of them are not relevant to the systems I own, some I have (too) many of, often their working condition is unknown. This takes up a lot of space while not bringing me anything. The moment comes when you either have to give/throw away everything (I'm not a hoarder) or you have to sanitize the lot. I chose the latter option. This means you have to test everything, and this takes up an unbelievable amount of time. It's like ripping a large CD collection: you want to do it only once, so you want to do it good the first time. So I threw away everything not worth that time (nobody wants a 1x180MHz CPU board for an Onyx2). If you test something you have to (1) then package the item such that it will not get damaged after you test it 'good', and (2) keep a record of it, and that's where the Excel sheet was born. It's trivial to create a 'mail merge' and print stickers from it.

The outcome is boxes full of neatly packaged, quality spares. And boxes full of electro-trash, and boxes full of neatly packaged, but useless (to me) stuff. The useless stuff has to go (I'm still not a hoarder). Some of it ends in the 'for sale/trade' forum here, some of it goes to local market places. If nobody wants it it goes to the recycling. In the end, I only want the boxes of neatly packaged, tested, labeled parts for my personal use. It will take time and dedication to get there. Maybe I never will.

The second purpose of my inventory: to help make a decision on whether I need to buy something if it pops up. I do not want to buy something I have (as a spare) already. And I will never remember what I already have.

It's really a matter of keeping it simple (the KISS principle). It has to be easy to edit or I won't bother and that would make it worthless. The little Excel sheet gets the job done, if I need a complicated system it's probably a sign of a bigger problem (like way to much crap I should really get rid of instead of cataloging it).
:PI: :Indigo: :Indigo: :Indy: :Indy: :Indy: :Indigo2: :Indigo2: :Indigo2IMP: :Octane: :Octane2: :O2: :O2+: Image :Fuel: :Tezro: :4D70G: :Skywriter: :PWRSeries: :Crimson: :ChallengeL: :Onyx: :O200: :Onyx2: :O3x02L:
To accentuate the special identity of the IRIS 4D/70, Silicon Graphics' designers selected a new color palette. The machine's coating blends dark grey, raspberry and beige colors into a pleasing harmony. ( IRIS 4D/70 Superworkstation Technical Report )
jan-jaap wrote:
johnnym wrote: So why not use a directory structure like an inventory?

This focuses very much on *where* a part is. As if things stay in the boxes they arrive in, in chronological order.

Not necessarily: you can move around things by simply mv ing the contents between boxes/directories (plus doing that in reality of course). I currently don't have an argument, where a tree like structure could be helpful in managing an inventory, but find using such quite manifest or natural.

jan-jaap wrote: I try to keep related parts together

Me too actually, as long as related parts fit in the same boxes or the available room close by - which is a problem in my computer room, I'm afraid.

jan-jaap wrote: It's really a matter of keeping it simple (the KISS principle). It has to be easy to edit or I won't bother and that would make it worthless. The little Excel sheet gets the job done, if I need a complicated system it's probably a sign of a bigger problem (like way to much crap I should really get rid of instead of cataloging it).

Using a table is indeed nice and simple, but the need for Excel or something heavy like Open/LibreOffice keeps me at distance. But I'm sure the flattened view of an inventory is advantageous for some operations (e.g. printing).
:Indy: :O2: :Octane: :Octane2: :O200: = :O200: - :O200: = :O200: (O200 cluster w/2 GIGAchannel cabinets)
[ ( hp ) ] 712/80 c3000 (dead) :hpserv: (J5600) c3700 c3750 c8000 rp2470 :rx2600: (rx2620) rx4640
| d | i | g | i | t | a | l | AXPpci33 AlphaStation 200 AlphaStation 255 PWS 500au AlphaServer DS20E AlphaServer DS25
C O B A L T Qube 2 Qube 3 RaQ RaQ 2 RaQ 4r RaQ XTR
OMG the directory approach is brilliant, and I'm ashamed to think that I've never thought of it before, and has many advantages compared to Excel.

Imagine the following two files

/computers/intellistation/dev/1gb-samsumg-ram-serial_number
and
/computers/intellistation/dev/scsi/20gb-ibm-scsi-hd-serial_number

These two devices have different specifications, so you can track, for example, for the RAM:

Code: Select all

manufacturer=samsung
capacity=4GB
number_of_chips=8
chip_capacity=512MB
device_type=hard-drive


And for the HD:

Code: Select all

manufacturer=ibm
capacity=20GB
rotational_speed=10000rpm
bus_type=parallel-scsi
bus_speed=320
device_type=ram-module


In Excel, it would be very hard to track the specifications of such different devices, unless you would restrict yourself to tracking only things like manufacturer and device type.

Now, if you move that RAM module to a drawer, you just send the file to:
/drawer2/20gb-ibm-scsi-hd-serial_number
and if later you install that hard drive, you can just move the file to /computers/fuel/scsi/

If you want to sell your computer later, with all that's inside, you can just print recursively the contents of the folders and you are done. =)

I love it!
Image Image
The File-system directory concept approach could be changed to using an LDAP server (Lightweight Directory Access Protocol)... OpenLDAP runs on lots of platforms (and it has portable versions to run without an actual installation on windows, linux,etc.. ) and there is Apache Directory Studio for a good GUI/front-end.. as well as JXplorer.. java-based, that runs on all sorts of platforms (even IRIX with a version supporting Java 1.4)
Structure would be similar:
ou=computers, cn=intellistation, cn= dev, cn=scsi, cn=20gb-ibm-scsi-hd-serial_number
Then attributes for cn=40gb-ibm-scsi-hd-serial_number:
manufacturer=samsung
capacity=4GB
number_of_chips=8
chip_capacity=512MB
device_type=hard-drive
Cut/paste/copy from one to another cn or another.
Search/filter, etc..

Then you deal with an LDIF file for import/export.. backups and sharing
It may be more intuitive for inventory purposes than an Excel spreadsheet or a filesystem directory layout.

You could also do something with XML too...
LDAP and/or XML most likely have some PHP/web-based browsing software out there too that could work on IRIX and other platforms. Speaking of PHP.. phpmyinventory would work with PHP and MySQL even on IRIX (at least it seems like it should)

Just a fancy idea ;)
Shiunbird wrote: In Excel, it would be very hard to track the specifications of such different devices, unless you would restrict yourself to tracking only things like manufacturer and device type.

I do not care about specifications. My inventory exists primarily to help me make decisions about buying and selling spare parts. I do not track what's in my systems (well, I guess I have 'hinv output...). I do not track generic or easy to replace parts like disks or memory. I only care about two things: (1) does it work and (2) do I need it. It is utilitarian, and not a purpose in itself.

Keep it simple. For me, that means I don't set up a database, or deal with java, LDAP, XML etc etc. when a simple flat list will do.

But hey, it's a free world.
:PI: :Indigo: :Indigo: :Indy: :Indy: :Indy: :Indigo2: :Indigo2: :Indigo2IMP: :Octane: :Octane2: :O2: :O2+: Image :Fuel: :Tezro: :4D70G: :Skywriter: :PWRSeries: :Crimson: :ChallengeL: :Onyx: :O200: :Onyx2: :O3x02L:
To accentuate the special identity of the IRIS 4D/70, Silicon Graphics' designers selected a new color palette. The machine's coating blends dark grey, raspberry and beige colors into a pleasing harmony. ( IRIS 4D/70 Superworkstation Technical Report )
necron2600 wrote: Then you deal with an LDIF file for import/export.. backups and sharing
It may be more intuitive for inventory purposes than an Excel spreadsheet or a filesystem directory layout.


You can always make a screenshot of the Device Manager. :mrgreen: :mrgreen: :mrgreen:

jan-jaap wrote: Keep it simple. For me, that means I don't set up a database, or deal with java, LDAP, XML etc etc. when a simple flat list will do.


Yea, I got you. I just got excited and inspired by your concept. =)
Image Image
I use a personal wikipedia (MoinMoin, written in python) to track what I have in the laboratory, so I can access it through a browser.

Fields are like: { what is it { hw,sw,fw}? does it work?, which is the last working branch?, how much did it cost?, how much I was paid for it? did they pay me {yes, no, N/A as it's personal project}? when I started the project?, which project is it?, where is stuff located? Whom have I to contact (email,phone,address)?, notes if any? }

But, I care more on red lines ( it's not a personal project, still waiting for the money ).
Head Full of Snow. Lemon Scented You
For cataloging. I just use a simple text list.
:Octane2: - :O2: - :Octane: - :Indigo2IMP:
jan-jaap wrote: The purpose of my inventory is not to know where things are. This should be obvious


Not so obvious if you travel a lot and you have parts hosted in different offices and warehouses!
For me, "where is it"? Is important, as well as the person I have to contact if I need parts shipped.

johnnym wrote: So why not use a directory structure like an inventory?


Some time ago (2011? ), I started a file system project, bTree-driven as it should be, but ... at some point I wanted to do a bit of research, so I started wondering why shouldn't I have to use multiple keys to point to an inode.

A tree is not a natural structure when you have do deal with some items which might have correlations.

Allowing correlation breaks the tree-shape of the information, so file systems don't allow them. But I wanted to see what could happen, so I applied some ideas to the engine, and tags were then allowed, so you can have relational items, and organize them into categories by the use of"tags".

A tag is a new actor, you have files with contain data and metadata, you have folders which contain files but only of those with the same mother node (classical concept of a classic folder), and you have tags which contain inode to files and folders (one, or many) from everywhere on the volume.

With a tree-based model you should have have multiple copies of the same information, or use "soft/hard link" (e.g. ln -s ... ) to point to the item (file or folder) you want to relate into a category, whereas the tag-design paradigm is to use tags to group items (files, folders) that can then be pulled into a view (workspace, or category); this allows you to have multiple view of the same information at once without the need to replicate them, and it's also more efficient to assign or reassign those tags and their related views on the fly. More efficient than having to deal with hard/soft links.

But, a tag makes the file system more fragile than a broken link, it's more hugger-mugger when things decide to break themselves, so ... of course there are problems with the consistency of the whole volume, and hey? Don't expect performances. Neither fix-tools, and be sure that a the first crash you will completely lose everything.

Not so good.

Btw, at the end of my research, I understood it's more useful for tasks like handling a library, like a collection of music, or ebooks, on your PDA, I mean when insert/delete events don't happen so frequently.
Head Full of Snow. Lemon Scented You
Y888099 wrote:
jan-jaap wrote: The purpose of my inventory is not to know where things are. This should be obvious


Not so obvious if you travel a lot and you have parts hosted in different offices and warehouses!
For me, "where is it"? Is important, as well as the person I have to contact if I need parts shipped.

johnnym wrote: So why not use a directory structure like an inventory?


Some time ago (2011? ), I started a file system project, bTree-driven as it should be, but ... at some point I wanted to do a bit of research, so I started wondering why shouldn't I have to use multiple keys to point to an inode.

A tree is not a natural structure when you have do deal with some items which might have correlations.

Allowing correlation breaks the tree-shape of the information, so file systems don't allow them. But I wanted to see what could happen, so I applied some ideas to the engine, and tags were then allowed, so you can have relational items, and organize them into categories by the use of"tags".

A tag is a new actor, you have files with contain data and metadata, you have folders which contain files but only of those with the same mother node (classical concept of a classic folder), and you have tags which contain inode to files and folders (one, or many) from everywhere on the volume.

With a tree-based model you should have have multiple copies of the same information, or use "soft/hard link" (e.g. ln -s ... ) to point to the item (file or folder) you want to relate into a category, whereas the tag-design paradigm is to use tags to group items (files, folders) that can then be pulled into a view (workspace, or category); this allows you to have multiple view of the same information at once without the need to replicate them, and it's also more efficient to assign or reassign those tags and their related views on the fly. More efficient than having to deal with hard/soft links.

But, a tag makes the file system more fragile than a broken link, it's more hugger-mugger when things decide to break themselves, so ... of course there are problems with the consistency of the whole volume, and hey? Don't expect performances. Neither fix-tools, and be sure that a the first crash you will completely lose everything.

Not so good.

Btw, at the end of my research, I understood it's more useful for tasks like handling a library, like a collection of music, or ebooks, on your PDA, I mean when insert/delete events don't happen so frequently.

What you have just described here is pretty much a graph database. One which focuses on the relationships between entities, rather than just the data itself. Dynamic data model!
Shiunbird wrote: [...]
Imagine the following two files

/computers/intellistation/dev/1gb-samsumg-ram-serial_number
and
/computers/intellistation/dev/scsi/20gb-ibm-scsi-hd-serial_number

These two devices have different specifications, so you can track, for example, for the RAM:

Code: Select all

manufacturer=samsung
capacity=4GB
number_of_chips=8
chip_capacity=512MB
device_type=hard-drive


And for the HD:

Code: Select all

manufacturer=ibm
capacity=20GB
rotational_speed=10000rpm
bus_type=parallel-scsi
bus_speed=320
device_type=ram-module


I didn't thought about doing this for machines, too. That's a cool idea! Using hinv is nice and handy and always current without keeping it up to date manually, but it only works when the machine is running and if it's SGI based. You can of course store the output but it will look differently compared to output from machines with other firmware (e.g. IEEE 1275 compatible). So why not create the hardware directory yourself the same way for all machines?

I'm currently experimenting with bar codes on boxes and bags for container like things. My recently acquired scanner (a Symbol LS 4004i with RS-232 interface which I can recommend) also understands most bar codes (for serial numbers or similar) I've seen on spare parts (except 2D bar codes like QR, though it should work with PDF417 , but I haven't tested this yet), so I use these to identify an item in a container instead of applying another bar code myself. Items are "containers" itself, in the sense that they contain meta data (manufacturer, type, description, state, etc.) as leafs (simple text files) and sometimes also other items, e.g. think of those hot-plug memory modules of a HP ProLiant DL580 G3/G4 containing four DIMMS.

So currently the directory names are solely made of numbers and further info can only be found in the meta data in each directory, unless you understand the numbering schemes of some manufacturers. I'm not yet sure if this is the way to go, but I'll see how this works out in the next weeks.
:Indy: :O2: :Octane: :Octane2: :O200: = :O200: - :O200: = :O200: (O200 cluster w/2 GIGAchannel cabinets)
[ ( hp ) ] 712/80 c3000 (dead) :hpserv: (J5600) c3700 c3750 c8000 rp2470 :rx2600: (rx2620) rx4640
| d | i | g | i | t | a | l | AXPpci33 AlphaStation 200 AlphaStation 255 PWS 500au AlphaServer DS20E AlphaServer DS25
C O B A L T Qube 2 Qube 3 RaQ RaQ 2 RaQ 4r RaQ XTR
spiroyster wrote: What you have just described here is pretty much a graph database.


sorry, I don't know graph databases, never used, never studied, I don't know their properties.

spiroyster wrote: One which focuses on the relationships between entities, rather than just the data itself.
Dynamic data model!


I don't get your point. "Data" is file's content, as well as file's metadata (access time, attributes {RWX RWX RWX}, owner, group, etc etc). Nothing different from a classic UNIX filesystem; I mean it's just a fs with tags, which allows items to be grouped within or without the the constraints of having the same mother node.

if you don't remove this constrain, you obtain a classic tree, with folders and files.

Relaxing one degree of freedom you obtain tags; basically a tag is like a folder (and they have a lot of routines in common), but it can points to an item { file, folder } n-degrees deeper in the tree hierarchy.

A tag can virtually groups other tags, including itself, but this might potentially cause endless disasters because in this case it adds itself twice in the entry block like a closed loop (deadlock). So, I removed this possibility: a tag can point to other tags, but it can't point to itself.
Head Full of Snow. Lemon Scented You
Y888099 wrote:
spiroyster wrote: What you have just described here is pretty much a graph database.


sorry, I don't know graph databases, never used, never studied, I don't know their properties.

spiroyster wrote: One which focuses on the relationships between entities, rather than just the data itself.
Dynamic data model!


I don't get your point. "Data" is file's content, as well as file's metadata (access time, attributes {RWX RWX RWX}, owner, group, etc etc). Nothing different from a classic UNIX filesystem; I mean it's just a fs with tags, which allows items to be grouped within or without the the constraints of having the same mother node.

if you don't remove this constrain, you obtain a classic tree, with folders and files.

Relaxing one degree of freedom you obtain tags; basically a tag is like a folder (and they have a lot of routines in common), but it can points to an item { file, folder } n-degrees deeper in the tree hierarchy.

A tag can virtually groups other tags, including itself, but this might potentially cause endless disasters because in this case it adds itself twice in the entry block like a closed loop (deadlock). So, I removed this possibility: a tag can point to other tags, but it can't point to itself.

No point really. Just say'in o.0

The situation described could be perhaps, more naturally resolved being represented in a graph database ;)

As you say, using a filesystem as a grouping mechanism allows you to structure you data in a tree like hierarchy. However, this means you need to know your data model in advance, and has it's own inherent problems, such as forcing a more relational like tree hierarchy/structure. Fine, until you require another grouping mechanism which does not fit the tree hierarchy representation, this as you say, can be solved with tags (tags on the nodes/data), but the fundamental data is ultimately structured/ordered in tree-like, so is limited with what the tags can do apart from them basically being metadata attached to a bit of data and have no hierarchy themselves (all at root level). This is like fudging two different data models together (one structured, and the other too flexible). All these issues could be (but don't have to be) resolved by using a graph database which doesn't impose a tree hierarchy (although it can emulate one) and allows you populate any metric you want (even ones which you do not care for at the time the data is acquired). It's how deep neural networks can operate since a graph databases scale well, and are extremely efficient with vast swarms of data allowing relationships to be generated at a later date. i.e. after there is a correlation between two bits of data that was originally thought mutually exclusive. The data has not changed, rather there is a new associated/relation/edge between them. Something was learned 8-)

Incidentally, a filesystem is an abstracted representation itself and probably has little to no relation to how the data is actually physically ordered/stored.

Anyhows, rights and wrongs of relational vs graph is a bit outside the scope of this diatribe, so I shall... as they say, reserve that for the reader.
spiroyster wrote: As you say, using a filesystem as a grouping mechanism allows you to structure you data in a tree like hierarchy. However, this means you need to know your data model in advance, and has it's own inherent problems, such as forcing a more relational like tree hierarchy/structure. Fine, until you require another grouping mechanism which does not fit the tree hierarchy representation, this as you say, can be solved with tags (tags on the nodes/data), but the fundamental data is ultimately structured/ordered in tree-like, so is limited with what the tags can do apart from them basically being metadata attached to a bit of data and have no hierarchy themselves (all at root level).


Have you ever worked with a CMDB? I hope you never ever ever have to deal with it.
https://en.wikipedia.org/wiki/Configuration_management_database

At work, we've moved from a custom-made Oracle DB to a proprietary DB by a German company built on top of Oracle (and the most awful front-end), and now we are going to the CLOUD (the miraculous solution to all IT problems). The migration is going to take years, because the vendor won't customize much of it for us so we need to review millions of entries.

You mention in a very theoretical way practical problems we always face:
- you need to enter a set of data that is organised in a way that doesn't fit the initial model. What do you do? Build a custom application that writes and reads in a specific way to the database, and tries to format the data in a way that the other entities accessing the database can utilize and understand.
- you have a mix of automatically-populated data and human-populated data, and then you have applications that rely on both kinds of data as FACTS.
- the odd use case.

For example, we had a colleague in Asia who had NO SURNAME. But the system won't allow blank last names, so:
So we had in different systems:
UserId=1234, FirstName=Name, LastName=.
UserId=1234, FirstName=Name, LastName=[blank]
UserId=1234, FirstName=Name, LastName=[space]
UserId=1234, FirstName=Name, LastName=Name
Most applications would use UserId as unique identifier among different systems (Active Directory, Lotus Notes, SSO, etc..), but some operated on basis of user name.
Her phone would periodically stop working and I've heard once she didn't get paid.

Back to our initial discussion...
We can use the IRIX inventory command, or export lsdev, and write scripts to have the formats matching. But once we have a different use case that reviews a limitation in our way of structuring the data, we have two options:
- review all legacy entries
- watch the beginning of chaos unfold in front of our eyes

The hardest of all is to have the discipline to keep everything updated as we go. I'd love to hear from anyone here who feels being successful with that.
Image Image
Shiunbird wrote:
spiroyster wrote: As you say, using a filesystem as a grouping mechanism allows you to structure you data in a tree like hierarchy. However, this means you need to know your data model in advance, and has it's own inherent problems, such as forcing a more relational like tree hierarchy/structure. Fine, until you require another grouping mechanism which does not fit the tree hierarchy representation, this as you say, can be solved with tags (tags on the nodes/data), but the fundamental data is ultimately structured/ordered in tree-like, so is limited with what the tags can do apart from them basically being metadata attached to a bit of data and have no hierarchy themselves (all at root level).


Have you ever worked with a CMDB? I hope you never ever ever have to deal with it.
https://en.wikipedia.org/wiki/Configuration_management_database

At work, we've moved from a custom-made Oracle DB to a proprietary DB by a German company built on top of Oracle (and the most awful front-end), and now we are going to the CLOUD (the miraculous solution to all IT problems). The migration is going to take years, because the vendor won't customize much of it for us so we need to review millions of entries.

You mention in a very theoretical way practical problems we always face:
- you need to enter a set of data that is organised in a way that doesn't fit the initial model. What do you do? Build a custom application that writes and reads in a specific way to the database, and tries to format the data in a way that the other entities accessing the database can utilize and understand.
- you have a mix of automatically-populated data and human-populated data, and then you have applications that rely on both kinds of data as FACTS.
- the odd use case.

For example, we had a colleague in Asia who had NO SURNAME. But the system won't allow blank last names, so:
So we had in different systems:
UserId=1234, FirstName=Name, LastName=.
UserId=1234, FirstName=Name, LastName=[blank]
UserId=1234, FirstName=Name, LastName=[space]
UserId=1234, FirstName=Name, LastName=Name
Most applications would use UserId as unique identifier among different systems (Active Directory, Lotus Notes, SSO, etc..), but some operated on basis of user name.
Her phone would periodically stop working and I've heard once she didn't get paid.

Back to our initial discussion...
We can use the IRIX inventory command, or export lsdev, and write scripts to have the formats matching. But once we have a different use case that reviews a limitation in our way of structuring the data, we have two options:
- review all legacy entries
- watch the beginning of chaos unfold in front of our eyes

The hardest of all is to have the discipline to keep everything updated as we go. I'd love to hear from anyone here who feels being successful with that.


No, will try to avoid, thanks :P .

Luckily, I'm a software engineer and we have a dedicated IT dept. to deal with this 8-) . I'm a recent convert, and graph databases have solved design problems for us on at least two occasions. Mainly due to the ability to have multiple data models all represented in a single database (single query interface), and their future proofing. Certainly, you need discipline, and at a higher level, require a vague idea of what you want from the data, but the details can come later ;) .

They are not the be all end all, and yes, I can approach from a theoretical point of view for fresh data, migrating existing databases would be a right pita (And thank lawd I don't have to do this for a job). We do have lots of third party 'manufacturers' data stored in SQL-esque form which we still need to update, deploy and subsequently query at runtime in the same context as our uber database. However, these can be hidden behind a compatible alternate query language interface rather than migrating it to a single database (although IRL, devs just know which database interface to use...atm... I WILL CHANGE THIS!), but I'm confident a graph database would be able to handle all aspects of this data (and more) should we decide to purify it all.

FWIW, I have implemented a graph-like bespoke database as a PoC for internal representation (which is in fact pretty straight forward, until you want to enforce stuff like ACID transactions o.0), however it looks like we will be using neo4j (although I take offence at the jre requirement, but it comes with loads of tools, including nice visual eye-candy tools). Ultimately it doesn't matter as I'm pretty sure most graph-esque query languages can be translated will little effort into other graph-esque queries (explicitly by the user, if not implicitly by the interface). Or at least, I haven't come across any yet o.0 (although I'll be the first to stick my hand up, and admit its a big wide world of databses out there o.0. I'm not hugely experienced with different graph database implementations, more the concepts required for data management).

The cloud is the future imo. I pretty much work via the cloud since I'm part of a distributed development team (we have a dev in NZ! which is literally the opposite side of the planet to me :shock: ) and so am I quite happy with its existence as I could not work the way I do without it. I like to think clouds mean we have come full circle, we don't have mainframes down the corridors fed by terminals, now we have some grid thingy in (potentially) another country that we interact with via dumb down ARM/Atom clients. In 5 years time, the only requirement for workstations will be for legacy software o.0. I speak from CAD perspective, although it seems like everything is turning into SaaS (even gaming :twisted: ), which will end up being facilitated puerly by 'the cloud' only. My hypothesis.

<ontopic>
I don't have an inventory of all my kit, but perhaps I should create one, and will post the case study here when I do.