29 July 2009

Missing ontological serinity in the world of software systems architecture

Updates: See bottom, but also this question on StackOverflow.

Ok, so let me say from the get go that I'm a little bit upset. Well, maybe angry and bewildered more than upset, but nevertheless not happy. And it all has to do with the dingbat way we architecture our various computer systems. So, yeah, quite generic and not really something we can do much about.

Let's rehash. I'm a SOA junkie, an EDA pimp, and I hate by default the bullshit in any Enterpise camp that promotes their way of doing must be right. And by SOA, I don't mean no ESB bullshit, I mean a hard-core focus on services for architectural means. I build ontologically driven systems, and care deeply about semantics where most others don't give a monkey's bottom.

Lately I've had to rehash my knowledge on plugin architectures (both implementation specific and theoretical), how to modularise complex pieces of software, and implement an event-driven platform on which to run my systems. So I've been snooping around, and there's a ton of models and architectures to be found. But being found is not the same as finding what you're after, especially as I have a few criteria to my search; I want to find something that's generic, simple (but not simplistic), elegant (as in, does not suck) and extendable, an architecture that's event-driven, modular and open. Nothing. I've found nothing. Of course they all claim to be amazingly fantastic and super and great, but looking under the hood, if allowed, reveals yet another staticly created shared library stack with some hooks for your software to use, using some misnomer like SOA or EDA or any of the hundreds of other Enterprise bullshit terms out there.

So, I set my goals lower in the hopes of finding anything of value, even went and asked real programmers what I thought was a simple question, making it specific enough to hopefully muster some replies. Nothing. It seems everybody's got their own way to handle their own little piece of the universe, that people cling to their silos of comfort or something, afraid of what might happen if we all agreed on something. Even when you dig into large architectures, like my own Linux Kernel which I'm using to write this post, there's tons of layers and shared libraries that's hubbled together in a way that does the job, ok, but doesn't make it, in my eyes, an easy job to do, elegant to extend or easy to change.

I guess I should clarify. I'm knee-deep in ontology work for software systems architecture, a field that's almost chemically free of any active community, has a few scattered experiements that went no where (and I'm tempted to put ADL in that category, too), a few papers here and there that talks about it in very generic terms (either as abstracts to academic stroke sessions, or a white paper claiming to be the second coming of Jeebus!), but as to hard-core practitioners like me who want to inject a Topic Map with events of given types that matches certain ontological expressions and Topic Map fragments of certain types of architectual patterns, tough! You're on your own, kid.

So, what am I after?

Well, many things, but I'll try to be a bit clear here. I've cut down on my wants, to, in order to try to find others out there doing similar things. So. I'd like to see a simple event-driven software stack that scales ontologically, and isn't bound to any technology, company or otherwise religious platform. This means that the stack with its names and values work just as well for a small plugin as it does for a larger system like an extra-OS or a cloud, works for potato-peelers as well as online booking agents, database connection pools and kernel space memory managers, but also can grow and shrink with need, in such a way that all other parts of it when they need to can find out what those changes are. This digs into creating an upper ontology for information science, of course, but more importantly it means I'd like to plug software into various parts of a stack, so that everything - and I mean everything! - is an event listener. I know some micro-kernels work in similar ways but highly statically bound, but regardless these ideas are way past the cradle stage by now and need to have a greater exploration in the real-world.

So when I download an open-source package of sorts and try to find out what its stack of operation looks like, why is this information so hard to find? Or compare the Java event model and the .Net model. Or OSes. It seems it's very hard to agree on these things, but I doubt the state of things isn't because they've tried and failed, but because they haven't tried. It's a big world and this is a big field, yet this has not been tried in any meaningful way.

Sure, the technologies promoted through OASIS, ECMA and W3C in themselves have various solutions and tries to bind stuff together in a coherent way as not to confuse us too much, but even within their own stacks of proposals and standards there are huge gaps, great leaps of faith, and generally no clear direction. Even W3C who pushes the semantic web movement hasn't got anything to say on the matter. It's starting to drive me bonkers.

Ok, I'm done. My steam has gone out, but I'm not feeling any better. Off to do my own thing, like the rest of them. :)

Update: Ok, it seems I'm not getting my message across. Let me create a simple (and wrong) example ;
  • SOA : Start
  • SOA : Configure
  • SOA : Map
  • ENV : Start
  • ENV : Configure
  • ENV : Map
  • APP : Start
  • APP : Configure
  • APP : Init
  • APP : Connect
  • APP : Perform
  • APP : Teardown
  • ENV : Teardown
  • SOA : Teardown
Here we got an application session events where SOA is, er, SOA, ENV the "environment" (whatever that should mean), APP is an application, and so forth. This list should be HUGE! Think of all the interesting events one could generate from you turn the computer on until someone gets Rickrolled on the other side of the planet! I want to map environments, systems and eco-systems, with labels. In some regards it's an enumerated list of points that any computer system traverses on its path from being loaded into memory until it leaves it. And possibly then some.

I want to map the software system world! I want to know what people call their various points on the software stack, what they call their events, how they see them work together, how they forsee workflow interactions, how they define system integrity, thoughts on implementation, named entities, the works.

I can find heaps of this stuff, but none of it is globally agreed upon, it's all tucked away in projects or companies, it's their own version of how things should be and what happens. Even big players such as Sun / Java and Microsoft / .Net have very different event models and ontologies, and they are not compatible in any meaningful way. I would expect some parts of CORBA had done work in this area, but what I've seen is very transaction oriented where clients already know the ontology and uses CORBA to travel through rather than be defined by.

As an example of the closest I've found so far in the realm of mapping machine-parts ("machine" here is "software systems") is the Open architecture computing environment which tries to define up the most important parts of software systems (although the final version was released in 2004 ... these things can be considered to be final? Where's the clouds?), but lacks the ontological and semantic definition, has no event or message structures or standards, nor does it have any notational value or end-points which, admittedly, I could spend the next couple of weeks doing, but let's see what else is out there.

Making any sense?

Update: To be even more specific, trawling through IPC is really what I have been doing for the last few days, but getting to the core ontology of all of this is soooo painful. Surely someone out there have done something like this? I've even gone through POSIX trying to gleam what nuggets I could find, but the system level of that beast is just so low it's not funny. Promising is the DBus architecture and event stack, but this again is very low-level, covers only a fraction of the software systems, and is littered with duplication of complexities.

Anything else I should hack at? Yes, I've gone through the most of the WS-* stack as well, digging into past knowledge I had hoped to never see again, but here as well as most other technologies out there they seem to be obsessed with being so flexible that they forget to be defining. So, we get a lot of scaffolding and frameworks that you can extend and define your stuff in, but no clear definitions of what the world looks like. Even an obvious contender like WS-Events and the less-know WS-Event from Hewlett-Packard have nothing more than a functional approach to defining and registering events but that's it, leaving the defining to some semi-ontological layer.

But I'm still convinced lots of people have done this sort of work, especially in these Semantic Web haydays. Browsing through the thousands of OWL ontologies in Swoogle for 'software architecture' (which doesn't really cut it, but is the closest term that yield results) leaves me just overloaded. Sure, the OpenGroup SOA ontology for example, does provide me with, eh, lots of interesting stuff, but again it's a special domain (SOA, obviously) using a certain moniker (service orientation, which sucks when you want to define events across operational stacks).

Argh! Can you tell I'm going bonkers?


  1. Not sure if this is what you are after, but were I you, I'd look closely at Atom/AtomPub + Pubsubhubbub (funny name, very cool stuff -- pubsub based on web hooks). HTTP all the way up & down. You might also have a looks at the ecosystem sprouting up around XMPP (Google Wave for one).

  2. I put a response on StackOverflow suggesting Spring, but I'm not sure it's the kind of all-consuming god framework you're talking about here :) But it is pretty nice for web apps. What you probably want is Spring plus RDF for its data store, then you'd be cooking.

  3. Tuple spaces might be interesting.

  4. I second Peter's advice to look at REST and it's incarnation in AtomPub. It won't describe the universe of software, but its a nice clean implementation.

    On the subject of mapping the software universe you might want to take a look at Richard Gabriel's Patterns of Software if you haven't seen it before.