Showing posts with label topic maps. Show all posts
Showing posts with label topic maps. Show all posts

28 March 2011

Options for an open-source project

I'm nearing completion of the next generation of xSiteable, which used to be a "website generator in XSLT using Topic Maps" but has evolved over the last 10 years into "RESTful SOA, event-driven developer-focused, less-is-more PHP framework and super-cool XSLT templating layer, with a Topic Maps engine, full-stack plugin architecture, life-cycle-based event model, shaped around HTML5, CSS and JQuery" kinda framework, a little something I've tinkered with to fit all my many needs over the years.

Now, in my current job I'm creating a document-control, intranet, social web app thingy for a health-care organisation, and we've agreed to open-source the lot. So, I could use some advice, but here's the current plan ;

  • xSiteable is BSD licensed, and consists of the framework itself
  • a yet unnamed application written using xSiteable is released with BSD license as well

How should I distribute the two? Are the licenses ok? What source repository to use? Google Code, GitHub, what? (We've got both in subversion on a local server right now) xSiteable is really a collection of reusable classes where some are dependent on others, but not completely. Where to document the API's? Where to document examples and guides? How to attract users? Should I create a blog for it? And on and on.

Thoughts or ideas?

18 August 2010

Updates and recommendations

Right, so here's where I'm up to these days ;

I'm working for a local (but fairly large) health care provider as their intranet guy (building an empire from scratch, and may well include Topic Maps) both in design, implementation, usability, and process management, a role that is being expanded crazily with every day as we discover new territories to conquer and submit to our new reign of knowledge management. So yes, I'm actually enjoying it, even though the challenges are sky-high and densely packed.

I'm writing a book tentatively (and probably) called "The well-tempered monkey" (with some fancy sub-title, I'm sure), and it's about evolution, baroque music, the IT industry, software development, human psychology and cognition, category theories, philosophy, geeks, procreation and laser-guns! I'm roughly 1/4 finished with the first draft, and it contains heavily edited blog posts, lots of new writing and thinking, and my own pictures and designs. (Can an eBook embed music? If yes, I'll put some of my music in as well for good measure) Looking for tips, but think I'll make it a eBook-friendly PDF with a donate button at this point, unless you have a better way.

A good friend and librarian / cataloger Saskia has started a new blog called "All things cataloged", and she is well-versed in the black art of Topic Maps and identity management. You should check it out, it's good stuff.

Today I stumbled upon a Dutch version of a Norwegian classic by Sigrid Undset on a lonely bookshelf in the corridors of a health care facility in Albion Park, Illawarra. Man, that was a seriously crazy moment!

I'm closing in on xSiteable RESTful event-driven resource-oriented PHP framework for enterprise application development with embedded Topic Maps / identity management. I've started documenting the thing, and I'll release it soon-ish, I think. It also feature a funky Topic Maps-based XSLT dynamic GUI templating framework that I think should be a project all by itself, but hey, I'll throw it in for good value.

I've created a number of upper and core ontologies that I might release at some point, some of them obviously designed for more fuzzy Intranet stuff, but I'm increasingly getting all representialist on my arse, outing basic category theory and generally thrashing the good name of entites everywhere. I feel a long blog coming on.

Did I mention that all enterprise knowledge management software friggin' sucks? Like, sucks balls? All of them. I've tried them all, extensively, and they all just fail the one simple rule I've got; make KM easy for people. Confluence, Atrium, Documentum, SharePoint, SocialText, LifeRay, all the portal apps and associated server technologies, Vignette, Microsoft, Oracle, SAP, Sun (hehe), I could go on and on, they all SUCKS BALLS! They are technologists solutions to human problems, and failing basic compassion and respect for the generic user! Usability is not about pretty friggin' colors and cute graphics! I'm disgusted with the state of affairs as the usability of these things have not improved much or at all in the last 20 years I've worked in this field. (And yes, I'll friggin' make my own, I'm sick of this ...)

Anyway, on that happy note, life isn't so bad, and I'm eagerly awaiting the next crazy chapter in my life. We'll talk soon.

3 June 2010

Topic Maps visualisation

I've had a screenshot of a demo I made some time ago just lying around on my desktop, sulking for being forgotten and ignored for so long. I guess I wanted to polish the demo up and make it live at some point, but I've been distracted by a pandemonium of butterflies that has taken up room in my house of late, so I'm just going to dump it here ;


What you're seeing is parts of the Opera Topic Map by Steve Pepper represented as a Treemap (similar to the Protovis thing that Patrick linked to today), and the concept work well as a way to browse around the Topic Map itself, even through smooth zooming in and out of large and small groups of things.

The next fun thing is another tool I've made that takes some YAML structure or XTM as input, and create pretty decent graph representations of the map using GraphViz ;


This one is an ontology that is especially made for a portal I'm doing for Claudio Monteverdi, the baroque composer (and a hero of mine). Here's what a partial input looks to get the idea ;


    Music:
        (c) Work:
            (i) L'incoronazione di Poppea:
            (i) Il ritorno d'Ulisse in patria:
        (c) Form:
            (i) Madrigal:
            (i) Motet:
            (i) Canzonet:
            (i) Mass:
            (i) Trio sonata:
            (i) Opera:
        (c) Style:
            (i) Primo prattica:
            (i) Secondo prattica:
    Person:
        (i) Monteverdi, Claudio:
        (i) Monteverdi, Baldasarre:


This method is perfect for sitting down with customers and clients, and just jot out models, ontologies and concepts; all you need is a text editor. (You'll notice that assocs are made through (c) (i) and a few others, which basically are assoc templates you can define more of, so a meta ontology for ontology construction. Yay! :) It's a couple of PHP classes (one of them being the PHP Topic Maps engine I've talked about before that I'll wrap up if I get some spare time) that generate DOT from the input, and serves up a generated image on refresh, so basically you edit the text file, save and hit refresh. Very fast. I might even consider an inbuilt editor in the browser, but pah!

Anyway, just a quick dump there. I'm also working on a far more complicated TM visualisation thingy using RaphaelJS to play directly on the HTML5 Canvas using native and SVG graphics, and it's looking very good so far. I'll show you more when things gets closer to something I won't hide in shame over ...

6 May 2010

Of models, frameworks and the paradigm shifts we don't see

I need desperately to talk about models. I hunger for them, I need them, I love them! They are pretty, of course, arousing us and making us do silly things like buy stuff we don't need, or fooling us into buying something we think we need but we later find out that we're a bit stupid, we were mislead by the models winks and beautiful attire. However, they are often more than just skin deep, more than just the abstract notion of our perversion to objectify everything we see. Confused yet? These models are not found on the cover of glossy magazines.

These models are everywhere. They are how the brain takes a group of concepts - be it physical objects or knowledge nuggets in our head - and plonk them all into a grouping of sorts, and draw lines of meaning between the new group and the old it knows about. Together they form anything from thoughts to language to the reasoning used when voting. Between person A and political issue B we draw a relation type "opinion" with opinion C. A few fuzzy million of these, and we can pin you down pretty well.

Where do models live in our various systems? Well, they live in language, for the most part. And this is not some cryptic ballyhoo I'm inventing here, so I'll demonstrate with the humble book. Let's look at a description of a book in the foreign and mysterious language called Eksemelle ;

The book

<book>
   <title>Some title</title>
   <author>Clemens, Samuel</author>
   <pompador>In a nutshell</pompador>
</book>

"book", "title", "author" and "pompador" mean something to someone. To most people who understand the English word "book", it means those rectangular objects made of dead-trees that's got some kind of letters and / or pictures in them. They are, as we say, semantic in that they have meaning to those who knows what those words mean. Here's the word "book", and it has a definition of sorts; that is explicit semantics. But then there's the implicit semantics of what those words mean in terms of this being an XML snippet, maybe from a specific bibliographic format, maybe with an even more specific XML schema (unless the XML schema is made explicit through something like a DOCTYPE). And finally there's the tacit semantics in the space between the model, the framework, and the people who work with it. Let's explore these shark-infested semantic parts.

How are they semantic? In what way are they meaningful, and to whom? Well, let's start with who. No model is really valuable unto itself; there's always some external framework that understands (and appreciates?) the model, some "thing" that looks at and interacts with that model in order to make it useful. Indeed, a models usefulness is often measured by how the semantics of various models match up. For example, the usefulness of the MARC model can be measured in terms by how well it matches the model librarians work with and need. Needless to say, the usefulness of the MARC model can also be measured how useful it is for a bricklayer, but more on this later.

So in what way are they meaningful? Every time we talk about models, we are really talking about a translation that's going on between models. The model of the words in this blog post is translated first from my brain and into the model used by my blogging software that uses some models of the Internet and computers and complex electronic networks and systems, and then translated to the model of your brain. We take a piece of semantics, and we try to make the transition from my brain to yours - through a multitude of other models - as smooth as possible. Have I succeeded so far? Does our models match up a little, much or not at all?

Things are meaningful when the models match up, when there is little or no difference between them to make understanding the thing in one model hard to understand in another. An example of a semantic mismatch in models are indeed the semantics of the 245$a field for a bricklayer looking for his bricks.

Constraints on entities

In the past I've talked about how models are constraints on entities, and this still holds true, but it needs a bit of clarification to make sense. First, what are entities?

Well, entities is one of those words that we can make to mean pretty much anything we like, but let's take a simple view of entities being "things you can talk about", similar (or exactly the same) as subjects in Topic Maps, "concept" in philosophy, or, to the layman, "things." Anything you can think of. Any subject, fictional or real, physical or surreal; a boat, a thought about the Moon, the idea of North, the concept of the number 1, the Eiffel Tower, MARC, a MARC record, an XML representation of that record, the book the MARC record represents, a physical book, the relationship between the book and the abstract notion of a book that the MARC record represents ... oh, the possibilities are - truer than anything! - endless.

Between the entities, in the cracks of our language and our understanding, flow their relationships. They are part of our model, those notions that give our entities a meaning of sorts, what makes the semantic;

   "This book" was written by "this author"

Look at what we found in the cracks; "was written by." Isn't it grand? This trifecta is known in the Semantic Web world as a triplet, basically a tuple with a subject - predicate - object structure. Make enough triplet statements about a thing, and it grows in semantics. Let's look at our book ;

   "This book" is our subject
   "was written by" is our predicate
   "this author" is the object

Let's make another triplet statement ;

   "This book" is still our subject
   "was published by" is a new predicate
   "this publisher" is another object

We now have a model a bit like this ;

   "This book"
   - "was written by" : "this author"
   - "was published by" : "this publisher"

We got a subject (or an entity) that we're attaching predicates and objects to, in many ways similar to how we might add named key/value pairs of properties to an object, or create a lookup-table with column indexes between two tables in a relational database, or set named properties on a Java Bean, or even scribble two statements about a book in its margin. We're attaching some semantics to something.

When we attach meaning to something, we are constraining its possibilities that those properties might be something else. We are saying that the title of the book is X, and by saying it is X we can infer that it isn't millions and millions of other titles that it could have been. Before we put that title to the book it really had only two options;

Either the book had no title, or it could be any title we could imagine. But by labeling the book with a specific title, we're constraining both of these options, changing them dramatically from the endless possibilities we had to one specific option. We constrained the book by giving it meaning. And the more meaning we give it, the less abstract it becomes, the more constrained it becomes.

Reflections in the mirror

The models we have in our heads rarely perfectly match the model we're interacting with. The model in my wife's head is not matched well with my own model in many ways, like shopping, views on the value of shoes, the model of interacting with people (she's the one with a model closer matched to the generic likable model shared by most social and nice people) and the concept of geekery (of which she has none). And she is a person that's quite well matched in general. It only gets worse from here.

Now imagine the semantic distance between me and a computer system I've designed. I can work with it. I understand it. I can get my job done. However, I show my perfect model to a customer, and they immediately start picking it apart, pointing out how my model doesn't match their (or their individual) model. How could I have been so blind?

Here's a little secret to why usability and user-centered design works; you test your model against most other people's model in order to get to some model that you all can reasonably match with. When you don't test your model against the users models, they are bound to suck.

Models are reflections of how we humans see our world. The MARC standard most certainly reflect how the librarians saw the world, how it matched their needs and wants. My programs are often a reflection of how I see things. Your browser is a reflection of its developers model of how your browsing should be. This blog post is a reflection of me. Your computer a reflection of some manufacturer. The operating system a reflection of yet more developers views.

Sure, we try to create standards that either try to reflect some common model of things, or at least a common language in which to describe this view. However, it is terribly difficult to come to models that match well across so many thousands and millions of possible models. I'm tempted to almost say we need some constraints of commonalities on our models in order to create semantics to better understand our various models, to better agree and share them.

Where models sleep at night

Where do we find the actual models when we peer into the computer systems we use? Somewhere, surely, is that model in which we try to model to our own model rests, somewhere in there amongst the code and the interface we can point to it and say, "There!"

Mostly we can't, however there is one place where you'll find a lot, one place so holy in computer science that people dedicate whole careers to dealing with its innards; the relational database.

I need to speak about the relational database a bit because, well, so much of most computer systems out there a) have one, and b) store much of their model in there. Yes, there's alternatives, and more and more technology pop up that tries to do different, but let's be realistic about what 99% of most computer systems use, those RDBM systems. Let's have a look at a small corner of one ;

What we see is tables; columns of fields, rows of entries. These form the entities of these models. The example given even uses a tricky third table to function as a lookup-table between two tables of entity data (I won't go into too much technical details here). To get data in and out of these tables, we use a query language like SQL to say things like this simple example, "get all fields (with their rows of data) from tables A and B, where table A has a field 'book_id' that matches its value with a field 'id' in table B, sort it by the field 'title' of table A in descending order, and give me the first 40 results."

One SQL statement that matches for that is a semi-cryptic "SELECT * FROM TableA,TableB WHERE TableA.book_id = TableB.id ORDER BY TableA.title DESC LIMIT 40", or some other variety (there's tons of different ways of saying the same, with or without JOINs, sets and filters).

Let's look for our model. First of all, it's the titles of the tables, the titles of each column in them, and lastly the contents of the rows. Notice here that this is three levels of semantics nested within each query, and you need to know all of them to make reasonable statements. But what else is in our model? Well, it's those pesky relationships between things, the constraints on our entities to make the meaningful, and they exist in the query itself, in your application. Think about that for a second, think about where these things go ;

The table, columns and rows go in the database system. The querying goes in your application. The user interface (which we haven't even dug far into but is a wormhole of complexity in its own right) that interacts with you also sits in the middle, in the application. So there's a model in your database, and you reconstruct that model in your application (otherwise, how could you query the database if you don't know what it looks like?) and yet do further things that are not embedded in the database (and so, you've got a super model ... fun with pun!), translate further between your users and the user interface, translate back into the application which translates back into the database ... ah, what fun spaghetti games that makes.

 Surely RDBMS and SQL is better than other alternatives? Well, it was for many years, and it was a way to solve the problem of doing things in far worse way, for sure. But we were also under the constraints of computing power in which we couldn't just do the right thing and still get a computer that gave you answers in time for Christmas. It was a compromise between the need for any answer to all your data, and that of a practical lots of answers to most of your data.

But this analogy can be taken further, especially into the world of MARC and the libraries. MARC itself was also designed with lots of peculiar constraints, funny rules and structuring, and then with the added AACR2 (and earlier friends) rules for manual data integrity, it surely reflected the best of breed at the time, reflected what they wanted and it was something that matched their onset. So we got the model of MARC (I've called it the culture of MARC in the past, but a model suits just as well) in MARC itself, in the rules we add to it, to our ILS, to our OPAC, our catalog, our acquisition, our collection management, everything. And then the model of MARC is everywhere, it even starts to dictate our human processes.

Time flies

But then time flies, and the world changes, sometimes unexpectedly and the harsher if it is. Just like there's a lot of push for alternatives to RDBMS these days because today and tomorrow is somewhat different from yesterday, the same with a push in the library world to go from MARC / AACR2 to something more like FRBR / RDA.

However.

When you create a model of the future you need to make sure it is future proof. You have to make sure that not only does the model match what you need right now, that it reflects the funky stuff you wanna do today, but it must be able to deal with the future. If it doesn't, well then you are going to have to go through the pains of changing the model sooner rather than later.

Here's a few thoughts on that process in the library perspective. FRBR was designed 15 years ago, when the world was sloooowly waking up the the new fresh brew of the Internet and technology. Take a good look at what the world was like back then, especially paying attention to the fact that books were still the main container for knowledge and information, mobile phones did nothing more than make calls, Internet devices were practically unheard of, no eBooks, no iPads, no eEducation, no eGovernment ... for Fraggs sake, the evil monster of Netscape was still alive and doing harm! I remember still writing Netscape specific code to deal with its quirks. This was a time before we all stopped hating the dying Netscape and focused on the evils of Internet Explorer instead. can you even remember back to that time and think that the world might be in your back pocket or pack in the shape of an eReader or iPhone, or that Amazon (in its infancy) would have a full-blown infrastructure including their own eReader with tons of titles a click away? Or perhaps even more profoundly, that WikiPedia would lead the way to the disjointed revolution of knowledge and information? That we would be twittering? That all higher educational institutions would move towards ePresses? That paper journals would turn to online journals? That the pricing models of online content would change? That the price of admission would change? That even the model of content negotiation would be different? That blogs would dominate the future of discourse, even the serious academic ones? That newspapers would ever fail?

Time flies. And models change. Some models are better at dealing with change, some are better at being future proof, but change they will. And when the models change, you must either change your own models, update your models, or use outdated models. FRBR and RDA are outdated models before they're even implemented. Please reconsider.

Model models

The last ten years or so there's been a stronger push towards meta models. That basically means "simple models in which you can create other models." One might wonder what such a crazy thing would do to help, but let me first exemplify through what I've seen again and again over the years I've worked as a consultant ;

The smallest changes to any complex system, where databases, tables, columns and rows must change, you've also got thousands of lines of query / SQL that also needs to change, every model along the way, from the hardwired entities of the database to the user interface controls must all be updated to this new way of looking at the world. It could even be the smallest of things, say, changing the name of a field in a table from "id" to "book_id" (some times even stupid things like this is needed because the people who create the original model [called schema] didn't worry about multi-join SQL statements that would have a hard time dealing with the ambiguity of the many varieties of "id" fields, or didn't have the foresight to think that more than one thing in your table could be rightly called 'id' ...) could cost in the millions. I know it sounds terribly stupid, probably even terribly untrue, but I swear on the grave of all those programmers who laid down their lives in pursuit of SQL ambiguity and integration and middle-tier testing, that it is a very sad truth.

The library is facing similar insurmountable trouble by switching to anything but MARC, and I suspect the cost analysis is in the multi-millions wherever you look. People are starting to ask if the change will be worth it (and with the criticism laid out about FRBR you might want to have a closer look before you leap), is there another way?

Well, sure, kinda. There's meta models, models that are somewhat ready but needs your lovely input and tweaking to get perfect. They are generally easier to deal with because, unlike models, they are designed to be a bit vague and, well, meta about it all. And yes, indeed; Topic Maps is such a technology. The model in topic Maps is simple ;

  • a subject is anything you can ever think of, anything you want to talk about, anything in the "real" world, like books and people and cars and thoughts and ideas and ... well, anything.
  • A subject is represented in a computer system by a Topic
  • Topics have multiple Names, can be of multiple Types, have multiple Identites and multiple Occurrences
  • Associations of different types tie them all together with roles

If you can wrap your head around such a concept, it's easy to build whatever you need with it, and here's the advantage ;

  • A standardized data model and a standardized reference model
  • A standardized XML format for exchange, import and export
  • A standardized query language and a standardized constraint language

It doesn't matter one bit what data you put into it; all of the above still applies. You can make model-agnostic queries into your data. You can mix whatever data you feel like; nothing can't be put into it. You determine yourself what level of indirection you want on your data. And you can have serious identity management to boot! Did I mention author records done right? Chuck a thesaurii or faceted navigation systems right in the model! Make software modules understand certain languages rather than the combination of languages and data, and share these! Want to see what your data merged with any other data might look like? It's right there in the standard, it comes out of the box. Play with your data, and invent new ways of interacting with it without dicking around for weeks with databases, filtering and merging. And on and on it goes.

But why do this? Well, since the model exist in a domain which is specially designed to handle the disjointed nature of your models and data, they are free to shape whatever solution you might think of (meaning, you can change the interface without changing the model nor the application logic) where you in the past were stuck with the original model design. You can copy and paste your little models and languages around. Try out new things. Merge stuff with ease. And, not the least, focus on application specifics without worrying about model integrity. Nor do you have to worry about user interface integrity, either. How to put it in a way that you could understand? It's like taking a bucket of apples and a bucket of bananas, which in the past would when mixed together make a sticky slushy fruity goo that no one really likes, would now be genetically merged to make the banapple that can still be split apart into its raw apple and banana parts if you felt like it.

Yes, I've whinged and raved about this in the library world (and other places) for years and years, but getting people up to speed on understanding models, their implications and how meta models might be a better bet, then demonstrate and convince everyone (including people with no technical background), all while standing in an elevator with some people who's off to tinker with some RDA or MARC or something. It's hard to get their attention when they don't actually see the problem.

But I'm not pushing Topic Maps, really. Well, a little, but more specifically I'm pushing meta models, and I'm pushing for better ways of dealing with your computer infra structure, to take a few good steps out of the litter sandbox that permutes the current library systems infra structural designs, and get jiggy with the future before it gets overrun by the cool kids with iPhones and iPads and whatsnot's that also, you know, have a model or two. Models that may or may not be compatible with whatever model you come up with next. If you do. Seriously, I thought librarians loved meta?

6 April 2010

Before I write what I write before the next time I write

It seems my last poll revealed that there are still people in the library world who hasn't rejected me, or, perhaps a stronger theory, likes to watch road accidents. So my next piece is being written about why the library world fails so badly at technology and seeing the future (or even their own relevance to it), but I'm somewhat busy these days with real work, so a few more days, ok?

However, all is not lost. I've got a few things to say about, well, the stuff I work with, that bucket I stick my head in every day to see if the crap I put in it yesterday has turned to gold yet. No luck so far.

There's a peculiar discussion going on in the Semantic Web mailing-list at the W3C, of which Bernard Vatant will fill you in. It's funny to watch; where are the success stories, where is the commercial viability, does it even work in academia, has it got traction, what do we do now? Why aren't more people doing it? Why haven't the world adopted this specific and undoubtedly brilliant world-view yet? Are we all mad!?

I'm sure you can fill in with our own Topic Maps echo here, but the more you dig, the more you discover that most of the sillies put up as a reason or a scapegoat for the lack of world dominance are things that, frankly, the Topic Maps community have figured out long ago, and some of those missing features in their world is a dominant feature in ours. And we haven't taken over the world, either. Bummer.

It's frustrating, I know, but what can we do? There's no amount of technology suave that can beat any status quo that feeds upon itself. No new ideas can beat old ones that seem to work, because, well, the definition of "works" is so multi-faceted and complex and, eh, making lots of money for lots of people. Semantic Web and Topic Maps doesn't make lots of many. Heck, they don't make money, period. They're convenient little technologies that will stay small and insignificant.

I have a plan, though, and it will piss off some of the Topic Maps purist (or, let's face it, even pragmatists) and hopefully some Semantic Web people as well. First, I'll rename it something cool - maybe something like NoSQL or something - and then rename the integral concepts, strip away the jargon, and make it web-friendly by injecting it straight into HTML5 based technology, and relate all queries through SQL. Mwuahaha, I might even throw some REST API's in there, just to stir it up some more. And I shall call it ; the web.

Man, I hate these technical wars over standards and ways of doing things. The thing I love about Topic Maps isn't the standard or the specs. No, it's the thinking I'm forced to do in rejecting some parts, while loving others. It's what I take from it. It's the epiphanies it yields.

NoSQL? Semantic Web? Topic Maps? SQL? They're all just abstract interfaces into a set of memory positions shaped by various registers, stacks and pops. Standardizing our ways is just a step on the ladder of the future, not a platform upon which we have to stand firm.

Anyway, the whole NoSQL thing is something I'll have to write about more later. Right now dinner and kids and cleaning the house beckons.

19 March 2010

Newbie tips for Topic Maps bliss

Well, hi there, pilgrim! So, you've noticed this fandangled thing called "Topic Maps", and you heard it was an interesting, smart or new way of solving hard problems of some sort? Well, you've come to the right place. Let me, as an elder of the movement, give you a few hints and tips of how to go about this complex notion ;

1. Don't do it

Yeah, I won't lie to you; unless you know more than a smidgen about information science and / or knowledge management, and especially unless you know quite a bit about data models and how to interact with them in complex computer systems, I'd urge you to stay right clear of it. Topic Maps is full of complex models, silly jargon, weird people and technical API's. Nothing in the normal world is easy, and the Topic Maps world only makes it harder. Topic Maps won't solve your problem, unless you already know how to solve it, at which point find some other technology that people actually know, ok? What's the point of building the best system out there with amazing technology that no one knows how to use, extend, appreciate or even keep a straight face while talking about it.

Also, at what technical level do you think you need it? If the answer is, not very technical at all, why care about the underlying technology? An Excel spreadsheet might fix your problem much better. Use that. Unless you know that multi-dimensional graph-based technology will save you, look another way. If you don't know this stuff it will lure you in with magical wistful promises of a better tomorrow that will never see the time of day.

2. Make someone else do it if you must

Ok, so this one isn't all that different from the first tip, but since Topic Maps indeed can solve hard problems in brilliant ways, there are people out there who could use it and help you solve them. The truth is that people who are steeped in this stuff, who knows it inside out, indeed can solve pretty much any complex issue you might have with it, and even help you become smarter in doing it, even teach you how it all works. And there's other benefits to letting them do it for you; you don't have to become one of them in the process.

The truth is that Topic Maps really is cool and brilliant and all that, but it is ridiculously hard to grasp and even harder to master, and it will change you into a weirdo in the process. Have you really got the time,  resources and personality traits it takes to get into this stuff? Really?

3. Topic Maps people are few and rare, and, uh, strange

I kid you not; these guys are not your average cup of tea, so tread gently, and expect to be surprised in some way or another. Expect them to say things that makes no sense whatsoever, they have their own language littered with technical jargon even technologists wouldn't understand. I don't think they get out much, at least not outside their own field, so you need to replace your own terminology for theirs if they are to made sense of, as they themselves rarely compromise and adapt to how the rest of the world see things.

Also, expect some slight geeky behavior like mistaking things like pages, tags, websites, business objectives, servers, networks, computers and pasta for topics, topics, topics, topics, topics, topics and topics, in that order. It's quite similar to that of Asberger's syndrome, but if you know how, you can use it to your advantage, but caution and patience must be urged, and just like with the real thing unfortunately there is no cure, only workarounds.

4. Patience is not a virtue, but stubbornness might be

Look, the mystical world of Topic Maps is full of concepts you never dreamed existed, things that makes you ask fundamental questions about identity philosophy, what is knowledge, and paradigms of models and technological culture. Patience is not enough to grasp this stuff, you need sheer stubbornness and bloodymindedness to get anywhere, and - dare I say it? - perhaps some weird personally trait. Maybe a limp or a monocle. Not only do you have to understand the technicality of the stuff, but also the weirdness of the culture itself and - perhaps even more important - the personal implications this knowledge might have upon your own thought processes. You may end up getting a cape.

Once you tread down the path of Topic Maps and actually get anywhere (and that in itself is a hallmark of your stubbornness), your brain will change, you will see things differently. I'm not going to say that that is a good thing, but it can be, especially if you like uprooting your preconceived notions and planting new ones. The world is built on foundations which are far removed from the Topic Maps world, but once you grasp this other world it is hard not to see your old world in a new light, and this can be challenging. You might even start to sound like one of these blubbering idiots yourself, saying topics, topics, topics, topics, topics when you used to speak a coherent language people around you actually understood. There's great danger in getting an epiphany or two.

5. If you like your job, stay clear

Hmm, I see a pattern in my tips, but the thing is that once you have converted your old job might look boring and infantile by comparison. You might get (morally repulsive) urges to work on Topic Maps, but your organisation probably won't understand what the hell you're on about (remember, the whole painful stubborn process you went through has to happen to each and every person in your whole organisation!), and you might be looking around for another job where the Topic Maps goodness is practiced. Don't be fooled!

These jobs don't really exist. No one thinks Topic Maps on your CV is a good thing, because they, too, haven't done that stupid painful stubborn path to enlightenment either, and you'll come across as a bit of a show-off with nothing to show for it. (The exception to this tip is if you live in Norway. If you want to know why, the answer is that, again, the Topic Maps culture is repellingly weird) No one who does real business gives a rats ass about Topic Maps, and no one who does real business in the future will either. Even people who does weird but similar things who also have modest success (like people doing Semantic Web / RDF work) shun Topic Mappers. For your own job security, stay clear.

6. However ...

However, if you are weird, not scared by overly complex or outlandish technologies, if you think that strange new cultures only makes you stronger (and you've got a strong immune system to boot), if you think job security is only the stuff of boring people, and, indeed, if you have a monocle, cape and a glass eye, perhaps this might be the place for you after all.

And if so, contact me; I get off on this stuff. Otherwise, you have been warned.

4 February 2010

Topic Maps, 10 years down the line

I'm told, by way of my own imagination based on loose rumors put out by flying pink fairies, that Topic Maps is a waning technology, poorly supported by the IT industry at large, hard to wrap your head around, and generally icky to deal with.

All of this is, unfortunately, true.

But, as in all stories told by only one side, there is an other side just waiting to come out into the light, just one day, real soon now. This day may never come, but here is my own little attempt to shed some light on a few of the issues with the Topic Maps world. It was about 10 years ago I first got a whiff of Topic Maps, so my first post in 2010 seems fitting to take some Topic Maps rumors, loose observations and vague statements, and make some comments along the way. Here we go ;

1. Topic Maps are hard

Why, yes, to a commoner or some person with a somewhat traditional approach to computing, Topic Maps can indeed seem like an alien concept at first. The first time I started reading up on it I was mesmerized and frightened at the same time, wondering where the magic would bring me and just how painful it would be for me when reality would kick in (and me) ; there were new notions and concept, new words, new paradigms everywhere! Reification, role types, associations, occurrences, occurrence type, typified information, subjects and topics, ontologies (upper, lower, specialized ones) the list goes on. It is terrifying indeed, and for many, many people they are so terrifying that SQL and C# and .Net and C and PHP seems like a comforting auntie lulling you back into things we know and know well, no hard thinking required (just lots of hair to pull out).

Until you realize a few things, that is. For example, the vocabulary is anchored in information science, and with a bit of research or learning it shouldn't take that long to get familiar with it. Even the complex issues of reification and ontologies after some time will be as normal and self-explainable as second-cousins and language. (And yes, there is a correlation between the examples given! See if you can find them!) And perhaps more importantly, the problems you can solve with Topic Maps can completely and utterly eradicate the major problems those traditional methods give us, one of the biggest bug-bears that I'd ever had! (Anyone wish to offer me a book deal on how to solve most of the main IT development problems in seriously interesting ways? :)

Can I just mention that having an small epiphany about Topic Maps have the effect of you never returning to the real world and look at it the same way, ever again? I have never met a person who got Topic Maps return to the old ways, at least not without making huge compromises. Getting it will change you in good ways, and is most definitely worth the effort despite the pain.

Tips to newbies: It's not really hard, even if it seems hard. But it requires you to change your mind on some key issues.

2. Topic Maps are poorly supported in the real-world

Oh yes, indeed. If you talk to anyone, any company in your immediate serenity (yes, a tautologically pun) and ask them about their use of Topic Maps, you'd most likely get a blank stare back and a careful "What would we need maps for?"

There's the odd technical-inclined person who might now a toddle about what these fabled Topic Maps are all about, but very, very few people understand what they are, and even less have implemented them into something useful. (The exception to this is, oddly enough, the country of Norway, and some scantily-clad areas of southern Germany) No mainstream software package comes with the stuff wrapped in, no word-processor touts its amazingness, no operating system comes with support for it, and no popular software of any kind use it.

But then, there's the odd system that use it. You'll find it also in the odd Norwegian government portal, which is bizarre in its own right, and perhaps deep down in some academic underfunded project or perhaps some commercial project where parts of the data-model masquerades as it. My old website use it. I have a framework or two. There's the odd other open-source project, a few API's, and a host of other well-meaning but obscure projects that perhaps has got it, albeit well hidden and kept away from children.

For a technology that stands out as something that can fix it all, I find it bizarre that it is found so seldom, but then bizarre is not the same as surprised. And when you look at the "competition", the well-funded, well-marketed, well-established world of the Semantic Web, championed by none other than the W3C and Tim Berners-Lee, well you have to concede that it shouldn't be much of a surprise at all, really. Topic Maps is a tiny group of enthusiasts (a few hundred, being liberal with statistics) who'll saw off their right leg if it meant we could get the specs done in time, while the Semantic World is littered with academia, organisations and companies (we're talking thousands upon thousands of people actively working on it), so no, you should not be surprised.

Tips to newbies: As the saying go, if a million flies eat it ... surely, it has some nutritional value or greater worth over, say, that green grass the cows are dumping it on?

3. Topic Maps is dying and obsolete; use RDF instead

There was a period about 10 years ago which I regard as the Topic Maps time of bloom ; the trees had beautiful flowers on, the pink and purple petals falling over the world of IT like a slow-motion rainfall of beauty. Everywhere you turned there was people talking about it and potential projects popping all the time.

But times went by. Topic Maps was too hard for most (see point 1 and 2), and not just the technical implications themselves and the language and terms used, but also the philosophy of it, the very idea of why we should be using it over, say, any relational database or traditional software stack. I mean, what's the point, really?

The point is easy to miss, admittedly. A technology that can be used for everything is hard to pin down and said to be good for something. And we have focused just too damn much on knowledge management systems, and not only that, but used our own special language in the process which often is quite remote from knowledge management speech in the enterprise arena (but you find it rife in academia). When the world looks to Topic Maps, all they see is a difficult way to do knowledge management. Ugh.

Myself, I'm using Topic Maps in highly non-traditional ways. I use maps for my application (definitions, actions and functionality), for functional topology (generic functionality in hyper-systems based on typification), for business logic (rules, conditions, interactions) and, perhaps just as important, for the actual development itself (modules and plugins, deployment, versioning, services) which makes for a highly (and this "highly" is quite higher than any normally used "highly") customizable and flexible framework for making great semantic applications. But more on the details at some later stage.

Tips for newbies: No, it's not dead nor dying, just not as popular as stuff that's easier or more accessible

4. Topic Maps is nothing new

Well, given its roughly 20 year history (and I'm counting from early days of HyTyme), in Internet years it's an old, old dog, so by that alone we can't say there's anything new, but most people would mean "new" here to mean something like "we've been doing X for years, so why do we need this?", where X usually points to some bit of the Topic Maps paradigm that indeed has been done before. Of course it has. There is nothing new in Topic Maps except, of course, putting it all together and standardize one cohesive and complete way of doing pretty damn most of what you would need for your complex data-model, identity management, semantic or otherwise relational, interoperable information and / or structural need, chucking in knowledge management, too, for good measure.

There are of course nothing new with Topic Maps, except that all that old stuff is bundled into a new thing, if you allow a 20 year old standard to be called "new." But then again, "the standard" is really a family of standards, all evolving and changing with the times. There's always a sub-standard (no pun intended ... well, not a lot of pun intended) in the woodworks, always some half-baked document to explain something or other, always something that is so damn specific and concise that the overall grooviness and funky bits are pushed to the side-lines.

Topic Maps is new and old at the same time, but it really is groovy and funky once you overcome the technical jargon and the concise nature of the standards.

Tips to newbies: The king is dead. Long live the king!

5. The Topic Maps community is, um, a bit tricky

Oh, yes indeed. And this one is the hardest to write about as I'm part of this community and know pretty much everyone, some more than others.

So let's say it this way; I'm a difficult person in certain ways, for example I talk a lot, I overflow with ideas rather than code, I don't care too much about political correctness, and I speak my mind and use language that could alienate people with too strong attachments to their ties or their social buckets.

And the core of the Topic Maps community is loaded with weirdos like me; highly opinionated, rough ideas, hard on woo, and soft on business. But the problem isn't the weirdos, but the low number of them. Any successful community with such a wide-ranging and all-encompassing area of what Topic Maps is all about (which is, uh, almost anything) going from epistemology to identity management to ontology work, well, you need a lot of personalities to match them all to make it seem like a lively place. We, on the other hand, have a handful of people, and the contrast between us all is sometimes just too great. And, I've noticed, we're not very good with newbies, either, so even if we answer their questions, quite often our answers are just too far out there for normal people to comprehend (and I've got a ton of circumstantial and anecdotal evidence to back it up).

I'm part of many different communities on the web, but there is only one champion of how fast an online discussion goes private (and it's not of the good kind; it's the kind where we need to express our frustrations in private [because, ultimately, we're nice people who don't want to offend anyone even when they deserve it, those bastards], lest we blow up and our eyes will bleed!), and that's the community which is located on a private server where you must write to the list owner in an email to be added. *sigh*

I tried my "question of the week" thing on the mailing-list for a while, and some of those went well, but too many of those question quickly descended into nothing or private arenas. So, I'm officially giving up on it for now. Maybe I'll come back stronger once my spine grows back, who knows?

Tips for newbies: Be strong, keep at it, ask for clarification! We don't know just how alien we are. And please join in as we need more weirdos.

6. What, exactly, is Topic Maps, anyways? I don't get it!

Yes, indeed, what exactly is this darn Topic Maps thing? The funny thing is that there is no correct answer to that question. First of all, it's a family of standards that we collectively call "Topic Maps", but it could also mean either the TMDM (Topic Maps Data Model) standard or the XTM (Topic Maps XML exchange format) XML standard, depending on your non-sexual preferences. Some might even go out on a limb (obviously not the limb cut off in point no. 2) and claim that it means the TMRM (Topic Maps Reference Model) which is a more abstract framework, or possibly even just the philosophical direction - or, dare I say it, zeitgeist? - of the thing, like a blueprint for how to build a key-value recursive property framework with identity- and knowledge management system. Your mileage may vary.

But then we have a problem as it is not a technology nor a format. It is more akin to a language, a model or a direction of sorts. No, not a language like SQL (even though the TMQL (Topic Maps Query Language) could be said to hold that place) that is to be parsed by a computer, nor a language like Norwegian or English. No, we're talking about a language that sits right in the middle between the computer and the human, a kind of mediator or translator, a model in which both machine and human can do things that each part understands equally well, a model which is defined through information science, math and human language.

So what is it? It's a language that both computers and humans can use without pulling too much in either direction, a language in the middle that, if spoken by many parties (computers and humans both), they can all join hands and sing beautiful knowledge management songs together, share and propagate with ease. But of course, Topic Maps isn't limited to just knowledge management, oh no. You can solve unsurmountable things with it as you can make it represent whatever you want it to, and I really, truly mean anything. If you want a topic to represent your thing, off you go. It's that flexible.

It can work as the basis for pretty much any system that has structures in it of any kind or shape, and that, by and large, is pretty much any system ever built. So it's actually quite hard to explain just what you can use it for, even though traditionally it's content management, portals and knowledge management.

Tips to newbies: It's only a model ...

So there you go, a quick summary of bits and bobs about Topic Maps. In my next installment, I'll summarize my naval fluff collection, next the train-table changes of Minnamurra station of the last 10 years, and finally I thought I'd summarize all the redundant technology that's gathering dust in my garage. Stay tuned for exciting times ahead!

15 October 2009

Ontological Ponderings

The last few months have been interesting for me in a philosophical sense. My job is on an architectural level in using ontologies in software development, both in the process (development, deployment, documentation), the infra-structure (SOA, servers, clusters) and the end result of it (business applications). So needless to say, I've been going a bit epistemental, so I promised myself yesterday to jot down my thoughts and worries, if for no other reason than for future reference.

One big thing that seems to go through my ponderings like a theme, is the linguistic flow of the definition language itself, in how the mode of definition changes the relative inference of the results of using that ontology over static data (not to mention how it gets even trickier with dynamic data). We usually say that the two main ontological expressions (is_a, has_a) of most triplets (I use the example of triplets / RDF as they are the most common ones, although I use Topic Maps association statements myself) defines a flat world from which we further classify the round world. But how do we do this? We make up statements like this ;

Alex is_a Person
Alex has_a Son

Anyone who works in this field understand what's going on, and that things like "Alex" and "Person" and "Son" are entities, and defined with URIs, so actually they become ;

http://shelter.nu/me.html is_a http://psi.ontopedia.net/Person
http://shelter.nu/me.html has_a http://en.wikipedia.org/wiki/Son

Well, in RDF they do. In Topic Maps we have these as subject identifiers, but pretty much the same deal (except some subtleties I won't go into here). But our work is not done. Even those ontological expressions have their URIs as well, giving us ;

http://shelter.nu/me.html http://shelter.nu/psi/is_a http://psi.ontopedia.net/Person
http://shelter.nu/me.html http://shelter.nu/psi/has_a http://en.wikipedia.org/wiki/Son

Right, so now we got triplets of URIs we can do inferencing over. But there's a few snags. Firstly, a tuple like this is nothing but a set of properties for a non-virtual property and does not function like a proxy (like for instance the Topic Maps Reference Model do), and in transforming between these two forms gives us a lot of ambiguity that quickly becomes a bit of a problem if you're not careful (it can completely render inferencing useless, which is kinda sucky). Now given that most ontological expressions are defined by people, things can get hairy even quicker. People are funny that way.

So I've been thinking about the implications of more ambiguous statement definitions, so instead of saying is_a, what about was_a, will_be_a, can_be_a, is_a_kindof_a? What are the ontological implications of playing around with the language itself like this? It's just another property, and as such will create a different inferred result, but that's the easy answer. The hard answer lies between a formal definition language and the language in which I'm writing this blog post.

We tend to define that "this is_a that", this being the focal point from which our definition flows. So, instead of listing all Persons of the world, we list this one thing who is a Person, and moves on to the next. And for practical reasons, that's the way it must be, especially considering the scope of the Semantic Web itself. But what if this creates bias we do not want?

Alex is_a Person, for sure, but at some point I shall die, and then I change from is_a to a was_a. What implications will this, if any, have on things? Should is_a and was_a be synonyms, antonyms, allegoric of, or projection through? Do we need special ontologies that deal with discrepancies over time, a clean-up mechanism that alters data and sub-sequentially changes queries and results? Because it's one thing to define and use data as is, another completely to deal with an ever changing world, and I see most - if not all - ontology work break when faced with a changing world.

I think I've decided to go with a kind_of ontology (and ontology where there is no defined truth, only an inferred kind-system), for no other reason that it makes cognitive sense to me and hopefully to other people who will be using the ontologies. This resonates with me especially these days as I'm sick on the distinction people make between language and society, that the two are different. They are not. Our languages are just like music; with the ebb and flow, drama and silence that makes words mean different things. By adding the ambiguity of "kind of" instead of truth statements I'm hoping to add a bit of semiotics to the mix.

But I know it won't fix any real problems, because the problem is that we are human, and as humans we're very good at reading between the lines, at being vague, clever with words, and don't need our information to be true in order to live with it. Computers suck at all these things.

This is where I'm having a semi-crisis of belief, where I'm not sure that epistemological thinking will ever get past the stage of basic tinkering with identity in which we create a false world of digital identities to make up for any real identity of things. I'm not sure how we can properly create proxies of identity in a meaningful way, nor in a practical way. If you're with me so far, the problem is that we need to give special attention to every context, something machines simply aren't capable of doing. Even the most kick-ass inferencing machines breaks down under epistemological pressure, and it's starting to bug me. Well, bug me in a philosophical kind of way. (As for mere software development and such, we can get away with a lot of murder)

I'm currently looking into how we can replicate the warm, fuzzy impreciseness of human thinking through cumulative histograms over ontological expressions. I'm hoping that there is a way to create small blobs of "thinking" programs (small software programs or, probably more correctly, script languages) that can work over ontological expressions without the use of formal logic at all (first-order logic, go to hell!) that can be shared, that can learn what data can and can't be trusted to have some truthiness. Here's to hoping.

The next issue is directional linguistics, in how the vectors of knowledge is defined. There's things of importance to what order you gain your knowledge, just like there's great importance in how you sort it. This is mostly ignored, and the data is treated as it's found and entered. I'm not happy with that state of things at all, and I know that if I was taught about axioms before I got sick of math, my understanding of axiomatic value systems would be quite different. Not because I can't sit down now and figure it out, but because I've built a foundation which is hard to re-learn when wrong, hard to break free from. Any foundation sucks in that way, even our brains work this way, making it very hard to un-learn and re-train your brain. Ontological systems are no different; they build up a belief-system which may prove to be wrong further down the line, and I doubt these systems know how to deal with that, nor do the people who use such systems. I'm not happy.

Change is the key to all this, and I don't see many systems designed to cope with change. Well, small changes, for sure, but big, walloping changes? Changes in the fundamentals? Nope, not so much.

We humans can actually deal with humongous change pretty well, even though it may be a painful process to go through. Death, devastation, sickness and other large changes we adapt to. There's the saying, "when you've lost everything, there's nothing more to lose and everything to gain", and it holds remarkably true for the human adventure on this planet (look it up; the Earth is not really all that glad to have us around). But our computer systems can't deal with a CRC failure, little less a hard-drive crash just before tax-time.

There's something about the foundations of our computer systems that are terribly rigid. Now, of course, them being based on bits and bytes and hard-core logic, there's not too much you can do about the underlying stuff (apart from creating quantum machines; they're pretty awesome, and can alter the way we compute far more than the mere efficeny claims tell us) to make it more human. But we can put human genius on top of it. Heck, the ontological paradigm is one such important step in the right direction, but as long as the ontologies are defined in first-order logic and truth-statements, it is not going to work. It's going to break. It's going to suck.

Ok, enough for now. I'm heading for Canberra over the weekend, so see you on the other side, for my next ponder.

29 September 2009

Library Pontifications

Once in a while I get some email from people who ask me some questions or ask me to clarify something I've said in some setting. The other day I ranted on the NGC4LIB (Next-generation catalog 4 libraries) mailing-list about, uh, something or other. And I got email, which I answered, but since I got no reply I'm posting it here in a blog-edited form so that it doesn't go to waste ;
I think I am starting to understand your rants against the culture of MARC, and I'd probably feel offended if I knew what all of the above meant.
Hmm. Well, it wasn't meant to offend anyone. I guess if people thought they were hardcore into persistent identity management, then maybe they would feel I've either overlooked their hard work or don't think what they're doing is the right kind, or something.

I usually have two goals with my "rants"; 1. flush out those who already are on the right track, and make them more vocal and visible, and 2. if no one is on the right track, inspire people in the library world to at least have a look at it. I can do this because I have no vested interest in the library world as such; I cannot lose my library job as I'm not working for a library. :)
Naturally, to feel outside of the mainstream creates a crisis of confidence in one's abilities. What does it mean these days to say that one is a cataloger or that one works in tech services, and is it perceived as a joke for those on the outside? Oh yeah...they still produce cards. What do they know about databases?
Librarians are from the outside an incredible gifted bunch of people who knows what they're doing, they have granted powers outside the realm of normal people (including professionals like software developers, believe it or not), and they know stuff we normal folks don't.

However, having been on the inside you get to glimpse the reality of an underfunded, underprioritized sub-culture of society who knows as little about the "real-world" as normal folks know of the library world. There is a great divide between them, and very little has been done to open up. The blame for this I put squarely on the library world (as the real-world is, well, real and out there) who for many years have demanded a library degree even for software development positions, and when we finally get there we are treated as second-class citizens because we don't have that mark of librarianship that comes from library school. It's a bizarre thing, really, and perhaps the most damaging one you've got, this notion of librarians must have a library degree, as if normal people will never understand the beauty of why a 245 c is needed, or the secret of why shelves must be called stacks, and so on.

One thing that has got me very disillusioned about the library way is philosophy. I deliberately sought out the library as a place to work because I have a few passions mixed with my skills which I thought was a good match, and one of the strongest passions were epistemology. One would think that if there was one institutional string of places that could appreciate the finer details of epistemology, it would be the libraries and the people within. That's what they concern themselves with, no?

Err, no. No, they don't. There's the odd person that ponders how a OCLC number can verify some book's identity, but these are very plain boring questions of database management. Then along came FRBR which does not only dip its toes into epistemology, but outright talks about it! The authors of it clearly had knowledge and wisdom about such things. So, one would think there was hope. Like, when it came out in 1993. That's more than 15 years ago. And people still haven't got it. How much time do you reckon it's going to take, and more importantly, how many years until it's way too late?

But no, RDA comes out of the woodwork and proves once and for all that there is no hope of libraries ever taking the issues at a philosophical nor practical level. Let me explain this one, as it sits at the core of much of my "ranting."

FRBR defines work, expression, manifestation, item, and these are semi-philosophical definitions that we're supposed to attach semantics and knowledge to. There's primarily two ways to do that; define entities of knowledge, or create relationships between entities. (Note these two basic ways of doing knowledge management; entities and relationships, as they spring up in all areas of knowledge representation)

Now, can you without looking stuff up tell me the difference between a work and an expression? Or between manifestation and an item? Sure, we can discuss if this or that thing is an item or something else, back and forth, but is that a good foundation upon to lay all future library philosophy? Because that's just what it is; a philosophical model we use to make sense of the real world. FRBR is confusing, even if it is a great leap forward in epistemological thinking, for example when it comes down to identity management (persistent identifiers for one thing can be expressed through a multitude, like a proxy, which FRBR fails at miserably, for example) it is right there in the centre of it, but a lot of it focuses on the wrong part of it, the part that involves human cognition to make decisions about identity.

Anyway, I guess at this point all I'm trying to say is that there are glimpses of what I'm talking about in the library world, and I was attracted to it, I wanted to dedicate parts of my life to fixing a lot what was broken in the real-world. I came to the library because they are the shining beacon of light in our society.

So, what happened?
Which is why I am interested smarting up about some of these things. Where should one go for a decent but not mind-blowing introduction to the types of things you have described lately?
It's hard to say what will blow your mind, and what will not. But since you're a library type person I'm going to go out on a limb here,and assume you're a smart person. :) So, I'm going to assume that http://en.wikipedia.org/wiki/Epistemology won't blow your mind. So let's assume we're using the definition for "subject" as such ;
  • An area of knowledge, a topic, an area of interest or study
In terms of philosophy we usually expand that definition a bit wider (so it will also include most discourse and literature) but I'll try to keep it simple. First, a question?

"What does it mean that something
is something?"

This is the basic question for identity, that something exists and that we can talk and refer to it. Refering to things is a huge portion of what the library does, not only as an archive, but as a living institution where knowledge is harboured. We're talking about subjects put into systems, about being subject-centric in the way we deal with things. Just like our brains do.

Now, for me there's a few things that have happened the last 20-30 years. The world has become more and more knowledge centric (they've gone from "all knowledge are in books" to "knowledge can be found in many places", and the advent of computers and the internet plays no small part in that), while libraries have become more book specific, more focused on the collection part rather than what the collection actually harbours in terms of knowledge (and I suspect this is because there are no traditional tracks within the library world for technology), probably because it's easier and fits better into budget driven government run institutions.

However, this isn't beneficial to the knowledge management part. Libraries are moving steady towards being archives, but the world wants them to become knowledge specialists. Ouch. And so the libraries will be closed down when they
don't deliver knowledge. Archives is what Google does best, and they're not that bad at harbouring basic knowledge. What hope in hell have you got then?

I'm running out of time right now, but feel free to ask any question and point to any of my wrongs, and laugh at it as well; I need the discourse as much as (I hope) you do. Let me just quickly run through that list with comments and pointers ; [
editors note : this is a list of things I felt the library world 'have no clue about' from my mail to the mailing-list]
  • No idea about digital persistent identification.
What happens to identifiers when people stop maintaining them? They lose their semantic and intrinsic value, and become moot. How many libraries maintain their age old software? No, a more human, less technological means of resolving is needed, and when when the world went digital the choice of multiple identities became not only possible but inevitable. Yet, when the library world manages identities as OCLC / LOC record numbers at the item level, things go horribly wrong and you cannot take what you've defined and learned into the philosophical space. Even if the OCLC / LOC numbers are maintained till the end of the world, they do not solve basic epistemological problems.
  • No subject-centricity.
FRBR does actually provide some, but it is not focused on the epistemological problems, only one of identifying the problem of identification without providing a mechanism (real or philosophical) for doing so.
  • No understanding of semantics in data modeling.
The AARC2 / RDA world is, in some definition of the terms, a data model. And between entities in data models there are semantics, meaning the relationships themselves, their names, roles and thought purpose. But you have to understand, as a human, all of AARC2 / RDA to be able to model anything with it; there's no platform on which to stand, there's no atomic parts you can use to build molecules and then cells and then beings. The whole model is, in fact, a hobbled-together set of fields without structure (and no, numbering them is not a structure :), and without structure there's only rules. And rules without structure is only human-enforceable.
  • No clue about ontologies, inferencing, guides by analogy
This is a stab at what the Semantic Web people are doing. They have a long background from AI and knowledge management, and if you guys were at least on par with that group, there could be some better understanding of the issues. The SemWeb crowd understand a lot of first-order logic, inferencing, analogy, case-based reasoning, and so forth, all stuff you need to have computers understand a tad bit better how your data is hobbled together, how they all interact, how entities and relationships (remember those? :) are mapped.

I should of course make a note here that I think that the SemWeb efforts are mostly wrong, and that they could learn an awful lot from librarians in the way to deal with collections and access, but that's a different discourse for some other time. :)
  • no real knowledge about collection management ( ... wait for it ...) with multiple hooks and identities
I was actually hoping people would jump on this one, getting offended that I said they had no real knowledge of collection management (which is their forte, it is what they do!), but I guess either they saw the hook and line of *identities*, and jumped over it. Dang.

It's all about the identity of what you are collecting. Crikey, publishers haven't even got ISBN to work (how many times to I put in one ISBN to get a completely different book ...), and one would think that would provide hints to why this is hard, and perhaps what to do otherwise. Hmm.

-- end of mail except some more personal ramblings not fit for generic consumption --

24 August 2009

What event model ontology?

Hmm, it seems that no one has blogged, tweeted or mentioned my blog post in my last plea, which I'm quite disappointed with. However, I'll chalk this one down to the complexity of what I'm trying to accomplish, and my failed attempt at explaining what it is.

In the mean time I've been working at it, converging various models from all sorts of weird places (anything from WebServices and SOAP stacks, to operating systems like Linux, to event models in Java and .Net, to more conceptual stuff in the Semantic Web world), but boy, you can tell that we live in a world shaped by iterative imperative paradigms of approaching the software world.

One thing I learned quite early was declarative and functional programming, introduced to me, of all places, with using XSLT many years ago. It may not be the most obvious place to find it, and this is one of those hidden gems of the language which still doesn't enjoy too much of a following. And no wonder; people come into it from the imperative stuff that dominates the world, polluting us all with filthy thoughts of changing variables (at least in Scala you can choose between var and val), functions that aren't truly functional, and the classical idea in object-oriented programming of a taxonomical structure that doesn't hold up to scrutiny.

Let me clarify that last point. Wht are we doing this stuff? Why are we creating computer programs?

To solve problems. And who are we solving problems for? For humans. It's the classical example (albeit extrapolated) of garbage in, garbage out. I've talked about this in the past a lot, about the constant translation that happens between huna and machine, and how we are creating translation models in both worlds in order to "move forward" and solve problems better. But this excercise becomes increasingly harder as our legacy grows, so trying to teach functional programming to people who don't understand certain basic principles of Lambda Calculus is going to be hard, just like it's hard to teach Topic Maps to people who live in a SQL world. Or like it's hard to teach auto-generating user-interfaces to a user-interface developer.

These are usually called paradigm shifts, where some important part of your existing world is totally changed as you learn some other even more important knowledge. You must shift your thinking from one way to a rather different other. And this is hard. Patterns of knowledge in your brain is maintained by traversing certain paths often, and as such strengthening that path (following the pattern that an often travelled path must be the right path). But if the path is wrong, there's some pretty strong paths you need to unlearn. Damn, that is hard! Which is why I urge you to try it out.

I'm currently using Topic Maps, human behaviour driven ontologies for auto-generating applications and user-interfaces over functional complete models of both virtual and concrete human domains, all with temporality and continous change as the central paradigms. Yeah, pretty hefty stuff, and I've spent years trying to unlearn stuff I learnt in the years before that. And those years were unlearning some other stuff before that. My whole life has been one huge unlearning experience, and I don't think any other way conceptually grasps the beauty of life better; nature and life both are in perpetual change. Needless to say, I'm enjoying every single crazy second of it!

But back to my event model ontology. I've learned one important thing in all this; Sowa has suggested a shift from logical inference to analogy, and this coupled with the OODA loop can create an intriguing platform for knowledge management and eco-system forsoftware applications. I'll let you know more as things progress from here. I'm excited!

And as always, I'd love to hear your comments on all of this. I beg you. Again. :)

5 August 2009

Can I ask you a favour? (Does social media actually work?)

Hi everybody. Could I ask you a favour? I'm not getting much response to my quest for a unified software architecture ontology, so could I humbly ask you to blog, tag, link or otherwise gossip about my previous post on the matter? I would really appreciate it, and I promise I'll share my findings with you all.

(My subtitle "Does social media actually work?" is a blatant attempt to get circulation going by mocking the whole debacle which I try to, ahem, you know, promote. Thanks.)

15 May 2009

Spilling a few beans

I think enough time has passed, don't you? I've been hinting to what I'm up to these days, but I've been rather careful about spilling the beans, I guess because, well, it's a brand new adventure and every storyteller should get their story together well before writing it down. I'm keen to talk about this stuff, though, because it is wickedly cool and I'm keen to not only do it, but to talk about it and involve more people in it as well.

As you probably saw from my last post I'm currently in India, and yes, my new employer is an Indian company, but I work from home (in gorgeous Kiama, Australia, 1.5 hours south of Sydney) and travel to India every so often (4-5 times a year as a rough guide). We work over the Internet, including video conferencing and remote controlling and the like, and as such is a new interesting challenge for me to be somewhat isolated from the smiles and sideways nods and the tacit knowledge floating down the hallways of our headquarters in Mumbai. I've got plenty of ideas of how to deal with that, so we'll see how it goes.

My company is Free Systems Technology Labs, a nifty medium-sized IT development company with main offices in Mumbai (from where I'm writing this) and most R&D and development in Bangalore (where I've been the last week), which is a daughter-company of another company mostly known for more hardware orientated stuff, like computer building, server hosting and various gadgets, but they only have a number of software outlets as well. I'll be working with anything from planning to execution, and mostly in the domain of Topic Maps. Yes, the very thing I've been talking about for the last 9 years is now going to be my main concern as opposed to secondary or third (or some periods not at all) at the whims of other jobs, and I can't even begin to tell you how excited the prospect of that is to me; I believe in the ideals and practice of Topic Maps so strongly, and it's going to be good for my soul to pour it into something as cool as what we're going to do. (More on that later) The guys here also happen to share many of my own ideals (open-source, development methods, goals, community and societal building, and so much more), and they've been spoiling me. I'll miss the tea, that's for sure.

I became part of this through a weird mix of happenstance, but mostly because the people involved here have been, put simply, a fantastic bunch, in terms of technical brilliance, sincerity and honesty, and in convincing me that I should join (they obviously think I'm good for something :). I've been with the company now almost three months where the first two months are more like a warm-up, but it's been a very good ride so far.

But I need to talk about something that's been on my mind ever since they got in touch with me last year, and that's prejudice. The world is full of it, and I entered this adventure with a slight degree of scepticism. No, not the bad kind, but a certain carefulness, because, you know, they're Indians, and Indians got their mouth full of rice, and you're not getting any! (A joke I got from an Indian friend, so that makes it alright, yeah? :) Not only did they have to convince me, but also my wife. "Honey, how about I drop my great-paying safe cushy job in one of the richest countries in the world, and rather work for strangers from a strange land full of poverty and strong smells and interesting hairdoos, and do it over internet?" Yeah, she was keen, as you can imagine.

You can't work for Indians! They are supposed to work for us!

Sure. But they kept talking with me, flew me to Belgium (they own half of a company there) and were not only completely honest with me but simply blew me away with their knowledge, seriousness, and most importantly their friendliness and openness. Me and the wife thought long and hard about it (probably longer and harder than my company wanted me to :), and here we are.

Everything I knew about India was either heavily adjusted, or simply wrong, but I've seriously enjoyed being corrected. I've embraced everything that's been thrown at me, including very hot food, weird drinks, amazingly crazy traffic and the sweltering heat, the chaos, the smells, the meetings and the way they interact, the attitudes and the values. I think the tagline "Incredible India" is truer than they think.

Ok, that's enough for a first intro, now I have to get to bed. I'm flying home tomorrow and I'm looking forward to seeing the wife and kids again (Lilje just won an award for her art at school, so I'm mighty proud as well), and we'll be spending the weekend together, and on sunday celebrate Norways national day in Sydney.

And then, a little bit later, I'll tell you about the wickedly cool stuff we're going to do with Topic Maps.

9 May 2009

Where in the world is Alexander?

Short answer; Bangalore, India.

Longer answer; my new employer which I started with a couple of months back is an Indian company with strong ties to back-end systems and support, hardware manufacture and design, and software services. I'll tell more as things progress, and I'll probably talk a lot more about how they plan to use Topic Maps to solve some really crazy and hard problems. But before I do those kind of detailed stuff, I wanted to just quickly show you this picture which pretty much summarises my first impression of this crazy, lively, contrasting, weird, interesting place, and if you can't read the sign, it says "Follow traffic rules." I realise that in India, if you ask kindly, they just might do what you ask, but riding as a passenger in a car through this traffic was, err, an experience I won't forget anytime soon. However, it's interesting that in a language such as my own (English, or Norwegian, or Swedish, or Danish) we base our expression mostly on words alone, while in India the reason traffic works is that they've got such a strong foothold in semiotics that makes it work. A honk here, two honks there as we pass a car, a blink of our beam lights racing past a "moto" (small scooter that's kinda rebuilt as a tiny car) ... I still have much to learn about this language. The cool thing is that it's global; even I can do it. Except I would never drive here. Never. Ever.

Anyway, I'm in India to train staff and meet and plan with them in all things black magic and drink their excellent Indian tea and eat their amazing food, and generally get a feel for the country, the culture, and most importantly, the people I'm working with, which so far has turned out to be a fantastic bunch. I'm here for another week or so, and I'll suss out the details and let you know all about it in due time. Until then, there's a chapati drenched in yummy chutney with my name on it. India is, truly, an amazing place.

P.S. Hey Barta, where can I get my sweaty hands on your TM as a filesystem? Would love a play with it right about now. Oh, and that near NLP query stuff you mentioned that one time in the back-alley while drinking gin and discussing the meaning of wife. Or life. Or whatever.