Knee-deep in SOA

Lately at work I've been knee-deep in SOA; Service-Oriented Architecture, a concept that for me has roots in web services but extends beyond that in that it has made my holistic thinking possible.

There is more to application design than to solve the problem at hand, far beyond the scope of any requirements document; it's more about supporting the infrastructure than whatever else you think you're doing. Maybe this requires some explanation;

Business Analysist create requirement documents to solve some business problem. However, unless that business analyst is especially sharp and holistic, there are so many undercurrants, twists and turns to the final solution that more often than not we shouldn't even attempt to solve problems. A lot of places employ Solution / systems / information Architects to try to rectify this problem, but often it simply ain't enough; detachement from technical solutions seems to be a huge problem for understanding business problems.

A lot of us are technically inclined people; we try to take the business requirements and make technical solutions of them. Anyone with a speck of experience in these areas know how dreadfully wrong it can go, and we say "oh, you need to employ the right people to make it work." Most of the time, that's true; with really super people these things will work much better. How often do we have the luxury of only working with top-dogs? I'll leave that an open question, of course.

Enter the SOA; think of your business requirements as services, tiny and disjointed or large and intertwined, and open up Interfaces to them, and what happens? Well, notice that bold word; interface. Not application. Not program. Not even requirement. It's a service, a service that programs, applications and requirements can use to solve their problems.

We all know web services by now; most of the time it means either a SOAP or REST call where XML is the carrier of various bits of information. Because of the openness of these technologies we can quickly cook up various other applications and programs from a smørgåsbord (yeah, look it up) of services that might address your issues or wants. They are completely disjointed from the applications that use them, meaning a clear separation of business logic, application frameworks and user-interfaces. If played correctly, it can have an amazing synergetic effect on everything you do.

If you're a geek like me, the prospect of this is great, but over the next little while I'd like to talk about all those things it affects in better business management, usability, information architecture, user-interface design, application design, application scalability and performance enhancements, and more.

To sign off though, I'd like to talk a little about what I've done so far. First, I've created a hug from which all web services come, something like http://ws.example.com/ which works as either an application context for your servlets/scripts/etc that are plain web services, or it works as a proxy for external services. This hub has a wrapper so that it doesn't matter if you want to use SOAP or REST or even partly RSS/Atom feeds of stuff.

Next, put some good services in there, like user-authentication, and you're half-way there to the single-sign-on pipe-dream. I've implemented it with some thesaurus services, authentication, an OPAC service and a wrapper for Amazon.com.

I can now, through PHP, create a completely new application to a few services that were written in Java and Perl. I can pass bits of information in to our OPAC and do more complex searches. As a test, we recently create a Lucene database prototype of about 11.4 million MARC records. The user-interface looked terrible, but through the SOA hub we split the requests in two (at random, fire your web service request at two different servers for load balancing), took the first record from the service and fed it to another lookup-service, fed the subject headlines from this request to the thesaurus, did a third search in the Lucene prototype for the subjects that had thesaurus entries, and Voila! we could present a new application with a good user-interface, all in two days from start to finish. And we really didn't break a sweat, either.

That's what I want to talk about; what happens once you've got a basic SOA in place; synergy!