The Library World and their lack of opinions

There's a mystical place out there, a place drenched in magic, adventure and fantastic experiences, where your soul and senses meets the challenge of the human spirit. No, not the church nor the congressional house, but the library.

I love this place. I love this culture. But it needs your help, because it is dying. There's many reasons for why this is so, but in my mind the two main reasons are that a) the book as a medium ain't good enough as the world hits a certain complexity, and b) librarians still think that the library is mostly a place to find books on shelves.

As much as I could go on at length on both these things - and I probably will in the near future - right now I'd like to tell you a secret. It's not a big thing and I might be wrong, but I've observed this little professional quirk the librarians embrace quite vigorously;

Librarians have no opinions.

Well, obviously not true as many of them have lots and great opinions which they've shared with me on a number of occasions. But when you ask them for information they will not tell you what book on the subject is the best, because, you know, we're human, you have to find out that piece of opinion for yourself. Librarians are not supposed to say things like "you'd enjoy that book", only (if they're big risk-takers and on the edge of library society) "I enjoyed this book," a doctrine that has a long and proud history in the library world. I know this is very academic and proper, but nowhere have I seen it so strong as in the library world.

I'm an avid library user. I've always loved libraries, and especially when I'm looking for some specific piece of information it's the number one place to go. And as such heavy librarian use I've become pretty good at telling good from bad. Let's define good as "getting you what you wanted or something even better" and bad as "getting something in the ballpark of what you wanted, with a feeling that there surely must be more?" These definitions work just as well with literature as with history and with science or any other reason you go to the library.

I think you can guess by now what I'm getting at; the good library experience comes through librarians who dare to challenge the stronghold of "no opinion." When the librarian daringly points out that I might enjoy this book by some other author that what I was looking at, that's when the magic happens! Serendipity!

Good librarians know to break this "no opinion" guideline. Good librarians know how to create magic and adventure. And I think good librarians know that this is their biggest and most wonderful weapon in the battle for the library of the future.

My good friend (who I miss dearly) Bobby "Slobo" Graham from the National Library of Australia kept telling me of a saying of sorts that I can't recall the origins of;

"Serendipity is when you go to the library to take out a book, and end up taking out the librarian."

At the library I've smiled, I've cried, I've danced, struggled, had love, made philosophy, drank the kool-aid and smelt victory. You should also. And tell those risque librarians that you love them. I know I do.

Why I trust Google

Low blog season for meg as life swooshes by and I'm preparing for the time that has already gone. But I do trust Google, and here's why.

Oh waily, waily me!

Ok, summer is over, and I'm back at work, the household is busy with their various stuff (night-school, school, kindergarten, mini-kindergarten, from oldest to youngest). We're an extremely busy little flock, I dare say.

However, amidst all this family busy life there is a slow, slow, slow realization of the self that is going on. I'm getting old. And I'm not working with what I want to do right now. Seriously, if you're going to have a super-busy life, at least I would like it to be busy with stuff I want to do. The family part is dandy. No, it's the work part that keeps me up at night.

The company I work for is great. Seriously. Really great. It's not their fault, it's mine. You see, I think a lot, and a lot of my thinking is about stuff that my work really don't do much of. I have dreams, passions and aspirations that travels well out of scope for the IT consultancy gig I do every day.

Like, building the most energy efficient, low-maintenance wave-based hydro-electro generator ever. Or the non-maintainence, micro-inducing, piggy-backed salt-to-freshwater plant ever thought of. Or the zero-impact house water system (gray and white at the same time). Or design the coolest child-friendly gadget that my daughter cries about at night for not getting. Or change organization processes and communication from bad to great. Or help people realize their own potential. Or, you know, make people happier.

I'm an ideas guy. I shouldn't really be a boring consultant, at least not for customers who wants me to do exactly as they say. I'm into innovation, into doing new and exciting (but not stupid) things, bordering on crazy but never go as far as sick. I love to work with people to come up with new stuff, or refine old stuff. Fix problems. Solve the impossible. Free Willy. Save the planet. Create art.

I think a lot, and I'm getting older.

...

Ok, back to tuning my Tomcat context to understand proper REST patterns without interfering too much, and then fine-tune some Topic Maps optimizations in my RDBMS. *sigh* *pining for the fjords*

Holiday blues

Sam's got the blues ...



... and we'll be back to "normal" next week. Hope you've all had a great time. We went to Denmark for a quickie, and to the mountains of Trysil (Norway) for a weeks cabin-fever fun. We've had a good time, but it's good to get back, too. Now, what was I doing again?

Round and round it goes

This morning was a good one. I got on the bus, armed with breakfast banana in hand, and right there in front of me sat fellow Topic Mapper Stian Danenbarger (from Bouvet), who happened to be living just literally down the road from me. I've been living at Korsvoll (in Oslo) for 6 months now without bumping into him, how odd is that?

Anyways, the last few days I've written about Language and Semantics and about context for understanding communication (all with strong relations to programming languages), and needless to say this became the topic (heh) of discussion on the bus this morning as well.

In this post I'll try to summarize the discussion so far, implement the discussion I had on the bus this morning, coupled with a discussion I've had with Reginald Braithwaite on his blog, from "My mixed feelings about Ruby". Let's start with Reginald and move backwards.

Background

Matz has said that Ruby is an attempt to solve the problem of making programmers happy. So maybe we aren’t happy with some of the accidental complexity. But can we be happy overall? Can we find a way to program in harmony with Ruby rather than trying to Greenspun it into Lisp?

I think that the goal of making programmers happy is a good one, although I suspect there's more than one way to please a programmer. One way is perhaps rooted in the syntax of the language at hand. Then there's the semantics of your language keywords. Another is to have good APIs to work with. Another is how meta the language is (i.e. how much freedom the programmer has in changing the semantics of the language, where Lisp is very meta while Java is not at all), and yet another is the community around it. Or the type and amount of documentation. Or its run-time environment. Or how the code is run (interpreted? compiled? half-compiled to bytecodes?).

Can we find ways in programming that would make all programmers happy? I need to now point back to my first post about Language and Semantics and simply reiterate that there's a tremendous lack of focus on why we program in most modern programming languages. Their idea is to shift bits around, and seldom to satisfy some overal more abstract problem. So for me it becomes more important to convey semantics (i.e. meaning) through my programming more than just having the ability to do so. Most languages will solve any problem you have, so what does the different languages offer us? In fact, how different are they most of the time?

At this moment in time I have extremely mixed feelings about Ruby. I sorely miss the elegance and purity of languages like Scheme and Smalltalk. But at the same time, I am trying to keep my mind open to some of the ways in which Ruby is a great programming language.

I think we really agree here. My own experiences with over 8 years of professional XSLT development (yes, look it up :) has taught me some valuable lessons about how elegant functional programming can be, just like Lisp and the mix-a-lot SmallTalk (which I like less of the two). But then I like certain ways that Ruby does things too, with a better syntax for one. I like to bicker about syntax. Yeah, I'm one of those. And I think I bicker about syntax for very good reasons, too;

Context

In "just enough to make some sense" I talk about context; how many hints do we need to provide in order to communicate well? Make no mistake; when we program, we are doing more than solving the shifting of bits and bytes back and forth. We are giving hints to 1) a computer to run the code, and 2) the programmer (either the original developer, or someone else looking at her code). Most arguments about syntax seems to stem from 1) in which 2) becomes a personal opinion of individuals rather than a communal excericse. In other words, syntax seems to come from some human designer trying to express hints best to the computer in order to shift bits about, instead of focusing entirly on their programming brothers and sisters.

In the first quote about Ruby being designed in order to please the programmer, that would imply that 2) was in focus, but the focus of that quoted statemement is all wrong; it pleases some programmers, but certainly not all, otherwise why are we even talking about this stuff?

Ok, we're ready to move on to the crux of the matter, I think.

I am arguing that while it is easy to agree that languages ought to facilitate writing readable programs, it is not easy to derive any tangible heuristics for language design from this extremely motherhood and apple pie sentiment.

Readability is an important and strong word. And it is very important, indeed. We need everything to be readable, from syntax to APIs to environments and onwards. I think we all want this pipe-dream, but we all see different ways of accomplishing it. Some say it's impossible, others say it's easy, while people like Reginald I think is right there in the middle, the ultimate pragmatic stance. And if I had never done Topic Maps I would be right there with him. Like Stian Danenberger said this morning, there's more to readability than just reading the code well.

Topic Maps

Yeah, it's time talk about what happens when you drink the kool-aid and you accept the paradigm shift that comes with it. There's mainly x things I've learned through Topic Maps;

• Everything is a model, from the business ideals and processes, to design and definition, our programming languages, our databases, the interaction against our systems, and the human aspect of business and customers. Models, models, everywhere ...
  
• All we want to do is to work with models, and be able to change those models at will
  
• All programming is to satisfy recreating those models

Have you ever looked at model-driven architecture or domain-driven design? These are somewhat abstract principles to creating complex systems. Now, I'm not going to delve into the pros and cons of these approaches, but merely point out that they were "invented" out from a need that programming languages didn't solve, namely the focus on models.

Think about it; in every aspect of our programming life, all we do is trying to capture models which somehow mimics the real-life problem-space. The shifting of bits wouldn't be necessary if there wasn't a model were working towards. We create abstract models of programming that we use in order to translate between us humans and those pesky computers who's not smart enough to understand "buy cheap, sell expensive" as a command. This is the main purpose of our jobs - to make models that translate human problems into computer-speak - and then we choose our programming language to do this in. In other words, the direction is not language first then the problem, but the other way around. In my first post in this series I talked about tools, and about choosing the "right tool for the job." This is a good moment to lament some of what I see are the real problems of modern programming languages.

What objects?

Object-oriented programming. Now, don't get me wrong, I think OOP is a huge improvement over the process-oriented imperative ways of the olden ways. But as I said in my last post, it looks so much like the truth, we mistakenly treat it as truth. The truth is there's something fundamentally wrong with what we know as object-oriented programming.

First of all, it's not labeled right. Stian Danenbarger mention that someone (can't remember the name; Morten someone?) said it should be called "Class-based programming", or - if you know the Linnean world - taxonomical programming. If you know about RDF and the Semantic Web, it too is based loosely on recursive key/value pairs, creating those tree-structures as the operative model. This is dangerously deceitful, as I've written about in my two previous posts. The world is not a tree-structure, but a mix of trees, graphs and vectors, with some semi-ordered chaos thrown in.

Every single programming approach, be it a language or a paradigm like OOP or functional, comes with its own meta model of how to translate between computers and the humans that use them. Every single approach is an attempt to recreate those models, to make it efficient and user-friendly to use and reuse those models, and make it easy to change the models, remove the models, make new ones, add others, mix them, and so on. My last post goes into much detail about what those meta models are, and those meta models define the communication from human to computer to human to computer to human, and on and on and on.

It's a bit of a puzzle, then, why our programming languages focus less on the models and more on shifting those bits around. When shifting bits are the modus operandi and we leave the models in the hands of programmers who normally don't think too much about those models (and, perhaps by inference, programmers who don't think about those models goes on to design programming languages in which they want to shift bits around ...), you end up with some odd models, which at most times are incompatible with each other. This is how all models are shifted to the API level.

Everyone who has ever designed an API knows how hard it can be. Most of the time you start in one corner of your API thinking it's going smooth until you meet with the other end, and you hack and polish your API as best you can, and release version 1.0. If anyone but you use that API, how long until requests for change, bugs, "wouldn't it make more sense to ...", "What do you mean by 'construct objects' here?", and on and on and on. Creating APIs is a test of all the skills you've got. And all of the same can be said about creating a programming language.

Could the problem simply be that we're using a taxonomic programming language paradigm in which we try to create a graph structured application? I like to think so. Why isn't there native support in languages for typed objects, the most basic building block of categorisation and graphing?

$mice = all objects of type 'mouse' ;

Or cleanups?

set free $mice of type 'lab' ;

Or relationships (with implicit cardinality)?

with $mice of type ('woodland')
add relationship 'is food' to objects of type 'owl' ;

Or prowling?

with $mice that has relationship to objects of type ('owl')
add type ('owl food') ;

Or workflow models?

in $workflow at option ('is milk fresh?') add possible response ('maybe')
with task ('smell it') and path back to parent ;

[disclaimer : these are all tounge-in-cheek examples]

I know you can extend some languages to do the basic bidding here, for example in JavaScript I can change the prototype for basic objects and types, but it's an extension each programmer must make and the syntax is bound to the limits of the meta model of the language, amking most such extensions look kludgy and inelegant. And unless they know all the problems that I think we've been talking about here, they really won't do this. This sort of discussion certainly does not appear where people learn programming skills.

No, most programming languages follow the tree-structure quite faithfully, or more precise the taxomatic model (which is mostly trees but with the odd jump (relationship) sideways in order to deal with the kludges that didn't fit the tree). Our programs are exactly that; data and code, and the programming languages define not only the syntax for how to deal with the data and code, but the very way we think about dealing with blobs of data and code.

They define the readability of our programs. So, Reginald closes;

Again we come down to this: readability is a property of programs, and the influence of a language on the readability of the programs is indirect. That does not mean the language doesn't matter, but it does make me suspicious of the argument that we can look at one language and say it produces readable programs and look at another language and say it does not.

Agreed, except I think most of the languages we do discuss are all forged over the same OOP and functional anvil, in the same "shifting the bits and byes back and forth" kind of thinking. I think we need to think in terms of the reason we program; those pesky models. Therein lies the key to readability, when the code resembles the models we are trying to recreate.

Syntax for shifting bits around

Yes, syntax is perhaps more important than we like to admit. Syntax defines the nitty-gritty way we shift those bits around in order to accomplish those modeling ideals. It's all in the eyes of the beholder, of course, just like every programming language meta model have their own answer. What is the general consensus on good syntax that convey the right amount of semantics in order for us all to agree to its meaning?

There's certain things which seems to be agreed on. Using angle brackets and the equal sign for comparators of basic types, for example, or using colon and equal to assign values (although there's a 50/50 on that one), using curly brackets to denote blocks (but not closures), using square brackets for arrays or lists (but not in functional languages), using parenthesis for functional lists, certain keywords such as const for constants, var for variables (mostly loosly typed languages, for some reason) or int or Int for integers (basic types or basic type classes), and so on. But does any of this really matter?

As shifting bytes around, I'd say they don't matter. What matters is why they're shifting the bytes around. And most languages don't care about that. And so I don't care about the syntax or the language quirks of inner closures when inner closures are a symptom of us using the wrong tools for the modeling job at hand. We're bickering about how to best do it wrong instead of focusing on doing it right. Um, IMHO, of course, but that's just the Topic Maps drugs talking.

Just like Robert Barta (who I'd wish would come to dinner more often), I too dream of a Topic Maps (or graph based) programming language. Maybe it's time to dream one up. :)

Just enough to make some sense

I've realized that my previous post on language and semantics could possibly be a bit hard to understand without having the proper context wrapped around it, so today I'll continue my journey of explaining life, universe and everything. Today I want to talk about "just enough complexity for understanding, but not more."

Mouses

Let's talk about mouse. Or a mouse. Mice. Let's talk about this ;

One can argue whether this is really enough context for us to talk about this thing. What does "mouse" mean here? The Disney mouse? A computer mouse? The mouse shadow in the second moon? In order for me to communicate clearly with my fellow human beings I need to provide just enough information so that we can figure this out, so I say "mouse, you know the furry, multivorus, small critter that ..." ;


This is too much information, at least for most cases. I'm not trying to give you all the information I know about mice, but just enough for me to say "I saw a mouse yesterday in the pantry." Talking about context is incredibly hard, because, frankly, what does context mean? And how much background information do I need to provide to you in order for you to understand what I'm talking about?

In terms of language "context" means verbal context as words and expressions that surrounds a word, and social context as the connection between the words and those who hear or read them based on the human constraints (age, gender, knowledge, etc.) There's also some controversy about this, and we often also imply certain mental models (social context of understanding).

In general, though, we talk about context as "that stuff that surrounds the issue", from solid objects, ideas, my mental state, what I see, what I know, what my audience see and knows, hears, smells, cultural and political history, musical tastes, and on and on and on. Everything in the moment and everything in the past in order to understand the current communication that takes us to the future.

Yup, it's pretty big and heady stuff, and it's a darn interesting question; how much context do you need in order to communicate well? My previous post was indeed about how much context we need to put into our language and definition in order to communicate well.

A bit of background

Back in 1956 a paper by the cognitive psychologist George A. Miller changed a lot of how we think about our own capacity for juggling stuff in our heads. It's a most famous paper, where further research since has added to and confirmed the basic premise that there's only so much we're able to remember at the same time. And the figure that came up was 7, plus / minus 2.

Of course that number is specific to that research, and may mean very little in the scheme of more specific settings. It's a general rule, though, that hints to the limits we have in cognition, in the way we observe and respond to communication. And it certainly helps us understand the way we deal with context. Context can be overly complex, or overly simple. Maybe the right amount of context is 7, plus / minus 2?

Just right



I'm not going to speculate much in what it means that "between 5 and 9 equally-weighted error-less choices" defines arbitrary constraints on our mental storage capacity (short-term especially), but I'll for sure speculate that it guides the way we can understand context, and perhaps especially where it's loosely defined.

We humans have a tendency to think that those things that looks like the truth must be the truth. We do this perhaps especially in the way we deal with computer systems, because, frankly, it's easy to define structures and limitations there. It's what we do.

An example of this is how we observe anything as containers that may contain things, that in themselves might be containers which might be things or more containers, and so on. Our world is filld with this notion, from taxonomies, to object-oriented programming, to XML, to how we talk bout structures and things, to how science was defined, and on and on and on. Tree-structures, basically.

But as anyone with a decent taxonomic background knows, taxonomies don't always work as a strict tree-structure. Neither does anyone who's meddled in OO for too long. Or fiddled with XML until the angle-brackets break. These things looks so much like the truth that we pursue them as truth.

things are more chaotic than we like. They're more, in fact, like graph structures, where relationships between things go back and forth, up and down, over and under already established relationships. It can be quite tricky, because the simple "this container contains these containers" mentality is gone, and a more complex model appears;


This is the world of the Semantic Web and Topic Maps, of course, and many of the reasons why these emerging technologies are, er, emerging is of course because all containers aren't containers at all, and that the semantics of "this things belongs to that thing" isn't precise enough when we want to communicate well. Explaining the world in terms of tree-structures puts too many constraints on us, so many that we spend most our time trying to fit our communication into it rather than simply defining them.

We could go back to frames theory as well, with recursive key/value properties that you find naturally in b-trees, where values are either a literal, or another property. RDF is based on this model, for example, where the recursiveness is used for creating graph structures. (Which is one reason I hate RDF, using anonymous nodes for literals)

Programming languages and meta models

Programming languages don't extend the basic pre-defined model of the language much. Some languages allow some degree of flexibility (such as Ruby, Lisp and Python), some offer tweaking (such as PHP. Lua and Perl), while others offer macroing and overloading of syntax (mostly C family), and yet more are just stuck in their modeling ways (Java). [note: don't take these notions too strictly; there's a host of features to these languages that mix and match various terms, both within and outside of the OO paradigm]

What they all have in common is that the defined meta model is linked to shifting bits and bytes around a computer program, and that all human communication and / or understanding is left in the hands of programmers. Let's talk about meta models.

Most programming languages have a set of keywords and syntax that make up a model of programming. this is the meta model; it's a foundation of a language, a set of things in which you build your programs on. All programming languages have more or less of them, and the more they have, the stricter they usually are as well. Some are object oriented languages, other functional, some imperative, and yet other mixes things up. If I write ;

Int i = new Int ( 34) ;

in Java, there's only so many ways to interpret that. It's basically an instance of the Integer class, that holds the integer number of 34. But what about

$i = new Int ( 34 ) ;

in PHP? There is no built-in class called Int in PHP, so this code either fails or produce an instance of some class called Int, but we do not know what that means, at least not at this point. And this is what the meta model defines; built-in types, classes, APIs and the overall framework, how things are glued together.

As such, Java and .Net has huge meta models defined, so huge that you can spend your whole career in just one part of it. PHP has a medium meta model, Perl even smaller, all the way down to assembler with a rather puny meta model. Syntax and keywords is not just how we program, but they define the constraints of our language. There's things that's easy and hard in every language, and there is no one answer to what the best programming language is. They all do things differently.

The object-oriented ways of Java differ to the ones of Ruby which differs to the ways of C++ which differs to the ways of PHP. The functional ways of Erlang differs to XSLT which differs to Lisp.

The right answer?

There is no right answer. One can always argue about the little differences between all thse meta models, and we do, all the time. We bicker about operator overloading, about whether mutliple inheritance is better than single inheritance, one the real difference between interfaces and abstract classes, about getter and setter methods (or lack thereof), about types should be first class objects or not, about what closures are, wheter to use curly-brackets or define programming structure through whitespace, and on and on and on.

My previous post was another way of saying that we perhaps should argue less about the meta model of our language, and worry more about the reason the computer was created more than how a certain problem was solved? We don't have the mental capacity to juggle too much stuff around in our brains, and if the meta model is huge, our ability to focus on perhaps the important bits become less.

There are so many levels of communication in our development stack. Maybe we should introduce a more semantically sane model into it to move a few steps closer to the real problem, the communication between man and machine? I'm not convinced that OO nor functional programming solves the human communication problem. let's speculate and draw sketches on napkins.

Language, semantics and software development

I just read Robert Barta's hilarious post on some of Java's (the programming language) antics, which, for those who's following from home, I agree with wholeheartedly. His post triggered a monologue in my head as I was going out for coffee today, and I felt the need to jot down what I think is my thoughts on languages, semantics and semiotics from that inner tirade.

No, this won't be on programming languages, at least not only. Language permutes every aspect of our human evolvement, and has been one of the main factors for our success for the last 100 000 years or more, and lays down the foundations for that one thing which everything else spins around; communication.

We communicate. We all do it, all the time, every day. In fact, apart from physically mundane things such as eating, resting and procreating, it is all we do.

But as much as we do it all the time, and as such you would expect us to get better at it, we need more than just communication, but the right kind of communication. Anything can communicate, but anything can't always communicate well. A dog barks at me, but I'm not always sure what that bark means (although the drool and glistening fangs provide valuable hints). Good communication is how to get the message across without ambiguity.

Developing software

Let's take a rather common problem; how do we develop software? Well, it's actually quite easy. Just open whatever tool you need, start writing code, hit "compile", and voila! The trick is of course not to develop software, but to develop software well, to remove the ambiguity between the tools we use and the customer who asked for it. And just like with communication, we expect that if we do this often and repeatedly, we learn something along the way, and get better at it.

But developing software still sucks. I've been in this business for almost 20 years working in more than 7 different countries, and every place had the exact same problem; how do we get the communication right?

There's an absolutely stunning focus amongst people in my line of work on the technological parts of the solutions we create. Of course we need those technological bits to make stuff happen, but most of the time they drive the solution more than the other way around. There's a sensible saying that says "choose the right tool for the job", but it seems to me that very few actually stop and ponder that very good piece of advice. What does it mean to choose the "right tool for the job"?

The job

There's a few indicators to the problem above. First of all, what is "the job"?

Our ideal target is a working functionally-complete system that solves some loosely defined problem. As professionals we put a lot of methodology and effort into translating that napkin sketch into documentation fit for enterprise development, which in turn will be translated into programming code, which in turn a GUI will translate to the users of the system. We can say with great confidence that "the job" is pretty much a translation job.

How many levels of translation can we find in our daily work? Let's see, first there's Bob who has an idea (level 1) who talks to an agreeing Pete (level 2), they talk to John (level 3) who agrees (level 4) who writes a mail (level 5) to the marketing manager (level 6) who brings that to a meeting (level 7) to get agreement (level 8). Then John takes the revised document (level 9) to the CEO (level 10) who agrees (level 11) and gives a go, to which the document is sent to Sam who will write a spec (level 12) which Bob, Pete, John, CEO, Marketing boss and Sam all agree on matches the original idea (level 13). Then the spec is sent to the development project manager Jim (level 14) for scheduling, and then to the development team (level 15) for estimation and thoughts (level 16). they come up with estimations and suggestions (level 17), there's a bit back and forth between them and marketing (level 18 through 22), before Jill the functional designer is given the roughly agreed upon documents (level 23) which she alters (level 24) and the leadership team agrees with (level 25). The document now goes back to the development team approved (level 26), and entered their schedule. Development starts with visual design sketches by Pop (level 27), which needs approval (level 28 through 34), which Jim further plans (level 35), and then each developer gets a slice of the new plan (level 36 through 40). A couple of iterations go by (level 41 through 46), and we're ready for development release 1 (level 47) which needs to be agreed upon by all bosses involved (level 48), with tweaks and continued development iterations ongoing (level 49 through 52), we're getting to first test version (level 53), which needs agreement from bosses (level 54 through 57), and finally the first production release that is to match Bob's idea (level 58), to which Bob says "WTF?" (level 59) and to which customers respond "WTF?" (level 60 through 65)

You get the picture. And it gets worse; not only must we translate from person to person, but from person to tool to person to tool to tool to tool to person to tool to person to person to tool to tool ... and on and on and on.

The tools

It's hence a given that our tools need to be good at translations between humans and computers, no? The whole idea of programming languages is to do just this, so choose your language wisely, young padawan. Some choose low-level languages to get closer to how the computer sees things, other choose high-level languages hoping those who built the underlying layers have all the semantics down pat. Or up.

Back to Robert's post; in Java people use a lot of syntax to express something that in other languages takes less. But is language about semantics? Is the difference between French and German the words they use? Of course not. The same goes for programming languages.

How many levels of translation do I find in my development stack these days? I'm doing a pretty straight-forward webapp wizard thing. We could start with the customer having an old webapp which sucks, and they want a new and improved one. So they tell us what they want (level 1) which we jot down on paper (level 2) while showing us the old one (level 3), of which we agree roughly on what needs to be done (level 4). Our functional designer draws up a few sketches (level 5), which the customer representative agrees to (level 6) which she takes up the ladder to get more agreement on (level 7), and we're handed responses (level 8) to which we update the sketches (level 9) ... and on and on and on.

But let's look at it strictly from a technical aspect. Let's pretend we have a joint human understanding of what the needs and solutions should be. First, there's legacy to look at, and here we've got Java all the way (level 1), running WebSphere app servers (level 2), using enterprise given code guides shaped with Java 1.6 / 6 (level 3), running in three environments (level 4), with Maven to build (level 5), Hibernate for db (level 6), Struts 2.0 for MVC (level 7) and some JSP semantics (level 8), FileTags for more JSP "fun" (level 9), and of course JSP itself with mixed in Java code (level 10), running Jetty in development (level 11), with proxied Maven repositories due to no internet connection in devel (!!) (level 12), our own JavaBean naming conventions (level 13), legacy-based property names (level 14), about 10 different Java tools that comes bundled (but not used) with the platform (level 15, just in case namespaces gets muddled) ... and on and on and on. And that's just development. I could add a cut-down version of Eclipse as IDE (levels 16 through 20), running through WM Ware (level 21) and Citrix for intranet access (level 22), servers running Linux (level 23 through 25) while we develop on Windows XP (level 26 through 28) ...

APIAPIAPIAPI?

You know, at this point we gotta mentions application programming interface (API), because, frankly, they drive me friggin' nuts! Basically they are lists of words (which are objects or functions) that convey semantics of whatever module they represent. Sorry, that was the interpretive version. I mean, they are "a set of declarations of the functions (or procedures) that an operating system, library or service provides to support requests made by computer programs." It's what we use - all the time, all over the place - to have bits of software talk to other bits. All the semantics (i.e. meaning) is left in the hands of those write the docs for those APIs, which must match the behaviour of the code. Without this perfect match of perfect symmetry, everything can quite quickly turn to batshit. And it does, because this is what programmers really do all day; try to use APIs the way they were intended. Communication, programmer to programmer. But the focus again is on technology, rarely on the meaning of the thing.

Did I mention that I hate automated documentation with a passion? The way an API works is left in the hands of programmers slapping keywords with short explanations in code? All the hundreds of layers of fragile communication is autogenerated from code?

I refer often to something I call API thinking, where the semantics of a problem space is defined through keywords. It's a bit like trying to explain the complex nature of the Iraq War (the current one) in terms of War, Bush, 1991, 2001, USA, Iraq, Oil, WMD, and few getters and setters on top. Let's call it the iWreck API ;

Dependency oil = new Dependency ( Dependency::RESOURCE_OIL ) ;
int startDate = new Date ( "2001" ) ;
War war = new War ( startDate, War::TIME_UNKNOWN ) ;
war.author = "Bush" ;
war.history = new HistoryFactory ( new HistoryContext ( new Context ( new Identifier ( ... ) ) ) ) ) ) ;

There, done. Easy-peasy. Now we know the meaning of the Iraq War (current). What else is there to say? Come to think of it, I reckon Bush would make an excellent programmer.

Stacking it up

How many levels of translating our communication from one unit to the other have we got when we add them all up, all the human ones, the tools and the systems? Bucketloads! So many it is quite crazy to think that any of the original ideas come through this enormous filter monster we have created!

Doing communication right is damn hard. How can I explain? Maybe through example. I had some JSP code similar to this ;

<framework>
<section>
<sub-section />
<sub-section />
<sub-section />
</section>
</framework>

When someone was going through the code for this, they found out that the JSP for the

tag didn't add any HTML nor did any logic in it, but simply told the compiler to travel up the stack. They hence removed

tags, because they didn't have any functionality attached. We now got;

<framework>
<sub-section />
<sub-section />
<sub-section />
</framework>

I'm not sure you've noticed, but you just lost the semantics between the <framework> and a <sub-section>. What is a sub-section without a section? It's API hell, that's what. It may not look like much of a deal to most people, but remember here what we're trying to do, remember that very thing, the most important thing in all that we do; communication! And when you're removing a section in which a sub-section lives, you get a sub-section out of context. The reason for the sub-sections' life is lost, and the technologists killed it thinking they cleared things up by removing redundant code.

My soul weeps. I see this sort of innocent but utterly devestating loss of semantics through every level of interpretation, from customer to developer to user. We remove little bits here and there, add a bit there, and the original idea comes out all warped and garbled, and we think it is a good thing!

Folks, if I have one request if you were to listen to anything, then please be careful about semantics and meaning, careful about adding it in, state purpose and meaning, write more about what the hell the point of what you're trying to achive, in your APIs, in code, in thinking, in documenting, in specing, in talking to eachother, in anything you do, and - perhaps and hopefully poignantly - in reading my little plea for semantic sanity in software development.

Thoughts on PHP

Yes, yes, I admit the herasy; I like PHP. No, no, PHP has tons of worts with it, so no, it's not better that alternative X, Y or Z for task Q, W or E. I hate comparing languages this way, feature by feature, syntax by semantics, and so on. I like to judge languages on two things;

Environment

I really like the CGI style for web resources. No, PHP ain't CGI (unless you're in a pain-self-inflicting mood) but most often a glorious Apache module which reuse all the goodness it can offer, but the model of a totally independent scripting engine which needs to mold its relationships, and then you throw it away when you're done, makes for a clean, fast and very scalable framework. PHP basically compiles together mostly C modules, and use its simple syntax to glue stuff together. Yes, there's a backlog of really shitty and badly written code out there, especially when people have no clue. And when the threshold is as low as it is for PHP, that's inevitable, but I hardly hold that against the language itself. The environment in which that shitty code runs is really good. To me, the environment is the best part of PHP. Perl falls somewhat into this same category.

For those in the know, this model is very similar to how a RESTful system works, where the interpreter is a manifestation of a resource. In resource-oriented development this means gold, and is very important to me as my environment supports my RESTful way of designing systems.

Style

And since style is something any good developer can control themselves (unless their language is super-strict), this really is the main thing that I like about PHP; It gives me the freedom to make things work for me. I can choose the methodology, whether a function, class or inline is best suited, and since PHP always is evaluated in run-time, I can make my environment depend on real-time parameters rather than a pre-compiled utopia.

Zend Framework

I've been using the Zend Framework for the last year or so, and it's a great framework as such, although the focus has been mostly to put OO wrappers around common PHP idioms and conventions. As such it works great to perhaps consolidate the features, and perhaps give PHP 6 a future direction. This is the best part of the Zend Framework.

The bad thing about the Zend Framework is that it imposes its own style, and somewhat alters the environment. Ouch, on both the things that make PHP my choice tool. I've struggled quite a bit in trying to reuse certain parts of the framework, extend others, and generally use some bits without using the whole shebang. There's not a lot of dependencies between the components, but just enough to make it tricky to do serious stuff (for example writing an alternative "threads-like" HTTP adapter to HTTP Request).

Now, people talk about the Zend Frameworks impact on the future of the PHP language. Yes, one can always hope that this is the case, but I think people are a little too preoccupied with the OO capabilities and forgetting perhaps what makes PHP really popular with those who choose to continue to use it past their first few applications. Do not fall into the trap thinking OO is somehow better than other ways.

As much as people think MVC is the best thing, I really don't care about that. MVC works great for some things, not for all (for example, I have a REST framework that use a completely different model, a more resource oriented model), so to impose MVC as the modus operandi is not good, and indeed something that makes reuse of other ZF modules a bit trickier. Instead of a MVC focus, there should be a strong highlighting of a uniform interface to the environment itself. It's the environment that's cool with PHP, so let's make interfacing to that better. There's some work on wrappers and readers for various aspects of the HTTP protocol, but only the most basic stuff is in there and needs serious work. With PHP I could do serious HTTP applications; with Zend Framework I'm limited, and need to hack and extend. Let's get HTTP savvy, not MVC drones.

Availability and diversity

PHP is everywhere. Period. I can use pretty much any ISP or in-house hosting to host most of what I need. And there are so many different open-source projects about that uses PHP (WikiPedia, WordPress, PHPBB, PHPMyAdmin, Drupal, Joomla, Flickr ... the list goes on) meaning both hosting and the amount of high-quality tools are abundant. The LAMP stack is pretty much supported on every hardware and software platform. Heck, I can even run it on the JVM.

I have written many tools over the years, some good, some bad, but I'm always happy to find out that I can copy my old files into any new environment and they will pretty much always run straight away, no tweaking. I can't tell you how important this is!

The shitty stuff

No system is perfect, to some specific definition of what "perfect" means. With PHP, I've learned to live with its odd bits, such as funny booleans and comparators, diverse and non-uniform APIs, sloppy exception handling, no shared memory (which may or may not be a good thing, but certainly a big part of its design), and a few syntactic snuffs (Why not introduce "$$" as a shortcut for "$this->", for example? Tidbits).

Some say that PHP code is crap, usually said when encountering - indeed! - shitty code. But shitty code is everywhere. Even in the most tied-down, static and controlled environments you still get shit. Some say less shit. I say shit with a different color, not different odor. I've been programming for almost 30 years soon, and let me tell you, I've encountered shitty code in every single environment and language I've ever tried, and I'm one of those who tries every language out there. There is no language who can hold a stick to the wonderful imaginitive mind of man, able to lay bricks where diamonds should have shone.

Some say the PHP language itself is immature, or unstructured, or some other parameter that their favourite language holds. No, sorry, but there's not that much which separates all our different languages (except BrainFuck ... that one is seriously different. :), and indeed a lot of the language wars are really more about their API designs than the languages themselves. For example, the syntax of Java is tolerable, but a lot of its APIs are not. The syntax of Python is ok, but some of its APIs are great. Ruby has good syntax, but to me some confusing APIs.

It's really mostly about APIs, and how we create models to solve our problems. It's all about models, as APIs in themselves are models. The API is not the language.

Round off

I should here of course mention the link between models and Topic Maps, as the latter is a way to define, work with and exchange / share the former. Couple this model openness with resource-orientation, and I think it makes for a very interesting environement. But as this whole shebang is part of the new framework I'm working on, expect more blogging on this in the very near future. Have a wonderful week.