Successful crap

It never ceases to amaze me peeking into various successful open-source projects, seeing the innards, and wonder how they even got this crap code past their own pride. Yes, I need to vent.

This weekend I was head-down in various content-management systems and their ilk, digging into anything from WordPress to Habari to Joomla to Simple CMS (which you would expect be simple) eZ publishing. All of them had rather abysmal code scattered throughout (with eZ publishing being the better of the lot), oddities, and all the worts you'd expect of systems hobbled together, where their success is more an afterthought.

But hang on, I can't criticise systems for their organic growth. But I can criticise them for not doing much about the trouble that comes from it. Sure, I understand that rewriting core parts of a system requires a huge ego and nerves of steel, and I understand how "if it ain't brok, don't touch it" rules the end of the day, but surely the end means is good software, right?

But.

It's crap. It's rubbish. And more importantly, as I can put up with crap if it gives me opportunities and love, it hinders innovation, flexibility and, well, love.

Let's pick on a random contender that I worked heaps with last week, WordPress. If we cut away comments, this is its index.php ;

define('WP_USE_THEMES', true);
require('./wp-blog-header.php');

If we look into wp-blog-header.php, here's what we get ;

if ( !isset($wp_did_header) ) {
$wp_did_header = true;
require_once( dirname(__FILE__) . '/wp-load.php' );
wp();
require_once( ABSPATH . WPINC . '/template-loader.php' );
}

Ok, so let's peak into wp-load.php, and we find about 20 lines of code and more includes. Don't you just love playing hide and seek with files to find out where it's going and why? These things maybe have come around from shuffling the organic growth of the system into other files, and left them with these sad little snippets that's hard to get an overview of and takes a slight performance toll, too, as well as eat away your sanity and good programming ethics. And they all contain that newbie error of putting this at the end of every business logic file ;

?>

It's not needed, and if you like your whitespace under reasonable control, you're stuffed. Not a big crime, mind you, but just one of those niggling little things. Then you've got functions and objects, some with the wp_ prefix, some without, business logic in files called wp_settings.php, repetition of code everywhere, hundreds of DEFINE's scattered about in different files, and so on and so on. (And yeah, I should contribute as it is open-source and all that, and I'm actually writing a embedded Topic Maps engine as a plugin, so we'll see)

But I'm not here to pick on WordPress per se. It's more about how this organic growth hinders innovation and opportunities. So let's talk about frameworks. All little pieces of code together form a framework, so we're not necessarily talking about a framework of disjoint classes or functions that aid developers making stuff like the Zend Framework, or Cake, or Symfony, or CodeIgniter, or 1000 others out there. Well, kinda; all those little things the app is made from also constitutes a framework, but it isn't disjoint nor refactored or synergetic or stable or well thought-out as you get in a more established framework, but never the less that's what it is.

PHP itself is a framework, of course, and most PHP frameworks are wrappers and added code to make PHP act more like a coherent system, fixing inaccuracies and bugs and niggles, stabilizing behaviour and increasing the need for spending hours and hours learning some new paradigm you can't use elsewhere.

Hmm. Where was I? Oh, right; every app is a framework. But when the framework isn't a perticulary good one, where the pieces are either too fragmented or too disjoined to make any sense, making stuff in that framework is going to be a pain. Like WordPress is a pain. And before you know it we get to the next rewrite, and this time we'll get it right, although we need to keep our legacy intact, and hence we write hacks on top of fresh code to drag it back to the hole it came from. The data model needs to rewritten, but it won't because "well, it works, doesn't it?" and the framework needs to be rewritten, but it won't because "well, it's not broken!"

When I can't replace MySQL with something else, that's a hindrance. If I can't change the way tagging works, I can't move forward. If I can't change the URI handling, I'm stuffed. If I can't use portions of it to write something else, nothing new will come. Sure, there might be a plugin architecture somewhere, perhaps a simple event model that one can tap into, but if I can't replace the model in which it operates I can't make it more beautiful. I am forced to accept the model and framework in which WordPress sits.

And it sits quite squarely on top of everything I want to do. I want to create better and typed links, I want to reuse a model for sequences and storage, I want to replace tags with guided controlled vocabularies (maybe even typed and binary linked to external WordNet sites), I want to use it as a CMS and skip the URI handling alltogether, and so on. But I can't, because WordPress wasn't designed with change and innovation in mind.

But a day will come when even the most successful project will face its own innards. And some people will branch it, some will stay on, some will create something new, and some will stop using it alltogether. And it's all a really good thing; this is organic growth, it's a framework that spawns other frameworks. And more crap will be successful. Things will be broken and ugly and hackish, just like some things hopefully won't.

And in a few iterations, something beautiful - with a probably different name - will emerge.

Update: It would be great to get your suggestions for open-source projects which are designed for change and has quality and / or elegant code to boot. Let's make a list!