• Larval - junior programmers - curious maybe, confused often.  Not sure what type of programmer, if any, they will eventually morph into.
• Diggers - The guys that are great at hunting down bugs.  They seem to have an infinite ability to pour through log files, read configurations, run countless GREPs across prior versions of the code, etc.  Often, these guys aren’t actually true programmers - they often can be testers, sysadmins, etc.  But the title doesn’t matter - what these guys do is dig.
• Housekeepers- The guy who seems to remember all the crazy legacy reasons for things, who knows in intricate detail how people use the system, and can usually tell you, almost instantly, what it will take to perform each particular enhancement or bug fix.  Oddly, these guys tend to be less good at actually producing code (though this is more due to mindset than ability).
• Directors - Much better at telling people how to produce code than actually able to produce code themselves.  Directors often become Business Analysts.
• Builders - these guys produce code.  It doesn’t really matter what they are working on -  they write it, no matter how large.
• Artists - These are the guys who don’t seem to be producing anything, and then disappear away for a 48 hour programming jag that produces a brilliant, elegant piece of software.  And then they disappear for a few more days, hopefully to sleep.
• Editors - These guys are oddly bad at producing code, but terrific at optimizing, debugging, and tweaking existing code.
• Duct-Tapers - A duct-tapers main goal is to get things working.  They are willing to, and often do, accept a less than elegant solution today in lieu of a perfect design tomorrow. Great at emergencies, hot-fixes and the like.

Where all of this becomes important is in the staffing of a project.  Different projects will require different staffing models.  Most staffing plans tend to think of staffing programmers by skillset (e.g. database programmer, VB guy, JMS expert) and/or experience level.  However, specific skills can most often be taught or brought in much more easily than mindsets can be changed.  A digger will always be a digger - even if he started out as a Database programmer and ended up a U/I guy.  So a successful staffing plan must take temperament into account as much as skill-set.

• Green field projects, especially small ones, are better staffed with Diggers, Artists and Duct-Tapers.  Artists lay down the big bigs, duct-tapers pull it together, and diggers smooth the rough ends.  Oddly, Builders are less useful here - simply because inspiration and innovation can often trump perspiration in the early going of a project. Directors and Editors are to be avoided and Housekeepers will be bored.
• Young projects (something is up and running, but not “complete”): Diggers and Builders.  Artists and Duct-Tapers will start to get bored - Directors will start to appear
• Refactoring, re-writes:  Great for Housekeepers, Directors and Diggers.
• Maintenance: Housekeepers, Builders.  A Duct-Taper is good here for small bits of continuous improvements.
• Integrations: Duct-Tapers with the help of a Housekeeper or Director.  Diggers are useful if the integration is big.

I have seen projects fail for reason I think can be directly attributed to the temperament of the people involved - a major application rewrite that fell apart because there were no diggers or housekeepers (”they built the first one wrong, why would we let them at the second?”), a new application that was delivered two years late because everyone on the project was a Director or Builder (the Builders just wanted to be told what to make and the Directors could only say what was wrong), and a product expansion loaded completely with Directors (ever seen a program where the same for loop is rewritten 28 times?).At this point, its probably obvious that of all of the programmer types, a duct-taper and diggers are the most useful (though, for full disclosure,  my career has mostly been green field and early phase stuff. Assume biases per your discretion).  I tend to try to find work that has attracted duct-tapers and has kept at least one or two around.

Ruby? Perl? Java, PHP, Python, Cold Fusion, .Net …

Rails, Struts, Spring, EJB3, Zope, Hinabe…

Apache, JBoss, Lighttpd …

Linux, Windows, Solaris

Oracle, MySQL …

Why didn’t I go to med school?

There are enough technologies and platforms out there that something as simple as building a website can lead to huge amounts of analysis paralysis. But the risk here is huge - if you choose poorly, the site won’t scale, everyone will be fighting the system trying to get new functionality in place, you will be hated, the company will fail and you will end up living in a box by the river in six months.

Or maybe not.

So what’s the right technology to implement your new project? The answer is actually very simple - it doesn’t matter.

No, seriously, it doesn’t matter.

No software project has ever succeeded or failed because someone chose Cobol over Ruby on Rails. Projects succeed or fail based off of three things:

1. Having “the secret sauce.” (which means your baby picture idea is going to fail in every language)

2. How quickly you can get the sauce in front of customers.

3. How well you can get the sauce in front of the right customers.

Its not about programming languages, scalability, open source or anything else. They will matter later, if you are lucky, but none of those things matter RIGHT NOW. So what technology do you chose?

Chose what’s in front of you. Your team knows Java? Then write the damned thing in Java, regardless of what 37signals.com is telling you. You like Ruby? Write it in Ruby and ignore the “it won’t scale” arguments. Are you poor? Use the LAMP stack.

But what happens when we get big? What happens if we could have written it faster in some other language? What happens…

Skip what ifs, because face it, the odds are against you. There is no framework out there that let’s you automate the build of eBay. PHP doesn’t have a <facebook> tag. If what you are trying to do is good and valuable and unique, then no one else has written it before, and you’ll need to implement it no matter what.

So when it’s all said and done, if you are lucky, you’ll end up with a few things:

1. A system you are reasonably familiar with.

2. A customer base that’s growing faster than you expected or planned for.

3. A whole mess of things you wish you’d done differently (functionality you wish you hadn’t wasted time on, poorly designed algorithms, a lack of a proper test harness)

Again, this is a very very good thing. Yes, it would have been easier to write a test harness and full set of unit tests earlier. Yes, you should have planned for load better. Yes, you really wish your code had better documentation so the new guys you hired could get up to speed faster. But you have customers, and hopefully that means you have revenue, which means you have a much better chance of surviving than if you had a scalable system with a great test harness but no revenue. Never ever worry about something until you absolutely positively have to.

This sounds like the advice given to a startup, but its true for projects in large companies too. Unless you have some brilliant political connections, or at least pictures of your CEO doing things in Vegas that should have stayed in Vegas, then you are going to be beholden to the same revenue trap as the startup down the street.

Of course, that leaves one big ugly open question: if I do everything right, I’m going to have to start worrying about everything I did wrong at some point. When do I know what that point is?

But I’ll save the answer to that for another day.

in Pseudo-code:

load variables;

start loop

    did I remember to load my variables? I better load them again.

    oh yeah, I need to reset the counter first

    output string

    output other string

    load more variables

    output another string

    check the counter

    //couple of random comments

   did I load that other thing I needed?  better check

end loop

Idiocy aside, there are often good reasons why small, simple ideas become 4000 lines of code behemoth functions. I’ll give you one example we spent months wrestling to the ground:

Most of our applications are very date sensitive, and one of the things we constantly need to know is how many days there are between two given dates. Since any function that we wrote would be called 100s of thousands of times during the normal daily execution of the program, the programmer strove for something that was simple (since everyone would rely on it and read it often) and fast (which was a classic example of premature optimization, but generally not that big a deal).

His first pass at the function looked like this:

function daysBetween(Date d1, Dated2) {

//compute milliseconds in a day as: 24 hours * 60minute * 60 seconds * 1000 milliseconds

int millisInDay = 24 * 60 *60 * 1000;

int delta = d1.time() - d2.time();

return int(delta/millisInDay);

}

So, for the most part, this function is decent enough. It’s rather fast (the longest part taking place in d1.time()) and small enough that it’s very simple to read. All its doing is taking the two given dates, subtracting one from the other, and dividing by the number of milliseconds in an average day. Four lines of code gives you something that is fast, elegant, and wrong.

It didn’t take long for us to start finding the first few problems. They were pretty simple, both to find and to fix. The first was that the function was only working if you passed in dates in the right order. Since the function subtracts the second date from the first, if the latter date is listed first, you start getting things like having -7 days between consecutive Sundays, which could be said to be mathematically correct, but still isn’t very useful.

I could take you through each of the iterations we went through, incrementally improving the code, but I suspect you can guess most of them (using an absolute value for delta, using a round instead of an int cast…). What really ended up killing us (and the inherent elegance of the function) was two things.

1. Having the number of days between two dates isn’t actually that helpful. 99 times out of 100, we found that as soon as we called our daysBetween() function, we were turning around and calling some second function to tell us what days are between day X and day Y. It eventually seemed to be a lot smarter to have one function that just returned an array of all days between a given two, and then you could get the bit of extra information as a one off of the more useful array (by looking at array.length…)

2. Timezones.

Timezones? Yes, unfortunately. After we modified our function to return an array of dates between Date X and Date Y, we found an odd situation when, occasionally, one of two things would happen: we’d either get one day less than we expected, or we we get the right amount of days, but only because we’d get one date twice.

Or, more accurately, I should say that our customers kept getting these situations - we didn’t. Our code passed all of our unit tests, our QA group signed off on it, our UAT group signed off on it, and our collection of beta customers had no troubles. It wasn’t really until we were out in production that we started having trouble (this is probably one of those cases that shows why testing is, at best, a risk mitigation strategy, but that is a point for another day…)

For a blog post, this is getting rather long, so I won’t go through the details of how we found the bug (imagine a group of us looking here ), but it all comes down to this line:

int millisInDay = 24 * 60 *60 * 1000;

Counting days in milliseconds is problematic in a couple of ways - the main one being that a day is actually 86,162,400 milliseconds long instead of 86,400,000, but that wasn’t the problem. The problem was Daylight Saving Time. In the United States, we mostly follow DST (well, except for Arizona and Indiana - which has caused us other problems in the past). The most recent time the clock flipped was at 0200 on November 4th, 2007. When our software needed to compare dates, we always used dates based off of the same time of day: midnight. So, when November 4th rolled around (or more accurately, when we compared a set of dates that included November 4th) we were mostly okay.

Other countries, however, change over to DST at midnight, so in a few cases (Brazil, I’m looking at you here), when you cross over certain dates, some days will get double counted - Add 24 hours (86400 seconds) to November 3rd at midnight, and you should get November 4th at midnight, but since November 4th at midnight rolled back to November 3rd at 2300, you get two November 3rds in the list. Simply put, there are dozens of days of the year (since not everyone switches on or off DST on the same day) where one day is not 86,400 seconds long.

Our final version of the daysBetween() function is 232 lines long, all of them necessary, but none of them particularly fun to look at. Listed as I have, the solutions encompassed between the two {} seem pretty trivial, but each one was hard fought, and more importantly, hard won. We have an application that works marginally slower than before, but we know that it will work correctly (at least w/r/t dates) in Brazil and Western Australia and Southern Spain and even, begrudgingly enough, Arizona.

Yes, that function is now almost 80 times larger than it previously was, which means its at least 10 to 15 times harder for a programmer to read, learn, and understand, but that’s the way of software. I’ve seen hundreds of enhancements that have increased the size of an application codebase by 15% or more, but I’ve never seen one that shrinks a codebase more than a few hundred lines. You only really have three choices: watch your program grow until you can no longer keep track of it all, force it to stay a specific size and lose a certain number of Kangaroo friendly customers, or watch the whole endeavour die, because no one wants what you’ve written anyway.

We, obviously, chose option 1.