Monday, February 9, 2009

It's a Job Title

I read two blogs asking whether we are engineers or developers. Their argument is that computer science does not produce engineers because we don't have hard and fast rules for what optimally solves a given problem. Engineers, to them, are people that really deal with the physical laws of the universe when producing a given object in a predictable, mathematically provable way. They have interesting points:

1. Jeremiah Clark, “Jeremiah Clark's Blog : Are Software Developers Really Engineers?,” http://blogs.msdn.com/miah/archive/2009/02/08/are-software-developers-really-engineers.aspx.

2. Scott Palmateer, “Software Alchemy - Scott Palmateer,” http://scottpalmateer.spaces.live.com/Blog/cns%21144A1BAAB1016256%21491.entry.

First, there is plenty of engineering in software development. We don't always see it, especially when we are working within generic frameworks, because the execution behavior of the framework changes the execution behavior of our code and thus removes one of the tenants of good engineering: predictability. Predictability in the physical world is being able to determine the crisis point of a given object well ahead of the crisis occurring. We can predictably tell exactly how much weight a given box made from a given material can contain before it bursts. In fact, you can look at the bottom of most generic brown card board boxes and see the burst point clearly labeled on the Box Manufacturer's Certificate by some box standards organization. Software development often doesn't have that level of predictability as the developer moves further away from the raw hardware.

A current Java based web application running Seam (Richfaces, JSF, EJB 3, JPA/Hibernate and more) on JBoss has very little predictability in my opinion. I believe this to be true because I watch how much injected and generated code is created as my application runs. I can watch my call stack grow as the injected and proxied objects get generated that manage a single request's life cycle to get to my first bean, let alone drill all the way down to my SQL server. I cannot predict exactly how much code will execute between the servlet that is actually processing the incoming request to my bean and I have no idea how long it will take to execute that code. I am left with the hope that it will perform well and that my code will execute in the way I had expected.

There is a concept called "programming by contract" and another "programming by coincidence". Programming by contract says that someone writes a rigid specification for a given API and I write my client code using only the specification as my guide. It also means that the results of any call have been laid out in the specification such that I can trust that with good data the API will respond with a valid result and with bad data that the API will return a specific error response. Programming by coincidence is when you call an API method and set a breakpoint on the line, examine the results and write the rest of the code to handle that response. The problem is that many specifications are either inaccurate or wrong so you end up having to write some measure of code that relies on programming by coincidence. Programming by coincidence almost always eliminates predictability because the behavior of the API or framework is not guaranteed and thus not predictable.

Predictability increases as you move closer to the hardware. You know precisely, or pretty accurately, what your low level language is going to load in order to execute your application and the overhead of those components. The fewer and lighter the external components are, the more likely you can reliably predict performance and side-effect impact. Higher level languages, like Java with some frameworks, don't have the same level of predictability because you can't always tell what the application is really going to load and how those components will effect your application until you start running it.

Note: I love Java as a development language and I couldn't accomplish a giant web application in a timely fashion if I wasn't using some kind of framework. I know though, from a purely scientific perspective, that the more I bring in from the framework, the more risk I take on that the component will not execute or perform to expectations.

Second, software development is about applying design and implementation principals to a specific situation and those principals may not work in other similar situations. MBA students talk about case studies where a professor presents either a real or fictitious scenario to help students to understand a decision making process. The professor may go through several different case studies that may have similar scenarios and vary enough to result in different outcomes. We have the same kinds of problems where one variation in the goals of the project can change components and the project's schedule.

Let's say you have to write a log file reader to extract all the SQL queries generated by Hibernate for two different companies. Company A wants the reader to run though at least 10,000 lines per second and Company B wants 5,000 lines. Company A wants to see the changes in real time and B wants to run through the log at the end of the day and produce a single report. So far both companies could probably be served by the same application. Company A says it wants to store all the output in a database, Company B wants it in memory. Finally Company B says it wants to support IBatis queries and to filter out certain queries. Now we have some design differences that may be significant.

We can see that the reading and translation of the queries is largely the same until Company B wanted to support more than one ORM. One design may take a polymorphic stance and try to generate objects that specifically read each kind of line. Another possible design is one where there is simply a branching statement that just runs the translation code without using a specific object for the kind of log statement. The storage mechanism may require something innovative to get the database updates to meet its performance needs while Company B may require something far more simple. You want to reuse as much code as possible, so how do you do it?

You have a lot of different options and any one of them is probably just fine. You may even decide that your hourly rate is greater than the cost of using something prepackaged so you just buy that application, configure it and then work on something else. You may also sit around and write something in python, perl, Java, C++ or any one of your favorite languages. Tell the same scenario to another developer and he may come back with an entirely different approach and it still meets the same requirements. One approach will be better than another in some respect but may fail in others, thus software becomes a negotiation process in dealing with the trade offs to accomplish the goals set by the customer.

Third, it's a job title damn it! So one company calls you a software engineer and another calls you a software developer and you think it matters? You write designs and implement code in the same way in both places. I write web applications all day and I argue that we mostly find ways to integrate components and systems together to build interesting apps. There's hardly any really innovative code beyond just making the web application work. I half expect to be retitled from Web Architect to Web Cobbler. I could probably resole your shoes while I'm at it.

Job titles are pretty nutty when you really look at them. One company's Software Engineer 4 is another company's junior software engineer. I know this from experience in that one previous employer ranked employees in pay bands so a software engineer made X dollars which caused him to be in the fourth pay band so I was a Software Engineer 4 even though it was an entry level job. My Magic Missile improved remarkably when I became a Software Engineer 5.

You see Principal and Principle Software Developer/Engineer on the job boards all the time. One means that your a very senior engineer and the other means you have a good moral compass. My favorite job title was a guy from Sun that was known as a Computational Theologist.

You have to look at the job responsibilities to understand whether the title is meaningful.

Now I'm not saying to ignore job titles when looking for a gig. Job titles actually do matter though some employers and pundits say it shouldn't. A senior job title should mean senior responsibilities. The title opens doors later in your career and we shouldn't discount titles as a result, especially when they show a track record of increasing responsibilities. That means your employers trust you to lead or do important things and future employers will like that.

Sometimes we get the chance to change a title to something goofy or funny. Grand Coder, Debugger Extraordinaire, "He that lurks within the kernel", Jackass, Meat Calculator, Chief Computer Scientist and so on may sound great but are probably less advisable if you want people to quickly understand what your last job was without having to go into obnoxious detail. While I chuckle that I'm a software cobbler, I won't be changing my title any time soon.

Monday, February 2, 2009

Losing Routine

English has this wonderful term called "Senioritis" where US high school seniors often give up on hard school work in the final days, weeks or months before graduation. They figure they can coast to the end and do just fine. There is an unspoken wisdom to the pressure valve opening, especially at the realization that childhood is supposed to be over and now you must go off and live your life. Adults know that the real transition is when you leave student academia for work and that transition may not happen for several years.

Software projects can go through a similar senioritis. It happens near the end, after some major milestone has passed and things go from frantic to calm, like the sea after a storm passes. There is a lot of churn and the flaws and damage are laid out for all to see, but the frantic pace and long hours are mostly over. I look at these senioritis times with a sense of cautious optimism.

I'm cautious because some people become sloppy and stop doing the things they have been doing for months. My favorite change is the sudden disappearance of the steps to reproduce a bug. I may get great bug reports with detailed steps to reproduce the error and the suddenly I get bugs from the same people that say "Dialog doesn't work" and I go into that dialog and find it does work. I try two or three more things but the error doesn't manifest itself. The normal thing is for the developer to mark the bug as not reproducible and wait for QA to reopen it after adding some new step or some missing clue.

I'm a stickler for following procedure and protocol. It can drive some people bonkers that I want nightly builds, comments with commits to source control, bug numbers with commits to source control, steps to reproduce bugs and the resolution comments for that bug to be something more than "fixed" unless it was remarkably easy like renaming a field's label. It isn't unusual for me to visit or email a staffer asking them to stick with the protocols we have been living with.

I know, someone is rolling his eyes saying that I'm just a procedure pushing dumbass. That I'm enforcing procedure rather than facilitating development. Some times the comments in a bug are enough to tell the developer how to fix a bug with no need to have steps to reproduce. Those people are correct up to a point. Common sense does kick in because we don't need to have steps to reproduce an error when we are talking about changing the label on a field though the change may be tracked in the bug tracking system. The point is that we follow procedure in the big and little things alike so that everything works consistently.

Consistency and reproducing consistency are valuable for limiting miscommunication. Skipping protocol means that details are lost and those details may need to be found again. We have to trudge through the base work to get back to the original point. Skip the daily builds and you don't know when a major build change occurred that breaks the product and prevents the next release. Skip documenting the steps to reproduce a bug and other people on the team can't figure out what is going on, or worse: a developer reproduces what looks like the bug but he really fixes an entirely different bug that exhibits an identical looking error.

A project experiencing senioritis sees the procedures breakdown that brought you so close to the finishing line. As details are lost, so are the hours before the project is done. The pressure comes back to the old frantic pace, rising just like the waves on the sea and pretty soon you find yourself in a storm in the hours before releasing the project. The moral of the story is to stick to procedure, be consistent throughout the project and don't let normal everyday processes slip until the project is done and shipped.