mobcode

Build a prototype instead of a proposal

When you first start working on an application you can create an end-to-end working system in a remarkably short time. I don’t just mean a false demo that provides screens that don’t work. I mean a real working application. Now it will be very short on features. It will be an anemic application. But, it will be a working end-to-end system.

Some developers will not deliver a working end-to-end system quickly. Instead they will write specification documents, build data models, create architecture diagrams, build frameworks, construct development environments, do UI design, create detailed estimates, etc.

Some developers are scared to show sponsors a working system early, because the sponsor will never understand that the application is not finished, it is merely started.

Use the fact that you can create a working application very quickly to your advantage. Use this to win business. Use this to win real engagement with your sponsor.

If you are bidding on a project, instead of producing a proposal, produce the first version of the application. Imagine the buyer comparing a written proposal with a working app.

If you want to really engage with a sponsor and get them excited about the project, then show them a working prototype instead of a proposal.

And don’t just do this once, keep doing it. If you deliver new features that the sponsor wants every week then you will become an ally of the sponsor rather than an antagonist for them to fight against.

Fires kill productivity

In order to be productive you need a short list of things to focus on. When you focus on an item you get it done.

Fires are the urgent items that pop-up throughout the day and demand immediate attention. These urgent issues kill productivity. They kill productivity because their urgency encourages us to take on too many tasks. Their urgency almost forces us to multi-task. And multi-tasking kills productivity.

So my question is, why do fire houses always appear so neat and orderly? If firemen are always fighting fires then how do they have the time and discipline to keep their trucks washed and everything put perfectly in its place?

Multitasking does not scale

It is one thing to bear the burden of working on ten things at once. But now apply that to a team. There are five team members. Each working on ten things. The team leader ends up multitasking between fifity different issues! Working on fifty things at once does not work very well.

Generate a real parser for your custom language

Many years ago I found myself in the position of writing telephony applications for the phone company. We were using an IBM system that expected you to use a graphical programming tool to create a state machine describing the program. The state machine had primitives like play a message to the caller, collect digits from the caller, etc. Writing programs as state machines in their dismal environment was brutally painful. You ended up doing ridiculous things like building while loops out of primitives that checked a condition, took an action, and then looped back to the top. All of this by adding blocks and drawing lines between them… ugh!

So I longed for a simple C style syntax to write my code. Not knowing any better I got a copy of the Dragon Book from the library, downloaded lex and yacc and created a prototype of a language that compiled to a state machine format the IBM system could read. Once I had a protoype working I was convinced I could really build such a thing. I switched to JavaCC and created a full-blown language.

The way tools like lex/yacc and JavaCC work is that you create an input file that specifies the grammar for your language. You tell it what the keywords are and what order they are to appear in. Then the tool generates a parser for your language. It produces an abstract syntax tree that you can then walk to produce the compiled output that you need.

This was undoubtedly the most fun I ever had programming. And this was before I became a serious professional programmer. In fact, this experience was one of the factors that drove me from engineering to programming as a profession. So I highly recommend the experience of creating a language in this way. Furthermore, if you are going to create your own language this is the way to do it. You need a real parser for it, not some home-grown monstrosity. So using something like yacc or JavaCC is the way to go. You can hand someone the grammar specification and they can read it to see how to speak your new language.

It is quite amazing the power you feel having such a power tool at your fingertips. On future jobs I would whip out JavaCC and create a code generator at the drop of the hat.

Back to the original telephony project… once I had the basic language working, I then proceeded to create another language: a domain specific language that compiled to my C style language. This DSL was tailored to creating the kinds of voice applications that we were building. Unlike the IBM graphical programming tool, my DSL made it trivial to create the apps that we needed to create. I became dramatically more productive than my teammates by using these tools.

So what was the next step? Getting my teammates to use it of course. And why wouldn’t they want to? It would save them tons of time. Well, as it turns out, I was never able to get anyone else to even look at it seriously, much less use it. What good was the formal specification of the grammar for the language? None. Nobody read it. And if they did look at it, it’s not the most self-explanatory thing. What you really need is lots of examples that show the language in use. Nobody learns a language by reading the grammar specification.

So you can create your own language for yourself, but there is not much chance of getting others to use it.

Don’t make up your own language

So you are working on an app and you find that you want to add some hooks to let users customize some aspect of the app. Say… you are building a task management application and you want to allow users to plug-in an algorithm for scheduling the order to work tasks. You might start out small and give users a custom field to add a order to each task. Then you realize they need more power so you let the users enter a simple expression that evaluates some of the fields on each task to determine its order. Then you go a step further and add the ability to call a sub-routine. They you add variables. You are sliding down a slippery slope of creating an ad-hoc custom language.

That is bad thing.

Instead of doing that just adopt some scripting language and add a hook to call a script. The default choice for a scripting language must be JavaScript. For a list of some other choices check out the Extension/embeddable languages section here http://en.wikipedia.org/wiki/Scripting_language.

Failure oblivious computing

Do you really need to understand all aspects of the program you are writing?

Is it more important for a program to be correct or simply keep running?

I certainly have strong reactions to these questions. My natural, strong responses are based on the principles of how software ought to be done.

This paper (link provided to me courtesy of Kyle) challenges my thinking in a startling way.RinardOOPSLA06.pdf

The results are so startling that on this page I was ready to discard the paper in complete disbelief:

Our Philosophy • Should be able to ignore addressing errors • Perform dynamic bounds checks • Discard out of bounds writes • Manufacture values for out of bounds reads • Continue executing • Called failure-oblivious computing

Did you get that? If the program writes to memory that it does not own… just throw away the write (!?). If the program reads from memory that it does not own… just make up some value to return (!?).

But then the author makes a compelling case based on a empirical data.

The startling conclusion is that in many cases we cannot afford to build correct software. When I think about reality, rather than my wishful thinking, this resonates deeply.

Let data flow through to users

Applications often need to be treated as pipes that carry data. In such cases the pipes should be smooth and should not consume data. Rather the data should flow out the other end.

I have seen this pattern in a variety of applications including billing systems and email handling systems.

Consider an application for routing email. The basic idea is that every incoming message needs to flow through the filter and end up coming out the other end.

Suppose there are fixed filtering rules that you are implementing. One rule might be that if the email is from domain “casino.com” then route the message to the folder “gambling”. That rule is implemented and the messages flow through. If a message arrives from casino.com then it ends up in the gambling folder. So this meets the objective of allowing the data to flow.

Now, consider another rule. The rule is that if the subject line contains the word “urgent” then the message should be handled specially. The users are unable to tell you how to handle it. The requirements are pending. But the users assure you that there are special rules for such messages and you will need to implement them. So what do you do? The right answer is to add a rule that detects the word “urgent” and puts those messages into the folder called “urgent”. This seems obvious, how could it be done wrong?

The wrong answer is to implement code that detects the word “urgent” and then places the messages in some internal holding area that the users do not have access to. This is violating the objective of being a smooth pipe. Data is sent in, but it never comes out. This leaves users peering into the pipe looking for their data, shaking the pipe, cursing the pipe. It leaves users unable to use the app because data comes in and disappears.

And users are the key to this issue. Users are people, not computers. That means users can adapt and do the right thing even if the system is only partially implemented. So even if you don’t know the complete requirements make sure that the data flows out the other end. This way users can implement the missing requirements. If you don’t do this then you will prevent users from being able to do their job simply because you did not have the needed requirements to build the system. Now it may not be your problem that the requirements are not available, but if you build a system that eats the data then you become the problem.

Now consider this advice in light of an agile, iterative development process. Never build a version of the code that hides data away in some corner. Every step of the way make a system that lets the data flow. The first step in building this email router is to make a no-op router that merely forwards every message on to a single destination. With that start, never build a system that eats messages.