Back when I started developing what could loosely be called software around fifteen years ago, I didn't know what I was doing. If it compiled, ran and produced (mostly) the expected results, then the job was done. 

As a new programmer, I was immensely productive. 

Of course, problems came when it was time to fix bugs or extend the software. It was often easier to just start again than to try and understand the rat's nest of poorly structured and unintelligible code. 

Fast forward ten years and I have been fortunate enough to have had exposure to some huge, well architected and complicated systems, not to mention some extremely clever people. As a result I realise I knew nothing back then. In another ten years I'll probably think the same about my current knowledge and ability. But that's the nature of software engineering. You never stop learning and evolving.

I do however miss those naive days of being able to crank out code with such velocity. It was fun back then. Yeah, what I did then might have been inefficient and probably quite flawed. But for the most part, it worked and served a purpose.

Back in those days of youthful ignorance I didn't have the experience to know what could go wrong when I hit the database fifty times to service a request. That was learnt the hard way. I just ploughed ahead, thinking in functional terms about how the application would behave for a user. I didn't worry about code being performant or extensible or even dream of coming up with my own framework. No yak shaving, I just focused on the task in hand. This was perhaps a good mindset to have.

A decade later, I am not suggesting that we should lash things together in this way. It'd be moronic to suggest we shun our collective experience, knowledge of patterns and the advanced features of programming languages. Not to mention security. However, I have found that as my knowledge increased, there was a dangerous tendency to obsess over tiny details relating to both scope and implementation and not produce anything at all.

"Will my peers think I am fool for using a TABLE element to display this on-screen calendar?"

"This web service I am creating isn't really RESTful, but if I make it RESTful it'll be really slow."

"Hmm. That query on a million records took over 75ms. Slow. :("

"This won't scale very well with more than 1000 concurrent requests. :("

"Yes, we're creating an online pizza shop but what if we want to support tapas or greek food as well? We may want to the ability to sell mountain bikes and custom coffee mugs using the same software at the some point...."

Would an engineer design a small, single lane bridge for a rural Northumberland village so that it could support the weight of a thousand double decker buses? No. So why do we, as software engineers try to do exactly this? That day will never come.

Why expend effort over engineering software in places that you don't yet know are important? Sure, it'd be nice to get a background task running in one second rather than ten - but if it's a one off nightly scheduled job, does it matter? 

I have come to the conclusion that there is no shame in producing well considered, simple, fit-for-purpose code that is just good enough.

Good enough doesn't imply half-arsed or lashed together. It should concisely meet the requirements at hand, not what you think the requirements might be next week. It doesn't mean you are naive and haven't considered the big picture, nor are you lazy or stupid. It doesn't mean you are a moron if you don't use wildcard generics and don't have a fetish for multiple inheritance.

I believe all developers should have a geek valve that prevents them from introducing overly-generic, indecipherable black magic to a codebase. In conversation you would look a bit unusual if you insisted on using flowery language to express a point that could be adequately conveyed in more standard terms. Some people may miss your point. The fact that their grasp of English isn't as advanced as yours doesn't make them stupid. It means you aren't communicating efficiently. Why can't the same logic apply to code? Favour explicit and clear over clever.

Software evolves over time, in some cases decades. If the architecture is kept as simple as it can be and is easily understood by all, on-going maintenance and evolution is likely to be a hell of a lot less risky. It will be more likely to be undertaken in a reasonable timeframe by anyone on the team. To me, that's true extensibility.

Almost two years ago, I had the idea for a mobile app that would let me know when the next Tyne and Wear Metro would be at my local stop. This meant I could decide whether to stroll to the station, or whether I needed to get a move on in order not to miss my train.

I hacked together an app for Android, as I was using a T-Mobile G1 at the time. It contained a simple database that I had manually entered from the timetables on the Metro web site. After using it daily for a few weeks, I realised that this was something that other people would find useful.I have long believed that the best solutions are those the author builds to make their own life easier, not primarily to make money or show off technical prowess.

After chatting with Paul and Jon at Never Odd or Even LLP, we decided to jointly take the idea to the Metro operator Nexus, with the view of being commissioned to produce an official Metro times app for the dominant mobile platforms. This proved to be a dead end, although they were supportive of the idea.

We didn't want to give up, but if the app wasn't going to official, we faced obstacles. Not just anyone can publish copyrighted timetable information in a mobile app. Although I'd like to think that my nifty Objective-C and Java code alone would be well worth the £1.79 asking price, I'm not stupid. It's all about the content and how you deliver it to the user. 

If I had finished the prototype app, we would have been selling content we did not own. TfL famously takes a hard line on any unofficial apps that use the Tube feeds without licence. Although the Metro operators are smaller, there is no reason why they wouldn't protect their IP. We did not want to waste time producing an app that would potentially get removed from the app store, or worse, face legal action.

The solution, discovered by Paul, was to licence the data from the Traveline NextBuses API which is used by many other mobile apps. Despite the name NextBuses, the system also contains Metro times. Next Metro, as it had become known, would need to be a client to this system, via an API, rather than timetable data being stored in the app.

The advantage to this approach was that we didn't need to maintain our own timetable database and have to enter bank holiday or engineering work deviations. The disadvantage was the app would be considerably slower as it would need to communicate with a server. One other drawback was that the API did not deliver live train data, just timetable information relative to the request time.The whereabouts of trains is not exposed anywhere outside of the Nexus system, so it was simply not possible for us to provide this.  If, however, this one day changes and the NextBuses API receives live train data - the app will show these, without modification. This has proven to be less of an issue than we first thought it would be as the Metro system does keep to timetable remarkably well. (Edit: Apart from the day after I posted this... people have been stealing live overhead cabling again ... 1500V DC ... braver men than I.)

We decided to first target the iPhone. This was absolutely the correct thing to do. Whether or not you like the iPhone or Apple, you would be hard pressed to argue that developers don't stand the best chance of making a small (or large) income on this platform. iPhone users buy the most apps and the App Store does an excellent job at promoting and delivering small developers' work to mass audiences, even if Next Metro was incredibly niche. You can't get more niche than a train times app for a slightly rickety, thirty-one year old regional train system somewhere to the North East of a small island.

The app did really well. It won an award. The income, even after royalties and costs, definitely exceeded my expectations. Not a notable fraction of a month's salary, but still nice enough. Not everyone saw the point - there was some inevitable critcism about the app not showing live train data and some thought it was pointless when you could download a PDF from the Nexus website. And we were charging for it! The cheek! Horses for courses. Better to delight the people who would find the app useful than to try and win over those who never will. 

When we released the iPhone version a year ago, Android users were by far the most vocal at requesting a version for their handsets. After politely ignoring them, about a month ago I decided to finally port the app to Android, largely as an experiment to see how the app would do commercially on what many believe to be the dominant mobile platform. Although I choose to use iOS and really like developing for it (I spend a lot of my day job in Xcode), developing for Android is great too. In many ways easier than iOS. I'm a techy at heart and don't really understand religious wars over technology. It's all good. Unless it's a touch screen BlackBerry.

So there you go. Thank you for reading this little ramble about the history of a small mobile app, which is now on two platforms. I can't wait to see how well the Android version does.

Next Metro

Next Metro for iPhone

Next Metro for Android

(download)

Click here to download:

next-metro-a-bit-of-history-and-android-release-zIiqwucizCkgmHdHJIIe.zip (74 KB)

I am currently handing over the system that I have worked on for the past three years. It is written in ColdFusion. 

I have noticed two things:

• some developers are initially wary of something less familiar, while some are keen to embrace something new
• an implicit assumption is made that you must be wedded to that platform you work in: "I am a .NET developer so you must be a ColdFusion developer."

In actual fact, I see myself as just being a developer.

Someone actually remarked that they were surprised that my new employer made use of ColdFusion. They don't, of course. Do developers get typecast? 

I develop in many different languages and frameworks. Java, mainly, but also C#, T-SQL, Python, Objective-C and Cocoa. Does this versatility make me a jack of all trades and a master of none? Some would say yes.

I just prefer to consider myself as someone who is not shackled to any particular platform. Us developers have transferrable skills which mature throughout our careers. Implementation languages and frameworks are only half of the story. I'm not saying that it is wrong to specialise and become expert in something. I am not saying it is wrong to specialise. I'm saying it's wrong to deny the existence of any other technology. 

I understand why management favour a unified and supported technology path. There's less risk. It makes sense. If you have a pool of .NET developers available, in theory they should be able to maintain each others code. 

It can, however, be a ball and chain, particularly in a mobile context. 

Successful companies and individuals will think outside their bubble and comfort zones.

It's simple economics. It makes sense to write software for the iPhone because it's huge at the moment. The tooling and documentation for the iPhone SDK are excellent.

Does this mean I'll burst into tears as Android's popularity continues to rise and the iPhone's popularity inevitably wanes and Apple go bankrupt (despite the billions) if we are to believe many of people I follow on Twitter? Will I have wasted my life in Xcode, typing square brackets? Not at all. I won't have called myself an iPhone developer. I'll still have been just a developer - developing for iPhone, Android, BlackBerry and ... whatever else.

Being a mobile developer in 2010 will be very different to being one in 2015, that's for sure.

Having sung the virtues of Google Web Toolkit for well over a year, I did an experiment. I rewrote a section of a prototype GWT application using JavaScript and YUI. It took me less than two hours, including time to consult the excellent documentation. The YUI version was deployed this afternoon.

Those of you who follow me on Twitter will be used to my occasional rants about web development. I love my job, but almost every day think that there has got to be a better way to do things, particularly on the client-side.

As we all know, the web started out as a way of displaying documents, regardless of computer type. This collection of hyperlinked documents somehow managed to become the runtime of the 21st century. It doesn't matter if you run OS X, Linux, Windows - a vast amount of what you do on a computer these days is in a web browser. Fact. Don't believe me? Unplug yourself from the network - turn off your wireless card. See how long you last.

Due to our reliance on the browser, JavaScript has become the world's most popular language.

Douglas Crockford, creator discoverer of JSON and author of one of the few decent JS books excellently sums up the misinformation and understanding that surrounds JavaScript. "...JavaScript has more in common with functional languages like Lisp or Scheme than with C or Java. It has arrays instead of lists and objects instead of property lists. Functions are first class. It has closures."

So why on earth did I attempt to write a web front-end in an arguably inferior language: Java, with the Google Web Toolkit?

I'll hold my hands up. Despite being able to churn out a lot of JavaScript code, I guess I didn't know JavaScript.

When Google released the Google Web Toolkit sometime in 2006, I was intrigued. I had dabbled with creating AJAX-y front ends of my own but almost consistently had ended with very brittle code. $$ and $A and $ functions made me feel unusual. Debugging was impossible. Things would stop working - yet no error messages would be given. With this in mind, GWT made perfect sense. A good UI widget library, strong typing, a compilation step to catch errors early, debugging, excellent IDEs, optimized code for each browser masking quirks and behaviour variation..... oh my, I was sold.

I enjoyed the rigidity of working in Java for client as well as server-side work. Using Eclipse I was soon refactoring frequently, being forced to define interfaces, unit testing, creating mock objects, using dependency injection, creating components, thinking in terms of events rather than forms and URLs... the list goes on and on.

However, in the last week I came to the realization that, with a bit of discipline, I could apply identical development approaches when using plain old JavaScript and YUI. For every perceived advantage of GWT, JavaScript had an answer. Unfortunately, GWT brought disadvantages of its own. 

• The built in widget library - calendars, drop down menus, trees, etc - whilst comprehensive, was fairly basic and hard to skin with CSS. Remember the promise of being shielded from browser quirks and variations? Forget it. Any web front-end library is all about the widgets. YUI (and others) are arguably superior.
• Compiling five versions for each supported browser proved to be very slow: 20 seconds on a 2.26Ghz MacBook Pro for a very small app. This may sound like a minor thing, but it did slow the development process down considerably. The rapid edit-refresh-edit development model of the web is extremely productive - live in-browser CSS and JavaScript editing are now also becoming common place. Some advantages of the compilation step can be matched by using a tool like jslint. jsunit also allows in-browser unit testing of JavaScript code.
• Eclipse is a fine Java IDE and the Google plug-in is an excellent addition. Code generation, hinting and completion were all welcome additions having come from a text editor. Refactoring GWT code within Eclipse was something I did frequently - although ultimately my use boiled down to a semi-intelligent search and replace. My time with GWT reminded me how important refactoring is. 
• Debugging within Eclipse only worked when running the Java code - you were not debugging the compiled JavaScript. There were differences between what happens in hosted mode and what happens in the browser. In addition, native JavaScript debugging within the browser has come a long way - particularly in recent versions of Safari and Firebug.
• Browser speed. It was easy to forget that one is targeting a web browser. It is natural to think in terms of components, event listeners and forget about the crazy HTML that is being produced somewhere. Although this is the logical assumption to make when using GWT, it is a mistake - the result is extremely slow code. Whilst this isn't such an issue in modern browsers, it is if you want your code to run well in Internet Explorer. For instance: a complicated list of objects took 20ms to render in Safari and 4000ms to render in IE7. 4000ms is unacceptable. The solution?  Render HTML in a stringbuffer (or on server) and use element.innerHTML = ".." to inject it onto the page. You throw away all the object-orientated/component/event goodness. The issues with speed are no doubt due to the slowness of DOM manipulation routines in certain browsers which aren't GWT's fault. Regardless, this low-level work around defeats the purpose of GWT to my mind.
• The event bus pattern I previously blogged about is trivial to achieve in JavaScript with a lot less code. Events are core to JavaScript within a browser. Custom event models are extremely easy to implement with a good library such as YUI or prototype.js. Whilst the GWT HandlerManager worked fine, I felt a disproportionately large amount of code was needed. 
• JavaScript can be fabulous. It has closures. More can be done in less code. A callback object can be defined inline, in literal form: { onSuccess: function(e) {}, onError: function(e) {} }. I intend to elaborate on this further with some some side-by-side GWT vs YUI code comparisons in a future post.

Maybe I'll be back in a future post saying that I didn't understand GWT (or Java) well enough. GWT will continue to evolve and Google Wave, which is built using GWT, will no doubt be a seriously impressive web front-end. 

GWT, particularly the Java-JavaScript cross compiler, is a stunning feat of engineering. Maybe the coolness of the hack is what attracted me to GWT in the first place. But like so many tools made by developers for developers, the beauty lies within - in this case the API and the compiler. The over-engineered end result just doesn't do all of the effort and cleverness justice.

There is much I like and admire about GWT. I may use it again. Regardless of whether I use it in future projects, I have GWT to thank for forcing me to apply proper software engineering approaches to web client development. 

Bespoke developments for internal use can fall into many traps. 

Technically, it is harder to justify refactoring or behind the scenes code improvements to your client or internal customer -- the completely reasonable arguments being "it works (albeit slowly); we cannot risk instability; we need new features". As a result, there is a temptation to write, or build upon poorly structured code to get things done.

From a user interface perspective, people are trained to use the system. They are a captive audience who cannot go off and use something else so often internal system UIs are fairly abysmal. The hideousness of the SAP web-front end proves this point. This is inexcusable - if people are being forced to use a system as part of their job for hours every day, extra effort should be made to ensure the system is user friendly and works with them in doing their job. A bit like having a comfy office chair that doesn't give you backache after half an hour.

Functionally, internal systems risk becoming blunt swiss army knives - they try hard be all things to all users yet end up doing nothing well. Even if you don't plan to commercialise or sell your software, I have learnt that it makes sense to consider every change request as if the system was being developed with real, paying customers in mind.

It sounds obvious but actually putting a price on tasks, considering the ROI and the number of users who would benefit makes it possible to plan changes more effectively. Competitive commercial edge, even if imaginary or indirect, is an important driver in ensuring a system moves forward.The early adopters of the system often get confused as to why their 'order' of a few feature is not implemented. An internal system with five users is a different prospect to one with several hundred. 

Users are the lifeblood of a system should be listened to, but it can be dangerous for them to be in charge. Did you see what happened when Homer Simpson's brother let him design his dream car?

When a software project is ready to release, there is a temptation to keep polishing and refining, rather than releasing it for consumption and inevitable criticism. A bit like a hen who doesn't want their eggs to hatch.

Bad similes aside, it is wrong to perpetually incubate work where it is out of harms way, under the pretence of being a perfectionist or having exceptionally high standards. In reality this could be indicative of a lack of confidence in what is being produced.

It is vital to set high standards. However the "release early and often" rule is still a good one. I have found that when code is put out there scrutiny it improves rapidly. This is obvious. Attention is paid to the right places, not where developers think the improvements need to be made. And when the code does go live to the world, it can be done so with utmost confidence.