Back when I started developing what could loosely be called software around fifteen years ago, I didn't know what I was doing. If it compiled, ran and produced (mostly) the expected results, then the job was done. 

As a new programmer, I was immensely productive. 

Of course, problems came when it was time to fix bugs or extend the software. It was often easier to just start again than to try and understand the rat's nest of poorly structured and unintelligible code. 

Fast forward ten years and I have been fortunate enough to have had exposure to some huge, well architected and complicated systems, not to mention some extremely clever people. As a result I realise I knew nothing back then. In another ten years I'll probably think the same about my current knowledge and ability. But that's the nature of software engineering. You never stop learning and evolving.

I do however miss those naive days of being able to crank out code with such velocity. It was fun back then. Yeah, what I did then might have been inefficient and probably quite flawed. But for the most part, it worked and served a purpose.

Back in those days of youthful ignorance I didn't have the experience to know what could go wrong when I hit the database fifty times to service a request. That was learnt the hard way. I just ploughed ahead, thinking in functional terms about how the application would behave for a user. I didn't worry about code being performant or extensible or even dream of coming up with my own framework. No yak shaving, I just focused on the task in hand. This was perhaps a good mindset to have.

A decade later, I am not suggesting that we should lash things together in this way. It'd be moronic to suggest we shun our collective experience, knowledge of patterns and the advanced features of programming languages. Not to mention security. However, I have found that as my knowledge increased, there was a dangerous tendency to obsess over tiny details relating to both scope and implementation and not produce anything at all.

"Will my peers think I am fool for using a TABLE element to display this on-screen calendar?"

"This web service I am creating isn't really RESTful, but if I make it RESTful it'll be really slow."

"Hmm. That query on a million records took over 75ms. Slow. :("

"This won't scale very well with more than 1000 concurrent requests. :("

"Yes, we're creating an online pizza shop but what if we want to support tapas or greek food as well? We may want to the ability to sell mountain bikes and custom coffee mugs using the same software at the some point...."

Would an engineer design a small, single lane bridge for a rural Northumberland village so that it could support the weight of a thousand double decker buses? No. So why do we, as software engineers try to do exactly this? That day will never come.

Why expend effort over engineering software in places that you don't yet know are important? Sure, it'd be nice to get a background task running in one second rather than ten - but if it's a one off nightly scheduled job, does it matter? 

I have come to the conclusion that there is no shame in producing well considered, simple, fit-for-purpose code that is just good enough.

Good enough doesn't imply half-arsed or lashed together. It should concisely meet the requirements at hand, not what you think the requirements might be next week. It doesn't mean you are naive and haven't considered the big picture, nor are you lazy or stupid. It doesn't mean you are a moron if you don't use wildcard generics and don't have a fetish for multiple inheritance.

I believe all developers should have a geek valve that prevents them from introducing overly-generic, indecipherable black magic to a codebase. In conversation you would look a bit unusual if you insisted on using flowery language to express a point that could be adequately conveyed in more standard terms. Some people may miss your point. The fact that their grasp of English isn't as advanced as yours doesn't make them stupid. It means you aren't communicating efficiently. Why can't the same logic apply to code? Favour explicit and clear over clever.

Software evolves over time, in some cases decades. If the architecture is kept as simple as it can be and is easily understood by all, on-going maintenance and evolution is likely to be a hell of a lot less risky. It will be more likely to be undertaken in a reasonable timeframe by anyone on the team. To me, that's true extensibility.

Permalink | Leave a comment  »

Almost two years ago, I had the idea for a mobile app that would let me know when the next Tyne and Wear Metro would be at my local stop. This meant I could decide whether to stroll to the station, or whether I needed to get a move on in order not to miss my train.

I hacked together an app for Android, as I was using a T-Mobile G1 at the time. It contained a simple database that I had manually entered from the timetables on the Metro web site. After using it daily for a few weeks, I realised that this was something that other people would find useful.I have long believed that the best solutions are those the author builds to make their own life easier, not primarily to make money or show off technical prowess.

After chatting with Paul and Jon at Never Odd or Even LLP, we decided to jointly take the idea to the Metro operator Nexus, with the view of being commissioned to produce an official Metro times app for the dominant mobile platforms. This proved to be a dead end, although they were supportive of the idea.

We didn't want to give up, but if the app wasn't going to official, we faced obstacles. Not just anyone can publish copyrighted timetable information in a mobile app. Although I'd like to think that my nifty Objective-C and Java code alone would be well worth the £1.79 asking price, I'm not stupid. It's all about the content and how you deliver it to the user. 

If I had finished the prototype app, we would have been selling content we did not own. TfL famously takes a hard line on any unofficial apps that use the Tube feeds without licence. Although the Metro operators are smaller, there is no reason why they wouldn't protect their IP. We did not want to waste time producing an app that would potentially get removed from the app store, or worse, face legal action.

The solution, discovered by Paul, was to licence the data from the Traveline NextBuses API which is used by many other mobile apps. Despite the name NextBuses, the system also contains Metro times. Next Metro, as it had become known, would need to be a client to this system, via an API, rather than timetable data being stored in the app.

The advantage to this approach was that we didn't need to maintain our own timetable database and have to enter bank holiday or engineering work deviations. The disadvantage was the app would be considerably slower as it would need to communicate with a server. One other drawback was that the API did not deliver live train data, just timetable information relative to the request time.The whereabouts of trains is not exposed anywhere outside of the Nexus system, so it was simply not possible for us to provide this.  If, however, this one day changes and the NextBuses API receives live train data - the app will show these, without modification. This has proven to be less of an issue than we first thought it would be as the Metro system does keep to timetable remarkably well. (Edit: Apart from the day after I posted this... people have been stealing live overhead cabling again ... 1500V DC ... braver men than I.)

We decided to first target the iPhone. This was absolutely the correct thing to do. Whether or not you like the iPhone or Apple, you would be hard pressed to argue that developers don't stand the best chance of making a small (or large) income on this platform. iPhone users buy the most apps and the App Store does an excellent job at promoting and delivering small developers' work to mass audiences, even if Next Metro was incredibly niche. You can't get more niche than a train times app for a slightly rickety, thirty-one year old regional train system somewhere to the North East of a small island.

The app did really well. It won an award. The income, even after royalties and costs, definitely exceeded my expectations. Not a notable fraction of a month's salary, but still nice enough. Not everyone saw the point - there was some inevitable critcism about the app not showing live train data and some thought it was pointless when you could download a PDF from the Nexus website. And we were charging for it! The cheek! Horses for courses. Better to delight the people who would find the app useful than to try and win over those who never will. 

When we released the iPhone version a year ago, Android users were by far the most vocal at requesting a version for their handsets. After politely ignoring them, about a month ago I decided to finally port the app to Android, largely as an experiment to see how the app would do commercially on what many believe to be the dominant mobile platform. Although I choose to use iOS and really like developing for it (I spend a lot of my day job in Xcode), developing for Android is great too. In many ways easier than iOS. I'm a techy at heart and don't really understand religious wars over technology. It's all good. Unless it's a touch screen BlackBerry.

So there you go. Thank you for reading this little ramble about the history of a small mobile app, which is now on two platforms. I can't wait to see how well the Android version does.

Next Metro

Next Metro for iPhone

Next Metro for Android

See the full gallery on Posterous

Permalink | Leave a comment  »

I am currently handing over the system that I have worked on for the past three years. It is written in ColdFusion. 

I have noticed two things:

• some developers are initially wary of something less familiar, while some are keen to embrace something new
• an implicit assumption is made that you must be wedded to that platform you work in: "I am a .NET developer so you must be a ColdFusion developer."

In actual fact, I see myself as just being a developer.

Someone actually remarked that they were surprised that my new employer made use of ColdFusion. They don't, of course. Do developers get typecast? 

I develop in many different languages and frameworks. Java, mainly, but also C#, T-SQL, Python, Objective-C and Cocoa. Does this versatility make me a jack of all trades and a master of none? Some would say yes.

I just prefer to consider myself as someone who is not shackled to any particular platform. Us developers have transferrable skills which mature throughout our careers. Implementation languages and frameworks are only half of the story. I'm not saying that it is wrong to specialise and become expert in something. I am not saying it is wrong to specialise. I'm saying it's wrong to deny the existence of any other technology. 

I understand why management favour a unified and supported technology path. There's less risk. It makes sense. If you have a pool of .NET developers available, in theory they should be able to maintain each others code. 

It can, however, be a ball and chain, particularly in a mobile context. 

Successful companies and individuals will think outside their bubble and comfort zones.

It's simple economics. It makes sense to write software for the iPhone because it's huge at the moment. The tooling and documentation for the iPhone SDK are excellent.

Does this mean I'll burst into tears as Android's popularity continues to rise and the iPhone's popularity inevitably wanes and Apple go bankrupt (despite the billions) if we are to believe many of people I follow on Twitter? Will I have wasted my life in Xcode, typing square brackets? Not at all. I won't have called myself an iPhone developer. I'll still have been just a developer - developing for iPhone, Android, BlackBerry and ... whatever else.

Being a mobile developer in 2010 will be very different to being one in 2015, that's for sure.

Permalink | Leave a comment  »

Tonight I will be speaking at SuperMondays about mobile development, with a live coding demo showing the build of a simple app for #Android powered devices. I hope to keep a good part of the talk generic so it appeals to as many as possible. If you're going, I hope it proves useful. There's a fair bit to get through but feel free to comment/shout/agree/disagree at any point during the talk.

Update: Here are my slides.

Exciting times. I am collaborating on some shiny new mobile projects of all shapes and sizes, planned for release in the coming months.

Also, as of yesterday, there's a new North East mobile developers group - follow @appnorth on Twitter for more information. First meeting in March.

Permalink | Leave a comment  »

Ask the Hoff is now available for Android phones! Scan the barcode below to download it!

I have been meaning to release a mobile app for some time, but wanted to wait until a novel enough idea came along. I was impressed with the ultra-simplicity of Ask the Hoff, developed by Paul (of @twitchhiker fame) and Jon, both of Never Odd or Even LLP. Because the app was simple but not trivial, I figured it would feasible to complete development in about a day. So, after taking the Hoff's advice via my iPod, I offered to port it to Android for them. They accepted!

Despite being a keen Mac user, I've never had an iPhone, despite thinking my iPod Touch is one of the best things I have ever bought. Maybe I'm old fashioned but I actually like having a 1990s personal organiser keyboard attached to my phone. I still use one of the first G1s to come out and have been a fan of the Android platform ever since: it has evolved rapidly in the space of a year. 

Android apps are programmed in Java inside the Eclipse IDE, despite not running in a JVM on the device itself. This is an environment I am extremely comfortable in. Overall, I am extremely impressed with the entire Android development experience: the IDE integration, debugger, emulator, the SDK and the extremely readable documentation on Android.com. The Eclipse integration is particularly good, although the GUI for laying out inter faces is nowhere near the slickness of Interface Builder. I generally edited the user interface layout XML directly. I will blog in more detail on the Android development experience later.

I look forward to working with Paul and Jon again (shame I'm not called George, then we'd only be missing a Ringo...) on future Android projects, as well as perhaps releasing a few apps of my own.

Edit: Forgot to say - many, many thanks to all the people who tested various builds of Hoff earlier this week.

Permalink | Leave a comment  »

Cleanly defining and managing the flow of screens in web applications, particularly multi-step processes or workflows can be challenging. Due to the stateless nature of HTTP, state has to be managed through cookies, hidden fields, URLs or session data held on the server. What if the user clicks the back button? Meh.

I was thinking, why can't we model web application flow like the monolithic programs we knocked together when we were young? (Or was that just me?)

10 PRINT "What is your name?"
20 INPUT NAME$
30 PRINT "Hello, ";NAME$
RUN

What is your name?
? Alex
Hello, Alex

It turns out I hadn't invented anything. An approach known as a continuation can make this happen. The Apache Cocoon and Seaside framework do just this in a web setting and this article from IBM describes it very well. A continuation is effectively a way of freezing code execution mid flow and squirreling it away to be continued later. When that happens, all prior state/variables are as they were and the code can continue to run, unaware that anything happened.

Somewhat like hibernating your laptop, rather than shutting it down, perhaps.

As it happens, this approach fits well with the HTTP request/response cycle. Code runs, needs more input - continuation stored and HTML form displayed. User supplies more input and clicks submit. Server runs continuation and does some processing - decides more data is needed, continuation stored and another HTML form displayed.

I imagine this kind of monolithic flow would make for very readable business process implementation. I like the idea of the logic of a process is defined as simply as possible in a single location, not scattered across user controls and databases.

One to ponder. This could be one of those cases where a disproportionate amount of complexity is introduced to make something else simpler. Maybe the way we program for the web isn't perfect, but isn't horribly broken either.

Permalink | Leave a comment  »

My view - your business logic always lives on the server.

User-interface JavaScript running on the client should essentially be a slave to the server. Any decisions or calculations must be made on the server.

Building logic in JavaScript is tempting. It's often fast as no server roundtrip is involved. It is very easy to code as you're not having to worry about shipping values to and from the server.

However, when running on the client you have little control over faulty language implementations or, more likely, people manipulating your code and XHR payloads with tools such as Firebug. 

If the server blindly accepts what it is given by the client, you're in trouble. In Web 1.0 terms, this is synonymous with having a form with only JavaScript validation - a browser without JavaScript enabled would be able to submit invalid data.

Always running logic on the server has a host of advantages in addition to closing security holes. You are in control of the runtime and can reliably test its behaviour to be consistent. You only need to test your deployment runtime, not every browser you plan to support. From a performance point of view, the .NET or Java runtimes are likely to be considerably faster than JavaScript running on a client PC.

Some applications could run logic on both the client and server but this may result in duplication. Offloading processing to the browser could make an application more scalable. However, logic on the client should be considered supplemental and should only invested in where there is tangible benefit. 

Permalink | Leave a comment  »

As developers we continually seek to improve our game. In good teams, people share approaches and patterns.

One approach I use a lot is to write each step of functionality as a one line comment. After all the steps have been 'defined' as comments, I review and refine them - possibly changing the order and refactoring before any code has been written. Once I am happy with the comments, I find that filling in the implementation between the comments is easy.

It keeps code focused and forces me to consider the whole problem before diving straight in.

I obviously haven't invented this approach but it works for me. If you prefix the comments with TODO: in Eclipse, it populates the To Do window. Neat.

So that's one oldie that works for me. What approaches along these lines would YOU recommend?

Permalink | Leave a comment  »

Having sung the virtues of Google Web Toolkit for well over a year, I did an experiment. I rewrote a section of a prototype GWT application using JavaScript and YUI. It took me less than two hours, including time to consult the excellent documentation. The YUI version was deployed this afternoon.

Those of you who follow me on Twitter will be used to my occasional rants about web development. I love my job, but almost every day think that there has got to be a better way to do things, particularly on the client-side.

As we all know, the web started out as a way of displaying documents, regardless of computer type. This collection of hyperlinked documents somehow managed to become the runtime of the 21st century. It doesn't matter if you run OS X, Linux, Windows - a vast amount of what you do on a computer these days is in a web browser. Fact. Don't believe me? Unplug yourself from the network - turn off your wireless card. See how long you last.

Due to our reliance on the browser, JavaScript has become the world's most popular language.

Douglas Crockford, creator discoverer of JSON and author of one of the few decent JS books excellently sums up the misinformation and understanding that surrounds JavaScript. "...JavaScript has more in common with functional languages like Lisp or Scheme than with C or Java. It has arrays instead of lists and objects instead of property lists. Functions are first class. It has closures."

So why on earth did I attempt to write a web front-end in an arguably inferior language: Java, with the Google Web Toolkit?

I'll hold my hands up. Despite being able to churn out a lot of JavaScript code, I guess I didn't know JavaScript.

When Google released the Google Web Toolkit sometime in 2006, I was intrigued. I had dabbled with creating AJAX-y front ends of my own but almost consistently had ended with very brittle code. $$ and $A and $ functions made me feel unusual. Debugging was impossible. Things would stop working - yet no error messages would be given. With this in mind, GWT made perfect sense. A good UI widget library, strong typing, a compilation step to catch errors early, debugging, excellent IDEs, optimized code for each browser masking quirks and behaviour variation..... oh my, I was sold.

I enjoyed the rigidity of working in Java for client as well as server-side work. Using Eclipse I was soon refactoring frequently, being forced to define interfaces, unit testing, creating mock objects, using dependency injection, creating components, thinking in terms of events rather than forms and URLs... the list goes on and on.

However, in the last week I came to the realization that, with a bit of discipline, I could apply identical development approaches when using plain old JavaScript and YUI. For every perceived advantage of GWT, JavaScript had an answer. Unfortunately, GWT brought disadvantages of its own. 

• The built in widget library - calendars, drop down menus, trees, etc - whilst comprehensive, was fairly basic and hard to skin with CSS. Remember the promise of being shielded from browser quirks and variations? Forget it. Any web front-end library is all about the widgets. YUI (and others) are arguably superior.
• Compiling five versions for each supported browser proved to be very slow: 20 seconds on a 2.26Ghz MacBook Pro for a very small app. This may sound like a minor thing, but it did slow the development process down considerably. The rapid edit-refresh-edit development model of the web is extremely productive - live in-browser CSS and JavaScript editing are now also becoming common place. Some advantages of the compilation step can be matched by using a tool like jslint. jsunit also allows in-browser unit testing of JavaScript code.
• Eclipse is a fine Java IDE and the Google plug-in is an excellent addition. Code generation, hinting and completion were all welcome additions having come from a text editor. Refactoring GWT code within Eclipse was something I did frequently - although ultimately my use boiled down to a semi-intelligent search and replace. My time with GWT reminded me how important refactoring is. 
• Debugging within Eclipse only worked when running the Java code - you were not debugging the compiled JavaScript. There were differences between what happens in hosted mode and what happens in the browser. In addition, native JavaScript debugging within the browser has come a long way - particularly in recent versions of Safari and Firebug.
• Browser speed. It was easy to forget that one is targeting a web browser. It is natural to think in terms of components, event listeners and forget about the crazy HTML that is being produced somewhere. Although this is the logical assumption to make when using GWT, it is a mistake - the result is extremely slow code. Whilst this isn't such an issue in modern browsers, it is if you want your code to run well in Internet Explorer. For instance: a complicated list of objects took 20ms to render in Safari and 4000ms to render in IE7. 4000ms is unacceptable. The solution?  Render HTML in a stringbuffer (or on server) and use element.innerHTML = ".." to inject it onto the page. You throw away all the object-orientated/component/event goodness. The issues with speed are no doubt due to the slowness of DOM manipulation routines in certain browsers which aren't GWT's fault. Regardless, this low-level work around defeats the purpose of GWT to my mind.
• The event bus pattern I previously blogged about is trivial to achieve in JavaScript with a lot less code. Events are core to JavaScript within a browser. Custom event models are extremely easy to implement with a good library such as YUI or prototype.js. Whilst the GWT HandlerManager worked fine, I felt a disproportionately large amount of code was needed. 
• JavaScript can be fabulous. It has closures. More can be done in less code. A callback object can be defined inline, in literal form: { onSuccess: function(e) {}, onError: function(e) {} }. I intend to elaborate on this further with some some side-by-side GWT vs YUI code comparisons in a future post.

Maybe I'll be back in a future post saying that I didn't understand GWT (or Java) well enough. GWT will continue to evolve and Google Wave, which is built using GWT, will no doubt be a seriously impressive web front-end. 

GWT, particularly the Java-JavaScript cross compiler, is a stunning feat of engineering. Maybe the coolness of the hack is what attracted me to GWT in the first place. But like so many tools made by developers for developers, the beauty lies within - in this case the API and the compiler. The over-engineered end result just doesn't do all of the effort and cleverness justice.

There is much I like and admire about GWT. I may use it again. Regardless of whether I use it in future projects, I have GWT to thank for forcing me to apply proper software engineering approaches to web client development. 

Permalink | Leave a comment  »

Bespoke developments for internal use can fall into many traps. 

Technically, it is harder to justify refactoring or behind the scenes code improvements to your client or internal customer -- the completely reasonable arguments being "it works (albeit slowly); we cannot risk instability; we need new features". As a result, there is a temptation to write, or build upon poorly structured code to get things done.

From a user interface perspective, people are trained to use the system. They are a captive audience who cannot go off and use something else so often internal system UIs are fairly abysmal. The hideousness of the SAP web-front end proves this point. This is inexcusable - if people are being forced to use a system as part of their job for hours every day, extra effort should be made to ensure the system is user friendly and works with them in doing their job. A bit like having a comfy office chair that doesn't give you backache after half an hour.

Functionally, internal systems risk becoming blunt swiss army knives - they try hard be all things to all users yet end up doing nothing well. Even if you don't plan to commercialise or sell your software, I have learnt that it makes sense to consider every change request as if the system was being developed with real, paying customers in mind.

It sounds obvious but actually putting a price on tasks, considering the ROI and the number of users who would benefit makes it possible to plan changes more effectively. Competitive commercial edge, even if imaginary or indirect, is an important driver in ensuring a system moves forward.The early adopters of the system often get confused as to why their 'order' of a few feature is not implemented. An internal system with five users is a different prospect to one with several hundred. 

Users are the lifeblood of a system should be listened to, but it can be dangerous for them to be in charge. Did you see what happened when Homer Simpson's brother let him design his dream car?

Permalink | Leave a comment  »

Permalink | Leave a comment  »

When a software project is ready to release, there is a temptation to keep polishing and refining, rather than releasing it for consumption and inevitable criticism. A bit like a hen who doesn't want their eggs to hatch.

Bad similes aside, it is wrong to perpetually incubate work where it is out of harms way, under the pretence of being a perfectionist or having exceptionally high standards. In reality this could be indicative of a lack of confidence in what is being produced.

It is vital to set high standards. However the "release early and often" rule is still a good one. I have found that when code is put out there scrutiny it improves rapidly. This is obvious. Attention is paid to the right places, not where developers think the improvements need to be made. And when the code does go live to the world, it can be done so with utmost confidence.

Permalink | Leave a comment  »