Happiness and other technical requirements – Clean slate

This post is the third part of a software development story. If you want to catch-up, here are the first and the second parts. Please make sure you read the disclaimer first.

After presenting a dysfunctional team, and then moving on to it’s recovery and improvement, we are now facing a defining challenge: a new project where the team could experiment their skills.

Even if the stakes from a business perspective where not that high, they were so for the team. It was a big bet on ourselves and on our newly found competences and attitudes.

The risk of getting something wrong was incredibly high, as we were moving in uncharted territory: developers were mainly PHP devs, who decided to write the backend in Ruby; automation testers were mainly Java experts, who transitioned to Cucumber and BDD; none of us had any experience developing for mobile, working with maps, storing on a No-SQL database or hosting on an cloud infrastracture.

So… dear reader, buckle-up, because it’ going to be a real roller-coaster!

Re-shaping the problem

A couple of business owners and a project manager approached us with a detailed, documented plan to extend an existing business application. The idea was to provide our door-to-door sales agents with a tablet, and to build them a digital, live version of the printed map and form which they were usually assigned.

The first thing we did was to heavily transform the project structure. Piece by piece we reshaped the waterfall plan in a more agile, iterative way. We convinced them to start with a prototype and evaluate the results. Mindful of our inherited mistakes, we put user satisfaction at the top. We challenged their business assumptions about a top-down plan.

Before even starting a proper discussion, we did something that shook them a bit: a couple of developers and our business analyst actually spent a day working with these door-to-door sales people, knocking at people doors, trying to sell our products.

It was a brilliant move. No-one likes door-to-door sales people. But that single act allowed us to empathise with them and their hard job. It gave us a totally new and fresh perspective on their real problems, and a concrete meaning behind what we were about to develop.

We used this new information to challenge some work items, introduce new ideas, and re-priotise the backlog.

We made the project our own. We developed a sense of ownership.

Mistakes

Everything was quite exciting and challenging, but only one part of the team was involved in these early explorations. When we started to make decisions that would affect the course of the project I saw the risk of not involving the rest of the team, but my colleagues were more eager to make progress quickly.

As mentioned in the previous post, as a Tech Lead I tried to have a very light touch and to rarely interfere in technical decisions. As we were figuring out what JavaScript framework to adopt, I felt both the pressure from our non-technical team mates to quickly decide and the danger of laying the wrong foundation for our codebase. I ended up pushing for my preferred option much more than I’d wanted.

I tried to explain later to the rest of the developers the reasons for my choice, and that I would have much preferred a collective approach. I reassured them that the project manager promised we would be given the opportunity to change framework and re-write our front-end if the prototype and the user trial proved we could have chosen better.

Nonetheless, along the course of the project that decision proved to be a real source of troubles. From a purely technical perspective the framework was well architected, flexible, and especially good at compensating some of our own weaknesses (such as visual design). But the learning curve was steep, and working with it was far from exciting. It required much more investment and motivation.

If there is something I learned from it is that you own the decisions you take, and being the only one involved in that decision, I become the only one with a sense of ownership over that piece of code.

I hope you dear readers noticed the irony in my mistake: I ended up relying on purely technical specs more than on collaboration and commitment.

I’ve asked myself for a long time what would I do differently and better if I had to face a similar critical situation. Once put in the right context, the answer is simple, and I hope by now you already figured it out: I just wouldn’t settle for an individual decision.

It really doesn’t matter what framework you adopt, if your whole team is not committed to it. And you can only commit to a decision if you have been involved in it and you have been allowed to voice your opinion.

Innovation

Once the whole team started working on the new project, the joy of having a clean slate and the curiosity for exploring new possibilities took over.

Being part of a big enterprise company, we inherited a slow and verbose release process. We already worked hard in the previous 6 months on our original project to speed it up by automating the manual steps and by removing technical obstacles for more frequent deployments.

With this new project we decided to move much further, adopting a Continous Deployment approach and a Continous Delivery strategy. Which essentially means we hugely invested and relied in automation tests and automated deployments.

Our deployment pipeline became a masterpiece of craftsmanship, a source for new experiments and learning, a backbone of our development process, but also a showcase of all our efforts and huge source of pride.

We supported our growing application with different kind of test suites: Cucumber/WebDriver feature tests, RSpec for our own REST API, Jasmine unit testing for JavaScript, integration testing and contract tests for external APIs.

All of these test suites were organically born out of real needs, so that we were efficiently testing only what was really required. For instance, we introduced a contract suite once we realised the external API we were using wasn’t reliable. We introduced JavaScript unit testing once the complexity of our asynchronous code become confusing.

Above all of them, a special place and importance had our Cucumber/BDD suite. It really was a point of intersection between the needs of developers and testers, between business requirements and technical features.

The test suites were distributed along multiple steps of our deployment pipeline, and executed on different automated environments, in a beautifully designed process that involved careful integration between different systems and technologies.

The Incredible Machine

A bit like one of those contraptions out of The Incredible Machine, everything started from a spark in our Git repo, moving through a set of jobs on our Jenkins CI server, crossing our farm of 20 virtual slaves running on VirtualBox/Vagrant and provisioned using Puppet, deployed on freshly spawn environments using CloudFoundry, smoke tested again via WebDriver, and finally ready for production.

I have a clear picture in my mind, a snapshot of this happy period. I’m leaving the office at the end of the day, and I notice that half of the team is still sitting at their desks. We have no deadlines, and no pressure to deliver yet. Still we sometimes work more than before just because we like what we are doing, staying late maybe just to finish that cool thing we were playing with. And I realise how far we’ve gone in the last 8 months, when, on the contrary, we had to work overtime on the weekends to meet an arbitrary deadline no-one really cared about.

Doubts

I left the team for one month, as I temporarily joined another team that needed some help with their delivery.

They say: hire great people, and get out of their way, and I always believed that the best thing a leader can do is to make oneself redundant.

But when I returned, in time for our regular team retrospective, things weren’t going that well.

Even if still no-one was using our app, more and more random bugs were caught by our testers, and we couldn’t identify the root cause.

The general mood during the retrospective was quite low, and people felt a lot of uncertainty.

With all this disruptive innovation going on, I found myself in the unexpected position of having to contain it a bit, or at least allow it to redistribute organically across the whole team, trying to make everyone feel comfortable with the fast pace.

Maybe because of the lack of my reassurance, or maybe just as a necessary phase, doubts creeped in.

As one of the developers emphatically expressed it at the end of the retrospective “essentially,  at this stage we have no confidence at all that this thing is going to work“.

What if the project turned out to be a failure?
Had we been too reckless?

You’ll find out in the next episode.

(suspense)

2 thoughts on “Happiness and other technical requirements – Clean slate

  1. Jonathan

    AAARGH ! you leave us hanging just as it was getting really interesting 🙂 Though the testing regimen sounds dreadfully complex !
    I’ve been there too, the energy and initial excitement meaning the whole team working long hours because it was fun – we were in a building with automatic lighting, so made a pact that we would fold up for the day when the lights went out. But our enthusiasm didn’t wane in the way you described – maybe that was because the team didn’t change as yours had to.
    Had they delivered anything during your absence?
    Looking forward to the next installment,
    Jonathan
    (Author of “Team Synergy Masterplan: 3 big challenges of IT teams”)

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s