Agile - Days of Future Past, Days of Future Present

On Monday night some colleagues and I attended a thought provoking presentation entitled The Future of Agile (If Any). I believe that agile patterns and practices are fundamental to the continued successful growth of an IT company, so hence the questions I left with felt weighty enough to share more widely:

1. What physical actions can you take today to increase the level of trust between your Development Team and the rest of your business?
2. When (not if) a major catastrophe is caused by software, how will we, in the IT Industry, answer the question: "how did you let this happen?"?

The link between these questions is a third:
3. What does the future hold for Agile Development Practices? Is it rosy or is bleak?

Uncle Bob, one of the founders of the Agile Manifesto, explored these questions for about an hour.

After his usual diversion into natural phenomena (this time it was interference patterns), Bob explored Agile's history, from it's humble beginnings to present day.

Kent Beck, at the creation of the Agile Manifesto, said: "We want to heal the divide between business and development". The group that met and formed the Manifesto recognised that this divide could be bridged by building trust, which in turn, is built by transparency. The original practices sought to provide this transparency.

One of my colleagues provided the definition of software as being soft "because you can squidge it about." TDD, Pairing, Refactoring and Simple Design help keep software soft. Bob stated that these practices could be replaced by others, but only if some core values (from Extreme Programming) could still be met:

• Simplicity
• Feedback
• Communication
• Courage
• Respect

In his opinion, TDD, for example, is the best way of getting feedback when developing code, but he welcomed any suggestions of alternative practices that still gave the same amount of feedback as TDD.

Bob recognised that Scrum has helped Agile move from the margin to mainstream, but questioned whether Scrum has allowed team autonomy and self-organisation to overrule minimum standards, practices and principles that give improve a software team's maturity (such as pairing and TDD). This was a particular concern for Agile's future, as, in his opinion, the community is very immature. This opinion was based on a rough and ready estimation that the number of developers world-wide is doubling every five years, and so hence the vast majority of developers in the world have less than 5 years experience.

He then looked to the future and posed the question: What major catastrophe will be caused by software? When it happens, how will Development defend itself against the accusation of "how did you let this happen?"?

On a positive note (in response to a question I asked) Bob stated that the disciplines encompassed by Agile Software Development are a positive step towards maturity (providing their adoption continues to spread and become the norm).

The main question I left with, was back to the original purpose of the manifesto: how do we increase the level of trust and transparency between Development and the rest of the business?

Experience report from the Kanban Coaching Exchange - Exploring Kanban through it's values

I enjoyed a great evening last night at the Kanban Coaching Exchange, hearing Mike Burrows explore Kanban via a value system. This post summarises my notes and observations from the evening.

Some sub-titles were:

• Kanban is "a humane start with what you do now approach to change"
• Kanban encourages "Leadership at every level"

We kicked of with some quick exercises to familiarise ourselves with the values (Mike's posted them on his blog). I enjoyed being part of a very like-minded group (including @drewpreston and @jose_casal), which finished the first exercise quickly and got full-marks :-) The beer and success must have gone to our heads though, as we missed 100% on the second :-(

The values were grouped into three Agendas:

1 - Sustainability

• Transparency
• Balance
• Collaboration

Collaboration is more than just "being nice to each other"; it's about creative relationships that, together, achieve something. Surprise, dissappointment or frustration in the team is often a sign that collaboration is missing or sub-optimal.

2 - Service Orientation

• Customer focus - how quickly does the team validate that they've met the customer's needs. Can they even do this (does the data exist and can they access it)?
• Flow
• Leadership (at every level)

3- Survivability

• Understanding
• Agreement
• Respect

These are disciplines (that take time and effort to do well), particularly relevant to making changes to a system/process. A related anti-pattern is "unsafe change", comprised of:

• Bravado (too fast)
• Complacency (too slow)
• Tampering (too random)

This was my first time at the meetup and I really enjoyed it. Great location, speaker, beer, pizza, discussion and people.

Team Safety

How safe is your team? Is team safety important?

My colleague raised these excellent questions in a recent interview we conducted for our Scrum-master vacancy. Then this week I was challenged by some uncomfortable, but important, questions. Unfortunately, this was before I read a thought-for-the-week article on how Jesus invited questions. All this got me thinking about why team safety is important, and how we can improve it.

http://iversonforest.com/safetyteam.html

Why is safety important?

When people feel safe in a team, they are not afraid to ask questions. These may be curious, or knowledge-seeking questions, or they may be challenging, uncomfortable questions. Either way, these kinds of questions benefit both the individuals and the team. Such questions help build and spread knowledge as well as improve artefacts, processes and behaviours.

Safety can also be viewed as the absence of fear. People tend to be afraid of making mistakes, of getting things wrong. But we often learn by our mistakes. It’s only in taking risks that we make new discoveries (think of Columbus, Neil Armstrong) and move things forward. When people feel safe in a team, they have the confidence to step out of their comfort zone and take a risk, even if it leads to failure. They know that their team-members will support them, whatever the outcome.

How can we improve team safety?

http://today.slac.stanford.edu/feature/2009/safe09-site-office.asp

It's got to be more than just issuing hard-hats and fluro-vests, or books and training courses.

People feel safe when they trust each other. People feel safe when they respect each other. The excellent book, Team Geek prefixes trust and respect with humility (HRT - spoken as “heart”), which is also a vital ingredient for a healthy team. If you work in an IT team and haven’t read Team Geek, I highly recommend it - we have several copies going round our office (I did say more than just books)!

I’ve been very challenged this year (already!) by how to live out these values, firstly for myself, and, secondly, to encourage them in the teams I lead and am part of (be they family, work or social). As a leader and manager of a software team, my goal (as put by the Agile Scout, Peter Saddington) is to love the team and inspire them. Inspiring the behaviour and values that increase team safety is quite a challenge, but a vital one to engage with.

I feel like it’s going to be a long year! Perhaps it’s time for a retrospective, starting with a Safety Check? This great technique provides a safe way for the team members to share their comfort/safety level, ahead of the team discussing potentially difficult, “touchy-feely” type subjects. Writing this blog post actually led me to the Safety Check article, so for me, this post has been very worthwhile, and I’m starting to feel just a tiny bit more confident already!

Agile Testing Days take-aways #3 - Infrastructure Testing with Jenkins, Puppet and Vagrant

Infrastructure-as-code (IaC) tools, such as Puppet and Chef, have fascinated me since I first became aware of them about three years ago, and I'm very excited that my organisation is now making use of them. Attending this session was thus an easy choice to make. Carlos Sanchez delivered a well balanced talk that was high-level enough for newbies to understand, but still provided enough meat to satisfy those familiar with the concepts. You can read the abstract and grab the slides from his blog.

I had two direct take-aways from his talk, one technical, the other cultural:

1. Carlos shared how RSpec-puppet can be used to assert that packages are set up correctly - think acceptance-test-driven-development for puppet.
2. There was a great quote and this picture at the very beginning, which gave the Printing Press as an example of a tool can lead to significant cultural change. The quote was from Patrick Debois: Tools can enable change in behaviour and eventually change culture. This is controversial point, because, in the case of BDD, for example, using tools too early can hinder or even prevent the right kind of culture change. Choosing the right tool at the right time is a difficult decision to make, but one we shouldn't be afraid of.

Following his talk, Carlos and I met up to have a deeper discussion on the following practices.

Environment specific variables and configuration servers

One of the benefits of IaC is that it helps overcome environmental configuration issues, by explicitly codifying each environment's configuration. A few years back, the concept of a configuration server was mooted, which also partly addressed this. The concept involves a central server and configuration repository, from which applications retrieve their configuration (via HTTP, for example), by telling the server where their running. We discussed how IaC solves this, and whether configuration servers are now redundant. Our conclusion was that environment specific variables (e.g. connection strings for QA, Pre-Prod, Live) should be separated from recipes into environment specific files. It's still worth considering the use of a configuration server, quite separate from Puppet/Chef. I think experimentation would be required to establish if this separation was beneficial or not. If any readers have experience of this, I'd love to hear it.

Should engineers develop on local, remote or cloud VM's?

IaC tools can lead to the increased use of Virtual Machines (VM's) throughout the development process, including an individual engineer's dev-environment. This raises the question of what's the best host for VM's used for this particular purpose. Is it better that they be hosted locally (on the engineer's own laptop), remotely on a VM Server, or remotely in the cloud (e.g. AWS). There are many trade-off's to make in deciding this, but our discussion concluded with the instinct that (beefy) local hosts were best.

What's best: immutable images or a clean base + recipes?

Martin Fowler has an excellent article that presents the concept of immutable images (a VM image that needs no configuration), and compares them to deploying binaries that need no further tinkering nor configuration. At first, this seems to go against the IaC concept of having base images that source-controlled recipes fully define the configuration of. When you evaluate this in practice though, a hybrid of the two practices gives the best of both worlds, as using an immutable image significantly speeds up chef execution and deployment. The immutable image should be created, though, by using a clean base image and a source-controlled set of recipes. The image and it's recipes should be updated every couple of weeks, depending on your release cycle.

Who chef's the chef / who puppet's the puppet?

Chef/Puppet recipes include definitions of required applications, where to get them from, and what version to get. So where do you define what version of chef to use (in order to keep it consistent and up-to-date)? Carlos suggested this could be achieved by a mix of running update scripts and using chef recipes themselves.

Agile Testing Days take-aways #2 - Accelerating Agile Testing

This is the second in my series of take-aways from Agile Testing Days.

The ever-enthusiastic Dan North did a high-energy keynote on Accelerating Agile Testing. His slides were as innovative as ever - this time a series of index cards stitched together using Cam Scanner. Check them out at Speaker Deck.

My personal take-aways were:

• UX is all about the experience a user has when they use the product - e.g. what drives Apple users to stay loyal and queue up overnight. How do we develop our culture so that it thinks as much as Apple does about UX?
• What we do reveals what we value: values and beliefs drive our capabilities which drives our behaviour. Consultants and coaches typically work at the capabilities level, unless they have permission to help people adjust their values. Working systemically can help make adjustments at the values level.
• Are we testing in all the right areas, in the right ways? Check the testing capabilities matrix:

Testing capabilities matrix

• We should be putting most of our testing effort in the upper right quadrant of the impact vs. likelihood matrix. Bear in mind that this matrix has several planes, with each being specific to a context/stakeholder.

Impact vs. likelihood matrix

• Explore other testing methods, consider opportunity cost (i.e. not everything is worth automating), test deliberately (consider how, where, what, when).

Agile Testing Days take-aways #1 - Visualising Quality

Last week I had the privilege of attending Agile Testing Days 2013, where I was a consensus speaker. This post starts a series summarising the main learning points I took away from the awesome conference.

First up is David Evans on Visualising Quality. David provided a very moving key-note that ranged from the Challenger disaster and Napolean's retreat to Lady Gaga and the McGurk effect. He very effectively communicated the care and thought needed when we construct visualisations. Some things I felt helpful to consider or try out were:

1. wordle.net - Pour in all of our test case names and see what it looks like.
2. The value of the information we provide is equal to the value of the decision it informs - provides a good rule for deciding how much effort to put into creating a visualisation.
3. We're not pilots, so don't build a dashboard that makes it looks like we are - good advice against going overboard with awesome dashboards.
4. Consider carefully how your information is presented: use a representation appropriate to the decision its informing and ensure that those making the decision understand what the data is saying. David gave many good examples (such as how the US voted during the 2012 elections) of how the choice of axis can distort or enhance the true picture. Each time we create a visualisation of something, we should carefully consider how appropriate the axis are (e.g. what's the best code-metric: LOC, files, test coverage, live usage?).

Hiring well

How you do hire the best people, in the most efficient way?

This question was the essence of a great conversation I had this evening with a young Agile Manager from a fast growing, Polish start-up. We're both in Bucharest for the grass roots Agile Lean Europe 2013 conference, so this will no doubt be one of many great conversations.

We compared notes and shared ideas. The take away was:

1. Attract the best talent. Through community involvement over a number of years, his company is way ahead in this area (thankfully they're in Poland, so not direct competition :). Joking apart, this reinforced the importance of developing and promoting your R&D Team as a brand - it pays off in the search for new talent.
2. Always phone screen first. If using agencies, have them do a pre-call, and get them to collect information to dig into on your own phone screen. Give the agencies a mix of open-ended and specific questions, e.g. "how to you learn?", "what have you learnt most recently?". Design these to give an early indication of height and cultural fit.
3. Show-me-your-code. If the candidate passes the phone screen, send them a coding exercise to complete. Their solution will give lots of insights, at very low cost.
4. Always meet face-to-face. If the candidate passes the coding exercise, then either do a Skype call (if remote working will be a significant part of the role) or skip straight to a face-to-face. The goal of the face-to-face is mostly to determine cultural fit. My friend put it this way: "only hire people you'd want to go for a beer with, so go for a beer with them". Our approach in more reserved England is a four-part face-to-face involving a code review and kata with an Engineer, a technical discussion with an Architect/Engineer and Engineering Manager, followed by a "cultural" discussion with a couple more Managers. The CTO often gets involved at this point too. A tour of Engineering is also done.
5. Use a tool that makes it easier to scale this process out to lots of people (in terms of both candidates and those doing the hiring). What was hilarious was that we both use Trello for this, and both have recently set up Zapier to allow candidates to be added by email (this was ahead of Trello's recent addition of their email-to-card feature).

Geeks, growing great teams and faith

I've recently finished reading Team Geek and would highly recommend it to any member of a Software Development Team (in fact, any member of any team, but more on that later).

The book's authors (Brian W Fitzpatrick and Ben Collins-Sussman) do a fantastic job of illustrating how Humility, Respect and Trust are key values within an effective team. The book is a very easy read. Each short, bite-sized chapter includes coal-face stories from the authors' work with Google and the open source community.

Despite being a short book (167 pages), it covers a wide range of topics, whilst maintaining a style and language that is "geek-friendly". Developers often struggle to engage with fluffy concepts such as feelings, emotions and motivation, but this book communicates accessible advice exceptionally well, on all these topics.

The challenge for me now is how to apply it to the teams I lead, thinking specifically of what tweaks I need to make personally, and also encourage others to make. An interesting aspect of the application of this book is that, as the authors state in the Epilogue, it's principles don't just apply to software teams, but to any community.

Whilst reading this book, I was also listening to a podcast series on the Shema, from Mars Hill. These focused on the application of love (to God, and people), via heart, mind and strength. The book and the podcasts had much in common, with each reinforcing the other. Time will tell how well I apply them both.

Our first public coding dojo

Tuesday evening saw NewVoiceMedia host its first public Coding Dojo, under the auspices of the Guildford Meetup Group.

Dojo literally translates from Japanese as “place of the way” (Wikipedia). It’s a place where students hone their skills in a given art. The art, in our case, is writing maintainable code. Dojo’s normally use katas to develop the skill: kata being the deliberate, repetitive practice of the technique, to improve one’s expertise in that area.

We were led by the excellent Chris Pitts who did a fantastic job of guiding the six students (four from NVM) through the FizzBuzz puzzle:

Output the numbers 1-100, but for numbers divisible by 3, output Fizz, and numbers divisible by 5, output Buzz. For numbers divisible by 3 and 5, output FizzBuzz.

This appears at first to be a very simple puzzle, which can be solved in under five minutes by an expert. Chris provided a great extension to this puzzle (applying the Single Responsibility and Open Closed Principles) that stretched even the most seasoned programmers present. All in all, it was a very fun and educational evening. The pizza and coffee was pretty good too!

We look forward to the next one!

Building simple, usable interfaces, architectures and products

I've recently finished reading a fantastic book: Simple and Usable. It's best read as a physical book, due to the stylish, magazine layout of text and full-page pictures. It's easy to read, but thought provoking. Concrete steps describe how to improve a user interface. These steps are easy to understand, but will no doubt require discipline to apply correctly.

Although each page offers a tasty morsel to digest, one page (76), entitled Solutions, not processes, jumped out at me. It asked the question: is there another way to solve this problem? I had recently read the very same words, in this architecture check-list, and they were then echoed again by Michael Dubakov's insightful article on Product Owners as a SPOF.

So often we rush to produce a solution, without considering the alternatives. Our team is currently trying to scale out and speed up in a big way, so we're focussing hard on optimising cycle-time. Despite the diversity of these articles, they all provide a clear reminder to not rush the creative process that leads to the best solutions. Don't rush it, but still optimise it - that's the challenge.

If you're a skilled User Interface Specialist that would like to be part of this challenge, please get in touch - we'd love you to join our team!

Going faster, growing faster

The team I help lead has a lot of work to do. In fact, a massive amount of work to do.

So what are we doing about it (given that we're already very agile)?

Three things:

1. Go faster, by focussing the teams on stabilising and reducing cycle-time.
2. Go faster, by focussing the teams on applying the 5-why's to problems that slow us down.
3. Grow faster, by hiring more Software Engineers, both permanent and, as of today, contract. So if you're interested, please get in touch!

The challenge is the balancing act: I'm well aware of the mythical man month, so will be focussing on ensuring that growing faster doesn't result in us actually going slower!

Watch this space for an update on how it goes!

A tale of two retrospectives

One of my responsibilities as a Scrum Master is to facilitate our team's fortnightly retrospective. I like to experiment quite a lot with most things, and retrospectives are no exception. Our last two sessions used different formats that I thought may be of interest to others.

As a caveat, many of my ideas (and this very blog post) was inspired by a recent article from the lovely people at +TargetProcess , on different retrospective styles (of course!).

Goal-orientated

This is a twist on a traditional format that focusses three reflective questions around some shared goals:

Create fast, secure, scalable features that our clients will rave about.

Continuously improve our ability to get things done and get things right.

Share your passion for your work internally and externally.

Be (mostly) awesome.

These goals apply to every Software Engineer at New Voice Media and so we should also embody them as a team (we're hiring, by the way, Developers, a Product Owner, Op's Engineers and more - contact me if you're interested).

I printed the goals out, with emphasis on the key words, and stuck this up on the whiteboard. With this as our visual focus, we then considered three questions:

In order to achieve these goals, what should our team:

Do more of?

Do less of?

Keep doing?

The team spent 5 minutes writing ideas on stickies which were colour coded by section. Once everyone had posted their stickies on the board, we discussed each one and then starred the ones requiring significant action.

This format was great as it provided a concrete, visual focus for guiding our reflections, as the picture below shows.

Retrospective output

Looking forward - preparing for change

If your team is about to experience significant change, then a retrospective is a useful setting for sowing seeds and facilitating preparation.

Our Development Team will increase in size by 25% over the course of January (did you miss that we're hiring), and we're likely to add another 25% over the remainder of Q1, some of whom will be in a separate location. For our last retrospective of the year, I therefore made this our focus. We discussed the short and longer term changes that the team would be experiencing (cultural, as well as numerical) and shared thoughts on how this would impact us.

Applying the retrospective label to this approach may be a stretch, but I felt that it was necessary to improve the team's readiness for change. I'll let you know how well we actually responded to the growth in a few months!

Retrospective retrospective

In summary, it's important to experiment with the retrospective format and hopefully the two I've shared will help you do that. To improve our improving (which is what retrospectives are all about), it's important that we regularly apply the adjust part of the plan-do-check-adjust cycle to the retrospective itself.

When two worlds collide - Systemic Practice meets Agile

In interviews, I often describe Software Development as being:

People + Process + Product

Although I'm passionate about all three, I think the People component is the most important, but hardest to get right. My wife is currently studying Systemic Practice, in the context of Family Therapy. In our frequent discussions about each other's work, I'm finding significant overlap between Systemic Practice and growing great software teams.

Systemic Practice seeks constructive improvement, through bringing out, sharing and respecting the views and stories of all involved. Although this approach has a strong connection to Family Therapy, it can also be applied to any group of people. For more details, visit the Association for Family Therapy site or the Systemic Practice Index.

A Crisp team article recently made a similar connection in applying what my wife calls the miracle question, to Agile Retrospectives:

If you had a magic wand / could perform a miracle, what would your new future be like?

The key with this approach is to focus on a more positive future, and the steps to reach it, instead of spending time wallowing in ones current problems. A couple of the teams I work with tried this "positive thinking" method in recent retrospectives, and found it very, well, positive! It gave a much greater focus on the end-improvement, and the actions required to get there. Time will tell if the goals are actually reached, but I have every confidence.

Another Family Therapy technique I've often found useful is the use of curiosity to subversively cause change. If you'd like a person/team to adopt a new behaviour, then instead of asking them directly, or (even worse) telling them to, try this approach: take a neutral stance on their behaviour, but just think out loud:

I wonder what would happen if you tried <describe desired behaviour>." 

Then wait for their response, without pushing or pressurising them for agreement. You may not get a positive response, or even any response at all, but given time and the right conditions, they may mull it over and their own curiosity will make them think this through. The result is often them picturing their change in behaviour and foreseeing some benefits, which they then voice themselves and buy into more readily than if you'd forced the change on them. This is because they are now the originator and owner of the change.

I've described just a couple of Systemic Practice techniques that can be applied to Software Teams. I'm sure there are many more, so I'll blog about new ones as I find them. I'd also love to hear from other  Agile Coaches or Teams applying Systemic Practice, so we can share our experiences of what works (and what doesn't!).

A user story story

Although I've known about user stories for about five years, I've only now been in the position to practise what I preach.

Having promoted them within my team (particularly following QCon 2007), I'm trying to use them on every project I work on and have to say I am blown away by their effectiveness! Although we captured them onto a wiki, their power immediately became apparent when we started using the real paper/card versions.

First off, estimating (complexity) became immediately easier: we analysed each card and divided them into three piles (easy, medium and complex). Unknowns and design notes were made direct onto each card, for later reference. The cards could also be simply arranged in dependency groups. Pieces of paper divided into piles made the shuffling back and forth incredibly easy and low cost.

We then assigned effort estimates (weeks) to each story and grouped them into possible iterations. Again, the ease with which changes could be made, and the effects so easily seen gave the analysis extra power. I've always previously used Excel to do this kind of work, but whilst it "looks" professional and does the sums for you, it lacks the speed and imprecision of paper!

The power of paper was further demonstrated when we met with a system commissioner, and talked them through our analysis. Each piece of paper could be referred to in turn, to address unknowns and discuss implementation considerations. The story groups could be simply seen, but also easily rearranged.

The ironic twist is that we kept our wiki versions of the story up-to-date as we changed the paper ones. However, we came to the conclusion that whilst the paper stories looked unprofessional (my writing is not the neatest :-), the real paper ones were just so much more powerful.

QCon 2007 - What I learnt about process - Part 2

Going through my notes I discovered this useful map of an agile life-cycle. I think it came from Rachel Davies's session on user stories.



As text it reads:

Bird's eye view of life cycle

Exploration

Explore

• business domain
• technology
• process
• scope

Keep as short as possible (< 2 weeks)

Release planning - create roadmap of releases

• May not happen until some initial iterations
• Flesh out plan as you progress
• Deliver high risk / high value items early
• Set concrete release dates but vary features

Iterations

Production - aim is to arrive here ASAP

Planning phases

Customer needs

Implementation considerations

Implementation tasks

Estimated task list

Customer reviews effort vs. necessity

UI Design is often done an iteration ahead,
to help users & team understand what is required