Avoiding too many user stories

…… 600 user stories in an application development backlog. All those user stories showed the team’s extensive due-diligence work in a complex project that involved a platform change. Unfortunately, having so many user stories was a sign of no consensus on goals
Avoiding too many user stories in software development

Quality Assurance

Let me set something straight. Quality Assurance (QA) is about improving the process that is used to create the end result. In other words (in software development) the project lifecycle and all it’s inherent processes. The wikipedia defintion is not that bad, so there is no excuse for rebranding of QA as purely about testing (though testing processes do come under QA remit). To misunderstand what Quality Assurance probably means you don’t really understand what testing is about either. In other words, leave it to the experts. I realise saying we need to QA this product sound vaguely more exciting than we need to test this product. Reason being is that most people don’t get what it is anyway. Mysterious – hence, cooler.

Why the good testers disappeared

… and why they shouldn’t have. It is relatively easy to see when and why some form of disillusionment occurred in testing world, after a few years of increasing hype around it. There were many healthy discussions around a tester’s place on modern Agile projects, and increased focus on test automation skills. It was when the context-driven school of testing put it’s hands up and fairly stated it could not be considered relevant. After all, what stays still in software development? The whole evolution is based on learning from mistakes, and improving programming. It is an interesting time, because once again testing has to reclaim it’s position, in the new surge of “testing is dead” mantras.

The place of ET

Last year, there was some heavy debate around Exploratory testing (ET) – mainly down to increased awareness of it’s existence, though those in testing it had been around in many forms since software development begun. It is very difficult to do complete requirements, and while in good Agile fashion, requirements should be continually reviewed and if necessary, edited or added to. Exploratory testing is simply part of that process, not random adhoc testing, which is unfortunately ET had reputation of being. And it can be, in the hands of amateurs. It should be planned and documented, and User Stories should be generated around them. There is no reason testers cannot contribute to user stories.

This is a great presentation from Elizabeth Hendrickson which succinctly demonstrates the place of ET, and how it’s actually an essential not an optional testing type.

Real-life Kanban

Have you ever found yourself in accidental Kanban on software development project? When the developer is efficiently managing his own WIP (Work in progress), and driving forward changes without clearing with the business? Not ideal, as a manager should be dealing with the WIP, and changes to plan should be by consesnsus. Developers like this don’t grow on trees, so there has to be a way of designing new processes (process fast becoming a dirty word, but as yet no effective substitute) to steer projects on a Kanban course.

Stop Starting Start Finishing (http://www.software-kanban.de)

Kanban introduces improvements in momentum and sustaining that momentum. It is a methodology for managing CHANGE – it is not a project or development methodology. As with any methodology, the simplicity of the fundamentals belies the underlying complexities. Managing WIP cannot be slapdash or applied weakly. Software development has careered forward in wake of Agile, and took hits on quality. With stakeholders and developers (supposedly) more actively testing, testers became an almost endangered species at some points. With shorter, smaller scale web projects, sometimes testing was victim of budget or time constraints. Quality is not omitted as a default, but degrees of success can be achieved without dedicated test resource, even on badly-run projects. In manufacturing this would not be possible for long-term (even short-term) survival.

Quality Assurance has long been in manufacturing industry, and carries weight at every stage of the process. In the thinking of end-to-end path to product from requirements, QA transcends industry – but for some reason, despite it’s lucrative nature, software development remain in a delivery-panic mode. The holy grail became how to make products as quickly and as marketable as possible. In the efforts to shorten the journey there have been compromises, mainly on quality. Though “We have had to cut the testing budget” sounds a lot less alarmist than “We have had to increase risk compromising quality”. I used a similar style recently, when defending case for proper validated HTML and CSS code (traditional back-burner task). In response to pushback I said “OK, so we are all happy with not fixing the invalid code, that may also be causing performance issues?”. OK, melodramatic, but I was just making a point.

With Lean IT, you can streamline development to be relevant and efficient – and Kanban is very effective way to manage the development cycle. Kanban ensures that rules are in place to streamline flow of requirements to code to production. It is oriented around concepts of ultizing your resource and skills, without overusing them. The review and improve in continuous flow is essential for this to work, and therein lies the biggest challenge for IT with Lean methodologies.

What struck me most about Kanban was in the implementation challenge, within different country and work cultures. The UK, along with others, had problems with the Agile ethos, and that was due to many cultural factors. For instance, there is a general lack of openness and transparency in British business. So concepts of learning through mistakes looks logical, but implementing it in a culture where failure is to be avoided, and In the US, they tend to have a more team-oriented way of thinking. Communication is a big deal in Lean and Agile thinking, and it’s importance and definition varies from country to country and industry to industry. The trouble is we can still get away with a lot in software development, and I speak from experience in development. To produce code of quality and robustness is difficult to demonstrate to the customer – it has no instant gratification for someone who will judge based primarily on what they see. But the customer can have visibility of the processes that transfer what that ask for, into end product.

What Agile suggested with a nudge, Lean/Kanban drives home with a sledgehammer – transparency, communication, adaptability.

Kanban Polyglot

…. the essence of software engineering consists of working out the specification, design, and verification of a highly precise and richly detailed set of interlocking concepts. What makes software development difficult is its essential complexity, conformity, changeability, and invisibility.
“No Silver Bullets—Essence and Accident in Software Engineering” (Fred Brooks, April 1987).

The Polyglot Man - Orange (Joan Miro - 1969)

The title sounds like a character from Hitchikers Guide To The Galaxy, but the two are very relevant to each other. Kanban has been rolling around my thoughts and online rants for a while now. I am still learning – I would never be arrogant enough to assume my learning has stopped. And that I can get trapped down a rabbit hole as easily as anyone else when trying to work out the best process. The wave that carried Agile was rough, but it was also an essential learning curve. But in widespread application as purely as a user guide, rather than using it as guidelines, there were errors of judgement. The most common being that the processes devised (or more likely copied) were not being followed through. This was down to general misunderstanding of what Agile was to start with, but also the resistance to change – a risk with any change within a company, not just software development.

Whilst business world nodded to Agile, liked the ethos, liked the idea of faster and better quality delivery, it singularly failed to go Agile itself. Or modernize at all. For all the advancement, I still see a gulf between business and development – in fact, even more so with Agile/Lean, because of their promotion of transparency, experimentation, embracing mistakes as learning opportunities. These aren’t natural bedfellows for business. But it is too easy to waste too much space whining about that problem. Perhaps a better strategy is to implement processes to development cycle by stealth. Largely business don’t actually care about the process, they care about the product.

Software development has become too machine like – the idea that anyone can be a programmers is a tempting chant. It’s accurate but let’s face it, just actual coding (especially with all the helpful IDE’s these days) is a small part of the project story.

I decided to learn a new language, or as it’s C++, relearn something I haven’t touched for many years.  Programming languages generally evolve over time, I mean the fundamental ones like C++ which really brought object-oriented programming to the fore.  When I was choosing, I decided to go for an established and still popular language.  It’s faster to run than many modern languages, and more robust and scalable.  The impatient fervour around Ruby on Rails was too early, as people essentially lactched to its flashy timesaving features, rather than being concerned with the substance.

Creating a programming language is one thing, but it has to go through years of implementations and learning to discover it pros and cons (and all languages have them).  Polygot programming illustrates ideas of using using languages for different features – one language is rarely appropriate for an entire system.  Choosing a fundamental language to learn seems a better approach, as when learning a fundamental languages, other languages can be learnt a lot more quickly.  The jump from C++ to Ruby is a lot less painful than vice versa.  Rather than waste time on more narrow fluff languages, I prefer to learn one that inspired many other languages.

http://polyglotprogramming.com/
http://www.jedi.be/blog/2010/02/12/what-is-this-devops-thing-anyway/

Agile Waterfall

It’s only recently I realised how much we tie ourselves in knots over process, and an obsession with adopting processes at a surface high-level, instead of properly analysing what is needed.  Processes are generally treated as if it is not possible to integrate them into other methodologies.  The classic case is the step-down approaches, which have long provided business with the milestone approach that makes them feel comfortable. Release or Sprint?  How about a Release containing multiple Sprints?

Less is More (Lean is Mean)

Lean is centered around preserving value with less work. Lean originated as a manufacturing & production practice that considers the expenditure of resources for any goal other than the creation of value for the end customer to be wasteful, and thus a target for elimination. This can easily been mirrored in the demands of modern web projects.

Less haste, more speed

You never change things by fighting the existing reality.
To change something, build a new model that makes the existing model obsolete.
— Richard Buckminster Fuller

With the news that one of the founding fathers of modern software development died (Dennis Ritchie – inventor of C and co-creator of UNIX), it made me think how this is the first time I feel a sense of history for modern computing. And how software development methodology has evolved, as people’s understanding of the impacts of process. In the earlier days, clients just accepted products without question – as computing was considered in realms of IT people, and it was assumed that non-IT just wouldnt understand the perceived intricacies of delivery software. Waterfall was a default throughout the 1970’s and 1980’s, and unless a client worked out everything they wanted from the analysis stage, they would not be able to change their minds (only in special circumstances and at great additional cost).

Waterfall methodology did not suit the increasing complexity of software development, and the dependncy on each phase completing before the next one started. There was no allowance for requirements change once analysis phase was complete, that was the key weakness. Waterfall is the oldest methodology and as such critising seems a ltitle akin to kicking an old man. The top-down approach did make sense in earlier days. Software engineering has been, and is continuing toi be, and ongoing learning curve. So it does make sense methodologies should account for this. As should project management. What there isnt an excuse for is taking easy path of simply slating the older ways of doing things, in order to make way for newer ways. The way to usurp is not attack, but develop another alternative that render the old way obselete,

The most successful of the Agile-related methodologies for management is Scrum (though scrum predates Agile by 9 years). Widely adopted, it was primarily aimed at smaller projects with teams 8-12. A lot of modern projects fit into this bracket. What has changed significantly since Agile Manifesto was first devised, is the concept that methodology will solve problems by themselves. This may be true to some degree with software development, but management is a different matter entirely. A lot rest on the skills of the management layer, with methodologies, the assumption being you have the right person in right role. A methodology cannot accommodate team weaknesses, lack of engagement from stakeholders. If you have a poor Product Owner or Scrummaster, just observing the principles of daily meetings and Sprint planning will not magically make your project go well. There is not magic.

As an example of how to solve these naturally occurring issues (after all every company, every team, is different), I will use Kanban. Kanban was devised by people from a company to deal with a problem of fully carrying through design requirements through to end-product. This was originated in manufacturing, but can easily be applied to software development. It takes an existing concept from manufacturing origins, and makes the Sprint itself more flexible. Sprint planning is not omitted, it is acknowledgement that events during a Sprint can nesessitate a change in plan. It is, if oyu like, a software development andidote to Scrum rigidity. Its not about breaking the rules, it bending them. And that is what you are supposed to do. Apply Scrum with kanban and you have a very flexible methodology – couple that with continuous integration, and you will have a cohesive process.

Continuous integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day). Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This is a fundamental Agile process.

The above combination is not intended to be a final solution, it is one of many potential combination and methodology customisations. Its possible that kanban is not relevant to your project, either because the stakeholder finds the concept, or there are immovable deadlines (for whatever reason). Too many Agile approaches don’t allow for the very real possibility of inflexible timescales. Applying Scrum rigidly the this scenario will keep project on track – just make sure you have the right team to do it!