Evolutionary change fuels revolutions

We commonly draw a distinction between revolutionary and evolutionary change. We categorize some changes like the invention of the PC as revolutionary, and others like a new development framework as evolutionary. I suppose there is something about that idea that makes sense. There are changes that have immediate and wide-ranging effects. But I’ve never seen this as a useful distinction when you are trying to effect change.

This distinction is commonly used to comment on big changes in an industry, product category or culture. Commentators see the big, evolutionary change leapfrogging over many smaller changes. They categorize changes into these categories as an either/or proposition. A given advance must fit into one bucket or the other. There is another group besides pundits that also commonly uses this paradigm to think about how change works: managers.

This kind of hard, binary distinction is something that only an observer and not a creator could come up with. Creators know that these kinds of changes are really two sides of the same coin. That’s not to say that we don’t need both managers and commentators, but their perspective is, for the most part, that of an outsider to the creative process. Creators know that there is a very fine, if even distinguishable, line between the evolutionary and the revolutionary.

A revolutionary change can only be sustainably made after many smaller changes lay a foundation. Wi-Fi was revolutionary, but it was built on the back of other technologies. The TCP/IP protocol, radio communication, low-power radio receivers, fast high-quality encryption and more led the way to making Wi-Fi possible. For the outside observer, this appears to be an evolutionary change out of the blue, but to anyone in the networking sphere it was the inevitable result of many smaller (from this perspective) innovations.

We must keep this in mind when we are responsible for changing systems, be they software, hardware or processes. Small evolutionary changes fuel the big, revolutionary change. New principles and ideas build on what came before them. To effect change in a large scale system, leaders and designers must emulate this evolutionary process.

I think the easiest example here is contemplating how a large development organization might make the move to an agile methodology. The leaders certainly can decide to just institute the process changes—daily stand-ups, backlogs, fast iterations and the like—and train the staff in how it should work. This is the way many companies do it. But it is usually not very effective. Because agile is not just a way of running projects, it’s a way of thinking about software, a very different way than many people were trained to think about it. Implementing the changes to the process without changing the way people think, without trying to change the culture, will not result in a meaningful change.

Small changes—like your team seeing the wisdom in the agile principles—is not always easy, and it does not occur overnight. A change like this is evolutionary by its very nature. By constantly fostering small changes you can build the foundation for the evolutionary change you want.

Big changes are possible—they happen every day—you just can’t skip building the foundation for them.

For software, stasis equals death

“Stasis equals death.” I picked up this phrase from a really good book on screenwriting, and found that it absolutely applies to the software development world. It applies because it is utterly true. For developers of modern software, products are either under active development or they are dying.

This dynamic was clear in Apple’s transition last year to iOS 7. Within a few weeks of the update, most of the top apps were updated to the new look, and those developers that didn’t felt heavy pressure from their users. Look at anyone’s iPhone home screen months later and there are likely no iOS 6 apps on it.

The apps that were not updated now feel old. They still work as intended, but they no longer meet the expectations of their users, and as a result, they might as well not exist.

In the world of traditional enterprise software we see the same issues. Projects to develop or enhance an application finish, the operations team is (hopefully) trained, and the development team moves on to the next project. Sometimes on the same application, sometimes not. Either way, the project in production might as well not exist to the developers unless it breaks. After all, they have the next big project to worry about. The application enters stasis.

This leaves the users out in the cold. If they have changes for the application or issues that they need fixed, they must submit their issues to the ops team and feature requests to the product manager, or the roadmap committee, or whatever bureaucratic madness the governance folks can cook up. The timeline on addressing those requests then tend to run in months, not days. Even good software at this point starts to feel old, and broken.

It’s not just the users who are left out, either. As requests for work on the app stack up, and the next big project holds all the attention, a technical debt stacks up on the application in production. The issues may not be visible to the project team, or they may be known but “de-prioritized” in favor of the existing roadmap. Either way, this team runs a very real risk of launching a new iteration of the product that is even further away from what the users actually want and need. Even scarier is this simple fact: the longer these projects take to complete, the farther away they can stray from the users’ actual needs.

I know, right now you’re shaking your head and thinking, “It’s a good thing I don’t have this problem, because we are Agile.” Well, you might be right. Maybe.

Agile as a idea is a good one, maybe even the right one. But ‘Agile’ as sold by consultants and book publishers is not always the answer. There are a lot of people who will disagree with this, but it’s usually because they are selling Some Agile Way™.

Agile is nothing more than a set of ideas, ideas that build a culture. Merlin Mann talks about this concept in the latest episode of Back to Work, but the culture you create will define every aspect of your organization. Adopting Agile methodologies can help, but only if you are committed to building a culture that really believes in the idea of agile. Either way, and agile approach will not solve our problem by default.

So our problem remains. How do you avoid software stasis? By focusing your work on the product and not the project. This idea is not new, but it should be reconsidered by most of the software development world. Your primary concern should not be the project, but it should be the application.

There are many implications of these ideas, but product ownership can be implemented through a few simple concepts:

  • Organizing development teams by product. Each application or service has it’s own dedicated team
  • Tracking and scoping requirements by product, not project. For strategic initiatives you certainly need to have some mapping of the initiative’s requirements to the product requirements, but the system of record should be at the product level
  • Projects will come and go, but the product teams endure, as does the product feature and fix backlog
  • The product team must really own the product, and be in a direct relationship with it’s users.
  • The development process must trust the team with ownership of the product

This breaks applications out of stasis. In this model, the product is never “done,” it evolves. With the team and work focused around the product the developer and his users can really own and drive the product forward together, without the long lag times of the project focused timeline. It puts the software and its use at the center of development activities.

This idea does not solve all the problems of a dev team. Like all things in life there is no silver bullet. But by viewing products as the key organizing factor, an organization can focus on the constant changes required to keep up with users evolving needs and demands. Isn’t that, after all, the key mission of a development team?

Simple is hard work

When solving any problem, we tend to look to the simple answers first. This is a good response, and one that should be nurtured. But can easily be misled by that simplicity. This typically happens in two ways. First, we think the simple answer means less time than other answers. We tend to equate simple with fast. Second, we assume that a simple answer is easy to implement. We assume that what is easily understood can be easily done. We equate simple with easy. As a result, we often misunderstand the impact of a simple answer to a hard problem. Because, truthfully, simplicity is hard work.

Let me make this more concrete. Take the idea of a microservices architecture, something I have been thinking about a lot lately. The microservices architecture is “an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.”

This is a simple idea. In fact, most laymen will be able to grasp it with about 10 minutes of explanation, and IT professional should be able to grok it in a matter of minutes. The idea is straight-forward and elegant. Break a single system down into multiple, small systems. Put application logic into the services, not the data or transport layers—”smart endpoints and dumb pipes” as Martin Fowler puts it. Call the services asynchronously, and run each of them independently. The concept is not complicated.

But implementing this idea is hard work. How do you deal with decentralized data stores? How do you coordinate operations between services without tightly coupling them? What about managing API documentation? Automated testing and deployments on multiple frameworks or platforms? All of these questions sit squarely in the middle of the microservices concept and must be answered.

With all of those questions in mind, is the idea still a simple one? I’d say so. The idea is still clear and easy to understand. But it doesn’t look easy now, does it?

One of the best lessons a good developer or architect should learn is how to continually ask the question, “then what?” Always push to the next question, to the next decision. Consider each one fully in turn, and then in light of the whole. You cannot be satisfied with the simple, obvious answer if you cannot work through all of its consequences. Don’t be fooled, in the software world simple is never easy. It’s hard work.

Find rest in other mediums

Last week was one of those weeks we all have occasionally, where the schedule is full, your mind is engaged all week, and yet somehow you're not dead at the end of it. Through a long hard week, I came out of it encouraged and satisfied. It was good, and I think there is a lesson in it for all of us creative types.

This weekend Lindsey and I were able to spend plenty of time together, and spend it immersed in the arts. Thursday night drove down to the historic Greune Hall to see local songwriter David Ramierez open up for Patty Griffin. It was a great night. David is one of the best songwriters out there. Period. If you've never heard him, you really should give him a listen. Patty Griffin was amazing, but that's more Lindsey's jam than mine. But it was a beautiful night. The weather was great, the venue classic, and both performances were outstanding.

Saturday morning we had our quarterly training session for the Austin Stone Story Team, and it was just as filling as it always is. To spend time with our artists, to sit under teaching about the gospel and creativity, and to consider where God has called me to use my talents was such a good, good thing.

Then, Saturday night we went to the Austin Symphony and saw our good friend Joseph play. Joseph is a great violinist and a dear friend. The program was outstanding, with a great variety. With a beautiful contemporary piece, Mozart and Strauss, it was really fun. We finished the night with burgers at Hopdoddy's with Joseph, as has become our tradition.

Spending time with Lindsey and great music was exactly what I needed. After a week of writing and editing for work, technical work and discussions there, I needed to immerse myself in art, specifically a form that I don't deal with every day. Sure, I am always listening to music, especially when I work, but it's not the same as live music. Live music is something special.

So, next time you need to refill, go find a medium that speaks to you, and isn't your own. Go experience new art. Hang out with other creatives. Talk about the things that inspire you. Dream a bit.