My first real exposure to TDD came during a project that slowly spiraled into development hell. This was our first agile project and for us meant going through the motions while defining our acceptance criteria to be “it should seem to work at least once”. All of our testing was done on target, and because the product we were developing was part of a distributed system this was extremely difficult. We really didn’t have the means to ensure that features would continue working (or really, whether they ever worked) so our open bug count grew without bound and we were unable to continue adding features under mounting technical debt.

At this point team morale was very poor and I truly hated my job. I didn’t feel that I was able to write software that I was proud of and I didn’t feel like a good software engineer because of how much I struggled to make even simple changes. I was frustrated by the lack of accountability within my team, but ultimately I was creating as many issues as everyone else. I felt powerless to make any significant positive change. Every day was a slog.

As a team we slowly came to the realize that we needed to change our approach and that agile software development simply wasn’t going to work without the right tools. As we explored agile and looked for answers, we found that test driven development was the change most frequently recommended. We ordered several copies of Test-Driven Development for Embedded C and hired James to first give a few days training via teleconference for this particular team and then a weeklong training session for the entire embedded development group.

Our initial optimism faded as it became clear that it was not going to be possible to flip a switch and become TDD wizards. My single biggest takeaway from James’ training was that we were going to need to design and build software much differently if we wanted to take advantage of test driven development. This meant we’d need to not only define a coherent hardware abstraction layer (HAL), but we’d need new mechanisms for making and keeping it abstract. This also meant we’d need to have a much more thoughtfully designed architecture and adhere to the SOLID principles so that we could achieve and maintain modularity.

Ultimately, the project and team were (mercifully) dissolved, but I took the lessons learned with me to my new role developing the libraries that would be used across all of our platforms. This was a from-the-ground-up effort that allowed me and my team to explore new ways to develop software. One particular challenge that we faced was that we didn’t have target hardware because we were developing code that was to run on all of our existing platforms as well as as-yet-undefined future platforms. TDD was critical in helping us to overcome this challenge because it allowed us to define and measure success in the abstract. This really hit home for us when we were able to get a new networking stack that we developed entirely on a PC with TDD up and running on target hardware within hours. To date we have never had more than a single open bug in this stack, and the total number of bugs identified is in the single digits. Learning test driven development was a tremendous amount of work for us, but even the early results justified the investment.

We’re now almost 5 years into our TDD journey and have recreated our team and team culture around the lessons we learned while learning and practicing test driven development. Accountability is a central focus of our team’s culture that is expressed through our heavy use of automated testing and continuous integration. We now have nearly an order of magnitude fewer bugs escape our team and we’re able to write new tests to identify and correct the bugs that we do find. Our testing culture has also allowed us to make continuous improvement a reality by providing a safety net for both new and experienced developers that want to make a positive change to existing code. Finally, we were able to largely divorce ourselves from the awful Windows-only toolchains that are legion within the embedded space and develop in a Unix environment while taking advantage of the wealth of tools and automation opportunities Unix presents. I contend that adopting TDD and thoughtfully applying the lessons learned while doing so have helped my team to make 10 years of progress in less than 5.

Developing without tests and “cowboy” coding are unfortunate realities within much of the embedded industry, but I am not willing to go back to working in that sort of environment. I have a short list of questions that I always ask prospective employers and “How's your testing culture? Do you develop using TDD?” is at the top.