The software development process needs improvement

For 10 years, it was the same old story: wait for the build, program the target, wait for the target to initialize, manually run complicated tests. Many times this involved multiple computer terminals and pieces of hardware. Then I would repeat the cycle of build and run the tests. After a while, I would run a bunch of long regression tests, only to find out that some code was broken. Who knows when that happened? Then I would hunt for the issue, refocusing on multiple routines until I fixed the issue. Something had to change!

Unit tests are faster than manual tests

An infrequent Zigbee event was causing an error. After a long wait, I was able to capture the over-the-air event in the debugger, and found that the issue was isolated to a small section of code. I set up a simple unit test environment in the IAR IDE using the simulator, then created tests for the message handling. The simulation took less than a minute to build and run. The fix took a couple of hours, including setting up the simulation. I continued looking for ways to test software off target. I attended James Grenning’s Test-Driven Development workshop where I learned the TDD process: creating a small test, writing only the code necessary to pass the test, restructuring, and repeating until the functionality was complete. I was eager to try TDD on a whole project.

TDD speeds up embedded project development

A Zigbee-based industrial sensor and controller had an end-of-life Zigbee chip. The new design required a complete rewrite of the software. Development with the Silicon Labs Zigbee stack can be tedious when everything is done on target. It takes over a minute to build the application. Then, there’s the time to program the target and run over-the-air tests involving multiple pieces of hardware. Many features require a lot of code to test anything, and so the new code must be tested and debugged all at once. This process prevents testing and proving out small steps in the code development. Additionally, manual regression tests take a long time to run. So instead, I decided to use the Test-Driven Development process to reduce test time and speed up the development. Since the Zigbee compiler is GCC-based, the code easily built and ran on a PC-based GCC compiler. Access to the PC debugging printing and file handling facilities gave much more information, and quicker test results than on-target methods. Running test data from files was particularly helpful. Using the TDD approach, I created tests and small sections of code. Both the product and stack simulation code were developed using TDD. I utilized portions of the Silicon Labs stack code to develop the simulation code. The testing application built and ran all tests in under 15 seconds. Progress smoothly moved forward. There were no time-consuming bug chases. There was very little on-target debugging. The target debugger was mostly used to generate test data for the unit tests or figure out how the stack behaved. System-level issues on the target were translated into unit tests. It was easy to get restarted after an interruption or leaving for the day. The tests documented my progress. Occasionally, I broke either the stack simulation or product code. I was usually less than 5 minutes from the error creation.

Rapid feedback made the errors easy to fix

When the code was complete, the units tests were complete as well. The testing was more thorough than previous projects where the unit testing was written after the fact. Code releases occurred frequently since only a short manual test was required to verify the code. This allowed the customer to test in parallel with the code development.

TDD reduced the testing time and sped up development

The third project was the PC GUI application for calibrating the sensors developed in the previous effort. GUI-based applications can take a lot of time to test because of the required user interaction. It takes five to ten minutes of pushing buttons to test a small change deep in the calibration logic. If you make one mistake in 20 steps, you have to redo the whole sequence. The GUI interface was minimized and isolated from the rest of the code. Its behavior was then simulated. Separate builds were used for the calibration sequence, com port operation and calibration math. This simplified the configurations and kept the compile-build-run cycle to around 15 seconds for each project. Like the Zigbee project, the development was smooth and steady with little difficulty. The GUI portion of the development was not as smooth, as there was some interaction between the controls. Because the code was kept to a minimum and isolated from the rest of the project, there were very few issues and they were pretty easy to find.

The customer shipped product in less than three months after the development start

Future projects will further reduce the code amount in the GUI and investigate TDD methods to exercise the GUI code. Using TDD on three different projects gave the same results: easier development, faster testing, more frequent testing and smooth development progress. I really enjoyed these projects because I did not have long difficult bug chases, and the testing was more thorough.

TDD definitely improves the development process