| 1. Unit testing for individual or a small set of classes
2. Integration testing in a simulated environment
3. Integration testing in a real time environment
4. System testing in a simulated environment |
Many product specific targets for simulated and real time environment |
1. C++ editor, compiler and debugger
2. Matlab / Simulink |
All pull requests to master branch must be approved by one colleague |
11-30 seconds |
20% |
60% |
20% |
Show
|
| depends on the project. Hardly even pure TDD, a bit of mixture of soft TDD and test created after coding. Both unit and integration tests on PC, the same on HW + System test. |
Embedded (ARM, etc.) |
All sort of Editors (depends on client) |
used a number of tools |
1-5 minutes |
40 |
30 |
30 |
Show
|
| Depending on the project, either with Ceedling or GoogleTest for unit tests and Python for tests that interface with hardware. |
Usually a microcontroller that's either running FreeRTOS or a custom Linux image built with Yocto. |
Atmel/Microchip Studio, VS Code, PuTTY, CMake, hardware JTAG debuggers, GitHub Copilot or ChatGPT |
We create pull requests and if a reviewer flags an issue we discuss it, though issues are rare. |
1-5 minutes |
25 |
25 |
50 |
Show
|
| Unit tests with Catch2, tests with simulated embedded SW |
Target system is a variable speed motor drive. |
Azure DevOps, Jenkins, Visual Studio |
PR needs to be approved. Reviews are usually online. |
5-30 minutes |
20 |
20 |
60 |
Show
|
| Unit tests with TDD
Host integration test
Target end to end tests |
Embedded systems, some running CentOS or Debian Linux, some bare metal. C and C++ applications |
CLion, CMake, Git, Jenkins, Docker |
Not required very often as dev is done using pair programming. Reviewed by senior devs during dev. |
5-30 minutes |
10 |
40 |
50 |
Show
|
| Catch2 unit test, end to end test using python |
varies |
CLion, Git, Gitlab |
continuous pair programming |
11-30 seconds |
33 |
33 |
34 |
Show
|
| Post-hoc with heavy mocking |
STM32 series microcontrollers |
CMake, GCC, CLion, cpplint, sonarqube, gitlab |
Two mandatory reviews per pull request, one dev, one QA |
11-30 seconds |
5 |
3 |
2 |
Show
|
| Unit and integration tests in Catch2
End to End tests running on hardware in Pytest |
STM32 microcontroller |
Clion
STprogrammer and STlink |
Minimal due to heavy use of pair programming. However diffs are reviewed between releases. |
11-30 seconds |
20 |
30 |
50 |
Show
|
| Combination of various ad hoc |
Embedded C on AMR M4 |
Eclipse |
Doesn't exist |
Under 10 seconds |
70 |
10 |
20 |
Show
|
| Catch2 with google mock |
Linux |
GCC |
use gitlab |
31-60 seconds |
40 |
40 |
20 |
Show
|
| Unit test, manual testing with hardware, automated tests (unit tests and hw tests) |
Embedded system with ultrascale soc and different peripherals. |
Git, ide, debugger sw, confluence, build tools |
We have code reviews when pull requests are being done. |
5-30 minutes |
20% |
30% |
50% |
Show
|
| Functional Testing. |
WPF Apps, and Embedded STM32 Devices. |
GitHub, Jira, Microsoft Visual Studio, STM32CubeIDE |
Short pull request reviews for minor changes, and a larger system review prior to first release. |
11-30 seconds |
50 |
30 |
20 |
Show
|
| breakpoints; set up constants in code |
windows |
GitLab, Jira |
very unofficial reviews between peers |
1-5 minutes |
50 |
20 |
30 |
Show
|
| Unity |
The next target I know of is called the Platform Development project. |
IAR, Codewarrior, MPLab |
Done online through DevOps |
Under 10 seconds |
30 |
30 |
40 |
Show
|
| CppUTest |
STM32xx w/ on-target tests in IAR and GCC + on-host tests w/ 32-bit windows GCC build |
IAR, GCC |
Pull request + CI automation checks |
1-5 minutes |
30 |
30 |
40 |
Show
|
| CppUTest |
This varies from project to project. |
IAR, VisualStudioCode |
Currently involves a select group of Software Engineers within a team who go through the code. |
1-5 minutes |
30% |
20% |
50% |
Show
|
| By writing unit and integration tests |
Don't have one.
It can change at any point. |
IDEs and Code Editors.
Version Control.
Package and dependency managers.
Debuggers
|
They go okay |
31-60 seconds |
40 |
50 |
10 |
Show
|
| Java unit tests. Build C++ code for some testing |
Java in a Linux environment |
IntelliJ, Qt |
Essentially non-existent, we don't have the time. |
31-60 seconds |
30 |
20 |
50 |
Show
|
| PC targets are tested using Unity, embedded software are tested also using Unity but using a PC simulator |
Multi-target, ARM Cortex-M |
KEIL
Makefile
Vendor IDE(s) : STMCubeIDE |
Small team so I am often my own reviewer |
1-5 minutes |
30 |
10 |
60 |
Show
|
| TDD |
ARM cortex-M3/M4 |
Keil IAR e2-studio |
DevOps |
31-60 seconds |
30 |
30 |
40 |
Show
|
| Debugger |
? |
Keil MDK ARM, Visual Studio |
No review |
Under 10 seconds |
5 |
5 |
5 |
Show
|
| Custom test harness. |
hetero multicore SoC |
Linux tool chains |
Ad hoc |
31-60 seconds |
5 |
0 |
15 |
Show
|
| CppUTest |
AMD X86 embedded, Arm V7 & V8, Alpine musl for static builds |
Make, CMake, VS Code, Sublime Text |
Unless I see a major reason for a change I generally only ask probing questions make recommendations |
31-60 seconds |
7 |
1 |
2 |
Show
|
| Unit and integration |
Embedded PIC16,PIC32 and ESP32 |
Notepad++ |
What code review? |
1-5 minutes |
80 |
10 |
10 |
Show
|
| TDD, testing locally before pushing to CI.
Tests run in CI using Jenkins pipelines. |
Data diodes and their surrounding software for customer ease of use. |
Nvim, GitLab, Jenkins, Docker, Jira |
We use pair programming for the purposes of code reviews. |
1-5 minutes |
10 |
10 |
80 |
Show
|
| Tdd mostly since last year, mncks, DLP |
Embedded Xilinx devices |
Qemu, libcheck, mocks |
Deep code review, understanding of requirements |
Under 10 seconds |
50 |
40 |
10 |
Show
|
| using debugger |
temperature controller pcb with ARM microcontroller |
Renesas E2 Studio |
Code reviews are done using DevOps. |
1-5 minutes |
50 |
25 |
25 |
Show
|
| Test to functional specifications and ensure the device functions as intended. Additionally some longer term testing is setup but its all quite unstructured. |
Battery powered small embedded systems. Usually containing a sensor and RF |
STM IDE |
Every pull request to a develop branch or to release goes through a pull request and is reviewed. |
11-30 seconds |
20 |
50 |
30 |
Show
|
| unit test |
Multiple peripheral system |
STM32 CUBE IDE |
2 CYCLE REVIEW |
Under 10 seconds |
30 |
30 |
40 |
Show
|
| With IDE. |
STM32 Microcontroller. |
STM32CubeIDE |
I don't have a code review experience. |
1 day or more |
10 |
10 |
80 |
Show
|
| Debugging tools |
STM32 and ESP32, we will have some other chips in the future. |
STM32CubeIDE
Visual Studio Code and a variety of plugins |
At work they have been minimal I hope this course helps us establish better practices. |
31-60 seconds |
20 |
40 |
40 |
Show
|
| Debugging on embedded hardware |
STM32 |
STM32CubeIDE, STM32CubeProgrammer, STM32CubePowerMonitor |
Peer reviews on pull request |
Under 10 seconds |
40 |
10 |
50 |
Show
|
| Manual testing |
STM32 |
STMCube IDE |
We dont have that process setup yet |
31-60 seconds |
4 |
2 |
4 |
Show
|
| Limited independent peer review, personal stress testing for edge conditions and critical logical conditions |
Not sure what is meant here. I don't have any specific procedure. |
Stock IDE tools for code generation and basic debugging. |
Informal and without a written description. Reviews performed by developers of the code. |
Under 10 seconds |
25 |
25 |
50 |
Show
|
| unit test, system test |
embedded |
VS Code |
- |
5-30 minutes |
40 |
40 |
20 |
Show
|
| Unit tests and SITL tests |
armv7/x86 qnx systems |
VS Code |
At lease two dev reviewers. |
1-5 minutes |
40 |
30 |
30 |
Show
|
| Bench test, toggle IO, printf, hw in the loop testing. |
Cortex-M7 with external memory,sensors, xcvrs. |
VS Code
GCC
CMake
cppUTest
WSL2
MSYS2
AtmelStudio
IAR |
Minimal |
1-5 minutes |
UNDEF |
UNDEF |
NaN |
Show
|
| Functional tests as I'm programming |
Linux |
VS Code
GCC
Docker
Git |
We don't have a lot of them, and don't have a process established yet. |
5-30 minutes |
50 |
25 |
25 |
Show
|
| Gtest |
Real-time Linux-like OS |
VS Code
Linux
clang
coverity |
Done through bitbucket PRs |
31-60 seconds |
50 |
10 |
40 |
Show
|
| Unit testing |
Armv7 SoC |
VS Code + language-specific extensions; git, Jira, valgrind |
They depend on who is doing the reviewing. |
1-5 minutes |
30 |
40 |
30 |
Show
|
| Simple running and debugging the code |
I can't say for a certain |
VS Code for coding |
Just reading the code and test it |
1-2 hours |
20 |
30 |
50 |
Show
|
| My role is not currently development, as I am mainly on Support and with some devops. I write code occasionally for demostrations an recreating customers issues.
I normally write test cases. |
No target on this |
VS Code or JetBrains Tools. I have used others in the past. |
No comment |
Under 10 seconds |
60 |
20 |
20 |
Show
|
| Unit, Integration (HW + SW), and system verification |
QNX Armv7 |
VS Code to develop, Docker to build |
Code reviews are completed on the pull request |
5-30 minutes |
20 |
70 |
10 |
Show
|
| run it on Linux laptop.
run it on robot.
run automated test framework. |
HUGO Surgical robot -- QNX and Windows x86 systems.
Linux laptops. |
VS Code.
g++, cmake, Qt tools |
every line of code is reviewed, often by more than one person. |
1-5 minutes |
50 |
25 |
25 |
Show
|
| Functional testing of the product |
Arm based real time control system |
VS code |
We do peer reviews (min 2) in pull requests |
5-30 minutes |
5 |
15 |
15 |
Show
|
| Unit tests
ATF |
Wind turbine converters |
VS code |
PRs |
1-5 minutes |
40 |
30 |
30 |
Show
|
| gtest |
X86 & Arm A |
VS code, Cmake |
we do it in Bitbucket Pull request. |
5-30 minutes |
30 |
30 |
40 |
Show
|
| Manual functional / behavioral test suite
I started at Whisker as a FW QA Engineer in July, I have become familiar with Ceedling and am in process of adding unit test capabilities to FW team. |
Main MCU is PIC but is being phased out with STM32, all targets have companion ESP32 for Wi-Fi. |
VSCode
Espressif's ESP-IDF
STM32CubeIDE |
Have not participated in one, but have recently set forth a plan to make this a regular occurance. |
1-5 minutes |
n/a |
n/a |
n/a |
Show
|
| Write the code and if time permits write unit tests.
Then update integration tests to make sure code base works as a whole.
Run robot framework on the output to make sure the behaviour is correct.
QA manually sign off that everything is working. |
The target is docker containers which run the data pipeline |
VSCode with Python Extensions
Copilot |
Done through the PR interface. Usually offline but if it is complex, done on a call |
Under 10 seconds |
30 |
10 |
60 |
Show
|
| For C++ code Cppunit and using mocks for hardware drivers. For FPGA I use System Verilog with assertions to implement TDD to verify behaviours to reduce the risk of regressions when making changes. |
Almost all my projects have an FPGA and chips around them that we interface to. |
VSCode with TerosHDL for FPGA work and VSCode with C/C++ extensions for linting/code completion etc. |
Generally I like to use checklists and have reviewers read standards beforehand. |
1-5 minutes |
2 |
4 |
4 |
Show
|
| QA Feature regession testing for functionality. Firmware engineer edgecase testing. |
ESP32 WROOM 32E SoC |
VSCode with extensions for C and python |
Currently we have 1 other FW engineer to a Pull Request |
1-5 minutes |
30 |
50 |
20 |
Show
|
| unit tests around Cunit framework |
embedded linux device running on beagle bone type hardware |
VSCode with gcc toolchain |
code is peer reviewed through git Pull Request from dev branch into feature branch for testing |
1-5 minutes |
40 |
30 |
30 |
Show
|
| Unit testing with CppUTest before merging and periodically performing integration tests |
Testing features of medical devices |
VSCode, IAR, QtCreator, Bitbucket |
Reviews completed before merging code |
1-5 minutes |
35% |
25% |
40% |
Show
|
| CppUTest |
custom embedded platform (Arm and x86) |
VSCode, Makefiles, git, Jenkins |
PRs are reviewed before merging to main. Who reviews is chosen by the PR author. |
31-60 seconds |
20% |
5% |
10% |
Show
|
| BDD, Manual QA, Black Box, Automation Tests of functionality, Integration testing, System Testing, Regression testing. |
Any complex software application |
VSCode, Notepad++, SQL, Docker, Cucumber, |
I don't do any. |
11-30 seconds |
1 |
7 |
2 |
Show
|
| On the current project we have a blend of Unit Tests, Integration tests, System Tests, performance tests and, unfortunately, a heavy contingent of manual tests still. |
Python Data Pipeline which pushes data into a Postgres Database fronted by a .NET / JS web app. |
VSCode, Visual Studio, Git, GitlabCI, Docker, Azure and associated tooling |
All code goes through peer review and gets signed off through a formal QA process on client data |
11-30 seconds |
50 |
30 |
20 |
Show
|
| GTest / Pytest |
qnx 7.1.0, ubuntu 20.04 |
VSCode, WSL, Docker |
Comments about things not done properly, improvements in speed/robustness (why and how) |
11-30 seconds |
30 |
65 |
5 |
Show
|
| Unit and integration tests |
x86 and arm7 target nodes running an OS |
VSCode, target IDEs for smaller applications |
Deploy/test burden makes iterations a lengthy process |
30-60 minutes |
40 |
30 |
30 |
Show
|
| Using TDD - unit, integration, e2e. Automated CI to run tests on hardware |
Linux based systems, Raspberry Pis, cloud |
VScode, Jenkins, Docker |
Usually pair program so the code review is done during development of a feature rather than after |
Under 10 seconds |
3 |
6 |
1 |
Show
|
| Unit tests Google Test framework. Functional testing and smoke testing with ABB's own testing framework. |
Uses μC/OS-II or FreeRTOS, NXP1060 with ARMv7 (Cortex-M7) |
VScode, Visual Studio, IAR EW for ARM, SCons |
1 person must review the code that is merged into the master |
1-5 minutes |
50 |
30 |
20 |
Show
|
| I test edge cases and then come up with a test plan for a technician to run. |
embedded devices |
VScode, eclipse based IDEs |
I have never done one |
1-5 minutes |
3 |
3 |
4 |
Show
|
| Varies |
Varies, but mostly Linux VMs (not embedded) |
Varies |
Varies |
1-5 minutes |
50 |
20 |
10 |
Show
|
| Manually |
Various |
Various IDE |
DevOps Pull Request |
30-60 minutes |
50 |
10 |
40 |
Show
|
| For code touching hardware, most is done with custom code that allows hardware to be exercised to verify functionality |
Windows PC and/or embedded target |
Vendor provided (MPLAB / Keil)
Visual Studio Code
|
Internal use - informal or None depending on scope
External - Formal peer reviews |
11-30 seconds |
30 |
40 |
30 |
Show
|
| As I generate code, I create test functions at the bottom of the same file. The tests are intended to catch edge cases and compare function output to known good output. The test functions are wrapped in #ifdef for easy removal. |
Embedded C on microcontrollers & some scripting with Python |
Vendor provided IDEs (Code Composer Studio, STM32CubeIDE, etc.) |
After verifying functionality, I look to senior programmers for feedback on style and implementation |
11-30 seconds |
40 |
20 |
40 |
Show
|
| GTest |
QNX 7.1 and Linux |
Vim, VSCode |
Done in the Bitbucket pull request. >= 2 reviewers need to complete; all comments addressed |
4-8 hours |
10 |
10 |
20 |
Show
|
| Smoke Tests, Integration Tests, Ad-Hoc Tests, Unit Tests |
Linux and QNX OS, ARM and x86 arch |
Visual Studio |
We use atlassian tools for code reviews |
11-30 seconds |
40% |
40% |
20% |
Show
|
| Just look to see if what I wrote does what I want it to do. |
I'm not sure |
Visual Studio
E^2
IAR
Arduino IDE
Unity3D |
Not what they should be. |
1-5 minutes |
45 |
5 |
50 |
Show
|
| Mostly system level tests that are automated. |
Microcontrollers |
Visual Studio
GCC
GDB
Bash
Python
Git |
GitLab to manage merge request. Colleagues will review the change. |
11-30 seconds |
33 |
33 |
33 |
Show
|
| Unitary tests, corner tests, functional tests and (if applicable) performance tests. |
Embedded Linux-based systems. GNSS receivers. System on chips. |
Visual Studio Code IDE (previously Eclipse).
gcov/lcov, cppcheck, Valgrind, clang-tidy/format |
Deep code reviews about the functionality of the code, and docs, convention, etc done by CI |
Under 10 seconds |
4 |
4 |
2 |
Show
|
| A mix of CppUTest and manual unit testing. |
I work on both embedded Linux and microcontrollers. |
Visual Studio Code, IAR Embedded Workbench, QT Creator, Visual Studio |
Currently, we conduct these via Bitbucket pull requests. |
31-60 seconds |
20% |
30% |
50% |
Show
|
| individual developers define their own Ad hoc tests unless specifics requirements are defined by the customer |
Linux |
Visual Studio Code, Sublime, other IDEs. Gitlab/Github |
Over the shoulder/ virtual screen share code reviews. Usually limited to a few hundred lines. |
11-30 seconds |
70% |
10% |
20% |
Show
|
| Creating Test Cases using the same language and IDE |
Microcontrollers (C/C++) and desktops (C#) |
Visual Studio, Spyder, Arduino |
I like to circulate code, then have an in-person review. |
1-5 minutes |
30 |
30 |
40 |
Show
|
| "Automated" testing by writing code to test code, not using a unit test framework such as the one provided by .NET. |
Windows machines, Xilinx devices |
Visual Studio, Vivado, SDK |
Using either a lead dev on a project or a group of people to review code prior to merging to main. |
1-5 minutes |
15% |
35% |
50% |
Show
|
| Mostly in a simulated system running on a PC or on target hardware. Unit tests when module is new or making modifications to module, which has unit tests. |
Embedded software running on a ARM Cortex A53 |
Visual studio, Visual studio code, GCC |
Every change to master requires approvals from at least 2 reviewers |
30-60 minutes |
25 |
50 |
25 |
Show
|
| Unit, systemtests, tests with realHW |
electric drive |
Visual studio/code |
Via pull requests |
1-5 minutes |
30 |
40 |
30 |
Show
|
| Test Benches, Prayer |
ARM processors on Xilinx MPSoCs |
Vivado, Vitis, Visual Studio, VS Code |
None |
5-30 minutes |
2 |
3 |
5 |
Show
|
| manually |
game consoles, pcs, etc. |
all and any |
solo developer so it's new terrain |
5-30 minutes |
50 |
40 |
10 |
Show
|
| static analysis
unit-test
functional-test
system software in system |
qnx
windows
linux |
clang toolchain
qnx toolchain
|
we use Atlassian online tools to review the code changes. |
1-5 minutes |
2 |
6 |
2 |
Show
|
| catch unit test
python integration and end to end tests |
microcontrollers embedded |
clion
gcc tool chain |
pair programming or formal document |
5-30 minutes |
30 |
50 |
20 |
Show
|
| mostly in hardware tests |
single chip / embedded device |
code IDE |
pull request reviewed by team member |
30-60 minutes |
2 |
2 |
2 |
Show
|
| see if it does what i want it to |
arm microcontroller |
code composer studio |
haven't done any |
Under 10 seconds |
40 |
20 |
40 |
Show
|
| Try to use TDD where Possible |
CN0510
https://github.com/graial/ad5940-examples-adjust-output |
cross core embedded studio (eclipse based) |
nil, one man team |
Under 10 seconds |
3 |
5 |
2 |
Show
|
| varied: Unit, integration, E2E tests. Very few human-in-the loop manual tests. |
varied: FPGA, MCU, Embedded C, Linux |
git, gcc, CMake, CLion, JIRA, Jenkins, Docker, Artifactory |
Pair program instead |
Under 10 seconds |
10 |
10 |
20 |
Show
|
| I try to cover as much of my code as I can with unit tests, and run functional and integration tests too. |
I primarily write code for embedded ARM processors in avionics systems. |
git, vscode, docker, gdb, github. |
My goal when going through code reviews is to keep them as small in size and scope as possible. |
11-30 seconds |
60% |
10% |
30% |
Show
|
| manually |
NetSuite customization and scripting |
gitlab, notepad++ |
all code and code changes gets reviewed by at least one person. Code reviewers rotate. |
31-60 seconds |
30 |
40 |
30 |
Show
|
| unit and integration tests |
arm, x64 |
jetbrains suite, vs code, sublime text |
3 reviewers on every change, 2 junior and 1 senior |
1-5 minutes |
5 |
3 |
2 |
Show
|
| Manual tests, custom python scripts, ci-fuzz, cpp-check |
ARM processors (STM32, nRF52) with Ethernet/Wi-Fi and various radios. RTOS or bare metal. |
makefiles, cmake, arm gcc, vim, github |
Each PR is reviewed by another developer. |
11-30 seconds |
30 |
40 |
30 |
Show
|
| System level test. Unit tests |
Each product have a different target to compile |
meson, scons, vs code |
In each pull request a review is done. |
1-5 minutes |
4 |
4 |
2 |
Show
|
| GTest + Adhoc |
RTOS = QNX; multiple nodes communicating over EtherCat bus; Simulink based motion control algorithms |
multiple... |
100 % reviews Fish-Eye and Bitbucket |
1-5 minutes |
40 |
20 |
40 |
Show
|
| n/a |
n/a |
n/a |
n/a |
Under 10 seconds |
n/a |
n/a |
n/a |
Show
|
| TDD |
n/a |
pycharm |
I do them |
2-4 hours |
50 |
10 |
40 |
Show
|
| gtest/gmock |
qnx/windows |
qcc
Visual Studio
git
Atlassian tool suite |
Code reviews are mandatory for every commit, several reviewers both internal and external of teams |
11-30 seconds |
60 |
20 |
20 |
Show
|
| googletest |
embedded |
various |
pair programming, walkthroughs, formal reviews. |
31-60 seconds |
30 |
40 |
30 |
Show
|
| unittest module in python |
embedded linux devices |
vim
vscode
pycharm |
we dont do them |
31-60 seconds |
40 |
20 |
40 |
Show
|
| Manual |
Just here to learn a better way to code. No current projects with target system. |
visual studio code |
currently limited in current job function |
11-30 seconds |
50 |
20 |
30 |
Show
|
| Trial and error. |
windows and linux |
visual studio code |
review with other engineers on the project. |
1-5 minutes |
50 |
25 |
25 |
Show
|
| On-target |
Embedded MCUs |
vscode, eclipse |
I get okay feedback but we don't have the software review culture I would have expected. |
1-5 minutes |
50 |
20 |
30 |
Show
|
| right now its non existent |
arm32 bit embedded systems running linux |
vscode, make, scripts for upload |
very informal, usually a small amount of discussion and a pull request soliciting teammates comments |
1-5 minutes |
10 |
10 |
80 |
Show
|
| unit tests, function tests, system tests, manual tests |
IoT devices, embedded product with connection to cloud |
zephyr, git, cmake |
Mine are usually pretty small and concise, |
5-30 minutes |
20 |
40 |
40 |
Show
|