Training Preparation -- Current Practices of 100 Attendees

How do you test your code? What is your target system? What development tools do you use? What is your code review practice? How much time does it take to do an incremental build, load, and start running your code so you can start to test it? What percentage of your development time is spent coding? What percentage of your development time is spent testing? What percentage of your development time is spent debugging?
I haven't really done much development work at my new company yet, so I'm not sure... Software running on a laptop, sometimes in areas where internet is spotty or non existent CLion At least two people must review the code. The reviewer should test the code to ensure it works. 11-30 seconds 40 40 20 Show
Running it on the target machine and exercising it manually or hard coding stuff to test corner cases. I've written some developer tests using CppUTest and Ceedling on different projects, but they are very immature and there are too few of them. Main project is 32-bit arm embedded linux device and there is a side project based on a dsPIC device CMake, GCC, XC-DSC, VSCode Just quick looks at PRs. 1-5 minutes 30 35 35 Show
Unity test on target. We are slowly shifting to the Ceedling framework to run unit tests on PC. RISC microcontroller. ARC EM7D CPU. CMake, Synopsys ARC toolchain, VS code. Pull request reviews within Azure Dev Ops. At least 2 approvals, resolve comments, test passing. 5-30 minutes 45 10 45 Show
Bluetooth Protocol-level test suites System-On-Chip with Bluetooth stack CMake, cross-compilers, debugger, protocol analyzers, conformance testers Reviews performed thru Azure DevOps with two reviewers required to push a PR. 11-30 seconds 20 50 30 Show
Manually... I wish we had a pipeline X86 laptop, or x86 NUC on a UxV Clion, VSCode 1 in process review at 50% completion, 1 at pull request with run testing 1-5 minutes 50 30 20 Show
Static analysis, system test, integration test Low power MCU, modem, RF chip Cmake, gcc compiler, vs code, st-link, cubemx programmer It is mostly to check if changes are made in line with acceptance criteria 1-5 minutes 30 20 50 Show
Always test behavior in system after code is written. Varies depending on function, but may include using printf statements or debugger breakpoints. Asset tracking device using ARM Cortex MCU GNU C Compiler, GDB debugger We use a web-based code reviewer to perform peer reviews before every commit to our trunk. 1-5 minutes 60 25 15 Show
unit tests, integration tests, e2e tests, performance tests N/A Git, Jenkins, just, GitHub Actions We do not often have code reviews, we use pair programming Under 10 seconds 20 60 20 Show
I run automated unit tests using CxxTest and stubs. We develop the ERTMS on-board system for railway applications. I use Visual Studio Code and Visual Studio 2022. We make peer reviews for every change 1-5 minutes 50 30 20 Show
Off-target unit-testing, on-target HIL tests. Distributed system with internal interfaces (UART), external Debug CLI. hard realtime. IDE Code coverage Lint Ad-hoc and retrospective. 30-60 minutes 40 10 50 Show
- Primarily unit tests (in gtest) - Secondary: system tests using a python based framework (built on top of pytest). - Manual tests on running system Distributed embedded system (but no real limitations to memory allocation or exceptions). IDE (Clion+IDE), sourcetrail, main project toolchain (debian12 gcc). Codescene, Coverity, copilot Our agile team is heavily invested in code reviews. 1-5 minutes 60% 30% 10% Show
build then test on target *_*, if it fails debug then repeat MCU based target IDE provided by MCU manufacturer there are almost no peer review, if I found something that may cause a problem I will fix it. 5-30 minutes 30% 10% 60% Show
Manual Platform? Windows/GNU+Linux IDE, clang-tidy Good. Learning a lot from them. Under 10 seconds 5 2 3 Show
n/a PC Jetbrains Visual Studio All check ins require a code review 1-5 minutes 6 1 3 Show
function and some gtest Embedded STM32's, esp's and nordic series of ICs. Bare metal, idf or Zephyr RTOS Jlink ultra, salea, scope, multimeter, SCA tools, lizard cognitive load Mob review, AI supported and PR with context 11-30 seconds 50 40 10 Show
Legacy code. Only perform UAT for any custom changes made to the code. Tried to implement Unit testing framework unity to the legacy code but didn't go well. We use 8bit Pic Microcontroller MPLAB X nil 30-60 minutes 5/10 3/10 2/10 Show
Sandbox functions - test functions by writing my own unit tests IoT systems MPLabX, Cypress Creator (PSOC), Simplicity Studio, Source meters, DMM, Review changes with other programmers. Categorize into low, med & high risk changes Under 10 seconds 30 40 30 Show
Unit Test (unity), integration tests (Python), code coverage currently performed on FPGA with LDRA, moving to GCOV and CMock. Embedded system based on ARC CPU for Bluetooth LE SoC. Metaware C Compiler/Debugger. Python Done via Azure DevOps during Pull Request integration. 31-60 seconds 30 40 30 Show
We can execute unit tests on the target or without the target. Bluetooth SOC using ARC CPU and our custom IPs. We write all the drivers and integrate a BLE stack. Metaware compiler/debuguer with Visual Studio Code Using Azure Devops pull request to make the code review 30-60 minutes 40 40 20 Show
Unit Tests, we are able to run them on the target or without the target. Regression (Integration) tests. We write python tests which send HCI commands over SPI to the target. Bluetooth SOC using ARC CPU and our custom IPs. We write all the drivers and integrate a BLE stack. Metaware compiler/debuguer with Visual Studio Code. We use Azure DevOps and git for pull requests. 31-60 seconds 30 50 20 Show
Using test suites with CxxTest I don't understand the question Microsoft Visual Studio Code Microsoft Visual Studio Tools: QAC++ static analysis, CxxTest framework, Make build system, integrated regression testing 1-2 hours 60 10 30 Show
I run it on the hardware and test for functionality Mostly STM32 Mostly STM32 Done more senior coders Under 10 seconds 30% 20% 50% Show
Unit and integration Embedded PIC16,PIC32 and ESP32 Notepad++ What code review? 1-5 minutes 80 10 10 Show
Using unit test and developer tests. Programming using Visual studio code and compiling debug code with MSVC/MSVC++ 31-60 seconds 5 2 3 Show
Tdd mostly since last year, mncks, DLP Embedded Xilinx devices Qemu, libcheck, mocks Deep code review, understanding of requirements Under 10 seconds 50 40 10 Show
Rust unit test, Boost Test Linux based embedded system Rust standard tools, c/c++ linter etc 1) Does it follow the standard 2) Understand the problem 3) Evaluate the proposed solution 5-30 minutes 40 20 40 Show
unit test Multiple peripheral system STM32 CUBE IDE 2 CYCLE REVIEW Under 10 seconds 30 30 40 Show
Mostly through firmware level verifications with hardware to sus out edge cases. Core functions are identified and individual development tests laid out. This often happens after much of the code is written though. Low level firmware in microcontrollers (AVR,ARM, PIC), but also GUI dev, webservers, embedded STM32 IDE, Microchip Studio, Visual Studio, Espressif IDE, Arduino IDE, MPLabs Code reviews are organic and change from project to project based on available senior developers 5-30 minutes 50 10 40 Show
TDD frame work stm32f427 STM32CUBEIDE no 5-30 minutes 50 30 20 Show
With IDE. STM32 Microcontroller. STM32CubeIDE I don't have a code review experience. 1 day or more 10 10 80 Show
Debugging tools STM32 and ESP32, we will have some other chips in the future. STM32CubeIDE Visual Studio Code and a variety of plugins At work they have been minimal I hope this course helps us establish better practices. 31-60 seconds 20 40 40 Show
Manual testing STM32 STMCube IDE We dont have that process setup yet 31-60 seconds 4 2 4 Show
I attempt to do TDD, but sometimes revert back to not writing tests (particularly when I'm rushing) Desktop, Web, and Embedded Cortex M processor for an IoT application Simplicity Studio Every feature change goes through code reviews before merging with the main code branch 1-5 minutes 50 20 30 Show
Printf statement to follow-up code execution, toggling GPIOs when performances are requested. Used VectorCast for unit and integration testing (long time ago). ARC CPU with few KBytes of memory. Synopsys ARC toolchain and debugger. Git for source code management. Digilent Digital Discovery Pull requests done when a branch has to be merged within the develop main branch. 11-30 seconds 30 30 40 Show
On Device Black box testing Multiple ARM 32-bit systems. One multi core WiFi/BLE, one BLE, one low level hardware driver. VS Code GDB Various hardware debug tools One Person Firmware team. Self Review and sometimes AI tools 31-60 seconds 75 10 15 Show
pytest + python + powershell Arm Cortex M7, comms mostly via CAN VS Code MCUXpresso Segger J-Link Picoscope non exist 31-60 seconds 30 40 30 Show
Blackbox System Level Test Multiple Cortex-MX devices VS Code + extensions Salea I am the only Embedded Developer on my team currently. I need better tools for test/etc 31-60 seconds 50 10 40 Show
Unit testing, Integration Testing, End-to-End Testing Windows/AMD64 VS Code, CMake, MSVC BuildTools, GoogleTest, Git, OpenCppCoverage, BitBucket 2-person review on pull-requests 1-5 minutes 40 20 10 Show
Ideally, unit-testing off target HW, if "required", on HW. Integration testing using our QA team's harness, then any BLE SIG tests. I also will do some basic personal smoke testing of my features iteratively. ARCv2 ASIC running BLE stack. VS Code, GDB, gcc toolchain when building on Linux, Docker, Metaware toolchain when built on target. First, I review over the intent of the change, then I go over other things like readability, tests. 5-30 minutes 40% 30% 30% Show
In simulator with JSON based inputs. Embedded system running embedded software. VS Code, Git etc. Code is thoroughly tested and reviewed. 5-30 minutes 45% 40% 15% Show
Unit testing, integration testing, qualification tests, and with an internally development test system. Embedded ARC VS Code, Metaware development environment, python Code reviews are preformed on an informal basis, but are required. 31-60 seconds 25 40 35 Show
unit tests, script/automation tests and manual tests embedded systems running Linux VS Code, Pycharm Colaboration tools: Confluence, Jira, Bitbucket, Jenkins mandatory in Bitbucket pull-requests 30-60 minutes 50% 30% 20% Show
Test cases EM made SoC VS Code, WSL2, Confluence n/a 1-5 minutes 8 8 10 Show
Unit tests and manual testing. A55 and Nordic processors running Zephyr. VS Code, gcc, git. All pull requests are reviewed by another person to check basic functionality and completion of DoD. 11-30 seconds 60% 20% 20% Show
Unit tests and integration tests run in CI/CD pipeline Linux, running as a controller for attached devices VS code Reviews are done by team members before any code is merged into the development branch. 30-60 minutes 4 2 4 Show
Mostly manual with occasional integration/unit testing Internal Engine Tooling VS, VSCode, Intellij Custom tool for code reviewws Under 10 seconds 2 3 5 Show
NUnit WINCE, moving to Linux later this year ARM based VS, VScode, Resharper Jira ticket -> Dev-branch -> PR -> CodeReview 11-30 seconds 40 50 10 Show
Manual, SITL, HITL Embedded linux or embedded VSCode We have peer reviews by a senior person 5-30 minutes 40 40 20 Show
Writing unit tests, integration tests and end-to-end tests N/A VSCode Talking through with another developer 11-30 seconds 7 1 2 Show
Yes Embedded Linux VSCode Bitbucket is used for code reviews. Under 10 seconds 6 2 2 Show
Manual functional / behavioral test suite I started at Whisker as a FW QA Engineer in July, I have become familiar with Ceedling and am in process of adding unit test capabilities to FW team. Main MCU is PIC but is being phased out with STM32, all targets have companion ESP32 for Wi-Fi. VSCode Espressif's ESP-IDF STM32CubeIDE Have not participated in one, but have recently set forth a plan to make this a regular occurance. 1-5 minutes n/a n/a n/a Show
TDD Battery-powered, embedded ARM A-core device VSCode and open-source compilers Quick checks of functionality and making sure the objectives of the task/story were accomplished. Under 10 seconds 40 40 20 Show
QA Feature regession testing for functionality. Firmware engineer edgecase testing. ESP32 WROOM 32E SoC VSCode with extensions for C and python Currently we have 1 other FW engineer to a Pull Request 1-5 minutes 30 50 20 Show
Occasional unit tests. By running the code. Mostly STM32 based boards. PX4 Autopilot. VSCode, Emacs occasionally. GDB, some of the simple SWD probes, STLink. A team member or lead reviews a pull request. Not done as a group. 11-30 seconds 80% 5% 15% Show
using shell on zephyr imx93 platform running Zephyr RTOS VSCode, GIT and bitbucket Average 50 hours we spend for every sprint of 2 weeks length. 11-30 seconds 10 5 5 Show
Unit testing with CppUTest before merging and periodically performing integration tests Testing features of medical devices VSCode, IAR, QtCreator, Bitbucket Reviews completed before merging code 1-5 minutes 35% 25% 40% Show
Unit testing, functional testing, manual testing, regression testing wireless embedded systems VSCode, Linux, IDEs peer reviews and SME approval 11-30 seconds 5 6 8 Show
Automated test cases Telecom optical equipment VSCode, Vim Constructive feedback will be provided and most of the corner cases will be evaluated 1-5 minutes 60 25 15 Show
A mix of unit tests and manual testing - VSCode, Visual Studio - 11-30 seconds 4 2 4 Show
manually ARM Cortex-A55 running Zephyr RTOS, and an ARM Cortex-M device (BLE radio) also running Zephyr. VSCode, gcc, git/bitbucket Code reviews are done manually for every pull request. 11-30 seconds 25 50 25 Show
Management Framework Integration Tests (MFIT), python scripting end2end tests. Unit Tests (Gtest) Debian OS for Telecommunications VSCode, with some extensions At least one reviewers must approve before delivery, and none can disapprove. 1-5 minutes 20 60 20 Show
I am the coach and faciliator I am not actively programming :) x86 VScode we used to 2 level code reviews and prefer code 1-2 hours 5 5 5 Show
Using prints and testing behavior Embedded system running Zephyr VScode JLink StateSmith All code is reviewed before merging to master. Code is reviewed and tested Under 10 seconds 50% 25% 25% Show
I test edge cases and then come up with a test plan for a technician to run. embedded devices VScode, eclipse based IDEs I have never done one 1-5 minutes 3 3 4 Show
Unit, integration & end-to-end tests, CI & manual Various embedded systems Various: VSCode, functional safety compilers, CI Scrum 11-30 seconds 25 10 25 Show
Iam not doing currently heavy programming. But I did test using automated unit tests with JUnit TDD as mindset to develop embedded app. Visual Code, Unity... using primarily Pull request. 1-5 minutes XX XX XX Show
First unit testing, then integration tests (automatic when possible), and finally manual system level partial testing (full system level verification and validation is done by other departments). Usually embedded systems (RTOS or linux), but sometimes PC Visual Code, makefile, cmake, propietary compilers, git, QAC static analyser Our processes include peer reviews before the feature branch is merged. 1-5 minutes 30 30 40 Show
Mix of manual and unit tests PC Visual Studio Mostly used to spot architectural inconsistencies and standards deviations. 11-30 seconds 60 10 30 Show
Primarily manually with some unit tests Cross-platform for Windows and macOS Visual Studio Often asynchronous, sometimes used for spotting bugs as opposed to purely for alignment/sharing 11-30 seconds 5 2 3 Show
gtest windows/linux/osx/ios/android Visual Studio Every change is reviewed, but not blocking. 11-30 seconds 7 1 2 Show
Initially, personally against requirements. Then some basic CID tests, then via games played on internal regions, then more extensively via Public Beta games. All flavours of PC/Mac and mobile. Visual Studio 1 - 4 reviewers required per change. Reviews via Swarm 31-60 seconds 50 10 40 Show
Scenario testing with scripted events, verfying certain outcomes. Automated testing with mouse / keyboard interactions. We test video game functionality and windows programs. Visual Studio We have SMEs look at code as well as TLs. We have a system for people to become reviewers. 5-30 minutes 60 15 25 Show
Unit tests, functional tests, manual QA Desktop and mobile devices Visual Studio Qt Creator Emacs Renderdoc Perforce Swarm Under 10 seconds 5 1 4 Show
Combination of unit and functional tests Game Engine Visual Studio VS Code Visual Assist Perforce Proprietary We have them for every checking from a selection of SME's team members 11-30 seconds 60 30 10 Show
Unitary tests, corner tests, functional tests and (if applicable) performance tests. Embedded Linux-based systems. GNSS receivers. System on chips. Visual Studio Code IDE (previously Eclipse). gcov/lcov, cppcheck, Valgrind, clang-tidy/format Deep code reviews about the functionality of the code, and docs, convention, etc done by CI Under 10 seconds 4 4 2 Show
Run through a series of test in editor and game to confirm expected behavior, and identify any edge cases We develop for PC, Mac, Android, and IOS, but primarily target PCs. The codebase is > 18 years old Visual Studio, RenderDoc, Sublime Text, We use Swarm, and Peer review our code, requiring at least one SME sign off 1-5 minutes 3 1 6 Show
C: flash it onto an MCU , step thru with debugger until something goes wrong. In RTOS, hopefully with logging messages CC2340, MSP430, python is for automating my lab Visual studio + copilot CCstudio None, solo 11-30 seconds 40 10 50 Show
Usually running in a debugger. Windows and Mac PCs Visual studio, neovim, vscode, xcode. When reviewing I focus on potential issues/bugs and maintainability. 1-5 minutes 20 30 50 Show
Ah-hoc playthroughs. PC games, typically low-end machines e.g. older laptops. VisualStudio + in-house tools. 2-4 people review with discussions. 11-30 seconds 30 30 40 Show
unit test, integration tests, system tests Card Application SW running on zync ultrascale arm cortex core processor bitbucket, eclipse, ms code, various plugins in bitbucket. Required review and approval for all PRs 1-5 minutes 30 40 30 Show
unit test (gtest), top level tests (system integratoin tests) huge legacy model (several million lines) that has existed over a decade, maintained by many teams c/c++/python/assembly insanely strict; including a must-add test case 5-30 minutes 25 35 40 Show
depends on the project. - on target with unity - off target with ceedling - system level integration tests it varies - arc core (ASICS and general SoC) - risc V core (ASICS) - some assembly (custom ISA) ci/cd pipelines clang format/tidy code coverage tools MISRA etc... requirement management lacking. It seems we never have enough time for good reviews. 31-60 seconds 5 3 2 Show
As ASPICE mandates (3 layers) It's a Bluetooth SOC. cmake, clang goodies, CPU supplier toolchain, IBM DOORS, Azure devops, etc. Formally, there is a checklist. Practically, we use our engineering judgement. Under 10 seconds 2 7 1 Show
some of the legacy code here is tested using tools developed after Grenning training (10+ years ago?). I know zero details beyond: "the tests run at build time" various embedded systems, more towards the 32 bit microcontroller eclipse, xdc tools informal if existent 11-30 seconds 3 3 3 Show
unit tests or python scripts embeeded device, with multiple applications mainly coded in c++ gcc, gdb, coverity, ctc, valgrind, visualcode, vim, libasan, cppcheck, codescene we have code reviews for each pull request 31-60 seconds 60 30 10 Show
Unit test, Integration Test, System Test, Regression Test x86, arm gcc, vscode, make, cmake NTR 31-60 seconds 40 20 40 Show
unit tests up to a minimum coverage, robot framework for system level tests, "manual" functional tests satellites on-board computer git, cmake as a build system, vscode, self-hosted LLM code reviews are requested by developers when they think they need it, focused on design 1-2 hours 40 30 30 Show
unit test with cFS, system test with robot framework, "manual" functional test spacecraft on-board computer git, cmake build system, vscode code reviews are requested by developers when they finish a task, they are mainly based on design 1-2 hours 30% 30% 40% Show
manually NetSuite customization and scripting gitlab, notepad++ all code and code changes gets reviewed by at least one person. Code reviewers rotate. 31-60 seconds 30 40 30 Show
Creating test procedures based on code features and validating against. Embedded devices, iot. ide, repository control, text editors, ai, debuggers. Have not had any yet with current company. 11-30 seconds 33 33 33 Show
Manual tests, custom python scripts, ci-fuzz, cpp-check ARM processors (STM32, nRF52) with Ethernet/Wi-Fi and various radios. RTOS or bare metal. makefiles, cmake, arm gcc, vim, github Each PR is reviewed by another developer. 11-30 seconds 30 40 30 Show
UT Yet to explore normal Team reviews 30-60 minutes 3 3 4 Show
mainly ml code, so includes validating data pipelines, model performance, and reproducibility using the unit tests, integration tests, and metric based evaluations microcontrollers, embedded devices, edge ai accelerators and npus pytorch / tensorflow, huggingface, scikit, xgboost, mlflow, onnx, nvidia focus on code clarity, reproducibility, data handling, evaluation logic. mix of software and ml 1-5 minutes 50 30 20 Show
in system testing various, mostly embedded MIPs various use codecollaborator 1-5 minutes 40 10 50 Show
json based test input and manual verification of output optical embedded system visual studio git bitbucket good. 2-4 hours 5 5 3 Show
using the bluetooth qualification tests and verifying manually in specific instances an embedded firmware system written in C running on target visual studio code as an IDE, cmake to build the firmware, ninja as a compiler code reviews happen in PRs by the people reviewing the change, rather informally done asynchronously Under 10 seconds 50 20 30 Show
manual testing in real products linux and freertos environment visual studio, logic analyzer, oscilloscope, jtag none 31-60 seconds 50 10 40 Show
Build it and run it. nRF54 and nRF52 devices and iMX93 devices. vsCode Zephyr West We have a pull-request system and reviews must be done. 11-30 seconds 3 2 5 Show
Manually Unity ARC MCU vscode clang-format PRs must have 2 approvals Under 10 seconds 20 40 40 Show
Most of the times it is manual test. We build and replace the modified binary/library in target system and check. SW applications running on Debian vscode, tabnine Bitbucket PR review 1-5 minutes 2 5 8 Show
The tag cloud words are from attendees 'Current test practice'