NTFC (NuttX Test Framework for Community)

NTFC enables automated testing for NuttX across simulator, QEMU, and real hardware via serial interface. The framework automatically detects available applications in NuttX images and executes applicable tests using pytest-based test cases.

The framework and official test cases are available at:

Detailed documentation is available in the framework repository.

CI runtime test cases use NTFC. Configurations with an NTFC runner include:

Running a CI Test Target Locally

Run NTFC from the top-level NuttX source directory. The relative paths in a target’s config.yaml file, including the NuttX configuration and image paths, are resolved from that directory. A working NuttX build environment and any emulator or hardware required by the target must already be available.

Create a Python 3.10 or newer virtual environment and install the NTFC and test case versions currently used by NuttX CI:

$ python3 -m venv ../ntfc-venv
$ . ../ntfc-venv/bin/activate
$ python -m pip install ntfc==0.0.1
$ git clone --branch release-0.0.2 --depth 1 \
    https://github.com/apache/nuttx-ntfc-testing ../nuttx-ntfc-testing

Build the target in the usual way. For example, the simulator CI configuration uses the Make build:

$ ./tools/configure.sh sim:citest
$ make

Configurations whose config.yaml refers to ./build/nuttx require an out-of-tree CMake build instead. For example, build qemu-armv8a:citest with:

$ cmake -B build -DBOARD_CONFIG=qemu-armv8a:citest -GNinja
$ cmake --build build

Then pass the test case checkout and the target’s NTFC configuration files to NTFC. For sim:citest this is:

$ python -m ntfc test \
    --testpath=../nuttx-ntfc-testing \
    --confpath=boards/sim/sim/sim/configs/citest/config.yaml \
    --jsonconf=boards/sim/sim/sim/configs/citest/session.json

For another target, replace the build target and both configuration paths with the files from that target’s configs/<configuration> directory. NTFC selects the applicable tests from the NuttX configuration and image and stores the test results under result/ by default.

NTFC exports test logs as CI artifacts. This allows test logs (including ostest output) to be downloaded directly from the CI for all targets where the citest configuration is enabled.