How to run DejaGNU tests

This article describes how to run testsuites of GNU projects for ARC. This article covers only baremetal toolchain. Tests can be run on different platforms - simulators or hardware platforms; this document covers only free nSIM.

Prerequisites

1. Get sources of the project you want to test.

2. Get source of the "toolchain" repository:

   $ git clone git@github.com:foss-for-synopsys-dwc-arc-processors/toolchain.git

3. Download free nSIM from webpage: https://www.synopsys.com/cgi-bin/dwarcnsim/req1.cgi. Note that requests are manually approved, hence it make take up to 2 workdays for download to be ready.

4. Build toolchain.

Preparing

Create a directory where artifacts and temporary files will be created:

$ mkdir tests
$ cd tests

Create a site.exp file in this directory. toolchain/dejagnu/example_site.exp can be used as an example: srcdir and arc_exec_prefix variables must be set to location of toolchain sources and location of installed toolchain respectively. Variable toolchain_sysroot_dir in that file shouldn't be set for baremetal toolchain testing.

Required environment variables are:

• ARC_MULTILIB_OPTIONS - should be set to compiler that to be tested. Note that first -m should be omitted, but only first. So for example this variables might take value: cpu=arcem -mnorm.
• DEJAGNU - should be set to path to the site.exp file in "toolchain" repository.
• PATH - add toolchain installation location bin directory to PATH - some tests ignore value set in the site.exp file.
• NSIM_HOME - should point to location where nSIM has been untarred.

Example example_site.exp file also specifies temporary directory as ./temp, which must be manually created:

$ mkdir temp

Note that many test still would ignore this directory and instead would create their temporary files in the current directory.

Some actions are specific to particular GNU projects:

• Newlib requires:

$ mkdir targ-include
$ ln -s /home/user/arc_gnu/INSTALL/arc-elf32/include/newlib.h targ-include/

• GCC might require (for some tests):

$ pushd home/user/arc_gnu/gcc/
$ ./contrib/gcc_update --touch
$ popd

• libstdc++ requires:

$ export CXXFLAGS="-O2 -g"

• GDB requires:

$ testsuite=$/home/user/arc_gnu/gdb/gdb/testsuite
$ mkdir $(ls -1d $testsuite/gdb.* | grep -Po '(?<=\/)[^\/]+$')

Also arc-nsim.exp board will require an environment variable ARC_NSIM_PROPS to be set and to contain path to nSIM properties file that specifies ISA options.

Running

Now, that all is set, test suite can be run with:

$ runtest --tool=<project-to-test> --target_board=<board>

Where <project-to-test> can be: gcc, g++, binutils, ld, gas, newlib, libstdc++ or gdb. board for free nSIM can be arc-nsim.exp or arc-sim-nsimdrv.exp. The former runs nSIM as a GDBserver, while the latter will run nSIM as a standalone simulator, which is faster and more stable, but not suitable to run GDB testsuite.