cachepc-linux

Fork of AMDESE/linux with modifications for CachePC side-channel attack
git clone https://git.sinitax.com/sinitax/cachepc-linux
Log | Files | Refs | README | LICENSE | sfeed.txt

running_tips.rst (8003B)


      1.. SPDX-License-Identifier: GPL-2.0
      2
      3============================
      4Tips For Running KUnit Tests
      5============================
      6
      7Using ``kunit.py run`` ("kunit tool")
      8=====================================
      9
     10Running from any directory
     11--------------------------
     12
     13It can be handy to create a bash function like:
     14
     15.. code-block:: bash
     16
     17	function run_kunit() {
     18	  ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run $@ )
     19	}
     20
     21.. note::
     22	Early versions of ``kunit.py`` (before 5.6) didn't work unless run from
     23	the kernel root, hence the use of a subshell and ``cd``.
     24
     25Running a subset of tests
     26-------------------------
     27
     28``kunit.py run`` accepts an optional glob argument to filter tests. The format
     29is ``"<suite_glob>[.test_glob]"``.
     30
     31Say that we wanted to run the sysctl tests, we could do so via:
     32
     33.. code-block:: bash
     34
     35	$ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNIT_ALL_TESTS=y' > .kunit/.kunitconfig
     36	$ ./tools/testing/kunit/kunit.py run 'sysctl*'
     37
     38We can filter down to just the "write" tests via:
     39
     40.. code-block:: bash
     41
     42	$ echo -e 'CONFIG_KUNIT=y\nCONFIG_KUNIT_ALL_TESTS=y' > .kunit/.kunitconfig
     43	$ ./tools/testing/kunit/kunit.py run 'sysctl*.*write*'
     44
     45We're paying the cost of building more tests than we need this way, but it's
     46easier than fiddling with ``.kunitconfig`` files or commenting out
     47``kunit_suite``'s.
     48
     49However, if we wanted to define a set of tests in a less ad hoc way, the next
     50tip is useful.
     51
     52Defining a set of tests
     53-----------------------
     54
     55``kunit.py run`` (along with ``build``, and ``config``) supports a
     56``--kunitconfig`` flag. So if you have a set of tests that you want to run on a
     57regular basis (especially if they have other dependencies), you can create a
     58specific ``.kunitconfig`` for them.
     59
     60E.g. kunit has one for its tests:
     61
     62.. code-block:: bash
     63
     64	$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit/.kunitconfig
     65
     66Alternatively, if you're following the convention of naming your
     67file ``.kunitconfig``, you can just pass in the dir, e.g.
     68
     69.. code-block:: bash
     70
     71	$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit
     72
     73.. note::
     74	This is a relatively new feature (5.12+) so we don't have any
     75	conventions yet about on what files should be checked in versus just
     76	kept around locally. It's up to you and your maintainer to decide if a
     77	config is useful enough to submit (and therefore have to maintain).
     78
     79.. note::
     80	Having ``.kunitconfig`` fragments in a parent and child directory is
     81	iffy. There's discussion about adding an "import" statement in these
     82	files to make it possible to have a top-level config run tests from all
     83	child directories. But that would mean ``.kunitconfig`` files are no
     84	longer just simple .config fragments.
     85
     86	One alternative would be to have kunit tool recursively combine configs
     87	automagically, but tests could theoretically depend on incompatible
     88	options, so handling that would be tricky.
     89
     90Setting kernel commandline parameters
     91-------------------------------------
     92
     93You can use ``--kernel_args`` to pass arbitrary kernel arguments, e.g.
     94
     95.. code-block:: bash
     96
     97	$ ./tools/testing/kunit/kunit.py run --kernel_args=param=42 --kernel_args=param2=false
     98
     99
    100Generating code coverage reports under UML
    101------------------------------------------
    102
    103.. note::
    104	TODO(brendanhiggins@google.com): There are various issues with UML and
    105	versions of gcc 7 and up. You're likely to run into missing ``.gcda``
    106	files or compile errors.
    107
    108This is different from the "normal" way of getting coverage information that is
    109documented in Documentation/dev-tools/gcov.rst.
    110
    111Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options:
    112
    113.. code-block:: none
    114
    115	CONFIG_DEBUG_KERNEL=y
    116	CONFIG_DEBUG_INFO=y
    117	CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
    118	CONFIG_GCOV=y
    119
    120
    121Putting it together into a copy-pastable sequence of commands:
    122
    123.. code-block:: bash
    124
    125	# Append coverage options to the current config
    126	$ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
    127	$ ./tools/testing/kunit/kunit.py run
    128	# Extract the coverage information from the build dir (.kunit/)
    129	$ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/
    130
    131	# From here on, it's the same process as with CONFIG_GCOV_KERNEL=y
    132	# E.g. can generate an HTML report in a tmp dir like so:
    133	$ genhtml -o /tmp/coverage_html coverage.info
    134
    135
    136If your installed version of gcc doesn't work, you can tweak the steps:
    137
    138.. code-block:: bash
    139
    140	$ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
    141	$ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6
    142
    143
    144Running tests manually
    145======================
    146
    147Running tests without using ``kunit.py run`` is also an important use case.
    148Currently it's your only option if you want to test on architectures other than
    149UML.
    150
    151As running the tests under UML is fairly straightforward (configure and compile
    152the kernel, run the ``./linux`` binary), this section will focus on testing
    153non-UML architectures.
    154
    155
    156Running built-in tests
    157----------------------
    158
    159When setting tests to ``=y``, the tests will run as part of boot and print
    160results to dmesg in TAP format. So you just need to add your tests to your
    161``.config``, build and boot your kernel as normal.
    162
    163So if we compiled our kernel with:
    164
    165.. code-block:: none
    166
    167	CONFIG_KUNIT=y
    168	CONFIG_KUNIT_EXAMPLE_TEST=y
    169
    170Then we'd see output like this in dmesg signaling the test ran and passed:
    171
    172.. code-block:: none
    173
    174	TAP version 14
    175	1..1
    176	    # Subtest: example
    177	    1..1
    178	    # example_simple_test: initializing
    179	    ok 1 - example_simple_test
    180	ok 1 - example
    181
    182Running tests as modules
    183------------------------
    184
    185Depending on the tests, you can build them as loadable modules.
    186
    187For example, we'd change the config options from before to
    188
    189.. code-block:: none
    190
    191	CONFIG_KUNIT=y
    192	CONFIG_KUNIT_EXAMPLE_TEST=m
    193
    194Then after booting into our kernel, we can run the test via
    195
    196.. code-block:: none
    197
    198	$ modprobe kunit-example-test
    199
    200This will then cause it to print TAP output to stdout.
    201
    202.. note::
    203	The ``modprobe`` will *not* have a non-zero exit code if any test
    204	failed (as of 5.13). But ``kunit.py parse`` would, see below.
    205
    206.. note::
    207	You can set ``CONFIG_KUNIT=m`` as well, however, some features will not
    208	work and thus some tests might break. Ideally tests would specify they
    209	depend on ``KUNIT=y`` in their ``Kconfig``'s, but this is an edge case
    210	most test authors won't think about.
    211	As of 5.13, the only difference is that ``current->kunit_test`` will
    212	not exist.
    213
    214Pretty-printing results
    215-----------------------
    216
    217You can use ``kunit.py parse`` to parse dmesg for test output and print out
    218results in the same familiar format that ``kunit.py run`` does.
    219
    220.. code-block:: bash
    221
    222	$ ./tools/testing/kunit/kunit.py parse /var/log/dmesg
    223
    224
    225Retrieving per suite results
    226----------------------------
    227
    228Regardless of how you're running your tests, you can enable
    229``CONFIG_KUNIT_DEBUGFS`` to expose per-suite TAP-formatted results:
    230
    231.. code-block:: none
    232
    233	CONFIG_KUNIT=y
    234	CONFIG_KUNIT_EXAMPLE_TEST=m
    235	CONFIG_KUNIT_DEBUGFS=y
    236
    237The results for each suite will be exposed under
    238``/sys/kernel/debug/kunit/<suite>/results``.
    239So using our example config:
    240
    241.. code-block:: bash
    242
    243	$ modprobe kunit-example-test > /dev/null
    244	$ cat /sys/kernel/debug/kunit/example/results
    245	... <TAP output> ...
    246
    247	# After removing the module, the corresponding files will go away
    248	$ modprobe -r kunit-example-test
    249	$ cat /sys/kernel/debug/kunit/example/results
    250	/sys/kernel/debug/kunit/example/results: No such file or directory
    251
    252Generating code coverage reports
    253--------------------------------
    254
    255See Documentation/dev-tools/gcov.rst for details on how to do this.
    256
    257The only vaguely KUnit-specific advice here is that you probably want to build
    258your tests as modules. That way you can isolate the coverage from tests from
    259other code executed during boot, e.g.
    260
    261.. code-block:: bash
    262
    263	# Reset coverage counters before running the test.
    264	$ echo 0 > /sys/kernel/debug/gcov/reset
    265	$ modprobe kunit-example-test