cachepc-linux

Fork of AMDESE/linux with modifications for CachePC side-channel attack
git clone https://git.sinitax.com/sinitax/cachepc-linux
Log | Files | Refs | README | LICENSE | sfeed.txt

coresight.rst (29240B)


      1======================================
      2Coresight - HW Assisted Tracing on ARM
      3======================================
      4
      5   :Author:   Mathieu Poirier <mathieu.poirier@linaro.org>
      6   :Date:     September 11th, 2014
      7
      8Introduction
      9------------
     10
     11Coresight is an umbrella of technologies allowing for the debugging of ARM
     12based SoC.  It includes solutions for JTAG and HW assisted tracing.  This
     13document is concerned with the latter.
     14
     15HW assisted tracing is becoming increasingly useful when dealing with systems
     16that have many SoCs and other components like GPU and DMA engines.  ARM has
     17developed a HW assisted tracing solution by means of different components, each
     18being added to a design at synthesis time to cater to specific tracing needs.
     19Components are generally categorised as source, link and sinks and are
     20(usually) discovered using the AMBA bus.
     21
     22"Sources" generate a compressed stream representing the processor instruction
     23path based on tracing scenarios as configured by users.  From there the stream
     24flows through the coresight system (via ATB bus) using links that are connecting
     25the emanating source to a sink(s).  Sinks serve as endpoints to the coresight
     26implementation, either storing the compressed stream in a memory buffer or
     27creating an interface to the outside world where data can be transferred to a
     28host without fear of filling up the onboard coresight memory buffer.
     29
     30At typical coresight system would look like this::
     31
     32  *****************************************************************
     33 **************************** AMBA AXI  ****************************===||
     34  *****************************************************************    ||
     35        ^                    ^                            |            ||
     36        |                    |                            *            **
     37     0000000    :::::     0000000    :::::    :::::    @@@@@@@    ||||||||||||
     38     0 CPU 0<-->: C :     0 CPU 0<-->: C :    : C :    @ STM @    || System ||
     39  |->0000000    : T :  |->0000000    : T :    : T :<--->@@@@@     || Memory ||
     40  |  #######<-->: I :  |  #######<-->: I :    : I :      @@@<-|   ||||||||||||
     41  |  # ETM #    :::::  |  # PTM #    :::::    :::::       @   |
     42  |   #####      ^ ^   |   #####      ^ !      ^ !        .   |   |||||||||
     43  | |->###       | !   | |->###       | !      | !        .   |   || DAP ||
     44  | |   #        | !   | |   #        | !      | !        .   |   |||||||||
     45  | |   .        | !   | |   .        | !      | !        .   |      |  |
     46  | |   .        | !   | |   .        | !      | !        .   |      |  *
     47  | |   .        | !   | |   .        | !      | !        .   |      | SWD/
     48  | |   .        | !   | |   .        | !      | !        .   |      | JTAG
     49  *****************************************************************<-|
     50 *************************** AMBA Debug APB ************************
     51  *****************************************************************
     52   |    .          !         .          !        !        .    |
     53   |    .          *         .          *        *        .    |
     54  *****************************************************************
     55 ******************** Cross Trigger Matrix (CTM) *******************
     56  *****************************************************************
     57   |    .     ^              .                            .    |
     58   |    *     !              *                            *    |
     59  *****************************************************************
     60 ****************** AMBA Advanced Trace Bus (ATB) ******************
     61  *****************************************************************
     62   |          !                        ===============         |
     63   |          *                         ===== F =====<---------|
     64   |   :::::::::                         ==== U ====
     65   |-->:: CTI ::<!!                       === N ===
     66   |   :::::::::  !                        == N ==
     67   |    ^         *                        == E ==
     68   |    !  &&&&&&&&&       IIIIIII         == L ==
     69   |------>&& ETB &&<......II     I        =======
     70   |    !  &&&&&&&&&       II     I           .
     71   |    !                    I     I          .
     72   |    !                    I REP I<..........
     73   |    !                    I     I
     74   |    !!>&&&&&&&&&       II     I           *Source: ARM ltd.
     75   |------>& TPIU  &<......II    I            DAP = Debug Access Port
     76           &&&&&&&&&       IIIIIII            ETM = Embedded Trace Macrocell
     77               ;                              PTM = Program Trace Macrocell
     78               ;                              CTI = Cross Trigger Interface
     79               *                              ETB = Embedded Trace Buffer
     80          To trace port                       TPIU= Trace Port Interface Unit
     81                                              SWD = Serial Wire Debug
     82
     83While on target configuration of the components is done via the APB bus,
     84all trace data are carried out-of-band on the ATB bus.  The CTM provides
     85a way to aggregate and distribute signals between CoreSight components.
     86
     87The coresight framework provides a central point to represent, configure and
     88manage coresight devices on a platform.  This first implementation centers on
     89the basic tracing functionality, enabling components such ETM/PTM, funnel,
     90replicator, TMC, TPIU and ETB.  Future work will enable more
     91intricate IP blocks such as STM and CTI.
     92
     93
     94Acronyms and Classification
     95---------------------------
     96
     97Acronyms:
     98
     99PTM:
    100    Program Trace Macrocell
    101ETM:
    102    Embedded Trace Macrocell
    103STM:
    104    System trace Macrocell
    105ETB:
    106    Embedded Trace Buffer
    107ITM:
    108    Instrumentation Trace Macrocell
    109TPIU:
    110     Trace Port Interface Unit
    111TMC-ETR:
    112        Trace Memory Controller, configured as Embedded Trace Router
    113TMC-ETF:
    114        Trace Memory Controller, configured as Embedded Trace FIFO
    115CTI:
    116    Cross Trigger Interface
    117
    118Classification:
    119
    120Source:
    121   ETMv3.x ETMv4, PTMv1.0, PTMv1.1, STM, STM500, ITM
    122Link:
    123   Funnel, replicator (intelligent or not), TMC-ETR
    124Sinks:
    125   ETBv1.0, ETB1.1, TPIU, TMC-ETF
    126Misc:
    127   CTI
    128
    129
    130Device Tree Bindings
    131--------------------
    132
    133See Documentation/devicetree/bindings/arm/coresight.txt for details.
    134
    135As of this writing drivers for ITM, STMs and CTIs are not provided but are
    136expected to be added as the solution matures.
    137
    138
    139Framework and implementation
    140----------------------------
    141
    142The coresight framework provides a central point to represent, configure and
    143manage coresight devices on a platform.  Any coresight compliant device can
    144register with the framework for as long as they use the right APIs:
    145
    146.. c:function:: struct coresight_device *coresight_register(struct coresight_desc *desc);
    147.. c:function:: void coresight_unregister(struct coresight_device *csdev);
    148
    149The registering function is taking a ``struct coresight_desc *desc`` and
    150register the device with the core framework. The unregister function takes
    151a reference to a ``struct coresight_device *csdev`` obtained at registration time.
    152
    153If everything goes well during the registration process the new devices will
    154show up under /sys/bus/coresight/devices, as showns here for a TC2 platform::
    155
    156    root:~# ls /sys/bus/coresight/devices/
    157    replicator  20030000.tpiu    2201c000.ptm  2203c000.etm  2203e000.etm
    158    20010000.etb         20040000.funnel  2201d000.ptm  2203d000.etm
    159    root:~#
    160
    161The functions take a ``struct coresight_device``, which looks like this::
    162
    163    struct coresight_desc {
    164            enum coresight_dev_type type;
    165            struct coresight_dev_subtype subtype;
    166            const struct coresight_ops *ops;
    167            struct coresight_platform_data *pdata;
    168            struct device *dev;
    169            const struct attribute_group **groups;
    170    };
    171
    172
    173The "coresight_dev_type" identifies what the device is, i.e, source link or
    174sink while the "coresight_dev_subtype" will characterise that type further.
    175
    176The ``struct coresight_ops`` is mandatory and will tell the framework how to
    177perform base operations related to the components, each component having
    178a different set of requirement. For that ``struct coresight_ops_sink``,
    179``struct coresight_ops_link`` and ``struct coresight_ops_source`` have been
    180provided.
    181
    182The next field ``struct coresight_platform_data *pdata`` is acquired by calling
    183``of_get_coresight_platform_data()``, as part of the driver's _probe routine and
    184``struct device *dev`` gets the device reference embedded in the ``amba_device``::
    185
    186    static int etm_probe(struct amba_device *adev, const struct amba_id *id)
    187    {
    188     ...
    189     ...
    190     drvdata->dev = &adev->dev;
    191     ...
    192    }
    193
    194Specific class of device (source, link, or sink) have generic operations
    195that can be performed on them (see ``struct coresight_ops``). The ``**groups``
    196is a list of sysfs entries pertaining to operations
    197specific to that component only.  "Implementation defined" customisations are
    198expected to be accessed and controlled using those entries.
    199
    200Device Naming scheme
    201--------------------
    202
    203The devices that appear on the "coresight" bus were named the same as their
    204parent devices, i.e, the real devices that appears on AMBA bus or the platform bus.
    205Thus the names were based on the Linux Open Firmware layer naming convention,
    206which follows the base physical address of the device followed by the device
    207type. e.g::
    208
    209    root:~# ls /sys/bus/coresight/devices/
    210     20010000.etf  20040000.funnel      20100000.stm     22040000.etm
    211     22140000.etm  230c0000.funnel      23240000.etm     20030000.tpiu
    212     20070000.etr  20120000.replicator  220c0000.funnel
    213     23040000.etm  23140000.etm         23340000.etm
    214
    215However, with the introduction of ACPI support, the names of the real
    216devices are a bit cryptic and non-obvious. Thus, a new naming scheme was
    217introduced to use more generic names based on the type of the device. The
    218following rules apply::
    219
    220  1) Devices that are bound to CPUs, are named based on the CPU logical
    221     number.
    222
    223     e.g, ETM bound to CPU0 is named "etm0"
    224
    225  2) All other devices follow a pattern, "<device_type_prefix>N", where :
    226
    227	<device_type_prefix> 	- A prefix specific to the type of the device
    228	N			- a sequential number assigned based on the order
    229				  of probing.
    230
    231	e.g, tmc_etf0, tmc_etr0, funnel0, funnel1
    232
    233Thus, with the new scheme the devices could appear as ::
    234
    235    root:~# ls /sys/bus/coresight/devices/
    236     etm0     etm1     etm2         etm3  etm4      etm5      funnel0
    237     funnel1  funnel2  replicator0  stm0  tmc_etf0  tmc_etr0  tpiu0
    238
    239Some of the examples below might refer to old naming scheme and some
    240to the newer scheme, to give a confirmation that what you see on your
    241system is not unexpected. One must use the "names" as they appear on
    242the system under specified locations.
    243
    244Topology Representation
    245-----------------------
    246
    247Each CoreSight component has a ``connections`` directory which will contain
    248links to other CoreSight components. This allows the user to explore the trace
    249topology and for larger systems, determine the most appropriate sink for a
    250given source. The connection information can also be used to establish
    251which CTI devices are connected to a given component. This directory contains a
    252``nr_links`` attribute detailing the number of links in the directory.
    253
    254For an ETM source, in this case ``etm0`` on a Juno platform, a typical
    255arrangement will be::
    256
    257  linaro-developer:~# ls - l /sys/bus/coresight/devices/etm0/connections
    258  <file details>  cti_cpu0 -> ../../../23020000.cti/cti_cpu0
    259  <file details>  nr_links
    260  <file details>  out:0 -> ../../../230c0000.funnel/funnel2
    261
    262Following the out port to ``funnel2``::
    263
    264  linaro-developer:~# ls -l /sys/bus/coresight/devices/funnel2/connections
    265  <file details> in:0 -> ../../../23040000.etm/etm0
    266  <file details> in:1 -> ../../../23140000.etm/etm3
    267  <file details> in:2 -> ../../../23240000.etm/etm4
    268  <file details> in:3 -> ../../../23340000.etm/etm5
    269  <file details> nr_links
    270  <file details> out:0 -> ../../../20040000.funnel/funnel0
    271
    272And again to ``funnel0``::
    273
    274  linaro-developer:~# ls -l /sys/bus/coresight/devices/funnel0/connections
    275  <file details> in:0 -> ../../../220c0000.funnel/funnel1
    276  <file details> in:1 -> ../../../230c0000.funnel/funnel2
    277  <file details> nr_links
    278  <file details> out:0 -> ../../../20010000.etf/tmc_etf0
    279
    280Finding the first sink ``tmc_etf0``. This can be used to collect data
    281as a sink, or as a link to propagate further along the chain::
    282
    283  linaro-developer:~# ls -l /sys/bus/coresight/devices/tmc_etf0/connections
    284  <file details> cti_sys0 -> ../../../20020000.cti/cti_sys0
    285  <file details> in:0 -> ../../../20040000.funnel/funnel0
    286  <file details> nr_links
    287  <file details> out:0 -> ../../../20150000.funnel/funnel4
    288
    289via ``funnel4``::
    290
    291  linaro-developer:~# ls -l /sys/bus/coresight/devices/funnel4/connections
    292  <file details> in:0 -> ../../../20010000.etf/tmc_etf0
    293  <file details> in:1 -> ../../../20140000.etf/tmc_etf1
    294  <file details> nr_links
    295  <file details> out:0 -> ../../../20120000.replicator/replicator0
    296
    297and a ``replicator0``::
    298
    299  linaro-developer:~# ls -l /sys/bus/coresight/devices/replicator0/connections
    300  <file details> in:0 -> ../../../20150000.funnel/funnel4
    301  <file details> nr_links
    302  <file details> out:0 -> ../../../20030000.tpiu/tpiu0
    303  <file details> out:1 -> ../../../20070000.etr/tmc_etr0
    304
    305Arriving at the final sink in the chain, ``tmc_etr0``::
    306
    307  linaro-developer:~# ls -l /sys/bus/coresight/devices/tmc_etr0/connections
    308  <file details> cti_sys0 -> ../../../20020000.cti/cti_sys0
    309  <file details> in:0 -> ../../../20120000.replicator/replicator0
    310  <file details> nr_links
    311
    312As described below, when using sysfs it is sufficient to enable a sink and
    313a source for successful trace. The framework will correctly enable all
    314intermediate links as required.
    315
    316Note: ``cti_sys0`` appears in two of the connections lists above.
    317CTIs can connect to multiple devices and are arranged in a star topology
    318via the CTM. See (Documentation/trace/coresight/coresight-ect.rst)
    319[#fourth]_ for further details.
    320Looking at this device we see 4 connections::
    321
    322  linaro-developer:~# ls -l /sys/bus/coresight/devices/cti_sys0/connections
    323  <file details> nr_links
    324  <file details> stm0 -> ../../../20100000.stm/stm0
    325  <file details> tmc_etf0 -> ../../../20010000.etf/tmc_etf0
    326  <file details> tmc_etr0 -> ../../../20070000.etr/tmc_etr0
    327  <file details> tpiu0 -> ../../../20030000.tpiu/tpiu0
    328
    329
    330How to use the tracer modules
    331-----------------------------
    332
    333There are two ways to use the Coresight framework:
    334
    3351. using the perf cmd line tools.
    3362. interacting directly with the Coresight devices using the sysFS interface.
    337
    338Preference is given to the former as using the sysFS interface
    339requires a deep understanding of the Coresight HW.  The following sections
    340provide details on using both methods.
    341
    3421) Using the sysFS interface:
    343
    344Before trace collection can start, a coresight sink needs to be identified.
    345There is no limit on the amount of sinks (nor sources) that can be enabled at
    346any given moment.  As a generic operation, all device pertaining to the sink
    347class will have an "active" entry in sysfs::
    348
    349    root:/sys/bus/coresight/devices# ls
    350    replicator  20030000.tpiu    2201c000.ptm  2203c000.etm  2203e000.etm
    351    20010000.etb         20040000.funnel  2201d000.ptm  2203d000.etm
    352    root:/sys/bus/coresight/devices# ls 20010000.etb
    353    enable_sink  status  trigger_cntr
    354    root:/sys/bus/coresight/devices# echo 1 > 20010000.etb/enable_sink
    355    root:/sys/bus/coresight/devices# cat 20010000.etb/enable_sink
    356    1
    357    root:/sys/bus/coresight/devices#
    358
    359At boot time the current etm3x driver will configure the first address
    360comparator with "_stext" and "_etext", essentially tracing any instruction
    361that falls within that range.  As such "enabling" a source will immediately
    362trigger a trace capture::
    363
    364    root:/sys/bus/coresight/devices# echo 1 > 2201c000.ptm/enable_source
    365    root:/sys/bus/coresight/devices# cat 2201c000.ptm/enable_source
    366    1
    367    root:/sys/bus/coresight/devices# cat 20010000.etb/status
    368    Depth:          0x2000
    369    Status:         0x1
    370    RAM read ptr:   0x0
    371    RAM wrt ptr:    0x19d3   <----- The write pointer is moving
    372    Trigger cnt:    0x0
    373    Control:        0x1
    374    Flush status:   0x0
    375    Flush ctrl:     0x2001
    376    root:/sys/bus/coresight/devices#
    377
    378Trace collection is stopped the same way::
    379
    380    root:/sys/bus/coresight/devices# echo 0 > 2201c000.ptm/enable_source
    381    root:/sys/bus/coresight/devices#
    382
    383The content of the ETB buffer can be harvested directly from /dev::
    384
    385    root:/sys/bus/coresight/devices# dd if=/dev/20010000.etb \
    386    of=~/cstrace.bin
    387    64+0 records in
    388    64+0 records out
    389    32768 bytes (33 kB) copied, 0.00125258 s, 26.2 MB/s
    390    root:/sys/bus/coresight/devices#
    391
    392The file cstrace.bin can be decompressed using "ptm2human", DS-5 or Trace32.
    393
    394Following is a DS-5 output of an experimental loop that increments a variable up
    395to a certain value.  The example is simple and yet provides a glimpse of the
    396wealth of possibilities that coresight provides.
    397::
    398
    399    Info                                    Tracing enabled
    400    Instruction     106378866       0x8026B53C      E52DE004        false   PUSH     {lr}
    401    Instruction     0       0x8026B540      E24DD00C        false   SUB      sp,sp,#0xc
    402    Instruction     0       0x8026B544      E3A03000        false   MOV      r3,#0
    403    Instruction     0       0x8026B548      E58D3004        false   STR      r3,[sp,#4]
    404    Instruction     0       0x8026B54C      E59D3004        false   LDR      r3,[sp,#4]
    405    Instruction     0       0x8026B550      E3530004        false   CMP      r3,#4
    406    Instruction     0       0x8026B554      E2833001        false   ADD      r3,r3,#1
    407    Instruction     0       0x8026B558      E58D3004        false   STR      r3,[sp,#4]
    408    Instruction     0       0x8026B55C      DAFFFFFA        true    BLE      {pc}-0x10 ; 0x8026b54c
    409    Timestamp                                       Timestamp: 17106715833
    410    Instruction     319     0x8026B54C      E59D3004        false   LDR      r3,[sp,#4]
    411    Instruction     0       0x8026B550      E3530004        false   CMP      r3,#4
    412    Instruction     0       0x8026B554      E2833001        false   ADD      r3,r3,#1
    413    Instruction     0       0x8026B558      E58D3004        false   STR      r3,[sp,#4]
    414    Instruction     0       0x8026B55C      DAFFFFFA        true    BLE      {pc}-0x10 ; 0x8026b54c
    415    Instruction     9       0x8026B54C      E59D3004        false   LDR      r3,[sp,#4]
    416    Instruction     0       0x8026B550      E3530004        false   CMP      r3,#4
    417    Instruction     0       0x8026B554      E2833001        false   ADD      r3,r3,#1
    418    Instruction     0       0x8026B558      E58D3004        false   STR      r3,[sp,#4]
    419    Instruction     0       0x8026B55C      DAFFFFFA        true    BLE      {pc}-0x10 ; 0x8026b54c
    420    Instruction     7       0x8026B54C      E59D3004        false   LDR      r3,[sp,#4]
    421    Instruction     0       0x8026B550      E3530004        false   CMP      r3,#4
    422    Instruction     0       0x8026B554      E2833001        false   ADD      r3,r3,#1
    423    Instruction     0       0x8026B558      E58D3004        false   STR      r3,[sp,#4]
    424    Instruction     0       0x8026B55C      DAFFFFFA        true    BLE      {pc}-0x10 ; 0x8026b54c
    425    Instruction     7       0x8026B54C      E59D3004        false   LDR      r3,[sp,#4]
    426    Instruction     0       0x8026B550      E3530004        false   CMP      r3,#4
    427    Instruction     0       0x8026B554      E2833001        false   ADD      r3,r3,#1
    428    Instruction     0       0x8026B558      E58D3004        false   STR      r3,[sp,#4]
    429    Instruction     0       0x8026B55C      DAFFFFFA        true    BLE      {pc}-0x10 ; 0x8026b54c
    430    Instruction     10      0x8026B54C      E59D3004        false   LDR      r3,[sp,#4]
    431    Instruction     0       0x8026B550      E3530004        false   CMP      r3,#4
    432    Instruction     0       0x8026B554      E2833001        false   ADD      r3,r3,#1
    433    Instruction     0       0x8026B558      E58D3004        false   STR      r3,[sp,#4]
    434    Instruction     0       0x8026B55C      DAFFFFFA        true    BLE      {pc}-0x10 ; 0x8026b54c
    435    Instruction     6       0x8026B560      EE1D3F30        false   MRC      p15,#0x0,r3,c13,c0,#1
    436    Instruction     0       0x8026B564      E1A0100D        false   MOV      r1,sp
    437    Instruction     0       0x8026B568      E3C12D7F        false   BIC      r2,r1,#0x1fc0
    438    Instruction     0       0x8026B56C      E3C2203F        false   BIC      r2,r2,#0x3f
    439    Instruction     0       0x8026B570      E59D1004        false   LDR      r1,[sp,#4]
    440    Instruction     0       0x8026B574      E59F0010        false   LDR      r0,[pc,#16] ; [0x8026B58C] = 0x80550368
    441    Instruction     0       0x8026B578      E592200C        false   LDR      r2,[r2,#0xc]
    442    Instruction     0       0x8026B57C      E59221D0        false   LDR      r2,[r2,#0x1d0]
    443    Instruction     0       0x8026B580      EB07A4CF        true    BL       {pc}+0x1e9344 ; 0x804548c4
    444    Info                                    Tracing enabled
    445    Instruction     13570831        0x8026B584      E28DD00C        false   ADD      sp,sp,#0xc
    446    Instruction     0       0x8026B588      E8BD8000        true    LDM      sp!,{pc}
    447    Timestamp                                       Timestamp: 17107041535
    448
    4492) Using perf framework:
    450
    451Coresight tracers are represented using the Perf framework's Performance
    452Monitoring Unit (PMU) abstraction.  As such the perf framework takes charge of
    453controlling when tracing gets enabled based on when the process of interest is
    454scheduled.  When configured in a system, Coresight PMUs will be listed when
    455queried by the perf command line tool:
    456
    457	linaro@linaro-nano:~$ ./perf list pmu
    458
    459		List of pre-defined events (to be used in -e):
    460
    461		cs_etm//                                    [Kernel PMU event]
    462
    463	linaro@linaro-nano:~$
    464
    465Regardless of the number of tracers available in a system (usually equal to the
    466amount of processor cores), the "cs_etm" PMU will be listed only once.
    467
    468A Coresight PMU works the same way as any other PMU, i.e the name of the PMU is
    469listed along with configuration options within forward slashes '/'.  Since a
    470Coresight system will typically have more than one sink, the name of the sink to
    471work with needs to be specified as an event option.
    472On newer kernels the available sinks are listed in sysFS under
    473($SYSFS)/bus/event_source/devices/cs_etm/sinks/::
    474
    475	root@localhost:/sys/bus/event_source/devices/cs_etm/sinks# ls
    476	tmc_etf0  tmc_etr0  tpiu0
    477
    478On older kernels, this may need to be found from the list of coresight devices,
    479available under ($SYSFS)/bus/coresight/devices/::
    480
    481	root:~# ls /sys/bus/coresight/devices/
    482	 etm0     etm1     etm2         etm3  etm4      etm5      funnel0
    483	 funnel1  funnel2  replicator0  stm0  tmc_etf0  tmc_etr0  tpiu0
    484	root@linaro-nano:~# perf record -e cs_etm/@tmc_etr0/u --per-thread program
    485
    486As mentioned above in section "Device Naming scheme", the names of the devices could
    487look different from what is used in the example above. One must use the device names
    488as it appears under the sysFS.
    489
    490The syntax within the forward slashes '/' is important.  The '@' character
    491tells the parser that a sink is about to be specified and that this is the sink
    492to use for the trace session.
    493
    494More information on the above and other example on how to use Coresight with
    495the perf tools can be found in the "HOWTO.md" file of the openCSD gitHub
    496repository [#third]_.
    497
    4982.1) AutoFDO analysis using the perf tools:
    499
    500perf can be used to record and analyze trace of programs.
    501
    502Execution can be recorded using 'perf record' with the cs_etm event,
    503specifying the name of the sink to record to, e.g::
    504
    505    perf record -e cs_etm/@tmc_etr0/u --per-thread
    506
    507The 'perf report' and 'perf script' commands can be used to analyze execution,
    508synthesizing instruction and branch events from the instruction trace.
    509'perf inject' can be used to replace the trace data with the synthesized events.
    510The --itrace option controls the type and frequency of synthesized events
    511(see perf documentation).
    512
    513Note that only 64-bit programs are currently supported - further work is
    514required to support instruction decode of 32-bit Arm programs.
    515
    5162.2) Tracing PID
    517
    518The kernel can be built to write the PID value into the PE ContextID registers.
    519For a kernel running at EL1, the PID is stored in CONTEXTIDR_EL1.  A PE may
    520implement Arm Virtualization Host Extensions (VHE), which the kernel can
    521run at EL2 as a virtualisation host; in this case, the PID value is stored in
    522CONTEXTIDR_EL2.
    523
    524perf provides PMU formats that program the ETM to insert these values into the
    525trace data; the PMU formats are defined as below:
    526
    527  "contextid1": Available on both EL1 kernel and EL2 kernel.  When the
    528                kernel is running at EL1, "contextid1" enables the PID
    529                tracing; when the kernel is running at EL2, this enables
    530                tracing the PID of guest applications.
    531
    532  "contextid2": Only usable when the kernel is running at EL2.  When
    533                selected, enables PID tracing on EL2 kernel.
    534
    535  "contextid":  Will be an alias for the option that enables PID
    536                tracing.  I.e,
    537                contextid == contextid1, on EL1 kernel.
    538                contextid == contextid2, on EL2 kernel.
    539
    540perf will always enable PID tracing at the relevant EL, this is accomplished by
    541automatically enable the "contextid" config - but for EL2 it is possible to make
    542specific adjustments using configs "contextid1" and "contextid2", E.g. if a user
    543wants to trace PIDs for both host and guest, the two configs "contextid1" and
    544"contextid2" can be set at the same time:
    545
    546  perf record -e cs_etm/contextid1,contextid2/u -- vm
    547
    548
    549Generating coverage files for Feedback Directed Optimization: AutoFDO
    550---------------------------------------------------------------------
    551
    552'perf inject' accepts the --itrace option in which case tracing data is
    553removed and replaced with the synthesized events. e.g.
    554::
    555
    556	perf inject --itrace --strip -i perf.data -o perf.data.new
    557
    558Below is an example of using ARM ETM for autoFDO.  It requires autofdo
    559(https://github.com/google/autofdo) and gcc version 5.  The bubble
    560sort example is from the AutoFDO tutorial (https://gcc.gnu.org/wiki/AutoFDO/Tutorial).
    561::
    562
    563	$ gcc-5 -O3 sort.c -o sort
    564	$ taskset -c 2 ./sort
    565	Bubble sorting array of 30000 elements
    566	5910 ms
    567
    568	$ perf record -e cs_etm/@tmc_etr0/u --per-thread taskset -c 2 ./sort
    569	Bubble sorting array of 30000 elements
    570	12543 ms
    571	[ perf record: Woken up 35 times to write data ]
    572	[ perf record: Captured and wrote 69.640 MB perf.data ]
    573
    574	$ perf inject -i perf.data -o inj.data --itrace=il64 --strip
    575	$ create_gcov --binary=./sort --profile=inj.data --gcov=sort.gcov -gcov_version=1
    576	$ gcc-5 -O3 -fauto-profile=sort.gcov sort.c -o sort_autofdo
    577	$ taskset -c 2 ./sort_autofdo
    578	Bubble sorting array of 30000 elements
    579	5806 ms
    580
    581
    582How to use the STM module
    583-------------------------
    584
    585Using the System Trace Macrocell module is the same as the tracers - the only
    586difference is that clients are driving the trace capture rather
    587than the program flow through the code.
    588
    589As with any other CoreSight component, specifics about the STM tracer can be
    590found in sysfs with more information on each entry being found in [#first]_::
    591
    592    root@genericarmv8:~# ls /sys/bus/coresight/devices/stm0
    593    enable_source   hwevent_select  port_enable     subsystem       uevent
    594    hwevent_enable  mgmt            port_select     traceid
    595    root@genericarmv8:~#
    596
    597Like any other source a sink needs to be identified and the STM enabled before
    598being used::
    599
    600    root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/tmc_etf0/enable_sink
    601    root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/stm0/enable_source
    602
    603From there user space applications can request and use channels using the devfs
    604interface provided for that purpose by the generic STM API::
    605
    606    root@genericarmv8:~# ls -l /dev/stm0
    607    crw-------    1 root     root       10,  61 Jan  3 18:11 /dev/stm0
    608    root@genericarmv8:~#
    609
    610Details on how to use the generic STM API can be found here:
    611- Documentation/trace/stm.rst [#second]_.
    612
    613The CTI & CTM Modules
    614---------------------
    615
    616The CTI (Cross Trigger Interface) provides a set of trigger signals between
    617individual CTIs and components, and can propagate these between all CTIs via
    618channels on the CTM (Cross Trigger Matrix).
    619
    620A separate documentation file is provided to explain the use of these devices.
    621(Documentation/trace/coresight/coresight-ect.rst) [#fourth]_.
    622
    623CoreSight System Configuration
    624------------------------------
    625
    626CoreSight components can be complex devices with many programming options.
    627Furthermore, components can be programmed to interact with each other across the
    628complete system.
    629
    630A CoreSight System Configuration manager is provided to allow these complex programming
    631configurations to be selected and used easily from perf and sysfs.
    632
    633See the separate document for further information.
    634(Documentation/trace/coresight/coresight-config.rst) [#fifth]_.
    635
    636
    637.. [#first] Documentation/ABI/testing/sysfs-bus-coresight-devices-stm
    638
    639.. [#second] Documentation/trace/stm.rst
    640
    641.. [#third] https://github.com/Linaro/perf-opencsd
    642
    643.. [#fourth] Documentation/trace/coresight/coresight-ect.rst
    644
    645.. [#fifth] Documentation/trace/coresight/coresight-config.rst