Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.
1 Coresight - HW Assisted Tracing on ARM 2 ====================================== 3 4 Author: Mathieu Poirier <mathieu.poirier@linaro.org> 5 Date: September 11th, 2014 6 7 Introduction 8 ------------ 9 10 Coresight is an umbrella of technologies allowing for the debugging of ARM 11 based SoC. It includes solutions for JTAG and HW assisted tracing. This 12 document is concerned with the latter. 13 14 HW assisted tracing is becoming increasingly useful when dealing with systems 15 that have many SoCs and other components like GPU and DMA engines. ARM has 16 developed a HW assisted tracing solution by means of different components, each 17 being added to a design at synthesis time to cater to specific tracing needs. 18 Components are generally categorised as source, link and sinks and are 19 (usually) discovered using the AMBA bus. 20 21 "Sources" generate a compressed stream representing the processor instruction 22 path based on tracing scenarios as configured by users. From there the stream 23 flows through the coresight system (via ATB bus) using links that are connecting 24 the emanating source to a sink(s). Sinks serve as endpoints to the coresight 25 implementation, either storing the compressed stream in a memory buffer or 26 creating an interface to the outside world where data can be transferred to a 27 host without fear of filling up the onboard coresight memory buffer. 28 29 At typical coresight system would look like this: 30 31 ***************************************************************** 32 **************************** AMBA AXI ****************************===|| 33 ***************************************************************** || 34 ^ ^ | || 35 | | * ** 36 0000000 ::::: 0000000 ::::: ::::: @@@@@@@ |||||||||||| 37 0 CPU 0<-->: C : 0 CPU 0<-->: C : : C : @ STM @ || System || 38 |->0000000 : T : |->0000000 : T : : T :<--->@@@@@ || Memory || 39 | #######<-->: I : | #######<-->: I : : I : @@@<-| |||||||||||| 40 | # ETM # ::::: | # PTM # ::::: ::::: @ | 41 | ##### ^ ^ | ##### ^ ! ^ ! . | ||||||||| 42 | |->### | ! | |->### | ! | ! . | || DAP || 43 | | # | ! | | # | ! | ! . | ||||||||| 44 | | . | ! | | . | ! | ! . | | | 45 | | . | ! | | . | ! | ! . | | * 46 | | . | ! | | . | ! | ! . | | SWD/ 47 | | . | ! | | . | ! | ! . | | JTAG 48 *****************************************************************<-| 49 *************************** AMBA Debug APB ************************ 50 ***************************************************************** 51 | . ! . ! ! . | 52 | . * . * * . | 53 ***************************************************************** 54 ******************** Cross Trigger Matrix (CTM) ******************* 55 ***************************************************************** 56 | . ^ . . | 57 | * ! * * | 58 ***************************************************************** 59 ****************** AMBA Advanced Trace Bus (ATB) ****************** 60 ***************************************************************** 61 | ! =============== | 62 | * ===== F =====<---------| 63 | ::::::::: ==== U ==== 64 |-->:: CTI ::<!! === N === 65 | ::::::::: ! == N == 66 | ^ * == E == 67 | ! &&&&&&&&& IIIIIII == L == 68 |------>&& ETB &&<......II I ======= 69 | ! &&&&&&&&& II I . 70 | ! I I . 71 | ! I REP I<.......... 72 | ! I I 73 | !!>&&&&&&&&& II I *Source: ARM ltd. 74 |------>& TPIU &<......II I DAP = Debug Access Port 75 &&&&&&&&& IIIIIII ETM = Embedded Trace Macrocell 76 ; PTM = Program Trace Macrocell 77 ; CTI = Cross Trigger Interface 78 * ETB = Embedded Trace Buffer 79 To trace port TPIU= Trace Port Interface Unit 80 SWD = Serial Wire Debug 81 82 While on target configuration of the components is done via the APB bus, 83 all trace data are carried out-of-band on the ATB bus. The CTM provides 84 a way to aggregate and distribute signals between CoreSight components. 85 86 The coresight framework provides a central point to represent, configure and 87 manage coresight devices on a platform. This first implementation centers on 88 the basic tracing functionality, enabling components such ETM/PTM, funnel, 89 replicator, TMC, TPIU and ETB. Future work will enable more 90 intricate IP blocks such as STM and CTI. 91 92 93 Acronyms and Classification 94 --------------------------- 95 96 Acronyms: 97 98 PTM: Program Trace Macrocell 99 ETM: Embedded Trace Macrocell 100 STM: System trace Macrocell 101 ETB: Embedded Trace Buffer 102 ITM: Instrumentation Trace Macrocell 103 TPIU: Trace Port Interface Unit 104 TMC-ETR: Trace Memory Controller, configured as Embedded Trace Router 105 TMC-ETF: Trace Memory Controller, configured as Embedded Trace FIFO 106 CTI: Cross Trigger Interface 107 108 Classification: 109 110 Source: 111 ETMv3.x ETMv4, PTMv1.0, PTMv1.1, STM, STM500, ITM 112 Link: 113 Funnel, replicator (intelligent or not), TMC-ETR 114 Sinks: 115 ETBv1.0, ETB1.1, TPIU, TMC-ETF 116 Misc: 117 CTI 118 119 120 Device Tree Bindings 121 ---------------------- 122 123 See Documentation/devicetree/bindings/arm/coresight.txt for details. 124 125 As of this writing drivers for ITM, STMs and CTIs are not provided but are 126 expected to be added as the solution matures. 127 128 129 Framework and implementation 130 ---------------------------- 131 132 The coresight framework provides a central point to represent, configure and 133 manage coresight devices on a platform. Any coresight compliant device can 134 register with the framework for as long as they use the right APIs: 135 136 struct coresight_device *coresight_register(struct coresight_desc *desc); 137 void coresight_unregister(struct coresight_device *csdev); 138 139 The registering function is taking a "struct coresight_device *csdev" and 140 register the device with the core framework. The unregister function takes 141 a reference to a "struct coresight_device", obtained at registration time. 142 143 If everything goes well during the registration process the new devices will 144 show up under /sys/bus/coresight/devices, as showns here for a TC2 platform: 145 146 root:~# ls /sys/bus/coresight/devices/ 147 replicator 20030000.tpiu 2201c000.ptm 2203c000.etm 2203e000.etm 148 20010000.etb 20040000.funnel 2201d000.ptm 2203d000.etm 149 root:~# 150 151 The functions take a "struct coresight_device", which looks like this: 152 153 struct coresight_desc { 154 enum coresight_dev_type type; 155 struct coresight_dev_subtype subtype; 156 const struct coresight_ops *ops; 157 struct coresight_platform_data *pdata; 158 struct device *dev; 159 const struct attribute_group **groups; 160 }; 161 162 163 The "coresight_dev_type" identifies what the device is, i.e, source link or 164 sink while the "coresight_dev_subtype" will characterise that type further. 165 166 The "struct coresight_ops" is mandatory and will tell the framework how to 167 perform base operations related to the components, each component having 168 a different set of requirement. For that "struct coresight_ops_sink", 169 "struct coresight_ops_link" and "struct coresight_ops_source" have been 170 provided. 171 172 The next field, "struct coresight_platform_data *pdata" is acquired by calling 173 "of_get_coresight_platform_data()", as part of the driver's _probe routine and 174 "struct device *dev" gets the device reference embedded in the "amba_device": 175 176 static int etm_probe(struct amba_device *adev, const struct amba_id *id) 177 { 178 ... 179 ... 180 drvdata->dev = &adev->dev; 181 ... 182 } 183 184 Specific class of device (source, link, or sink) have generic operations 185 that can be performed on them (see "struct coresight_ops"). The 186 "**groups" is a list of sysfs entries pertaining to operations 187 specific to that component only. "Implementation defined" customisations are 188 expected to be accessed and controlled using those entries. 189 190 Last but not least, "struct module *owner" is expected to be set to reflect 191 the information carried in "THIS_MODULE". 192 193 How to use the tracer modules 194 ----------------------------- 195 196 Before trace collection can start, a coresight sink needs to be identify. 197 There is no limit on the amount of sinks (nor sources) that can be enabled at 198 any given moment. As a generic operation, all device pertaining to the sink 199 class will have an "active" entry in sysfs: 200 201 root:/sys/bus/coresight/devices# ls 202 replicator 20030000.tpiu 2201c000.ptm 2203c000.etm 2203e000.etm 203 20010000.etb 20040000.funnel 2201d000.ptm 2203d000.etm 204 root:/sys/bus/coresight/devices# ls 20010000.etb 205 enable_sink status trigger_cntr 206 root:/sys/bus/coresight/devices# echo 1 > 20010000.etb/enable_sink 207 root:/sys/bus/coresight/devices# cat 20010000.etb/enable_sink 208 1 209 root:/sys/bus/coresight/devices# 210 211 At boot time the current etm3x driver will configure the first address 212 comparator with "_stext" and "_etext", essentially tracing any instruction 213 that falls within that range. As such "enabling" a source will immediately 214 trigger a trace capture: 215 216 root:/sys/bus/coresight/devices# echo 1 > 2201c000.ptm/enable_source 217 root:/sys/bus/coresight/devices# cat 2201c000.ptm/enable_source 218 1 219 root:/sys/bus/coresight/devices# cat 20010000.etb/status 220 Depth: 0x2000 221 Status: 0x1 222 RAM read ptr: 0x0 223 RAM wrt ptr: 0x19d3 <----- The write pointer is moving 224 Trigger cnt: 0x0 225 Control: 0x1 226 Flush status: 0x0 227 Flush ctrl: 0x2001 228 root:/sys/bus/coresight/devices# 229 230 Trace collection is stopped the same way: 231 232 root:/sys/bus/coresight/devices# echo 0 > 2201c000.ptm/enable_source 233 root:/sys/bus/coresight/devices# 234 235 The content of the ETB buffer can be harvested directly from /dev: 236 237 root:/sys/bus/coresight/devices# dd if=/dev/20010000.etb \ 238 of=~/cstrace.bin 239 240 64+0 records in 241 64+0 records out 242 32768 bytes (33 kB) copied, 0.00125258 s, 26.2 MB/s 243 root:/sys/bus/coresight/devices# 244 245 The file cstrace.bin can be decompressed using "ptm2human", DS-5 or Trace32. 246 247 Following is a DS-5 output of an experimental loop that increments a variable up 248 to a certain value. The example is simple and yet provides a glimpse of the 249 wealth of possibilities that coresight provides. 250 251 Info Tracing enabled 252 Instruction 106378866 0x8026B53C E52DE004 false PUSH {lr} 253 Instruction 0 0x8026B540 E24DD00C false SUB sp,sp,#0xc 254 Instruction 0 0x8026B544 E3A03000 false MOV r3,#0 255 Instruction 0 0x8026B548 E58D3004 false STR r3,[sp,#4] 256 Instruction 0 0x8026B54C E59D3004 false LDR r3,[sp,#4] 257 Instruction 0 0x8026B550 E3530004 false CMP r3,#4 258 Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1 259 Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4] 260 Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c 261 Timestamp Timestamp: 17106715833 262 Instruction 319 0x8026B54C E59D3004 false LDR r3,[sp,#4] 263 Instruction 0 0x8026B550 E3530004 false CMP r3,#4 264 Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1 265 Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4] 266 Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c 267 Instruction 9 0x8026B54C E59D3004 false LDR r3,[sp,#4] 268 Instruction 0 0x8026B550 E3530004 false CMP r3,#4 269 Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1 270 Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4] 271 Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c 272 Instruction 7 0x8026B54C E59D3004 false LDR r3,[sp,#4] 273 Instruction 0 0x8026B550 E3530004 false CMP r3,#4 274 Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1 275 Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4] 276 Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c 277 Instruction 7 0x8026B54C E59D3004 false LDR r3,[sp,#4] 278 Instruction 0 0x8026B550 E3530004 false CMP r3,#4 279 Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1 280 Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4] 281 Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c 282 Instruction 10 0x8026B54C E59D3004 false LDR r3,[sp,#4] 283 Instruction 0 0x8026B550 E3530004 false CMP r3,#4 284 Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1 285 Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4] 286 Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c 287 Instruction 6 0x8026B560 EE1D3F30 false MRC p15,#0x0,r3,c13,c0,#1 288 Instruction 0 0x8026B564 E1A0100D false MOV r1,sp 289 Instruction 0 0x8026B568 E3C12D7F false BIC r2,r1,#0x1fc0 290 Instruction 0 0x8026B56C E3C2203F false BIC r2,r2,#0x3f 291 Instruction 0 0x8026B570 E59D1004 false LDR r1,[sp,#4] 292 Instruction 0 0x8026B574 E59F0010 false LDR r0,[pc,#16] ; [0x8026B58C] = 0x80550368 293 Instruction 0 0x8026B578 E592200C false LDR r2,[r2,#0xc] 294 Instruction 0 0x8026B57C E59221D0 false LDR r2,[r2,#0x1d0] 295 Instruction 0 0x8026B580 EB07A4CF true BL {pc}+0x1e9344 ; 0x804548c4 296 Info Tracing enabled 297 Instruction 13570831 0x8026B584 E28DD00C false ADD sp,sp,#0xc 298 Instruction 0 0x8026B588 E8BD8000 true LDM sp!,{pc} 299 Timestamp Timestamp: 17107041535 300 301 How to use the STM module 302 ------------------------- 303 304 Using the System Trace Macrocell module is the same as the tracers - the only 305 difference is that clients are driving the trace capture rather 306 than the program flow through the code. 307 308 As with any other CoreSight component, specifics about the STM tracer can be 309 found in sysfs with more information on each entry being found in [1]: 310 311 root@genericarmv8:~# ls /sys/bus/coresight/devices/20100000.stm 312 enable_source hwevent_select port_enable subsystem uevent 313 hwevent_enable mgmt port_select traceid 314 root@genericarmv8:~# 315 316 Like any other source a sink needs to be identified and the STM enabled before 317 being used: 318 319 root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20010000.etf/enable_sink 320 root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20100000.stm/enable_source 321 322 From there user space applications can request and use channels using the devfs 323 interface provided for that purpose by the generic STM API: 324 325 root@genericarmv8:~# ls -l /dev/20100000.stm 326 crw------- 1 root root 10, 61 Jan 3 18:11 /dev/20100000.stm 327 root@genericarmv8:~# 328 329 Details on how to use the generic STM API can be found here [2]. 330 331 [1]. Documentation/ABI/testing/sysfs-bus-coresight-devices-stm 332 [2]. Documentation/trace/stm.txt