Opened on 12/19/18 at 07:01:36
Closed on 12/19/19 at 09:26:09
#3665 closed enhancement (fixed)
Add low level event recording infrastructure
Reported by: | Sebastian Huber | Owned by: | Sebastian Huber |
---|---|---|---|
Priority: | normal | Milestone: | 5.1 |
Component: | lib | Version: | 5 |
Severity: | normal | Keywords: | |
Cc: | Blocked By: | ||
Blocking: |
Description
Add low level event recording infrastructure for system and user defined events. The infrastructure should be able to record high frequency events such as
- SMP lock acquire/release,
- interrupt entry/exit,
- thread switches,
- UMA zone allocate/free,
- Ethernet packet input/output, etc.
It should allow post-mortem analysis in fatal error handlers, e.g. the last events should be in the record buffer, the newest event overwrites the oldest event. It should be possible to detect record buffer overflows for consumers that expect a continuous stream of events, e.g. to display the system state in real-time.
The framework should support high-end SMP machines (more than 1GHz processor frequency, more than four processors).
The existing capture engine tries to solve this problem, but its performance is not good enough for high-end production systems. The main issues are the variable-size buffers and the use of SMP locks for synchronization. To fix this, the API would change significantly.
Add a new API instead. The implementation should use per-processor data structures and no atomic read-modify-write operations. It is pretty much a per-processor ring buffer for record events.
Use the CPU counter to get the time of events. Combine it with periodic uptime events to synchronize it with CLOCK_REALTIME.
Here is an example of the
/** * @brief Produces a record item. * * @param event The record event without a time stamp for the item. * @param data The record data for the item. */ void rtems_record_produce( rtems_record_event event, rtems_record_data data );
function PowerPC machine code generated by GCC:
00000000 <rtems_record_produce>: 0: 7d 00 00 a6 mfmsr r8 4: 7c 00 01 46 wrteei 0 8: 7d 2e 82 a6 mfspr r9,526 c: 7d 50 42 a6 mfsprg r10,0 10: 81 4a 02 b4 lwz r10,692(r10) 14: 55 29 50 2a rlwinm r9,r9,10,0,21 18: 7d 23 1b 78 or r3,r9,r3 1c: 81 2a 00 00 lwz r9,0(r10) 20: 80 ca 00 08 lwz r6,8(r10) 24: 38 e9 00 01 addi r7,r9,1 28: 7d 29 30 38 and r9,r9,r6 2c: 55 29 18 38 rlwinm r9,r9,3,0,28 30: 7d 2a 4a 14 add r9,r10,r9 34: 90 69 00 48 stw r3,72(r9) 38: 90 89 00 4c stw r4,76(r9) 3c: 7c 20 04 ac lwsync 40: 90 ea 00 00 stw r7,0(r10) 44: 7d 00 01 06 wrtee r8 48: 4e 80 00 20 blr
Just 19 instructions, no branches, no stack frame, no atomic-read-modify-write, just a light weight synchronization to ensure that the consumer reads not half finished items.
Change History (46)
comment:1 Changed on 01/29/19 at 12:57:00 by Sebastian Huber <sebastian.huber@…>
comment:3 Changed on 02/01/19 at 08:52:23 by Sebastian Huber <sebastian.huber@…>
comment:4 Changed on 02/01/19 at 08:52:25 by Sebastian Huber <sebastian.huber@…>
comment:8 Changed on 08/20/19 at 20:51:11 by Sebastian Huber <sebastian.huber@…>
comment:10 Changed on 08/27/19 at 06:47:37 by Sebastian Huber <sebastian.huber@…>
In e273e7a9/rtems:
comment:11 Changed on 08/27/19 at 06:52:40 by Sebastian Huber <sebastian.huber@…>
comment:12 Changed on 08/27/19 at 06:52:42 by Sebastian Huber <sebastian.huber@…>
comment:13 Changed on 08/27/19 at 06:52:44 by Sebastian Huber <sebastian.huber@…>
comment:14 Changed on 08/27/19 at 06:52:46 by Sebastian Huber <sebastian.huber@…>
comment:15 Changed on 08/27/19 at 06:52:48 by Sebastian Huber <sebastian.huber@…>
comment:16 Changed on 08/27/19 at 06:52:50 by Sebastian Huber <sebastian.huber@…>
comment:17 Changed on 08/28/19 at 14:02:53 by Sebastian Huber <sebastian.huber@…>
In a2684c2b/rtems:
comment:18 Changed on 08/28/19 at 14:02:56 by Sebastian Huber <sebastian.huber@…>
In 3eb8d783/rtems:
comment:21 Changed on 08/29/19 at 14:05:45 by Sebastian Huber <sebastian.huber@…>
comment:23 Changed on 08/30/19 at 07:01:22 by Sebastian Huber <sebastian.huber@…>
In 11f196d6/rtems:
comment:25 Changed on 08/30/19 at 09:49:35 by Sebastian Huber <sebastian.huber@…>
In 8ace7ead/rtems:
comment:26 Changed on 08/30/19 at 09:53:09 by Sebastian Huber <sebastian.huber@…>
comment:28 Changed on 08/30/19 at 17:50:29 by Sebastian Huber <sebastian.huber@…>
comment:29 Changed on 09/02/19 at 05:56:58 by Sebastian Huber <sebastian.huber@…>
In e41e9961/rtems:
comment:30 Changed on 09/02/19 at 06:14:50 by Sebastian Huber <sebastian.huber@…>
comment:31 Changed on 09/03/19 at 13:03:20 by Sebastian Huber <sebastian.huber@…>
comment:32 Changed on 09/03/19 at 13:03:22 by Sebastian Huber <sebastian.huber@…>
comment:33 Changed on 09/04/19 at 11:48:07 by Sebastian Huber <sebastian.huber@…>
comment:34 Changed on 09/04/19 at 11:48:10 by Sebastian Huber <sebastian.huber@…>
comment:35 Changed on 09/04/19 at 11:48:12 by Sebastian Huber <sebastian.huber@…>
comment:36 Changed on 09/04/19 at 11:48:15 by Sebastian Huber <sebastian.huber@…>
comment:37 Changed on 09/04/19 at 11:48:17 by Sebastian Huber <sebastian.huber@…>
comment:38 Changed on 09/04/19 at 12:03:39 by Sebastian Huber <sebastian.huber@…>
comment:39 Changed on 09/04/19 at 12:43:10 by Sebastian Huber <sebastian.huber@…>
comment:40 Changed on 09/05/19 at 08:50:08 by Sebastian Huber <sebastian.huber@…>
comment:41 Changed on 09/05/19 at 08:50:26 by Sebastian Huber <sebastian.huber@…>
comment:42 Changed on 09/05/19 at 08:50:28 by Sebastian Huber <sebastian.huber@…>
comment:45 Changed on 10/02/19 at 04:38:31 by Sebastian Huber <sebastian.huber@…>
In a314544a/rtems:
comment:46 Changed on 12/19/19 at 09:26:09 by Sebastian Huber
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
All the basic functionality is integrated. New features/bugs should use new tickets.
In dca6184/rtems: