<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 02 Oct 2015, at 04:32, Meng Xu <<a href="mailto:mengxu@seas.upenn.edu" class="">mengxu@seas.upenn.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="gmail_default" style="font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: small;">I'm Meng Xu, a PhD student at the PRECISE lab at the University of Pennsylvania.</div><div class="gmail_default" style="font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: small;"><br class=""></div><div class="gmail_default" style="font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: small;">I'm trying to measure the context switch overhead with the feather trace tool on Freescale IMX6 ARM board. The board has 1GB RAM and 4 cores. </div><div class="gmail_default" style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">I followed the description at<span class="Apple-converted-space"> </span><a href="https://wiki.litmus-rt.org/litmus/Tracing" class="">https://wiki.litmus-rt.org/litmus/Tracing</a>.<br class="">I measured the context switch overhead with the command:<br class=""><br class="">ST_TRACE_PATH=/dev/shm<br class="">ST_TRACE_NAME=GSN-EDF<br class="">OH_EVENTS="CXS_START CXS_END"<br class="">ftcat /dev/litmus/ft_cpu_trace0 ${OH_EVENTS} > ${ST_TRACE_PATH}/oh-${ST_TRACE_NAME}-0.bin &<br class="">ftcat /dev/litmus/ft_cpu_trace1 ${OH_EVENTS} > ${ST_TRACE_PATH}/oh-${ST_TRACE_NAME}-1.bin &<br class="">ftcat /dev/litmus/ft_cpu_trace2 ${OH_EVENTS} > ${ST_TRACE_PATH}/oh-${ST_TRACE_NAME}-2.bin &<br class="">ftcat /dev/litmus/ft_cpu_trace3 ${OH_EVENTS} > ${ST_TRACE_PATH}/oh-${ST_TRACE_NAME}-3.bin &<br class=""><br class="">I run a rtspin RT task under GSN-EDF scheduler for 20 seconds and then combine the bin files as one "oh-GSN-EDF-all.bin ".</div><div class="gmail_default" style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">When I use ft2csv command to parse the bin file, it shows no complete events. </div><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""># ft2csv CXS oh-GSN-EDF-all.bin</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Total : 11936</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Skipped : 1</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Avoided : 0</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Complete : 0</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Incomplete : 4941</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Non RT : 0</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Interleaved : 0</span><br style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Interrupted : 1033</span><div style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_default" style="font-size: small;">I tried to increase the buffer size of the overhead trace to a larger value, but it cannot be larger than 1024MB since RAM size is only 1024MB. </div><div class="gmail_default" style="font-size: small;">/dev/shm has size 512MB.</div><div class="gmail_default" style="font-size: small;"><br class=""></div><div class="gmail_default" style="font-size: small;">I also observe that there may be some memory leaking in the feather trace?</div><div class="gmail_default" style="font-size: small;">When I boot the system, there is about 800MB free memory.</div><div class="gmail_default" style="font-size: small;">After I run the feather trace, the free memory becomes 538MB. (ftcat has been killed already at this time.)</div></div></div></blockquote><div><br class=""></div><div>Well, Feather-Trace does allocate large buffers. I didn’t think they were leaking, but if you find evidence to the contrary I’d appreciate a patch. Keep in mind that free memory is expected to decrease to near-zero during normal operation of Linux due to the buffer cache.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_default" style="font-size: small;">The commit point I used is 7f051e3fa168eb60386b0e8d970551c06696befb, which is committed by Bjorn on Jun 12th, 2014.</div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_default" style="font-size: small;"><b class="">My question is:</b></div><div class="gmail_default" style="font-size: small;">Did I configure anything wrong that cause no complete event is recorded?</div></div></div></blockquote><div><br class=""></div><div>No, I don’t see anything obviously wrong.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_default" style="font-size: small;">Do you have any suggestion/advice on how I can get the complete events?</div><div class="gmail_default" style="font-size: small;"><br class=""></div><div class="gmail_default" style="font-size: small;">I have tried to set the ftcat to highest priority under FIFO scheduler, it didn't solve the problem. </div><div class="gmail_default" style="font-size: small;">I was suspecting it is because the buffer is not consumed by ftcat fast enough so that the old events are overwritten by new events. However, I only run on rtspin task which leave at least 3 full cores idle to consume the buffer. </div></div></div></blockquote><br class=""></div><div>If ftcat is starved, then you get some incomplete records, but typically not 100% incomplete records. This must be something else.</div><div><br class=""></div><div>Have you looked at the trace with ftdump? What does it look like? </div><div><br class=""></div><div>- Björn</div><br class=""><div class=""><br class=""></div></body></html>