[LITMUS^RT] local APIC tick lost

[Andrea Bastoni]

On 01/19/2013 05:18 AM, Hang Su wrote:
> Thanks for your reply. Kernel runs on real machine. 
> 
> Let me introduce my platform first.
> There are two Intel xeon E5405 in my machine. Each one contains four cores. So,
> I have eight cores in total.
> I used LITMUS way to align tick across cores. Then, I add a dummy scheduler
> module on the top of RT scheduler module like LITMUS. Actually, it is really
> similar to RT, only allocating X ticks for a process/thread. In my small test
> case, I utilize the first six core and create six threads, therefore each thread
> scheduled by dummy scheduler occupies one cpu for x ticks interval (Assume x is
> large, 10e7)
> 
> The purpose of my test case is to test arrival difference of local APIC
> interrupts across cores under different HZ settings, such as (100, 200, 400,
> 800, 1000). My arrival difference threshold (ADThres) is set to 300us. And the
> arrival difference is measure in this way under 1000 HZ:
> /kernel/sched.c
> 
> void scheduler_tick <http://lxr.linux.no/linux+v2.6.34/+code=scheduler_tick>(void){
> 
>    ...
> 
>    rq->curr_tick_stamp = sched_clock <http://lxr.linux.no/linux+v2.6.34/+code=sched_clock>();
> 
>    if( rq->curr_tick_stamp > rq->prev_tick_stamp + 1000000(depends on HZ setting) + ADThres ){
> 
>        error report
> 
>    }
> 
>    rq->prev_tick_stamp = rq->curr_tick_stamp;
> 
> 
>    ...
> 
> }
> 
> 
> Under 1000HZ setting, it is easy to report error. At error tick, the arrival
> difference is rough 500 us, which means local APIC tick interrupts arrive in the
> middle of next supposed tick. I am wondering this phenomenon have a big
> influence on tick based scheduler. However, it rarely reports error under 100 or
> 200 HZ setting.

Hi Hang,

Your measuring points do not seems too LITMUS-specific. Is the issue happening
also on a plain vanilla kernel? Which kernel version are you using? And is
CONFIG_NO_HZ = n?

Thanks,
- Andrea

> 
> Then, I focused on accuracy of local APIC. According to [1], it gives an example
> in period mode of LAPIC
> 
> For example, for a 2.4 GHz CPU with an external/bus frequency of 800 MHz, if the
> Divide Configuration Register is set to "divide by 4" and the initial count is
> set to 123456; then the local APIC timer would decrement the count at a rate of
> 200 MHz and generate a timer IRQ every 617.28 us, giving a rate of IRQs of
> 1620.01 Hz.
> 
> My machine's FSB is 1333 MHZ and it has the capability of providing nanoseconds
> accuracy.
> 
> [1]. APIC timer, http://wiki.osdev.org/APIC_timer
> 
> 
> 
> 
> On Thu, Jan 17, 2013 at 2:14 AM, Björn Brandenburg <bbb at mpi-sws.org
> <mailto:bbb at mpi-sws.org>> wrote:
> 
> 
>     On Jan 17, 2013, at 3:42 AM, Hang Su <hangsu.cs at gmail.com
>     <mailto:hangsu.cs at gmail.com>> wrote:
> 
>     > Hello everyone:
>     >
>     > Is there anybody has experienced local APIC timer tick lost under HZ =
>     1000 and x86?
> 
>     On real hardware, no. I suspect this might be possible in QEMU/kvm with
>     lot's of debugging options enabled. Under which scheduler plugin is this
>     happening, and how many cores do you have?
> 
>     - Björn