[LITMUS^RT] IPI vs. Cache Affinity

[Glenn Elliott]

On Sep 27, 2013, at 7:17 PM, Andrea Bastoni <bastoni at sprg.uniroma2.it> wrote:

> Hi Glenn,
> 
> On 09/27/2013 10:46 PM, Glenn Elliott wrote:
>> I've been getting more acquainted with the latest litmus code and I see that
>> GSN-EDF/C-EDF prefer to schedule a task on the local CPU if the CPU is
>> available.  This avoids an IPI.  However, this may result in higher cache
>> migration costs because this scheduling decision is made before checking CPU
>> affinity.
> 
> Umm, what is the use case you've in mind? Evaluating a local condition is
> relatively cheap (as you've already all the locks you need). Preempting a remote
> CPU is more expensive (locks to be reacquired, interrupt to be sent and
> received, etc.).
> 
>> Are IPI costs (significantly) higher than cache affinity loss?
> 
> What's your working set size, what's the load of the system, what's your
> architecture?
> 
> Also, an IPI will cause an interrupt on the destination CPU, thus disturbing the
> execution of the task already running there. This perturbation may be completely
> pointless in the case the task is eventually not scheduled on the CPU being
> preempted by the IPI.
> 
>> Are we making
>> the assumption that affinity has already been lost due to cache polluters?
> 
> Are you assuming worst-case conditions?
> 
> Thanks,
> - Andrea
> 
>> Do we have empirical data to support the current implementation?  I'm not
>> arguing in favor of one method or the other---I am just interested in the
>> motivations behind the changes.
>> 
>> Thanks,
>> Glenn
>> _______________________________________________
>> litmus-dev mailing list
>> litmus-dev at lists.litmus-rt.org 
>> https://lists.litmus-rt.org/listinfo/litmus-dev
> 
> -- 
> Andrea Bastoni, PhD <bastoni at sprg.uniroma2.it>
> Dept. of Computer Science, Systems, and Industrial Engineering
> University of Rome "Tor Vergata", Via del Politecnico, 1 - 00133 Rome
> 
> _______________________________________________
> litmus-dev mailing list
> litmus-dev at lists.litmus-rt.org
> https://lists.litmus-rt.org/listinfo/litmus-dev


Hi Andrea,

Thank you for the deeper explanation.  I certainly appreciate the simplicity of local scheduling.  I will be investigating response times of cache heavy tasks under G-EDF/C-EDF over the next few weeks.  I'll report back if I see any noticeable trade-offs between local-first vs. affinity-first scheduling.

-Glenn