[LITMUS^RT] [LITMUS-RT] Modified pfp_schedule for migration

[Sebastiano Catellani]

Hi everyone,

I'm modifying the pfp_schedule function of the sched_pfp.c file. The
behavior that I want to obtain is to migrate a task from a cpu to another
everytime that it is preempted by a task with higher priority.

With the following task set, I'd like that p2 migrates to the second cpu
when it is preempted by p1 and migrates back to the first cpu when
preempted by p3.

-p 1 -z 1 -q 3 5 20 //(p1)
-p 1 -z 1 -q 4 5 18 //(p2)
-p 2 -z 1 -q 3 3 20 //(p3)

The following code show a piece of my modified function:

/*   */
                  if ((!np || blocks) && (resched || !exists)) {

 if(prev && preempt && ((smp_processor_id() == 1) || (smp_processor_id() ==
2))) {
TRACE_TASK(prev, "[PFP:%llu]: [cpu %d] preempted! I will migrate!\n",
litmus_clock(), smp_processor_id());
 } else {
if (pfp->scheduled && !blocks  && !migrate)
requeue(pfp->scheduled, pfp);
 }

next = fp_prio_take(&pfp->ready_queue);
 if (next == prev) {
 struct task_struct *t = fp_prio_peek(&pfp->ready_queue);

TRACE_TASK(next, "next==prev sleep=%d oot=%d np=%d preempt=%d migrate=%d "
 "boost=%d empty=%d prio-idx=%u prio=%u\n",
sleep, out_of_time, np, preempt, migrate,
is_priority_boosted(next),
 t == NULL,
priority_index(next),
get_priority(next));

if (t)
TRACE_TASK(t, "waiter boost=%d prio-idx=%u prio=%u\n",
is_priority_boosted(t),
 priority_index(t),
get_priority(t));
} else {
 if(prev && blocks) {
if(smp_processor_id() == 1) {
pfp_migrate_to(2, prev);
 }
if(smp_processor_id() == 2) {
pfp_migrate_to(1, prev);
 }
}
}

 /* If preempt is set, we should not see the same task again. */
BUG_ON(preempt && next == prev);
 /* Similarly, if preempt is set, then next may not be NULL,
* unless it's a migration. */
BUG_ON(preempt && !migrate && next == NULL);
 }

/*   */

When the task is in the situation indicated above, it won't requeue in the
pfp_domain or pfp_ready_queue and in the else branch will migrate in the
correct cpu.

This is the error I get when I execute the run_exps.py script:

=============================================
[ INFO: possible recursive locking detected ]
3.10.5-litmus2013.1 #101 Not tainted
---------------------------------------------
rtspin/1586 is trying to acquire lock:
 (&rt->ready_lock){......}, at: [<ffffffff81299940>]
pfp_migrate_to+0x80/0x120

but task is already holding lock:
 (&rt->ready_lock){......}, at: [<ffffffff8129ba35>] pfp_schedule+0x35/0xab0

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&rt->ready_lock);
  lock(&rt->ready_lock);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

2 locks held by rtspin/1586:
 #0:  (&rq->lock){-.-.-.}, at: [<ffffffff8155f105>] __schedule+0x175/0xa70
 #1:  (&rt->ready_lock){......}, at: [<ffffffff8129ba35>]
pfp_schedule+0x35/0xab0

stack backtrace:
CPU: 1 PID: 1586 Comm: rtspin Not tainted 3.10.5-litmus2013.1 #101
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
 ffffffff81c0f0c0 ffff88001dba9a08 ffffffff8155ae4e ffff88001dba9af8
 ffffffff81078217 0000000000000000 ffffffff82172630 000000000003f5f0
 ffff88001dba9b20 ffffffff81c0f0c0 ffff88001ea52140 ffff88001dba9b38
Call Trace:
 [<ffffffff8155ae4e>] dump_stack+0x19/0x1b
 [<ffffffff81078217>] __lock_acquire+0x757/0x1e90
 [<ffffffff81077eca>] ? __lock_acquire+0x40a/0x1e90
 [<ffffffff812abf6f>] ? number.isra.1+0x2ff/0x330
 [<ffffffff810b9c0a>] ? get_page_from_freelist+0x4fa/0x690
 [<ffffffff81079f65>] lock_acquire+0x95/0x140
 [<ffffffff81299940>] ? pfp_migrate_to+0x80/0x120
 [<ffffffff81560ff6>] _raw_spin_lock+0x36/0x50
 [<ffffffff81299940>] ? pfp_migrate_to+0x80/0x120
 [<ffffffff81299940>] pfp_migrate_to+0x80/0x120
 [<ffffffff8129c255>] pfp_schedule+0x855/0xab0
 [<ffffffff810763bd>] ? trace_hardirqs_off+0xd/0x10
 [<ffffffff812a199d>] ? sched_trace_log_message+0xdd/0x110
 [<ffffffff81069e70>] pick_next_task_litmus+0x40/0x500
 [<ffffffff8155f1aa>] __schedule+0x21a/0xa70
 [<ffffffff8155fa24>] schedule+0x24/0x70
 [<ffffffff8155d1ec>] schedule_timeout+0x14c/0x200
 [<ffffffff81561207>] ? _raw_spin_unlock_irq+0x27/0x60
 [<ffffffff8105e3ed>] ? get_parent_ip+0xd/0x50
 [<ffffffff8105e589>] ? sub_preempt_count+0x69/0xf0
 [<ffffffff8155ffdb>] wait_for_completion_interruptible+0xcb/0x140
 [<ffffffff81060e60>] ? try_to_wake_up+0x470/0x470
 [<ffffffff8129266f>] do_wait_for_ts_release+0xef/0x190
 [<ffffffff81292782>] sys_wait_for_ts_release+0x22/0x30
 [<ffffffff81562592>] system_call_fastpath+0x16/0x1b

Thanks in advance for any tips.

Sebastiano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.litmus-rt.org/pipermail/litmus-dev/attachments/20140110/b9d86042/attachment.html>