<div dir="ltr">Hi everyone,<div><br></div><div><div>I'm modifying the pfp_schedule function of the sched_pfp.c file. The behavior that I want to obtain is to migrate a task from a cpu to another everytime that it is preempted by a task with higher priority.</div>
<div><br></div><div>With the following task set, I'd like that p2 migrates to the second cpu when it is preempted by p1 and migrates back to the first cpu when preempted by p3.</div></div><div><br></div><div><div>-p 1 -z 1 -q 3 5 20 //(p1)</div>
<div>-p 1 -z 1 -q 4 5 18 //(p2)</div><div>-p 2 -z 1 -q 3 3 20 //(p3)</div></div><div><br></div><div><div>The following code show a piece of my modified function:</div></div><div><br></div><div>/* */</div><div><div> if ((!np || blocks) && (resched || !exists)) {</div>
<div><span style="white-space:pre-wrap"> </span><br></div>
<div><span style="white-space:pre-wrap"> </span>if(prev && preempt && ((smp_processor_id() == 1) || (smp_processor_id() == 2))) {</div><div><span style="white-space:pre-wrap"> </span>TRACE_TASK(prev, "[PFP:%llu]: [cpu %d] preempted! I will migrate!\n", litmus_clock(), smp_processor_id());</div>
<div><span style="white-space:pre-wrap"> </span>} else {</div><div><span style="white-space:pre-wrap"> </span>if (pfp->scheduled && !blocks && !migrate)</div><div><span style="white-space:pre-wrap"> </span>requeue(pfp->scheduled, pfp);<span style="white-space:pre-wrap"> </span></div>
<div><span style="white-space:pre-wrap"> </span>}</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>next = fp_prio_take(&pfp->ready_queue);</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>if (next == prev) {</div>
<div><span style="white-space:pre-wrap"> </span>struct task_struct *t = fp_prio_peek(&pfp->ready_queue);</div><div><br></div><div><span style="white-space:pre-wrap"> </span>TRACE_TASK(next, "next==prev sleep=%d oot=%d np=%d preempt=%d migrate=%d "</div>
<div><span style="white-space:pre-wrap"> </span>"boost=%d empty=%d prio-idx=%u prio=%u\n",</div><div><span style="white-space:pre-wrap"> </span>sleep, out_of_time, np, preempt, migrate,</div><div><span style="white-space:pre-wrap"> </span>is_priority_boosted(next),</div>
<div><span style="white-space:pre-wrap"> </span>t == NULL,</div><div><span style="white-space:pre-wrap"> </span>priority_index(next),</div><div><span style="white-space:pre-wrap"> </span>get_priority(next));</div>
<div><br></div><div><span style="white-space:pre-wrap"> </span>if (t)</div><div><span style="white-space:pre-wrap"> </span>TRACE_TASK(t, "waiter boost=%d prio-idx=%u prio=%u\n",</div><div><span style="white-space:pre-wrap"> </span>is_priority_boosted(t),</div>
<div><span style="white-space:pre-wrap"> </span>priority_index(t),</div><div><span style="white-space:pre-wrap"> </span>get_priority(t));</div><div><span style="white-space:pre-wrap"> </span>} else {</div>
<div><span style="white-space:pre-wrap"> </span>if(prev && blocks) {</div><div><span style="white-space:pre-wrap"> </span>if(smp_processor_id() == 1) {</div><div><span style="white-space:pre-wrap"> </span>pfp_migrate_to(2, prev);</div>
<div><span style="white-space:pre-wrap"> </span>}</div><div><span style="white-space:pre-wrap"> </span>if(smp_processor_id() == 2) {</div><div><span style="white-space:pre-wrap"> </span>pfp_migrate_to(1, prev);</div>
<div><span style="white-space:pre-wrap"> </span>}</div><div><span style="white-space:pre-wrap"> </span>}</div><div><span style="white-space:pre-wrap"> </span>}</div><div><br></div><div>
<span style="white-space:pre-wrap"> </span>/* If preempt is set, we should not see the same task again. */</div><div><span style="white-space:pre-wrap"> </span>BUG_ON(preempt && next == prev);</div><div>
<span style="white-space:pre-wrap"> </span>/* Similarly, if preempt is set, then next may not be NULL,</div><div><span style="white-space:pre-wrap"> </span>* unless it's a migration. */</div><div><span style="white-space:pre-wrap"> </span>BUG_ON(preempt && !migrate && next == NULL);</div>
<div><span style="white-space:pre-wrap"> </span>}<br></div></div><div><br></div><div>/* */<br></div><div><br></div><div><div>When the task is in the situation indicated above, it won't requeue in the pfp_domain or pfp_ready_queue and in the else branch will migrate in the correct cpu.</div>
<div><br></div><div>This is the error I get when I execute the run_exps.py script:</div></div><div><br></div><div><div>=============================================</div>
<div>[ INFO: possible recursive locking detected ]</div><div>3.10.5-litmus2013.1 #101 Not tainted</div><div>---------------------------------------------</div><div>rtspin/1586 is trying to acquire lock:</div><div> (&rt->ready_lock){......}, at: [<ffffffff81299940>] pfp_migrate_to+0x80/0x120</div>
<div><br></div><div>but task is already holding lock:</div><div> (&rt->ready_lock){......}, at: [<ffffffff8129ba35>] pfp_schedule+0x35/0xab0</div><div><br></div><div>other info that might help us debug this:</div>
<div> Possible unsafe locking scenario:</div><div><br></div><div> CPU0</div><div> ----</div><div> lock(&rt->ready_lock);</div><div> lock(&rt->ready_lock);</div><div><br></div><div> *** DEADLOCK ***</div>
<div><br></div><div> May be due to missing lock nesting notation</div><div><br></div><div>2 locks held by rtspin/1586:</div><div> #0: (&rq->lock){-.-.-.}, at: [<ffffffff8155f105>] __schedule+0x175/0xa70</div>
<div> #1: (&rt->ready_lock){......}, at: [<ffffffff8129ba35>] pfp_schedule+0x35/0xab0</div><div><br></div><div>stack backtrace:</div><div>CPU: 1 PID: 1586 Comm: rtspin Not tainted 3.10.5-litmus2013.1 #101</div>
<div>Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011</div><div> ffffffff81c0f0c0 ffff88001dba9a08 ffffffff8155ae4e ffff88001dba9af8</div><div> ffffffff81078217 0000000000000000 ffffffff82172630 000000000003f5f0</div><div>
ffff88001dba9b20 ffffffff81c0f0c0 ffff88001ea52140 ffff88001dba9b38</div><div>Call Trace:</div><div> [<ffffffff8155ae4e>] dump_stack+0x19/0x1b</div><div> [<ffffffff81078217>] __lock_acquire+0x757/0x1e90</div>
<div> [<ffffffff81077eca>] ? __lock_acquire+0x40a/0x1e90</div><div> [<ffffffff812abf6f>] ? number.isra.1+0x2ff/0x330</div><div> [<ffffffff810b9c0a>] ? get_page_from_freelist+0x4fa/0x690</div><div> [<ffffffff81079f65>] lock_acquire+0x95/0x140</div>
<div> [<ffffffff81299940>] ? pfp_migrate_to+0x80/0x120</div><div> [<ffffffff81560ff6>] _raw_spin_lock+0x36/0x50</div><div> [<ffffffff81299940>] ? pfp_migrate_to+0x80/0x120</div><div> [<ffffffff81299940>] pfp_migrate_to+0x80/0x120</div>
<div> [<ffffffff8129c255>] pfp_schedule+0x855/0xab0</div><div> [<ffffffff810763bd>] ? trace_hardirqs_off+0xd/0x10</div><div> [<ffffffff812a199d>] ? sched_trace_log_message+0xdd/0x110</div><div> [<ffffffff81069e70>] pick_next_task_litmus+0x40/0x500</div>
<div> [<ffffffff8155f1aa>] __schedule+0x21a/0xa70</div><div> [<ffffffff8155fa24>] schedule+0x24/0x70</div><div> [<ffffffff8155d1ec>] schedule_timeout+0x14c/0x200</div><div> [<ffffffff81561207>] ? _raw_spin_unlock_irq+0x27/0x60</div>
<div> [<ffffffff8105e3ed>] ? get_parent_ip+0xd/0x50</div><div> [<ffffffff8105e589>] ? sub_preempt_count+0x69/0xf0</div><div> [<ffffffff8155ffdb>] wait_for_completion_interruptible+0xcb/0x140</div><div> [<ffffffff81060e60>] ? try_to_wake_up+0x470/0x470</div>
<div> [<ffffffff8129266f>] do_wait_for_ts_release+0xef/0x190</div><div> [<ffffffff81292782>] sys_wait_for_ts_release+0x22/0x30</div><div> [<ffffffff81562592>] system_call_fastpath+0x16/0x1b</div></div><div>
<br></div><div><div>Thanks in advance for any tips.</div><div><br></div><div>Sebastiano</div></div></div>