<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Sep 19, 2014 at 4:25 AM, Björn Brandenburg <span dir="ltr"><<a href="mailto:bbb@mpi-sws.org" target="_blank">bbb@mpi-sws.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class=""><br>
On 19 Sep 2014, at 08:12, Christopher Kenna <<a href="mailto:cjk@cs.unc.edu">cjk@cs.unc.edu</a>> wrote:<br>
<br>
> The stack traces might have been useful in determining the call path into gsnedf_schedule(). Why did you remove them? It's hard to say anything about the state of interrupts, especially if there's no context around how gsnedf_schedule() was called.<br>
><br>
> -- Chris<br>
<br>
</span>Indeed, please don't truncate error reports; it makes diagnosing issues much more complicated.<br>
<span class=""><br></span></blockquote><div><br></div><div>Here's the stack traces that I removed.</div><div><br></div><div><div>3484 P0 [vprintk_emit@kernel/printk.c:1564]: 4------------[ cut here ]------------</div><div>3485 P0 [vprintk_emit@kernel/printk.c:1564]: 4WARNING: at kernel/smp.c:382 smp_call_function_many+0xa5/0x290()</div><div>3486 P0 [vprintk_emit@kernel/printk.c:1564]: dModules linked in:3487 P0 [vprintk_emit@kernel/printk.c:1564]: </div><div>3488 P0 [vprintk_emit@kernel/printk.c:1564]: dCPU: 0 PID: 1959 Comm: release_ts Not tainted 3.10.5-kvm-test-pagefault #240</div><div>3489 P0 [vprintk_emit@kernel/printk.c:1564]: dHardware name: , BIOS QEMU 01/01/2007</div><div>3490 P0 [vprintk_emit@kernel/printk.c:1564]: 00000000000000093491 P0 [vprintk_emit@kernel/printk.c:1564]: ffff880072c9fab03492 P0 [vprintk_emit@kernel/printk.c:1564]: ffffffff815025113493 P0 [vprintk_emit@kernel/printk.c:1564]: ffff880072c9fae83494 P0 [vprintk_emit@kernel/printk.c:1564]: </div><div>3495 P0 [vprintk_emit@kernel/printk.c:1564]: ffffffff8102eb3b3496 P0 [vprintk_emit@kernel/printk.c:1564]: ffff880079aaea403497 P0 [vprintk_emit@kernel/printk.c:1564]: 00000000000000003498 P0 [vprintk_emit@kernel/printk.c:1564]: 00007ff19e08b0003499 P0 [vprintk_emit@kernel/printk.c:1564]: </div><div>3500 P0 [vprintk_emit@kernel/printk.c:1564]: ffff8800737b84583501 P0 [vprintk_emit@kernel/printk.c:1564]: ffff880079aaecf03502 P0 [vprintk_emit@kernel/printk.c:1564]: ffff880072c9faf83503 P0 [vprintk_emit@kernel/printk.c:1564]: ffffffff8102eb853504 P0 [vprintk_emit@kernel/printk.c:1564]: </div><div>3505 P0 [vprintk_emit@kernel/printk.c:1564]: Call Trace:</div><div>3506 P0 [vprintk_emit@kernel/printk.c:1564]: 3507 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81502511>] dump_stack+0x19/0x1b</div><div>3508 P0 [vprintk_emit@kernel/printk.c:1564]: 3509 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8102eb3b>] warn_slowpath_common+0x6b/0xa0</div><div>3510 P0 [vprintk_emit@kernel/printk.c:1564]: 3511 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8102eb85>] warn_slowpath_null+0x15/0x20</div><div>3512 P0 [vprintk_emit@kernel/printk.c:1564]: 3513 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81075545>] smp_call_function_many+0xa5/0x290</div><div>3514 P0 [vprintk_emit@kernel/printk.c:1564]: 3515 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81029ca0>] ? do_flush_tlb_all+0x60/0x60</div><div>3516 P0 [vprintk_emit@kernel/printk.c:1564]: 3517 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff815067fc>] ? _raw_spin_unlock+0x2c/0x40</div><div>3518 P0 [vprintk_emit@kernel/printk.c:1564]: 3519 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81029d69>] native_flush_tlb_others+0x29/0x30</div><div>3520 P0 [vprintk_emit@kernel/printk.c:1564]: 3521 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8102a118>] flush_tlb_page+0x98/0xe0</div><div>3522 P0 [vprintk_emit@kernel/printk.c:1564]: 3523 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8127746c>] walk_pte_entry+0x15c/0x240</div><div>3524 P0 [vprintk_emit@kernel/printk.c:1564]: 3525 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff810d7e56>] walk_page_range+0x366/0x430</div><div>3526 P0 [vprintk_emit@kernel/printk.c:1564]: 3527 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81278a23>] gsnedf_schedule+0x523/0x960</div><div>3528 P0 [vprintk_emit@kernel/printk.c:1564]: 3529 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8128c05f>] ? number.isra.1+0x2ff/0x330</div><div>3530 P0 [vprintk_emit@kernel/printk.c:1564]: 3531 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81277310>] ? gsnedf_finish_switch+0x40/0x40</div><div>3532 P0 [vprintk_emit@kernel/printk.c:1564]: 3533 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81067db0>] pick_next_task_litmus+0x40/0x4e0</div><div>3534 P0 [vprintk_emit@kernel/printk.c:1564]: 3535 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff810605e8>] ? update_curr+0x58/0x140</div><div>3536 P0 [vprintk_emit@kernel/printk.c:1564]: 3537 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81060840>] ? __enqueue_entity+0x70/0x80</div><div>3538 P0 [vprintk_emit@kernel/printk.c:1564]: 3539 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81505212>] __schedule+0x1f2/0xa70</div><div>3540 P0 [vprintk_emit@kernel/printk.c:1564]: 3541 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8150683e>] ? _raw_spin_unlock_irqrestore+0x2e/0x40</div><div>3542 P0 [vprintk_emit@kernel/printk.c:1564]: 3543 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8105e211>] ? try_to_wake_up+0x1c1/0x400</div><div>3544 P0 [vprintk_emit@kernel/printk.c:1564]: 3545 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81505b62>] preempt_schedule+0x42/0x60</div><div>3546 P0 [vprintk_emit@kernel/printk.c:1564]: 3547 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8150683e>] _raw_spin_unlock_irqrestore+0x2e/0x40</div><div>3548 P0 [vprintk_emit@kernel/printk.c:1564]: 3549 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff8105857d>] complete+0x4d/0x60</div><div>3550 P0 [vprintk_emit@kernel/printk.c:1564]: 3551 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81273c78>] sys_release_ts+0xf8/0x1d0</div><div>3552 P0 [vprintk_emit@kernel/printk.c:1564]: 3553 P0 [vprintk_emit@kernel/printk.c:1564]: [<ffffffff81507852>] system_call_fastpath+0x16/0x1b</div><div>3554 P0 [vprintk_emit@kernel/printk.c:1564]: 4---[ end trace 74e2249803801847 ]---</div></div><div><br></div><div>smp_call_function_many() should not be called with interrupts disabled. But, if I want to call set_pte() in gsnedf_schedule to modify page table entry, I need to call flush_tlb_page() which invokes the tlb flush function on remote CPU.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class="">
> On Thu, Sep 18, 2014 at 8:18 PM, Namhoon Kim <<a href="mailto:namhoonk@cs.unc.edu">namhoonk@cs.unc.edu</a>> wrote:<br>
> Hi All,<br>
><br>
> I'd like to modify page permissions to invoke a page fault. So, I used set_pte() with modified pte access bits in gsnedf_schedule() . According to kernel documentation (<a href="https://www.kernel.org/doc/Documentation/cachetlb.txt" target="_blank">https://www.kernel.org/doc/Documentation/cachetlb.txt</a>), I should call flush_cache_page() before set_pte() and call flush_tlb_page() after set_pte(). But, when I run several rtspin tasks, the following warning messages are shown.<br>
></span> </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class="">
> 3194 P0 [vprintk_emit@kernel/printk.c:1564]: 4------------[ cut here ]------------<br>
> 3195 P0 [vprintk_emit@kernel/printk.c:1564]: 4WARNING: at kernel/smp.c:244 smp_call_function_single+0x12f/0x190()<br>
> 3196 P0 [vprintk_emit@kernel/printk.c:1564]: dModules linked in:3197 P0 [vprintk_emit@kernel/printk.c:1564]:<br>
> 3198 P0 [vprintk_emit@kernel/printk.c:1564]: dCPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 3.10.5-kvm-test-pagefault #223<br>
> 3199 P0 [vprintk_emit@kernel/printk.c:1564]: dHardware name: , BIOS QEMU 01/01/2007<br>
> ~~~(stack traces)<br>
><br>
> The codes at kernel/smp.c:244: (<a href="https://github.com/LITMUS-RT/litmus-rt/blob/master/kernel/smp.c#L243" target="_blank">https://github.com/LITMUS-RT/litmus-rt/blob/master/kernel/smp.c#L243</a>)<br>
> /*<br>
> * Can deadlock when called with interrupts disabled.<br>
> * We allow cpu's that are not yet online though, as no one else can<br>
> * send smp call function interrupt to this cpu and as such deadlocks<br>
> * can't happen.<br>
> */<br>
><br>
><br>
> WARN_ON_ONCE(cpu_online(this_cpu) && irqs_disabled<br>
> ()<br>
> && !<br>
> oops_in_progress);<br>
> Are interrupts disabled when we enter gsnedf_schedule()? Does anyone know how to modify page access bits in schedule() properly?<br>
<br>
</span>Yes, of course. Interrupts are always disabled while scheduling in Linux to defend against scheduler invocations within interrupts. You can easily see this for yourself by observing that gsnedf_schedule() is called from within schedule(), which disables all interrupts.<br>
<span class=""><font color="#888888"><br></font></span></blockquote><div><br></div><div>I can put the set_pte() and flush_tlb_page() in Linux schedule() after interrupts enabled, but I think it is not the proper way. I don't want to touch any Linux functions. What is the proper way to use smp_call_functions in LITMUS schedule()? </div><div><br></div><div>-Namhoon<br></div></div></div></div>