[LITMUS^RT] Modifying Page Table Entry in gsnedf_schedule()

Fri Sep 19 17:56:39 CEST 2014

On Fri, Sep 19, 2014 at 4:25 AM, Björn Brandenburg <bbb at mpi-sws.org> wrote:

>
> On 19 Sep 2014, at 08:12, Christopher Kenna <cjk at cs.unc.edu> wrote:
>
> > The stack traces might have been useful in determining the call path
> into gsnedf_schedule(). Why did you remove them? It's hard to say anything
> about the state of interrupts, especially if there's no context around how
> gsnedf_schedule() was called.
> >
> >  -- Chris
>
> Indeed, please don't truncate error reports; it makes diagnosing issues
> much more complicated.
>
>
Here's the stack traces that I removed.

3484 P0 [vprintk_emit at kernel/printk.c:1564]: 4------------[ cut here
]------------
3485 P0 [vprintk_emit at kernel/printk.c:1564]: 4WARNING: at kernel/smp.c:382
smp_call_function_many+0xa5/0x290()
3486 P0 [vprintk_emit at kernel/printk.c:1564]: dModules linked in:3487 P0
[vprintk_emit at kernel/printk.c:1564]:
3488 P0 [vprintk_emit at kernel/printk.c:1564]: dCPU: 0 PID: 1959 Comm:
release_ts Not tainted 3.10.5-kvm-test-pagefault #240
3489 P0 [vprintk_emit at kernel/printk.c:1564]: dHardware name:  , BIOS QEMU
01/01/2007
3490 P0 [vprintk_emit at kernel/printk.c:1564]:  00000000000000093491 P0
[vprintk_emit at kernel/printk.c:1564]:  ffff880072c9fab03492 P0
[vprintk_emit at kernel/printk.c:1564]:  ffffffff815025113493 P0
[vprintk_emit at kernel/printk.c:1564]:  ffff880072c9fae83494 P0
[vprintk_emit at kernel/printk.c:1564]:
3495 P0 [vprintk_emit at kernel/printk.c:1564]:  ffffffff8102eb3b3496 P0
[vprintk_emit at kernel/printk.c:1564]:  ffff880079aaea403497 P0
[vprintk_emit at kernel/printk.c:1564]:  00000000000000003498 P0
[vprintk_emit at kernel/printk.c:1564]:  00007ff19e08b0003499 P0
[vprintk_emit at kernel/printk.c:1564]:
3500 P0 [vprintk_emit at kernel/printk.c:1564]:  ffff8800737b84583501 P0
[vprintk_emit at kernel/printk.c:1564]:  ffff880079aaecf03502 P0
[vprintk_emit at kernel/printk.c:1564]:  ffff880072c9faf83503 P0
[vprintk_emit at kernel/printk.c:1564]:  ffffffff8102eb853504 P0
[vprintk_emit at kernel/printk.c:1564]:
3505 P0 [vprintk_emit at kernel/printk.c:1564]: Call Trace:
3506 P0 [vprintk_emit at kernel/printk.c:1564]: 3507 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81502511>] dump_stack+0x19/0x1b
3508 P0 [vprintk_emit at kernel/printk.c:1564]: 3509 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8102eb3b>] warn_slowpath_common+0x6b/0xa0
3510 P0 [vprintk_emit at kernel/printk.c:1564]: 3511 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8102eb85>] warn_slowpath_null+0x15/0x20
3512 P0 [vprintk_emit at kernel/printk.c:1564]: 3513 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81075545>] smp_call_function_many+0xa5/0x290
3514 P0 [vprintk_emit at kernel/printk.c:1564]: 3515 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81029ca0>] ? do_flush_tlb_all+0x60/0x60
3516 P0 [vprintk_emit at kernel/printk.c:1564]: 3517 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff815067fc>] ? _raw_spin_unlock+0x2c/0x40
3518 P0 [vprintk_emit at kernel/printk.c:1564]: 3519 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81029d69>] native_flush_tlb_others+0x29/0x30
3520 P0 [vprintk_emit at kernel/printk.c:1564]: 3521 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8102a118>] flush_tlb_page+0x98/0xe0
3522 P0 [vprintk_emit at kernel/printk.c:1564]: 3523 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8127746c>] walk_pte_entry+0x15c/0x240
3524 P0 [vprintk_emit at kernel/printk.c:1564]: 3525 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff810d7e56>] walk_page_range+0x366/0x430
3526 P0 [vprintk_emit at kernel/printk.c:1564]: 3527 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81278a23>] gsnedf_schedule+0x523/0x960
3528 P0 [vprintk_emit at kernel/printk.c:1564]: 3529 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8128c05f>] ? number.isra.1+0x2ff/0x330
3530 P0 [vprintk_emit at kernel/printk.c:1564]: 3531 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81277310>] ? gsnedf_finish_switch+0x40/0x40
3532 P0 [vprintk_emit at kernel/printk.c:1564]: 3533 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81067db0>] pick_next_task_litmus+0x40/0x4e0
3534 P0 [vprintk_emit at kernel/printk.c:1564]: 3535 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff810605e8>] ? update_curr+0x58/0x140
3536 P0 [vprintk_emit at kernel/printk.c:1564]: 3537 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81060840>] ? __enqueue_entity+0x70/0x80
3538 P0 [vprintk_emit at kernel/printk.c:1564]: 3539 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81505212>] __schedule+0x1f2/0xa70
3540 P0 [vprintk_emit at kernel/printk.c:1564]: 3541 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8150683e>] ? _raw_spin_unlock_irqrestore+0x2e/0x40
3542 P0 [vprintk_emit at kernel/printk.c:1564]: 3543 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8105e211>] ? try_to_wake_up+0x1c1/0x400
3544 P0 [vprintk_emit at kernel/printk.c:1564]: 3545 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81505b62>] preempt_schedule+0x42/0x60
3546 P0 [vprintk_emit at kernel/printk.c:1564]: 3547 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8150683e>] _raw_spin_unlock_irqrestore+0x2e/0x40
3548 P0 [vprintk_emit at kernel/printk.c:1564]: 3549 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff8105857d>] complete+0x4d/0x60
3550 P0 [vprintk_emit at kernel/printk.c:1564]: 3551 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81273c78>] sys_release_ts+0xf8/0x1d0
3552 P0 [vprintk_emit at kernel/printk.c:1564]: 3553 P0
[vprintk_emit at kernel/printk.c:1564]:
 [<ffffffff81507852>] system_call_fastpath+0x16/0x1b
3554 P0 [vprintk_emit at kernel/printk.c:1564]: 4---[ end trace
74e2249803801847 ]---

smp_call_function_many() should not be called with interrupts disabled.
But, if I want to call set_pte() in gsnedf_schedule to modify page table
entry, I need to call flush_tlb_page() which invokes the tlb flush function
on remote CPU.

> On Thu, Sep 18, 2014 at 8:18 PM, Namhoon Kim <namhoonk at cs.unc.edu> wrote:
> > Hi All,
> >
> > I'd like to modify page permissions to invoke a page fault. So, I used
> set_pte() with modified pte access bits in gsnedf_schedule() . According to
> kernel documentation (
> https://www.kernel.org/doc/Documentation/cachetlb.txt), I should call
> flush_cache_page() before set_pte() and call flush_tlb_page() after
> set_pte(). But, when I run several rtspin tasks, the following warning
> messages are shown.
> >

> 3194 P0 [vprintk_emit at kernel/printk.c:1564]: 4------------[ cut here
> ]------------
> > 3195 P0 [vprintk_emit at kernel/printk.c:1564]: 4WARNING: at
> kernel/smp.c:244 smp_call_function_single+0x12f/0x190()
> > 3196 P0 [vprintk_emit at kernel/printk.c:1564]: dModules linked in:3197 P0
> [vprintk_emit at kernel/printk.c:1564]:
> > 3198 P0 [vprintk_emit at kernel/printk.c:1564]: dCPU: 0 PID: 0 Comm:
> swapper/0 Tainted: G        W    3.10.5-kvm-test-pagefault #223
> > 3199 P0 [vprintk_emit at kernel/printk.c:1564]: dHardware name:  , BIOS
> QEMU 01/01/2007
> > ~~~(stack traces)
> >
> > The codes at kernel/smp.c:244: (
> https://github.com/LITMUS-RT/litmus-rt/blob/master/kernel/smp.c#L243)
> > /*
> >  * Can deadlock when called with interrupts disabled.
> >  * We allow cpu's that are not yet online though, as no one else can
> >  * send smp call function interrupt to this cpu and as such deadlocks
> >  * can't happen.
> >  */
> >
> >
> > WARN_ON_ONCE(cpu_online(this_cpu) && irqs_disabled
> > ()
> >                 && !
> > oops_in_progress);
> > Are interrupts disabled when we enter gsnedf_schedule()? Does anyone
> know how to modify page access bits in schedule() properly?
>
> Yes, of course. Interrupts are always disabled while scheduling in Linux
> to defend against scheduler invocations within interrupts. You can easily
> see this for yourself by observing that gsnedf_schedule() is called from
> within schedule(), which disables all interrupts.
>
>
I can put the set_pte() and flush_tlb_page() in Linux schedule() after
interrupts enabled, but I think it is not the proper way. I don't want to
touch any Linux functions. What is the proper way to use smp_call_functions
in LITMUS schedule()?

-Namhoon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.litmus-rt.org/pipermail/litmus-dev/attachments/20140919/17ca62f0/attachment.html>