[LITMUS^RT] prop/sched-domains

[Björn Brandenburg]

On 27 Feb 2014, at 09:11, Björn Brandenburg <bbb at mpi-sws.org> wrote:

> 
> On 25 Feb 2014, at 23:37, Glenn Elliott <gelliott at cs.unc.edu> wrote:
> 
>> Arg!  I used GFP_ATOMIC in all of my code, not realizing Linux’s proc_create_data() would call plain kmalloc().  Here are two possible solutions:
>> 1) Use kworker to set up the proc files after successful activation.
>> 2) Use kworker to perform plugin activation.
>> 
>> I have run into a similar problem in my GPUSync branch.  Here, I need plugin deactivation to wait/join on interrupt handling threads that are signaled to exit/shutdown.  In GPUSync, an echo to /proc/litmus/active_plugin triggers an event on kworker, which invokes the real activation routine.  Problem solved.  However, plugin activation becomes asynchronous.  You have to wait for some epsilon after activating a plugin to know if it was successful.  That’s not very nice.  I don’t know how else to approach the problem though.
>> 
>> Back to my sched-domains patch, normally I would advocate solution #1, since plugin activation remains synchronous.  However, since I’ve run into this problem before, I am willing to entertain solution #2.  I think we can defend against race conditions by serializing all plugin activation/deactivations on the same worker thread (perhaps we set up a reserved kworker just for litmus).
>> 
>> Questions of the Litmus community:  Does anyone have an opinion one way or the other?  Has atomic plugin activation caused you problems in the past?
> 
> Hi Glenn,
> 
> is there a way to avoid introducing kthreads for this? How about we call a “prepare_plugin()” callback from switch_sched_plugin() (https://github.com/LITMUS-RT/litmus-rt/blob/staging/litmus/litmus.c#L447) before executing in atomic mode?
> 
> Basically: 
> 
> 1) acquire switching mutex
> 2) check preconditions (plugin known, no RT tasks, etc.)
> 3) call prepare_plugin() on new plugin while sleeping is ok, abort if non-zero return value
> 4) call stop_machine(), recheck preconditions, etc., do the actual switch
> 5) call cleanup_plugin() on old plugin
> 6) release switching mutex
> 
> This would keep the switch synchronous, allow sleeping to take place as part of prepare_plugin(), and does not require kthreads. Anything I’ve overlooked?


I've coded up a simple patch for this. It seems to work ok.

	https://github.com/LITMUS-RT/litmus-rt/compare/staging...prop%2Fdomain-non-atomic-alloc

As byproduct, these patches introduce a new API for disabling plugin switches, which I need for some other project anyway.

Please let me know if I accidentally broke something with the patch. Otherwise, I'll merge it for the next release, which I intend to prepare soon.

Thanks,
Björn