[LITMUS^RT] proposing liblitmus API additions and changes

[Glenn Elliott]

Litmus Developers,

I've pushed two branches to github that extend liblitmus:
* prop/cpu-affinity
* prop/pthreadification

#1: prop/cpu-affinity
The first branch is prop/cpu-affinity.  Litmus requires that when a thread becomes a real-time task that this thread is executing on a CPU within the thread's desired cluster/partition.  This precondition can be achieved by properly setting up the thread's CPU affinity mask.  Unfortunately, liblitmus's be_migrate_to() is troublesome to work with because it takes a single CPU parameter.  It is entirely possible that the caller thread only knows what cluster/partition it should run on, not a particular CPU within that cluster/partition.  Things are further complicated if you add release master into the mix---tasks shouldn't include the release master (which can have an arbitrary value) in their affinity mask.  In the end, figuring out what CPU affinity mask to set can be complicated.

(8d65a8e4db) introduces the following functions (and other related functions):

* migrate_to_cpu(): Has the same functionality as be_migrate_to().

* migrate_to_partition(): Migrate a task to specified partition.  CPUs are indexed 0..m (or 0..m-1, if release master is enabled).  The release master can be an arbitrary CPU.  The partition-to-CPU mapping will just skip over the release master CPU.

* migrate_to_cluster(): Migrate a task to specified cluster. The caller just gives the cluster index and cluster size (small changes to C-EDF should allow us to remove the cluster size parameter in the future).  If release master is used, then the cluster size parameter should be the size of the largest cluster.  The release master is still excluded from the affinity mask (unless forced by __migrate_to_cluster()).  If cluster size is 1, then migrate_to_partition() is called instead.

* num_online_cpus(): Returns the number of online CPUs.

* release_master(): Returns the CPU that is the current release master.  -1 if no release master is in use.


#2: prop/pthreadification
This branch introduces a new pthread-styled API to setting up real-time task parameters.  The sporadic_* functions and macros are taking ever more parameters.  Worse, these parameters do not always apply to the active scheduler (ex. fixed priority).  This API should hopefully simply things with a familiar interface and make code more expressive.

(d9daa24e24) This patch introduces the core API.  Everything is centered around setting up a litmus_attr_t and then applying these attribute parameters to a thread.  This is similar to pthread's pthread_attr*() functions.  In the most common cases, callers will only need to do the following to set up their real-time parameters:

litmus_attr_t attr;
litmus_attr_init(&attr);
litmus_attr_set_execution(&attr, exe_time, time_units); /* time_units enumerated: TIME_NS, _US, _MS, _S */
litmus_attr_set_period(&attr, period_time, time_units);
litmus_attr_set_partition(&attr, partition);
litmus_set_attr(gettid(), &attr);  /* calls set_rt_task_params */

Yes, this is more code than even a very long function call or macro.  However, consider: (1) that existing macros can wrap this new API; (2) this approach allows thread attributes to be copied around; (3) time units are made explicit (no more guessing!); (4) very clear and expressive; (5) caller only needs to specify the parameters they care about (no need to deal with budgets or fixed priorities); (6) uses prop/cpu-affinity to make cluster/partition assignment easy.  We can also consider combining some of these setters into reasonable groups, such as: litmus_attr_set_params(&attr, exe, period, time_units).

I believe that there has also been some discussion about moving task configuration parameters to control pages or /proc.  I understand the motivation for this, but I thought a familiar API might be easier to work with.  Also, there is no reason why the parameters set in litmus_attr_t could not be directed to control pages or /proc instead of set_rt_task_params().  Should the user care where these parameters go (syscall, control page, or /proc)?


(fd5a0c8163) This patch builds on (d9daa24e24) to make a litmus_rt_task_create() function.  It's basically pthread_create() with an additional litmus_attr_t parameter.  I'm not entirely sold that this patch is needed, but I am interested in feedback.  It currently has two limitations/drawbacks: (1) all binaries must now include -pthread when compiling and linking; (2) errors in setting Litmus parameters are not reported back to the caller---not sure how to get around this one yet.


Please note that I have not fully tested this code; no doubt there are bugs.  There may be a lot of interest in these patches, so I wanted to wait for comments before committing more time.  If feedback is generally positive, then maybe we can discuss updating the rest of liblitmus to use the API and the stripping away of redundant code.

Thanks,
Glenn