[LITMUS^RT] Testing tools

[Glenn Elliott]

On Aug 31, 2012, at 5:31 PM, Jonathan Herman <hermanjl at cs.unc.edu> wrote:

> I'd like to get an inventory of the Litmus testing tools people have
> coded for their own use. For instance, I know Chris and I have similar
> debugging scripts, and he has some code lying around that we have used
> to safely automate and repeat multi-hour / day tests. Could people
> give a short description of their tools and their use?
> 
> Why: I'd like to settle on one or two different frameworks / systems
> for testing that would be somewhat officially supported. These would
> be the default tools we would point new people at if they want to get
> started with serious litmus development.
> 
> 1. Some kind of simple debugging script, used for quick one-off tests
> to validate general correctness of plugins and other kernel changes.
> This should take very little effort to setup and use.
> 
> 2. An extendable system which can handle longer and repeated
> experiments, like the test suites we use to generate results put in
> papers.
> 
> If there is something brilliant enough to do (1) and (2), we would
> only need one to.
> ______
> do_trace: called like
> $ do_trace <file> <scheduler> <duration>
> $ do_trace color-4.sched GSN-EDF 10
> 
> <file> contains lines of parameters (not -w) to pass to rtspin, like:
> 10 20
> -p 3 30 60
> -c a 70 90
> 
> 1. Runs rtspins in parallel
> 2. If the kernel was built with TRACE debugging, outputs TRACE to log.slog
> 3. If the kernel was built with overhead measurements, outputs
> overheads to ft-*.bin files
> 4. If the kernel was built with sched_trace measurements, outputs data
> to st-*.bin files
> 5. If the kernel was built with kernelshark debugging, outputs data to trace.dat
> 
> 
> -- 
> Jonathan Herman
> Department of Computer Science at UNC Chapel Hill


I can think of the following that may be of interest to others:

1) A tool that will detect priority inversions in sched_trace logs for EDF schedulers.  Its pretty rough, but could be reworked.  C++.
2) A tool that will compute response time statistics, including cumulative distribution functions, from sched_trace logs.  C++.
3) Scripts for running tests.  Uses 'numactl' to set up clustering across NUMA boundaries.  Sends email notification when tests have completed.  Python.
4) Script for controlling Linux's throttling of SCHED_FIFO/_RR CPU allocations.
5) Schedulability test code implementing Jeremy's compliance-vector-based test.  C++.  (Does not implement his latest method that modifies deadlines.)

-Glenn