[LITMUS^RT] Testing tools

[Jonathan Herman]

Could you put (3) in a publicly available place? I'd like to check it
out. Also, what is (4)?

On Mon, Sep 10, 2012 at 11:15 AM, Glenn Elliott <gelliott at cs.unc.edu> wrote:
>
> On Aug 31, 2012, at 5:31 PM, Jonathan Herman <hermanjl at cs.unc.edu> wrote:
>
>> I'd like to get an inventory of the Litmus testing tools people have
>> coded for their own use. For instance, I know Chris and I have similar
>> debugging scripts, and he has some code lying around that we have used
>> to safely automate and repeat multi-hour / day tests. Could people
>> give a short description of their tools and their use?
>>
>> Why: I'd like to settle on one or two different frameworks / systems
>> for testing that would be somewhat officially supported. These would
>> be the default tools we would point new people at if they want to get
>> started with serious litmus development.
>>
>> 1. Some kind of simple debugging script, used for quick one-off tests
>> to validate general correctness of plugins and other kernel changes.
>> This should take very little effort to setup and use.
>>
>> 2. An extendable system which can handle longer and repeated
>> experiments, like the test suites we use to generate results put in
>> papers.
>>
>> If there is something brilliant enough to do (1) and (2), we would
>> only need one to.
>> ______
>> do_trace: called like
>> $ do_trace <file> <scheduler> <duration>
>> $ do_trace color-4.sched GSN-EDF 10
>>
>> <file> contains lines of parameters (not -w) to pass to rtspin, like:
>> 10 20
>> -p 3 30 60
>> -c a 70 90
>>
>> 1. Runs rtspins in parallel
>> 2. If the kernel was built with TRACE debugging, outputs TRACE to log.slog
>> 3. If the kernel was built with overhead measurements, outputs
>> overheads to ft-*.bin files
>> 4. If the kernel was built with sched_trace measurements, outputs data
>> to st-*.bin files
>> 5. If the kernel was built with kernelshark debugging, outputs data to trace.dat
>>
>>
>> --
>> Jonathan Herman
>> Department of Computer Science at UNC Chapel Hill
>
>
> I can think of the following that may be of interest to others:
>
> 1) A tool that will detect priority inversions in sched_trace logs for EDF schedulers.  Its pretty rough, but could be reworked.  C++.
> 2) A tool that will compute response time statistics, including cumulative distribution functions, from sched_trace logs.  C++.
> 3) Scripts for running tests.  Uses 'numactl' to set up clustering across NUMA boundaries.  Sends email notification when tests have completed.  Python.
> 4) Script for controlling Linux's throttling of SCHED_FIFO/_RR CPU allocations.
> 5) Schedulability test code implementing Jeremy's compliance-vector-based test.  C++.  (Does not implement his latest method that modifies deadlines.)
>
> -Glenn
>
>
> _______________________________________________
> litmus-dev mailing list
> litmus-dev at lists.litmus-rt.org
> https://lists.litmus-rt.org/listinfo/litmus-dev



-- 
Jonathan Herman
Department of Computer Science at UNC Chapel Hill