[LITMUS^RT] Testing tools

[Björn Brandenburg]

On Aug 31, 2012, at 11:31 PM, Jonathan Herman wrote:

> I'd like to get an inventory of the Litmus testing tools people have
> coded for their own use. For instance, I know Chris and I have similar
> debugging scripts, and he has some code lying around that we have used
> to safely automate and repeat multi-hour / day tests. Could people
> give a short description of their tools and their use?
> 
> Why: I'd like to settle on one or two different frameworks / systems
> for testing that would be somewhat officially supported. These would
> be the default tools we would point new people at if they want to get
> started with serious litmus development.
> 
> 1. Some kind of simple debugging script, used for quick one-off tests
> to validate general correctness of plugins and other kernel changes.
> This should take very little effort to setup and use.
> 
> 2. An extendable system which can handle longer and repeated
> experiments, like the test suites we use to generate results put in
> papers.

Hi Jonathan,

I have a bunch of Python scripts that do 2) for me, and that I also use for 1).

On the plus side, I have a fairly nice wrapper of the LITMUS^RT API in Python (e.g., activating plugins, setting cluster sizes, etc.).

On the negative side, a bunch of stuff is not as flexible as a general solution would have to be (hardcoded machine topology info, etc.). In addition, it depends on Schedcat for task set description and parsing.

I'd be happy to share the code, but in its current form it probably needs a lot of work to turn it into a tool that's very useful to others. (I don't have the time to do that right now.)

However, I fully support the goal to have a semi-official set of scripts that accomplish this, so if you come up with a consensus solution I'll try to adopt it and contribute. However, I would really hope for some solution that it is extensible enough to support task sets with locks.

Thanks,
Björn