Delta debugging testcases

Description

Richard Chu created the design of the test cases based on his understanding on the JUnit test-driven development environment. The idea is to set up a standard Design Pattern for future test cases that allow modification and flexibility. This pattern will, in turn, interact with the rest of the Delta Debugging Framework, but this relationship needs to be understood for the interaction to work well.

Test Cases Design

There are many elements involved in the test cases, though they are split into categories.

Source Code

• This is where the actual program to be tested resides. The program can be in any language, but has to be able to be compiled and executed in a given environment (i.e. for a C++ program, there should be a C++ compiler in the system or Cygwin environment).
• It is suggested that this source code resides in a directory separated from the rest of the framework. The test will focus on that directory, and the framework should reside around it. Observe the following diagram:

 /home
   +-- /tests
     +-- /HelloWorld
       +-- HelloWorld.c

In this diagram, the framework should reside on the tests directory to allow it to wrap the source code.

• The idea is that this source code used to pass a test, but no longer due to recent changes in the code. User should then back up the source code, set up the framework, and run the Delta Debugging Algorithm to find the last revision that passes the test.

Build Script

• This is a Perl file (.pl), and should be named accordingly (e.g. HelloBuild.pl, SudokuBuild.pl).
• This file contains the script needed to properly build the source code. This script is located in the build() function, which will be called by the algorithm.
• The separation of the build file from the rest of the code allows user to select and customize various build systems or compilers that exist in the market. Whether the source code requires certain compiler, or uses Makefile for the GNU Make, this file handles the customization.
• Follow the HelloWorld and Sudoku template on how to create Build Scripts. Modify package name accordingly.

Test Module

• This is a Perl module (.pm), and should be named accordingly (e.g. HelloTestCase1.pm, SudokuTestCase1.pm).
• This module contains the actual test the source code will run against. If the user already has a test function that fails, that function can be registered and called here. The user can then describe what condition(s) would warrant the test to pass or fail.
• Should there be multiple tests, multiple test modules are suggested (hence the number in the name). The framework is designed to support multiple tests, and can be designed to run them all.
• Follow the HelloWorld and Sudoku template on how to create Test Modules. Modify package name accordingly.

Delta Debugging Script

• This is a Perl file (.pl), and should be named accordingly (e.g. DeltaDebugHello.pl, DeltaDebugSudoku.pl).
• The heart of the test case, this file contains the script that runs the Delta Debugging Algorithm. This is the script where you register your Build Script and Test Module. The SVN repository information should also be registered here to allow the algorithm to access the repository automatically.
• Follow the HelloWorld and Sudoku template on how to create Delta Debugging Script. Modify package name and included files accordingly.

Test Cases

• Test case 1 - Hello World
• Test case 2 - Sudoku