[gs-code-review] regression test system generation 2

[Jack Moffitt]

I've attached a tarball containing the new generation of the regression
test system.  Here are the highlights:

 * reorganized the code (it is much simpler now and more capable).  the
   new database values are dicts instead of strings, which means they
   can store arbitrary data, not just md5sums.

 * new tests for runtime discrepancies, stdout/stderr differences, and
   pdfwrite encryption

 * test failures have more informative output.  if a stdout/stderr 
   difference is detected, the output is a unified diff.  if rasters are
   different, the output is a fuzzy comparision report.

 * daily test results are only saved if the entire suite ran to
   completion.  if the test is aborted or the system crashes, an 
   incomplete daily test result database will not be saved.

 * handling of parameters for update_baseline, update_specific, etc was
   improved.  now there is a unified way to do this.

 * update_specific now maintains and uses cached builds for speedier
   operations.  the number of environments to cache is configurable and
   each environment represents a single cvs datespec.

 * segfaults are now properly detected when running ghostscript.

 * A default set of tests is defined, but tests can be overridden on a 
   per file basis by creating a file.cfg file in the comparefiles 
   directory.  Getting a list of tests to run has been abstracted, so
   even more advanced behavior is easily added here.

 * Most of the core code is now in a package called "gstest".

 * Adding new types of tests should be much easier now with the
   refactored design.

A few notes about the code:
 
 * Python 2.4 is now required (decorated syntax on classmethods and
   possibly other things).

 * The __main__ module must define several special variables depending
   on the needed functionality.  __config__, __resultdb__, 
   __ghostscript__, and __dailydb__.  This makes the code simpler.

 * The pdfwrite test now compares the resulting pdf md5sums instead of 
   generating rasters.  

 * The default parameters could use customization on the pdfwrite and
   pdfenc tests.  I just made some guesses.

 * A few scripts are not windows ready, such as update_specific which
   needs to build and configure Ghostscript.

 * I have tested on small regression sets, but I have tried to excercise
   the bulk of the code.  Small problems are likely to crop up, but
   this code should clean up most of the previous annoyances.
 
I hope you enjoy :)

Regards,
jack.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: testsg2.tar.gz
Type: application/octet-stream
Size: 26341 bytes
Desc: not available
Url : http://ghostscript.com/pipermail/gs-code-review/attachments/20050604/167d455c/testsg2.tar-0001.obj