[Gs-devel] Report of Oct. 3 quarterly Artifex/artofcode/Aladdin meeting

[L. Peter Deutsch]

Early in each quarter, Miles, Raph, and Peter get together to discuss topics
of mutual interest about the present and future of Ghostscript.  We had the
third such meeting in Berkeley on Oct. 3.  Here is a summary report.  Please
note that all statistics, project statuses, etc. are as of Oct. 3.

The main agenda was various quality issues.  The report follows the logical
organization, not the order in which we discussed them.

We came up with a lot of "xxx should happen" statements, but rarely
identified a person to do them or a time frame.  I (Peter) fear this is
going to wind up meaning that most of it won't get done: I'll bring this up
at the January meeting if I remember.

Macro-quality issues
====================

The existing code together with the existing documentation is not
modularized well.  This makes it a lot harder for new people to wrap their
heads around the code, and to make changes without inadvertently damaging
the structure.  There is also no clear vision of how to address this issue
going forward.

    Action item: Raph will write a "Future of Ghostscript" document that
    includes some discussion of this issue.  (His "Ghostscript Roadmap"
    internal document will be a starting point.)

There are significant speed and output quality issues in some areas:

	* Output quality from Type 1 hinting (both monobit and gray-scale)
	  and sub-pixel positioning.  Igor did some work on this, but it
	  isn't clear what path we should take next.  (Raph suggests that
	  hinting should be turned off by default if anti-aliasing is
	  selected.)

	* Shading quality and speed.  Raph will eventually rewrite the
          shading code, but there is no identified timescale.

	* Speed of interpolated images.  The right thing to do here is to
	  replace the current computationally expensive digital filter with
	  the much simpler geometric interpolation scheme that Adobe uses.
	  The architecture for this is all in place, and there is even a
	  very bad sketch of the code already (siinterp.{c,h}): it just
	  needs an engineer to work on it.

The "(end-)user experience" of Ghostscript is pretty bad, even for
Unix/Linux users who are savvy enough to compile the code.  Some ideas for
improving the situation include:

	* Redoing the way devices are selected, moving this to a separate
	  configuration file.

	* Using autoconf on Unix/Linux platforms to handle environment
	  issues (NOT user-selectable options).

	* Switching to an external-driver model processing an intermediate
          bitmap for new printers, so users don't have to recompile
          Ghostscript for their own configuration.  (This is apparently
          easier for users than using the dynamically loaded driver model.
	  There may also be technical issues.  Raph?)

Raph is by default the person to worry about all of this, since he accepts
the responsibility of being the Ghostscript process-and-tools maintainer and
developer.

The remaining quality areas are, in comparison, "micro".

pdfwrite
========

While pdfwrite has improved tremendously in the past 2 years, it still
doesn't quite challenge Acrobat.  We identified specific areas requiring
work:

	* Images must be fully stored to handle auto-compression and to
	  avoid premature termination, which screws up JPEG compression.
	  Fixing this will require an architectural change, but not a big
	  one.

	* Incrementally loaded fonts don't always work correctly,
	  specifically in the case of save ... load some CharStrings ...
	  restore.  Fixing this might require some substantial changes.

	* Fill/stroke/text colors are always converted to either DeviceRGB
	  or DeviceCMYK.  Fixing this requires some rethinking of the driver
	  procedures that do color mapping, and also generalizing the
	  current set of native device color models.  This isn't causing a
	  lot of concern right now: it should be addressed as part of the
	  ever-receding DeviceN color project.

	* General CIEBased color spaces should be converted to ICCBased.
	  All of the new code is written: it awaits some refactoring of
	  gsciemap.c to complete it.  This is a big issue for images
	  specified in CIEBased color spaces, which are currently broken
	  down into zillions of tiny rectangles, with horrendous space and
	  time consequences.

	* TrueType fonts continue to cause endless grief with Acrobat
	  Reader, mostly because the Adobe documentation on this issue is
	  hopelessly inadequate.  We (Peter, as an Adobe-recognized reviewer
	  of the PDF specification) will probably have to take this up again
	  with Adobe.

Peter will continue to handle this code until Stefan is ready to tackle
redoing pswrite to merge it with pdfwrite.  There is also some perception
that pdfwrite isn't stable, although in fact it is about as reliable as the
rest of Ghostscript.

Release management
==================

We discussed the concept of "brown bag level" of quality -- the quality
level above which the maintainers / authors are no longer so embarrassed
that they have to put a brown paper bag over their head to avoid being
identified in public.  (Thanks to Linus Torvalds for this concept.)  Raph
and Peter have different philosophies about code visibility: Raph thinks
it's often OK to make sub-brown-bag code visible, as long as it's identified
as such, in order to get it out in front of people earlier, and Peter
doesn't.

The current Release.htm is out of date and not very useful to Raph or other
release builders.  Raph and Peter will get together in the near future to
update it to a document that Raph will sign an oath in blood to actually
follow when making releases.

Release numbering
-----------------

After much thrashing around, we concluded that something very close to
Aladdin's numbering and branching policy was the way to go.

	* Version numbers N.0x and N.5x will be reserved for stable
	  releases.

	* Immediately after making a N.00 (N.50) stable release, the HEAD
	  version number will be incremented to N.10 (N.60).

	* Version numbers N.0x / N.5x, 1 <= x <= 9, will be reserved for
	  short, stubby stable branches off of the N.00 / N.50 releases.
	  The only changes going into these branches will be:

		* Bug fixes, not enhancements;

		* From a subsequent HEAD release;

		* That have major consequences if left unfixed;

		* And that can be fixed with only localized code changes.

	* All other version numbers will be used for development or beta
	  (HEAD) releases at reasonably frequent intervals (3-6 weeks or so
	  for development, as often as needed for beta).  The version number
	  in CVS will be incremented immediately after each numbered
	  release, so that the History files will accurately describe all
	  numbered releases other than the one currently in CVS.

All of this will be documented in the updated Release.htm (see above).
(Most of it is already.)

Windows exe files
-----------------

Windows users give a lot of valuable test feedback, but they need pre-built
executables.  The current delay in posting Windows executables discourages
Windows users.  The only reason Raph doesn't build them as part of a release
is that he doesn't have a Windows development environment.  He will get one,
and then build and test Windows binaries for every release.  In fact, if we
can figure out how to script it, we'd like the nightly regression test run
to build and post Windows executables of the current CVS tree.

Regression testing
------------------

The automated regression test scripts were exposed before they reached brown
bag quality (see above).  We agreed they should get put in the Ghostscript
release toolbin directory ASAP so that people other than their primary
developer can have a chance to improve them.

ps2pdf testing
--------------

ps2pdf (pdfwrite) is not getting tested.  Ray has some good ideas about how
to do this.  Comparing PS -> raster output with PS -> PDF -> raster output
seems like the way to go.  We agreed this should get added to the nightly
regression test.

Documentation
-------------

Develop.htm is getting neglected.  We agreed to run hrefcov as part of the
nightly regression test.

Support
=======

The weekly bug summary report is wonderful.  However, the number of
SourceForge bugs has been hovering around 85 for a long time, vs. a goal of
40-50, 24 of these are still assigned to "nobody" (with a median age of
about 45 days), and almost half of the "high priority" bugs are over 3
months old.  It is claimed that triage is working better on the newer bugs,
but this isn't actually true: of the 26 bugs less than 2 months old, 8 are
assigned to nobody, a slightly *larger* percentage than for the list as a
whole.

We propose a once- or twice-yearly "bug scrub" (see "warning scrub" below)
to get down to the goal.  If 5% of engineering time is devoted to various
forms of scrubbing, that is approximately 2 weeks a year for each engineer,
which is enough time to accomplish quite a lot.

Code base
=========

icclib integration
------------------

It's unfortunate that icclib is integrated in a way that doesn't allow using
a pristine copy of the public library code, as for the other 3 libraries.
This happened simply because of priority triage: fixing it wasn't important
enough.

    Action item: Peter will fix this on some rainy day when he doesn't feel
    like doing anything really important.

Code hygiene
------------

It's unfortunate that the number of (gcc) warnings not sanctioned by
C-style.htm has been growing, and that some of them actually do reflect
underlying coding problems.  We came up with a good plan for addressing
this.

	* Since different compilers care about different things, pick 2-3
          compilers as the warnings yardstick.  (Proposed: gcc, MSVC, maybe
          Code Warrior or Intel P4 or IA-64 compiler.)  Use the automated
          regression test to detect the appearance of new unsanctioned
          warnings (by diff'ing the warning messages stripped of their line
          numbers), and treat these as real problems.

	* Do a one-time "warning scrub", in which for a week or so (say) the
	  entire engineering team devotes significant time simply to getting
	  rid of unsanctioned warnings.

There are two other areas in which one-time "scrubs" seem like they would
pay off:

	* Further reducing the use of macros where appropriate.

	* Implementing our documented commitment to ANSI-only code by
	  removing the Pn macros and the conditional use of __PROTOTYPES__.

We did not identify a leader or a schedule for the above, but it's in Raph's
territory.

-- 

L. Peter Deutsch     |    Aladdin Enterprises   |  203 Santa Margarita Ave.
ghost at aladdin.com    |  http://www.aladdin.com  |  Menlo Park, CA 94025

	The future of software is at http://www.opensource.org