[gs-devel] Page (clist) queuing -- Seperate Rendering thread(s)
Igor V. Melichev
igor.melichev at artifex.com
Mon Nov 12 10:04:58 PST 2007
I agree with Ray's functionality and disagree with Ray's architecture.
Ray assumes that both clist writer and reader exist in same memory space.
I don't think that it's an useful constraint.
IMO they may run at different sides of the Earth
or on different planets. Besides others, dropping this constraint
gives a simpler architecture, simpler coding,
and more flexibility as I'll explain below.
In my view the module gxpageq.c provides some useful
function prototypes but it is incomplete.
Particularly it is restricted with FIFO discipline.
Here is an example why FIFO is not acceptable.
Suppose an user prints a 100 page document
with a conveyor-like hardware, which performs
3 actions at a time :
1. Take the next page from the input tray and warm it;
2. Print the current page.
3. Cut off the last with TrimBox page and stock it to the output tray.
If a glitch happens, the user may want to continue
the conveyor printing because it is fast,
and to reprint the failed page later.
The user will insert the re-printed page manually into the booklet.
I did so multiple times.
In this example the failed page must not
be dequeued until it is reprinted, but
other pages to be dequeued
to save the printer's RAM.
gxpageq.c doesn't allow that.
Another hard problem with the old design
is a wrong subclassing. We need a class for
normal rendering and another class
for delayed rendering, and we want
to apply each of them to each raster device.
Instead doing so gdevbmpa.c simply duplicates
some devices, and hopes that we will continue
the duplicating till the hell. Thanks, I won't.
The right way is to derive with 2 base classes,
or to attach the second base class dynamically.
I think these 2 problems are enough for requesting another design.
So here is my one.
1. Any raster device may employ clist
if the user requests a delayed rendering.
2. If a delayed rendering is requested,
Each page is written as (blist,clist) into
a separate couple of files (bfile, cfile).
They will get unique file names.
3. If a delayed rendering is requested,
'showpage' does not run the clist interpreter.
Instead that it just writes a page title to the output file.
The title contains the file names mentioned above.
Then it applies flush() to the output file.
That's all changes to the old product.
gxpageq.c and gp_wsync.c will die.
4. In addition to document interpreter we will ship
another application, which reads the page headers from
a file and runs the clist interpreter for each page.
The user can connect it with the first application
with a pipe. The new application frees page data
5. If somebody will need to render pages
in a random order chosen by a human intervention,
he will need to insert a filter, which changes the
order of pages in the pipe. Also it will take care of
freeing page data when he finds it useful.
Therefore pages are stored in a file system
rather than in a single file.
Thanks to Ralph for noticing the "file system"
in the last meeting.
Actually the pager were never stored in a single file
due to (bfile and cfile).
6. In this design, it is not important
whether a build uses romfs or
the underlying OS's file system.
7. The project name to be changed to
"delayed page rendering".
The appearance of the word "async" in gdevprna.c
is misleading and must be removed.
8. An additional advantage of this design
is no change to the language switch build
and other top level makefiles.
That's all. I believe that the pipe and flush()
provides enough synchronization,
which is pretty portable.
Moreover, pipe exactly matches the
needs of the suggested functionality,
i.e. it doesn't set any extra constraint.
We will ship the technology,
and customers will be able to
create various solutions with it.
BTW, gp_wsync.c assumes too much about
the underlying OS, so it should't go to production
due to poor portability.
----- Original Message -----
From: "Ray Johnston" <Ray.Johnston at Artifex.com>
To: "gs-devel" <gs-devel at ghostscript.com>
Sent: Thursday, November 01, 2007 10:50 PM
Subject: [gs-devel] Page (clist) queuing -- Seperate Rendering thread(s)
> The below was sent to 'Tech' back in May. Henry suggested
> that this be posted to gs-devel to possibly get more dicussion
> This _was_ attached in my previous posting (it's that way in my
> "Sent" folder), but it didn't make it here. Trying again with the
> previous contents simply pasted in...
> Some notes on implementing 'async rendering' in a more convenient and
> way than the old 'gdevprna + gdevbmpa' technique...
> In general, this is based on the 'save page' functionality that Peter
> implemented for
> CalComp circa 1997 that was applied to a very feature rich RIP that used
> pages (on disk) to support separate writer(parser) and reader(renderer)
> on a RTOS that did _not_ have processes with separate memory spaces -- the
> OS 'tasks' were essentially threads. The features included:
> - Scheduling: Pages would be selected from the queue if the media in the
> was compatible with the requested parameters (media type, width,
> length, ink
> - Imposition: nesting small pages across a larger web, provided all of
> those pages
> had compatible "properties" where the properties included things like
> type (7 types), ink type (dye or pigment), resolution (360/720),
> quality mode
> (draft, normal, best)
> - Operator Priority Selection. Pages could be processed out of order if an
> or web based operator identified a "priority" job/page.
> - Reprint N: Since this was a disk based RIP, pages/jobs were retained and
> could be reprinted "N" times by an operator after examination of the
> - Collation of copies from jobs with multiple pages.
> The key to this functionality was the "saved page" that was a serialized
> (text) form structure that retained the information about the page(s) and
> jobs. This was serialized to a text file so that it could be written to
> (non-volatile in the case of power failure). The clist "c" and "b" file
> were part of the information in the saved page structure.
> Going forward:
> These concepts can be extended to any "page" device managed by gdevprn
> and to memory based clist (BAND_LIST_STORAGE=memory) modes.
> The design goals for the more general implementation are:
> - Easy to add to any clist capable device
> - Provide a framework queue that can be memory or disk based ( with
> an example implementation that has the basic functionality
> corresponding to the gdevbmpa device).
> - Queue entries (RAM or disk) contain the "pagedevice" parameters that
> were in effect at end of page (including PageSize, HWResolution,
> NumCopies, ProcessColorModel, ...) that might be usef..ul in selecting
> from the queue. We may want to add something for "job" association to
> support features like multiple copy collation.
> - Queue access that supports "peek" functions (not currently in gxpageq.c)
> to access the pagedevice parameters (including wild card _any_ match)
> - De-queue _any_ element to allow processing pages out of queuing order
> (for nesting/imposition, priority task, ...)
> - Queue support for "file" or "memory" (currently only memory) with
> (BAND_LIST_STORAGE) in either "memory" or "file". Using memory
> based clist will require
> - A default, simple queue server that processes pages in FIFO order that
> works with either the memory or file based queue.
> - The simple queue method is a standard runtime action based on a printer
> device parameter, e.g. -dQueuePages=true or the PS equivalent:
> << /QueuePages=true >> setpagedevice (PCL and XPS would rely
> on the -d method or calling the graphics library).
> Please let me know if there are any other features that you think need
> to be in the default implementation. Once real embedded customers
> start using this, we may find something else they need, but we can
> "build that bridge when we come to the creek"..
> gs-devel mailing list
> gs-devel at ghostscript.com
More information about the gs-devel