[gs-code-review] 688177 Eliminate the 64 bit pixel size limit

[Leonardo]

Dan,

> five lines of
> code that have to be changed for every reference to 'gx_color_index'

This statement isn't true in the word 'every'.

> Now you are suggesting that it may be desirable to 'encode' the pixel
> color data

I'm saying that hardware printers may use an encoding.
There is no strong need to use an encoding for the internal representation
of the color index
neither in device drivers nor in the graphics library,
rather a compression may be useful for the output raster buffer.

However a pixel data encoding isn't a single way for the compression,
and I guess that most printers will fist separate planes (inks),
then compress each plane independently - this is how
most modern graphic data formats are already organized.

A pixel encoding may be useful for organizing raster buffers for overprint,
but I think that a banding with a separable compression is better.
Also I think that multiple inks rasters are mostly useful with
wide printers (44+ inches), which need a banding by definition.

Leo.



----- Original Message ----- 
From: "Dan Coby" <dan.coby at artifex.com>
To: "Leonardo" <leonardo at artifex.com>; "Gs-Code-Review"
<gs-code-review at ghostscript.com>
Cc: "Miles Jones" <miles.jones at artifex.com>
Sent: Friday, September 02, 2005 2:02 AM
Subject: RE: [gs-code-review] 688177 Eliminate the 64 bit pixel size limit


>
> Leo,
>
>>Not sure why "many months of effort".
>>2 weaks should be enough.
>>A right way to choose a right alternative is to implement both methods
>>and then measure performance etc.
>
> I think that two weeks is a grossly low estimate. It would be like me
> estimating
> that ps2write would take two weeks. If I guess that there are five lines
> of
> code that have to be changed for every reference to 'gx_color_index' then
> there
> will be around 5000 lines of code that need to be changed. Obviously much
> of
> the changes required will be simple and will go very quickly. However
> there will
> be sections that will require extensive rework. My estimate is that just
> getting
> the halftoning back to working will take more than two weeks. Testing and
> debugging
> of the graphic library changes will take more than two weeks.
>
>
>>4. The output raster buffer.
>>There is some smart point here.
>
> Yes.
>
>
>>4.1. Old device drivers with small number of colorants won't work
>>for a big DeviceN colorants numbers exactly due to
>>the large "plain" representation of a color index. Such device drivers
>>will need a special encoding, which is not
>>necessarily equal to Dan's one.
>>Printer developers will develop many other encoding
>>schemes, and they will need special drivers for each scheme.
>
> Now you are suggesting that it may be desirable to 'encode' the pixel
> color data.
> This is exactly the idea that I proposed. I made it very clear in my first
> email
> that other schemes were possible and would not conflict with my effort and
> that my effort did not prohibit the use of other encoding schemes.
> Likewise
> my scheme can be used with or without Igor's proposal.
>
>
>>Trying to guess what will they need isn't a good way of
>>development, except you've got a special request.
>>Have you ?
>
> We do not have a specific request. However an encoding scheme was part of
> the
> discussion with one of our possible customers and I wanted to be ready
> with an
> example implementation to demonstrate the practicality of the idea. I do
> not claim
> that my encoding is the only one possible. I think that the one that I
> have proposed
> is a good one. I sent my efforts to gs-code-review to see if anyone has
> any
> suggestions about making it better.
>
>
> Dan
>
>
>
> -----Original Message-----
> From: Leonardo [mailto:leonardo at artifex.com]
> Sent: Thursday, September 01, 2005 12:16 PM
> To: Dan Coby; Gs-Code-Review
> Cc: Miles Jones
> Subject: Re: [gs-code-review] 688177 Eliminate the 64 bit pixel size limit
> xefitra
>
>
> Dan,
>
>> 1. The execution time for all color processing and drawing operations
>> would be
>> increased.
>
> Yes, in 0.1%.
> What's about your one ?
>
>> Compare
>> that to the many months of effort that would be required for Igor's
>> proposal
>
> Not sure why "many months of effort".
> 2 weaks should be enough.
> A right way to choose a right alternative is to implement both methods
> and then measure performance etc.
>
>> With Igor's approach, either the size of the raster buffers
>> would have
>> to be increased
>
> I'm not sure what raster buffers are you about.
> I can see 4 points here.
>
> 1. Buffers for temporary data allocated on stack.
> There are no big arrays in there.
> One can see few arrays of GS_IMAGE_MAX_COLOR_COMPONENTS elements,
> which will allocate 64*64*4 = 16K bytes on stack.
> The graphics library already has other big data on stack
> (the type1 interpreter, the filling algorithm, etc.)
> which allocate over 30K. So it should not cause a problem.
>
> 2. Raster buffers for  transparency.
> The blend color space can't be DeviceN.
>
> 3. Raster buffers for overprint.
> As I know, it uses the output raster buffer.
>
> 4. The output raster buffer.
> There is some smart point here.
> 4.1. Old device drivers with small number of colorants won't work
> for a big DeviceN colorants numbers exactly due to
> the large "plain" representation of a color index. Such device drivers
> will need a special encoding, which is not
> necessarily equal to Dan's one.
> Printer developers will develop many other encoding
> schemes, and they will need special drivers for each scheme.
> Trying to guess what will they need isn't a good way of
> development, except you've got a special request.
> Have you ?
> 4.2. New device drivers with new encodings
> will be developed for both approaches.
>
>> someone
>> tried to put 16 different inks at the same spot
>
> No problem with die sublimation.
> I'm not sure why do you think that an ink is a liquid.
> There are many hard ones.
>
> Leo.
>
> ----- Original Message -----
> From: "Dan Coby" <dan.coby at artifex.com>
> To: "Leonardo" <leonardo at artifex.com>; "Gs-Code-Review"
> <gs-code-review at ghostscript.com>
> Cc: "Miles Jones" <miles.jones at artifex.com>
> Sent: Thursday, September 01, 2005 10:36 PM
> Subject: RE: [gs-code-review] 688177 Eliminate the 64 bit pixel size limit
>
>
>>
>> Leo,
>>
>>>The main advantage of Igor's approach is to weeken the constraint
>>>forever,
>>>so that we'll not need to redo this effort after half year.
>>
>> I agree that by going to a pointer to an object that can be made
>> arbitrarily large,
>> that many of the constraints on the number of inks, etc. could be
>> eliminated.
>> And I do like that part of Igor's approach. In the past, I have also
>> advocated changing
>> to a pointer type as part of the solution to this problem. I like the
>> fact
>> that Igor's
>> approach can handle both a large number of colorants and a large number
>> of
>> bits per colorant.
>>
>> However there are several problems with Igor's approach. Some of them
>> are:
>>
>> 1. The execution time for all color processing and drawing operations
>> would be
>> increased. The execution time increase would be due several causes.
>> These
>> include the added overhead to reference pointer based data and the extra
>> memory
>> requirements of Igor's approach.
>>
>> This increase in execution time would be true even for non DeviceN
>> devices. So we
>> would seen an increase in execution time for simple CMYK, RGB and gray
>> devices.
>> This would affect our efforts in the embedded market.
>>
>> 2.  The memory usage with Igor's approach would be larger. To support 64
>> colors
>> with 8 bits per color, Igor's approach would require 64 bytes per pixel.
>> My approach
>> would only require 8 bytes.
>>
>> 3. In my approach, it is much simpler to detect and add spot colors as a
>> file is
>> processed. There is no change in the data in the raster buffers if a new
>> spot color
>> is detected. With Igor's approach, either the size of the raster buffers
>> would have
>> to be increased when a new spot color is detected or else the raster
>> buffers would
>> have to be allocated for the largest number of spot colors possible.
>>
>> 4. There would have to be major changes in the graphics library and the
>> various
>> device drivers. A quick grep of the source files show 1060 references to
>> 'gx_color_index'. Of these references, only 674 are inside of the device
>> driver
>> sources. The remainder are inside of the graphics library. For my
>> proposal,
>> I have put in approximately 1 and 1/2 weeks of part time effort. The
>> effort is not
>> complete but it does represent the major part of the development
>> required.
>> Compare
>> that to the many months of effort that would be required for Igor's
>> proposal. The
>> probability of creating bugs in the graphics library and devices is much
>> higher with
>> Igor's approach. In my approach, the changes are primarily in the
>> encode_color
>> and decode_color methods for the DeviceN devices. Thus the effects are
>> much
>> more localized and do not have possible side effects upon the operation
>> of
>> other
>> devices.
>>
>>
>>> ...  for example, if a tensor shading
>>>uses 4 source colors of 4 inks each, in the middle of the
>>>shading will use up to 16 inks, which overflows your constraint.
>>
>> Yes. I addressed this issue in my first email. If 16 colorants are
>> actually being
>> used in any one pixel, then my approach would be unable to encode the
>> result.
>> However my approach will happily work with 64 colorants (or more). The
>> only
>> requirement is that the user does not try to put them all on a single
>> spot
>> on the
>> page. Consider what would happen with a real printed page in which the
>> someone
>> tried to put 16 different inks at the same spot. The paper would be
>> unable
>> to absorb
>> the ink and the ink would run down the paper. I think that your example
>> is
>> more
>> of a theoretical problem than a real problem. With the files that I have
>> tested, to
>> date, the largest number colorants at a single spot has been six.  My
>> code
>> was
>> created to handle 14.
>>
>>
>> I realize that my approach has some theoretical problems with contrived
>> situations.
>> I do not think that they will appear in real data files. They definitely
>> have not appeared
>> in the testing to date. I discussed the theoretical problems in my first
>> email.
>>
>> Initially I also was in favor of going to a pointer based approach.
>> However I now think
>> that using a packed encoding scheme is better. The approach is much
>> simpler to
>> implement, it uses much less memory, and it is much less likely to have
>> bugs.
>>
>>
>> Dan
>>
>>
>> -----Original Message-----
>> From: Leonardo [mailto:leonardo at artifex.com]
>> Sent: Thursday, September 01, 2005 8:20 AM
>> To: dan.coby at artifex.com; Gs-Code-Review
>> Cc: Miles Jones
>> Subject: Re: [gs-code-review] 688177 Eliminate the 64 bit pixel size
>> limit
>> (xefitra)
>>
>>
>> Dan,
>>
>> The main advantage of Igor's approach is to weeken the constraint
>> forever,
>> so that we'll not need to redo this effort after half year.
>>
>> One uneasy thing of changing to a non-integer type
>> is to find out places of color index comparison.
>> The compiler may not find them automatically.
>> There are few ones in the graphics library code,
>> and likely they all are already found.
>> However customer's devices may need some debugging.
>>
>> By another hand, the numerous changes through entire graphics library
>> don't look as a disadvantage. Now color index operations
>> are encoded with C language operations, but the encoding is not 1-to-1.
>> A problem happens with gx_no_color_index, you know.
>> In Igor's code an absence of an index is a NULL pointer,
>> and it is different than any existing color index, which
>> requires a non-NULL pointer. Therefore such change
>> would eliminate the old hack, and it should be considered
>> as a progression.
>>
>> Igor's approach requires us to write an instruction for developers
>> how to upgrade code of their devices. It should start with
>> running a compiler with the old code and get a list of errors,
>> which points to lines, which to be changed.
>>
>> Besides that, you looks to take a big care of DeviceN process color
>> model,
>> and a small care about DeviceN colors in documents.
>> Your patch sets a constraint for document designers
>> not to use mixes of multiple inks. However such mix
>> may unintentionally appear in shadings : for example, if a tensor shading
>> uses 4 source colors of 4 inks each, in the middle of the
>> shading will use up to 16 inks, which overflows your constraint.
>>
>> Leo.
>>
>>
>>
>> ----- Original Message -----
>> From: "Dan Coby" <dan.coby at artifex.com>
>> To: "Igor V. Melichev" <igor.melichev at artifex.com>; "Gs-Code-Review"
>> <gs-code-review at ghostscript.com>
>> Cc: "Miles Jones" <miles.jones at artifex.com>
>> Sent: Thursday, September 01, 2005 12:29 AM
>> Subject: RE: [gs-code-review] 688177 Eliminate the 64 bit pixel size
>> limit
>> (xefitra)
>>
>>
>>>
>>> Igor,
>>>
>>> I am not sure about how your scheme works.  I did look at the URL that
>>> you
>>> gave in your email and also at the patches that you posted to the bug
>>> 688177
>>> bug report.  However I am still not sure about what you are doing.  If I
>>> am
>>> wrong about what you are doing then let me know.
>>>
>>> It looks like you are changing a gx_color_index to be a non integer
>>> type.
>>> If
>>> so then this is both good and bad.  The major advantage is that once
>>> that
>>> change
>>> is made, then it will probably be fairly easy to support things like a
>>> very
>>> large number of colorants.  The major disadvantages of using a non
>>> integer
>>> type
>>> are that changes have to be made throughout the entire graphics library
>>> and all
>>> of the current device drivers.  This is a very large effort.  Besides
>>> changing
>>> every reference to a gx_color_index type variable, there are issues with
>>> memory
>>> management for things like changing raster buffer sizes as spot
>>> colorants
>>> are
>>> found while processing a file.  This sort of change is also likely to
>>> make
>>> the general operation of the graphics library slower.  I do see that you
>>> seem
>>> to have a scheme which allows switching between an integer an non
>>> integer
>>> types
>>> at compile time.  This does eliminate some worries but it also creates
>>> other
>>> issues about testing the different versions.
>>>
>>> My first email gave a list of the advantages and disadvantages of my
>>> scheme.
>>> I would like to emphasize the advantages of this method include that it
>>> is
>>> quick
>>> to implement, it does support 64 (or more) colorants and it does not
>>> affect the
>>> majority of the graphics library or any other device.  This makes issues
>>> about
>>> support of customer and GPL devices much simpler and the likelihood of
>>> introducing
>>> bugs into the graphics library is eliminated.
>>>
>>>
>>> Dan
>>>
>>>
>>> -----Original Message-----
>>> From: gs-code-review-bounces at ghostscript.com
>>> [mailto:gs-code-review-bounces at ghostscript.com]On Behalf Of Igor V.
>>> Melichev
>>> Sent: Wednesday, August 31, 2005 6:46 AM
>>> To: Gs-Code-Review; dan.coby at artifex.com
>>> Cc: Miles Jones
>>> Subject: Re: [gs-code-review] 688177 Eliminate the 64 bit pixel size
>>> limit
>>>
>>>
>>> Dan,
>>>
>>> Your approach changes a very strong restriction to a strong one.
>>> IMO there is a way to change it to a very weak one,
>>> at least for the kernel of the graphics library :
>>>
>>>     color components <= 64
>>>     color component bits <= 16
>>>
>>> See gxcindex.h Revision: 1.7.2.2, Mon Jun 20 22:47:12 2005 UTC (2
>>> months,
>>> 1
>>> week ago) by igor
>>>
>>
> http://cvs.ghostscript.com/cgi-bin/viewcvs.cgi/cvs/ghostscript/gs/src/gxcindex.h?rev=1.7.2.2&only_with_tag=COLOR_INDEX_2_64&view=log
>>>
>>> Igor.
>>>
>>> ----- Original Message -----
>>> From: "Dan Coby" <dan.coby at artifex.com>
>>> To: "Gs-Code-Review" <gs-code-review at ghostscript.com>
>>> Sent: Wednesday, August 31, 2005 11:25 AM
>>> Subject: [gs-code-review] 688177 Eliminate the 64 bit pixel size limit
>>>
>>> [...skipped...]
>>>
>>>>  Number of non zero components    Index bits Bits per component
>>>> 0 to 5    24      8
>>>>    6    16    8
>>>>    7     8    8
>>>>    8          8    7
>>>>    9     8    6
>>>>   10     8    5
>>>>   11     8    5
>>>>   12     8    4
>>>>   13     8    4
>>>>   14     8    4
>>>>     More than 14 Not enclosable
>>>
>>> _______________________________________________
>>> gs-code-review mailing list
>>> gs-code-review at ghostscript.com
>>> http://www.ghostscript.com/mailman/listinfo/gs-code-review
>>>
>>>
>>
>>
>> _______________________________________________
>> gs-code-review mailing list
>> gs-code-review at ghostscript.com
>> http://www.ghostscript.com/mailman/listinfo/gs-code-review
>>
>
>
> _______________________________________________
> gs-code-review mailing list
> gs-code-review at ghostscript.com
> http://www.ghostscript.com/mailman/listinfo/gs-code-review
>