| | <<<Back 1 day (to 2012/07/19) | 2012/07/20 |
| kens | chrisl Ian sent me this URL: | 07:08.32 |
| | http://www.bbc.co.uk/news/uk-england-london-18911599 | 07:08.32 |
| | "However, our supplier substituted one font for another so that the wrong alphabet was used for the Arabic message, rendering it meaningless." | 07:08.46 |
| | We should keep the URL and direct customers who seem unable to grasp the problem with font substitution to it :-) | 07:09.16 |
| chrisl | Frankly, it rather sums up our Olympics :-( | 07:10.43 |
| | kens: I'm also trying *real* hard to feel sympathetic about this one: http://www.bbc.co.uk/news/business-18917906 | 07:17.11 |
| kens | Oh yeah that was on the radio this morning too :-) | 07:17.27 |
| | One of Uncle Fester's bigger mistakes | 07:17.49 |
| chrisl | I'd say that ranks as quite a howler! | 07:18.23 |
| kens | chrisl are you sure these fonts are being written in PFB format ? | 07:30.21 |
| | THey are certainly being written in binary but I don't see any of the PFB markers being written | 07:30.43 |
| chrisl | kens: well, it *looks* like the pfb output from t1asm | 07:31.18 |
| kens | Hmm, I'm not entirely sure that's the same thing | 07:31.30 |
| | I'll decode the output and look | 07:31.35 |
| chrisl | I *thought* pfb was the only binary encoding allowed for Type 1 fonts | 07:32.00 |
| kens | I'll check the T1 spec. My memory is that PFB is a disk-based format with sections headed by specific binary codes. | 07:32.45 |
| | Anyway, I'll look some more | 07:32.52 |
| | I think harsha9502 needs their wrist slapped | 07:34.25 |
| chrisl | I couldn't find specific pfa and pfb specifications | 07:35.45 |
| kens | IIRC PFB is documented somewhere else | 07:36.02 |
| | Because ist a shipping format for fonts, not a downloadable format | 07:36.13 |
| | chrisl what I'm thinking of is the 'IBM PC Format' of Font File Formats, described in section 3.3 of tech note 5040 'Supporting Downloadable PostScritp Language Fonts' | 07:41.21 |
| | Those are usually shipped as '.pfb' files while '.pfa' files are usually shipped as 'Hexadecimal format' (section 3.1) | 07:42.07 |
| | Your patch seems to disable the LZW encoding of rhe font, is that what you intended ? | 07:42.52 |
| chrisl | kens: only when !pdev->CompressFonts | 07:44.30 |
| kens | Well, leaving aside discussion of 'PFB' what do you tihnk it does in terms of filter loading ? | 07:45.01 |
| | I'm inclined not to make such a change, I'd rather add a new exemption for the Kyocera printer, unless you think there's a good readon not to | 07:46.20 |
| chrisl | With my patch, and -dCompressFonts=false we shouldn't add any filters in pdf_begin_data_stream() | 07:46.49 |
| kens | Yes, that's true | 07:47.03 |
| chrisl | The as we write out the type 1 font, we'll use ASCIHex for eexec encoded section | 07:47.34 |
| kens | The actual problem is that the Kyocera interpreter can't handle reading font data from a compressed stream ? | 07:47.40 |
| chrisl | No, it can't handle the binary eexec data | 07:47.57 |
| kens | That sounds ridiculous | 07:48.09 |
| | Stupid interpreter, that data is *supposed* to be binary | 07:48.23 |
| | Or is it ? | 07:48.27 |
| | Actually it can be either I think | 07:48.36 |
| chrisl | Well, as far as I've found, we're the only emitter that produces binary eexec data | 07:48.57 |
| kens | But its perfectly legal I believe ? | 07:49.13 |
| kens | goes to open teh T1 spec | 07:49.22 |
| chrisl | Yes, it is legal - but not recommended, IIRC | 07:49.35 |
| kens | So this patch does *2* things, one fixes the binary eexec, why the other change ? | 07:51.37 |
| chrisl | It seems pointless to ASCII85 encode data that is already ASCII | 07:52.08 |
| kens | Is that all the binary stuff does ? I thought it did more | 07:52.54 |
| chrisl | No, I think that's all it does, in this case, anyway | 07:53.39 |
| | FWIW, I've got no problem if you want to control it with a -dType1ASCII option, or something, my patch was mainly to show you how I thought we could get te ASCII Type 1 data out. | 07:55.17 |
| kens | I'm just trying to understand the pathc really | 07:55.32 |
| | I didn't realise that you were controlling 2 differnt filter setups here, I was only expecting one | 07:55.53 |
| | Also, terminology confusion over 'PFB'. | 07:56.33 |
| chrisl | Well, I'm still unable to find a definition of pfb, other than "binary encoding for Type 1 fonts" | 07:57.12 |
| kens | its been used (probably incorrectly) to describe the font format used for shipping fonts on disk, probably because such files generally end in '.pfb' | 07:58.11 |
| | It is (or was) the normal format for distribution of type 1 fonts on PCs. | 07:58.35 |
| chrisl | I'm quite happy to change the title of the bug...... | 07:59.02 |
| kens | No that's OK, its too late to save me confusion ;-) | 07:59.20 |
| | BTW I don't see any reccomendation in the T1 spec about using ascii rather than binary eexec data | 08:00.13 |
| | Its crap that the Kyocera printer can't handle legal data, but its such a poor implementation we probably shouldn't be surprised. | 08:00.41 |
| chrisl | sorry on phone | 08:07.04 |
| kens | NP | 08:07.30 |
| chrisl | kens: it's a little strange - about the middle of page 64 in the T1 spec is the "Note" I was thinking of | 08:21.25 |
| kens | Yes, which refers to binary potentially being a problem if you don't use 'eexec' | 08:21.49 |
| | But doesn't really mention the possibility of eexec data being binary anyway :-) | 08:22.04 |
| chrisl | Despite the fact that just above that it binary eexec data..... | 08:22.45 |
| kens | Exactly, sill y spec writer :-) | 08:22.59 |
| chrisl | Anyway, we do seem almost unique in PS emitters in not using the hex encoding | 08:23.24 |
| kens | chrisl I've chosen to *always* write the t1 eexec data as ASCIIHex | 08:23.29 |
| | And reformatted the text earlier to not check for is_ps2write | 08:23.48 |
| | In case we end up with a ps3write or extend ps2write to handle level 3 | 08:24.06 |
| | So I'm going to punt this off to the cluster for testing | 08:24.18 |
| | After that I'd like to create a file and ask fm__ to test it again. | 08:24.40 |
| chrisl | So, even for pdfwrite, we'll use the hex encoding? | 08:24.41 |
| kens | chrisl pdfwrite always converts type 1 to CFF | 08:24.53 |
| | But even so, yes. | 08:25.09 |
| | As you say we are unique in having binary eexec anyway, so we may as well be consistnet | 08:25.30 |
| chrisl | The conversion to CFF can be disabled, can't it? | 08:25.47 |
| kens | chrisl it can yes, in which case yo get what you deserve :-) | 08:26.02 |
| | I would be surprised if other PDF producers don't do the same anyway | 08:26.31 |
| | (ascii eexec portions) | 08:26.41 |
| chrisl | Fair enough. I thought there might be a specific pdfwrite "preset" which disabled CFF | 08:26.59 |
| kens | chrisl no, its really there for ps2write actually | 08:27.18 |
| | Its part of the work I did removing explicit tests for the device name | 08:27.29 |
| chrisl | Oh, right, well in that case, my patch was over complicated...... | 08:27.42 |
| kens | Not a problem | 08:27.51 |
| | But like I said, I feel a need to cluster test it, and ask fm__ to retest the output | 08:28.13 |
| | I'll also make the output art a more sensible resolution and see if that helps the performance | 08:28.28 |
| chrisl | kens: as far as our uniqeness is concerned, I looked at the output from Acrobat, the windows PS driver, xpdf/poppler, and Jaws/xlat, and they all use the hex encoding. | 08:29.03 |
| kens | Yeah, good enough for me | 08:29.14 |
| | I'd rather not mess around with another command line variable, we have too many as it is | 08:29.50 |
| chrisl | kens: for fm__ to test the output, I'll need to apply the cups "workflow gubbins" to the PS file first. | 08:29.58 |
| kens | chrisl OK can't I just rip the header off the existing file and replace teh one I generate ? | 08:30.21 |
| chrisl | It's more than just the header, there's setup stuff part way through, and then the PJL trailer, too | 08:30.52 |
| kens | OK well I'll pass it over to you then ;-) | 08:31.05 |
| | I'd suggest making two examples to test, -r300 and -r600 and see what the output is like qualitatively, and the performance | 08:31.30 |
| | Should I send you a diff ? | 08:31.54 |
| fm__ | thanks for working on it chrisl and kens | 08:31.55 |
| kens | fm__ : No problem, thanks for doign the donkey work of testing | 08:32.07 |
| chrisl | ken: Actually, I'm not sure that will make a difference - I think the "high res" images are images in the PDF source :-( | 08:32.15 |
| kens | Well, that is a nuisance | 08:32.33 |
| | You coudl apply DownsampleImages :-) | 08:32.40 |
| chrisl | I don't know how that works..... | 08:32.55 |
| kens | But maybe we should just stick with getting the output working :-) | 08:32.56 |
| | chrisl I'll send you a command line. | 08:33.12 |
| | Let me amke you a diff first so you can try out the font | 08:33.28 |
| chrisl | Also, does DownsampleImages work if the image is in a pattern? | 08:33.30 |
| kens | chrisl as far as I'm aware, yes | 08:33.45 |
| | But it might get confused by the matrix | 08:33.54 |
| | So you might get inappropriate downsampling | 08:34.08 |
| chrisl | That's what I wondered - I thought it might just disable it to avoid confusion | 08:34.23 |
| kens | I don't believe we can, we don't know whether an image is in a pattern definition I don't think | 08:34.49 |
| chrisl | kens: if it's easier for you to send me the diff, I'm okay with that - whatever is easier for you | 08:35.21 |
| kens | Well I don'[t want to co commit until its been tested by the cluster, and preferably on the printer | 08:35.50 |
| | cluster is running now | 08:36.07 |
| | I thought if I sent you a diff, you could get a file to fm__ to test while I waited for the cluster run | 08:36.25 |
| chrisl | Yeh, okay. I'm looking through the previous Kyocera performance issue I worked through, trying to fathom what the conclusion was - it wasn't as simple as resolution.... | 08:38.57 |
| kens | Silly printer, remind me not to buy one :-) | 08:39.19 |
| chrisl | Or at least, not until they upgrade their Postscript to be as good as their PDF ;-) | 08:39.58 |
| kens | OK diff sent | 08:40.57 |
| kens | mneeds coffee, back in a minute | 08:41.41 |
| chrisl | too | 08:43.58 |
| fm__ | chrisl, regarding your comment above. is my printer able to interpret pdf directly? | 08:49.30 |
| chrisl | fm__: I've no idea - some of Kyo's higher end lasers are able to handle PDF directly, and I happen to know what interpreter they use...... | 08:52.33 |
| fm__ | ;) | 08:52.52 |
| chrisl | fm__: can you try this one, please? http://www.ghostscript.com/~chrisl/printout-016.ps | 09:01.30 |
| kens | cluster test finished, looks fine | 09:01.48 |
| chrisl | well, I cluster tested my patch, and it was fine - after I released about the EOD flag in the ASCIIHex filter state! | 09:02.26 |
| fm__ | chrisl, sent to printer at 11:02:02 | 09:02.41 |
| kens | Yes I noticed that, its clearly an oversight bu whoever coded this | 09:02.44 |
| | Oops, I've accidentlaly cluster tested some intermi code as well, no problem though | 09:03.29 |
| fm__ | chrisl, i predict it takes 5 minutes for the printer to start moving :-( | 09:03.46 |
| chrisl | I wondered if the state initialisation originally didn't set it, and since that route through the code hasn't been in use, the change in behaviour was never noticed. | 09:03.57 |
| kens | chrisl that is possible, certainly the fact that the code path couldn't get exercised masked the problem. | 09:04.35 |
| | Oh, interesting. | 09:04.56 |
| | Big change in CPU time | 09:05.06 |
| | Biut all in GS, how strange | 09:05.25 |
| chrisl | Less decoding needed? | 09:05.56 |
| kens | Well I wondered that, but I'm not sure how Marcos breaks out the timings | 09:06.18 |
| | total CPU time dropped by 2 hours | 09:06.33 |
| | whihc is about 6%, quite a bit really | 09:07.11 |
| | That might be in comparison to my last cluster push though, and I may have picked up some new performance improvements from Robin and Michael | 09:07.45 |
| chrisl | I'm not sure that counts as significant, given the variability in the results..... | 09:08.16 |
| fm__ | chrisl, print finished at 11:08:05 | 09:08.26 |
| kens | Really ? Seems like quite a lot to me | 09:08.30 |
| fm__ | so it takes 6:03 minutes to print that page | 09:08.53 |
| kens | Give me a minute to experiment and I'll see if I can get ther images reduced | 09:09.13 |
| Robin_Watts | The patch I am testing consistently times out, and I have no idea why. | 09:09.16 |
| chrisl | fm__: yes, there's something strange there with the performance - the 1320D had performance problems with certain filter combinations, but the file you are testing shouldn't have any of those problem combinations :-( | 09:09.55 |
| | kens: FWIW, I have a *very* string suspicion that at least one of the two HP printer problems where the users "went dark" was down to this binary/ascii font thing, too | 09:11.49 |
| kens | chrisl I seem to recall we suspected that at the time, this wasn't entirely new, but as you say they went away and we weren't sure | 09:12.27 |
| chrisl | Yes, exactly - I had just created a test job with the Type 1 fonts converted to all ASCII, and that was the test I never got a reply about! | 09:13.16 |
| kens | :-( | 09:13.33 |
| chrisl | fm__: if you want to investigate the performance issue further, it would be best if you create a new Ubuntu bug (it's a totally separate issue to "won't print"). | 09:15.44 |
| kens | Hmm, not seeing any reduction in file size, its going to take a while to find out why, and I need to do some other stuff instead. | 09:19.11 |
| | GOt a customer problem I need to look at | 09:19.17 |
| | However, this does seem to resolve the failure to print, so I'll go ahead and commit it. | 09:19.40 |
| chrisl | kens: I'm not convinced the image resolution is the issue, anyway..... | 09:19.54 |
| kens | chrisl not sure, really, woudl need some more major investigation | 09:20.13 |
| | Difficult to do, too, because we don't have the pritner | 09:20.34 |
| fm__ | chrisl, https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1026974 | 09:20.42 |
| chrisl | kens: Well, like I say, the last Kyo performance problem turned out to be because we had to instances of ASCII85Decode in the filter chain! | 09:21.07 |
| kens | I suppose we could compare to the poppler output, if that runs significantly faster | 09:21.07 |
| | Which is daft I admit | 09:21.18 |
| fm__ | kens, poppler takes 10s compared to the 363 of gs ... | 09:21.48 |
| chrisl | fm__: I've subscribed myself to the new bug, thanks | 09:22.08 |
| | fm__: do you any bandwidth restrictions we need to worry about - I'd like you to test a file, but it's 2.9Mb...... | 09:24.36 |
| fm__ | chrisl, no bandwith restrictions | 09:25.12 |
| | flatrate and at least 1MB/s on my end | 09:25.25 |
| chrisl | fm__: cool. I'm just adding the printer specific bumph to our output, and I'll upload the file | 09:25.46 |
| fm__ | btw. has the ghostscript project always been that active? During university I had many files which did not print successfully, although reporting bugs on everything I never bothered to report them ... :-( | 09:27.32 |
| chrisl | fm__: can you try this, and see what the speed is like? http://www.ghostscript.com/~chrisl/printout-017.ps | 09:28.20 |
| kens | fm__ : I can only speak for the tenure of my employment, its been this active for the last 5 years | 09:28.30 |
| | Patch committed, that's a good one to get into the 9.06 release | 09:29.16 |
| chrisl | fm__: Ghostscript's relationship with the open source community has varied a lot - the ESP fork did a lot of damage to that relationship until the "rift" was sealed.... | 09:29.48 |
| fm__ | chrisl, somehow, my netcat does not finish ... | 09:33.06 |
| chrisl | fm__: it may take sometime to transfer the file to the printer, given that it's so much bigger..... possibly? | 09:33.37 |
| fm__ | it is three times bigger, so it should not take more than 20 times longer, should it? | 09:34.09 |
| chrisl | it rather depends on how the printer consumes the data, but I wouldn't expect *that* much longer, no. Damn, that's annoying :-( | 09:34.53 |
| fm__ | printer does not accept any file anymore. even the last one. i have restarted it and i can ping it. any idea? | 09:38.32 |
| kens | fm__ : you power-cycled it and it still won't respond ? :-O | 09:39.24 |
| chrisl | Well, my idea would be send the thing back - it's clearly broken in several ways! | 09:39.47 |
| fm__ | ;) | 09:40.16 |
| chrisl | kens: I can't seem to change the compression use for image data in an inline image, is that right? | 09:41.04 |
| kens | Correct | 09:41.18 |
| | inline images are always written with 'losslesss' compression | 09:41.30 |
| | Which (rather horribly) means they always end up with LZW | 09:41.49 |
| | I cna hack that of course. | 09:41.55 |
| chrisl | So, I can disable compression, but not change it? | 09:41.56 |
| kens | chrisl I'm not entirely certain you cna disable that compression... | 09:42.15 |
| | In any event, no you cna't change it | 09:42.21 |
| | Without hacking the code | 09:42.31 |
| chrisl | Using -dEncodeColorImages=false I get them just ASCII85 encoded, no compression | 09:43.33 |
| kens | OK I guess that bypasses the lossless stuff at a higher level | 09:43.57 |
| chrisl | And with 13 inline image in the file, I'm *not* going to change the compression "manually"...... | 09:45.22 |
| kens | Well you can hack the code | 09:45.38 |
| | Easiest way is to force inline to false | 09:46.24 |
| | You could also try setting MaxInlineImageSize to 0 | 09:46.47 |
| chrisl | I thought that didn't work with ps2write? | 09:47.03 |
| kens | Though of course that does force images to be references and changes the way the file works | 09:47.10 |
| | chrisl could be, I haven't checked, I'm working on somethign elsee | 09:47.23 |
| tkamppeter | chrisl, kens, can I simply apply the patch of bug 693197 to GS 9.05 (for patch release) or does it require additional patches? | 09:47.26 |
| kens | tkamppeter : should work with 9.05 | 09:47.49 |
| | I wouldn't like to try earlier releases though teh HaveCFF flags is fairly new | 09:48.08 |
| | chrisl look at 'psdf_setup_lossless_filters' | 09:48.50 |
| fm__ | chrisl, i guess i have a backchannel now | 09:49.36 |
| chrisl | Thanks | 09:49.36 |
| | fm__: huh? | 09:49.50 |
| fm__ | the nc just quits when the print is done | 09:49.51 |
| kens | Actually, make that psdf_setup_image_filters when ;lossless' is true | 09:49.54 |
| fm__ | running "date && nc -w1 192.168.178.3 9100 < printout-017.ps && date" now | 09:51.56 |
| chrisl | Oh, I can change the filter :-) | 09:52.08 |
| kens | Yes | 09:52.25 |
| | chrisl in setup_image_compression lossless_tamplate is defined | 09:52.56 |
| tkamppeter | kens, thanbk you, wil try | 09:53.03 |
| kens | You cna hack that. Look slike you cna also set UseFlateCOmpression and lie about pdev->version | 09:53.48 |
| chrisl | Oh, flate isn't Level 2, is it....... | 09:54.44 |
| kens | No | 09:54.52 |
| | That's why we don't use it | 09:54.59 |
| | Hmm, looks like the PDF interpreter isn't preserving the Flags of annotations, that sucks | 09:55.41 |
| chrisl | kens: silly question: is there a reason in-line images only ever use lossless compression? | 09:56.43 |
| fm__ | chrisl, your optimized file takes at least more then 6 minutes | 09:56.44 |
| | than | 09:56.59 |
| kens | chrisl I'm not entirely sure | 09:57.02 |
| | I think it may be a pdfwrite'ism rather than ps2write | 09:57.17 |
| fm__ | chrisl, 6:55 | 09:57.18 |
| chrisl | fm__: it's isn't optimized - if anything, it's entirely non-optimised, compared to the "default" | 09:57.26 |
| kens | For PDF they 'must' be small by definition, and so thre's no point in using a lossly compression method | 09:57.37 |
| | Of course this is not true for pdfwrite | 09:57.56 |
| | ps2write* | 09:58.02 |
| tkamppeter | kens, the patch applies to GS 9.05 and builds. | 10:00.50 |
| chrisl | fm__: so, I'm going to guess that the actual *printing* time was about the same, and the extra time was the transmission overhead of the larger, uncompressed file | 10:05.50 |
| kens | tkamppeter : if it compiles, then it should be just fine, otherwise you'd have got an 'undefined' on the flag entry in the structure | 10:05.55 |
| | chrisl that commit showed the same improvement in processing time with gs, weird.... I'm not going to complain though | 10:06.31 |
| tkamppeter | kens, chrisl, now trhere is also a performance problems: https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1026974 Have you already seen it? | 10:07.37 |
| kens | tkamppeter : chrisl asked fm__ to raise it if he wanted to pursue it | 10:08.00 |
| chrisl | Well, like I say, the only thing I can think of is reducing the time in the filters - maybe ASCIIHex is more efficient than ASCII85? | 10:08.01 |
| kens | I guess it must be, I'm just surprised that it comes out in the reporting. Puzzled I guess | 10:08.28 |
| tkamppeter | chrisl, what do you mean with reducing the time in the filters? fm__ gets this time by sending the PS file to the printer with nc, so the time is the pure processing time of the printer itself. | 10:09.35 |
| kens | tkamppeter : he means the filters used to compress the data in the PostScript file | 10:09.57 |
| | Not CUPS filters | 10:10.03 |
| chrisl | kens: based on fm__'s result with the uncompressed images, I'm not convinced changing to DCT will speed things up, either :-( | 10:16.01 |
| fm__ | chrisl, the one bug report has the poppler based ps attached, does that help to see the difference? | 10:18.00 |
| kens | Perhaps its not the images, have you tried stripping them out ? | 10:18.39 |
| | Or redefining /image :-) | 10:18.51 |
| chrisl | Other than images, there's not much else in there - well, the type 1 pattern containing an image might be an issue..... | 10:19.34 |
| kens | My guess would be the pattern | 10:19.52 |
| | First place to look | 10:20.00 |
| chrisl | Well, I can probably cut that out pretty easily.... | 10:20.12 |
| kens | Hacking the file to replace the pattern with a flat fill ? | 10:20.15 |
| chrisl | yeh | 10:20.31 |
| kens | A pattern with an image is the sort of thing I would expect to give a printer indigestion | 10:20.55 |
| | Can you send me the poppler output ? | 10:21.10 |
| chrisl | kens: it's here: https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1025061/comments/10 | 10:21.37 |
| kens | Thanks | 10:21.47 |
| | I see its actually larger than our output | 10:22.12 |
| chrisl | Well, the poppler output doesn't have a pattern in it | 10:23.40 |
| kens | Indeed | 10:23.47 |
| | I haven't looked at where the pattern is coming fomr | 10:24.12 |
| Robin_Watts | Mustafa Kadhum, the BBC's Arabic Online news editor, said "...Arabic words are always written with joined up letters, with some exceptions." | 10:24.30 |
| kens | However if its flattning patterns to images, I woudl expect problems with other printers/files | 10:24.31 |
| | Robin_Watts : yes, that's true | 10:24.41 |
| Robin_Watts | I'm always late, except when I'm not. | 10:24.43 |
| kens | The exceptions are on the order of 'I before E except after C' | 10:25.04 |
| | There is a non-cursive alphabet 'naskh' which nobody likes | 10:25.23 |
| Robin_Watts | It's yet another example of piss poor english, from the BBC news website. | 10:25.32 |
| kens | Well, that is true | 10:25.38 |
| | I would have said 'generally' rather than always | 10:25.47 |
| Robin_Watts | "Arabic words are usually written with joined up letters." would have been far more accurate. | 10:25.59 |
| kens | I suspect he was trying for emphasis | 10:26.52 |
| | But to be honest, if you don't know any Arabic, you wouldn't know that text is joined up, so the point is rather irrelevant | 10:27.24 |
| | Seems hte pattern is the background to the actual ticket | 10:34.11 |
| | I bet that's a single imaghe which is bein applied aas a pattern (stupidly) in the original PDF. | 10:34.30 |
| | Possibly there's an optimisation in poppler which is spotting the single usage in the pattern and simply emitting the image (suitably cropped and scaled) | 10:35.02 |
| chrisl | fm__: can try try this, please: http://www.ghostscript.com/~chrisl/printout-018.ps | 10:35.46 |
| | kens: PDF produced by cairo, so they might be detecting their own crapness :-( | 10:36.46 |
| kens | Lol | 10:36.53 |
| | its seems to be not that simple | 10:38.06 |
| | The pattern seems to use a fomr whihch has transparency (probably not required, as usual for Cairo) | 10:38.40 |
| fm__ | chrisl, that prints in 15s or so, have to remeasure | 10:38.57 |
| | yeah! | 10:39.01 |
| kens | So it sht use of the pattern | 10:39.13 |
| | WHich is caused by the fact that the pattern uses a form which specifies transparency | 10:39.25 |
| | Bascially, stupid PDF file | 10:39.30 |
| | Yes, the rectangle is filled with a pattern, the pattern uses a form, the form uses an image | 10:40.21 |
| | Standard Cairo crazy PDF file | 10:40.32 |
| | NB the image is interpolated too :-) | 10:40.42 |
| chrisl | Does ps2write write the interpolated samples? | 10:41.59 |
| kens | I don't know. I suspect that (because of the transparency in the form) the image is rendered. So my guess would be yes | 10:42.42 |
| sebras | tor8: any opinions on sebras/master | 10:43.28 |
| | ? | 10:43.31 |
| kens | chrisl the poppler output isn't a faithful representation of the input. I wonder what woudl happen if the form actually *used* transparency :-) | 10:44.00 |
| | I can't really see any way to optimise this I have to say. | 10:44.50 |
| chrisl | kens: poppler can spot "real" transparency, and drop non-effective transparency | 10:45.13 |
| kens | chrisl I expect that is what it is doing then. | 10:45.32 |
| | I guess it must also be undoing the form as well, and presumably spotting the pattern use. | 10:45.52 |
| | If it can do this, why does it emit such crap PDF files ? | 10:46.03 |
| | Sorry, why does Cairo emit such crap PDF files | 10:46.14 |
| chrisl | Because they can't be bothered to fix it | 10:46.26 |
| kens | Well I'm not sure thre's anything we can realistically do about this | 10:46.49 |
| | In order to spot this kind of stupidity we owuld have to slow down our own processing. | 10:47.10 |
| | And I'm not sure we could reliably spot this anyway | 10:47.21 |
| chrisl | It come out of the group clist thing Ray's going to look into | 10:48.00 |
| kens | Hmm, that's possible | 10:48.09 |
| chrisl | s/it/it may | 10:48.10 |
| kens | At least as far as the trnasparency usage goes | 10:48.25 |
| | But he form and pattern usage won't | 10:48.34 |
| | pdfwrite doesn't have the luxury of analysing the PDF file for this kind of inanity before it starts :-( | 10:49.13 |
| chrisl | fm__: I'm expecting this to be slow again, but can you try this: http://www.ghostscript.com/~chrisl/printout-019.ps | 10:53.32 |
| fm__ | chrisl, fast as hell ;) | 10:54.39 |
| | BUT BROKEN | 10:55.00 |
| chrisl | Broken? How | 10:55.10 |
| fm__ | i get %!PS-Adobe-3.0 ... | 10:55.17 |
| chrisl | Sorry, forgot the printer specific crap - I hate this :-( | 10:55.35 |
| | fm__: can you redownload please, it should work this time | 10:57.35 |
| fm__ | chrisl, broken | 11:00.13 |
| | OFFENDING COMMAND: m | 11:00.22 |
| | this time with a 180 | 11:00.26 |
| chrisl | I'm getting confused, I wonder if I edited the wrong file - hang on.... | 11:01.08 |
| | fm__: apologies, I'm not multi-tasking very well today.... http://www.ghostscript.com/~chrisl/printout-020.ps | 11:07.29 |
| fm__ | chrisl, 20 is reasonable fast as well | 11:08.40 |
| | 12s i guess | 11:08.45 |
| chrisl | And looks okay? | 11:09.00 |
| fm__ | first sight yes, let me do it more thourougly | 11:09.34 |
| chrisl | fm__: the places you might see differences are in the backdrop of the box under "ICE Fahrkarte", and the one at the bottom of the first page, and then the six "icons" for the paragraphs on the second page | 11:12.07 |
| fm__ | chrisl, i cannot see any difference | 11:13.21 |
| chrisl | fm__: I would have been surprised if you could, but worth checking anyway! | 11:13.44 |
| | I think this is probably something we should address *outside* of Ghostscript | 11:14.20 |
| fm__ | ?? | 11:14.53 |
| chrisl | All the images in the original PDF request interpolation, and the GS Postscript output faithfully reproduces that, which is what we want. In the case of your printer, however, it seems interpolation is unacceptably slow. | 11:16.09 |
| kens | :-( | 11:16.42 |
| Robin_Watts | chrisl: Is it feasible to ensure that the resolution we output from gs is correct for the printer? | 11:17.03 |
| kens | No | 11:17.14 |
| | Image data is not generally altered | 11:17.30 |
| | We could do that for rendered transparency, but that wouldn't have interpolation set anyway | 11:17.57 |
| Robin_Watts | and it would be painful to add a flag to say "do any interpolation within gs so the output file can always have interpolation turned off" ? | 11:18.13 |
| | chrisl: as a matter of interest, are we interpolating up or down here? | 11:18.35 |
| kens | yes very painful we'd have to interpolate ourselves to an unknown (bogus) resolution | 11:18.51 |
| chrisl | I've no idea - *we* are not interpolating at all | 11:18.51 |
| Robin_Watts | no, but the printer is. | 11:19.04 |
| chrisl | So this is the printer's problem, not ours | 11:19.29 |
| kens | I'm very much against us doing the interpolation | 11:20.10 |
| | I'd be happier to add a flag that said 'turn off interpolation if its on' | 11:20.26 |
| tor8 | Robin_Watts: tor/master a few patches with simple fixes for some of sumatra's font loading issues | 11:20.47 |
| chrisl | We have that - pdfwrite/pswrite could honour the -dNOINTERPOLATE flag | 11:21.00 |
| Robin_Watts | tor8: Will look in a mo. | 11:21.10 |
| kens | chrisl that would make some sense | 11:21.19 |
| | except that we might not then use it for rendering when we should (or vice versa) | 11:21.46 |
| Robin_Watts | Certainly having pdfwrite honor -dNOINTERPOLATE seems to be a sensible thing. | 11:22.01 |
| kens | Considering a transparent region with an interpolated image in it | 11:22.06 |
| | ps2write would render this | 11:22.17 |
| | If we set -DNOINTERPOLATE it wil be rendered 'wrong' | 11:22.28 |
| | Am I making sense ? | 11:23.36 |
| Robin_Watts | It would be rendered consistently with ps2writing the same image in a non-transparent area with -dNOINTERPOLATE and then printing it. | 11:24.01 |
| chrisl | It could be argued that since our interpolation won't match the target printer's, we're rendering it "wrong" now...... | 11:24.10 |
| kens | I think there could be problems, I'd rather see a separate flag | 11:24.32 |
| Robin_Watts | Setting -dNOINTERPOLATE should have the same effect on images in both transparent and non-transparent regions. | 11:24.50 |
| kens | currently it does not | 11:26.03 |
| Robin_Watts | Suppose I have a file which has 2 copies of an image in it. Both request interpolation, but one is in a transparent region and one is not. | 11:26.44 |
| chrisl | FWIW, it's pretty trivial to give tkamppeter a little PS snippet which cups can prepend to the prolog for this (and maybe other) printers, which will disable interpolating images. | 11:27.30 |
| Robin_Watts | At the moment, if I pdfwrite that with -dNOINTERPOLATE, then the transparent one will appear uninterpolated in the output, and the non transparent one will appear interpolated, right ? | 11:27.31 |
| kens | chrisl that makes more sense to me | 11:27.46 |
| | Robin_Watts : I am not certain | 11:27.58 |
| | I would have to check | 11:28.02 |
| | But I believe that it would depdend ont the printer you send it to dfor starters | 11:28.17 |
| Robin_Watts | kens: Fair enough. IF that *was* the case though, then I reckon it's wrong. | 11:28.26 |
| | and making pdfwrite honour -dNOINTERPOLATE would correct that. | 11:28.36 |
| kens | I'm not happy about overloading the flag | 11:28.54 |
| Robin_Watts | kens: assuming the printer correctly honors interpolate. | 11:29.10 |
| | I'd claim it's not an overloading. | 11:29.18 |
| chrisl | fm__: sorry, (hopefully just) one more file to try: http://www.ghostscript.com/~chrisl/printout-021.ps | 11:29.21 |
| kens | But as cgrisl poitns out if we do interpolation ourselves, and the final printer does noen (or differnet interpolation) the two results will be different anayway | 11:29.22 |
| Robin_Watts | kens: right, implementations of interpolation do not have to match. | 11:29.45 |
| kens | I htink we should have 2 flags, one affects rendering one affects high level devices. | 11:30.13 |
| Robin_Watts | And I disagree. | 11:30.35 |
| kens | Then we are at an impasse. | 11:30.45 |
| Robin_Watts | Indeed, and it's your ball, so you win, but it won't stop me trying to make a case :) | 11:31.04 |
| kens | I also belive it makes sense to be able to strip interpolation from images when (for example) we downsample them | 11:31.07 |
| chrisl | And I think pdfwrite and ps2write should *always* endeavour to reproduce the input as faithfully as possible..... | 11:31.22 |
| kens | Or possibly add interpolation fro the same case | 11:31.24 |
| Robin_Watts | interpolation is more important on downsampled images than upsampled ones. | 11:31.25 |
| | lack of interpolation on downsampled images gives dropouts. | 11:32.03 |
| kens | So I woudl add a flag which affects interpolation as 'Preserve' 'Add pr 'remove' | 11:32.08 |
| Robin_Watts | lack of interpolation on upsampled images merely gives jaggies. | 11:32.35 |
| kens | Re-using flags just gets us into trouble, if we want to do this then I will add a new flag | 11:33.28 |
| chrisl | The poppler produced PS does not preserve the interpolate flag, nor does it seem to interpolate during the conversion..... | 11:34.02 |
| kens | Which is the worst case I would say | 11:34.15 |
| chrisl | That would depend on the printer ;-) | 11:34.33 |
| kens | A printer which handled both PF and PS, and did interpolation (likely in this case, it must be a high-end printer) woudl give differnt results with the 2 inputs | 11:35.27 |
| fm__ | chrisl, 21 looks good to my eyes and prints fast | 11:36.43 |
| chrisl | kens: True, but I think 12 seconds to 6+ minutes without and with interpolation is pretty broken, anyway! | 11:37.06 |
| | fm__: great, thanks. I'll update the Ubuntu bug with what tkamppeter will need to know to resolve this in the cups workflow | 11:37.35 |
| kens | we've had similar results with GS in the past | 11:37.49 |
| chrisl | I know, and I don't think that's acceptable, either...... | 11:38.03 |
| fm__ | tkamppeter, your train ticket is quite popular by now ... ;) | 11:38.06 |
| tkamppeter | chrisl, note that I let GSA always use -dNOINTERPOLATE in the whole printing workflow, also in the pdftops filter. | 11:40.53 |
| chrisl | tkamppeter: that doesn't affect ps2write - it still preserves the flag | 11:41.16 |
| kens | lunches | 11:42.34 |
| chrisl | tkamppeter: when you have a moment, can you review my comment on the bug (https://bugs.launchpad.net/bugs/1026974), just to make sure you're clear with what I'm suggesting, please? | 11:45.25 |
| | I think I will also go for some lunch now..... | 11:46.00 |
| Robin_Watts | Early morning Michael. | 11:53.53 |
| tkamppeter | chrisl, that's OK, I will add it to the filter´s workaround code. So all what is needed to fix this bug is to add this workaroud code? | 11:55.33 |
| chrisl | tkamppeter: yes, that addresses the performance problem | 12:09.17 |
| fm__ | tkamppeter, where do i have to put it, to test it? | 12:19.35 |
| kens | chrisl the missing '%%BeginResource' comment turned out to be more difficult to track down than I expected, but I think I've got it now. | 13:19.12 |
| chrisl | kens: That's kind of annoying for something that's probably never used! | 13:21.01 |
| kens | Its hte same problem that is causing me grief with linearisation, the mad way that pdfwrite emits fonts. | 13:21.38 |
| chrisl | mentally substitutes the words "emits fonts" with the words "does everything"! | 13:23.02 |
| fm__ | chrisl, i have the following in my printer handbook "Emulations PCL6 (5c, XL), KPDL 3 (PostScript 3 compatible) with auto- matic emulation switching, Line Printer, IBM Proprinter X24E, Epson LQ-850, Diablo 630, PDF – 1.5, XPS Printing" | 13:31.50 |
| chrisl | fm__: so it looks like it does support direct PDF printing, but there may be restrictions on it, and I don't know whether cups can send PDF to the printer | 13:34.42 |
| fm__ | at least nc < .pdf does not work ;) | 13:35.17 |
| kens | Pity it would be interesting to see its speed | 13:35.48 |
| fm__ | i guess it would need some preamble as well, how should the printer know if it should do duplex, which tray to use, etc | 13:36.33 |
| chrisl | It almost certainly doesn't support PDF transparency (despite 1.5 claim) - it might have interesting output with the cairo PDF..... | 13:37.31 |
| kens | job ticket woudl give duplex and stuff | 13:38.01 |
| | But I suspect the direct PDF printing is (as usual) a s light exaggeration | 13:38.23 |
| chrisl | fm__: long shot, but you could try this: http://www.ghostscript.com/~chrisl/printout-001.pdf | 13:40.17 |
| fm__ | chrisl, close | 13:44.43 |
| | it prints after three minutes. | 13:44.54 |
| | the first page has a massive font problem, while the second is perfect | 13:45.08 |
| chrisl | Well, I suggest you report this one to Kyocera! | 13:46.16 |
| fm__ | i guess you have something to sell them ;) | 13:48.46 |
| Robin_Watts | tor8: ping ? | 14:20.06 |
| tor8 | Robin_Watts: yes? | 14:20.17 |
| Robin_Watts | Just looking at your patches. | 14:20.25 |
| | Weren't you supposed to be afk today ? | 14:20.38 |
| tor8 | Robin_Watts: that was y'day :) | 14:20.49 |
| Robin_Watts | Ah, ok. | 14:20.54 |
| | The "Attempt to load CID fonts with invalid cid collections" one. | 14:21.07 |
| | You've ended up with 2 calls to pdf_load_substitute_font when you could easily have one. | 14:22.32 |
| tor8 | with a goto? | 14:22.44 |
| Robin_Watts | It's a trivial minor point - otherwise that one looks fine. | 14:22.45 |
| tor8 | or am I just being stupid? | 14:22.54 |
| Robin_Watts | No, just return after some of the other places. | 14:23.12 |
| | pdf_load_substitute_cjk_font(...); return; | 14:23.36 |
| | so you fall out the bottom of the if | 14:23.58 |
| tor8 | right. 4 returns vs a duplicate load_substitute_font. I didn't like either of the choices | 14:24.01 |
| Robin_Watts | yeah, I was just looking at the diff; looking at the full code, it's less attractive. | 14:24.53 |
| | Ignore me then :) | 14:24.59 |
| | The next patch looks sensible. | 14:28.31 |
| | The final one... There is an fz_strlcpy that has been removed from the code with no obvious replacement. | 14:28.58 |
| | Is that deliberate? | 14:29.02 |
| tor8 | Robin_Watts: hm, that diff hunk should be moved to 869b4a7 | 14:29.38 |
| Robin_Watts | ok, I'd not understand it there either :) | 14:31.03 |
| tor8 | it's just copying the fontdesc name into the fz_font struct | 14:31.20 |
| Robin_Watts | except now it doesn't ? | 14:31.39 |
| tor8 | the other patch passes the fontdesc in when creating the fz_font, so it's always set and not just for the fontdesc case | 14:31.44 |
| | s/fontdesc/font name/1 | 14:31.54 |
| Robin_Watts | Oh, so that line was "overriding" an existing font name with a better one ? | 14:32.52 |
| tor8 | yeah. | 14:33.21 |
| Robin_Watts | I can see that 869b4a7 passes the fontname though to more places. | 14:33.32 |
| tor8 | the font name creation in fz_font has always been a bit messy | 14:33.34 |
| | since it wasn't really used anywhere except for debugging | 14:33.47 |
| Robin_Watts | so, that means it no longer needs the overriding ? | 14:33.50 |
| tor8 | but now the text device uses it | 14:33.52 |
| | correct. | 14:33.57 |
| Robin_Watts | Gotcha. | 14:34.00 |
| | OK. Review passed for all 3 then. | 14:34.19 |
| | I'll let you push 'em in case you want to move that line. | 14:34.28 |
| tor8 | the same 'fontname' is passed in to pdf_load_*_font now instead | 14:34.29 |
| | do another remote update, there's one I pushed just a minute before you pinged | 14:34.47 |
| Robin_Watts | I'm looking at the web view. | 14:35.13 |
| | and I still only see 3. | 14:35.19 |
| | Oh. "Give precedence to /Subtype field?" | 14:35.39 |
| tor8 | no, 'Prefer the more specific one...' | 14:36.02 |
| | well, the top 4 on the web view anyway | 14:36.16 |
| Robin_Watts | Yeah, that's the one with the extra line in. | 14:36.19 |
| | I'd missed the 4th one. | 14:36.30 |
| tor8 | oh, right | 14:36.36 |
| | with these we get a few progressions, and some files that had problems still have problems, but different ones | 14:37.03 |
| Robin_Watts | The 4th one could be if (!strcmp(...)) ... *else* if (!strcmp) ... | 14:37.17 |
| | Would avoid some needless repeated function calls. | 14:37.34 |
| | could even do: subtype = pdf_to_name(pdf_dict_gets(dict, ...)) | 14:38.31 |
| | then if (!strcmp(subtype, ...)) ... else if (!strcmp(subtype, ...)) ... | 14:38.48 |
| | but sure, consider that passed too. | 14:39.40 |
| tor8 | right. that's an easy change. | 14:39.52 |
| Robin_Watts | Only 10 patches from sebras today. Honestly, the boy's slacking. | 14:40.29 |
| tor8 | Robin_Watts: I checked them earlier. a few english fixes needed in one of them, but otherwise they look fine to me. | 14:41.22 |
| Robin_Watts | ah, saved me looking then, fab. | 14:41.34 |
| sebras | Robin_Watts: :) | 14:43.36 |
| | Robin_Watts: I only chopped up my gargantuan patch in to easily reviewable pieces. tends to keep maintainers happy. | 14:44.28 |
| Robin_Watts | We are bears of small brain. | 14:45.33 |
| henrys | I live about 30 miles from the shooting, another dark day for colorado feels like Columbine again. | 15:28.53 |
| Robin_Watts | henrys: Ah, heard about that on the radio this morning. | 15:31.20 |
| henrys | I shouldn't be such a nit picker but assignments in conditions always bother me. I always think the person intended comparison, if it doesn't bother others I'll shutup. | 16:02.07 |
| Robin_Watts | henrys: Generally, I reckon it should be avoided. But sometimes there are times in C when it can be worth it. | 16:11.03 |
| | I try, in such instances to make it clear. instead of doing if (a = b), I try to do: if ((a = b) != 0) | 16:11.39 |
| | or if (!!(a = b)) | 16:11.53 |
| | henrys: Whose patch are you looking at? (sods law says it will be one of mine) | 16:12.34 |
| henrys | ray's patch | 16:13.30 |
| Robin_Watts | ah, because I have just been guilty of that too :) | 16:14.02 |
| | Yeah, with rays one, I'd definitely prefer the assignment then the if. There is nothing but confusion to be gained by rolling the two together, IMHO. | 16:15.37 |
| henrys | hopefully he'll see the logs otherwise I'll email him. | 16:16.57 |
| Robin_Watts | I have a patch on my casper ghostscript repo that I'd like ray's opinion on (and anyone else that's interested). | 16:17.35 |
| chrisl | henrys: Shelly noticed the assignment instead of comparison in Ray's commit - he said he'd e-mail Ray about it | 16:17.36 |
| Robin_Watts | chrisl: It's not instead of, it's 'as well as'. | 16:18.19 |
| | (AIUI) | 16:18.27 |
| henrys | the code is correct - just confusing. | 16:18.32 |
| ray_laptop | morning, all | 16:26.01 |
| marcosw | morning ray_laptop | 16:26.07 |
| Robin_Watts | Morning ray_laptop | 16:26.12 |
| | ray_laptop: I've got a patch up on my casper repo for the colors_used stuff. | 16:26.46 |
| ray_laptop | chrisl: I saw a comment earlier about "Ray's group clist thing". What is that and how do you expect it to help Cairo output performance ? | 16:27.05 |
| Robin_Watts | Could I ask you to cast your eyes over it please? | 16:27.07 |
| ray_laptop | Robin_Watts: I shall. Thanks | 16:27.14 |
| henrys | hi ray_laptop see above about your last commit. | 16:27.38 |
| ray_laptop | henrys: OK. | 16:27.49 |
| chrisl | ray_laptop: it was about detecting whether transparency is "real" or has no effect | 16:27.56 |
| ray_laptop | I'm down to 11:16 UTC in the logs | 16:28.06 |
| chrisl | ray_laptop: it was just a "by the by" comment, not a suggestion of a solution..... | 16:28.33 |
| ray_laptop | chrisl: I see -- well it only works w.r.t. the page level and then only causes bands to be rendered without transparency. Would this help ? | 16:29.00 |
| chrisl | ray_laptop: no I was meaning the long term project of using a clist so pdfwrite can flatten transparency without running out of memory | 16:29.46 |
| ray_laptop | kens: chrisl: what's the bug that has the file ? | 16:30.00 |
| chrisl | ray_laptop: there isn't one, really, it was a theoritical point | 16:30.33 |
| ray_laptop | chrisl: oh -- flattening. But that would _still_ run transparency mode through the clist and generate a flattened image. | 16:30.47 |
| marcosw | henrys: I don't think we need a support meeting this week, unless I've missed something. | 16:30.56 |
| henrys | marcosw:right I think we're good also. | 16:31.14 |
| ray_laptop | it wouldn't be able to detect where "real transparency" is used | 16:31.22 |
| | henrys: I see your comment about "assignment in conditions". I agree, I should have used if ((ncdev->is_planar = cdev->is_planar) != 0) would that be better ? | 16:33.46 |
| henrys | yes that would be fine. | 16:34.05 |
| ray_laptop | Robin_Watts: which is what you suggested and what I usually have done. Frankly, I don't even like if (!pointer) return_error vs. if (pointer == NULL) | 16:34.55 |
| Robin_Watts | It would be better. | 16:34.56 |
| | It wouldn't be "fine", IMHO. | 16:35.06 |
| | I'd still rather see the assignment done before the test. | 16:35.21 |
| | I also dislike if (!pointer) | 16:35.36 |
| ray_laptop | OK. I'll go ahead with that, but only because you didn't flame me ;-) | 16:35.50 |
| Robin_Watts | There are times when doing assignments in expressions is useful (like loops, or in chained expressions), but unless you are getting some benefit from it, it's best avoided, IMAO. | 16:37.39 |
| ray_laptop | well it save an extra line in the source ;-) That used to matter when we had 80x24 screens | 16:39.47 |
| henrys | bbiab | 16:40.45 |
| ray_laptop | I do tend to use if ((code = somefunction(...)) < 0) return_error code | 16:40.47 |
| | with the 'return_error' indented on the next line, of course | 16:41.12 |
| Robin_Watts | The time saved by not having to scroll the screen is massively outweighed by the extra time required to stare at the code when debugging. | 16:41.54 |
| | ray_laptop: That's a slightly different case though, and a style that's prevalent throughout gs. | 16:42.12 |
| ray_laptop | Robin_Watts: that's where I picked up the bad habit of assignments in if clauses :-) | 16:43.29 |
| | oh, great. Now I get a hang in NumRenderringThreads=1 | 16:46.41 |
| Robin_Watts | And I'm getting a SEGV in a banded psdcmyk clist pattern. it's like all the worst bits of gs ganging up at once. | 16:52.49 |
| | +transparent | 16:53.16 |
| ray_laptop | whew... it was some prelim patches on top of a fix -- I hadn't done the git reset --hard origin to get back to a clean version | 16:53.30 |
| | Robin_Watts: that _IS_ an impressive stack of modes | 16:58.31 |
| | Robin_Watts: the only thing missing is image interpolation ;-) | 16:58.55 |
| Robin_Watts | Ah, this SEGV is independent of the number of color components though, so I may ignore it for noe. | 16:59.47 |
| | now | 16:59.49 |
| ray_laptop | Robin_Watts: well, based on my recent fix, make sure that the pattern_clist is in planar mode | 17:00.44 |
| | does it SEGV when tiling the pattern (clist playback) ? | 17:01.27 |
| Robin_Watts | cdev->is_planar = 1 | 17:02.17 |
| | Yes. | 17:02.20 |
| | It's when it does the pattern load during clist playback | 17:02.33 |
| | Hmm. The clist color info says it's 1 component 8 bit color. | 17:03.15 |
| | The pattern cache is set up for 24 bit color. | 17:03.52 |
| kens | OK goodnight all, running away now.... | 17:04.12 |
| ray_laptop | bye, kens | 17:04.17 |
| | Robin_Watts: that sounds like a problem | 17:04.34 |
| | Robin_Watts: which bug are you digging into ? | 17:07.14 |
| Robin_Watts | bug 693185 | 17:07.46 |
| ray_laptop | guesses 693144 might be related | 17:07.59 |
| Robin_Watts | Alex says he gets lots of SEGVs when we push max components to 32. I've solved that, but I still get some differences. | 17:08.23 |
| | (but generally small ones). | 17:08.31 |
| ray_laptop | Robin_Watts: yes, but you said you are only seeing a RGB pattern | 17:09.04 |
| Robin_Watts | Yes, I have 4 SEGVs left, and it turns out that this one was here anyway. | 17:09.37 |
| ray_laptop | Robin_Watts: so it's hard to see why 32 components would be relevant | 17:09.38 |
| Robin_Watts | hence this is independent of the my changes. This is just "a random SEGV on the trunk" | 17:10.04 |
| | 693144 is in fact spot on, thanks. | 17:11.40 |
| ray_laptop | Robin_Watts: OK. So it *might* be related to bug 693144 (we have to get lucky sometimes) | 17:11.41 |
| Robin_Watts | It's that very file. I should have checked bugzilla :( | 17:11.58 |
| ray_laptop | oops | 17:12.06 |
| Robin_Watts | so I shall move on. Let's try for 64 components in a cluster test. | 17:12.32 |
| mvrhel | 693144 is due to the issue that I mentioned in comment 1 | 17:25.02 |
| | or comment 0 I guess | 17:25.13 |
| Robin_Watts | mvrhel: Yes, it was my bad for not finding the bug. | 17:25.23 |
| | mvrhel, ray_laptop: If you look at http://ghostscript.com/~regression/robin/ those are the differences I get with pushing the max components up to 32. | 17:26.15 |
| ray_laptop | mvrhel: so a softmask is _supposed_ to be 8 bit monochrome, right ? Robin is seeing the pattern cache as 24-bit | 17:28.07 |
| Robin_Watts | ray_laptop: I think we should leave it for now. | 17:28.33 |
| | It's not a blocker, and mvrhel has enough on his plate. | 17:28.49 |
| | And it's not in the way of me moving onwards with the max components stuff. | 17:29.06 |
| mvrhel | ray_laptop yes that is what bug 693144 is all about | 17:29.15 |
| ray_laptop | Robin_Watts: OK. It sounds rare enough | 17:29.29 |
| Robin_Watts | I'm guessing most of the stuff in the bmpcmp can be ignored (single pixel differences), but stuff like 84 probably needs investigating. | 17:30.20 |
| | and number 1. | 17:31.18 |
| | I'm hoping we can ignore things like 3 and 4 as being due to diffences in bandheight or something. | 17:31.40 |
| | 24/25/26 are odd though - I wondered if you had any thoughts ? | 17:32.36 |
| ray_laptop | bbiab | 17:35.55 |
| mvrhel | 27 I can actually see a line | 17:38.58 |
| | how could bumping up to 32 cause that | 17:39.19 |
| Robin_Watts | indeed. | 17:39.25 |
| | I'll look into it on monday. I thought I'd mention it in case either you or ray (or anyone else) could instantly say "Ah, that might be the transmogrifier going wrong" | 17:40.17 |
| mvrhel | wish I had some advice on that | 17:40.31 |
| | I am certain it is something that I did in the planer/sep commit | 17:41.01 |
| | that commit has had quite a bug tail | 17:41.22 |
| | I am going to take another look at 693144 now | 17:42.43 |
| | ok. so the segv is only ocurring when the pattern is a clist | 17:46.21 |
| ray_laptop | ok, back | 17:48.34 |
| | yeah, like Robin_Watts said -- pretty much all of the hot button processing issues (maybe except ICC profiles, image interp and spot colors) | 17:49.34 |
| mvrhel | well there was a lot of code in that commit so there was plenty of opportunity for there to be issues | 17:50.34 |
| | ok looks like the push of the pdf14 device for the pattern is not getting properly set up | 18:21.10 |
| | ugh. ok so we have a soft mask, which has a pattern. so we push the pdf14 gray device for the pattern, but then there is an RGB group in the softmask | 18:33.03 |
| ray_laptop | I see the issue with NumRenderingThreads and psdcmyk (it isn't planar, it's the way we call 'get_bits' multiple times to get each plane | 18:44.47 |
| mvrhel | that is done that way since get_bits can't return all the planes at once | 18:45.30 |
| ray_laptop | which currently still renders ALL planes, then throws away data. mvrhel get_bits _does_ return all planes at once | 18:45.56 |
| mvrhel | I dont think so. Not if you want planar output | 18:46.19 |
| | I had a talk with Robin_Watts about this yesterday | 18:46.32 |
| | since I would like to have such an option for the overprint device | 18:46.45 |
| Robin_Watts | You can do a get bits for a single plane, I think. | 18:47.11 |
| mvrhel | are you still here Robin_Watts ? | 18:47.12 |
| | right | 18:47.34 |
| | that is the only way to get planar parts | 18:47.43 |
| | the way it is currently implemented | 18:47.58 |
| | iirc | 18:48.00 |
| Robin_Watts | ray_laptop: If you do a get bits on a planar device, then it can efficiently return a single plane. | 18:48.20 |
| | (I believe) | 18:48.24 |
| | it'll render all of them, and just return one. but nothing is 'thrown away'. | 18:48.55 |
| ray_laptop | Robin_Watts: mem_planar_fill_rectangle_hl_color still loops through all of the planes | 18:49.04 |
| mvrhel | because that is the only way to get the individual planes | 18:49.22 |
| ray_laptop | Robin_Watts: what I said -- it renders ALL of the planes, then only uses one plane's worth of data | 18:49.46 |
| Robin_Watts | Right, but exactly how is stuff 'thrown away' ? | 18:49.59 |
| ray_laptop | well, it is never used | 18:50.08 |
| Robin_Watts | It's used on the next call. | 18:50.16 |
| ray_laptop | the next call is for the next band OF THE SAME PLANE. it only uses the data from the next plane (colorant) after the entire page has been rendered, and by then the data no longer there | 18:51.21 |
| Robin_Watts | Ah, yes, that's true. | 18:51.45 |
| ray_laptop | Robin_Watts: note that it also goes through the 'downscaler' for all of the planes. I guess the issue is that psd_write_image_data needs to use all colorants of data at once (chan_idx loop INSIDE the 'j' loop) | 18:54.33 |
| Robin_Watts | I'm just checking into getbits to see if there is a way to get multiple planes returned at once. | 18:55.13 |
| mvrhel | that would be very helpful in several spots | 18:55.28 |
| marcosw | Robin_Watts: BTW, a box for you arrived at my house yesterday. I had a bit of a panic before I realized it was for you, Adorama boxes showing up at my house is a bad thing as far as the wife is concerned. | 18:56.04 |
| ray_laptop | the tiffsep device looks smarter -- it has the comp_num loop inside the 'y' loop | 18:56.09 |
| Robin_Watts | marcosw: Ah. That'll be the little lens. | 18:56.27 |
| marcosw | I thought it was a bit light for a camera body. | 18:56.40 |
| Robin_Watts | I reordered the camera + lens last week after amazon cancelled my preorder due to canon doing a recall. | 18:56.53 |
| | s/last week/this morning/ | 18:57.09 |
| | they cancelled last week. | 18:57.14 |
| | They don't have a date for stock yet, so you have lots of time to forget again :) | 18:57.34 |
| mvrhel | ray_laptop: the psd data is packed in planar form in the file | 18:57.36 |
| ray_laptop | Robin_Watts: there is GB_SELECT_PLANES in the 'options' | 18:57.38 |
| mvrhel | ray_laptop: one issue that I saw when trying to get get_bits to give me planar is that it ended up doing planar to chunky and then back to planar iirc | 18:58.35 |
| Robin_Watts | yes, I'm looking at the code here. | 18:58.49 |
| | I think if you try to get anything other than a single plane it returns chunky data. | 18:59.03 |
| mvrhel | yes | 18:59.07 |
| | exactly | 18:59.09 |
| | and so it was doing two conversions | 18:59.19 |
| ray_laptop | but psdcmyk only wants a single plane | 18:59.40 |
| marcosw | do you want me to open it to check it's okay? The box doesn't look beat up but UPS has been known to throw things around a bit :-) | 18:59.41 |
| Robin_Watts | marcosw: Feel free. | 18:59.53 |
| | I am tempted by a battery grip as well, but some small percentage of them arrive DOA, and returning it would be a hassle. | 19:00.59 |
| | ray_laptop: So, AIUI, the psdcmyk device should run through, fetching all the data for plane 0, and outputting. | 19:02.15 |
| ray_laptop | Robin_Watts: although it looks arbitrary that clist_render_rectangle only allows a single plane to be returned -- it seems like it _could_ return multiple planes by looping internally (for all planes where params->data[i] != 0) | 19:02.57 |
| Robin_Watts | So that will render every band on the page, and only use 1 plane. Then it will run through and get the data for plane 1, again rendering everything. | 19:02.59 |
| | ray_laptop: Right, but 1) that's not in the interface, and 2) we'd be limited to 16 planes. | 19:03.31 |
| ray_laptop | Robin_Watts: or psdcmyk should invert the loops like tiffsep does | 19:03.31 |
| Robin_Watts | ray_laptop: Can't. | 19:03.38 |
| ray_laptop | Robin_Watts: why can't psdcmyk invert the loops ? | 19:03.58 |
| Robin_Watts | The attraction of psdcmyk from the clusters perspective is that we write the file with no seeking. | 19:04.12 |
| | i.e. we can output to a pipe. | 19:04.28 |
| | This is why we can't test tif under the cluster. | 19:04.47 |
| | The data in a psdcmyk file comes out plane by plane, not interleaved. | 19:05.06 |
| ray_laptop | well, as long as it used (one or more) temp files, it could do the final output in the proper sequential order | 19:05.07 |
| Robin_Watts | Right, but tempfiles would be nasty. | 19:05.22 |
| ray_laptop | note that the pdfwrite and ps2write device use internal tempfiles | 19:05.31 |
| | for much the same reason -- they need to reorder the output | 19:05.49 |
| Robin_Watts | I inverted the loops in psdcmyk a while ago and then wondered why nothing worked :) | 19:06.02 |
| mvrhel | I had tried the same thing when I worked on it | 19:06.17 |
| Robin_Watts | ray_laptop: Why is this a problem? Other than the inefficiency? | 19:06.36 |
| mvrhel | temp files would work but does seem icky | 19:06.38 |
| ray_laptop | Robin_Watts: tempfiles are not as nasty as rendering/downscaling 4 times as often (IMHO) | 19:06.39 |
| mvrhel | well we don't really use that now that AA is working | 19:06.56 |
| | oh | 19:07.03 |
| | sorry | 19:07.05 |
| | misunderstood your point | 19:07.17 |
| Robin_Watts | Does something in num_rendering threads rely on the page only being drawn once ? | 19:07.27 |
| mvrhel | yes or more than 4 if we have CMYK+ spots | 19:07.29 |
| ray_laptop | mvrhel: I was talking about playing back the clist 4 times | 19:07.40 |
| mvrhel | ray_laptop: yes I understand | 19:07.50 |
| Robin_Watts | Well, tempfiles are a possibility, but I'd like to understand if this is a performance optimisation, or a correctness optimisation? | 19:08.03 |
| ray_laptop | Robin_Watts: it has a bug in it (that I am fixing) | 19:08.04 |
| | Robin_Watts: performance only, although it would have masked my bug :-) | 19:08.31 |
| Robin_Watts | OK, so open a performance enhancement to make psdcmyk work using temp files? | 19:08.33 |
| ray_laptop | with modern filesystems and disks, is one tempfile that we seek around in and write still worse than N tempfiles ? | 19:09.31 |
| Robin_Watts | I'd really hope not. | 19:09.55 |
| ray_laptop | we would want to write an entire band, however, not just one line at a time | 19:10.24 |
| Robin_Watts | We'd want to read a single line at a time from each plane, and write that out to the temp file. | 19:10.53 |
| | So the tempfile contained all the data interleaved. | 19:11.01 |
| | Then we'd read through the file n times when writing out. | 19:11.28 |
| ray_laptop | you think that would be better than writing the band in situ so that the we just read once | 19:12.03 |
| Robin_Watts | I think writing a single block is better than jumping through the file writing in several places. | 19:12.40 |
| | Some SSDs/Flash drives do compression etc, and I'm not sure that OSs don't assume a single 'leading edge' of a file for buffering as we write. | 19:13.21 |
| | Seeking may cause a sync. | 19:13.28 |
| mvrhel | what about doing separate temp files for each plane | 19:13.53 |
| Robin_Watts | 64 temp files? No thanks. | 19:14.06 |
| mvrhel | ok | 19:14.12 |
| | fair enough | 19:14.14 |
| ray_laptop | I guess it depends on the random access cache effectiveness when writing vs. to overhead of moving 4 times as much data | 19:14.16 |
| mvrhel | actually psd is limited to 56 | 19:14.28 |
| | which is something we probably don't currently check against | 19:14.53 |
| ray_laptop | mvrhel: thinking about it, 64 tempfiles is no different than doing the seeks -- to update the file the drive will still seek | 19:15.03 |
| mvrhel | but that is not a per/line seek | 19:15.38 |
| ray_laptop | mvrhel: I was presuming a per-band seek | 19:15.51 |
| mvrhel | oh I see what you are saying | 19:16.26 |
| | doing the writing in-sitiu vs. individual files | 19:16.47 |
| | yes, the same number of seeks | 19:16.56 |
| ray_laptop | and the device _could_ establish a block size independent of the bandheight and just buffer up that number of lines in memory then do the seek/write of planes | 19:17.05 |
| mvrhel | this sounds like quit a bit of work for a device that hardly anyone uses..... | 19:17.43 |
| ray_laptop | Robin_Watts: for an SSD I agree that multiple reads might be preferable to seeking, particularly if compressed files | 19:17.59 |
| | mvrhel: well, it would speed up our regression runs :-) | 19:18.22 |
| | which would make "real" performance hits show up more easily | 19:18.43 |
| Robin_Watts | I suspect that on modern machines, the speed of data being sucked from a disc is so great that it will be dwarfed by the processing we do on it. | 19:18.51 |
| mvrhel | true | 19:18.53 |
| Robin_Watts | hence reading linearly through a file n times and only using 1/nth of it will be efficient. | 19:19.22 |
| ray_laptop | Robin_Watts: that's why I was thinking that doing N read passes to collect the planes would be a killer. | 19:19.47 |
| Robin_Watts | ray_laptop: sorry? | 19:20.21 |
| | We write: plane0line0,plane1line0,plane2line0,....plane0line1,plane1line1,.... | 19:20.51 |
| ray_laptop | Robin_Watts: compared to rendering (and possibly downscaling) N times would depend on how complex the file is to render | 19:20.52 |
| Robin_Watts | That way we are guaranteed that every getbits is efficient; first one renders, next n just get data from the rendered buffer. | 19:21.45 |
| ray_laptop | Robin_Watts: so if we write line interleaved data, we don't do any seeking when writing, but have N passes to read then write the result | 19:21.48 |
| Robin_Watts | ray_laptop: Exactly. | 19:21.56 |
| | Now, reading in bulk linear data to memory is what drives are good at. | 19:22.12 |
| | So we do a loop that just sucks the data in to a buffer, and we copy single lines out. | 19:22.40 |
| ray_laptop | Robin_Watts: but if we call get bits for a rectangle (bigger than a line) then write the block for each plane at the correct position in the tempfile, we only read once | 19:23.04 |
| | Robin_Watts: and only render and downscale once | 19:23.40 |
| Robin_Watts | We only render and downscale once anyway. | 19:23.51 |
| ray_laptop | Robin_Watts: right, if we use a tempfile. | 19:24.13 |
| Robin_Watts | indeed. | 19:24.17 |
| ray_laptop | Robin_Watts: but N passes reading is a lot of IO time | 19:24.43 |
| Robin_Watts | We're discussing the relative benefits of an interleaved tempfile, and a non-interleaved tempfile. | 19:24.47 |
| | If we can read the bandsize out in advance then you may be right. | 19:25.10 |
| ray_laptop | Robin_Watts: right | 19:25.11 |
| Robin_Watts | (Interestingly, I wonder if the downscaler is smart enough to ensure that the bandheight is a multiple of the downscaler value) | 19:25.45 |
| ray_laptop | Robin_Watts: you can get the bandheight, but also psdcmyk can allocate a temp buffer and just fill it, independent of bandheight | 19:25.53 |
| Robin_Watts | (probably not, as it doesn't need to be it just reads a line at a time) | 19:26.19 |
| ray_laptop | but if we are in page buffer mode, we'd really like to skip the tempfile | 19:26.38 |
| Robin_Watts | ray_laptop: Well, it sounds like a straightforward enough thing to do. | 19:26.54 |
| ray_laptop | right. I'll open an enhancement, make it bountiable, and get back to making NumRenderingThreads>1 work correctly | 19:27.39 |
| | it looks pretty easy to fix. Note that it could also cause freed memory to be written (depending on the number of bands vs. NumRenderingThreads) | 19:30.37 |
| | i.e. it only worked if the number of bands was an integer multiple of NumRenderingThreads -- oops. | 19:31.19 |
| mvrhel | bbiab] | 20:03.39 |
| alexcher | marcosw: You've filed bug 692881. I cannot reproduce the problem. Both command lines take about 1 second of wall time. | 21:00.30 |
| marcosw | it may have been fixed, I filled it over 4 months ago. I'll check to see if I can reproduce it with master. | 21:01.52 |
| mvrhel_laptop | aha. now I remember that sep devices have special handling of groups in the pdf14 device which is causing us confusion | 21:47.00 |
| | or me and the code confusion at least | 21:47.20 |
| | Forward 1 day (to 2012/07/21)>>> | |
IRC Logs