| <<<Back 1 day (to 2012/07/17) | 2012/07/18 |
sebras | tor8 Robin_Watts: oh and at the top of sebras/master is a patch that require math-skills and logical reasoning to be sharp in order to be reviewed. I'm not convinced that it is correct. parts of this patch makes mupdf handle ghostscript bug 693033 gracefully. | 00:21.01 |
| let me know what you think. | 00:21.23 |
Robin_Watts | sebras: I'll look tomorrow. | 00:21.53 |
sebras | Robin_Watts: yeah, of course. math skills tend not to peak at 1am (or 2am like here, hence my patch requiring review ;) ). | 00:22.48 |
marcosw | apparently ubuntu broke building 32 bit binaries in 12.04 since "in some cases the -dev packages are not coinstallable (e.g. libxml2-dev) in that case you have to change the installation on each target compile" | 03:33.09 |
henrys | marcosw:we probably should give you sudo access for apt-get on all the machines. | 03:42.18 |
marcosw | based on me screwing up fathoms just now I don't think i should have sudo access on any machine. Somehow I managed to uninstall gcc :-) | 03:43.29 |
henrys | can we just do something a simple test on one 32 bit machine for alexcher's change? | 03:45.23 |
| something like | 03:45.43 |
| I don't think this comes up often enough to spend a lot of time on, but whatevery you think. | 03:46.42 |
marcosw | well, except for the problem with fathoms it was easy. I think I spent a total of an hour or so on it. | 04:01.30 |
mvrhel | henrys: so I figured out what the issue is with 693184. It should not be too bad. Probably a couple days of work and a day to debug | 05:13.12 |
| overprint device needs a copy_planes procedure | 05:13.48 |
| actually it may go faster than that | 05:19.08 |
| the fill rectangle hl_color proc that I wrote is going to be very similar to this | 05:19.30 |
| but I am going to take care of it tomorrrow | 05:19.42 |
fm__ | chrisl, 04 resuts again in two pages ... | 08:18.19 |
chrisl | fm__: I can't get my head around that - there's nothing in there to prompt a page ejection, so the most I'd expect is one page with an error..... | 08:19.30 |
fm__ | same as the file before, one blank page first and then one with the undefined error and 179 :-( | 08:21.32 |
chrisl | Okay, I'll chop out some of the prolog, and we'll see what that gets | 08:23.19 |
fm__ | In case you need it for the record the detailed version according to the integrated web frontend is: | 08:24.04 |
| MODELL: FS-1350DN | 08:24.05 |
| VERSION: 2H4_3000.006.004 | 08:24.05 |
| sadly, i think there is no way to update firmware ... :-( | 08:24.26 |
chrisl | fm__: that's really no help - I don't have that printer, and no better way to debug it. That's why it would be nice to get some cooperation from Kyocera on these problems: they ought to be able to quickly track down what in the PS causes the problem, even if they can't update the printer firmware. | 08:25.44 |
fm__ | i searched all of the webfrontend, there seems to be no option to get debug logs from the rip or whatever ... | 08:28.53 |
chrisl | fm__: You would probably only get the same error you get printed on the paper. How is the printer connected to the computer? | 08:30.05 |
fm__ | chrisl, it has an integrated printserver, it is just connected via lan | 08:30.33 |
| device for Kyocera_FS-1350DN: socket://192.168.178.3 | 08:31.23 |
chrisl | fm__: if you can find the netcat incantation to send PS to the printer, *some* printers pipe "back channel" output back across the network - that would be useful, because I could dump a lot of debugging info without wasting paper. | 08:32.10 |
| tkamppeter: ping | 08:32.24 |
tkamppeter | chrisl, hi | 08:32.48 |
fm__ | chrisl, do you have the netcat command with parameters at hand? | 08:32.55 |
chrisl | tkamppeter: hi, can you remind us of the netcat incantation to send PS to a networked printer, please? | 08:33.16 |
tkamppeter | fm__, it is "nc -w1 <IP of printer> 9100 < FILE | 08:33.30 |
chrisl | tkamppeter: thanks! | 08:33.43 |
fm__ | chrisl, should i send 04 there again? | 08:34.25 |
chrisl | fm__: just give me sec, I'm about to upload another cut down file | 08:34.55 |
| fm__: http://www.ghostscript.com/~chrisl/printout-005.ps | 08:35.52 |
| That's cut down more, and also has a test to see if you get backchannel printing from nc | 08:36.27 |
fm__ | chrisl, no back channel. :-( however 05 does not emit any page ... | 08:38.18 |
| nc command exits after a fraction of a second ... | 08:38.53 |
chrisl | fm__: hmm, annoying about the lack of back channel (but not a huge surprise). At least we're starting to narrow things down a bit! | 08:45.08 |
| fm__: http://www.ghostscript.com/~chrisl/printout-006.ps | 08:50.10 |
| That has all the font definitions removed | 08:50.20 |
fm__ | chrisl, prints nothing | 08:51.22 |
chrisl | fm__: so, the problems seems to be somewhere in the font definitions...... | 08:52.13 |
fm__ | chrisl, 05 did not print anything as well | 08:53.38 |
chrisl | fm__: 05 had both font and color space definitions removed, the one I'm uploading now has the patterns removed, but the fonts left in (just to be sure we're looking a the fonts) | 08:54.45 |
fm__ | btw. these documents are 05 and 06 are somehow special. the integrated job history view of my printer gives the name "doc" + id instead of printout.pdf | 08:55.15 |
| ok, thanks | 08:55.20 |
chrisl | Hmm, I haven't changed the DSC comments that include the job title - I can't even guess at the logic Kyo are using for their logging | 08:56.39 |
| fm__: http://www.ghostscript.com/~chrisl/printout-007.ps - no pattern definitions | 08:56.51 |
fm__ | 07 errors with two pages and 179 | 08:57.58 |
| chrisl, i guess you get closer! | 08:58.33 |
chrisl | The two pages still confuses me, but yes, getting slowly closer - I warned you it would take a lot of iterations! | 08:59.02 |
fm__ | and paper :-( | 08:59.28 |
chrisl | fm__: well, without the backchannel, the only feedback we get is on the page :-( With care, you can reuse, of course...... | 09:00.27 |
fm__ | ;) | 09:00.43 |
| tkamppeter, i keep reading that cups is moving to a pdf based flow to be closer aligned to the os x print workflow. however renderer=gs for me does not sound like pdf, could you shade some light where we are going? ;) | 09:05.19 |
| is this just for the first part of app -> cups -> printer :-( | 09:06.06 |
chrisl | fm__: you can't send PDF to your printer - cups queues the PDF, then uses gs to convert it to *something* to send to your printer - in this case, PS | 09:06.14 |
| fm__: http://www.ghostscript.com/~chrisl/printout-008.ps | 09:06.51 |
fm__ | chrisl, 08 prints nothing | 09:08.03 |
tkamppeter | chrisl, nc is bi-directional, if the printer answers something back to the sender, it appears on the screen, right after the nc command, if not, replace "-w1" by "-w10" or drop the "-wX". | 09:09.04 |
fm__ | tkamppeter, using -w200 ;) | 09:09.23 |
| so what is the fundamental diference between the gs based approach and pstopdf? pstopdf comes from poppler which is factored out of xpdf if I understand correctly ... what is the role of gs? | 09:10.50 |
chrisl | pstopdf isn't involved anywhere, is it? | 09:12.07 |
| fm__: basically, gs supports better quality output (particularly color), and we have plans to improve the color fidelity further - in ways that poppler doesn't support | 09:13.24 |
fm__ | chrisl, in my first bug tkamppeter asked me to try as cups option pdftops-renderer-default=pdftops and pdftops-renderer-default=gs. I assume in the first case pdftops is used | 09:14.32 |
chrisl | You said "pstopdf" not "pdftops" | 09:15.11 |
fm__ | ok sorry ;) | 09:15.40 |
kens | chrisl I'm heading out for ~2 hours, I'll read teh lgos when I get back so let me know here if you need anything from me. | 09:15.44 |
chrisl | ah, I'm about to go out for a couple of hours, too! | 09:16.21 |
| fm__: the answer still stands - moving forward, they're looking for cups to become a fully "color managed" workflow, which gs now has the infrastructure (and quite a few features) in place to support. | 09:17.54 |
fm__ | and what happens to printerd? | 09:18.27 |
chrisl | What is printerd? I'm not a cups dev, just gs..... | 09:19.05 |
fm__ | chrisl, i am just a user. but it was announced as new cups http://cyberelk.net/tim/2012/05/10/announcing-printerd/ | 09:20.04 |
chrisl | fm__: given that it is leaveraging the existing cups back ends, I would expect we'll have the same benefits and problems from using gs | 09:21.46 |
| http://www.ghostscript.com/~chrisl/printout-008.ps - other font stuff cut out | 09:22.25 |
| fm__: I need to step out for a couple of hours - if you're still around, we'll pick up when I get back..... | 09:22.56 |
fm__ | chrisl, i assume you meant 09 | 09:23.59 |
chrisl | darn, yes, sorry | 09:24.11 |
fm__ | 09 gives blank plus 179 | 09:25.04 |
| chrisl_away, ok, thanks alot for your help | 09:25.24 |
Robin_Watts | sebras: Your patch looks plausible to me. | 10:37.56 |
| In load_sample_func you have a dangling else { } | 10:40.00 |
| In load_exponential_func you've removed some tests on /N that seemed reasonable. | 10:41.11 |
| And throughout, you silently accept stuff where we have too many values rather than too few. | 10:41.54 |
| Maybe an fz_warn would be appropriate? | 10:42.12 |
paulgardiner | Robin_Watts, tor8: Just want to check I'm planning to do the right thing. | 11:48.57 |
Robin_Watts | zipper in the front. | 11:49.45 |
paulgardiner | When "\u20ac" is read in as a string it's left in that form, whereas when \243 is read in, it is interpreted and turned into a pound sign | 11:50.08 |
Robin_Watts | By what and where? | 11:50.42 |
paulgardiner | It's ok, getting my trousers on around the right way hasn't been a problem since I asked you last week. | 11:50.53 |
| Robin_Watts: by MuPDF's file reading stuff | 11:51.12 |
Robin_Watts | Urm... You mean the pdf lexer? | 11:51.58 |
paulgardiner | Yep. | 11:52.03 |
Robin_Watts | That makes sense. | 11:52.21 |
paulgardiner | Yeah, I thought that was probably right. | 11:52.38 |
Robin_Watts | \243 is an octal char (see section 3.2ish of the spec) | 11:52.40 |
| \u2ac smells like a javascripty thing. | 11:52.55 |
paulgardiner | When I pass strings onto v8 \u2ac is fine, but a pound sign is seen as a strange bit of utf8 | 11:53.40 |
Robin_Watts | takes the scrollbar handling for the windows ports out and shoots it in the back of the head. repeatedly. | 11:54.04 |
paulgardiner | So I was planning on preprocessing strings before sending to v8: if they have any top-bit-set chars, convert to utf8 | 11:54.15 |
Robin_Watts | If v8 is expecting to be fed utf8 strings, then converting strings to utf8 before feeding them to v8 would seem sensible. | 11:55.35 |
paulgardiner | I mean interpret top-bit-set XX as unicode 00XX and congert to utf8 | 11:55.45 |
| convert | 11:55.54 |
Robin_Watts | In the absence of any other encoding information for the pdf strings, that's probably not a bad idea. | 11:56.18 |
tor8 | what's the source of these strings? | 11:56.37 |
| user input? | 11:56.43 |
paulgardiner | tor8: direct from the file | 11:56.51 |
sebras | Robin_Watts: the tests for pdf_is_int() and pdf_is_real() are not evil, but we don't do it elsewhere. we rely on that pdf_to_int()/pdf_to_real() return 0/0.0f for unparse:able datatypes. | 11:56.55 |
paulgardiner | strings in the file | 11:56.57 |
sebras | Robin_Watts: so the tests for /N that I removed are now handled silently by the later call to pdf_to_real(). | 11:57.23 |
Robin_Watts | sebras: Right. But we had a test for it before, and now we don't. | 11:57.32 |
paulgardiner | tor8: I was thinking that for user entered text the app would send utf8 to the lib | 11:57.34 |
Robin_Watts | sebras: Not quite. | 11:57.38 |
| Previously we would have spit an error, now we silently accept them. | 11:57.50 |
paulgardiner | tor8: I mean pdf strings within the pdf file | 11:58.16 |
sebras | Robin_Watts: yes, but isn't that the policy on parse issues? | 11:58.19 |
Robin_Watts | I'd be tempted to leave the tests in there with an fz_warn. | 11:58.19 |
sebras | Robin_Watts: right, adding a fz_warn() is not an issue at all. :) | 11:58.35 |
chrisl | fm__: so 09 gives error? | 11:58.52 |
tor8 | paulgardiner: maybe I'm confused but didn't you do this already with the pdfdocencoding stuff? | 11:58.52 |
paulgardiner | Hmmm. I guess actually they are in pdfdoc enc, so I should convert from pdfdocenc to unicode to utf8 | 11:58.53 |
Robin_Watts | sebras: We have a new thing in the cookie, where we count the errors encountered during rendering. | 11:58.57 |
sebras | Robin_Watts: actually I have them for the most part but removed them for /N. | 11:58.59 |
Robin_Watts | So people can get a rendering out, and yet know that it's possibly corrupt. | 11:59.18 |
| At the moment it only triggers on errors, not on warns, but at some point warnings should trigger it too. | 11:59.57 |
| and so having an fz_warn in there would be better. | 12:00.09 |
| Lots of places you fz_warn if we don't have enough of something, but don't warn if we have too many. | 12:00.32 |
| I reckon that deserves warnings too. | 12:00.48 |
paulgardiner | tor8: the only place I currently take care of encodings is when synthesising appearance strings from output of v8. And that was added only yesterday. Before that I had no conversions in the system | 12:00.49 |
| tor8: I put something in the report a few weeks back, but that was plans not stuff already done. Maybe that's what you are thinking of | 12:02.07 |
sebras | Robin_Watts: I'll fix those too of course. | 12:05.06 |
| tor8: do you have any input? | 12:08.44 |
paulgardiner | tor8: It's me that's confused. Yes, we'd discussed this before, and decided how it should all work when done propertly. I'm currently looking at finding a simpler fix for the najority of files, and didn't notice that in this case the quick fix is just to do it properly. | 12:12.30 |
Robin_Watts | kens: ping | 12:14.49 |
| kens: I've just pushed a change to the scrollbar handling in windows for the gs/pcl etc display device window (and the gswin32.exe console window). | 12:15.29 |
kens | Robin_Watts : pong | 12:15.39 |
| I have to say I've never looked at that code | 12:15.55 |
Robin_Watts | They now properly handle proportional scrollbars. If you have time, can you give it a whirl just to check it behaves as you'd expect? | 12:16.09 |
kens | OK let me update and rebuild | 12:16.24 |
Robin_Watts | lunches. bbs. Thanks | 12:16.30 |
kens | Robin_Watts : the scrollbar looks good to me, useful to have. I'm a little puzzled why we ahve a scrolling (text) window for the main GS interface, even when there are only a few lines in it, but its always done that, so its not a problem, just a curiosity | 12:23.36 |
chrisl | kens: I have found a *very* minor bug in the DSC comments in the ps2write output...... | 12:44.34 |
kens | And that makes a difference to the printer ? | 12:44.47 |
| What's hte problem ? | 12:44.58 |
chrisl | I seriously doubt it affects the printer! | 12:45.06 |
| The objects for the FontFile streams have an EndResource comment, but no BeginResource comment | 12:45.51 |
kens | Oh, that semms odd.... | 12:46.32 |
chrisl | I'm not sure if there is a missing BeginResource, or an extra EndResource | 12:46.34 |
kens | I'll have to look | 12:46.42 |
| Ah, Fonts are 'different' | 12:47.10 |
chrisl | Aren't they always? | 12:47.32 |
kens | Yes, but even more so in this case. Its fonts that are preventing me doing linearisation now too. | 12:47.58 |
| I can fix that easily enough I think, I probably should, I'll look at it now. | 12:48.45 |
chrisl | As I say, I doubt it will affect any printers - It would be *very* depressing if DSC comments could cause an error on a printer | 12:49.45 |
kens | Yea, I should fix it though. | 12:50.17 |
chrisl | I just mean that I don't think it's going to be the "get out of jail free card" for the current Ubuntu printer problem :-( | 12:51.11 |
kens | No, and I see fm__ has gone quiet | 12:51.35 |
| Have you considered inserting 'save showpage restore' sequences in the file and trying to binary chop to the error ? | 12:52.08 |
paulgardiner | Robin_Watts, tor8: pdf_to_utf8 is a public function, so I guess it would be bad if I changed it to take an xref in place of a context? | 12:52.35 |
chrisl | kens: I started that route, but I didn't get a result I could understand - I added a showpage instead of the marking operations, and we got two pages with an error printed on them..... | 12:53.18 |
Robin_Watts | Hm. That sounds like it should maybe be fz_to_utf8 if it takes a context. | 12:53.20 |
kens | chrisl that's fairly weird. Sounds like its barfing in the prolog ? | 12:53.52 |
chrisl | kens: it's barfing on one of the font definitions - that's how I stumbled on the comments problem | 12:54.21 |
kens | Oh great :-( | 12:54.32 |
Robin_Watts | paulgardiner: I'd support making it an fz_ function, I think. Have to see what tor8 thinks. | 12:54.35 |
paulgardiner | Robin_Watts: I'd just like it to work with streams in addition to strings which is why it would be handy if it took an xref | 12:54.37 |
Robin_Watts | oh, ignore me. | 12:54.56 |
| It takes a pdf_obj. | 12:54.59 |
kens | Robin_Watts : can I pester you about linearisation, or are you busy ? | 12:55.04 |
paulgardiner | yep | 12:55.07 |
Robin_Watts | Then, yes, it should take an xref, I think. | 12:55.07 |
| kens: I am nicely between jobs, so go for it | 12:55.17 |
kens | OK I definitely can't do anything with pdfwrite | 12:55.30 |
| So I have to write the whole file and then reorder it | 12:55.38 |
Robin_Watts | kens: Thanks for checking the scrollbars, btw. | 12:55.39 |
kens | NP | 12:55.44 |
Robin_Watts | kens: Right... | 12:55.47 |
paulgardiner | Robin_Watts: and changing it is ok even though it's declared in mupdf.h? | 12:55.49 |
kens | This means renumbering the objects | 12:55.53 |
| How did you go about htis ? Did you reparese the file you had already emitted ? Looking f ro x y obj and x y R ? | 12:56.23 |
Robin_Watts | paulgardiner: It's not ideal, but it's making it more consistent, so I think I prefer it. | 12:56.25 |
| We have the file in memory. | 12:56.47 |
kens | Ah, I have it on disk, but the question is similar | 12:56.59 |
paulgardiner | Robin_Watts: Ok, I'll do that. Can always back it out and find an alterntaitve if tor8 has objections. | 12:57.00 |
kens | How did you g about renumbering the objects ? | 12:57.28 |
tor8 | it's pdf specific (since it uses pdf doc encoding) so I think it should still live in the pdf_ namespace | 12:57.44 |
Robin_Watts | kens: The logic is all in pdf_write.c | 12:58.24 |
tor8 | changing the first argument would be awkward, but if you need a pdf_document instead of a fz_context, go ahead | 12:58.28 |
| awkward since it's public, but if there's a good reason we can still do it | 12:58.47 |
kens | Hmm, I guess I'll have to go get teh MuPDF sources.... | 12:58.49 |
Robin_Watts | kens: http://git.ghostscript.com/?p=mupdf.git;a=blob;f=pdf/pdf_write.c;h=76ab1cb1c6a9e97dfa25198239c1b82c4e74bc89;hb=c47b3796f4f0314a86e4a82f7d467f8130c96b6c | 12:59.07 |
kens | aha, thanks Robin_Watts | 12:59.20 |
| Robin_Watts : looks like you have more information than I do I think | 13:00.08 |
paulgardiner | tor8: ok thanks tor8 | 13:00.14 |
Robin_Watts | So pdf_write_document is line 2074 onwards | 13:00.16 |
| First we make sure that we have all the object streams in memory. | 13:00.48 |
kens | Yeah, I cna't do that | 13:00.59 |
paulgardiner | Robin_Watts, tor8: as an aside, I wonder if pdf_is_stream(pdf_to_num, pdf_to_gen) can give false positives. | 13:01.03 |
Robin_Watts | Then we do any garbage collection (i.e. mark/sweep to detect unused objs). | 13:01.20 |
| Then we attempt to detect any duplicate objects (repeated images etc) | 13:01.42 |
| Then we compact the xref (that does a renumber) | 13:02.10 |
kens | reumbering the objects is my problem | 13:02.19 |
paulgardiner | Robin_Watts, tor8: pdf_to_num produces 0 if applied to a non-indirect object and obj(0,0) may be a stream. | 13:02.21 |
Robin_Watts | or rather it builds a map for how the renumbering should be done. | 13:02.28 |
| Then the actually renumbering is done in renubmerobjs. | 13:03.02 |
| We can get away with this because we have it all in memory. | 13:03.10 |
| and all references point through an xref entry. | 13:03.30 |
kens | Robin_Watts : my problem is in doing the renumbering. If a page uses (eg) a Form XObject and the Form XObject uses an image and the image uses a colorspace, I need to renumber all of these. But the objects are on disk byb now not in memory | 13:03.50 |
Robin_Watts | kens: Right. You'd need to build a 'renumber map' in the same way we do. | 13:04.10 |
kens | Robin_Watts : That isn't a problem per se | 13:04.29 |
| Its identifying which objects 'contain' references to other objects | 13:04.56 |
| I can't do my renumbering until after the fonts are written to disk, bnecause that action adds more objects | 13:05.14 |
Robin_Watts | Right. Where you have a contents stream that contains references to other places. | 13:05.21 |
| You need to read every stream back in, rewrite it, then write it out. | 13:05.30 |
| We are at pains to do the renumbering before writing streams. | 13:05.42 |
kens | Yes, that's what I'm thinking | 13:05.44 |
Robin_Watts | Actually, ignore me. | 13:05.50 |
| Streams never contain pointers to other things. | 13:05.57 |
kens | Robin_Watts : that's what I wanted to do, but I can't | 13:05.58 |
| No, but the dictionary can | 13:06.06 |
Robin_Watts | stream dictionaries do, but ... right. | 13:06.10 |
kens | So I'm thikning I'll have to read and parse every object as I read it back, and if its a dictionary, check for object references that ened altering. | 13:06.51 |
Robin_Watts | yes. | 13:06.57 |
| which will potentially change the length of the object when you write it out. | 13:07.11 |
kens | ah well, I was hoping to acoid that, so I thought I'd ask if you had a clever way to avoid it. | 13:07.22 |
| I already know the length will change so I was going to build a new xref as I make the new file | 13:07.38 |
Robin_Watts | which will mean you need to write everything out again, before calculating offsets for use in the hint streams. | 13:07.42 |
kens | Robin_Watts : Yes, so it looks like I will end up writing the file 3 times | 13:08.00 |
Robin_Watts | yeah. joy. | 13:08.06 |
kens | The only alternative is to completely rewrite the way pdfwrite stores objects | 13:08.31 |
Robin_Watts | All our linearisation code is in pdf_write, in (hopefully) a well commented form. | 13:08.32 |
| so you may find it useful. | 13:08.41 |
| If you spot any WTFs in it, please let me know. It's not been hugely tested. | 13:08.56 |
kens | Robin_Watts : thanks, I'm not sure how relevant it is, but I will look | 13:08.58 |
Robin_Watts | I'd imagine it may be useful for the hint stream generation. | 13:09.14 |
kens | Hint streams yes, possibly, I'm already stealing some of that from pdfopt.ps | 13:09.31 |
fm__ | chrisl, i am back | 13:09.45 |
kens | Or more accurately I will be ;-) | 13:09.45 |
Robin_Watts | The ordering of objects is interesting too. | 13:09.56 |
kens | Yes, but I think that is OK. | 13:10.25 |
Robin_Watts | You have to identify which objects belong to which bits of the file (page 1, shared, etc) | 13:10.25 |
tor8 | paulgardiner: object 0 must be 'free' in the xref according to the spec | 13:10.50 |
kens | Robin_Watts : Yes, and that is looking tricky with the fonts stuff | 13:11.03 |
paulgardiner | tor8: Ah good. Thought that might be it | 13:11.07 |
kens | Robin_Watts : I think the only thing that matters is 'is it on page 1' if so it has to come first, otherwise it comes later | 13:11.44 |
| After all teh structure stuff. | 13:12.00 |
Robin_Watts | kens: Well... not according to the letter of the spec. | 13:12.04 |
kens | OK which bit did I miss ? | 13:12.13 |
Robin_Watts | The letter of the spec says that all page 2's objects should come next. then all page 3's then all page 4's... then the shared objects. | 13:12.31 |
kens | Well, that's just not going to be possible. | 13:12.45 |
Robin_Watts | And the hint streams identify the regions of the files which each pages objects fall into. | 13:12.55 |
kens | I thought the ordering after page 1 was optinal | 13:13.06 |
| optional | 13:13.13 |
Robin_Watts | so a smart client can know what regions of the file it needs to read for a given page. | 13:13.16 |
| kens: I thought that too, but the hint stream is NOT optional. | 13:13.34 |
| and to make a well formed hint stream, the rest needs to be in order. | 13:13.50 |
kens | No sure, let me reread that bit, I've just opened the file | 13:13.51 |
| Robin_Watts : I'm not certain that it is possible to create a PDF file like this with teh current pdfwrite. | 13:15.40 |
Robin_Watts | It requires some significant analysis of the file to work out which objects go where. | 13:16.11 |
kens | Obviously I could write something that would extract each page and deal with its resoruces, writing tehm back as required, but that would be quite a bit of parsing | 13:16.15 |
Robin_Watts | This stuff was a massive pandoras box. Every time I got to the next bit I wished even harder that I'd never started. | 13:16.36 |
kens | I'm feeling like that right now | 13:16.47 |
| It doesn't help that the way pdfwrite works is massively non-optimal for this | 13:17.06 |
Robin_Watts | Again, I had the advantage that everything is in memory (or effectively so) for me. | 13:17.16 |
| So I can walk the structure of the file with relative ease. | 13:17.32 |
kens | SOme of the stuff isn't even created when we write the file for me | 13:17.34 |
Robin_Watts | Yes. I'm working from 'a complete well formed pdf file', which you aren't. | 13:18.02 |
kens | Well, I can be, but it means writing (in effect) a stand-alone lineariser which can work with the fiel after its output by pdfwrite | 13:18.42 |
Robin_Watts | kens: That feels like a sensible approach to me. | 13:19.00 |
| What is wrong with pdfopt.ps ? | 13:19.09 |
| (Genuine question) | 13:19.21 |
kens | Its written in PostScript ? | 13:19.21 |
Robin_Watts | So? | 13:19.29 |
kens | So its slow and it has bugs, and I can''t call it from pdfwrite so you need a post-proces invocation of GS | 13:19.45 |
Robin_Watts | So it's hard to maintain? So it's slow? So there are certain things it can't do? | 13:20.01 |
kens | Anbd you have to use GS, so you can't use it to klinearise the output from (eg) ghostpcl without also having GS | 13:20.13 |
Robin_Watts | Write a linpdfwrite device that uses pdfwrite to produce a temporary file, then system("mubusy clean -l temp.pdf out.pdf") :) | 13:20.45 |
kens | Yeah, but that isn't very cross-platform | 13:21.10 |
| and relies on MuPDF being present (cf earlier GS) | 13:21.20 |
Robin_Watts | system is ansi C :) | 13:21.21 |
tor8 | Robin_Watts: something in the commit that makes all the min/max/abs macros into inlines causes the XObject in 1522*.pdf to disappear from the rendering. the mudraw -x output is identical though, so probably something in the draw device. any suggestions where to start looking? | 13:21.35 |
Robin_Watts | Yes. | 13:21.35 |
| tor8: if it was me, I'd drop the old .c file back in. | 13:22.09 |
kens | SO it looks like I'm going to have to replicate a shed load of stuff to parse the file back again. I do have some information but its all terribly opaque in the traditional 'lets hide everything' pdfwrite manner | 13:22.40 |
Robin_Watts | kens: Yeah, you have my utmost sympathies. | 13:23.02 |
kens | Maybe I'll just gie up and do something else | 13:23.17 |
Robin_Watts | tor8: If I broke it, would you like me to try to fix it ? | 13:23.24 |
tor8 | the only other diffs from the commit are single pixel differences in a few places | 13:23.56 |
Robin_Watts | presumably cos of float/double conversions or something? | 13:24.18 |
tor8 | but the images in the ImagePixelFormats.xps are off quite regularly by a pixel, so maybe some float/int function mismatch? | 13:25.03 |
| yeah, I wouldn't worry about the majority of these diffs | 13:25.12 |
Robin_Watts | tor8: could be. I'm disappointed if that is the case though, cos I thought I checked carefully, and I remember running it through the cluster several times and bmpcmping | 13:25.47 |
tor8 | http://tor.dlinkddns.com/sane/report/1.0.68-8ff2db0-1.0.69-ee382eb.html | 13:26.09 |
Robin_Watts | The ImagePixelFormats thing, it looks like the background is off a bit. | 13:27.47 |
| possibly just an antialias difference on the left edge? | 13:27.58 |
tor8 | looks like the edge difference only | 13:28.46 |
Robin_Watts | yes. I can't tell offhand if it's that we now extend 1 pixel further left, or that the leftmost pixel is differently antialiased. | 13:29.21 |
| but either way, la la la, I can't hear you. | 13:29.35 |
tor8 | 1522 is disconcerting though... | 13:30.00 |
Robin_Watts | Yeah. Let me prod it with a stick for a mo. | 13:30.13 |
tor8 | kk | 13:30.25 |
Robin_Watts | There is lots of stuff in that Do - it's not just an image. | 13:41.13 |
| and if I remove it all and replace it with a grey rectangle I see it. | 13:43.20 |
| Hmm. pdf clean is producing horribly broken stream offsets :( | 13:48.38 |
| oh, it's marking all the objects as free. | 13:50.25 |
paulgardiner | Robin_Watts, tor8: there are three small commits on paul/forms if you have a moment later. | 14:10.54 |
Robin_Watts | tor8: OK. I found the stupid pdfclean breakage I caused. | 14:14.15 |
| oh... | 14:17.46 |
| -2147483648 -2147483648 m | 14:18.00 |
| -2147483648 2147483647 l | 14:18.02 |
| 2147483647 2147483647 l | 14:18.04 |
| 2147483647 -2147483648 l | 14:18.06 |
| tor8: That looks like a candidate to me :) | 14:18.13 |
| tor8: ping | 14:40.23 |
| Something odd I spotted while looking at this 1522. fz_flatten_fill_path, in the top of the FZ_MOVETO we have if (i && ...) | 14:41.01 |
| i will always be true, as we've just passed an i++, right? | 14:41.15 |
| I think we should remove the i && there. | 14:42.06 |
| The problem with the disappearing diagram is in fz_insert_gel | 14:43.29 |
| fy1 = 2.72e10. (int)fy1 = -2147483638 | 14:58.18 |
| tor8: ping ? | 14:58.54 |
| It looks like I need a 'safe' way to cast floats to ints. | 14:59.09 |
| actually, scratch that. | 14:59.24 |
| tor8: 3 patches on my master to review. One of them fixes 1522.pdf | 15:10.22 |
fm__ | chrisl, chrisl_ are you back? | 15:20.40 |
chrisl_ | fm__: yes, been back for a while - so 009 gave you the "normal" error? | 15:21.06 |
fm__ | great, thanks | 15:21.19 |
saper | you can't safely cast floating point into int :( | 15:21.34 |
fm__ | chrisl_, yes 009 gives standard error with the 179 | 15:22.13 |
| two pages, first blank | 15:22.31 |
chrisl_ | Okay, here's the next one http://www.ghostscript.com/~chrisl/printout-010.ps | 15:23.54 |
fm__ | chrisl_, erros | 15:25.19 |
chrisl_ | OKay...... | 15:25.31 |
| fm__: http://www.ghostscript.com/~chrisl/printout-011.ps | 15:26.48 |
fm__ | chrisl_, erros | 15:28.05 |
| btw. do you have any usefull editor to remove the different elements from the file, or are you using a text editor? ;) | 15:29.22 |
chrisl_ | Using a text editor | 15:29.37 |
| fm__: excuse this one - this is a double check, since we were down to the final font object, I want to be sure it is the problem: http://www.ghostscript.com/~chrisl/printout-012.ps | 15:30.51 |
fm__ | chrisl_, that one does nothing | 15:31.49 |
chrisl_ | fm__: good, so I know what object is causing the problem..... | 15:32.27 |
Robin_Watts | paulgardiner: Other 2 reviews all look great. | 15:33.55 |
paulgardiner | Robin_Watts: Thanks. I will add some comments to the first one. I'll let you know when I push. | 15:34.34 |
chrisl | fm__: http://www.ghostscript.com/~chrisl/printout-013.ps - font definition in a different form | 15:39.36 |
fm__ | chrisl, does nothing | 15:41.23 |
chrisl | fm__: okay, that's good (in a way). I'll need a few minutes to make the next test for you | 15:41.58 |
fm__ | looking at your diffs i assume ASCII85Decode somehow gives a problem? | 15:43.10 |
paulgardiner | Robin_Watts: done | 15:43.17 |
chrisl | fm__: no, I don't think the filter is the problem in that case | 15:43.35 |
Robin_Watts | paulgardiner: Pushed. | 15:43.52 |
paulgardiner | ta muchly | 15:44.00 |
Robin_Watts | you're very welcome. | 15:44.24 |
chrisl | fm__: another file for you to try, please: http://www.ghostscript.com/~chrisl/printout-014.ps | 15:53.03 |
fm__ | no error yet, but seems to be unbelievable slow, it is blinking for two minutes now ... | 15:56.03 |
| is it maybe missing some terminating part? the data transfer led is blinking, while nc has already exited 3 minutes ago | 15:56.44 |
chrisl | fm__: the only thing I changed was the font streams | 15:57.27 |
fm__ | chrisl, printed | 15:59.15 |
| however it takes arround 5 minutes | 15:59.25 |
| :-( | 15:59.28 |
kens | Odd | 15:59.35 |
fm__ | I had reported this earlier wait a second | 15:59.37 |
chrisl | fm__: I'm not worried about that, there are other measures in place for better printing times | 16:00.00 |
fm__ | https://bugs.launchpad.net/ubuntu/+source/cups-filters/+bug/980616 this is my earlier report when it took 5 minutes | 16:00.31 |
| tkamppeter, made it somehow work better back then ... | 16:01.02 |
chrisl | fm__: there are objects in the PDF we have to render to image to display in PS, and they are probably being rendered at high res, and your printer is (slowly) downsampling. | 16:03.32 |
| kens: so, this looks like the printer doesn't like the fonts being in pfb format | 16:04.13 |
kens | chril that doesn't totally surprsie me. | 16:04.27 |
| chrisl | 16:04.31 |
| I can probably fix that, given a little time | 16:04.51 |
chrisl | No, frankly, I'm surprised it hasn't caused us more problems..... | 16:04.52 |
kens | I'll look into it tomorrow | 16:05.01 |
| Will be logging off shortly | 16:05.12 |
chrisl | Well, at least I think we're at the root of the problem | 16:05.35 |
| fm__: I'm going to update the Ubuntu bug | 16:05.56 |
fm__ | thanks | 16:06.31 |
chrisl | fm__: the reason I think it's the pfb font is because there are loads of other things ASCII85 encoded in there, and those work fine. | 16:07.41 |
kens | I can probably get a file to test tomorrow, with just hte .pfb format recoded | 16:08.09 |
chrisl | kens: that's what I just had fm__ test | 16:08.35 |
kens | OK, presumably the slowness is the resolution of the images ? | 16:08.59 |
Robin_Watts | mvrhel: Ping? | 16:09.13 |
fm__ | kens, http://www.ghostscript.com/~chrisl/printout-014.ps just as a bit of images ... | 16:09.36 |
kens | There are images there, certainly | 16:10.15 |
chrisl | kens: Yes, I think so - I think it's leaving ps2write's default resolution, which tkamppeter aleady put in place facilities to handle it | 16:10.18 |
kens | Ah, well the default would be 720 dpi | 16:10.37 |
chrisl | Yep, and on an (alleged) 600dpi printer, that's suboptimal...... | 16:11.32 |
kens | Indeed | 16:11.41 |
fm__ | chrisl, should be 1200dpi | 16:12.03 |
kens | images at 1200 dpi on a 1200 dpi printer is wasteful, 300 dpi is enough | 16:12.35 |
chrisl | fm__: in the PS, drawn from the PPD: << /HWResolution [600 600] /PreRenderingEnhance false >> setpagedevice | 16:12.46 |
Robin_Watts | kens: depends on the image, and the printer. | 16:24.54 |
kens | Yeah I'm generalising | 16:25.12 |
Robin_Watts | If you've got a 1200dpi fax image, then you'd really rather like to get that out as 1200dpi :) | 16:25.32 |
kens | Robin_Watts : monochrome images are different | 16:26.29 |
| I was thinking of contone | 16:26.36 |
ray_laptop | kens: good. I got here when you are still around | 16:29.18 |
kens | Just in time ray I'm off very shortly | 16:29.32 |
ray_laptop | kens: I am changing the fill_rectangle_hl_color device proc (discussed with others after you left yesterday) | 16:29.51 |
henrys | next time I promise a new gl/2 feature to a customer somebody please slap me. | 16:29.53 |
kens | ray_laptop : I saw that in the logs | 16:30.03 |
ray_laptop | I am getting rid of the 'pcpath' | 16:30.04 |
| kens: I'm keeping the 'pis' although everyyone ignores it except pdfwrite which needs it | 16:30.40 |
kens | yes, pdfwrite needs it always | 16:30.53 |
ray_laptop | I don't know why the hl_fill needed the pcpath | 16:31.08 |
kens | Is that a clip path ? | 16:31.16 |
ray_laptop | kens: yes | 16:31.27 |
kens | Seems odd for a rectangle... | 16:31.37 |
ray_laptop | kens: maybe Igor was thinking about using it with a shaded color to pick up the boundaries of a shape | 16:32.06 |
kens | Beats me :-) | 16:32.26 |
ray_laptop | kens: but I put in a print statement and ran through 1500 PDF files (that might have shadings) and it was never called for anything other than a full page fill. | 16:32.54 |
kens | Well, that seems pretty conclusive to me | 16:33.11 |
ray_laptop | kens: fine. I'll just make appropriate comments in the log when I commit | 16:33.27 |
kens | Sure, was that it ? | 16:33.37 |
ray_laptop | kens: yep | 16:33.41 |
kens | Oh, that was easy :-) | 16:33.48 |
ray_laptop | kens: have a good evening | 16:33.54 |
kens | And a good day to you ray :-) | 16:34.02 |
| Goodnight all | 16:34.08 |
henrys | bye kens | 16:34.15 |
Robin_Watts | Night kens | 16:34.16 |
ray_laptop | kens: I just wanted to make sure I wasn't going to zap something you were going to be using someday | 16:34.17 |
| whew... just in time | 16:34.48 |
| Robin_Watts: I saw in your commit message for the interp optimization: whole line. We still calculate contributions for the whole of the images | 16:39.04 |
| Robin_Watts: why doesn't that slow things down ? | 16:39.25 |
henrys | I was starting to get concerned about the birthday paradox when I didn't see multiple birthdays showing up on facebook each day but now I see lots of folks don't include their birthdays. | 16:40.03 |
Robin_Watts | contributions are an internal table used by the interpolation routine. | 16:40.14 |
henrys | when mvrhel we should talk about how much effort we are dedicating to this customer (gemma). seems a bit much. | 16:42.03 |
ray_laptop | henrys: probably so, but we have others doing seps now. Unfortunately, most of them seem to be free GPL users | 16:42.42 |
| ghostscript apparently is a pretty useful tool for print shops | 16:43.16 |
Robin_Watts | ray_laptop: Sorry, off phone now. | 16:45.03 |
| contributions are calculated once at the start of an image. One table gives the weights to use when combining pixels when scaling on X. Another gives the weights to use when combining pixels on Y. | 16:46.02 |
| Hence the act of making the contributions is 2 O(n). | 16:46.18 |
| The act of using the contributions is O(n^2). | 16:46.29 |
| My patch reduces the O(n^2) work, not the O(n) work, but then the O(n) work is dwarfed by the O(n^2). | 16:47.38 |
ray_laptop | Robin_Watts: thanks. I didn't know they were only calculated once. | 16:48.30 |
| and that x and y were separate tables | 16:49.15 |
| but it makes sense now that I know what 'contributions' mean | 16:49.36 |
| brb... | 16:51.19 |
henrys | nice work on the interpolation Robin_Watts! | 17:06.26 |
Robin_Watts | henrys: Yes, glad to have that one out of the way. | 17:08.17 |
| The other customer bug from the same people looks to have a trivial fix too. | 17:08.32 |
| but I want to check it with mvrhel. | 17:08.44 |
tor8 | Robin_Watts: sorry, was out for a brief walk and decided to grab something to eat | 17:09.20 |
Robin_Watts | tor8: np. | 17:09.25 |
tor8 | we've run into those INT_MAX numbers being truncated or overflowed before... what changed in the clamp commit to affect that though? | 17:10.06 |
Robin_Watts | I used fz_clampi rather than fz_clamp | 17:10.37 |
| I figured I might as well force the cast early rather than late. | 17:10.48 |
| and of course that failed. | 17:10.53 |
| casting after having clamped is safer as we know it can't over/underflow. | 17:11.08 |
| It's all on my master branch awaiting review (3 patches) | 17:11.29 |
tor8 | Robin_Watts: ah! | 17:12.53 |
ray_laptop | mvrhel: et al.: Do we want to bump the GS_CLIENT_COLOR_MAX_COMPONENTS up (to 64) ? And since the planar code breaks beyond 64, put in a #error if it gets set higher ? | 17:15.09 |
| now that we have performance reporting on regressions, we can see if it hurts in any measurable way | 17:15.46 |
Robin_Watts | ray_laptop: Possibly. Now that marcos has timings.... yes | 17:15.50 |
ray_laptop | I think mvrhel should weigh in on it, but I had just seen alexcher comment on bug 693185 | 17:16.41 |
Robin_Watts | So it looks like there are some problems at least. | 17:18.36 |
ray_laptop | Robin_Watts: I assume that getting past the 64 component max in the planar code isn't trivial. Is that correct ? | 17:19.37 |
Robin_Watts | It's not the planar code. | 17:19.52 |
ray_laptop | oh, sorry. | 17:20.06 |
Robin_Watts | Its the rest of gs that assumes that you need at least 1 bit per component in the index. | 17:20.13 |
| It *may* be easy enough to do, I've not looked. | 17:20.31 |
ray_laptop | Robin_Watts: OK. Thanks. | 17:20.42 |
Robin_Watts | The SEGV with it set to 32 is worrying. | 17:21.00 |
ray_laptop | Robin_Watts: oops -- I hadn't gotten to comment #3 in that bug. | 17:22.03 |
| but it may be related to the segv I am tracking down, although I only see a problem with NumRenderingThreads > 0 | 17:23.10 |
| I'm going to go ahead and clusterpush my changes to see if any SEGV's show up (with normal regression NumRenderingThreads=0) | 17:24.31 |
tor8 | Robin_Watts: yeah, I don't understand that if (i && clause either. maybe a remnant from when I tested without cx/cy against bx/by and only checked if it was the first element? | 17:33.14 |
Robin_Watts | tor8: I'd not put any money on it not being me that did it :) | 17:33.33 |
tor8 | there's a similar test at the bottom after the loop too | 17:33.40 |
| Robin_Watts: pdfwrite and clamp fixes LGTM | 17:34.44 |
Robin_Watts | tor8: Right. That one *has* to be there. | 17:35.01 |
| because we might have an empty path. | 17:35.11 |
| in which case i =0, and we don't want to trigger. | 17:35.22 |
| oh, but the other conditions are probably enough for that too. | 17:35.34 |
tor8 | they ought to be yes | 17:36.17 |
Robin_Watts | ok, I'll fix that too. | 17:36.28 |
| paulgardiner: mudraw mjs generation changes pushed to my forms branch. | 17:55.29 |
| mvrhel: For the logs: At the moment update_lop_for_pdf14 sets the pdf14 bit in the lop in certain conditions. The sole purpose of this, AIUI, is to make it so lop_is_idempotent tests fail. | 18:09.16 |
| Specifically this test is used while stroking to tell if we need to handle the whole path at once rather than one segment at a time. | 18:10.00 |
| One of the conditions is that "the blend mode isn't normal". | 18:10.14 |
| I reckon Darken and Lighten are idempotent. | 18:10.27 |
| so we can safely make the condition "blend mode != normal && blendmode != darken && blendmode != lighten" | 18:10.54 |
| That is sufficient to solve the PW-889.pdf bug. | 18:11.09 |
| I've cluster pushed and I get just a few 1 pixel differences here and there, which I think are all fine. | 18:11.34 |
| I'd like mvrhels blessing before I commit it though in case there is something smart I've overlooked. | 18:11.58 |
ray_laptop | Robin_Watts: sounds like a good thing to me (those blend modes are _way_ overkill, IMHO) | 18:22.43 |
mvrhel | Robin_Watts: I am back. Sounds good to me about the blend modes being idempotent | 19:56.15 |
| henrys: yes this customer (gemma) has taken a huge slice of engineering time | 19:58.33 |
| hopefully we are at the tail end of it all though | 19:58.40 |
Robin_Watts | mvrhel: Thanks. | 20:01.07 |
| mvrhel: Famous last words (Gemma) | 20:01.19 |
mvrhel | yes | 20:02.56 |
tor8 | Robin_Watts: I'll be away from the computer most of tomorrow. Is there anything you need from me now? I'll be reachable by text tomorrow if you need to get ahold of me. | 20:07.49 |
Robin_Watts | tor8: I don't think so. | 20:08.17 |
tor8 | Robin_Watts: bug 693187 looks like something for you -- transparency or knockouts would be my guess | 20:08.19 |
Robin_Watts | tor8: I'll take it, but I suspect I should be looking at why we get crashes with gs when the max number of components is pushed to 32/64. | 20:09.18 |
tor8 | if you're busy with gs, I can take a first examination to see what's going on. but if it's blending or knockout related I'll drop it like a hot potato :P | 20:10.13 |
kens | chrisl ping ? | 20:33.18 |
| For the logs then. I forgot that I'm taking the day off tomrrow. I'll send an email round in the morning. So I'll look into the ps2write thing on Friday | 20:35.07 |
chrisl | drat - too slow...... | 20:36.10 |
| oh well, off now, too | 20:36.21 |
mvrhel | Robin_Watts: are you there | 20:40.18 |
Robin_Watts | I am. | 20:40.29 |
mvrhel | simple question for you | 20:40.35 |
| with the overprint device, it would be nice to be able to do a copy of a single plane | 20:40.58 |
| we dont have that capability do we? | 20:41.08 |
| the copy planes proc will do all of them yes? | 20:41.28 |
Robin_Watts | mvrhel: yes. | 20:41.36 |
mvrhel | ok. I can work around this, but I just wanted to check if such a creature was available | 20:41.50 |
| basically what I have to do then is do a get bits | 20:42.22 |
Robin_Watts | yeah, not immediately. | 20:42.24 |
mvrhel | replace the plane that I want to change | 20:42.29 |
Robin_Watts | get bits? no. | 20:42.30 |
mvrhel | then copy planes | 20:42.37 |
| no? | 20:42.47 |
Robin_Watts | Oh, right, the problem is that you have to have data in all planes to write out. | 20:43.21 |
mvrhel | right | 20:43.26 |
Robin_Watts | and the best you can do is to write back what's already there. | 20:43.30 |
mvrhel | yes | 20:43.33 |
| exactly | 20:43.36 |
Robin_Watts | So yes, get_bits :( | 20:43.52 |
mvrhel | this happens in the overprint fill rect | 20:44.00 |
| and now in the overprint copy planes | 20:44.06 |
Robin_Watts | get_bits for each plane you don't want to change. | 20:44.15 |
mvrhel | fill rect hl color that is | 20:44.21 |
Robin_Watts | or get_bits for everything, then copy over the data you want to change, and write out might be more efficient due to get_bits being inefficient when getting a single plane sometimes. | 20:44.53 |
mvrhel | yes. and there is not a get bits to get all the planes also | 20:44.56 |
| I have to have this loop to get each one | 20:45.04 |
Robin_Watts | Fair enough. | 20:45.15 |
| I guess if you're using copy_planes you're going to a planar device, so get bits of a plane will be efficient. | 20:45.42 |
mvrhel | Robin_Watts: right. OK. just checking that my thinking was right. thanks | 20:48.55 |
| there is room for improvements in that flow, but overprint is not that common | 20:49.25 |
sebras | hm. I can't substantiate it yet, but I'm worried about putting fz_warn inside pdf_eval_function() friends. I'm thinking that we may end up with a HUGE number of warnings (possibly alternating ones for inputs/outputs that are not caught by the rate limiter). | 22:24.21 |
| whereas checking the number of arguments in pdf_load_function() is likely to happen much less seldom and so wouldn't cause the same issues. | 22:25.17 |
| Forward 1 day (to 2012/07/19)>>> | |