| | <<<Back 1 day (to 2015/02/04) | 20150205 |
| jhabjan | kens: ping | 09:41.52 |
| kens | pong | 09:41.56 |
| jhabjan | if you recall the setdash from the other day | 09:42.54 |
| kens | Yep | 09:42.59 |
| jhabjan | well.. after explaining the problem to the user.. i just got response back | 09:43.34 |
| kens | OK | 09:43.38 |
| jhabjan | he claims it works with other devices | 09:43.48 |
| kens | Possibly it does, but that doesn't make it correct | 09:44.02 |
| jhabjan | I did try png256 device | 09:44.18 |
| kens | Acrobat for example is well known for ignoring illegal PDF files | 09:44.22 |
| jhabjan | and it does not report any errors | 09:44.28 |
| | png is produced | 09:44.33 |
| kens | Oh you mean other GS devices ? Really ? | 09:44.33 |
| jhabjan | yep | 09:44.37 |
| kens | I'm... surprised.... | 09:44.43 |
| jhabjan | only display device reports that error | 09:44.44 |
| kens | I can't think why that would be. | 09:44.56 |
| jhabjan | right, that's why I want to tell you that :) | 09:45.13 |
| kens | Best thing to do probably is open a bug report, I'm in the middle of something else at the moment | 09:45.16 |
| jhabjan | no worries | 09:45.23 |
| chris_away | jhabjan: what was the name of the PDF? | 09:47.58 |
| jhabjan | chrisl: https://www.dropbox.com/s/llv3e1duesous5z/i1.pdf?dl=0 | 09:48.48 |
| chrisl | jhabjan: that's odd - if I use -dNODISPLAY I get the "/rangecheck in --setdash--", but a whole bunch of other devices see to work fine..... | 09:51.58 |
| kens | Strange, probably the simplest thing is to break in setdash and see what's throwing the error. | 09:52.27 |
| jhabjan | yep | 09:52.43 |
| kens | I must admit I thought that [0] was illegal, because its setting both the 'draw' and 'gap' sizes to 0 | 09:53.00 |
| chrisl | Well, as the current code doesn't throw an error with any device I can see, I'm not inclined to spend much time on it | 09:53.17 |
| kens | The current code does drop the annotation as illegal though | 09:53.34 |
| | on the display device, I didn't try any others | 09:53.41 |
| chrisl | It does give a warning, I suppose, so..... | 09:53.46 |
| kens | Just did a quick check, on my branch code it doesn't throw an error with tiff24nc | 09:54.33 |
| jhabjan | http://bugs.ghostscript.com/show_bug.cgi?id=695815 | 09:56.06 |
| kens | OK I'll look at it when I have a moment free. | 09:56.22 |
| jhabjan | sure, no worries | 09:56.29 |
| chrisl | Baffling: the error comes from line 186 in base/gsline.c - and I see nothing there, or in the call tree, that is device dependent..... | 09:56.37 |
| kens | Well, that code looks like I'd expect. | 09:57.09 |
| | I'll reduce the file to just the annotation when I have the time. | 09:57.30 |
| | And I can have the PDF itnerpreter print the parameters its sending to setdash | 09:57.55 |
| chrisl | Duh, with ppmraw, zsetdash isn't being called....... | 09:59.18 |
| kens | Hmm, that's peculiar | 09:59.33 |
| | I htought we would draw the annotation at least. | 09:59.48 |
| chrisl | There is a redefinition of /setdash in gs_init.ps, but I'm not clear how that would be device dependent | 10:01.40 |
| kens | Its a Link annotation, so possibly we shouldn't be drawing it i any device (except pdfwrite) | 10:02.32 |
| | Hmm, looks like we do call drawborder for link annotations | 10:04.12 |
| | and drawwidget apparently | 10:04.22 |
| chrisl | With ppmraw, /annotvisible returns false so we don't try to draw it, but with -dNODISPLAY it returns true, so we do try to draw it | 10:06.47 |
| | Oh, ick, I see..... | 10:10.16 |
| | We decide whether or this is a "Printer" device depending on the existence of the "OutputFile" key in the page dictionary | 10:11.50 |
| kens | Well, I can't say I'm surprised | 10:15.20 |
| | Probably better to check if its a HighLevelDevice | 10:15.46 |
| chrisl | That would change the behaviour, though | 10:16.07 |
| kens | Yes, but its inconsistent now | 10:16.18 |
| | And thining about it, it will change pdfwrite too | 10:16.39 |
| | RIght now display, bbox, and maybe the inkcov devices won't render the annotations, which is wrong for the inkcov devices | 10:17.34 |
| | Its all pretty icky in there. actually | 10:18.22 |
| chrisl | jhabjan: so, it looks like, for now anyway, you can enforce consistent behaviour by adding to the Ghostscript parameters: -c "userdict /Printed true put" -f <input file> | 10:18.48 |
| jhabjan | chrisl: gr8, ill do that | 10:19.24 |
| kens | TBH doing different stuff based on whether its a printer looks wrong to me. THat would potentially mean that the bbox device would give incorrect results when compared against the rendering of the tiff device. | 10:19.48 |
| chrisl | That way the display device will behave the same as ones like png where these annotations are concerned | 10:19.48 |
| | kens: I'd agree with that.... | 10:20.53 |
| kens | Ugh, it seems to be something to do with annotationflags. | 10:22.21 |
| | Some annotations only redner if printed, not to a viewer (!) | 10:22.51 |
| | In any event, I think we should be consistent, no matter what the device. I'll change it at some point | 10:23.36 |
| chrisl | I still think it's a very poor way to tell whether it's a printer device | 10:23.38 |
| kens | Its a *terrible* way to tell..... | 10:23.50 |
| | A customer's device (for a printer) may well not have a /OutputDevice key set | 10:24.12 |
| jhabjan | chrisl: is userdict /Printed true put a same as: -dPrinted ? | 10:27.07 |
| chrisl | jhabjan: Erm, not quite, but the effect will be the same: -dPrinted I think will put the value in systemdict rather than userdict | 10:27.46 |
| jhabjan | ok | 10:28.04 |
| chrisl | I hadn't noticed it before, -dPrinted will be just fine | 10:28.16 |
| jhabjan | gr8 | 10:28.51 |
| kens | Huh, its even documented. We have too many options..... | 10:29.28 |
| jhabjan | both solution works .. with -c or -dPrinted.. thanks | 10:30.16 |
| chrisl | jhabjan: that should see you through until we can look at the setdash thing properly | 10:32.24 |
| kens | The setdash parameter is genuinely illegal. I could, of course, 'fix' it in the PDF interpreter | 10:33.03 |
| chrisl | I take it Acrobat just ignores it, as if setdash had never been called? | 10:33.43 |
| kens | Not absolutely sure to be honest, it either ignores the setdash or doesn't draw anything, I'd have ot reduce the file to check | 10:34.18 |
| | I think it simply drops the border silently though | 10:34.41 |
| chrisl | kens: Okay, if you want me to look at it, just let me know | 10:34.54 |
| kens | Not a problem, I'll get to it, just trying to sort out these forms | 10:35.10 |
| | Now that I've got rid of Zoltan again, for the moment | 10:35.23 |
| chrisl | Back to fonts hell..... | 10:35.31 |
| | Or possibly coffee first | 10:35.41 |
| kens | I have that yet to come :-( | 10:35.45 |
| | coffee, oh yes I knew I was forgetting something..... | 10:36.00 |
| jhabjan | any idea why tiff devices would not work when output is set to pipe ? | 12:22.17 |
| | other devices works well | 12:22.27 |
| kens | because they need to be able to seek in the output file | 12:22.42 |
| | I'm reasonably sure pdfwrite won't work if you try to pipe it, and it definitely won't if you try to Linearise the output | 12:23.31 |
| kens | lunches | 12:23.44 |
| jhabjan | pdfwrite device also works | 12:24.15 |
| chrisl | IIRC, pdfwrite "cheats" by writing the entire output to a tmp file, and then redirecting that the stdout | 12:24.57 |
| jhabjan | ok, got it | 12:25.28 |
| chrisl | We could do something similar for TIFF, but at the time it came up, no one had time to spend on it, so throwing an error when we couldn't write a valid file was the next best thing | 12:26.15 |
| | It also *can't* be made to work with tiffsep(1), so it's arguable we're better being consistent across all the tiff devices | 12:27.13 |
| jhabjan | no worries | 12:28.49 |
| kens | While its true that pdfwrite writes all the stream content and external resources to temporary files, and then creates the output file from those, I have a sneaking suspicion that there are some cases where it writes 'emoty space' into the file and then goes back and fills it in later when it knows what's supposed to be there. For Linearised output, it rewrite the PDF file completely *after* its been created, so that really won't work with a pipe. | 13:00.27 |
| chrisl | Yeh, I remember it *definitely* doesn't work for Linearised | 13:09.30 |
| hyper_ch | hi there, I have some issue with using ghostscript to combine multiple files and provide a layout with a pdfmarks file. It seems when I use: gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile="new.pdf" *.pdf "pdfmarks", then GS will use the existing pdfmarks in the last file selected by the glob and then add the info from pdfmakrs instead of replacing it | 14:21.15 |
| | so I wonder how I can remove existing pdfmarks | 14:21.37 |
| chrisl | You could possibly redefine pdfmark as a null op whilst processing the pdf file | 14:26.00 |
| | It would depend on exactly how pdfmark is incorporated into the pdf interpreter - a lot of stuff used by the pdf interpreter can't be redefined like that | 14:27.38 |
| hyper_ch | chrisl: not sure what you mean... I thought by providing the pdfmarks file it would use that and replace the stuff | 14:29.47 |
| chrisl | hyper_ch: no, that's not how it works at all | 14:30.10 |
| hyper_ch | well, what do you mean then by redefine pdfmark? | 14:31.20 |
| chrisl | pdfmark is (at least partly) implemented in Postscript, so it can be redefined | 14:31.48 |
| | You need to understand Postscript and some Ghostscript internals to do it, *if* it's possible at all | 14:32.23 |
| hyper_ch | what do you mean by that? | 14:32.27 |
| | I mean if I can tell it to add pdfmarks, why can't I tell it to remove existing stuff? | 14:33.31 |
| chrisl | I'm not aware of a pdfmark operation for removing stuff | 14:34.20 |
| hyper_ch | :( | 14:34.29 |
| chrisl | So, the only way to do it would to prevent the original pdfmarks (strictly speaking not pdfmarks, but that's another matter) from having any effect on the output | 14:35.53 |
| | But that could result in unwanted side effects | 14:36.09 |
| hyper_ch | and how would I achieve that and why isn't it pdfmarks? | 14:36.29 |
| chrisl | pdfmark is a (non-standard) Postscript operator, your input is PDF, so cannot actually contain pdfmarks. It *happens* we use pdfmarks to preserve certain metadata content from pdf input to output. | 14:37.36 |
| | But that metadata wasn't pdfmarks in the original PDF | 14:37.57 |
| hyper_ch | adobe calls it pdfmark... so if it's integreated in a pdf I fail to see why it's not pdfmark | 14:38.23 |
| chrisl | It's not integrated in a pdf, pdfmark is Postscript | 14:38.49 |
| hyper_ch | ok, you got a point there | 14:40.19 |
| | but still, how can I remove existing metadata then? | 14:40.40 |
| chrisl | As I said, there's a small chance you might be able to redefine the pdfmark operator to be a null op | 14:41.17 |
| hyper_ch | I don't know what you mean by taht | 14:41.31 |
| chrisl | Like I said, you'd need to understand Postscript and (some) Ghostscript pdf interpreter internals | 14:42.03 |
| hyper_ch | iieeks | 14:42.10 |
| chrisl | Even then, I'm not totally sure it would work.... | 14:42.41 |
| | hyper_ch: is this is one-off, or is this something you'll need to do regularly? | 14:42.59 |
| hyper_ch | regularly | 14:43.44 |
| chrisl | Like I said, even if it did work, you may well get unwanted side-effects - stuff going missing you don't intend | 14:44.38 |
| hyper_ch | maybe pdftk has a solution to remove metadata | 14:45.27 |
| chrisl | It may do - don't forget that Ghostscript's pdfwrite is intended to produce as accurate (in display terms) as possible a representation of it's input - it is *not* intended as a way to manipulate pdf files. | 14:46.47 |
| hyper_ch | ghostscript is great | 14:48.11 |
| | but I just need to remove the original info | 14:48.20 |
| chrisl | At what it does, yes | 14:48.37 |
| | And certain types of that kind of manipulation of the output content has become possible over the years, but it's not its core function, nor intent | 14:49.47 |
| hyper_ch | I found a way in pdftk to remove some metadata but not the bookmarks | 14:51.32 |
| chrisl | hyper_ch: aha, try doing "-dDOPDFMARKS=false" on your gs command line | 14:53.19 |
| hyper_ch | can it be that simple? | 14:53.40 |
| chrisl | Maybe...... | 14:53.52 |
| | That's why I said to try it | 14:54.00 |
| hyper_ch | let's try | 14:54.12 |
| | no, still has them | 14:55.16 |
| | gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -dDOPDFMARKS=false -sOutputFile="new.pdf" *.pdf | 14:56.18 |
| | gs v9.14 | 14:57.01 |
| chrisl | Oh, I see, that's really annoying - the code checks for the existence of the DOPDFMARKS key, but ignores it's value :-( | 14:57.18 |
| | hyper_ch: okay, try this: gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -dWRITESYSTEMDICT -sOutputFile="new.pdf" -c "systemdict /DOPDFMARKS undef" -f *.pdf | 14:59.35 |
| hyper_ch | no luck | 15:01.17 |
| | why did you add the -f ? | 15:01.54 |
| chrisl | When use use "-c" to inject Postscript, you need to add the -f to tell gs that the postscript has finished and the next argument is a file | 15:02.39 |
| hyper_ch | I see | 15:03.00 |
| | I think I have a work around | 15:03.26 |
| | I dump the data with pdftk | 15:03.54 |
| | then I remove all lines with Bookmark | 15:04.12 |
| kens | chrisl you can try redefining pdfmark as {cleartomark} | 15:04.26 |
| hyper_ch | then I use pdftk to update_info to create a pdf without bookmarks | 15:04.37 |
| | and finally re-use gs with the pdfmark file | 15:04.46 |
| chrisl | kens: yes, the problem is, the pdf interpreter may have already bound to the original definition before we get there | 15:05.08 |
| hyper_ch | it's not pretty but it can be done :) | 15:05.27 |
| kens | Hmm, possibly, I thought I'd used the construct, but of course I can cheat and stick it in gs_init | 15:05.37 |
| chrisl | kens: the problem is, I don't have a good test case that I can try, and know what to look for in the output to see if it worked | 15:06.15 |
| | It may be possible with -dDELAYBIND | 15:06.47 |
| kens | DOPDFMARKS is really intended to be set to true when you have a non-high-level device that you *do* want to be sent pdfmarks, rather than turning them off for high-level devices | 15:06.48 |
| hyper_ch | chrisl: you need pdf files? | 15:06.57 |
| chrisl | kens: Looking at the code, I see that now | 15:07.06 |
| kens | chrisl any metadata will do the job, so any file which has a Creator, Title etc. | 15:07.12 |
| chrisl | Okay, give me a few minutes, and I'll try it | 15:07.29 |
| kens | yes, sorry I wasn't around, my mother and sister are visiting..... | 15:07.40 |
| chrisl | OKay, so removing the metadata can done | 15:12.08 |
| hyper_ch | what's the magic word? | 15:12.34 |
| chrisl | -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile="new.pdf" -c "/pdfmark {cleartomark} def" -f *.pdf | 15:12.38 |
| rayjj | morning, all | 15:14.01 |
| kens | Morning.... | 15:14.08 |
| | rayjj do you have any idea what's causing the memory inflation with those files you posted ? Is there good reason to think its the PDF interpreter ? | 15:14.30 |
| hyper_ch | chrisl: that doesn't seem to generate a new.pdf file for me | 15:15.41 |
| rayjj | chrisl: Do we have a number (estimate pending fonts with more glyphs) for URW 136 OTC/CFF fonts (compressed with romfs) ? IIRC, you munged the fonts into CFF | 15:15.43 |
| hyper_ch | lol, my mistake | 15:16.00 |
| | s/gd/gs/ | 15:16.05 |
| chrisl | rayjj: no, I haven't done that yet | 15:16.10 |
| rayjj | kens: well, I know it's *not* the clist, and not openjpeg (since it is the same for luratech JPX), but I haven't looked further. | 15:16.40 |
| | kens: I did note (in the bug) that the worst file is designed for a larger page size and -dFitPage reduces it | 15:17.25 |
| kens | Well, I'm not at all sure how to go about investigating it..... It seems unlikely to be the PDF interpreter (unless it was something like creating a huge bitmap for a JP2K image) | 15:17.30 |
| hyper_ch | chrisl: great, works :) I just need to pass gs twice... first combine without any meta and then again to add pdfmarks | 15:17.48 |
| kens | The PDF interpretger doesn't create *that* much in the way of intermediate stuff, I didn't think | 15:17.53 |
| rayjj | kens: that's the one that is almost 900 Mb | 15:17.54 |
| chrisl | hyper_ch: yeh, I was hoping it might be viable to do it in one pass - still looking..... | 15:18.15 |
| kens | rayjj I don't really think a large page size and FitPage can be causing the problem, that just alters the CTM I think | 15:18.39 |
| hyper_ch | chrisl: it's already better than my approach using pdftk to extract metadata, use grep to filter out the Bookmark lines, use pdftk again to create a new pdf with no boomarks and then us gs to combine pdfmarks again | 15:19.08 |
| rayjj | kens: no, it just means that there is lots of stuff on the page. | 15:19.22 |
| kens | Well that is true, but there's very little that would be created persistently by the PDF interpreter. | 15:19.45 |
| | Fonts are one caus,e large images could be another, most everything else just marks the page. | 15:20.01 |
| hyper_ch | gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile="noBM.pdf" -c "/pdfmark {cleartomark} def" -f *.pdf "pdfmarks" | 15:20.03 |
| | gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile="new.pdf" "noBM.pdf" "pdfmarks" | 15:20.04 |
| | that works fine now :) | 15:20.08 |
| kens | rayjj I'll look at it when I finish this forms stuff | 15:20.59 |
| | Ah good, looks like I've solved getting the stored form marked as a resource used on teh current age. So, lots of tidying up to do in the PostScript, but first, lets check the original input file. | 15:23.54 |
| chrisl | hyper_ch: okay, I *may* have something that work for one pass, but the command line is a bit long so I'll use pastebin.... give me a sec | 15:24.08 |
| hyper_ch | geez :) | 15:24.48 |
| chrisl | hyper_ch: see if you can make sense of this: http://pastebin.com/QDygkuCi | 15:25.14 |
| | hyper_ch: you'll note there are now two "-c" options and two "-f" options | 15:26.10 |
| hyper_ch | it works... no idea why... but it does | 15:26.34 |
| kens | First he defines a 'holder' for the original definition of pdfmark, then he redfeines pdfmark to discard all the pdsfmark arguments and do nothing, then runs the PDF files. Then, using the stored definition, redefine pdfmark back to what it started as, then run your pdfmarks. | 15:27.54 |
| chrisl | kens types faster than I do..... | 15:28.13 |
| kens | But wiht lots of typos | 15:28.21 |
| chrisl | For the records, the command line is: -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile="new.pdf" -c "userdict /opdfmark systemdict /pdfmark get put /pdfmark {cleartomark} def" -f *.pdf -c "/pdfmark userdict /opdfmark get def" -f pdfmarks | 15:28.36 |
| | Just in case someone finds this in a search, follows the conversation, and finds an expired pastebin link | 15:29.23 |
| hyper_ch | chrisl: kens: thx, need to adjust my two script and push it to git | 15:30.01 |
| chrisl | hyper_ch: I'm glad (and surprised!) it worked! | 15:30.21 |
| kens | chrisl did all the hard work | 15:30.28 |
| chrisl | TBH, our pdf interpreter is renowned for *not* allowing that kind of flexibility...... | 15:31.02 |
| hyper_ch | https://github.com/sjau/pdfForts/blob/master/combinePDF/combinePDF.sh line 53 and https://github.com/sjau/pdfForts/blob/master/bookmarkPDF/bookmarkPDF.sh line 39 | 15:31.54 |
| chrisl | I love it when a plan comes together ;-) | 15:33.00 |
| hyper_ch | and a few people did actually try it: http://kde-apps.org/content/show.php/pdfForts?content=163700 | 15:33.03 |
| kens | Ecellent teh customer's file is now 4.3 Mb instead of 81 Mb and processes in ~16 seconds (same as Distiller) instead of 20 minutes. | 15:33.16 |
| hyper_ch | chrisl: second time you've helped me on that bundle ;) | 15:33.17 |
| chrisl | hyper_ch: glad to help | 15:33.46 |
| hyper_ch | it was nagging me | 15:34.09 |
| | I created those scripts because I make a lot of electronic submissions | 15:34.26 |
| | so I need an fast way to manipulate the pdfs | 15:34.40 |
| | stamping them for numbering, combining, exctracting pages etc :) | 15:34.54 |
| Robin_Watts | kens: Nice. | 15:37.24 |
| kens | I'm pretty pleased, just need to tidy it up and run a cluster psuh to make sure I didn't break anything | 15:37.47 |
| rayjj | kens: WOW! | 15:37.58 |
| kens | Well, almost all the content is forms, so if you don;t rerun the form PaintProc several times you get a lot of performance benefit ..... | 15:38.28 |
| | s/several/severalthousand/ | 15:38.41 |
| | Of course, I just broke the PostScript while cleaning it up. I wonder what I did wrong this time..... | 15:39.52 |
| chrisl | rayjj: for the rom footprint, how did you get the sizes of the individual resource directories? | 15:42.04 |
| Robin_Watts | freenode seems unhappy for me today. Anyone else seeing disconnections? | 15:46.12 |
| kens | Only seeing you disconnect | 15:46.27 |
| chrisl | It's been okay for me | 15:46.35 |
| kens | The few disconnects I've had have been my end | 15:46.58 |
| chrisl | I'm connected to leguin.freenode.net | 15:47.10 |
| hyper_ch | works all fine here | 15:47.43 |
| kens | No idea which server I'm on, is there a command for that ? | 15:47.50 |
| hyper_ch | sinisalo.freenode.net SSL | 15:48.18 |
| chrisl | Don't know, smuxi has a separate window with the server interactions in it - I looked there | 15:48.45 |
| Robin_Watts | none of my other connections die, so it's not my internet connection disconnecting. | 15:48.51 |
| hyper_ch | kens: I just checked in my server tab to which I connected | 15:48.55 |
| kens | Oh yeah I have a separate network windo, I'm on hitchcock | 15:49.28 |
| Robin_Watts | I've been kicked off 3 different servers so far I think. | 15:49.45 |
| hyper_ch | I connected through znc to sinisalo... not sure when it got the last time kicked there | 15:50.28 |
| chrisl | Robin_Watts: You're obviously just a trouble maker..... | 15:50.38 |
| Robin_Watts | chrisl: New lcms release. | 15:50.52 |
| chrisl | Robin_Watts: is that down to me now? | 15:51.09 |
| Robin_Watts | Actually, this is just rc1. | 15:51.26 |
| | chrisl: Dunno. | 15:51.33 |
| kens | chrisl who else is going to do it ? :-( | 15:51.52 |
| Robin_Watts | I haven't got bandwidth to look at it for the next few days at least. | 15:52.00 |
| chrisl | I've got it noted as being on mvrhel_laptop's desk | 15:52.07 |
| Robin_Watts | When are you planning the gs release? | 15:52.26 |
| chrisl | Next month | 15:52.36 |
| Robin_Watts | I think it's usually me that does it. | 15:52.41 |
| | ah, march? I may have time before then. | 15:52.48 |
| kens | After staff meeting I presume | 15:53.08 |
| chrisl | Yes, after the staff meeting. It's been in March since, erm, last March...... | 15:53.29 |
| | Robin_Watts: I can handle pulling in a new lcms release, it's not a big deal, but when we discussed it way back when, henrys wanted mvrhel_laptop to do it since he'd be best placed to pass judgement on the inevitable differences that arise | 15:56.01 |
| Robin_Watts | chrisl: Right. Our lcms has optimisations/customisations in from me. | 15:57.12 |
| chrisl | Robin_Watts: I thought they'd all been pulled into the "real thing"? | 15:57.36 |
| henrys | chrisl: inevitable? can we clusterpush it and see what we are in for, then decide? | 15:57.47 |
| chrisl | henrys: if there aren't any changes, why would there be a new release? | 15:58.44 |
| Robin_Watts | chrisl: Bugfixes, yes. | 15:59.29 |
| | optimisations, I don't think so. | 15:59.34 |
| chrisl | Oh, okay | 15:59.48 |
| henrys | progressions speedups ... | 16:00.07 |
| kens | My WIndows seems to be locking up, I think I'd best reboot while I can | 16:00.07 |
| chrisl | henrys: even a crash fix can result in rounding differences somewhere in the transform | 16:00.54 |
| rayjj | chrisl: I didn't get all the Resource directories since some are required (Init) or small. I did break out the CMap, CIDFSubst and Font directories since those are the big ones | 16:16.13 |
| chrisl | rayjj: I just have some sizes of the entire romfs object file with the 136 fonts in it. | 16:17.08 |
| rayjj | chrisl: For the potential customer, I'd like to get the estimate for the 136 CFF compressed (even if all the glyphs aren't there) and let them know that we will provide the updated numbers as soon as we get the remainder of the fonts with the extra glyphs | 16:17.34 |
| | chrisl: that'll work since I know how much the other stuff (besides the 35 fonts that I used) was | 16:18.12 |
| chrisl | So, with the fonts in pfb: gsromfs1.o was 11Mb | 16:19.10 |
| | Compression made no appreciable difference | 16:19.22 |
| | With the fonts in OTF/CFF: gsromfs1.o was 8.2Mb | 16:20.00 |
| | With the fonts in OTF/CFF compressed: gsromfs1.o was 7.1Mb | 16:20.25 |
| | With the fonts in Type 2: gsromfs1.o was 7.8Mb | 16:20.55 |
| | With the fonts in Type 2 compressed: gsromfs1.o was 6.8Mb | 16:21.18 |
| henrys | rayjj, chrisl:they've already licensed the fonts from URW so providing the font size estimate seems like a bad plan to me, because you may be way off, but whatever you want to do. | 16:24.30 |
| mvrhel_laptop | so what exactly do you need me to do with regard to lcms? You want me to bring in the new version? | 16:25.16 |
| | chrisl^^ | 16:25.21 |
| chrisl | henrys: how can we give them an estimate of the gs rom footprint without including the fonts? | 16:25.26 |
| | mvrhel_laptop: Robin_Watts mentioned to me there was a new lcms release pending, but I have lcms noted down as being yours to handle.... | 16:26.21 |
| henrys | chrisl: take the gs compile init size and subtract the fonts? | 16:27.12 |
| Robin_Watts | It's just rc1 at this stage. Proper release won't be for a week. | 16:27.20 |
| henrys | rayjj: didn't we agree to do with and without fonts ? | 16:27.44 |
| chrisl | henrys: but that's not going to be an accurate number - it'll be underestimated by the size of the fonts | 16:28.02 |
| henrys | chrisl: right and we will tell them to add their urw font release which we really don't know anything about. URW is not allowed to tell me. | 16:29.15 |
| kens | is puzzled why we spent time worrying about CFF/OTF/Type 1 fonts, if the customer isn't going to use the fonts we ship anyway | 16:29.46 |
| chrisl | henrys: they'll need to add it into our romfs to get a number, in which case, why did we bother with any of this exercise? | 16:30.13 |
| mvrhel_laptop | ok. I will look into it | 16:30.33 |
| | bbiab | 16:30.44 |
| henrys | kens: we know that they are getting the 136 he won't tell me specific glyph coverage or if there is more to the order beyond that like a CJK which could be quite a hit. | 16:32.22 |
| kens | Do we even know what format their fonts are being delivered ? | 16:33.10 |
| | Because if not, this exercise with OTF seems to have been somewhat redundant | 16:33.23 |
| | Good for future requests though | 16:33.40 |
| chrisl | So, the potential customer is going to have to repeat a bunch of stuff Ray has just done to arrive at their own numbers. | 16:34.23 |
| | kens: there's one thing (so far) about switching gs to OTF/CFF that worries me..... | 16:35.56 |
| kens | Just the one ? :-) | 16:36.09 |
| chrisl | We specifically check for Type 1 fonts, and load those into global VM | 16:36.20 |
| henrys | chrisl, rayjj: I don't see what we can do about that if they won't tell me what they are getting from URW. They've specifically asked for the ROM and RAM numbers and we are way overdue with an answer. Something needs to go out today. | 16:36.42 |
| chrisl | kens: At least three QL test files rely on that or they fail (not just fail tests, the files error out) | 16:37.51 |
| kens | Well, I guess we cna do the same for CFF font | 16:38.10 |
| chrisl | That's what I've got at the moment. | 16:38.25 |
| henrys | I'm already beating myself in the head for not making sure we had this available in the documentation - 2 weeks to get answers to this stuff is *really* bad for a new customer. | 16:38.53 |
| kens | chrisl I can't think of another solution, if the CET tests expect the fonts to be in global VM | 16:39.23 |
| chrisl | kens: But changing a whole class of fonts to be forcibly loaded into global VM may have unforeseen side effects | 16:39.34 |
| | henrys: as we can't give them accurate ROM numbers anyway, it all seems a little futile..... | 16:40.08 |
| kens | Probably, but no worse than changing to TT fonts would heva been..... And I can't believe its too terribly bad, since Type 1 fonts were already like that, and we mainly only see CFF in PDF files, whiich will be more tolerant | 16:40.23 |
| chrisl | kens: that's what I'm worried about: loading embedded CFFs into global VM | 16:41.10 |
| kens | Well, there's only really one way to find out isn't there ? :-) | 16:41.40 |
| chrisl | Remember, we rely on the fonts being restored away at the end of the page..... | 16:42.06 |
| kens | I'm going to assume the "33444 regression file(s) had differences but matched at least once in the previous 25 runs" is something to do with the font changes...... | 16:42.28 |
| henrys | chrisl: we are giving them accurate rom numbers ... they will respond that they aren't sure how to add in the font numbers and we continue. | 16:42.44 |
| kens | chrisl but surely that's no worse than the case we already have with type 1 fonts ? | 16:42.45 |
| chrisl | kens: we don't get type 1 fonts embedded in PDF files..... | 16:43.11 |
| kens | Really ? I thought we could | 16:43.22 |
| chrisl | Well, possible, but they're almost always Type 1C | 16:43.38 |
| kens | I'm reasonably sure some of our test files have type 1s and they 'seem' to work. | 16:44.05 |
| | I can't think of any solution other than 'put it in and we'll try and resolve any problems as they arise' | 16:44.28 |
| chrisl | I'm sure you are right, it's just a nagging doubt I have | 16:45.00 |
| kens | Well, we already have the ps2write problem to look at. But all of these are good reasons not to do this change right before a release ;-) | 16:45.30 |
| chrisl | There's been a few other pitfalls with the OTFs, but they seem to be working for non-high level devices, now | 16:46.05 |
| kens | Hmm, you have more problems than just the weird type 0 and ps2write ? | 16:46.32 |
| chrisl | One was that when we parsed a CFF font, we didn't mark the FontBBox array as executable - a couple of files failed on that | 16:47.19 |
| kens | Oh yeah, I remember that one, another daft requirement | 16:47.38 |
| chrisl | Another was that the font file pre-scan code returned a type of 42 for an OTF/CFF font, when it should be 2 | 16:48.19 |
| kens | Yeah I guess that is not too surprising | 16:48.33 |
| | I was thinkingof problems specific to high-level devices though | 16:48.54 |
| henrys | chrisl, kens : the rationale for the fonts is the compressed format and being able to build a language switch build that shares fonts. Clearly this is a very good change for us, I don't understand what you mean by an exercise. | 16:49.11 |
| chrisl | kens: I've not been testing high level devices with OTF (yet) | 16:49.43 |
| | I've been focussed on getting the interpreter end working | 16:50.02 |
| kens | henrys I understand the desire for a single font format. But if the customer isn't going to use the fonts we bundle, it seems we were 'panicking' somewhat needlessly over getting the OTF/CFF working. We can take our time | 16:50.05 |
| chrisl | And why the discussion couldn't have happened at the staff meeting, rather than over IRC | 16:51.13 |
| henrys | chrisl: in person meeting? | 16:52.15 |
| chrisl | henrys: yeh, I sort of felt there was quite a lot of confusion, given the heated nature of the argument, on IRC | 16:52.55 |
| rayjj | chrisl: I'd like to say: "the 136 fonts we have from URW are ___, and the compressed size after we put them in our ROM file system is ___" | 16:53.41 |
| | chrisl: can you give me those two numbers ? (just the romfs size with the other stuff is enough since I know how big that is) | 16:54.31 |
| chrisl | rayjj: the romfs sizes I listed above | 16:55.08 |
| rayjj | chrisl: but I want to make sure that you've enabled romfs compression for fonts (since the default build has fonts with '-b' | 16:55.13 |
| henrys | chrisl: as things are now I'll be flying to see the customer before the next meeting. If it could have waited I would have certainly put it off for in person discussion. | 16:56.01 |
| rayjj | chrisl: sorry -- I had a phone call and it was so far back, I missed it (since it wasn't tagged as 'ray'). I think I have what I need. | 16:56.37 |
| chrisl | henrys: but the customer won't be using our fonts so, as kens pointed out, what we ship is moot as far as they are concerned | 16:56.52 |
| | rayjj: do you want the size of the normal font files on disk? Or is that no necessary | 16:57.29 |
| rayjj | chrisl: but as long as we tell them what *we* got from URW and how we magically compressed it, then they can estimate based on their original font sizes | 16:57.47 |
| kens | rayjj not if their fonts a re a different format | 16:58.10 |
| rayjj | kens: I will be specific about that | 16:58.27 |
| | (about what our numbers are based on) | 16:58.41 |
| kens | If we want to be thorough we shoud quote all possible options; type 1, OTF, plain CFF etc | 16:58.57 |
| henrys | rayjj: indeed for the pcl fonts the romfs total and total for just a tar.gz of the fonts is very similar. They can simply compress their font set and be in the ball park right? | 16:59.29 |
| rayjj | chrisl: kens: I'll mention that "we started with Type 1 from URW, converted them (losslessly) to CFF, and then compressed the CFF" Does that sound accurate | 17:00.16 |
| kens | Nope | 17:00.25 |
| | As far as I'm aware we started with OTC/CFF from URW didn't we ? | 17:00.41 |
| rayjj | kens: I wasn't going to mention OTF since, iirc, chrisl said it didn't hurt much | 17:00.47 |
| | kens: I didn't think we had CFF for the 136 from URA | 17:01.10 |
| | URW | 17:01.13 |
| henrys | rayjj: I'm suggesting give them the numbers without fonts then tell them to tar.gz their release from URW and add it up. | 17:01.16 |
| kens | From chrisl's figures it seems that OTF added about .4 MB to the total | 17:01.17 |
| | I give up, I no longer recall what we had, or what we did, since I didn't do it.... | 17:01.41 |
| rayjj | kens: right -- not really serious amount | 17:02.10 |
| henrys | rayjj: objections? | 17:02.15 |
| rayjj | chrisl: is romfs compressed fonts total the same size as tar.gz of the set ? | 17:02.48 |
| chrisl | rayjj: the numbers I listed above were, indeed, for the 136 type 1 fonts, converted to OTF/CFF or Type 2 (CFF) | 17:02.49 |
| kens | Well, its 5% bigger. | 17:02.49 |
| chrisl | rayjj: I've no idea.... | 17:03.11 |
| rayjj | chrisl: can you do the tar.gz on your Resource/Font directory (if the 136 CFF) ? | 17:03.45 |
| chrisl | rayjj: that, or whole Resource directory? | 17:04.29 |
| henrys | rayjj: it is for pcl well 5.5 vs 5 meg romfs is a bit bigger. | 17:04.29 |
| rayjj | henrys: IME, compressing a tarball is different than compressing individual files as romfs does, but for fonts I have no idea | 17:04.30 |
| | chrisl: just the fonts | 17:04.42 |
| chrisl | rayjj: that comes to 2.6Mb | 17:05.06 |
| henrys | rayjj: I just gave you the numbers from pcl. | 17:05.27 |
| chrisl | It does seem to be a similar ballpark | 17:06.29 |
| | So, that seems to be Ghostscript with OTF/CFF fonts running the entire test suite to raster device(s) with no errors thrown..... | 17:08.05 |
| | I think that's fairly encouraging | 17:09.53 |
| henrys | chrisl: that is good really good | 17:10.25 |
| chrisl | I'll do a full test just to check that the high level problem we know about is the only one, and then I'll this to one side until after the release | 17:11.37 |
| henrys | chrisl: release after meeting right? | 17:12.08 |
| chrisl | henrys: yes. I'm not going to try to rush out even an RC before the meeting - especially with taking time off after the meeting | 17:12.52 |
| rayjj | chrisl: so the 6.8Mb replaces 5.4Mb I had in my build (for the entire romfs), so the 136 fonts should be (6.8Mb - 3.6Mb) = 3.2Mb (since the other stuff was the 3.6Mb) | 17:12.53 |
| chrisl | rayjj: so, how did you get to the separate numbers? | 17:13.33 |
| rayjj | chrisl: and the Type 1 set you started with (uncompressed) = (11Mb - 3.6Mb) = 7.4Mb | 17:14.16 |
| | chrisl: taking out the 3.6Mb that makes up everything except the Type 1 Fonts (1.8Mb) in my tests of "standard" builds. i.e., 5.4Mb - 1.8Mb = 3.6Mb (where the 35 fonts are 1.8Mb) | 17:15.27 |
| henrys | chrisl: I just wanted to make sure. I completely agree. | 17:15.33 |
| chrisl | rayjj: so did you just do a build with the font files removed to get the "everything else" size? | 17:16.46 |
| rayjj | chrisl: yes, but my 3.6Mb does include the DroidSansFallback as well as the CMaps and everything else | 17:18.02 |
| chrisl | rayjj: okay, thanks. | 17:18.19 |
| rayjj | chrisl: I've identified that the fallback font accounts for 1.8Mb | 17:19.04 |
| chrisl | rayjj: yeh, unfortunately, we can't switch to a different format for that, because I'm fairly sure it will break the CIDFont substitution if it's not a Truetype :-( | 17:19.57 |
| rayjj | chrisl: yes, but at least the DroidSansFallback does compress: len=3727996, compressed size=1877489 | 17:21.45 |
| | chrisl: hmm... I did make a mistake. The "everything else" is 3.9Mb *not* 3.6Mb, so the original 136 Type 1 is 7.1Mb and the compressed Type 2 is 2.9Mb | 17:23.41 |
| chrisl | Type 2 does seem to be the sweet spot for size | 17:24.30 |
| rayjj | which is still reasonable given your compressed tarball was 2.6Mb (just slightly better) | 17:24.35 |
| kens | henrys commit 17131c7a7fdf9d8537c4715e538c49b29c8945a8 should resolve the Forms problem raised by the customer if you want to let them know. | 17:25.11 |
| rayjj | chrisl: so all we (not me) have to do is to get PCL to work with the Type 2 fonts :-) | 17:25.17 |
| | kens: AWESOME | 17:25.28 |
| henrys | rayjj: I notice that adding up the compressed size of the pcl fonts individually as reported by romfs durinng the build is smaller than the total reported by romfs. AFAIK pcl only includes the romfs so that's something of a mysteryu | 17:25.53 |
| chrisl | kens: crikey, it's not nearly as horrific as I expected | 17:25.53 |
| kens | rayjj it wasn't so hard, fortunately I'd already done half the work just getting FOrms detected in high-level devices. But this way is *much* nicer, I'm quite happy with the result | 17:26.06 |
| | chrisl, I did say I didn't exepct it to be too bad. Though that's always a dangerous statement with pdfwrite involved.... | 17:26.48 |
| henrys | rayjj: nvm the I didn't count all the fonts hang on. | 17:26.55 |
| chrisl | kens: pdfwrite factor was what I was worried about | 17:27.12 |
| kens | Ah, I even remembered to add the customer number when I opened the bug, so that will get properly accounted for by Marcos. | 17:27.29 |
| | chrisl, yeah that was the worrying part ;-) | 17:27.37 |
| | Worked out not too abd in the end. | 17:27.52 |
| henrys | kens: you sure you don't want that to sit for a while or are you good with telling them now? | 17:27.55 |
| chrisl | Damn, there are some rendering differences with the OTF/CFF fonts I'll need to investigate :-( | 17:28.25 |
| kens | henrys I don't plan to review it, it works wellon their file and causes no problems with a regression test. FOrms aren't that common but we do have a couple of examples in the suite, so I reckon its as good as its going to get | 17:28.35 |
| henrys | kens: I'm thinking about the overnight and weekly testing marcosw does, but maybe that's irrelevant. | 17:28.36 |
| kens | henrys, I do;t mind waiting if you want to let Marcos' scripts bash on it. | 17:28.54 |
| | Tomorrow I'll try and look at these ATS files, but now, off to cook. | 17:29.37 |
| | Goodnight all | 17:29.40 |
| chrisl | I think I'll also finish now - I have to brave the supermarket..... | 17:33.58 |
| henrys | kens for the logs awesome speedup wow! | 17:36.49 |
| faLUCE | hello, pdftops -paperw 493 a.pdf <--- the output ps file doesn't have a 493 points width, why? | 18:12.46 |
| henrys | faLUCE: your best bet is to come back during GMT business hours and ask "kens" about that one. He may know off the top his head. Did you search stackoverflow? That's a good source for ghostscript answers too. | 18:28.14 |
| faLUCE | henrys: already googled | 18:28.36 |
| henrys | oh faLUCE pdftops is not ours - poppler. | 18:29.25 |
| faLUCE | henrys: anyway, how can I change the canvas size of a pdf with gs, without rasterizing it? | 18:30.00 |
| henrys | faLUCE: I don't think gs is what you want. ghostscript will completely decompose the pages and rebuild them. you might what to check pdftk. | 18:36.25 |
| Robin_Watts | henrys: If he's converting from pdf to ps, then he needs to completely decompose the pages, right ? | 18:42.18 |
| henrys | faLUCE: that said you can do something like: http://stackoverflow.com/questions/12675371/how-to-set-custom-page-size-with-ghostscript without knowing what exactly you want to do. | 18:42.37 |
| | Robin_Watts: that wasn't how I read his last question. | 18:42.48 |
| Robin_Watts | henrys: Ah. | 18:43.12 |
| | faLUCE: What is your input format? And what output format? | 18:43.34 |
| henrys | faLUCE: sorry didn't complete the sentence: without know exactly what you want to do kind of difficult to suggest other stuff. | 18:44.43 |
| | Robin_Watts: have you watched the Americans tv show... just started it, not bad. | 18:45.34 |
| Robin_Watts | henrys: Yes, have seen the first 2 series. | 18:45.46 |
| | The broadcaster has dropped it for S3 in the UK :( | 18:46.03 |
| henrys | Robin_Watts: you've seen everything... | 18:46.06 |
| | FX? | 18:46.22 |
| Robin_Watts | ITV | 18:46.28 |
| henrys | yeah FX has been demanding money from providers here some paid, others were dropped. | 18:47.29 |
| Robin_Watts | I think it was just a question of ITV showing it on saturday night here, and the ratings didn't justify the cost. | 18:48.14 |
| henrys | ah | 18:49.12 |
| Robin_Watts | There have been a lot of shows changing channels here recently. Supernatural was dropped by Sky and has gone to E4. | 18:49.46 |
| | so someone else may pick it up. | 18:49.51 |
| rayjj | henrys: I just send the "cleaned up" and hopefully more understandable email for final review -- let me know if you have questions and if you want me to send it. | 18:51.22 |
| henrys | rayjj: yes reviewing now | 18:51.33 |
| rayjj | henrys: If you want to add/edit it and just send it, please feel free | 18:51.39 |
| | (I sanitized out the potential customer name) | 18:52.07 |
| henrys | rayjj: the only problem I see is you would never get pcl by itself with 136 fonts, but I think this is what we need to get the ball running. Send it privately once we get some feedback (if we do) we'll bring in support. | 18:55.59 |
| faLUCE | henrys: nm, np, solved with pdftops | 18:56.23 |
| henrys | faLUCE: okay | 18:58.20 |
| rayjj | henrys: I thought the 136 was a superset of the 80 that we usually ship | 19:05.54 |
| | henrys: or are you saying that we should have stripped it to just the 80 ? | 19:06.29 |
| henrys | rayjj: it is but if you were making a pcl-only printer you wouldn't need 136... but like I said it's really fine, let's just get them talking to us. | 19:06.56 |
| rayjj | henrys: I'll add a comment in the PCL section that if they include less than the 136, for example fonts corresponding to the 80 that is usually seen on PCL only printers, the total will be reduced. | 19:08.23 |
| henrys | rayjj: great | 19:08.40 |
| rayjj | henrys: I'll cc you and Miles. Scott also ? | 19:12.24 |
| henrys | rayjj: yup | 19:12.37 |
| rayjj | OK | 19:13.02 |
| | henrys: sent. FINALLY! | 19:14.02 |
| | henrys: the nice thing is that the sanitized version is pretty much usable as is for others (the one I cc'ed to tech) | 19:15.13 |
| henrys | rayjj: https://www.youtube.com/watch?v=lg4WU-iuETw | 19:21.27 |
| rayjj | henrys: ha ha | 19:27.35 |
| | I have to run an errand. bbiab | 19:27.55 |
| mvrhel_laptop | henrys: | 22:04.31 |
| | the stuff with Stefan is going to take me a bit longer than I though | 22:05.33 |
| | thought | 22:05.35 |
| | hopefully just a few more days | 22:05.46 |
| rayjj | mvrhel_laptop: not that you need to get involved in it, but one of the really large RAM usage seems to be due to ICC profiles being collected (bug 695813). Either kens or I will track it down. | 22:06.58 |
| henrys | mvrhel_laptop: take your time with that I would like the CM to be good and they can really exercise it. Good use of time IMHO | 22:08.11 |
| mvrhel_laptop | ok. that is good to know | 22:08.27 |
| | rayjj: weird | 22:08.53 |
| rayjj | mvrhel_laptop: I thought "that is good to know" was a response to me saying "not that you need to get involved" ;-) | 22:09.25 |
| mvrhel_laptop | the good to know was to henry | 22:09.45 |
| | the weird was to you | 22:09.58 |
| | I wonder why they are not getting released | 22:10.08 |
| | over 1000! | 22:10.27 |
| | somewhere there is an rc_decrement missing | 22:10.55 |
| | does the file have transparency by chance? | 22:11.09 |
| | anyway I wont get involved, but I will be curious to hear what you find rayjj | 22:11.37 |
| | back to deviceN fun | 22:11.52 |
| | doh. I see what is going on with this now | 22:34.08 |
| | the alternate color space is getting invoked even if the direct replacement has happened. Should not be too hard to make this work the way we want | 22:35.29 |
| | famous last words when playing with DeviceN colors | 22:36.49 |
| | Forward 1 day (to 2015/02/06)>>> | |
IRC Logs