| | <<<Back 1 day (to 2014/08/31) | 2014/09/01 |
| leenasn | Had raised a question in Bugzilla about Adobe Illustrator feature "grouping" "ungrouping" as part of an issue http://bugs.ghostscript.com/show_bug.cgi?id=695451 | 06:30.19 |
| | The question is whether there is any specific setting in gs which can retain the "object group" in EPS [created using Adobe Illustrator] while converting it into PDF using ghostscript gs command | 06:31.53 |
| chrisl | leenasn: asked, and answered it appears | 07:11.41 |
| leenasn | chrisl: I am not sure whether I understood the answer | 07:14.01 |
| chrisl | leenasn: "No" is the most important part | 07:14.53 |
| | leenasn: PDF does not support that type of grouping (neither does Postscript) | 07:16.06 |
| leenasn | Thank you chrisl . Missed out the most important "no" | 07:25.23 |
| chrisl | leenasn: no problem | 07:26.53 |
| Mario_ | Hi! | 08:07.54 |
| | Is there any Ghostscript documentation re. how PDFs should ideally be prepared to work with the application? | 08:08.14 |
| kens | THere is plenty of Ghostscript documentation. Why does it matter how PDFs are prepared ? | 08:08.45 |
| Mario_ | Because we are getting some separations that are missing parts of the file | 08:10.14 |
| jogux | Hmm. the wiki seems to be dead. casualty of the casper upgrade I guess :-S | 08:11.06 |
| chrisl | Mario_: then it's either a bug, an error in configuration, or a PDF that is not constructed correctly according to the spec | 08:11.16 |
| Mario_ | KenKen I wrote you an email on Friday about that | 08:11.24 |
| kens | SOrry internet hiccuped. About what ? And what was the title of the email ? | 08:11.48 |
| Mario_ | Bug attachment access permissions | 08:12.15 |
| kens | Mario_ == Mariusz ? | 08:12.21 |
| Mario_ | yes :) | 08:12.25 |
| kens | I replied to the email I believe | 08:12.40 |
| Mario_ | yes | 08:12.45 |
| | you also asked about command input: | 08:13.25 |
| | gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=tiffsep | 08:13.26 |
| kens | I'm still waiting for the logs to catch up so I can read what I missed... | 08:13.27 |
| Mario_ | -r#{dpi} | 08:13.32 |
| | or I will just mail you that | 08:13.41 |
| kens | Well I always need the command line | 08:13.55 |
| | And the resolution is important | 08:14.06 |
| | But your files worked fine for me on a simple test | 08:14.24 |
| | I thought ghostbot updated the lgos every 5 minutes..... | 08:14.59 |
| kens | still waiting for the bit I missed... | 08:15.13 |
| | Aha, now it arrives | 08:15.22 |
| chrisl | Mario_: what version of gs are you using? | 08:15.38 |
| kens | Mario_ : your email said you were gettng errors, not missing output | 08:15.59 |
| Mario_ | Yes, there are actually two types of problems | 08:18.00 |
| kens | Well you know, if you only describe one problem to me, that's the only one I'll look at..... | 08:18.23 |
| Mario_ | Some files get processed without an error but some contents is missing | 08:18.24 |
| | probably because of transperency issues | 08:18.42 |
| kens | Maybe...... | 08:18.52 |
| | Without a command line, we can't investigate problems properly | 08:19.11 |
| Mario_ | email is comming | 08:19.19 |
| kens | OK | 08:19.23 |
| | Mario_ : it would be kind of useful to know which of the 3 resolutions you quote causes a problem, and with which of the 2 files you sent me. | 08:23.26 |
| | I hsould probably also mention at this point that free users don't qualify for support from us. We may look at problems, but its entirely at our discretion. | 08:24.38 |
| Mario_ | Ken, I understand, so thank you for you help so far | 08:26.22 |
| | We are looking in to getting a paid version as we are in contact with Scott | 08:27.59 |
| kens | OK. | 08:28.06 |
| | I'm just updating my 64-bit Linux VM, I previously used Windows. I need to do a clean rebuild, so this w2ill take a few minutes | 08:28.35 |
| farnoy | Hello Ken, I'm here if you have any questions. | 08:29.33 |
| kens | Well, it woul dbe good to know a precize command line which exhibits a problem. WHich file and which resolution. Also, for missing content, it would be nice to know which separation and what (approximately) is missing | 08:30.20 |
| kens | typing is not improving any :-( | 08:30.37 |
| farnoy | The line Mariusz mailed you is pretty much how we're using it. Besides that we also use icc profiles but for that file it doesn't make any difference iirc | 08:33.16 |
| kens | OK at 384 dpi I can reproduce a rangecheck but only with the tiffsep device | 08:33.19 |
| | farnoy the 'problem' with the command line is that it has 3 resolutions, ideally we want a *precise* command line,not one where we have to try three times ;-) | 08:34.06 |
| | On Linux that same file rangecheck's at 96 dpi, but *not* with the display device, so its something specifric to tiffsep it seems. | 08:34.36 |
| farnoy | I believe the problem happens on every resolution | 08:34.51 |
| kens | Seems to for me, yes. Like I said, when giving us a command line we ideally want just one that will go wrong. | 08:35.25 |
| | Interesting, the tiff device also fails | 08:36.15 |
| | Oh, no it doesn't, I just can't type | 08:36.26 |
| | Its defintiely specific to the tiffsep device. | 08:36.50 |
| | SO lets tgry it on WIndows | 08:36.58 |
| | Yep, also fails. At least I'll be able to look at it. | 08:37.57 |
| farnoy | Can you trace the error precisely where it's coming from? I tried but it's coming somewhere from the execution of a postscript program and I got lost. | 08:39.01 |
| kens | In a bit I will, yes. Its most likely generated by the device itself. | 08:39.26 |
| | The error is thrown in tiffsep_print_page() in gdevtsep.c, need to dig a little deeper | 08:42.02 |
| | Ah, there are 10 separations, chrisl do you remember the maximum spot colours these days in the default build ? | 08:43.46 |
| chrisl | It's more than 10.... I thought it was 32 | 08:44.15 |
| kens | Oh, can't be that then. Oh well.... | 08:44.25 |
| chrisl | Erm, well...... | 08:45.15 |
| kens | No, its not that, it opens all 10 output files | 08:45.26 |
| chrisl | Okay, it's just we have "GX_DEVICE_MAX_SEPARATIONS" and "GS_SOFT_MAX_SPOTS" | 08:45.50 |
| kens | Well, I have all 10 sep files and the composite file opened. rendering now.... | 08:46.14 |
| chrisl | The files being open may not be a useful indication, since that's managed by the device | 08:46.52 |
| kens | Yes indeed, have now also rendered each of the sep files though | 08:47.15 |
| | Moving on to the composite | 08:47.24 |
| | And that gets written too. Looks like hte only possible culprit is the downscaler. | 08:47.58 |
| | (only written one scan line so far, but it proves all files are written | 08:48.21 |
| | Yep, the downscaler throws an error | 08:48.51 |
| | On the 95th scan line | 08:49.07 |
| | Interesting, given that the scale factors are all 1..... | 08:50.38 |
| Robin_Watts_ | kens: open a bug for me? | 08:51.14 |
| kens | Its a clist problem Robin_Watts_ | 08:51.21 |
| | The error returns from clist_get_bits_rect_mt | 08:51.36 |
| | So I'll open a bug for Ray | 08:51.50 |
| Robin_Watts_ | OK :) | 08:51.55 |
| chrisl | reboots | 08:53.55 |
| Mario_ | kens Thank you for your support. I hope that this solves the issue :) | 08:57.36 |
| kens | Well, it'll need Ray to look into it, as I know nothing about the clist, and Ray is currently busy on another project. But he may well want to look at this for a break from his other stuff. | 08:58.16 |
| | Its definitely a bug though | 08:58.32 |
| | Mario_ : I've opened a new bug report here: | 09:07.06 |
| | http://bugs.ghostscript.com/show_bug.cgi?id=695459 | 09:07.06 |
| | You can track the bug by adding your email address to the CC list (Bugzilla doesn't seem to want to let me add you) | 09:07.06 |
| farnoy | It must have disconnected me, I'm using the Web client on the train. Do you need anything kens? | 09:34.05 |
| kens | farnoy : nope, there's a bug report opened | 09:34.25 |
| | It will need someone else to work on it though, not my area. | 09:34.38 |
| farnoy | I understand, thanks! | 09:35.18 |
| kens | Mario_ : has put his email on the CC list for the bug, so he can keep you up to date with it | 09:35.25 |
| tor8 | marcosw1: you around? | 09:52.35 |
| Robin_Watts_ | I think it's almost 3am at marcosw1's house :) | 09:54.34 |
| tor8 | Robin_Watts_: well, you never know :) | 09:58.24 |
| Robin_Watts_ | I'm fairly sure of the time (within an hour or so), but I agree you can never be sure marcosw won't be here :) | 09:58.57 |
| tor8 | just wanted to ask him if he was aware that git:// is broken | 09:58.58 |
| | he probably forgot to install that daemon | 09:59.07 |
| Robin_Watts_ | git-daemon-run is installed | 10:01.17 |
| | tor8: So, we need to revisit this strtod thing. | 10:06.02 |
| tor8 | Robin_Watts_: two problems with it in bugzilla | 10:06.14 |
| Robin_Watts_ | Firstly, Google fucked up, and that's a pain in the ass. | 10:06.18 |
| tor8 | one: lack of strtof on android | 10:06.23 |
| Robin_Watts_ | but I'd ignore it if it was just that. | 10:06.27 |
| tor8 | two: localization rears its stupid head again | 10:06.35 |
| Robin_Watts_ | The other is the fact that strtod... yeah. | 10:06.40 |
| tor8 | setlocale is the dumbest idea ever to make it into libc | 10:06.50 |
| Robin_Watts_ | locales in general, I think. | 10:07.04 |
| tor8 | sadly, we're just a library so can't just force setlocale("C") | 10:07.12 |
| Robin_Watts_ | I think I had a standalone fz_atof in the codebase at one point? | 10:07.42 |
| | Or was that you? Or did I dream it? | 10:07.50 |
| tor8 | *everything* is going to break, we depend on a lot of different atof/strtod/sscanf like things | 10:08.04 |
| Robin_Watts_ | There will be lots of tiny invisible changes to bitmaps, yes. | 10:08.29 |
| tor8 | Robin_Watts_: we might be better off to swallow the full dtoa.c source | 10:09.27 |
| Robin_Watts_ | From where? | 10:09.42 |
| | I rather assumed that any sensible atof implementation for a platform would be tuned to that platform, and would hence have assembly and heavy FP usage. | 10:10.39 |
| | but if there is a portable vanilla C version we can steal and cut down, great. | 10:11.00 |
| tor8 | Robin_Watts_: the original source from David Gay (like on http://netlib.org/fp/dtoa.c) | 10:12.44 |
| | it's icky and hacky and full of #ifdef voodoo though, so I wouldn't call it "portable vanilla C" | 10:13.13 |
| Robin_Watts_ | Can we take an (almost) unchanged version of that, and our own one, and allow people to pick? | 10:13.49 |
| tor8 | Robin_Watts_: soon we'll be shipping our own libc :) | 10:15.14 |
| | Robin_Watts_: might be possible to retro-fit the musl-libc version from here http://git.musl-libc.org/cgit/musl/tree/src/internal/floatscan.c | 10:16.03 |
| Robin_Watts_ | dtoa.c looks to me like it will at least be fast. | 10:16.48 |
| | so it's attractive in that regard. | 10:17.01 |
| tor8 | Robin_Watts_: yeah. it's the basis of all the open source libc's and probably most others too | 10:17.32 |
| Robin_Watts_ | Then probably we should go with it, cos it's had more testing than we can hope to throw at it. | 10:18.00 |
| tor8 | Robin_Watts_: I agree. | 10:22.40 |
| | this has caused enough trouble that I'm willing to give the finger to libc | 10:23.10 |
| | Robin_Watts_: this is really needed for mujs as well | 10:23.44 |
| Robin_Watts_ | mulibc ? :) | 10:24.00 |
| tor8 | so the question is where to put it (or just duplicate it for each project, it should be small enough) | 10:24.01 |
| | Robin_Watts_: don't tempt me :) | 10:24.07 |
| Robin_Watts_ | I'd duplicate it for now. | 10:24.34 |
| tor8 | Robin_Watts_: If you get a copy working for mupdf and I'll pull it into mujs | 10:24.57 |
| Robin_Watts_ | If we end up with more than one such function we can consider a musupport lib later. | 10:25.02 |
| | tor8: I'll stick it on my todo list, but SOT takes precedence for now. | 10:25.27 |
| tor8 | Robin_Watts_: we've survived for years without it, so I wouldn't call it high priority | 10:26.00 |
| | Robin_Watts_: http://plan9.bell-labs.com/sources/plan9/sys/src/libc/port/strtod.c is another portable variant | 10:28.20 |
| Robin_Watts_ | We really want strtof, not strtod, right? | 10:29.04 |
| tor8 | for mujs we need strtod | 10:29.26 |
| Robin_Watts_ | or does mujs want something different? | 10:29.28 |
| tor8 | but ideally, yes | 10:29.29 |
| Robin_Watts_ | right. | 10:29.30 |
| tor8 | for mupdf we only need strtof | 10:29.36 |
| | but I'll live with strtod and then casting to float | 10:29.45 |
| | Robin_Watts_: I use the plan9 dtoa in mujs, but haven't bothered sucking in the strtod. I could do that, run some tests and then we could see if that's useful for mupdf as well. | 10:33.28 |
| Robin_Watts_ | ok. | 10:36.21 |
| tor8 | Robin_Watts_: the compiled object files for the plan9port implementations of dtoa and strtod take 10kb | 11:11.19 |
| | so I think I'll be okay with duplicating them | 11:11.27 |
| Robin_Watts_ | and most of that 10k is probably symbols etc. | 11:14.39 |
| tor8 | Robin_Watts_: another big worry is all the printf's but we've got our own printf now at least | 11:16.16 |
| Robin_Watts_ | printf's where? | 11:16.43 |
| | in strtod etc? | 11:16.49 |
| tor8 | everywhere! especially in pdfwrite and pdfclean | 11:17.02 |
| | pdf_print_token, uses sprintf with %g which is going to break in odd locales :( | 11:20.40 |
| | plenty of stuff like that :( | 11:21.29 |
| Robin_Watts_ | oh, I see. | 11:22.00 |
| | yeah, we should move over to our new printf for everything like that. | 11:22.22 |
| sebras | d | 11:48.20 |
| | dis was a mistake. | 11:48.42 |
| kens | Hmm according to the reg, GoDaddy is having problems.... | 11:58.28 |
| chrisl | At least it's not just us, then! | 11:59.46 |
| kens | Hmm, well I have code which works for all cases except bugs692075.pcl. That file causes a call to pl_main_universe_select which deactivates the device (PCL device in this case I think) but does not close the gs device. This means that the font cache is freed. However, nothing sets it back up again, so when we get to the end of the job, it all falls apart for the same old reasons. | 13:02.14 |
| chrisl | kens: I'm not sure there is going to be an easy solution for this, because *once again* the PCL interpreter is, to my way of thinking, breaking the interface with the graphics library | 13:03.32 |
| kens | chrisl I believe it most certainly is. In this particular case there is nothign further I can do. The PCL interpreter has freed the font cache, but cartried on processing a file (I'm not exctly sure what's going on here). To my mind this means the code from this point on is running without a font cache, which seems suboptimal anyway, and thoroughly breaks pdfwrite. There is only one cluster file which exhibits this behaviour, so | 13:05.44 |
| | I'm going to pass this one back to Henry as teh PCL interpreter maintainer. I believe thbis is only possible to address in the PCL interpreter(s) | 13:05.44 |
| | I also believe its unreasonable to throw away big chunks of the ingterpreter before closing the device. I think that when finisihing and exiting closing the device should be the first thing which gets done. | 13:06.51 |
| chrisl | kens: if norbert is right, then this is the case where we have, probably, PCL5 inside a PXL stream? | 13:06.52 |
| kens | chrisl that's 'possible' | 13:07.01 |
| | But I'm unsure, and don't have the experience to tell. In any event, its not something I believe I can fix, I think its up to Henry to fix this one | 13:07.33 |
| | Certainly there is no way that pdfwrite can be modified to work with this, its important that the device is closed as one of the first things which gets done when finished with. | 13:08.11 |
| chrisl | In that case there is at least one mode where the PCL5 stream has to run in a complete "fresh" interpreter, and leave nothing (except the marks on the page) behind. So we effectively create a totally new PCL5 interpreter context, run the PCL5 stream, then destroy that context. | 13:08.32 |
| kens | Note that, as far as I can tell, once the cache is freed, nothing sets it back up again, so even if pdfwrite wasn't a problem, it seems to me that freeing the cache wuld cause a performance penalty as glyphs would (at best) then be uncached. | 13:09.17 |
| | chrisl, I believe you but, at present, it seems the font/glyph cache, and contents are not freed, and this does not seem to be a problem. | 13:09.45 |
| chrisl | No, because the "parent" PXL context, with its own font cache still exists. | 13:09.59 |
| kens | chrisl the font cache is not freed, that's my point | 13:10.26 |
| | But if the parent cache is still present, then we owuldn't have a problem in pdfwrite. | 13:10.46 |
| | THe problem in pdfwrite occurs when the cache is freed before the device is closed, because the font/glyph code wants to add the 'new' (copied) font to the cache, and there isn't one..... | 13:11.28 |
| chrisl | We would, because pdfwrite will access the cache via the font object, and the font object will point to the cache that disappeared | 13:11.30 |
| kens | The copied font isn't *in* a cache | 13:12.07 |
| | THat's the problem, when we get the glyph info from it the font code tries to add it to a cache | 13:12.25 |
| chrisl | So where does pdfwrite get the reference to the cache from? | 13:12.31 |
| kens | It doesn't | 13:12.37 |
| chrisl | I don't follow - it must be, at least indirectly, using a pointer to a cache | 13:13.36 |
| kens | It has a copy of the font which it tries to get glyph infor from, that causes the code to try to look up the fm pairs, and add the pair if not already present. But if there's no cacxhe, it seg faults trying. I presume the cache is being used form the pdfwrite device, or something | 13:13.42 |
| chrisl | *Where* is there no cache? | 13:14.07 |
| kens | I don't recall where the font code gets it from. | 13:14.27 |
| | I suspect ists form the pdfwritre device structure | 13:14.40 |
| | Which is why the device needs to be closed before the memory is discarded. | 13:14.57 |
| | Give me a minute and I'll put this back to reproduce the fault and see where 'dir' is loaded form. | 13:15.16 |
| chrisl | PCL, IIRC, never puts a reference to a font directory in the graphics library context (which, again, I think is incorrect) | 13:15.16 |
| kens | chrisl looks like it is getting it from the font copy | 13:17.35 |
| | So presumably whatever cache was present when the font was copied | 13:17.54 |
| chrisl | It will be the referenced by the original font | 13:18.14 |
| kens | Yeah that's what I mean | 13:18.27 |
| | So Henry can't go around throwing the cache away, any of them. | 13:18.41 |
| | Or we have to alter the font code to return glyph info woithout adding the font to a cache, or handle a cache 'dir' of 0. | 13:19.30 |
| | All of which soud bad to me. | 13:19.40 |
| | sound* | 13:19.45 |
| chrisl | It seems odd that pdfwrite would want to add a glyph to the cache, though...... | 13:20.08 |
| kens | It doesn't want to | 13:20.14 |
| | It calls font->procs.glyph_info | 13:20.37 |
| | THt is in this case gs_type42_glyph_info | 13:20.56 |
| | Which ends up in gs_type42_glyph_outline | 13:21.13 |
| | Which calls gx_lookup_fm_pair | 13:21.25 |
| | And if the cache has been freed, it seg faults | 13:21.36 |
| chrisl | Well, I can't immediately see a way to make the existing cache machinery handle the requirements of these PCL passthrough cases | 13:23.15 |
| kens | Well, we can't 'gsave' the cache(s) | 13:23.55 |
| | pdfwrite can't change the way it works for building font descriptors. | 13:24.46 |
| | SO the only alternatives are firstly to not free the cache (which how we work now) or modify the glyh_info code so that it doesn't try to retrieve the font/glyph form the cache, or add it to it. Which might have performance implications for 'normal' code. | 13:25.59 |
| chrisl | We could keep a list of glyph caches until the actual end of job, and free them then, but that could use up a *lot* of memory | 13:27.19 |
| kens | Presumably no more than its already using, at the moment we don't free the caches anyway | 13:27.43 |
| chrisl | Yes, but isn't that the bug? | 13:28.05 |
| kens | The patch in the bug report includes code to free the caches, and that's what causes the problem | 13:28.11 |
| | chrisl AIUI the bug is that it 'leaks' because that memory is *never* freed. | 13:28.34 |
| chrisl | I'd still be worried because it's completely open ended - there is no limit to the number of "passthrough" streams in a given job | 13:29.12 |
| kens | Quite so, but this is (from my limited understanding) no worse than we have now anyway. | 13:29.36 |
| | As far as I can see the PCL interpreter never frees the cache anyway | 13:30.01 |
| chrisl | True. But my feeling is we'd be better ensuring that the font's dir entry is nulled when the cache is freed, and handle that case in the cache code | 13:31.15 |
| kens | Well I'd be happy with that, but how do you plan to null the font's 'dir' entry in the pdfwrite copy of the font ? | 13:31.56 |
| henrys | oh god it's a holiday no gs font cache today ;-) | 13:32.22 |
| kens | hrnys then you better cover your easrs :-) | 13:32.38 |
| chrisl | It would need to be done with a "notification" method from the font directory. | 13:33.15 |
| kens | Right. wELL, IT MAKES SENSE.... | 13:33.35 |
| | SOrry, caps | 13:33.54 |
| chrisl | And that probably adding the entries needed for that in the font directory structure...... | 13:33.59 |
| kens | BTW that's not going to be enough | 13:34.04 |
| henrys | chrisl: absolutely incredible http://redrocksonline.com/concerts-events/detail/joe-bonamass-8-31-2014 -- never seen anyone play like that. | 13:34.09 |
| kens | gx_ttf_outline for example uses gs_currentgridfittt(pfont->dir) | 13:34.28 |
| chrisl | henrys: Joe B was almost certainly the best concert I've ever seen! | 13:34.58 |
| kens | Essentially our code assumes that any fopnt will always be associated with a valid cache (even if its 0 bytes I guess) | 13:35.07 |
| | OMG its worse than that, pfont is NULL here, I wonder why :-( | 13:35.45 |
| chrisl | kens: oh, how about we keep a list of the caches we've used, and free them at the end of job. And where the current patch frees the cache, we just empty it. | 13:36.12 |
| kens | Aha because append_outline_fitted uses the fm pair. So that's never going to work | 13:36.25 |
| chrisl | So we'll still accumulate memory from every cache, but only the "bare" cache, and no contents | 13:36.54 |
| kens | chrisl Yes I'd already changed the patch so that it emptied the cache, and only freed it when the device was released. | 13:36.55 |
| | chrisl I'm happy enough with that I think. We would need to be able to add to the cache, not merely have it empty, necause (see above) code relies upon using the fm pair entry | 13:37.47 |
| chrisl | kens: yeh, but it will only use the extra memory for pdfwrite, and not for any other devices | 13:38.16 |
| henrys | chrisl: albert hall? | 13:38.18 |
| chrisl | henrys: no, I went the year before that - Hammersmith Apollo | 13:38.40 |
| kens | chrisl yes, the only reason I mention it is because at one point while doing this I ended up with a valid cacge show maimum entry size was 0, and I don't think that will work with the TT code. | 13:39.04 |
| | sigh 'valid cache whose maximum' | 13:39.24 |
| chrisl | kens: it *really* should! | 13:39.39 |
| | But that's probably a separate issue | 13:39.54 |
| kens | chrisl if I can't add a fm pair, then append_outline_fitted won't work | 13:40.02 |
| chrisl | kens: then how did we render TTF glyphs with the cache disabled? | 13:40.48 |
| kens | chrisl did we not disable the glyph cache in that case, rather than the font cache ? | 13:41.09 |
| chrisl | kens: possibly.... can't remember | 13:41.45 |
| kens | Me neither.... | 13:41.52 |
| chrisl | henrys: so did you go to that JB concert? | 13:42.02 |
| kens | But its quite clear in gs_type42_glyph_outline | 13:42.04 |
| chrisl | kens: I think if we get around the crazy sh*t the PCL interpreter is doing, we can look at other issues as they arise. | 13:43.28 |
| kens | chrisl fair enough. Are you volunteering to take this on ? | 13:43.57 |
| chrisl | kens: I can certainly take a punt at it | 13:44.11 |
| kens | OK then I'll change the ownership to you. | 13:44.22 |
| chrisl | We can swap - I've got a bug to pass to you :-) | 13:45.07 |
| henrys | chrisl: yea red rocks is a great venue also. He comes here a lot. | 13:45.17 |
| kens | chrisl OK I'm taking that one anyway | 13:45.33 |
| | Give me a minute to finish writing on it. | 13:45.44 |
| chrisl | henrys: Cool, I envy you. I'd love to see him again | 13:45.46 |
| henrys | chrisl: if you've determined pcl is doing something crazy let me know. I'll read the logs too. | 13:46.05 |
| chrisl | henrys: it's the way it "manages" the font/glyph cache, especially for the PCL passthrough case | 13:46.48 |
| | henrys: the fact it creates and destroys a new cache for each PCL passthrough context is, I think, contrary to the design of the graphics library | 13:47.40 |
| henrys | chrisl: yes passtrhough was an absolute nightmare and I regret the way I did it. But really it is was just the stupidest move HP could have made. | 13:47.48 |
| chrisl | henrys: we're agreed on that! If you ever find out who at HP agreed to it, I'll hold him, while you hit him! | 13:48.35 |
| henrys | I remember when XL 3.0 came out and I read the spec, I just shrunk in disbelief | 13:49.40 |
| kens | The problem is that the font copy gets the same cache as the original font. So if you throw the cache away, then the font is now pointing to free memory. On top of that, the code which retrieves glyph information (eg widths) for a TrueType font requires that the font be in the font cache. So if our copied font isn't in the cache (which it won't be, its a copy) then we first add it to the cache. If the cache has been freed then it seg faults. | 13:49.48 |
| | Yeah, adding PCL5 pass through was just insane | 13:50.32 |
| chrisl | Oh, I may have another idea...... | 13:51.43 |
| kens | is intrested ? | 13:51.54 |
| chrisl | So, IIRC, the PCL interpreter *never* sets the font_dir pointer (or whatever the name is) in the graphics lib context...... | 13:52.30 |
| | What if PCL created a "sacrificial" font directory (which contains a cache) and put it in the graphics lib context, and pdfwrite *always* used the cache from the graphics lib for its copied fonts? | 13:53.42 |
| kens | Ugh, that means me figuring out all the places pdfwrite creates or copies fonts :-( | 13:54.12 |
| chrisl | Oh, I thought the actual copying was always done by the same routines :-( | 13:54.56 |
| kens | I think there are multiple ones for differetn font typoes, and then there are the cases where we make new fonts (certain types of CIDFont and many types of PCL weird font) | 13:55.36 |
| | Its not impossible, but it might take a while to track it all down | 13:56.13 |
| chrisl | Leave it with, and I'll have a poke at it | 13:56.31 |
| | s/with/with me | 13:56.39 |
| kens | Presumably it would be equally easy for pdfwrite to manufacture its own cache. | 13:57.10 |
| chrisl | kens: the difference would be that, with my proposal, Ghostscript would still behave exactly the same | 13:57.46 |
| kens | Mine too I htink, since all the changes wold be hidden in the device | 13:58.11 |
| | I'd just set the copied/manufactured fonts to use the pdfwrite cache. | 13:58.28 |
| | But yes, it might be problematic for rendering | 13:58.41 |
| chrisl | I was meaning that in gs, gs_font_base->dir == gs_lib_ctx->font_dir | 13:58.53 |
| | kens: neither idea should impact rendering at all | 13:59.09 |
| | In fact, yours could arguably have less, I suppose | 13:59.40 |
| kens | Not sure, in my case it would only affect the pdfwrite copied fonts, which should never be rendered anyway (I think) | 14:00.07 |
| | Hmm, looks like it might be easier than I though, seems we use gs_copy_font() throughout | 14:00.34 |
| chrisl | Yes, whereas my suggestion would have PCL create and destroy (but never use when rendering) an extra cache | 14:01.00 |
| kens | Well, either way we have an extra cache :-) | 14:01.21 |
| chrisl | Not with yours when we're rendering | 14:01.43 |
| kens | I guess that's so, I could set mine up during close_device | 14:02.04 |
| | Actually, no I couldn't, I'll need it when copying the fonts | 14:02.23 |
| chrisl | It's a bit swings and round-a-bouts: in my case, Ghostscript would behave exactly the same, but would mean PCL creating a unused cache when rendering. In your case, Ghostscript would behave differently (but should be invisibly so), but PCL wouldn't have to do anything extra...... | 14:04.11 |
| kens | PCL woudl still have to free its caches, which it doesn't do right now. But Henry's patch woudl work. | 14:04.46 |
| chrisl | If we go the route of a pdfwrite specific cache, I'd say it should be created at initialisation and destroyed at shutdown | 14:05.12 |
| kens | Indeed. THat's exactly what I was thinking | 14:05.36 |
| henrys | chrisl: the concert was a tribute to muddy water - and to hear you shook me covered on zeppelin's first album also was epic. | 14:05.44 |
| kens | I cna't find out where we create type 3 fonts at the moment :-( | 14:06.27 |
| chrisl | henrys: One of the first records I ever got was Muddy Waters - Live at the Newport Jazz Festival. It's still one of my favourites | 14:07.25 |
| henrys | chrisl: they were filming a PBS special at the concert, hopefully you'll get to see it. | 14:08.45 |
| chrisl | henrys: Even better, they might bring it out on DVD! | 14:09.30 |
| Robin_Watts_ | There is a company now that will do DVDs of a performance available to pick up at the end of the performance. | 14:10.38 |
| chrisl | It's probably not going to be terribly well mixed or edited, though | 14:11.35 |
| Robin_Watts_ | I would imagine not. | 14:11.56 |
| henrys | but sabrina and I will be on it screaming our fool heads off... | 14:12.20 |
| chrisl | henrys: I doubt they'd spend too long focussed away from the stage! | 14:14.06 |
| henrys | chrisl: seemed like the camera came by a lot but they may edit it that out. | 14:14.52 |
| chrisl | JB will be playing in London next March..... might just see about going :-) | 14:17.14 |
| | kens: so, if you want to use the pdfwrite specfic cache, it probably means changing the gs_copy_font() interface..... | 14:22.44 |
| kens | OK looks like I can ignore manufactured type 3 bitmap fonts, we never have a basefont for those, so a font cache is not required. I believe all others go through gs_copy_font or gs_copy_font_complete and all in a single routine in pdfwrite, so it *should* be possible to insert our own font cache right there. I'll need to check PCL stick fonts and other exotica but this looks plausible | 14:22.55 |
| | chrisl can't I just replace font->dir with something else on return from copy_font ? I haven't looked there yet | 14:23.31 |
| chrisl | kens: I suppose you could, yes | 14:24.18 |
| kens | That's pretty much what I was thinking of, but if there's a reason not to..... | 14:24.34 |
| chrisl | Just less tidy, I think. Maybe do a pdf_copy_font() which calls gs_copy_font() and then replaces the font dir? | 14:25.43 |
| kens | I'll need a copy_font_complete too I think | 14:26.11 |
| | For subset fonts it looks like we copy the original | 14:26.25 |
| chrisl | Well, you see what I mean: just to avoid in the future adding a call, and not realising the font directory needs replacing | 14:27.15 |
| kens | I was just going to hack pbftont->copied and pbfont->complete in pdf_base_font_alloc | 14:27.17 |
| | I can easily enough make a routine. I think I'd like to see it work first though ;-) | 14:27.58 |
| chrisl | Okay, do you want to run with this after all, then? | 14:28.36 |
| kens | If we do it this way its probably best if I do it :-( | 14:28.55 |
| | Where do we alloc the cache ? | 14:29.07 |
| | I need some code to copy ;-) | 14:29.13 |
| chrisl | Erm....... | 14:29.15 |
| kens | Definitely not going to be garbage collected memory either | 14:29.37 |
| chrisl | There should be no need for it to be | 14:29.52 |
| | gs_font_dir_alloc() | 14:30.22 |
| kens | Which C file is that ? | 14:30.34 |
| chrisl | base/gsfont.c | 14:30.55 |
| kens | got it, thanks | 14:31.01 |
| | Hmm, gs_font_dir_alloc2... So what happened to the original I wonder | 14:31.36 |
| chrisl | Oh, the usual probably! | 14:32.07 |
| kens | I guess it must be in a .h file, it gets called, but doens't seem to be defined | 14:32.39 |
| chrisl | Yes, it's a macro, gsfont.h | 14:32.59 |
| | #define gs_font_dir_alloc(mem) gs_font_dir_alloc2(mem, mem) | 14:33.11 |
| kens | Of course it is, not even slightly confusing at all | 14:33.16 |
| chrisl | We should just get rid of gs_font_dir_alloc() - seems pointless, now | 14:35.02 |
| kens | I have to agree | 14:35.10 |
| chrisl | gs_font_dir_alloc2() pre-dates the first commit into CVS in 1998!! | 14:36.07 |
| | PCL and XPS are the only uses of gs_font_dir_alloc() in our code base, too | 14:37.56 |
| kens | :-) | 14:38.14 |
| tor8 | Robin_Watts_: marcosw1: I fixed the git-daemon config (the --base-path argument in the launcher was wrong) so now git:// works | 14:43.36 |
| sebras | d | 14:45.50 |
| kens | sigh, as usual I've managed to pick a test file that doesn't work properly.... | 14:50.49 |
| | chrisl my quick hackery seems to work, surprisingly. | 14:57.18 |
| chrisl | kens: that sounds positive | 14:57.56 |
| kens | I implemented henrys patch, ran the file that caused a seg fault for me and it didn't seg fault. | 14:58.23 |
| | I verified it was going through henrys patch and freeing the cache, and also going through my code and replacing the cache | 14:58.42 |
| | Still work to do tidying up, but its better than I expected. | 14:59.02 |
| | And I don't actually have code to free the pdfwrite cache yet. | 14:59.28 |
| chrisl | One thing that vaguely concerns me is whether pdfwrite could get the kind of cache clash that the PCL code's cache juggling is there to avoid? | 15:00.45 |
| kens | Well, to be honest I don;t think pfdwrite actually cares about hte cache at all. | 15:01.07 |
| chrisl | No, you are probably right. Plus, it's not the glyph cache exactly, so it's probably fine | 15:01.53 |
| kens | Its only needed to satisfy the glyph_info calls. We don;t want the character bitmaps. But I guess you're right, if we resort to rendering glyphs that could be a problem. I think that owuld be extremely rare for pcl though | 15:02.07 |
| | Once I can figure out how to free the cache I'll try the cluster | 15:02.36 |
| | I guess I could steal Henrys code from the patch :-) | 15:03.03 |
| chrisl | I probably wouldn't be a huge deal to flush the pdfwrite cache after each font is flushed | 15:03.24 |
| | s/I/It | 15:03.29 |
| kens | Yeah, I guess tht would work, I'll see if it causes a problem first though I think | 15:04.02 |
| chrisl | No, again, I think you are right, if you get to that stage, there should be no reason to render the glyphs from PCL | 15:05.03 |
| kens | Hmm, I already have a 'pdf_free_font_cache', I bet it doesn't do what I want though..... | 15:05.28 |
| | And I was right. | 15:05.49 |
| | well, cluster test produced *lots* of seg faults | 15:29.21 |
| | LOL now MuDraw checks all the bytes in images and surprise surprise, its slower , well duh........ | 15:48.58 |
| rayjj | kens: I saw the bug report and the logs on the topic. I am able to see the rangecheck, and (with a debug build) I get a segfault in copy_separation_name duing tiffsep_prn_close | 15:50.12 |
| kens | Oh I did (once) get a seg fault, but I dismissed it because I couldn't reproduce it. | 15:50.36 |
| rayjj | kens: slower than gs, or just slower than previously ? | 15:50.39 |
| kens | rayjj its the support email, I'm guessing slower than MuDraw was before, not referenced to GS I guess | 15:51.02 |
| | I was just amused by them (only because its not my problem). Previously they didn't like it because it didn't check every sample of every image to see if it was actually gray. Now that ti does that, they don't like it because its slower thanit was. I guess we should be employing magic to make them happy. | 15:52.42 |
| rayjj | I was thinking that a random number guess might be faster than even opening the file ;-) | 15:53.55 |
| Robin_Watts_ | oh, joy. I will construct a reply. | 15:54.19 |
| kens | Does it start with 'Dear muppet' ? | 15:54.37 |
| chrisl | I feel like you just insulted Kermit, tbh....... | 15:59.04 |
| kens | 'Dear Fozzie' then ? | 15:59.39 |
| chrisl | Maybe go with a Fraggle? | 16:00.09 |
| kens | Jar-Jar binks ? | 16:00.33 |
| chrisl | Yep, that's appropriate - and just as annoying, too! | 16:01.02 |
| kens | :-) | 16:01.12 |
| Robin_Watts_ | Anyone here know how to get to the picas ftp site? | 16:03.21 |
| kens | No, I couldn't do it before. | 16:03.35 |
| Robin_Watts_ | I'll have to wait for marcos then. | 16:03.47 |
| rayjj | Robin_Watts_: just a sec... | 16:04.05 |
| | Robin_Watts_: email from Marcos to Sriran on 7/18 Subject: Re: FW: Performance related issue has the user and password that work for me | 16:07.48 |
| Robin_Watts_ | Ta. | 16:07.57 |
| rayjj | Robin_Watts_: that was cc'ed to support | 16:08.04 |
| | if you don't see it, we can go on private chat and I'll give it to you | 16:08.29 |
| Robin_Watts_ | I have it, thanks. | 16:09.48 |
| rayjj | Robin_Watts_: OK. If you can't get in, I can always grab the files for you and put them on casper | 16:11.07 |
| Robin_Watts_ | I am copying the first one now. | 16:11.31 |
| rayjj | Robin_Watts_: good (I guess) | 16:11.53 |
| Robin_Watts_ | hmm. bandwidth isn't great. I'm being predicted 6.5 hrs for 192Meg. | 16:14.48 |
| kens | OK enough, goodnight all | 16:23.48 |
| rayjj | Robin_Watts_: do you want me to benchmark mudraw -T vs. gs for the performance degraded. They mention the reason they are even looking at mudraw is that gs was slow -- so if mudraw is faster than gs, then the reason for sing mudraw is still balid | 16:28.25 |
| Robin_Watts_ | rayjj: Could do. | 16:29.40 |
| | We can possibly improve performance a bit. | 16:29.56 |
| | At the moment, we look at every pixel in an image to calculate the max difference before we bale out. | 16:30.21 |
| | We could check whether to bale out every 1000 pixels or something. | 16:30.32 |
| | For pages which have a single large image on that would be a saving on color files. | 16:31.02 |
| rayjj | Robin_Watts_: the 'trick' to get the per page color/mono status with gs was in an email from me on 6/18 to Sriram, Subject: Re: FW: Reg: Ghost Script Queries (in the attached zip is the "magic" args.cmd) | 16:31.05 |
| | Robin_Watts_: gs processes the entire image(s) as well, _and_ writes the clist :-( | 16:31.59 |
| | but at least with the 'args.cmd' we don't actually render the page | 16:32.58 |
| Robin_Watts_ | We could possibly pass the transform through the test device too. | 16:37.21 |
| | That way we could subsample images etc. | 16:37.33 |
| | so a color image might become greyscale through averaging. | 16:37.51 |
| | But that sounds like a far corner case. | 16:38.00 |
| | tor8: ping | 16:58.32 |
| | The first example from that customer shows a real problem. | 16:59.06 |
| | Page 1 of 50Mix.pdf is multiple jpeg2000 images tiled together to make 1 image. All of which give warnings from openjpeg. | 16:59.49 |
| | I don't care about the warnings. | 16:59.53 |
| | What *is* a problem though, is that because of the way we load jpeg 2000s, we always load and decode all the images on a page, even when we know that the page is greyscale. | 17:00.32 |
| | Any other type of image can avoid the decode for all but the first image. | 17:00.47 |
| rayjj | Robin_Watts_: how do you *know* the page is gray ? | 17:05.12 |
| Robin_Watts_ | s/greyscale/colour/ :) | 17:05.38 |
| rayjj | Robin_Watts_: since a JPX image can have a different colorspace than what's in the PDF Image dict | 17:05.46 |
| Robin_Watts_ | oh, yes... | 17:05.56 |
| rayjj | Robin_Watts_: OK, right. If you already know it's color, you can skip | 17:06.13 |
| | kens: (for the logs). I see the problem with Bug 695459 -- When processing an SMask, we change to monochrome colorspace (num_components == 1), but the color was written to the clist as DeviceN with 2 components (C and K). So the reading logic gets confused | 17:13.01 |
| | so now I have to track down where it's being written, and fix that. | 17:13.51 |
| Robin_Watts_ | ok, so not 192Meg. 1.8Gig. | 17:23.08 |
| tor8 | Robin_Watts_: yes? | 17:59.50 |
| Robin_Watts_ | See my burblings just after the ping. | 18:00.12 |
| | The first example from that customer shows a real problem. | 18:00.48 |
| | Page 1 of 50Mix.pdf is multiple jpeg2000 images tiled together to make 1 image. All of which give warnings from openjpeg. | 18:00.53 |
| | I don't care about the warnings. | 18:00.57 |
| | What *is* a problem though, is that because of the way we load jpeg 2000s, we always load and decode all the images on a page, even when we know that the page is greyscale. | 18:01.01 |
| | I've always been vaguely unhappy about our special cased handling of the jpx stuff. | 18:01.32 |
| | I suspect we need a new entry point for the openjpeg lib to be able to fix it though; something to decode just enough of the stream to know the colorspace. | 18:02.14 |
| tor8 | Robin_Watts_: we could set the ignore-image device hint once we detect that it is colorful | 18:03.38 |
| | that should solve the performance issue of decoding jpeg2k's | 18:03.48 |
| Robin_Watts_ | tor8: Ah! | 18:03.57 |
| tor8 | since then the pdf interpreter will bail early and not bother to even draw it | 18:04.22 |
| | which should save (a minimal amount) for other images as well that don't have the stupid icky I want to punch the person who wrote the spec special behaviour of jpeg2k | 18:04.58 |
| Robin_Watts_ | tor8: Much faster! | 18:10.38 |
| | tor8: Will commit that in a bit. got to run to the shop. | 18:12.07 |
| | tor8: 1 commit on robin/master then. | 18:29.04 |
| tor8 | Robin_Watts_: LGTM | 20:24.04 |
| | Robin_Watts_: a handful of commits for the strtod issue on tor/master | 20:24.22 |
| marcosw | tor8: sorry I didn't catch that, I did try a clone but it used the ghostcript.com:/home/git/ghostpdl.git address. | 20:48.23 |
| | Forward 1 day (to 2014/09/02)>>> | |
IRC Logs