| | <<<Back 1 day (to 2014/11/12) | 20141113 |
| rayjj | hi, marcosw -- Are you testing the cluster ? I see jobs coming and going for you | 04:41.23 |
| | I wouldn't usually notice, but I'm just watching my jobs go through | 04:42.00 |
| matu | Hello | 08:41.11 |
| ghostbot | Welcome to #ghostscript, the channel for Ghostscript and MuPDF. If you have a question, please ask it, don't ask to ask it. Do be prepared to wait for a reply as devs will check the logs and reply when they come on line. | 08:41.11 |
| matu | I'm using trying to use ghostscript with a automated printing solution | 08:42.12 |
| | C:\Program Files\gs\gs9.15\bin\gswin64c.exe" -dNOPAUSE -dBATCH -sDEVICE=ljet4 -sOutputFile="%printer%hp" C:\www\Litsentskawe.pdf | 08:42.16 |
| kens | matu and ? | 08:51.46 |
| matu | i can't user newer printres | 09:05.54 |
| | *printers | 09:05.59 |
| | nothing is coming out | 09:06.05 |
| kens | Sounds liek the printer isn't compatible with LaserJet 4 language. | 09:06.41 |
| matu | yes | 09:06.48 |
| | is there a way to update built in devices? | 09:06.59 |
| kens | Sure, download the source, alter the device, and rebuild | 09:07.17 |
| | You could also try the pjxl device, but as far as I know, the ljet4 device works with modern printers | 09:08.18 |
| matu | ok | 09:44.04 |
| | will try | 09:44.06 |
| | thank you! | 09:44.08 |
| kens | Our PCL expert will be online in a few hours you could come back and ask again then. Probably in around 4 hours or so | 09:45.00 |
| matu | perfect! | 09:45.25 |
| tor8 | Robin_Watts: gah. I regret caving to your suggestion to make fz_matrix and fz_rect pass by pointer... it makes writing code so much more tedious, having to always create temporary storage to mutate them | 10:08.29 |
| Robin_Watts | tor8: code is better performing for it though. | 10:15.21 |
| tor8 | possibly, the original pre and post measurements weren't reproducible were they? | 10:16.57 |
| | still, not significant enough to warrant the inertia of writing code using the new api:s, IMO | 10:17.13 |
| Robin_Watts | I think there is a discipline that comes from maintaining your own storage for stuff, rather than forcing the compiler to repeatedly copy structs. | 10:35.33 |
| | And my memory is that the performance difference was clear, especially on ARM. | 10:35.52 |
| paulgardiner | Not sure I get that. I wasn't aware of any advantage other than performance. | 10:39.27 |
| tor8 | in the end it's storage on the stack no matter whether it's an auto variable or a parameter | 10:40.17 |
| paulgardiner | ... but then I'm increasingly moving to the "don't use C unless you have to" camp. | 10:40.23 |
| Robin_Watts | Yes, it's a pure performance thing. It's malloc vs gc all over again. | 10:40.36 |
| nsz | paulgardiner: why not? | 10:40.46 |
| | Robin_Watts: why again? | 10:41.27 |
| Robin_Watts | nsz: Why is malloc better than gc? | 10:42.07 |
| tor8 | Robin_Watts: at least it's not like java where every single matrix or rect would have to be heap allocated and garbage collected. | 10:42.12 |
| Robin_Watts | It's horses for courses. gc makes it very easy to produce programs - it removes lots of worries about remembering to deallocate etc. | 10:42.49 |
| nsz | i mean it was always like that | 10:42.52 |
| | at least i didnt notice significant changes in the malloc vs gc thing for a long time | 10:43.11 |
| Robin_Watts | But it's necessarily inefficient. | 10:43.20 |
| paulgardiner | nsz: because it is hard enough to get things right even when you aren't worrying about where things are stored, plus, although you can do OO in C, it is really nice to have language support for it and lastly having blocks (or closures) is just wonderous | 10:43.25 |
| Robin_Watts | nsz: yes, I'm not claiming any recent changes in this stuff. | 10:43.31 |
| nsz | ok | 10:43.44 |
| tor8 | malloc vs gc only really matters when you can't have the periodic performance stalls while the gc runs | 10:44.03 |
| Robin_Watts | paulgardiner: All of which are arguments for convenience over performance. And all valid in that context. | 10:44.07 |
| | but we want mupdf to be performant. | 10:44.22 |
| nsz | paulgardiner: closures are evil in libraries that can do lowlevel things | 10:44.30 |
| | actually any callback is | 10:44.35 |
| tor8 | i.e. video and audio playback if you're cpu limited, and all animated graphics | 10:44.46 |
| | but we've never considered gc for mupdf, being C and all | 10:45.17 |
| jogux | tor8: Apple got some massive performance gains switching OS X apps from GC to automatic reference counting iirc. like performance of some tasks in xcode got 10 times faster. | 10:45.26 |
| paulgardiner | Robin_Watts: yeah I don't think I'm disagreeing with you (I rarely do). I was just wondering what you meant by "there's a discipline"... if there was something else that was good about arranging storatge yourself | 10:45.32 |
| nsz | my position is that you must use c if you want to write a library that solves a single problem that will be widely used | 10:45.35 |
| tor8 | jogux: sounds backwards to me... ARC needs to touch memory -all- the time a pointer is copied | 10:45.55 |
| | unless they're smarter about it than a naive ARC approach would be | 10:46.07 |
| nsz | gc touches memory all the time too, no? | 10:46.22 |
| jogux | the compiler does a lot of static analysis AIUI. | 10:46.24 |
| paulgardiner | nsz: yes very true. Although I don't use C unless I have to, I accept it is very often the best language to use | 10:46.54 |
| jogux | [it also often outperforms manually managing memory, at least if you're using autorelease pools, which are pretty much unavoidable] | 10:46.55 |
| Robin_Watts | paulgardiner: A mental discipline. It's a certain amount of pain for a certain amount of benefit. | 10:47.02 |
| tor8 | nsz: only at periodic intervals; gc doesn't dereference the pointer to read/write a reference count every time the pointer is accessed | 10:47.03 |
| | jogux: autorelease pools are a fantastic idea; probably the only thing I like about ObjC :) | 10:47.38 |
| jogux | :-) | 10:47.47 |
| Robin_Watts | tor8: depends how you implement gc. | 10:47.49 |
| paulgardiner | Robin_Watts: right | 10:47.57 |
| Robin_Watts | When I did the actionscript engine for the picsel flash player, that had gc in it. | 10:48.11 |
| paulgardiner | tor8: nothing else you like about objC? | 10:48.26 |
| Robin_Watts | It was a combination of reference counts and then mark/sweep to remove cycles. | 10:48.26 |
| | the cost of the reference counting was more than made up by the fact that mark/sweep had to do so much less. | 10:48.49 |
| tor8 | paulgardiner: the selector dispatch mechanisms are neat, but not enough to make me want to use the language | 10:48.53 |
| | Robin_Watts: my xhtml+css mupdf module is at the stage where it can be used to read HTML documents! | 10:49.55 |
| paulgardiner | tor8. I like having blocks. I like the way it makes reference counting trivial (although gc would make that unnecessary). I like the way it makes accessors so low maintenance | 10:50.21 |
| Robin_Watts | Wow! Nice one. | 10:50.23 |
| tor8 | with proper margins, padding and text-indent | 10:50.23 |
| nsz | hm i assumed ref counting could elide a fair amount of ref count modification by 'borrow without taking ownership' semantics | 10:50.56 |
| tor8 | paulgardiner: I'm just not into the object oriented programming style, so most of what objc gives you I don't care about :) | 10:51.01 |
| paulgardiner | tor8: that's astounding | 10:51.01 |
| nsz | and then the memory access should not be that bad | 10:51.12 |
| tor8 | paulgardiner: but yes, blocks are neat for multi-threading, I'll have to give them that | 10:51.17 |
| paulgardiner | I find blocks useful for all sorts of things | 10:51.47 |
| nsz | well it's ok if you dont use it in apis | 10:52.25 |
| tor8 | and font styles and sizes, etc | 10:52.39 |
| jogux | henrys: I have a skype question about newsletter when you have a minute :) | 12:41.49 |
| henrys | open source .net ... huh | 15:30.07 |
| kens | When you cna't beat htem.... | 15:30.35 |
| Robin_Watts | henrys: eh? | 15:33.14 |
| kens | MS is open sourcing .NET | 15:33.26 |
| | http://www.theregister.co.uk/2014/11/12/release_microsoft_net_from_its_windows_chains_mono_and_xamarin_guy_miguel_de_icaza_on_open_source_net/ | 15:33.45 |
| Robin_Watts | open sourcing. will that lead to cross platform support? | 15:33.47 |
| jogux | they're putting some of it into mono apparently. so yeah. | 15:33.59 |
| kens | WIndows Linunx and OS/X yes | 15:34.02 |
| Robin_Watts | So, how does that leave mvrhels windows gsview? | 15:34.32 |
| | The reason that wouldn't work cross platform was cos of the lack of .net on linux/macosx. | 15:35.09 |
| rayjj | such as can we have a Windows style UI for linux users -- they'll love that :-/ | 15:35.26 |
| | mvrhel_laptop: were your earrs ringing ? | 15:36.06 |
| mvrhel_laptop | uhoh | 15:36.14 |
| rayjj | Robin_Watts and henrys were just mentioning .NET on linux and OS/X -- so we could use your gsview there | 15:37.00 |
| | (oh, and kens actually posted the URL) | 15:37.34 |
| kens | I posted *a* URL, there are several | 15:37.45 |
| mvrhel_laptop | right | 15:37.50 |
| kens | THe Reg had about 3 articles on it yesterday, that was just the 1st one I found | 15:38.21 |
| henrys | well they've open sourced it... I imagine it'll be a little while before we see anything working on linux or osx ... | 15:38.45 |
| mvrhel_laptop | yes | 15:38.55 |
| rayjj | so MS is probably going to be hiring some people with open source and linux experience :-) | 15:38.58 |
| henrys | rayjj: they are supposed to be assisting the mono group | 15:39.25 |
| kens | Another link: | 15:41.14 |
| | http://www.theregister.co.uk/2014/11/12/microsoft_to_open_source_dot_net/ | 15:41.14 |
| rayjj | chrisl: we probably had this discussion (at least two times before), but I'd like to bring it up again -- For cust 532, we are loading the font once per page, even though every page uses the same object | 15:42.53 |
| | chrisl: obviously, the reload is due to the "PDFSave restore" at the end of each page that causes us to "forget" | 15:43.27 |
| chrisl | rayjj: yes, we can't safely hold onto the fonts between pages | 15:44.19 |
| henrys | this -20 C stuff is getting old fast... | 15:45.11 |
| rayjj | I removed the PDFSave save ... PDFSave restore and even on my laptop it is measurable on this 16p test file (> 10%) and on the full PLRM-3rd it results in 1.8x parsing improvement. | 15:45.12 |
| chrisl | rayjj: we have at least a couple of files in our regression tests that will fail to render correctly with that change | 15:45.56 |
| kens | But is that any use if the output is incorrect ? | 15:45.58 |
| | Fast but wrong is not better than slow but correct | 15:46.15 |
| rayjj | chrisl: actually, I ran a regression and it only showed 9 pages with minor differences | 15:46.29 |
| | and that's with the "brute force" approach that just zaps the save...restore | 15:47.11 |
| chrisl | rayjj: I can assure you, it will cause problems | 15:47.24 |
| Robin_Watts | "We can't safely hold onto fonts between pages" - why is that? Is that because of the architecture of the PDF interpreter? Is there something that could be done within the interpreter to make it safe? | 15:47.36 |
| kens | I no longer remember the details of this, if someone can dig it out of Git I'll look again | 15:47.59 |
| rayjj | chrisl: I understand font names aren't useful, but PDF objects s/b safe, right ? | 15:48.08 |
| kens | The commit that changed it I mean | 15:48.09 |
| chrisl | rayjj: it stems from the fact we load font resources by name, and font names in PDFs are not sufficiently unique | 15:48.22 |
| rayjj | kens: dig "it" out of git ??? | 15:48.24 |
| kens | Find the commit that added the page level save, I can't remember why it was done | 15:48.44 |
| | Presumably you oculd use Git blame since you have the line number | 15:48.57 |
| chrisl | "PDF objects s/b safe" - huh? | 15:49.00 |
| rayjj | chrisl: for PDF font resource objects, we keep the PSFont in the object dict and re-use it during the page | 15:49.06 |
| kens | We don't load PDF fonts by 'object' we load them by name | 15:49.18 |
| Robin_Watts | chrisl: So we might have several font objects within a PDF file that specify the same name, but different contents? | 15:49.38 |
| rayjj | kens: once we resolve a "resourcefont | 15:49.40 |
| chrisl | Robin_Watts: yes | 15:49.44 |
| rayjj | kens: we retain whatever (substituted, munged, etc) font as PSFont in the object | 15:50.05 |
| chrisl | Robin_Watts: Luckily, so far, we haven't hit any that clash on the same page | 15:50.11 |
| kens | rayjj what Chrisl is saying | 15:50.23 |
| rayjj | kens: chrisl: the same object ALWAYS resolves to the same font, AIUI | 15:50.53 |
| Robin_Watts | chrisl: So, (and forgive me if I am showing my ignorance here), we could change from using the font name specified in the PDF object, to using a faked up name derived from the PDF object/gen number? | 15:50.57 |
| kens | rayjj if we get a sedonde font object, using the name of an already laoded font, we don't want to take a copy of the already loaed font, it may *not* be the same font | 15:51.27 |
| rayjj | Robin_Watts: we don't even need that -- we retain the font in the object dict for the resource | 15:51.35 |
| chrisl | Robin_Watts: we could, and I have branch somewhere that has the beginnings of that approach - but no one was interested, so I had to stop work on it | 15:51.36 |
| kens | s/sedonde/second/ | 15:51.40 |
| rayjj | kens: it _does_ also check the UID for the loaded font | 15:52.00 |
| kens | Robin_Watts : chris and I have discussed that approach at length, and its the only 100% sure way to proceed, but its not how things work at the moment | 15:52.12 |
| | rayjj IIRC that makes no difference | 15:52.25 |
| Robin_Watts | chrisl: OK, so this is a "it's a limitation of the PDF interpreter. It could be lifted with sufficient investment of time and effort." bug then. | 15:52.35 |
| rayjj | kens: and I am not talking about a _second_ font object, but EXACTLY the same PDF object | 15:52.43 |
| chrisl | Robin_Watts: indeed, yes | 15:52.48 |
| Robin_Watts | chrisl: Gotcha. Ta. | 15:52.53 |
| kens | rayjj But we don't *know* that its the same font object. as far as I recall | 15:53.04 |
| rayjj | 1.8x on the PLRM seems pretty useful | 15:53.09 |
| kens | rayjj fintd the commit that added the save/restore andwe can discuss this better. | 15:53.40 |
| | The point is not reuse of the same object, its encountering a *different* object with the same font *name* | 15:54.08 |
| | In that case, we would load the already instantiated font, and that can cause trouble | 15:54.26 |
| rayjj | kens: we get the object resource for the font. That dict has PSFont (if it has been loaded for that page) and we re-use it. I am talking about that not going away across pages | 15:54.39 |
| chrisl | rayjj: when we create a PDF font object, we do, in at least some cases, a findfont to find the PSFont object - that relies on the font name, which isn't reliable | 15:55.02 |
| kens | If you are not talking about going across pages, then why do you care about the page level restore ? | 15:55.08 |
| rayjj | kens: PDF Font Resource resolves to an object (dict) *NOT* a name | 15:55.13 |
| kens | And what's your problem with the page level restore then ? | 15:55.30 |
| rayjj | kens: sorry bad phrasing. Currently it doesn't go across pages. It gets "forgotten" by the "restore". I *want* it to be remembered across pages | 15:56.15 |
| kens | If you aren't going across pages, the p[age level restore is irrelevant, if you are going across pages then the point is that you may load a new object, which uses the same name as the font on a preceeding page, if you do, then a dindfont will find the existing font, and that can be incorrect. | 15:56.33 |
| | rayjj, the point is that if you don't forget existing fonts at the end of page, then a new font object can reuse an existing font, incorrectly | 15:57.05 |
| rayjj | kens: you can't (or shouldn't) load a different font for the same object | 15:57.09 |
| kens | rayjj NO | 15:57.16 |
| | NOT THE SAME OBJECT, A *NEW* OBJECT | 15:57.32 |
| | Object 1 uses font foo | 15:57.48 |
| | enf od page, new page | 15:57.53 |
| | page 2 uses object 2 | 15:57.59 |
| | object 2 is also a font called 'foo' but it is not the same font as object 1 | 15:58.20 |
| rayjj | kens: I don't care about a different object. The optimisation is only for objects (currently effective only for within the page) | 15:58.33 |
| kens | If we don't forget the font for object 1, then we load it for object 2 | 15:58.38 |
| | rayjj you most certainly do care about a different object, if it resolves to the wrong font | 15:58.58 |
| Robin_Watts | rayjj: The problem is, AIUI, that the PDF interpreter does not find fonts at the object level. It finds them (at least partially) by name. | 15:59.37 |
| kens | If you remove the save/restore then you leave teh font object behind, which leaves it possible for a new font object to load the existing PostScript font | 15:59.40 |
| rayjj | kens: I don't care about (or want to change) the way an object resolves to an actual font object. I just want to avoid all the work of preparing (finding, munging, etc) the font for a particular obect | 16:00.09 |
| Robin_Watts | rayjj: If I'm following this properly, when a font is requested, the pdf interpreter does a findfont for that fonts name. If it exists, it is reused. If it does not exist, it sets it up again from the font object. | 16:00.40 |
| kens | rayjj unless you can arrange that a new PDF font object cannot find the existing PostScript font (loaded for the preceding font object) then you can potentially load the wrong PostScript font. | 16:01.12 |
| rayjj | kens: I _don't_ want to get rid of the restore. That is critical for other reasons. I just want to pass the PSFont font in an object around the restore | 16:01.17 |
| kens | rayjj I don't know how you can do that, but that doesn't seem to be what you were saying before. | 16:01.43 |
| chrisl | rayjj: the solution to that is to load the font into global VM - but that is a *huge* nightmare | 16:01.49 |
| kens | I've a feeling the fonts in global VM won't work for other reasons, though the details escape me now | 16:02.11 |
| rayjj | kens: getting rid of the restore might result in what you are saying (a new font object getting the wrong font by name) | 16:02.14 |
| kens | rayjj yes, I was under the impression you wanted to get rid of the page level restore | 16:02.30 |
| chrisl | rayjj: the problem with loading fonts into global VM is that by the time we know creating a font object, we have already created a *lot* of (composite) objects in local VM | 16:03.21 |
| | Also, some fonts don't work well in global VM, but I suspect we don't have to worry about those | 16:03.46 |
| rayjj | chrisl: IIRC that fonts not working in global VM is mostly having to do with the loading process of Type 1 fonts (that play games) | 16:07.35 |
| chrisl | rayjj: yeh, like I said, I don't think we have to care about those. But the problem of not knowing it's a font until we're a long way through reading it, that's a problem | 16:08.31 |
| rayjj | chrisl: what about copying the font to a global dict after it was loaded into local VM? Then the PDF interp would have the global copy to re-use | 16:08.38 |
| kens | Hmm, if we load a font into global VM, won't it still persist after a restore ? In which case we're back to potentially finding it via fndfont, aren't we ? | 16:09.13 |
| rayjj | in the PDF interp (pdf_fonts.ps) the last thing we do is tuck the font away as PSFont in the object dict | 16:09.23 |
| chrisl | rayjj: I did briefly try that, but I couldn't get it to work - again, no one cared at the time I raised it, so I had to stop working on it | 16:09.43 |
| kens | I seem to recall the PS font object carries some important stuff around with it, which the interpreter accesses directly | 16:10.05 |
| rayjj | kens: I don't think so. We use the FontDirectory -- we don't search all dicts for something named "Helvetica" | 16:10.08 |
| kens | rayjj as long as its no longer loaded into FontDirectory, yes | 16:10.29 |
| | WHich I guess is subject to save/retore | 16:10.38 |
| rayjj | kens: exactly -- FontDirectory _is_ one of the things that 'restore' takes care of | 16:11.04 |
| chrisl | if the font isn't in the FontDirectory we have to do a definefont don't we? In which case, we're pretty much back to recreating the font | 16:11.16 |
| | rayjj: But GlobalFontDirectory isn't which is where the font will end up if it's in global VM | 16:11.46 |
| kens | I'm unsure, I recall running across some strange dependencies in the PS interpreter, but I no longer remember what they were | 16:11.59 |
| rayjj | kens: ideally we'd not want to bother making a global copy of the font unless that object was shared on another page, but that's a bit more work (but not that much more than checking for transparency) | 16:12.14 |
| kens | and yes, I thought the font woudl still be a font, until the EOJ restore | 16:12.18 |
| chrisl | However, we *can* store the object number in the font dictionary, and use that to validate the font before we use it a subsequent time | 16:12.29 |
| kens | Yes, the object number is a *much* better way to proceed on this | 16:12.47 |
| rayjj | chrisl: we don't need the font in *either* FontDirectory if we get it from the PDF object number | 16:13.03 |
| | (sort of what Robin_Watts mentioned) | 16:13.21 |
| chrisl | rayjj: but we can't use a font that's not in one or other FontDirectory | 16:13.26 |
| kens | rayjj I have a sneaky suspicion that won't work, though I cannot currently remember why | 16:13.31 |
| henrys | does anyone know if adobe puts cached fonts in global vm? A font with a buildchar procedure that had a side effect could show that. Check if buildchar is called after restore again? | 16:13.35 |
| kens | I have a feeling the font carries a pointer to the cache, that might be the problem | 16:13.50 |
| rayjj | chrisl: are you sure ? 'scalefont' doesn't refer to the font directory | 16:14.17 |
| chrisl | rayjj: Hmm, I could be getting confused with another PS interpreter...... | 16:14.46 |
| rayjj | kens: why is the cache a problem (I'm not sure that the font _does_ point to the cache, btw) | 16:14.48 |
| | font cache hits are resolved by font-id,glyph-id,matrix as keys AIUI | 16:15.48 |
| chrisl | rayjj: I'm also not sure that copying the font dictionary will also copy the internal font object in a useful way (gs_font *) | 16:15.49 |
| kens | rayjj I'm pretty sure it does point to the cache, because I trip[ped over it while trying to do the 'no output fonts' stuff for pdfwrite (the equivalent of -dNOCACHE with pswrite). It may not be a problem *unless* you need to use the cache, just saying that I remember it causing me problems | 16:15.50 |
| rayjj | and that font-id is our internally generated gs_id, so _is_ unique | 16:16.29 |
| henrys | it does reference the cache - see gsfont.c line 72 ... | 16:16.33 |
| kens | WHile it may be possible to work around the problem, my own preference would be to fix it so that we check the object number as well, when dealing with PDF fonts | 16:16.46 |
| rayjj | henrys: that comment is about GC | 16:17.33 |
| henrys | the enum ptr procedure references the cache. | 16:18.09 |
| rayjj | henrys: I think that font directory is different than the PS level FontDirectory that is subject to restore | 16:18.45 |
| kens | Neverhteless, a gs_font carries a pointer to a gs_font_dir | 16:19.27 |
| | Its not clear to me that you can have a font which isn't in a gs_font_dir, though whether that is visible at the PostScirtpt level I have no idea | 16:20.10 |
| chrisl | I'm fairly sure that once you copy a font dictionary, the new copy is not a usable font - it needs to be definefonted again | 16:21.16 |
| rayjj | chrisl: I can try that readily enough ... | 16:22.37 |
| henrys | chrisl, rayjj : len said 5 seconds? what percentage slowdown? | 16:23.59 |
| rayjj | henrys: 5 seconds ??? the problem is < 2 seconds for 16 pages (the entire job is 24 seconds with 906) | 16:25.29 |
| chrisl | rayjj: "/Helvetica findfont dup length dict .copydict 10 scalefont" -> Error: /invalidfont in --scalefont-- | 16:25.50 |
| henrys | rayjj: he wrote GS871 is faster by about 5 seconds on the target ... | 16:26.08 |
| kens | I thikn 5% maybe ? | 16:26.19 |
| rayjj | henrys: gs871 is about 22.5 seconds. gs906 is 24.5 | 16:26.26 |
| kens | Or which mail is htis ? | 16:26.28 |
| chrisl | I saw the mail where he mentioned 5 seconds, but I've never seen overall times | 16:26.51 |
| rayjj | that's from the email with the JPG's of the instrumented code | 16:27.07 |
| | from 11/7 (1.9Mb -- 6 attachments) | 16:28.05 |
| henrys | if 24.5 and 22.5 are correct and I assume this is all text jobs effected that is quite a hit. | 16:28.49 |
| rayjj | the JPEG files are charts, but come from actual target numbers | 16:29.01 |
| chrisl | I'm not at all clear where the slow down is occurring | 16:29.15 |
| rayjj | now that thing is that 871 text processing was the same in this respect as with 906 -- both reload the font. In parallel I am looking (with Len) into potential color setting issues | 16:30.09 |
| | but if we can speed up the text processing, that's good as well. | 16:30.36 |
| chrisl | Avoiding reloading the fonts is a non-trivial project | 16:30.49 |
| rayjj | (or even better since it helps us overall) | 16:30.50 |
| chrisl | It's not a quick or easy fix | 16:31.21 |
| kens | We'd love to get rid of the font laoding, but its not easy, as chrisl said, he and I have discussed this several times | 16:31.22 |
| chrisl | I'm also concerned that since the page level save/restore was introduce a considerable time ago, there *may* be a lot of newer code that relies on it to work correctly | 16:32.32 |
| kens | That wouldn't surprise me either, though again if that's the case, it should be undone and made to work better :-) Another large project probably, although Ray's cluster push is encouraging | 16:33.19 |
| chrisl | Fonts are the biggest concern - but I doubt my old patch would work or even apply now | 16:34.07 |
| rayjj | kens: I also don't want to get rid of page level restore | 16:34.40 |
| henrys | do we have profiles there are some other things that have changed. Robin_Watts changing mem_copy_mono which is very important to font rendering changed as well? | 16:34.56 |
| kens | I think getting rid of that save is the way to go myself, trying to make fonts persist without that looks like a nightmare to me | 16:35.18 |
| chrisl | henrys: the VS profiler output was complete nonsense, even when I eventually got it to run | 16:35.34 |
| henrys | chrisl: we have a file right? we can profile on linux no? | 16:36.01 |
| chrisl | henrys: I've never got their code to build on anything other than the simulator in VS | 16:36.30 |
| kens | Is there a measurable difference on standard GS ? | 16:37.08 |
| henrys | chrisl: I was just thinking of doing 871 and 906 compare not their stuff... | 16:37.15 |
| chrisl | henrys: I doubt that will give meaningful results | 16:37.34 |
| | kens: not that I could detect using as close to similar configurations as seemed reasonable | 16:38.00 |
| kens | THen profiling on Linux is a non-starter I guess | 16:38.13 |
| chrisl | Stock 9.06 is a lot slower because we *always* applied all the TTF hints in that, while cust 532 disable TTF hints | 16:38.55 |
| rayjj | chrisl: I tried: /Helvetica findfont true setglobal dup length dict copy /X1 exch definefont false setglobal /X1 50 selectfont 100 500 moveto (X) show showpage | 16:38.57 |
| henrys | I guess I don't understand it seems pretty clear there is a performance regression. Why wouldn't we see it. | 16:39.02 |
| | ? | 16:39.03 |
| rayjj | henrys: you're kidding, right ? | 16:39.21 |
| chrisl | rayjj: yes, so you repeated the "definefont" which is one of the most time consuming parts of the process | 16:39.46 |
| rayjj | 9.xx _is_ slower than 871 -- we *know* that | 16:39.50 |
| | chrisl: is it ? | 16:40.09 |
| chrisl | rayjj: definefont is where almost all the validation happens, and where we build the gs_font object | 16:40.42 |
| henrys | rayjj: then why wouldn't it be useful to create 871 and 9.xx profiles on linux? | 16:40.54 |
| rayjj | henrys: profiles for just this 16p file ? | 16:41.34 |
| chrisl | henrys: 9.xx does more stuff than 8.71 - doing more stuff takes more time....... | 16:41.48 |
| henrys | yes but we'd like to know if it is save/restore, mem_copy_mono, icc profiles, you should see that quite clearly in a profile. | 16:43.04 |
| rayjj | henrys: for cust 532, their code is really *not* our 8.71 | 16:43.07 |
| | and we have some 'pseudo profiles' from their target that is the primary thing I'm using. Anything else is mostly misleading | 16:43.56 |
| chrisl | rayjj: the thing point is, I agree entirely that what we have now (page save/restore) is not the right solution, and I would be happy to take it up as a project (given that I started on it >2 years ago), but it's not going to be a quick fix. | 16:44.48 |
| rayjj | the difficulty is that the pseudo profiles are from instrumenting the code and it's hard (impossible) to get exactly the same instrumentation | 16:45.00 |
| chrisl | rayjj: my reading is that the manually instrument the code - is that correct? | 16:45.35 |
| | s/the/they | 16:45.46 |
| rayjj | chrisl: I'm not sure that getting rid of restore per page will ever get there. mooscript is probably a better place to spend time | 16:46.10 |
| chrisl | rayjj: I'm not sure that keeping the save/restore and fiddling with fonts would be best, either. It would need investigating | 16:47.05 |
| rayjj | chrisl: it depends on which set of docs you are looking at, but the JPEG charts and the .csv files are from instrumented code on the target. The XLS files are from the "AQtime" tool on Windows simulator and are best to take with a pound of salt | 16:47.48 |
| chrisl | rayjj: it just seems to me that instead of using a profile to work out where to investigate, they guess at where the problem is, and add instrumentation in there to prove or disprove the guess | 16:49.05 |
| rayjj | chrisl: on the time needed for 'definefont' that would only be to create the copy that gets retained (by object number) for re-use | 16:49.13 |
| | so that would only be done once for all pages | 16:49.28 |
| chrisl | rayjj: but the definefont puts the font in the FontDirectory, and we're back to needing a way to validate the font | 16:50.29 |
| rayjj | chrisl: yes. The problem is that their 'hprof' profiling tool is a piece of crap that doesn't give us actual function names reliably -- never been resolved | 16:50.34 |
| | chrisl: "validate" ? | 16:50.54 |
| kens | "Make sure we aren't loading an existing font for a new object" | 16:51.20 |
| chrisl | Wot kens said...... | 16:51.32 |
| rayjj | a PDF Font Resource comes from a PDF object/generation number that *is* unique | 16:51.49 |
| chrisl | But we don't store that in the font dictionary | 16:52.02 |
| kens | rayjj, yes, but its not the PDF font object we're discussing | 16:52.04 |
| rayjj | so as long as we pull in the font saved for that object, we are "valid" | 16:52.23 |
| | kens: it is _exactly_ the PDF object that I am looking to re-use, avoiding reloading | 16:52.51 |
| chrisl | When we're loading a new font object, we don't have a font saved for it | 16:52.54 |
| kens | When we load a new PDF font object, we may try to find an underlying PostScript font for it (ie findfont). If a previous font object has already loaded that PS font it will be in FontDirectory, which can lead to us using an inappropriate font for the new object | 16:52.57 |
| | rayjj you are trying to acoid reloading a font for an existing object, we are trying to avoid using an existing font for a new object | 16:53.39 |
| | Your interest is in getting performance, ours is in getting correct output | 16:53.56 |
| rayjj | kens: I don't care about a new object | 16:54.14 |
| kens | rayjj, you will if you run across one of these files. | 16:54.27 |
| chrisl | rayjj: so you're happy using the wrong font? | 16:54.34 |
| rayjj | when we do a page level restore, the elements in FontDirectory get forgotten | 16:54.45 |
| chrisl | Not if they are in global VM | 16:54.55 |
| kens | ANythign you do which results in a PS font remaining in FontDirectory means that potentially a new PDF font object can use that font, instead of laoding a new font | 16:54.59 |
| | rayjj and the global VM fonts are in GlobalFontDirectory ? WHich I think we also search with findfont | 16:55.30 |
| rayjj | kens: but they are in global VM in globaldict, not FontDiretory and the 'definefont' is done with a different key | 16:55.47 |
| chrisl | A font created in global VM will end up in the GlobalFontDirectory | 16:56.02 |
| kens | chrisl beat me to it | 16:56.21 |
| | rayjj copying the font dictionary in a way which removes the font name from all places where fiondfont might find it is absolutely safe. Anything else isn't. | 16:57.50 |
| | The problem seems to be that if you copy the font dictionary that way, you can't then use the font, which makes it less than useful for rendering text, so you have to 'findfont' it or similar again. WHIch chrisl believes is the largest part of the time anyway | 16:58.40 |
| chrisl | We remove a font from the FontDirectories, but I don't know what implications that might have for future uses of the font | 16:59.07 |
| rayjj | kens: findfont _is_ a problem. chrisl mentioned 'defineifont' | 16:59.18 |
| kens | He did also mention findfont, as did I. The problem is for a new PDF font object, which will start by trying to see if it already exists, by using findfont on the font name (potentialyy, at least) | 16:59.55 |
| | The problem is that (for some very bad PDF files) the font *does* already exist (from an earlier page) and has the same name, but its not the same font..... | 17:00.35 |
| chrisl | rayjj: as I've said, these problems can all be worked out, *but* it will take time | 17:01.42 |
| rayjj | kens: so we need to get rid of the font names that we don't want (not the object # based one) from GlobalFontDirectory ? | 17:01.52 |
| kens | rayjj from GLobalFontDirectory if you ptu the fonts in global VM, and from FOntDirectory if you put them in local VM | 17:02.20 |
| | But as I said before, I'd rather see Chris spend time on his approach than adding more hairy code that does weird special sh*t behind the scenes...... | 17:03.02 |
| rayjj | kens: AIUI, FontDirectory gets handled by the page level restore | 17:03.11 |
| kens | Yeah, if you put the fonts in global VM you don't need to worry about the local VM FontDirectory. | 17:03.41 |
| rayjj | kens: I agree that chrisl probably shouldn't spend time on it (mooscript is more useful) | 17:03.47 |
| chrisl | Er, actually, IIRC, at least in some cases we put global fonts in the local FontDirectory, too..... | 17:04.22 |
| kens | rayjj I think getting the existing PDF interpreter to work with object numbers for fonts would be more suseful. Mooscript is a long, long way from being useful, and not getting any closer | 17:04.30 |
| chrisl | Argh, git blame is less than helpful tracking down the reason for the PDFSave addition.... :-( | 17:05.55 |
| kens | TBH I've never had a lot if success with Git blame, I assumed it was because I just don't know how to use it properly | 17:06.22 |
| chrisl | The problem is, it's not clear in this case, if the lines were added in the commit, or moved to their current location in the commit | 17:07.16 |
| kens | Oh..... | 17:07.30 |
| chrisl | http://git.ghostscript.com/?p=ghostpdl.git;a=commitdiff;h=f0524b5d | 17:07.43 |
| | oh, no, actually, the lines were moved.... but how do I find the history before that | 17:08.11 |
| rayjj | chrisl: it's been in place since before 2002 as far as I can tell | 17:08.17 |
| kens | Yeah, seems they were moved | 17:08.19 |
| | I notice there's at least one other thing there about needing tro be inside the page save | 17:08.47 |
| chrisl | rayjj: yeh, I'm guessing it's been there since the original commit to cvs in '98 or so | 17:09.02 |
| | kens: I don't really want to push fonts onto the stack between pages - even it would work ;-) | 17:09.46 |
| | s/even/even if | 17:09.55 |
| kens | didn't think I'd suggested that..... | 17:10.22 |
| rayjj | kens: I confirmed that I _can_ still use the font defined as X1 in GlobalFontDirectory even if I 'undef' Helvetica and NimbusSanL-Regu from GlobalFontDirectory | 17:10.39 |
| chrisl | I assumed you meant the Repaired flag | 17:10.39 |
| kens | Oh no, I was just looking at the comment for pdfshowpage_setcspacesub | 17:11.00 |
| rayjj | chrisl: right. invalidrestore | 17:11.06 |
| henrys | I was sort of curious so I went back to earlier version 8.71 and it seems to be significantly slower than the latest for text dominant stuff. Huh | 17:11.36 |
| kens | rayjj are you doing the undef in PostScript ? | 17:11.43 |
| rayjj | kens: yes (I zapped gs_init so that GlobalFontDirectory was not readonly) | 17:12.11 |
| kens | Hmm, that's not ideal really | 17:12.29 |
| rayjj | kens: "ideal" ??? we are talking about Ghostscript's PDF interpreter here | 17:13.00 |
| kens | Thoguh I'm happier with a PS solution in the PDF itnerpreter than some hacky code in C in the PS interpreter | 17:13.01 |
| | rayjj you're talking about the PS interpreter there, maing GlobalFontDirectory read/write | 17:13.34 |
| | The PDF interpreter is impossible to make worse | 17:13.52 |
| rayjj | kens: but yes, I'm coming around to something that *might* work with mods to pdf_font.ps | 17:13.57 |
| | internally we have .forceundef | 17:14.29 |
| kens | While it would be nice to have, it doesn't really address Len's problem, it just hides it. Though maybe that's good enough | 17:14.42 |
| rayjj | kens: Len is looking for any and all solutions, including "hiding" | 17:15.15 |
| kens | Well, utilising nasty PS extensions in the PDF interpreter has plenty of provenance :-( | 17:15.16 |
| rayjj | I'm going to let this percolate in the "little gray cells" and look into the colorspace issue now, but thanks for the discussion (in particular the hint about 'definefont', chrisl) | 17:17.15 |
| | BTW, I did confirm that just copying a font loaded with findfont doesn't make a usable font -- I *do* need to use definefont, but then it is OK | 17:18.16 |
| kens | If you can knock something up quick, it would be interesting to see what effect doing that has on their file | 17:19.10 |
| rayjj | kens: yes, and also the PLRM | 17:19.35 |
| kens | Yes, I thought that might be harder as there are more fonts and font types to worry about | 17:19.53 |
| chrisl | For the couple of examples I got to work way back when, the performance improvement was useful - in one case, dramatic, but that was a bonkers file. | 17:20.14 |
| rayjj | kens: maybe, but just zapping the PDFSave restore with the PLRM was dramatic | 17:20.29 |
| kens | Yes, but that knocks out a lot of stuff, you probably won't get as much from just fonts, but it would be interesting to know | 17:20.59 |
| rayjj | but I don't know how much time was actually the save/restore time | 17:21.14 |
| kens | THat's what I mean yes | 17:21.21 |
| rayjj | (as kens said) | 17:21.25 |
| chrisl | PDFSave does, indeed, date right back to '98 | 17:21.35 |
| kens | :-( | 17:21.46 |
| | OK I'm off, night all | 17:22.06 |
| henrys | chrisl: about caching across jobs - at one time I had the pcl interpreter using the ufst cache and it would shove bitmaps in the gs cache. I haven't looked at that code in a long time but I wonder if that would be away to get some benefits of caching across pages without messing with gs. | 17:31.38 |
| chrisl | henrys: it's possible, but I doubt it would have a great deal of benefit in the PS/PDF world these days. Embedded fonts are the order of the day | 17:33.02 |
| henrys | chrisl: yeah just a thought... | 17:33.26 |
| chrisl | henrys: case in point, the file from Len uses Helvetica, but embeds a TTF version of it | 17:34.05 |
| | henrys: I've never messed with the ufst cache, though. I briefly played with the Freetype one, and didn't see any benefit (given our existing cache arrangement) | 17:35.25 |
| henrys | chrisl: no problem you were going to fix the ufst embedded stuff anyway ;-) | 17:35.43 |
| chrisl | henrys: I'd love to, but I think it's an impossible task without major tweaking to their code - so I gave up...... | 17:36.35 |
| henrys | chrisl: I never understood how anyone could get it to work with PS and PDF | 17:37.43 |
| chrisl | henrys: with the code as it ships, you can't, really, genuinely can't....... | 17:38.31 |
| henrys | I suspect the UFST is mostly a PCL tool and most folks have Adobe or their own font solution for the other languages. | 17:38.36 |
| chrisl | Everyone I know that's used it as their main solution has made more than trivial changes to it to work with Postscript, at least | 17:39.37 |
| henrys | now that monotype is partnered with adobe there's little reason to improve it, I can't imagine Adobe handing over font rendering to the UFST. | 17:40.53 |
| Robin_Watts | Adobe handed over some font rendering code to freetype recently IIRC. | 17:41.41 |
| | Possibly that may be a sign that they are moving to using freetype. | 17:42.02 |
| chrisl | Interpreter, not rendering, I think | 17:42.12 |
| Robin_Watts | chrisl: Sorry, yes. | 17:42.21 |
| chrisl | I think it was the CFF interpreter | 17:42.29 |
| | And, IIRC, they handed over the bare code, and someone else was left to integrate, but I could be wrong...... | 17:43.42 |
| | Is there a way in bugzill that I can search for bugs that I've attached a patch to? | 17:45.52 |
| henrys | chrisl: attachment is patch in custom search | 17:47.26 |
| | I haven't tried it myself | 17:47.41 |
| chrisl | Aha! So it is. Thanks henrys | 17:47.56 |
| henrys | rayjj, chrisl while I'm sticking my nose in... wouldn't it be better to expand that pdf file out to say 200 pages or so and having len do his profile again? | 17:53.37 |
| chrisl | henrys: I don't have the confidence in their profiling that it would buy us much...... | 17:55.22 |
| mvrhel_laptop | henrys: I am going to try to get my xps device changes committed today | 17:57.31 |
| henrys | mvrhel_laptop: yeah I was going to ask you about that. I have a large change too and I didn't want to force you to merge. | 17:57.59 |
| chrisl | rayjj: do you know if the major time delay is due to recreating the font, or due to discarding and recreating the cached the glyphs? | 17:58.32 |
| henrys | mvrhel_laptop: have you pushed without problems, I would expect a lot of pixel shifts with your code, at least. | 17:58.57 |
| mvrhel_laptop | henrys: not yet. I was just cleaning up a few things. | 18:04.18 |
| | marcosw: are you here? | 19:10.03 |
| | henrys: so the cluster push says there are lots of diffs. unfortunately, bmpcmp does not seem to work | 19:11.21 |
| | one segv that I will look at now | 19:12.52 |
| | it does not have any images but I am wondering if this has always been here since this appears to be the first run of this | 19:13.30 |
| henrys | mvrhel_laptop: do you have to do anything special to get it tested? does xpswrite have to change? | 19:14.23 |
| mvrhel_laptop | henrys: no I just did a clusterpush | 19:14.49 |
| | I don't know if xpswrite had to change | 19:15.05 |
| henrys | mvrhel_laptop: if it isn't the image stuff assign a bug to me | 19:15.19 |
| mvrhel_laptop | ok let me verify it first | 19:15.35 |
| henrys | mvrhel_laptop: marcos has reported 5 bugs but no crashes | 19:16.18 |
| mvrhel_laptop | ok | 19:16.25 |
| | well let me make sure | 19:16.31 |
| | hmm file ran fine here | 19:17.51 |
| | other than the fact that the output is giant... | 19:18.29 |
| | 3K goes to 132,000K | 19:19.06 |
| | oh and the xpsviewer will not open it | 19:20.24 |
| | it is a file with shading | 19:20.46 |
| | I am going to do a clean push just to see what comes out | 19:21.15 |
| henrys | mvrhel_laptop: this is before your changes? | 19:21.15 |
| mvrhel_laptop | no this is after my changes | 19:21.24 |
| | henrys: I will check from before too | 19:21.37 |
| rayjj | chrisl: aargh!! seems that even though we are in a long TJ operator we are doing the ENTIRE text setup for EVERY string in the TJ (Show is doing the entire showfirst procedure) | 19:21.43 |
| chrisl | rayjj: Yeh, I noticed that a few weeks ago. I was meaning to see if there would be a way to do it more efficiently | 19:23.03 |
| mvrhel_laptop | my fixes were not for shading. that will be another issue. xps is limited compared to pdf for its shading. we may be able to pack in some of the stuff in vector form (same for transparency). The rest we should pack in as an image (which is the part that I did add) | 19:23.27 |
| chrisl | rayjj: I figured it would be easy enough for Tr, but not sure what else is required | 19:23.37 |
| rayjj | chrisl: AFAIK, there is no way to change anything except text position when processing the TJ params | 19:23.41 |
| | chrisl: I'm going to see if there's a rabbit down this hole :-) | 19:24.33 |
| chrisl | rayjj: I was hoping we could avoid the text setup for all the operators unless something had changed - probably easier/safer to just target TJ though | 19:24.33 |
| rayjj | chrisl: If this works, I'll cc you (of course) and maybe leave more general optimization in side of BT/ET to you ? | 19:25.53 |
| chrisl | rayjj: Okay, if it looks like it's going to be useful, maybe put it in a bug? So we don't forget..... | 19:26.44 |
| rayjj | chrisl: right. | 19:26.53 |
| | tome for some food. | 19:27.08 |
| chrisl | FWIW, I *think* we need to smarter than just doing the setup on a BT operator, stuff can change inside BT/ET IIRC | 19:27.40 |
| | Right, my screen is starting to look fuzzy, so I better stop..... | 19:30.13 |
| | Goodnight all | 19:30.23 |
| henrys | mvrhel_laptop: we do need bmpcmp, I'll send marcosw email | 19:31.41 |
| mvrhel_laptop | ok thanks | 19:31.52 |
| | henrys: so a clean cluster push gives me this | 19:42.12 |
| | tests_private/pdf/PDF_1.7_FTS/fts_06_0626.pdf.xps.ppmraw.300.0 beards Seg_Fault_reading_Ghostscript_produced_PDF/PS/XPS_File | 19:42.14 |
| | oh it segv's on the reading of the xps file | 19:42.35 |
| | the xps file can't be read either by the MS xps viewer. So really there are 2 issues with this | 19:43.15 |
| | 1) we should not be doing the segv with gxps 2) the xps output from xpswrite is not good | 19:43.45 |
| | henrys: so once marcosw has bmpcmp working then I will go through my diffs | 19:44.43 |
| | I will open a bug about the above | 19:44.57 |
| | henrys: bug commited. Number 695680 assigned to you | 19:48.00 |
| | in the mean time, I will look at the ones that apparently resulted in errors with xpswrite | 19:51.43 |
| | which must be from my change | 19:51.53 |
| | 330 of them, hopefully a single issue... | 19:52.04 |
| | hmm I may have mucked up something. | 20:01.04 |
| | ok yes. my change screwed up something | 20:11.06 |
| | found the issue | 21:16.34 |
| henrys | mvrhel_laptop: any other parameters when creating that crash file for gxps? I am not seeing it. | 21:42.24 |
| mvrhel_laptop | henrys: I don't get it locally either | 21:42.45 |
| | it just occurred though on a clean cluster push | 21:42.57 |
| henrys | mvrhel_laptop: it's just a whole lot of rectangle so it's a memory thing I suppose. | 21:43.05 |
| mvrhel_laptop | henrys: and that is for gxps | 21:43.17 |
| | xpswrite creates an output | 21:43.24 |
| | but the MS viewer wont view it | 21:43.34 |
| | and gxps crashes on it | 21:43.37 |
| | henrys: I have to head out for a few minutes | 21:44.37 |
| henrys | mvrhel_laptop: no parameters to gs when creating the xps other than -sDEVICE=xpswrite? | 21:45.03 |
| mvrhel_laptop | that is all I had | 21:48.12 |
| | that wont crash though | 21:48.46 |
| | I think the crash is occurring when gxps tries to read the output of xpswirte | 21:49.05 |
| henrys | I understand - the page size is 10x the usual, could that be it? | 21:49.59 |
| | I wanted to make sure I was looking at the same xps input as you are. If you changed stuff on the gs command line my file would be different | 21:51.02 |
| | Forward 1 day (to 2014/11/14)>>> | |
IRC Logs