| | <<<Back 1 day (to 2014/10/16) | 20141017 |
| henry__ | Hi all, how can I save the pdf file into filesystem after MuPDFCore_addInkAnnotationInternal ? | 01:21.38 |
| | ps: I am using mupdf-1.6 | 01:22.19 |
| tomty89 | chrisl: hello | 07:01.11 |
| chrisl | tomty89: hi | 07:01.51 |
| tomty89 | chrisl: i have some questions about the urw fonts | 07:02.08 |
| chrisl | I'll answer if I can.... | 07:02.40 |
| tomty89 | chrisl: what are the relationship between the font files in Resource/Font and the urw-core35-fonts.git | 07:03.00 |
| | like, how are they maintained respectively | 07:03.24 |
| chrisl | They are the same fonts, only the file names are different | 07:03.30 |
| | Oh, actually, the font names are different, too. But the outlines are the same | 07:03.55 |
| tomty89 | but it seems to me the file sizes are a bit different | 07:04.32 |
| chrisl | Because the names are different | 07:04.40 |
| tomty89 | oh | 07:04.55 |
| | so when there're updates, i suppose urw-core35-fonts.git would also get updated? | 07:05.19 |
| chrisl | Basically, the urw-core35-fonts.git is *exactly* what we get from URW, whilst the Resource/Font ones have some tweaks to the names for Ghostscript | 07:05.53 |
| tomty89 | hmm okay | 07:06.18 |
| chrisl | Yes, every time we get an update from URW, that update will go in urw-core35-fonts.git | 07:06.30 |
| | Note that this will *probably* change with the next update: I *think* I'll keep the font names the same (but still rename the files) and modify Ghostscript to use the new URW names | 07:07.20 |
| tomty89 | do you know what are difference between them and these http://svn.ghostscript.com/ghostscript/trunk/urw-fonts/ http://svn.ghostscript.com/ghostscript/tags/urw-fonts-1.0.7pre44/ ? | 07:07.29 |
| | seems these are much larger in file sizes | 07:07.38 |
| chrisl | Those have (very badly designed) Cyrillic glyphs in them | 07:07.54 |
| tomty89 | so i should switch to those in the git without losing any "features"? | 07:08.34 |
| chrisl | You will lose Cyrillic.... but really, you shouldn't be using them for Cyrillic glyphs, given the poor quality | 07:09.12 |
| tomty89 | i see | 07:09.35 |
| chrisl | And Cyrillic glyphs are not required to be present in the Postscript base fonts | 07:10.16 |
| tomty89 | so one more question, any reason that only the pfb goes into Resource/Font? (is that a dumb question? because ghostscript only needs pfb?) | 07:11.10 |
| chrisl | Yeh, GS only needs the font files | 07:11.42 |
| | That's one of the main reasons that urw-core35-fonts.git exists | 07:12.08 |
| tomty89 | hmm so urw is still maintaining the fonts...i thought they just donated them and let you guys maintain it | 07:12.25 |
| chrisl | We occasionally *pay* URW to update the fonts | 07:13.00 |
| kens | We aren't a font foundry, we don't have the time to maintain fonts | 07:13.13 |
| tomty89 | oh | 07:13.19 |
| | but urw-core35-fonts.git is the most "upstream" source anyway, right? | 07:13.54 |
| kens | We are acting as upstream providers simply because nobody else will, its a conscious decision but not one we are totally happy with | 07:13.59 |
| chrisl | tomty89: yes, urw-core35-fonts.git is the upstream source | 07:14.22 |
| tomty89 | i see | 07:14.25 |
| chrisl | As I said, the relationship between the Resource/Font files and the contents of urw-core35-fonts.git will closer in the future | 07:15.04 |
| tomty89 | okay thanks a lot. i was quite confused because seems not me quite some linux distros still stick with "1.07pre44" | 07:16.58 |
| henrychan | how to save pdf file into file system after call MuPDFCore_addInkAnnotationInternal ?? | 09:10.09 |
| dinamic | tor8: do you have a fewm minutes ? | 10:44.24 |
| kens | chrisl I think this idea of subclassing gdevvec and gdevprn isn't going to work. THe problem is that, unlike our former experience, we cannot modify the device pipeline. In order to install a 'forwarding' device, I need to create a device which forwards itself to the final device. I can't insert a forwarding device before the current one. | 10:45.18 |
| | So I'd need to modify every instance of the device we want to create a forwarding device, then point itself to the device we actually intended to create in the first place. I'm more than somewhat dubious about this. | 10:46.13 |
| tor8 | dinamic: sure | 10:46.45 |
| dinamic | tor8: I'm thinking of possibilities of a js_async_call(state, args, _callback_finish(), user_data), would it be possible by having a state = js_copystate(state) | 10:46.56 |
| kens | The only solution I can think of at present is to have the initial device make a new device based on the original, then rewrite itself into a forwarding device. | 10:47.13 |
| dinamic | tor8: the problem im trying to solve is: i have a global object in js space with a function that is ansynchronous | 10:47.49 |
| kens | But that would copy all the pointers and stuff from the original device, which in the case of pdfwrite is somewhat horrifying | 10:47.51 |
| dinamic | tor8: during time something could potential call function from js space which would use same state and corrupt stuff.. | 10:48.28 |
| chrisl | kens: I remember discussing it, but I can't now remember the details of what I was thinking - I'd have to ponder it again | 10:48.35 |
| kens | I've3 been banging my head against this particular brick wall all morning, and I can't see a good solution :-( | 10:49.05 |
| tor8 | dinamic: that would be most difficult, since any form of pre-emptive multi-threading would have to have locks around the garbage collection structures and any shared environment records | 10:49.16 |
| | not to mention all objects shared between the contexts | 10:50.20 |
| dinamic | tor8: i was afraid of that answer.. but could it copy everything ? and not beeing shared ? | 10:50.50 |
| tor8 | or you could make a complete copy of the entire state, but then you'd lose all communication between threads | 10:50.55 |
| dinamic | ^ thats what i want and is an restriction that could be lived with | 10:51.19 |
| chrisl | kens: it *must* be possible to insert a device before the already installed one, otherwise compositor devices wouldn't work | 10:51.28 |
| dinamic | tor8: thats why is isolated it to a js_async_call() which is the only thing that would run on a isloated copy of a state | 10:52.03 |
| tor8 | dinamic: I guess you want to be able to fork the state to keep variables around for the new function? | 10:52.44 |
| | dinamic: it's not something I'm going to add to the main distribution, but it should also not be too difficult for you to write yourself | 10:53.12 |
| | dinamic: there are global lists that have pointers to all the objects used for the garbage collector that you can use, but you'll need to have a 'remap' data structure to rewrite all the pointers in the objects | 10:54.15 |
| | and also make a copy of the string interning table | 10:54.24 |
| dinamic | tor8: that sounds like what i should do... | 10:54.37 |
| tor8 | dinamic: if you don't mind a heavy weight solution -- have you considered using fork() instead? | 10:55.10 |
| dinamic | hmm | 10:55.46 |
| tor8 | I expect that would be about the same startup penalty (or less) than making a complete copy of the runtime state | 10:55.53 |
| dinamic | maybe that's easiest approach, ill look into it.. | 10:56.14 |
| | but it could be hard, my service uses mujs for plugins which provides implementations | 10:56.59 |
| tor8 | why do you need async calls? | 10:57.22 |
| dinamic | so it's not the most elegant approach | 10:57.22 |
| tor8 | neither approach would be elegant; the elegant solution would be to add multi-threading to mujs but that's not really part of the scope for mujs | 10:58.09 |
| | it would add a significant amount of extra complexity | 10:58.17 |
| dinamic | agreed ^ | 10:58.24 |
| Robin_Watts | tor8: Does mujs admit of a similar threading solution to mupdf? | 10:59.16 |
| tor8 | dinamic: if you just need the illusion of async calls, you could split the js stack and do co-operative multi-threading | 10:59.41 |
| Robin_Watts | Namely that a user could provide lock/unlock hooks in the context, so mujs remains threading agnostic? | 10:59.41 |
| dinamic | my main process is singel threaded, and i dont want my plugins to hang the mainloop | 10:59.45 |
| tor8 | Robin_Watts: there'd be so many locks around object manipulation that performance would be significantly hampered | 11:00.09 |
| | Robin_Watts: not that we *really* care about performance | 11:00.22 |
| | but it'd get ugly fast | 11:00.26 |
| Robin_Watts | tor8: lock at a higher granularity? | 11:00.36 |
| | on entry to the interpreter? | 11:00.43 |
| tor8 | dinamic: so stick mujs in a worker thread, and hand off jobs from the main thread to it and then monitor its progress | 11:01.00 |
| dinamic | hmm | 11:01.28 |
| | maybe its my design that is wrong | 11:01.42 |
| tor8 | dinamic: or you could set an instruction limit in the interpreter to bail if the runtime exceeds what you expect | 11:02.23 |
| dinamic | tor8: thanks for your input on the matter, i need to check the options you give.. | 11:03.46 |
| tor8 | dinamic: I'd either set a time limit for the js_call or call mujs from a worker thread. it's trivial to add a counter to the interpreter main loop, if that's what you need. | 11:05.23 |
| dinamic | actually, i might just create a thread worker per plugin, and live with all operations on plugin is synchronous | 11:05.24 |
| tor8 | dinamic: what's the project you're using mujs for, if you don't mind my asking? | 11:05.51 |
| dinamic | prototyping for a media service for chromecast | 11:06.18 |
| | https://github.com/hean01/castio | 11:06.21 |
| | tor8: will your iterator patch will be pushed upstream ? | 11:07.28 |
| tor8 | dinamic: yes | 11:07.39 |
| dinamic | great | 11:08.09 |
| | anyway, thanks for your time.. | 11:08.56 |
| tor8 | dinamic: glad to be of help | 11:10.41 |
| dinamic | oh one more thing :) | 11:12.36 |
| | it would be awesome if js_try() / js_endtry() also was public | 11:13.37 |
| Robin_Watts | New adobe reader. | 11:31.49 |
| tor8 | nsz: thanks for your patches! | 12:00.16 |
| kens | Robin_Watts : new reader, or new version number ? I already have reader XI | 12:01.14 |
| Robin_Watts | Just XI. I didn't know about it. | 12:02.47 |
| kens | Ah OK, I've had it a while. Did you notice they are pushing their web conversion tools in it ? | 12:03.18 |
| Robin_Watts | They were in X too. | 12:13.56 |
| kens | They seem to be pushing it a bit harder.... | 12:18.46 |
| rayjj | kens: I saw in the logs that you are struggling with the device chain. AIUI the 'compositor' devices (such as pdf14 or overprint) _do_ stick themselves in as the current device, and then forward selected calls to their 'target' | 15:41.16 |
| | kens: isn't that what you need for the PCL FirstPage implementation ? | 15:41.40 |
| kens | rayjj, yes that's what I need, but they don't do it themselves, the interpreter does it. I need the device to insert a new device in front of itself. It can't be done. At lreast not simply | 15:42.24 |
| rayjj | basically a compositor that eats marking ops until the FirstPage is reached | 15:42.36 |
| Robin_Watts | kens: I have a trace device that cheats to do it. | 15:43.10 |
| rayjj | kens: so why can't the PCL parser insert the compositor ? | 15:43.14 |
| | that's the interpreter level | 15:43.39 |
| kens | rayjj I want the device to do it so the interpreter doesn't have to, otherwise I have to rewrte all the interpreters, and it isn't easy to get the PostScript interpreter to do it, since it relies on an entry in systemdict (FirdtPage) | 15:44.11 |
| | Robin_Watts : I'd be interested to know how you do that. | 15:44.31 |
| | I have one idea left, and its nasty | 15:44.46 |
| Robin_Watts | kens: I stash the function pointers, then rewrite them function pointers | 15:45.34 |
| | So my new function pointers do what I need, then pass on control to the old ones. | 15:46.09 |
| kens | THat's what I was considering, but you also have to worry about the data from the device | 15:46.16 |
| Robin_Watts | The really 'hacky' bit is *where* I store the old function pointers. | 15:46.29 |
| | kens: That didn't matter for my purposes. | 15:46.46 |
| kens | Hmm, OK so you don't rewrite the device, hadn't thgought of | 15:46.55 |
| rayjj | kens: and you've discarded the notion of having the pdfwrite device have a unique set of procs to plug in when FirstPage is > 1, and plug in the "real" procs when the number of pages is reached ? | 15:46.57 |
| kens | rayjj pdfwrite isn't the problem. | 15:47.11 |
| rayjj | we have several devices that keep multiple sets of procs (and orig_procs) | 15:47.31 |
| henrys | rayjj: we want it to work with all device, pcl performance sucks as it works now with raster devices... | 15:48.10 |
| rayjj | kens: I thought that pdfwrite/ps2write *was* the problem (that they accumulated all of the stuff up to the FirstPage) | 15:48.18 |
| henrys | rayjj: that was the original problem but then we thought the entire business was not done well. | 15:49.02 |
| rayjj | kens: FirstPage with raster devices is really only debugging | 15:49.04 |
| kens | rayjj I'd like to solve it for all devices, so we don't waste time rendering stuff and calling erasepage | 15:49.19 |
| rayjj | no "real world" printer uses it, and we don't care about performance for those that want to use it on computers | 15:49.48 |
| | the problem (AIUI) was that the pdf we created was bloated | 15:50.07 |
| kens | I think we do worry about performance, ask norbert, he uses FirstPage too I belive | 15:50.27 |
| rayjj | hmm... I wonder why norbert would be using it ??? | 15:50.55 |
| kens | Robin_Watts : Its an intersting idea. I'll explore it more tomorrow. | 15:51.04 |
| henrys | rayjj: I'd like it work well, I think it's horrible as it is now. | 15:51.32 |
| kens | henrys even if I get this to work it'll be horrible :-) | 15:51.53 |
| henrys | kens: it should be an improvement over erasepage | 15:52.28 |
| rayjj | Robin_Watts: kens: the prn device swaps procs for going back and forth from page rendering to clist rendering | 15:52.37 |
| kens | henrys it'll be faster at least | 15:52.39 |
| rayjj | clist erasepage (recent) actually clears the clist files (it used to accumulate stuff) | 15:54.02 |
| | iirc, norbert always uses clist mode, so he won't be rendering stuff to then erase it (but it will be writing the clist and resetting the file pointers) | 15:55.28 |
| | since the prn device already keeps orig_procs (for swapping to/from clist procs) handling a set of 'non-marking' procs there should be easy to add | 15:56.54 |
| Robin_Watts | kens: If you are in control of both the device that wants to have a new device put in front of it, and the device that you want to put in front, you're in good shape. | 15:59.00 |
| | If you only have control of the device that you want to put in front, then it's trickier. | 15:59.16 |
| kens | Robin_Watts : Like I said I'll look at it tomorrow | 15:59.17 |
| Robin_Watts | Cos where do you put the function pointers you are replacing? | 15:59.33 |
| kens | In a new place in the device structure, which I will add | 15:59.54 |
| Robin_Watts | I hid them in the place of one of the unused function pointers. Possibly one of the pattern managing ones. | 16:00.05 |
| henrys | can they be stuck in the library context in mem? | 16:00.06 |
| rayjj | kens: if you want to hand off doing it for prn devices to me, let me know. It seems pretty straightforward. I thought you were working on it only for pdfwrite | 16:00.31 |
| Robin_Watts | henrys: Ew. | 16:00.37 |
| rayjj | why not add orig_procs to the device structure (not just the prn_device) ? | 16:01.22 |
| kens | rayjj : I want it for all the devices I can, that means device from gdevprn.c and gdevvec.c. If a device isn't derived from one of those it won't work, and TBH I don't think I care about those. | 16:01.23 |
| | rayjj see above^^ | 16:01.35 |
| Robin_Watts | Oh, actually, there is a big drawback to working this way. | 16:02.17 |
| rayjj | how far above (about adding an extra pointer to all devices ?) | 16:02.18 |
| Robin_Watts | In order to work properly, when you pass control onto the other device, you should put all the function pointers back, then pass it on, then take 'em out again. | 16:02.57 |
| | That can be painful. | 16:03.01 |
| kens | Where I said in a new place in the device structure, which I will add | 16:03.21 |
| rayjj | kens: OK. | 16:03.36 |
| kens | Robin_Watts : for your device that's a problem, for mine its not. | 16:04.33 |
| rayjj | Robin_Watts: I don't quite follow that. We do this for clist/page mode in the prn devie and it isn't that bad. Recall this "switch" is only on page boundaries | 16:05.17 |
| henrys | kens: so your saying add non marking procs to all devices? Well a pointer to an structure of non marking functions. | 16:05.40 |
| kens | henrys, more or less yes | 16:06.05 |
| | Only not like that. Swap the pointers to the procs so they either pass on or not | 16:06.25 |
| rayjj | henrys: actually we probably need to be "smart" about swapping in the non-marking procs (as the prn device is) | 16:06.42 |
| | some procs get retained for the non-marking device (kept the same as normal) | 16:07.21 |
| | s/non-marking device/non-marking mode/ | 16:07.41 |
| henrys | good luck kens I'll return to my salt mine. | 16:08.19 |
| kens | Thanks henrys | 16:08.29 |
| rayjj | and I back to cust 532's latest mess | 16:08.46 |
| henrys | as I do this refactoring chrisl pointed out I just want to rip out everything of plmain, what a mess. | 16:09.23 |
| rayjj | I assume all of you got the email to MQ. Interesting that mupdf did so well on performance. | 16:09.48 |
| | henrys: I wouldn't be sorry to see plmain get re-done. It sort of grew over time and shows it | 16:10.39 |
| Robin_Watts | ray: "However, since PDF files must be completely downloaded and stored before processing can begin"... | 16:12.21 |
| zlnk | Hi, is there someone? | 16:12.31 |
| Robin_Watts | mupdf can work on linearised stuff and hence be faster. | 16:12.38 |
| rayjj | Robin_Watts: yes? | 16:12.42 |
| Robin_Watts | it could even arrange to optimise the transfer to the printer given a smart enough driver. | 16:13.07 |
| henrys | zlnk: was the a philosophical question? | 16:13.08 |
| zlnk | Hi, I want to make a question, It happens that I have problems using the function get_globals | 16:13.08 |
| rayjj | Robin_Watts: only if the file is linearised _and_ the file is sourced from HTTP | 16:13.27 |
| Robin_Watts | zlnk: This is MuPDF on Android, I'm guessing. | 16:13.30 |
| zlnk | yes | 16:13.36 |
| | I followed the idea of ChoosePdfActivity | 16:13.54 |
| Robin_Watts | rayjj: HTTP or some other "directable" source. | 16:13.56 |
| zlnk | with the difference, that on mupdf.c I have an array of globals structures | 16:14.15 |
| rayjj | Robin_Watts: yes, we could use gs on the host to create pdf in the "ordered" form (streamable) as we do for ps2write | 16:14.35 |
| zlnk | so, on the function get_globals, I can not return the structure for one | 16:15.22 |
| rayjj | Robin_Watts: but I'm working with "current" code. GS could also process linearised PDF's from a suitable source without too much difficulty | 16:15.32 |
| | Robin_Watts: and actually I realized that mupdf -B should really be compared to gs using -dBandHeight= (not -dBufferSpace=) to be equivalent | 16:16.35 |
| | I'm going to compare RAM and performance for gs in that mode and send an update if it differs much | 16:17.30 |
| | Robin_Watts: mupdf _could_ also, fairly easily I would imagine, do the equivalent of BGPrint (rendering the display list in a separate thread while loading a new display list in the main thread), right ? | 16:18.55 |
| Robin_Watts | rayjj: oh yes. | 16:19.25 |
| | and we can run on multiple threads for rendering too. | 16:19.44 |
| rayjj | but given the VERY high level of the display list for mupdf, the amount of time creating the display list for a page is probably much smaller than gs creating the clist | 16:20.04 |
| Robin_Watts | yes, which means the important time is the rendering time, and that does get divided up nicely. | 16:20.48 |
| rayjj | Robin_Watts: mudraw can't (currently) -- also fairly easy to add. It's all sort of wrapped up in "it's newer different code" | 16:21.07 |
| Robin_Watts | mudraw is deliberately free of threading, yes. | 16:21.41 |
| rayjj | Robin_Watts: unfortunately, if a feature isn't there, it's hard to justify adding it just to do performance benchmarks for a non customer on speculation | 16:23.06 |
| Robin_Watts | rayjj: True, I wasn't suggesting we add any of this. | 16:23.45 |
| | just mentioning that the possibilities exist. | 16:23.59 |
| rayjj | Robin_Watts: and the problem is that for a "real world" printer, PS and PCL are the main formats (except maybe for linux). PDF is still just getting started | 16:24.27 |
| jogux | rayjj: the best selling 'PC' last year can /only/ print to pdf ;-) | 16:25.38 |
| rayjj | Robin_Watts: right. I realize that a lot of these could be added fairly easily to the mupdf code base (more easily than adding stuff to gs). | 16:25.49 |
| | jogux: yeah, but that *usually* uses the cloud to process the PDF to what the printer wants | 16:26.24 |
| | but our customer 532 _does_ take AirPrint PDF's directly | 16:26.53 |
| | the screwy PDF's we get from iPad's was a source of some bugs | 16:27.25 |
| jogux | rayjj: :-) yeah, airprint / ipad was what I was referring to. | 16:27.27 |
| rayjj | jogux: I guessed that from the "PC" :-) | 16:27.42 |
| jogux | :-) | 16:27.44 |
| rayjj | for company M's customer base, offerering *both* mupdf (for PDF) and GhostPDL (for PCL or PS) isn't that bad. mupdf is only an extra 10Mb and if we share fonts and CMap resources, could be MUCH less overhead | 16:29.21 |
| | certainly no worse that cust 532 that has 3 different graphics engines (us for PDF, their own PS+PCL and a third party for XPS) | 16:30.17 |
| | but at least they share fonts | 16:30.39 |
| | and Adobe PS + GhostPCL is in some real world printers, too | 16:31.02 |
| | Robin_Watts: BTW, I did look at memory usage and performance numbers for the timing files in XPS with mupdf and they were so horrendous I decided to just leave XPS out and mention it only in passing | 16:32.49 |
| Robin_Watts | More horrendous than anything else doing xps? | 16:33.11 |
| | or just that xps is hideous compared to pdf ? | 16:33.22 |
| rayjj | yeah, surprisingly even worse than gxps | 16:33.28 |
| Robin_Watts | That's... odd, given it's the same code, essentially. | 16:33.46 |
| rayjj | but I don't want to waste any time on XPS | 16:33.52 |
| | I'll leave it for Tor to investigate ;-) | 16:34.35 |
| rayjj | bets tor8 is fervently hoping we never get a real XPS customer | 16:36.15 |
| | Robin_Watts: when using threads to render bands, does mupdf use a common (shared by all threads) memory allocator that locks on each alloc/free ? | 16:39.00 |
| | mutex locks are a killer on Windows | 16:39.49 |
| Robin_Watts | rayjj: mupdf calls user specified alloc functions. | 16:48.39 |
| | The user is free to implement how he would like. | 16:48.55 |
| | but there is no specific support for 'different' allocators per thread. | 16:49.06 |
| rayjj | Robin_Watts: I see. So the memory allocator isn't part of the rendering context ? | 16:50.01 |
| Robin_Watts | indeed not. | 16:50.39 |
| rayjj | Robin_Watts: but does the allocator get the context passed to it ? | 16:50.43 |
| Robin_Watts | No, and that wouldn't help if it did. | 16:51.57 |
| | Cos you render to the display list in one thread (so the allocs are done there) | 16:52.15 |
| | then you use and destroy the display list in another thread (so the frees are done there) | 16:52.31 |
| rayjj | Robin_Watts: do the calls get the fz_alloc_context passed to them ? | 16:52.33 |
| Robin_Watts | No, they get the 'user' passed to them. | 16:52.44 |
| rayjj | Robin_Watts: OK, so at least the 'user' struct can handle making locking less frequent than just calling system malloc/free | 16:53.48 |
| | Forward 1 day (to 2014/10/18)>>> | |
IRC Logs