| | <<<Back 1 day (to 2013/10/09) | 2013/10/10 |
| mvrhel_laptop | Robin_Watts: for the logs, does mupdf fail for you on page 13 of Altona_Technical_v20_x4.pdf? seems to be a jpeg 2000 image issue | 04:32.53 |
| | I get a openjpeg error: Bad BPCC header box (bad size) | 04:33.59 |
| | which leads to error: Failed to read JPX header | 04:34.16 |
| | and I seem to have something amiss in my windowsrt codes since the throw is not getting back to my catch that I have in my code for some reason | 04:38.19 |
| | leading to an abrupt exit :( | 04:38.35 |
| henrys | mvrhel_laptop: it should fail on gs also but we'd see that in the logs no? | 04:39.20 |
| mvrhel_laptop | henrys: yes. I was surprised to see this | 04:39.56 |
| | let me double check with gs | 04:40.07 |
| | ok we get the error in gs too | 04:43.04 |
| | but, | 04:43.21 |
| | it keeps pushing on with the other images. this is likely all due to my poor error catching | 04:43.40 |
| | hmm | 04:44.11 |
| | actually the gs output looks good | 04:44.16 |
| | in fact, the AR output is wrong | 04:44.22 |
| | it fails on the CIELAB encoded image | 04:44.31 |
| henrys | you probably return an error back to the pdf parser and it keeps going. | 04:45.36 |
| mvrhel_laptop | yes | 04:45.59 |
| henrys | I met a guy in chicago who worked on the venerable Altona tests. | 04:46.20 |
| mvrhel_laptop | oh wow | 04:46.25 |
| | ok. I am going to call it a night. i will trace through the error return tomorrow and get this fixed up | 04:47.08 |
| henrys | mvrhel_laptop: see you tomorrow | 04:47.28 |
| mvrhel_laptop | yes | 04:47.32 |
| Vignesh__ | Hi, I have integrated MuPDF1.3 Successfully in Android. But I am not able to navigate page to particular PDF page number. | 05:51.06 |
| | Now I am able to open PDF files. | 05:51.41 |
| | How to load PDF file with Particular page number? | 05:52.11 |
| | Hi buddies. | 05:53.24 |
| sebras2 | vignesh__: you may have to wait a while for the european mupdf devs to wake up. | 06:18.17 |
| vignesh__ | Thanks for your info sebras. | 06:37.20 |
| sebras2 | tor7: Shanghai says good morning. | 07:36.10 |
| tor7 | sebras2: hey. so, back at work then? | 07:36.32 |
| sebras2 | tor7: yup, since tuesday. I think I mentioned this to chrisl, but maybe you weren't awake yet. | 07:37.57 |
| Robin_Watts | tor7: Some reviews on robin/master when you get a mo. | 09:34.27 |
| tor7 | Robin_Watts: naming question -- pixmap_as_png, or new_png_from_pixmap? | 09:40.09 |
| Robin_Watts | tor7: I'd be happy with either. | 09:41.14 |
| | pixmap_as_png seems right to me, as we're getting a version of one thing as another. | 09:41.32 |
| | but I'm notoriously poor at such decisions. | 09:41.48 |
| tor7 | the X_from_Y is what I usually do for conversion functions | 09:42.24 |
| | X_with_Y for constructors that take arguments but are not conversions | 09:42.41 |
| Robin_Watts | pixmap_from_png then ? | 09:43.01 |
| tor7 | on the second one, you use scissor_len and scissor_max, I'd prefer scissor_cap for consistency with other code | 09:43.03 |
| | png_from_pixmap, but png isn't a type so maybe what I said doesn't apply cleanly | 09:43.35 |
| kens | IMOif its a destructive c onversion then its pixmap_as_png and if its non-destructive then its new_png_from_pixmap. But I know nothing :-) | 09:43.36 |
| tor7 | new_buffer_with_pixmap_as_png would be the apple way of naming :) | 09:43.52 |
| Robin_Watts | draw device uses stack_max. I should fix that to be _cap too then. | 09:44.28 |
| | likewise pdf-type3.c | 09:45.59 |
| | kens: it's non destructive. | 09:46.50 |
| | so fz_pixmap_from_png would be my preference. | 09:47.13 |
| | And windows wants me to reboot. brb. | 09:47.36 |
| tor7 | I hope you mean png_from_pixmap... | 09:52.36 |
| kens | I'm sure he does | 09:52.46 |
| tor7 | Robin_Watts: IMO it should be new_png_from_pixmap (to make clear that the caller owns the resulting pointer) | 09:53.33 |
| Robin_Watts | tor7: ok. | 09:54.28 |
| tor7 | Robin_Watts: other than the naming, the 4 look good | 09:55.59 |
| Robin_Watts | Ta. | 09:57.34 |
| | tor7: OK. Tweaked versions, plus a stack_max -> stack_cap one on robin/master | 10:11.54 |
| tor7 | Robin_Watts: s/know/now/ in commit message | 10:14.39 |
| Robin_Watts | D'Oh. | 10:17.55 |
| | Fixed. | 10:18.37 |
| | tor7: happy with them now? | 10:34.06 |
| tor7 | Robin_Watts: yes. | 11:02.12 |
| Robin_Watts | tor7: Thanks | 11:57.26 |
| pulse | Hi, I'm struggling with rezise of a PDF. I would like to take a PDF in A3 (or larger) format and convert it to an a4 postscript file. Currently I manage to get an a4 postscript, but the content is not resized (meaning I get just a corner of the original PDF)? Any suggestions? | 12:37.41 |
| | I have tested: gs -sOutputFile=85529442_2.ps -sDEVICE=pswrite -dFitPage -sPAPERSIZE=a4 -dFIXEDMEDIA -sOutputFile=85529442.ps 85529442.pdf | 12:37.56 |
| kens | -dPDFFitPage | 12:37.57 |
| pulse | Together with the rest? | 12:38.22 |
| kens | Well you have to set the size of media you want, and you have to make it fixed or requests will cahnge it. So yes | 12:38.58 |
| | -dFitPage is new though and its *supposed* to set -dPDFFitPage | 12:40.01 |
| | What version of Ghostscript are you using ? | 12:40.22 |
| pulse | Ghostscript 5.50 | 12:41.19 |
| kens | THat's a joke right ? | 12:41.26 |
| pulse | nope, it's not... Is it that old? | 12:41.49 |
| kens | Old, well you could say that. The current version is 9.01 and the switch you are using opnly exists in the current version | 12:42.10 |
| | 9.10 | 12:42.15 |
| | It will have no effect on that elderly a version of Ghostscript | 12:42.33 |
| Robin_Watts | pulse: 8.71 is 4 years old now. | 12:42.47 |
| pulse | ok, thanks alot... I will complain to the "installation department" | 12:43.00 |
| Robin_Watts | Actually... it's possible that that's GNU Ghostscript rather than our Ghostscript. | 12:43.19 |
| kens | Well, we can't support that :-) | 12:43.30 |
| Robin_Watts | IIRC GNU Ghostscript stopped at 5.50 - and I'm not sure their version numbers match up to ours. | 12:43.41 |
| pulse | Right, it is GNU Ghostscript 5.50 (2000-2-13) | 12:43.54 |
| kens | Ghostscritp 5.50 was released in September 1998, so its 15 year sold now | 12:44.07 |
| | OK so that version is 13 years old then | 12:44.22 |
| | You're going to have to use something newer | 12:44.40 |
| kens | assumes it took 2 years to 'port' Aladdin Ghostscript (as it was then) 5.50 to GNU 5.50 | 12:45.23 |
| pulse | I will ask the persons who installed it, why they installed thet version. It was actually installed last week... | 12:45.45 |
| Robin_Watts | pulse: Possibly it's the version in their distros repo. | 12:48.45 |
| | Possibly they downloaded 'GNU Ghostscript' rather than 'Ghostscript'. | 12:48.56 |
| pulse | Do you know if there are newer versions for AIX 7.1 | 12:49.12 |
| Robin_Watts | It's a confusing thing, as our Ghostscript (the real ghostscript, the one that is actually developed) is released both under the GNU GPL and commercially. | 12:49.32 |
| | but some diehard zealots decided to repackage our releases as "GNU Ghostscript" a while ago, hence that name is polluted. | 12:50.08 |
| | Our version will work on AIX, I believe. | 12:50.16 |
| pulse | and is that a comersial version? | 12:50.41 |
| Robin_Watts | no, both the GNU GPL version and the commercial version will work on AIX. | 12:50.59 |
| | (The differences between the commercial version and the GNU GPL version are minimal) | 12:51.32 |
| pulse | and where can I find a newer version for AIX? | 12:51.49 |
| Robin_Watts | downloads.ghostscript.com | 12:52.08 |
| | You'll need to download the source and build it. We don't ship binaries. | 12:52.20 |
| pulse | ok, thanks alot | 12:54.22 |
| Farmer_Dan | exit | 13:15.39 |
| Robin_Watts | tor7: Another review for you. | 13:55.58 |
| tor7 | Robin_Watts: LGTM. | 14:09.06 |
| Robin_Watts | ta. | 14:10.29 |
| | fts_01_0106.pdf is showing up a problem with the gradient stuff. | 14:10.51 |
| | tor7: Currently all our fz_new_pixmap functions return us solid white pixmaps. | 14:20.34 |
| | I need one that's transparent I think. | 14:20.46 |
| | Currently we have no 'clear this pixmap to be transparent' function | 14:21.00 |
| | ignore me. | 14:26.59 |
| | tor7: Another quick fix for you on robin/master. No rush.Thanks. | 14:37.13 |
| kens | I have to go out to the dentist for a checkup, back in a bit | 14:38.58 |
| henrys | oh no biro will be an entertaining customer, now that he's paying his bills | 14:45.40 |
| Robin_Watts | tor7: fts_09_0910.pdf - we don't show the dingbats line. Presumably that's cos we don't have a fallback font for /ZapfDingbats? | 15:12.37 |
| selim | hello | 15:22.11 |
| Robin_Watts | hi | 15:22.21 |
| ghostbot | niihau | 15:22.21 |
| selim | I would like to give a title and author generated PDF but without a file directly from the command line thank you | 15:23.45 |
| Robin_Watts | selim: sorry, I didn't understand that. Can you be more explicit please ? | 15:24.36 |
| tor7 | Robin_Watts: new_pixmap returns uninitialised data. fz_clear_pixmap_with_value is what you already found, I reckon :) | 15:24.49 |
| Robin_Watts | tor7: No, fz_clear_pixmap was what I wanted :) | 15:25.06 |
| | fz_clear_pixmap_with_value makes it solid. | 15:25.16 |
| | but, basically, yes. Thanks :) | 15:25.40 |
| selim | I want to give a title and author name to my pdf but without using "Pdrmark" file. | 15:28.48 |
| Robin_Watts | selim: You're using gs to generate you a pdf file from some input, and you want to specify the title and the author without using a separate file? | 15:29.57 |
| selim | yes | 15:32.58 |
| Robin_Watts | You can probably do what you want using -c, but I'm no expert. kens? | 15:33.46 |
| kens | Just got back | 15:34.10 |
| chrisl | You can certainly pass pdfmark with the -c option | 15:34.20 |
| kens | You can feed PostScript into GS fomr teh command line, see the -c switch | 15:34.23 |
| | construct an appropriate pdfmark and it should be fine. | 15:34.39 |
| | You cannot do it without specifying a pdfmark though | 15:35.04 |
| selim | i'm sad :( | 15:47.18 |
| mvrhel_laptop | good morning | 16:27.01 |
| Robin_Watts | mvrhel_laptop: Morning. | 16:29.53 |
| | It seems that SVG 1.1 doesn't support any meaningful blend modes. | 16:30.08 |
| | and 1.2 (which was stillborn) supports them in an even more complex way than PDF. | 16:30.33 |
| | so I think I'm gonna punt on that for now. | 16:31.08 |
| mvrhel_laptop | question for you Robin_Watts | 16:32.17 |
| | so if fz_load_jpx does a fz_throw, when in a call stack from pdf_load_image how is the error handled? | 16:34.12 |
| | because right now, I am ending up in exit(EXIT_FAILURE) | 16:34.40 |
| Robin_Watts | let me look. | 16:34.49 |
| | The error is caught in pdf_load_image and then rethrown. | 16:35.27 |
| | sorry... | 16:35.44 |
| mvrhel_laptop | ? | 16:35.51 |
| | that is what I would have thought | 16:35.58 |
| Robin_Watts | fz_load_jpx is called from fz_load_jpx | 16:35.58 |
| mvrhel_laptop | ? | 16:36.06 |
| | its recursive? | 16:36.11 |
| Robin_Watts | I am incapable of typing today. let me try again. | 16:36.15 |
| | fz_load_jpx is called from pdf_load_jpx. | 16:36.26 |
| | IF the former throws the latter catches and rethrows. | 16:36.34 |
| mvrhel_laptop | yes that is working fine | 16:36.49 |
| Robin_Watts | pdf_load_jpx is called from pdf_load_image_imp | 16:37.02 |
| | and again that will catch and rethrow. | 16:37.15 |
| mvrhel_laptop | yes that works fine too | 16:37.36 |
| | but should pdf_load_image not have a catch also? | 16:38.26 |
| Robin_Watts | That is called either recursively or from pdf_load_inline_image or from pdf_load_jpx or from pdf_load_image | 16:38.31 |
| mvrhel_laptop | well, in the case I have here, it is coming from pdf_load_image | 16:38.49 |
| kens | OK i'moff, night all | 16:38.50 |
| Robin_Watts | and pdf_load_jpx is only called from pdf_load_image_imp. | 16:39.00 |
| | Sorry, I'm following this through. bear with me. | 16:39.13 |
| | So it looks to me like both pdf_load_image and pdf_load_inline_image can throw exceptions. | 16:39.33 |
| mvrhel_laptop | I don't know about pdf_load_inline_image, but pdf_load_image seems to | 16:40.37 |
| Robin_Watts | So, anything that calls pdf_load_image (or pdf_load_inline_image) should be prepared to catch any errors that are thrown. | 16:41.10 |
| mvrhel_laptop | right | 16:41.25 |
| Robin_Watts | pdf_load_image looks to be called by pdfextract.c (and we can ignore that) or from pdf_run_Do | 16:41.44 |
| mvrhel_laptop | and rethrow I would assume | 16:41.53 |
| | since higher up it will decide what to do | 16:42.05 |
| | and just not draw that object likely | 16:42.24 |
| Robin_Watts | mvrhel_laptop: Well, it can only rethrow if whatever is higher up is expecting to catch things, but yes. | 16:42.28 |
| mvrhel_laptop | well higher up from pdf_load_image the call stack is catching | 16:42.50 |
| | but pdf_load_image is not throwing anything | 16:42.58 |
| Robin_Watts | pdf_run_Do is called from pdf_run_keyword within it's own try/catch/rethrow. | 16:43.02 |
| | So what problem are you seeing here? You're seeing it throw, but not seeing it be caught? | 16:43.37 |
| | Can you pastebin the callstack ? | 16:43.51 |
| mvrhel_laptop | Robin_Watts: just run page 13 of the Altona file | 16:44.07 |
| | Altona_Technical_v20_x4.pdf | 16:44.54 |
| Robin_Watts | ok. | 16:45.59 |
| | so I get errors thrown and I get the page rendered with an incomplete image. | 16:46.22 |
| | What's the problem? | 16:46.30 |
| mvrhel_laptop | well for me I end up with the following | 16:46.40 |
| | I have a throw in load-jpx.c line 128 | 16:47.38 |
| | which is rethrown in pdf_load_jpx | 16:48.12 |
| Robin_Watts | right. Have you caught this in the debugger? | 16:48.21 |
| mvrhel_laptop | yes | 16:48.26 |
| | which is rethrown in pdf_load_image_imp | 16:48.48 |
| Robin_Watts | I *think* you can grab the callstack out of MSVC. | 16:48.48 |
| mvrhel_laptop | which is then ends up in the else statement of the throw in error.c | 16:49.42 |
| Robin_Watts | Click in the call stack window. Ctrl A, Ctrl C, then http://pastebin.com and paste it in there. | 16:49.53 |
| mvrhel_laptop | since ex->top >= 0 is false | 16:49.54 |
| | when do you want me to grab the stack? | 16:50.11 |
| | when it first is thrown? | 16:50.17 |
| Robin_Watts | yes. | 16:50.22 |
| mvrhel_laptop | ok hold on | 16:50.25 |
| Robin_Watts | Cos that way I can trace it all the way back. | 16:50.47 |
| mvrhel_laptop | ok it is pasted | 16:51.50 |
| Robin_Watts | You need to give me the URL :) | 16:52.06 |
| mvrhel_laptop | he ehe | 16:52.32 |
| | http://pastebin.com/fDxqGkkE | 16:52.34 |
| | sorry | 16:52.39 |
| Robin_Watts | My mentalism is a bit worn out by the end of the week :) | 16:52.52 |
| | Ok, let me catch it in the same place and we can walk it back up the stack together. | 16:53.49 |
| mvrhel_laptop | sounds like a great idea | 16:54.03 |
| | maybe I don't have something set up correctly | 16:55.35 |
| Robin_Watts | ok, so I've got a break in fz_throw and another in fz_rethrow | 16:56.00 |
| mvrhel_laptop | let me know when you are at the top | 16:56.01 |
| Robin_Watts | ok, so I'm in fz_load_jpx and that's throwing, same as for you. | 16:56.35 |
| mvrhel_laptop | ok I added break points there too | 16:56.40 |
| Robin_Watts | and that ends up back in pdf_load_jpx | 16:57.01 |
| mvrhel_laptop | yes in the catch | 16:57.26 |
| Robin_Watts | That rethrows to pdf_load_image_imp | 16:57.30 |
| mvrhel_laptop | for the rethrow | 16:57.37 |
| Robin_Watts | yeah. | 16:58.03 |
| mvrhel_laptop | yes, I am back in pdf_load_image_imp | 16:58.06 |
| | and I think this is where we will part ways | 16:58.21 |
| Robin_Watts | From there it ends up in pdf_run_keyword (oops, that's in an fz_rethrow_message) | 16:58.26 |
| mvrhel_laptop | from the rethrow in the catch in pdf_load_image_imp I am going to end up exiting | 16:59.25 |
| Robin_Watts | mvrhel_laptop: OK, so something very odd is up. | 16:59.56 |
| | Let me get back to that rethrow. | 17:00.07 |
| | ok, so I'm stopped in pdf_load_image_imp at the start of the 'catch' block. | 17:01.04 |
| | and I'm looking at the value of ctx->error | 17:01.19 |
| | I have top = 19, errcode = 1, message = "Failed to read JPX header" | 17:01.42 |
| mvrhel_laptop | I have top is -1 | 17:01.43 |
| Robin_Watts | That'd be the problem then :) | 17:01.58 |
| mvrhel_laptop | which is the source of my problem | 17:02.01 |
| Robin_Watts | OK, so something is corrupting the error stack. | 17:02.13 |
| | Let's rerun and stop the program at the very first throw and see what the value is there. | 17:02.26 |
| mvrhel_laptop | ok | 17:02.30 |
| Robin_Watts | I have top = 21 there. | 17:02.47 |
| mvrhel_laptop | hold on are you just running page 13 by itself? | 17:03.01 |
| Robin_Watts | I am. | 17:03.17 |
| mvrhel_laptop | one thing I have to say is that I am opening the file, and the thumbnails are off getting rendered leading to this | 17:03.30 |
| | so... | 17:03.34 |
| Robin_Watts | The error stack should reset itself after each and every page render. | 17:03.45 |
| mvrhel_laptop | when I say the file, I mean the whole document | 17:03.47 |
| | ok | 17:03.48 |
| | let me get just the single page though | 17:04.05 |
| | hold on | 17:04.08 |
| Robin_Watts | mvrhel_laptop: The exact number doesn't matter (as long as it's not -1 :) ). I could imagine that we'd be slightly different due to different numbers of try/catch in the calling code. | 17:04.52 |
| mvrhel_laptop | ok | 17:04.59 |
| | hold on | 17:05.04 |
| | so where do you want me to catch it? | 17:05.57 |
| | with the break point? | 17:06.03 |
| Robin_Watts | In the very first fz_throw | 17:06.06 |
| | yeah. | 17:06.08 |
| mvrhel_laptop | ok that is fz_throw(ctx, FZ_ERROR_GENERIC, "Failed to read JPX header"); | 17:06.28 |
| Robin_Watts | yeah. | 17:06.39 |
| | What is ctx->error->top there? | 17:06.46 |
| mvrhel_laptop | and the value is 1 | 17:06.51 |
| Robin_Watts | Ok, that's clearly screwed. | 17:07.05 |
| | every time we go through a try, error->top should get incremented by 1. | 17:07.21 |
| mvrhel_laptop | maybe it is due to the cloning that I am doing? | 17:07.28 |
| Robin_Watts | mvrhel_laptop: What cloning ? | 17:07.39 |
| | context cloning? | 17:07.44 |
| | Largely, the purpose of cloning the context is so that every new cloned context gets its own error stack. | 17:08.21 |
| mvrhel_laptop | oh ok | 17:08.27 |
| Robin_Watts | so we should never have 2 threads using the same error stack at the same time. | 17:08.39 |
| mvrhel_laptop | well let me put a break proint at my call into fz_run_page | 17:08.52 |
| Robin_Watts | Can you see the other threads in the debugger? | 17:09.13 |
| mvrhel_laptop | and watch if the ctx error count is getting incremented | 17:09.13 |
| | yes | 17:09.35 |
| Robin_Watts | I wonder if this is to do with the context changing at some point down the call stack... | 17:09.52 |
| mvrhel_laptop | but I am doing a single page now | 17:09.54 |
| | so we should not be running into that problem | 17:10.14 |
| | I would hope... | 17:10.21 |
| Robin_Watts | The file ends up bound to a single context. | 17:10.51 |
| | Let me try and find the language so I can remind myself of it. | 17:11.04 |
| mvrhel_laptop | let me put a breakpoint earlier and monitor error->top | 17:11.33 |
| Robin_Watts | ok. | 17:12.05 |
| mvrhel_laptop | so before my call into fz_run_page I am in a fz_try and the value is 0 for error->top | 17:12.25 |
| Robin_Watts | You could breakpoint fz_push_try | 17:12.34 |
| | That's called on every fz_try. | 17:12.47 |
| | but go on. | 17:13.02 |
| mvrhel_laptop | ok let me do that | 17:13.03 |
| Robin_Watts | Ah, I wonder if this is to do with the cloning. I bet the value of ctx is changing part way down the callstack. | 17:15.01 |
| | You'll probably see it as ctx->error->top suddenly dropping to 0. | 17:15.29 |
| mvrhel_laptop | it is floating around between 4 and -1 as we go through the page | 17:15.52 |
| | yes | 17:15.57 |
| | so I need to do my interface differently | 17:16.21 |
| | so should I not be doing the context clone like I am in RenderPage in muctx.cpp? | 17:16.58 |
| Robin_Watts | Ok, so look at: docs/overview.txt | 17:16.59 |
| mvrhel_laptop | ok | 17:17.03 |
| Robin_Watts | In the Multi-threading section. There are 3 rules. I think you're running foul of rule 2. | 17:17.23 |
| | all the fz_ functions take an fz_context *, and that's how we find the error stack. | 17:19.14 |
| mvrhel_laptop | I guess I was confused due to the fact that multi-threaded. c in renderer it does a clone | 17:19.45 |
| | at line 91 | 17:20.00 |
| Robin_Watts | When we start calling pdf_ functions, they all take a pdf_document *. We didn't want to have to make them take both a pdf_document * and an fz_context *, so we put the fz_context * in the pdf_document *. | 17:20.09 |
| mvrhel_laptop | and a comment above | 17:20.15 |
| | implies why you do that | 17:20.20 |
| Robin_Watts | mvrhel_laptop: Right. We clone the context so we can safely be using a different thread. | 17:22.01 |
| | but we only access a display list from within that thread. | 17:22.15 |
| | We never touch the original document. | 17:22.19 |
| | We make the original context in main. And we open the document with that. That binds that context to that document. | 17:23.12 |
| | then in main, we pull out each page from the document into a display list. | 17:23.31 |
| mvrhel_laptop | still on the main thread | 17:23.46 |
| Robin_Watts | Yes. Those pages are then rendered by the 'renderer' threads. And each of those uses a new context. | 17:24.01 |
| mvrhel_laptop | I am doing the fz_load_page after the clone :( | 17:24.06 |
| | ok | 17:24.15 |
| | A subtle point | 17:24.26 |
| Robin_Watts | Yeah, sorry. | 17:24.32 |
| | I have to confess to not appreciating this flaw when I designed the whole context thing. | 17:24.59 |
| mvrhel_laptop | It would probably be good to clarify where that boundary exists. I did read the doc and looked over the code | 17:25.06 |
| | but missed this | 17:25.18 |
| | looks like I have some work to do... | 17:25.41 |
| Robin_Watts | I think the docs are clear, but the example may not stress this enough. | 17:25.43 |
| | I will try to improve the comments in the example. | 17:26.04 |
| | If you can think of any improvements to the docs, please say, as the author is never best placed to be critical of his text :) | 17:26.35 |
| Micha` | Hollo werld! | 17:27.15 |
| mvrhel_laptop | sometimes, it is helpful to have a newbie try to create something using the documentation to see if all is clear. when the designer does it, it is easy to leave out what he/she considers to be intuitively obvious to the casual observer | 17:27.19 |
| Robin_Watts | mvrhel_laptop: Indeed. | 17:27.36 |
| | I don't consider this easy or obvious. | 17:27.45 |
| Micha` | Quick question here; why every PDF tool I tried ship the Ink annotations together with a bitmap of what they should like? | 17:28.00 |
| mvrhel_laptop | well, it could be that I am not the smartest bear either | 17:28.01 |
| | let me rework my code and I may bug you later | 17:28.16 |
| | tomorrow or early next week | 17:28.26 |
| Robin_Watts | I consider it a nasty limitation, but it's one we can't easily lift. Hence me trying to make it clear in the docs, and failing, obviously. | 17:28.26 |
| | mvrhel_laptop: Sorry. | 17:28.32 |
| mvrhel_laptop | no problem | 17:28.35 |
| | now I know | 17:28.38 |
| Robin_Watts | ray_laptop: Did you see the email from Len ? | 17:28.44 |
| mvrhel_laptop | and Robin_Watts, it was the cloning as I suspected. i.e. I was wondering when I wrote the code if what I was doing was correct | 17:29.22 |
| ray_laptop | Robin_Watts: yes. I guess he sent it to you because I was already working on something for them (monochrome detection in their 8.71+++ mess) | 17:29.31 |
| mvrhel_laptop | ick. | 17:29.43 |
| Robin_Watts | ray_laptop: Right. | 17:29.54 |
| | I don't have a clue how the selective transparency works, but I'll have a go :) | 17:30.18 |
| | Can you hang around on here so I can ask questions? :) | 17:30.31 |
| ray_laptop | I'm firing up the discovery sim now to see what he's talking about. I did the selective transparency anyway | 17:30.38 |
| | Robin_Watts: I don't think there's much point in pulling you off of working with mvrhel_laptop on mupdf | 17:31.21 |
| Robin_Watts | I think mvrhel_laptop and I are done for now. He's going to be busy stabbing the voodoo doll for a few days. | 17:31.59 |
| mvrhel_laptop | yes | 17:32.15 |
| Robin_Watts | mvrhel_laptop: So you make the display lists on background threads? | 17:32.45 |
| mvrhel_laptop | yes | 17:32.55 |
| Robin_Watts | We could have a 'server thread' that you called to make display lists from the other threads? | 17:33.43 |
| | That server thread would be the only thing using the doc (and hence the only thing using that context). | 17:34.10 |
| | That might save you from having to change the structure of the code wholesale ? | 17:34.33 |
| mvrhel_laptop | that is a good idea | 17:34.37 |
| | luckily though, my interface to mupdf is fairly simple though | 17:34.55 |
| Robin_Watts | So each of your existing worker threads would block on calling the server thread. | 17:35.05 |
| mvrhel_laptop | right | 17:35.17 |
| | let me look things over. I may bug you more about this | 17:35.49 |
| Robin_Watts | no worries. | 17:36.04 |
| Micha` | --- if my question isn't clear, what I'm asking is the reason why there is an AP entry pointing to an Image XObject attached to every Ink annotation, even though the Ink annot already has the InkList of points. Plus, I can't find where this is actually added in the library, pdf_set_ink_annot_list looking innocuous. | 17:36.28 |
| Robin_Watts | Micha`: Sorry, was busy elsewhere. | 17:37.49 |
| | I am not the expert on annotations, but I can try to give you my understanding of the situation. | 17:38.24 |
| Micha` | Thanks :-) This is my first dwelling into PDF internals, so I may be missing something obvious. | 17:39.24 |
| Robin_Watts | Every annotation type generally includes the 'raw information' for the annotation (in this case, for an Ink annotation, it's the list of points), and can optionally include an 'Appearance stream' (the AP entry) that shows what that actually looks like. | 17:39.31 |
| henrys | ray_laptop:geez did you know 801 has made substantial changes to the banding code? If so have you reviewed them? | 17:39.32 |
| Robin_Watts | The process of going from the 'raw data' to the actual rendered appearance is called 'appearance stream synthesis'. | 17:40.38 |
| | Many PDF viewers are 'dumb' in that they don't know about all the different types of annotation that are out there. So they just render the appearance stream given in the document. | 17:41.15 |
| | This means that if new annotation types are added in future, old viewers can still display them - they just can edit them etc. | 17:41.40 |
| | MuPDF was like that until recently when we started the annotation/forms work. | 17:41.58 |
| | So it's 'good form' for a PDF editor to fill out the 'AP' entry as well as putting in the raw points when editing an annotation. | 17:42.51 |
| | Micha`: Does that make sense? | 17:43.07 |
| Micha` | This definitely makes sense; I was thinking about it backwards, as if the stream was preferred to raw data rendering when it is present. However, there's a reason for that, I think: most viewers tend to render better ink annotations when the stream is present... Why is that? | 17:43.32 |
| Robin_Watts | most viewers don't know how to do the appearance stream synthesis. | 17:46.02 |
| | or if they do perhaps they do it badly. | 17:46.29 |
| ray_laptop | Robin_Watts: FWIW, the transparency problem Len reported is not due to selective transparency -- it is doing transparency on all bands | 17:46.47 |
| Robin_Watts | ray_laptop: ok. | 17:47.03 |
| | so it might be a clist problem? | 17:47.16 |
| | I will try to reproduce it here, and cut down the file as much as possible. | 17:47.45 |
| ray_laptop | henrys: I haven't looked at their clist changes. They said they have moved to 9.10, so I hoped they would stay with our code going forward | 17:47.46 |
| Micha` | True that. Evince does a really poor job wrt ink annotations without the stream. But with the stream, I'm surprised that I can still zoom in without seeing the pixels. | 17:47.47 |
| Robin_Watts | Micha`: The appearance stream is not a bitmap. | 17:48.09 |
| Micha` | AH! | 17:48.13 |
| Robin_Watts | It's a sequence of pdf operators; move here, stroke here etc. | 17:48.24 |
| henrys | ray_laptop:yeah they do sse2 stuff in gxclthrd.c not sure that's buying them a lot. | 17:48.45 |
| Robin_Watts | (Well, it COULD be a bitmap, but that'd be bonkers for ink annotations) | 17:48.57 |
| ray_laptop | Robin_Watts: it looks to me like the part that is dark gray in the color view is coming out light gray | 17:49.00 |
| Robin_Watts | ray_laptop: I'm still not at the point of having looked at the file. | 17:49.31 |
| | This is discovery, not dolphin right? | 17:49.38 |
| ray_laptop | Robin_Watts: right. | 17:49.48 |
| | I've got it running with discovery2_es startup | 17:50.21 |
| Micha` | Robin_Watts: Now I do understand. So basically, the extra coordinates are the arcs obtained using interpolation by the PDF creator. That's great. | 17:50.37 |
| Robin_Watts | yeah. | 17:50.46 |
| ray_laptop | Robin_Watts: First I'm going to look at the file with 8.71 | 17:50.53 |
| Micha` | Well, I think we got it all covered, except that I still can't find where this is added in the library. But I'll have a deeper look now that I know what I'm searching for :-) | 17:51.29 |
| ray_laptop | Robin_Watts: ahh.. it's broken in 8.71 the same way as discovery | 17:51.59 |
| | I think I'll go to peeves and do git bisect | 17:52.36 |
| | (it build faster than my laptop) | 17:52.51 |
| henrys | bbiab | 17:53.03 |
| Robin_Watts | ray_laptop: OK, that's good news, I guess. | 17:53.14 |
| | Is it broken in 9.10 ? | 17:53.25 |
| | if not, there might be a clist fix we can backport easily? | 17:53.49 |
| | "we" :) | 17:53.56 |
| ray_laptop | Robin_Watts: it looks OK in 9.10 (at least I think so) | 17:54.57 |
| Robin_Watts | Is it worth me trying to cut the file down? | 17:55.19 |
| ray_laptop | Robin_Watts: not yet. Let me do the bisect first | 17:56.11 |
| Robin_Watts | ray_laptop: fts_24_2526.pdf? | 17:59.36 |
| | Does he mean fts_24_2426.pdf ? | 17:59.47 |
| ray_laptop | 25_2526 | 18:00.41 |
| Robin_Watts | ok, cos 24_2426 looks fine :) | 18:01.00 |
| | oh he gets it right in the title, but wrong in the text. | 18:01.20 |
| ray_laptop | at least that is one that is broken in 8.71 and 9.06 (differently) and looks closer in 9.10 | 18:01.23 |
| Robin_Watts | ray_laptop: How is the file broken? Looks Ok to me. | 18:04.52 |
| ray_laptop | the output is wrong with 9.06 | 18:05.22 |
| Robin_Watts | I'm looking at acrobat and their sim output in wheel. | 18:05.38 |
| | oh, I see I think. | 18:08.06 |
| | grid should be lighter than the cells, not the other way around? | 18:08.36 |
| ray_laptop | Robin_Watts: right | 18:09.18 |
| | and with 9.06 the cells and the grid are almost the same | 18:09.45 |
| | (both light) | 18:09.54 |
| Robin_Watts | With the current code they are almost the same (both dark) | 18:10.18 |
| ray_laptop | Robin_Watts: I think I need to have Acrobat spit out a gray file for me to know what Adobe thinks it should be. | 18:15.58 |
| Robin_Watts | ray_laptop: I'm doing that as we speak. | 18:16.13 |
| ray_laptop | OK. HEAD matches (sort of) Adobe. | 18:20.30 |
| Robin_Watts | adobe looks like the grid is lighter. | 18:20.47 |
| ray_laptop | the grid is lighter than the cells and the center shape is darker than the cell bg | 18:21.23 |
| Robin_Watts | yes. | 18:21.38 |
| | wheras the gs output (from the pgm device) looks darker overall. | 18:21.52 |
| ray_laptop | Robin_Watts: but it's close enough. Just a color management diff | 18:22.05 |
| Robin_Watts | The stroke colors from gs look way too dark IMHO. | 18:22.56 |
| ray_laptop | well, close enough in general. | 18:23.24 |
| rayjj | I'm bisecting (3 runs done) | 18:24.58 |
| Robin_Watts | I'm not convinced, but for the purposes of satisfying len, I won't argue. | 18:25.13 |
| ray_laptop | hmm... git bisect keeps giving me the same merge base :-( | 18:26.44 |
| Robin_Watts | marcosw has binaries stored so he can bisect very fast. | 18:32.04 |
| | marcosw: Are you here? | 18:32.09 |
| ray_laptop | I guess I have enough space on peeved to do that, too | 18:33.07 |
| Robin_Watts | I have the space on my NAS, just not the inclination :) | 18:33.50 |
| ray_laptop | except it would take several days to generate them | 18:34.18 |
| | hmm... 8.71 and 9.06 both work the same as HEAD on that file on linux, but Windows doesn't work the same way | 18:46.22 |
| | Robin_Watts: what platform are you on ? | 18:46.33 |
| Robin_Watts | windows. | 18:46.40 |
| ray_laptop | so now I have to bisect on Windows ? | 18:48.06 |
| Robin_Watts | I reckon 9.06 is wrong, and master is right on windows. | 18:48.43 |
| ray_laptop | Robin_Watts: I'm starting the bisect process on Windows ... | 18:49.31 |
| Robin_Watts | ray_laptop: I think it's somewhere between bbfbdf7 and 3a4439b | 19:22.07 |
| ray_laptop | Robin_Watts: git bisect isn't working for me :-( I did: git bisect start ghostscript-9.06 72afba4 and it said | 19:25.07 |
| | Bisecting: a merge base must be tested | 19:25.08 |
| | [443ad5a4885be7abf5a1e0777275eefbc5322cd2] | 19:25.10 |
| Robin_Watts | ray_laptop: Yeah. I hate git bisect. | 19:25.59 |
| | I do it by hand. | 19:26.02 |
| ray_laptop | since I already know that one is "good" (really bad), I did: git bisect good and it moved to d723d72b3c | 19:26.04 |
| Robin_Watts | between 2a3bf5a and 3a4439b | 19:26.45 |
| ray_laptop | which is only a few days later (Aug 1 2012) | 19:26.46 |
| Robin_Watts | Hmm. I'm testing in december 2012 at the moment. | 19:28.25 |
| ray_laptop | Robin_Watts: git bisect is broken as far as I can tell | 19:30.22 |
| Robin_Watts | between ee7fdd1 and fea783c | 19:31.19 |
| | I'd guess that 6ca50dd might be the puppy. | 19:32.25 |
| | no. between 12763a0 and ee7fdd1 | 19:35.07 |
| | Ok, so it's 12763a0 that fixes it. | 19:35.53 |
| | ray_laptop: ping | 19:36.00 |
| ray_laptop | Robin_Watts: OK. | 19:36.23 |
| Robin_Watts | 1276ea0 even, sorry. | 19:36.59 |
| ray_laptop | Robin_Watts: I don't see that one in the git log | 19:37.04 |
| Robin_Watts | ray_laptop: I'm being called for dinner. I will check back in afterwards. | 19:37.23 |
| ray_laptop | Robin_Watts: OK. | 19:37.30 |
| Robin_Watts | It looks like a plausible commit to me. | 19:37.45 |
| ray_laptop | that one does sound more plausiible | 19:37.46 |
| | Robin_Watts: and the bug 693498 mentions the same file. We should have searched the bug tracker first :-/ | 19:39.06 |
| | I'll go ahead and pull that in and try it on their code... | 19:39.37 |
| | OK. Got the changes in. building to test ... | 20:03.49 |
| | mvrhel_laptop: Robin_Watts just found something you fixed with fts_25_2526, but both he and I did it the hard way, rather than checking the bug tracker. Bug 693498 :-/ | 20:05.17 |
| mvrhel_laptop | I remember that one | 20:05.59 |
| ray_laptop | Robin_Watts: Yep. That fixed it. | 20:06.07 |
| | I'll email Len | 20:06.15 |
| Robin_Watts | ray_laptop: Ha! typical. | 20:15.45 |
| mvrhel_laptop | Robin_Watts: are you back from dinner? | 20:23.19 |
| Robin_Watts | I am. | 20:23.26 |
| mvrhel_laptop | so if I already have a mutex lock set up for ctx is that not sufficient for me to just do the multi-threaded rendering the way that I am? | 20:24.22 |
| | the reason, I ask, is that I replaced all the clones with the global context and all seems fine | 20:24.41 |
| | or is there concern about the document access? | 20:25.06 |
| | i.e. the get page | 20:25.10 |
| | or fz_load_page | 20:25.27 |
| | that is | 20:25.29 |
| Robin_Watts | You can use the global context everywhere if you have a lock so that only one thread can ever be using the context at a time. | 20:25.31 |
| mvrhel_laptop | ok that is the way it is set up now | 20:25.46 |
| Robin_Watts | The context contains the error stack, so only one thread can ever be using it at a time. | 20:25.59 |
| mvrhel_laptop | and what about fz_document | 20:26.15 |
| Robin_Watts | and the doc is hardwired to the context it is created under. | 20:26.24 |
| mvrhel_laptop | can multiple threads read from that to get different pages? | 20:26.28 |
| Robin_Watts | No. | 20:26.36 |
| mvrhel_laptop | oh the doc is under the ctx | 20:26.59 |
| | so if the ctx is locked then access to the doc will also be locked? | 20:27.14 |
| Robin_Watts | No. The doc has a ctx field. (doc->ctx) | 20:27.19 |
| mvrhel_laptop | ok this is where I think I still have an issue then | 20:27.31 |
| Robin_Watts | Right. | 20:27.37 |
| mvrhel_laptop | though I don't know why it is working | 20:27.38 |
| Robin_Watts | luck :) | 20:27.48 |
| | In your scheme, the thing to do would be to do something like: | 20:28.01 |
| mvrhel_laptop | all my operations use the document. usually to load the page | 20:28.58 |
| | for example GetHTML, loads the page, runs the page | 20:29.14 |
| Robin_Watts | TAKE LOCK, get the page (using the base context), run the page to get the display list (using the base context), DROP LOCK, clone the context, run the list to render it(using the clone context), drop the clone context. | 20:29.23 |
| mvrhel_laptop | so how do I do that while threads are off rendering thumbnails | 20:29.42 |
| Robin_Watts | Like I just said. | 20:30.12 |
| mvrhel_laptop | I see. So put a lock around all the doc access in my code | 20:31.13 |
| Robin_Watts | right. | 20:31.20 |
| mvrhel_laptop | and then let mupdf deal with the locks around the context | 20:31.30 |
| | I have that defined | 20:31.37 |
| Robin_Watts | eh, what? | 20:31.38 |
| mvrhel_laptop | but mupdf knows it already | 20:31.43 |
| Robin_Watts | "and then let mupdf deal with the locks around the context" ? | 20:32.05 |
| mvrhel_laptop | i.e. it was set up with the context this->mu_ctx = fz_new_context(NULL, &mu_locks, FZ_STORE_DEFAULT); | 20:32.08 |
| Robin_Watts | mvrhel_laptop: Right, mupdf uses that to ensure that things like the glyph cache, image cache etc are all threadsafe. | 20:32.47 |
| | It doesn't use that to protect file access. | 20:32.56 |
| mvrhel_laptop | right. | 20:33.02 |
| | and why do I need to do the cloning though? | 20:33.11 |
| | i.e. you have " run the list to render it(using the clone context), drop the clone context." | 20:33.24 |
| Robin_Watts | You need to use a context to run the display list. | 20:33.41 |
| mvrhel_laptop | yes | 20:33.45 |
| Robin_Watts | You can only use each context in 1 thread. | 20:33.51 |
| mvrhel_laptop | ok | 20:34.04 |
| Robin_Watts | hence you can't be rendering the display list with a context while another thread is using it too. | 20:34.20 |
| | So either you need to hold the lock for the WHOLE time (which would be simple, but would suck performance-wise), or you need to use a different context for the rendering. | 20:34.53 |
| | Does that make sense? | 20:35.27 |
| mvrhel_laptop | so do the clones share a glyph cache, image cache etc? | 20:36.52 |
| Robin_Watts | Yes. | 20:36.57 |
| mvrhel_laptop | ok so now it makes sense | 20:37.05 |
| Robin_Watts | The only thing that isn't shared by the clones is the error stack. | 20:37.12 |
| | Now, what I described above (TAKE, get display list, DROP, render display list) will work, and will probably be your shortest route out of this, but it's not ideal. | 20:38.07 |
| mvrhel_laptop | so I am still a little confused why my error stack is getting corrupted though when I have the clones | 20:38.33 |
| Robin_Watts | ok. | 20:38.45 |
| | So at the moment you're calling fz_run_page(cloned_context, page); right? | 20:39.14 |
| | so that does an fz_try(cloned_context) and that calls pdf_run_page(page->doc) or something, right? | 20:39.45 |
| | and pdf_run_page does an fz_try(page->doc->ctx) or something. | 20:40.09 |
| mvrhel_laptop | yes | 20:40.17 |
| Robin_Watts | and the problem is that cloned_context != page->doc->ctx | 20:40.25 |
| mvrhel_laptop | I see | 20:40.39 |
| Robin_Watts | so you have the outside fz_var set up on the cloned_context stack, and the inner one on a different stack, and it all goes tits up. | 20:40.50 |
| mvrhel_laptop | so a different ctx is getting incremented and decremented | 20:40.53 |
| Robin_Watts | yeah, we're using 2 different error stacks. | 20:41.12 |
| mvrhel_laptop | ok. so what is the ideal way that this would be done | 20:41.29 |
| Robin_Watts | ok, the problem with what I described above is what happens when you have several threads waiting for a page to be loaded. | 20:42.04 |
| | Do you have a thread per page or something? | 20:42.37 |
| mvrhel_laptop | currently at the top level, I only have an opaque object which is the document. it is not until the lower level (right before the calls in fz_*) that I do anything with various pages | 20:42.53 |
| | and that call from the opaque level is a new thread | 20:43.09 |
| | so render_page(x, document) | 20:43.20 |
| Robin_Watts | OK. I'm imagining a situation where you have several threads all waiting for the lock at once. | 20:43.22 |
| mvrhel_laptop | is done up high on a new thread | 20:43.26 |
| Robin_Watts | one thread might be a background one that's doing thumbnails. | 20:43.55 |
| | another might be for a page that's actually needed for rendering on the screen. | 20:44.06 |
| mvrhel_laptop | yes more or less, there may be multiple threads even doing thumbnails | 20:44.32 |
| | or doing up coming pages at full resolution even | 20:44.45 |
| Robin_Watts | Right. That was the case I was thinking of. | 20:44.58 |
| | So in that case, you'd like the 'active' page to be highest priority. | 20:45.13 |
| | and the upcoming ones next priority | 20:45.24 |
| | and the thumbnail the lowest priority. | 20:45.32 |
| mvrhel_laptop | I can assign priority at the high level, but I have not done that yet | 20:45.33 |
| | that is, I can do it at my opaque to mupdf level | 20:45.53 |
| Robin_Watts | With the simple 'TAKE lock, get list, DROP lock, render list' meachanism if you have multiple threads waiting on the lock you can't say which one will get the next lock. | 20:46.22 |
| | Wheras if you implement a server thread that takes requests, and returns lists, that can easily have priorities in it. | 20:46.50 |
| mvrhel_laptop | right | 20:47.12 |
| Robin_Watts | so each thread would do 'get list from server, render list' | 20:47.22 |
| | and the server thread would be the only thing that ever uses doc/base_context. | 20:47.53 |
| mvrhel_laptop | ok. this is going to take a little thought for me with the way that I have things designed now with the whole create_task and continuations going on currently in the code. | 20:49.28 |
| Robin_Watts | yeah. | 20:49.41 |
| mvrhel_laptop | but I think it is doable | 20:49.42 |
| Robin_Watts | cool. | 20:49.47 |
| | If you could spare the time to look over docs/overview.txt and tell me how to make it clearer, I'd appreciate it. | 20:50.09 |
| mvrhel_laptop | if they are always just calling the same "server object" and it is designed to be thread safe then I should be ok | 20:50.18 |
| Robin_Watts | mvrhel_laptop: yeah. by having a server object you serialise it all. | 20:50.40 |
| mvrhel_laptop | yes | 20:50.45 |
| | I like that approach. I think I will work in that direction. It is pretty clear to me. | 20:51.27 |
| | Thanks Robin_Watts | 20:51.29 |
| | I have to head out for bit right now | 20:51.35 |
| Robin_Watts | no worries. | 20:51.38 |
| | I'm done for the night now, I think. | 20:51.43 |
| mvrhel_laptop | have a good night | 20:52.00 |
| Robin_Watts | ta. you too. | 20:52.30 |
| | Forward 1 day (to 2013/10/11)>>> | |
IRC Logs