| | <<<Back 1 day (to 2016/11/06) | 20161107 |
| sebras | Robin_Watts: looks like tor is still trying to sort out iOS. | 10:02.14 |
| Robin_Watts | possibly. I think we should probably release and fix ios later. How late are we already? :( | 10:02.53 |
| | Raeds bug is a clipping screw up in MuPDF. | 10:03.56 |
| | I have a solution for it, but it's showing up some other diffs I need to understand. | 10:04.11 |
| | At least one of which is us getting something horribly wrong. | 10:04.23 |
| sebras | Robin_Watts: I think maybe we need to discuss how to do release a bit at the next meeting. ;) | 10:04.46 |
| | tor8: good morning. | 10:04.55 |
| Robin_Watts | (i.e. current code is horribly wrong, new code is merely wrong) | 10:05.06 |
| tor8 | sebras: morning. | 10:16.33 |
| | Robin_Watts: I think just triggering a whole page render whenever we delete an annotation is the easiest fix to ios | 10:17.16 |
| Robin_Watts | tor8: Seems a shame, but sure. | 11:05.12 |
| tor8 | Robin_Watts: the code always re-renders the page display list and recreates *all* annotation display lists and rerenders them if anything changes | 11:25.27 |
| | that seems suboptimal | 11:25.30 |
| | even if it only rerenders the bits of the page bitmap that are changed | 11:25.49 |
| | if annotations have overlapping bboxes, the same area of the page will be rendered multiple times as well | 11:26.00 |
| | a "better" solution would be to hold the page bitmap sans annotations, a transparent bitmap holding the rendered annotations, and then in the UI overlay the two | 11:26.48 |
| | and only re-render the annotation overlay bitmap when annotations change | 11:26.58 |
| | if we want faster updates | 11:27.06 |
| Robin_Watts | I dislike the idea of the app controlling overlays. | 11:33.00 |
| tor8 | why? | 11:33.44 |
| Robin_Watts | the smart way to do it would be to 'merge' annotation rectangles to avoid overpainting. | 11:34.00 |
| | but whatever works at this stage. | 11:34.07 |
| tor8 | re-rendering the page bitmap for each annotation change could be very slow, in case of images | 11:34.10 |
| | I'm just arguing that we have two bitmaps, and let the UI toolkit do the final compositing of page bitmap and semi-transparent annotation bitmap | 11:34.50 |
| | and we only need to re-render the annotation bitmap parts that change | 11:35.07 |
| Robin_Watts | yeah, and that sends shudders down my spine. | 11:35.16 |
| | You clearly have more trust than I do in the ability of UI layers to correctly overlay one bitmap on another. | 11:35.40 |
| tor8 | well... yeah. I guess :) | 11:36.11 |
| Robin_Watts | What if you have transparent annotations that use blend modes ? | 11:36.30 |
| tor8 | writing my own UI layer for the viewers I care about has probably blinded me to the insanities of iOS and android ;) | 11:36.32 |
| | then you're hosed. and doing something so stupid you deserve to be punished ;) | 11:37.00 |
| | but yes, blend mode annotations won't work and will need a full re-render with the whole transparency madness involved | 11:37.33 |
| | but the way the ios and android code creates and destroys a single display list for all annotations on a page strikes me as odd | 11:38.08 |
| | wouldn't it be more useful to keep a separate display list for each annotation, and only need to recreate the ones that change? | 11:38.25 |
| Robin_Watts | Yes. | 11:38.34 |
| tor8 | Robin_Watts: okay, looking at fred's ios patch for the deleted annotations | 11:46.53 |
| | I think he's halfway there | 11:46.56 |
| | he factors out the pixmap rendering code from loadPage into a renderPage method | 11:47.43 |
| | but then in updatePageAndTileWithTileFrame he calls that refactored method instead of updatePixmap | 11:48.57 |
| | Robin_Watts: I've got a commit on tor/master that should fix the iOS issues. But it's untested, and might not even compile... | 12:05.57 |
| sebras | tor8: if fz_bound_annot() throws, you leak rect_list. | 12:11.50 |
| | in deleteAnnotation(). | 12:12.07 |
| | not sure this is a case you care about, but still.. | 12:12.14 |
| | tor8: also I notice that updatePage() may return pointers to dirty_rect_list which may have changed betweeen calls. and updatePageAndTileWithTileFrame() may be called twice at different times by update() AIUI. | 12:22.03 |
| tor8 | sebras: yes. I considered not returning dirty_rect_list at all. | 12:22.45 |
| sebras | tor8: doesn't that mean that one dispatch_async:ed call to drop_list() may interferen with the other? | 12:22.49 |
| tor8 | this way had a smaller diff | 12:22.50 |
| sebras | tor8: or updatePage() should clear dirty_rect_list before returning it..? | 12:23.10 |
| tor8 | called twice, how? | 12:23.17 |
| sebras | cannot update() be called twice? | 12:23.24 |
| tor8 | that will just queue up calls to updatePageAndTileWithTileFrame | 12:24.01 |
| sebras | are they called sequentially? | 12:24.12 |
| | one waiting for the next to finish..? | 12:24.21 |
| tor8 | yes. they are queued up to the same queue, which is emptied sequentially. | 12:24.25 |
| sebras | previous. | 12:24.26 |
| tor8 | yes. each dispatch queue is a single worker thread, which does things in order. | 12:24.42 |
| sebras | ok. | 12:24.46 |
| | I was worried about the use of the dirty_rect_list global in updatePageAndTileWithTileFrameAndThisIdentifierIsTooLongAndTooCaMeLToType(). :) | 12:25.27 |
| tor8 | still shorter than CMMetadataFormatDescriptionCreateWithMetadataFormatDescriptionAndMetadataSpecifications :) | 12:26.27 |
| sebras | yes, I'm happy that this is the case. | 12:27.24 |
| tor8 | or the constant kCMSampleBufferConduitNotificationParameter_UpcomingOutputPTSRangeMayOverlapQueuedOutputPTSRange | 12:27.24 |
| sebras | I thought they were supposed to be function identifiers, not function essays. | 12:29.16 |
| | to the best of my abilities that ios patch seems reasonable. | 12:29.37 |
| tor8 | tell that to the NeXTStep developers ;) | 12:29.46 |
| | I think we'll need to wait for fred or jogux to be able to test build it | 12:30.06 |
| norbertj | Hello Robin_Watts: as of this morning I cannot build for windows anymore. I get a complaint about: Cannot open include file: 'sys/time.h' when compiling xmlparse.c Do you know if something changed? | 12:31.20 |
| Robin_Watts | norbertj: Ugh. I probably broke something. | 12:31.59 |
| sebras | tor8: deleteSelectedAnnotation() is enqueued on the same dispatch queue, right? otherwise deleteAnnotation() may meddle with dirty_rect_list after updatePage() has returned it. | 12:32.25 |
| Robin_Watts | norbertj: What version of MSVC? What configuration? | 12:32.27 |
| norbertj | It looks like the WIN32 define is not passed/set? | 12:33.00 |
| Robin_Watts | Ah, chrisl just updated expat. | 12:33.37 |
| norbertj | Both the ghostpdl.sln (converted to VS2008) and the nmake commandline: nmake -f psi/msvc.mak MSVC_VERSION=${ver} WIN32=1 XCFLAGS=-DPSI_INCLUDED USE_LARGE_COLOR_INDEX=1 UFST_BRIDGE=1 UFST_ROOT=../ufst63 COMPILE_INITS=1 PCLXL_ROMFS_ARGS= DEVSTUDIO= gpdl 2>&1 | tee gpdl32.log | 12:34.08 |
| chrisl | Erm, oops..... | 12:34.29 |
| Robin_Watts | Why are we using WIN32 rather than _WIN32, which is set automatically (I think). | 12:36.27 |
| chrisl | We?? | 12:36.42 |
| sebras | tor8: the ios issue is the last thing before 1.10? | 12:38.36 |
| chrisl | I guess changing "EXPAT_CFLAGS=/DHAVE_MEMMOVE" to "EXPAT_CFLAGS=/DHAVE_MEMMOVE /DWIN32" is what's needed | 12:38.45 |
| Robin_Watts | s/are we/is our thirdparty code/ | 12:38.49 |
| tor8 | sebras: yes! it better be... | 12:39.07 |
| | we're late enough as it is... | 12:39.14 |
| sebras | tor8: yeah, I was more thinking if I can help to make 1.10 happen in anyway. | 12:39.55 |
| chrisl | Robin_Watts: You'd have to ask the expat/cups/luratech devs | 12:39.58 |
| | Robin_Watts, norbertj: I'm just having some lunch - I can fix this after | 12:41.06 |
| norbertj | chrisl: no hurry... | 12:42.34 |
| tor8 | sebras: only if you can build and test iOS ;) | 12:42.45 |
| sebras | tor8: care to port to swift? | 12:45.23 |
| Robin_Watts | muswift | 12:46.22 |
| | Couple of no brainer fixes: http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=ac56cc7292b6f06f763db9cd50bf67684961239d | 12:52.35 |
| tor8 | Robin_Watts: LGTM | 12:52.58 |
| Robin_Watts | One slightly (but only slightly) more complex one: http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=7456ccc6c3ec48490dffebaf26f85f9f5529fbe0 | 12:53.08 |
| tor8 | Robin_Watts: if (fz_colorspace_n(ctx, colorspace) == 4) is what we usually do for that test | 12:54.46 |
| Robin_Watts | tor8: OK. | 12:54.58 |
| tor8 | I feel it would be better to replace all of those tests with a new fz_is_subtractive_colorspace() function | 12:55.05 |
| Robin_Watts | We probably ought to have a flag in the colorspace... yeah. | 12:55.15 |
| tor8 | Robin_Watts: anyway, that fix LGTM this way or with n==4. either way is fine. | 12:56.28 |
| | until we get color management and can render to spot color pixmaps | 12:56.49 |
| Robin_Watts | tor8: Will commit as is, and then do a followup. | 12:57.02 |
| tor8 | Separation and DeviceN ought to also be subtractive colorspaces; but we never have never tried using them as destinations | 12:57.21 |
| sebras | tor8: source/fitz/load-*.c ought to use fz_is_subtractive_colorspace() in that case. | 12:58.35 |
| tor8 | sebras: yes. I grepped for 'subtractive' comments and found all the ones that I think need changing | 12:59.17 |
| Robin_Watts | I can leave that followup to one of you if you want? | 13:03.30 |
| sebras | tor8: ok. | 13:03.31 |
| | Robin_Watts: I think tor8 already finished it. | 13:03.46 |
| Robin_Watts | ok, 2 regressions left in my clipping changes :( | 13:06.29 |
| sebras | in fz_copy_pixmap_rect() why are src->n == 2 && dst->n == 4 not the same as src->n == 1 && src->alpha && dest->n == 3 && dest->alpha? | 13:07.56 |
| Robin_Watts | You mean: Why are "if (src->n == 2 && dst->n == 4)" and "if (src->n == 1 + src->alpha && dest->n == 3 + dest->alpha)" not equivalent ? | 13:10.30 |
| | cos src->n == 1, src->alpha = 0, dest->n = 3, dest->n = 1 would trigger the latter, but not the former. | 13:11.06 |
| sebras | I do, and those are + not &&. | 13:12.18 |
| Robin_Watts | ok, 1 of the changes is a transfer function, I can ignore that. | 13:19.46 |
| | and the other is actually a progression. | 13:23.43 |
| sebras | Robin_Watts: since you have assert()s in template_*() I'm guessing you might as well have one for sa == da in fz_paint_pixmap_with_mask() too. | 13:32.55 |
| Robin_Watts | sebras: Yeah. | 13:33.07 |
| | Ok, 2 more commits on robin/master good to go. You can see the diffs (modulo a small threshold) in my bmpcmp area. | 13:39.49 |
| sebras | Robin_Watts: do we want to remove FZ_REVERSE_COMBINE()? it's no longer used. | 13:40.04 |
| Robin_Watts | sebras: ooh, yes. | 13:40.13 |
| | Thanks. | 13:40.36 |
| sebras | I'm struggling to understand the math for d0,d1, but then again I don't understand the original code. | 13:40.49 |
| | yet. | 13:40.52 |
| Robin_Watts | The SWAR code? | 13:41.10 |
| | General blending is new_destination = old_destination * (1-alpha) + source * alpha | 13:41.42 |
| | we work in 256ths, so that's: | 13:41.53 |
| | new_d = (d * (256 - a) + s * a) / 256 | 13:42.18 |
| | or... new_d = ((d<<8) - d*d + s*a)>>8 | 13:42.44 |
| | ahem. | 13:43.00 |
| | or... new_d = ((d<<8) - d*a + s*a)>>8 | 13:43.04 |
| | or... new_d = ((d<<8) + (s-d)*a)>>8 | 13:43.34 |
| | so 1 mul, 2 shifts and an add. | 13:44.15 |
| | and with the marvels of SWAR we can do 2 components at once. | 13:44.39 |
| Robin_Watts | lunches. | 13:44.59 |
| sebras | Robin_Watts: ok, that makes sense. | 13:49.57 |
| | with my limited knowledge about blending i cannot fault your patch. | 13:53.18 |
| | so LGTM, but you'd better wait for tor8 too. | 13:53.37 |
| tor8 | Robin_Watts: oh, yeah. that does simplify things a bit! | 14:18.15 |
| chrisl | norbertj: well, that was a bit more complex than I'd hoped, but I've pushed the expat on Windows fix | 14:30.02 |
| Robin_Watts | tor8: I'm going to have a look at better layer control for Raed. | 14:37.28 |
| tor8 | Robin_Watts: querying/toggling OCGs or exposing the "usage" parameter to fz_document fz_run_page? | 14:38.24 |
| Robin_Watts | tor8: Yes :) | 14:41.17 |
| | Something like that. Haven't entirely decided yet which. | 14:41.44 |
| | I'll run the proposed API past you. | 14:41.54 |
| | I'm going to assume that "that does simplify things a bit!" is "lgtm" :) | 14:42.52 |
| tor8 | Robin_Watts: yes. my eyes glaze over the SWAR stuff, but I'll take your word and the cluster push's word for it :) | 14:46.41 |
| Robin_Watts | tor8: ta. | 14:46.52 |
| | and another tiny one: http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=be56635e063e84848389c4dd99990f78c35b6519 | 14:47.47 |
| | (just moves 2 functions to the bottom of the file to leave all the #definery at the top) | 14:48.04 |
| tor8 | Robin_Watts: I think I can live with that. | 14:56.22 |
| Robin_Watts | Ta. | 14:56.32 |
| norbertj | chrisl: thanks. I already thought it to be a bit more complex as only replacing HAVE_MEMMOVE with WIN32 in msvc.mak. And it compliles ok now. | 15:02.32 |
| Robin_Watts | tor8: OK, so the exact API we offer is going to depend on how much functionality we want to bundle together. | 15:27.26 |
| | Every PDF file can have a set of optional content groups. Call them OCG1 to OCGN. | 15:28.08 |
| | It can also have Optional Content Membership Dictionaries (groups of OCGs, kinda) | 15:29.28 |
| | Then it can have different 'configurations' for viewing the document. | 15:29.51 |
| | each different configuration can present the OCGs in a different order, with different nesting, and can control them turning on/off differently. | 15:30.35 |
| | Some OCGs can be turned on individually. Others are mutually exclusive with other ones. | 15:31.04 |
| tor8 | each OCG is an XObject called by a 'Do' operator? | 15:34.10 |
| Robin_Watts | tor8: No. | 15:34.22 |
| | each OCG is a BDC EMC thing. | 15:34.29 |
| tor8 | there is code in pdf_process_Do that checks pdf_is_hidden_ocg | 15:34.46 |
| | as well as the BDC EMC thing | 15:34.55 |
| | so the event parameter to pdf_run_page_with_usage selects the different configurations? | 15:37.51 |
| | or is that something completely different? | 15:37.59 |
| | huh, that should be a const char *! | 15:38.10 |
| Robin_Watts | tor8: The event param allows us to use broad strokes to control it. | 15:38.53 |
| | For proper finer control, I suspect we need an 'OCGState' structure. | 15:39.38 |
| | An API like: "How many OCGConfigs are there?" | 15:39.56 |
| | "Get the name for OCGConfig i" | 15:40.17 |
| | "Get me an OCGState for OCGConfig i" | 15:40.33 |
| | That would return a pointer to a structure that has names, nestings, etc in. | 15:41.05 |
| | And then "Toggle entry j in OCGConfig i". | 15:41.27 |
| | to allow for clicking on things in the UI. | 15:41.39 |
| | Then 'Run this page with this OCGstate' | 15:41.50 |
| tor8 | would a simple list of configs and a parameter to select config N not be enough? | 15:41.59 |
| | so the UI can provide a drop down of possible configs that the user will select | 15:42.17 |
| Robin_Watts | tor8: Your config might be "view" or "design" | 15:42.46 |
| | but within "view" you might still want to turn layers on or off. | 15:43.00 |
| | Within view, you might want to restrict it so that layer 1 and layer2 can't be on at the same time. | 15:43.44 |
| tor8 | and with layer that's a group that can be toggled which then in some mad insanity toggles sub groups? | 15:43.50 |
| Robin_Watts | But within design, layer1 and layer2 can be on at the same time. | 15:44.07 |
| | tor8: yes (but there is more complexity even than that) | 15:44.22 |
| tor8 | the PDF 1.6 /VE entry.... | 15:44.38 |
| Robin_Watts | You could have an OCMG "layer 3" that appears whenever layer1 or layer2 are enabled, but not otherwise. | 15:44.48 |
| | yeah, and then there is the VE crap. | 15:44.57 |
| | I don't propose to implement all of this immediately. | 15:45.16 |
| | I'd like to put a framework in that's powerful enough to extend to it though. | 15:45.35 |
| tor8 | PDF 1.5 only has View and Design names | 15:46.06 |
| | so yeah, I guess you'd want access to the actual groups to toggle them | 15:46.23 |
| Robin_Watts | View and Design are "Intents". | 15:46.39 |
| tor8 | how about a bitmask (array of chars, one char per layer) to the run_page | 15:46.42 |
| | then some utility function to do the actual toggling and insane PDF logic | 15:46.53 |
| | and if no bitmask array is provided, make a default one from the intent | 15:47.13 |
| Robin_Watts | tor8: That doesn't address the need to do the selection with mutual exclusion etc in some cases. | 15:47.36 |
| | I figure that if we're going to offer that, then we might as well just pass the OCGState to run page (or something derived therefrom) | 15:48.13 |
| tor8 | it might be useful to just have a togglable flat list of layers to show at the fz_document level | 15:48.50 |
| | in case we add a viewer for photoshop or gimp images | 15:49.04 |
| Robin_Watts | tor8: Kind of - except that flat list should encapsulate nesting etc too. | 15:49.24 |
| tor8 | and then add the funky logic of how they depend on intents etc to pdf_specifics | 15:49.31 |
| Robin_Watts | That's what I have OCGState in mind. | 15:49.33 |
| | That's what I hope OCGState should be, effectively. | 15:49.52 |
| | Let me knock up a header file for you to comment on. | 15:50.02 |
| tor8 | PDF 2.0 removes the wording that there may be future intents. | 15:50.19 |
| | It just states there are two intents: Design and View. | 15:50.31 |
| | oh, and hidden in a table it also says that it may be a "second class name". and the appendix says second class names are developer specific names with namespace prefixes. | 15:51.38 |
| Robin_Watts | Yeah, I'll stick to const char *'s :) | 15:54.27 |
| tor8 | Robin_Watts: don't forget this should be easy to expose to JS and JNI, so don't make the OCGState too intricate :) | 15:55.45 |
| sheel | which one should be used for Linux based PDF flattening app - GhostPDF or muPDF? | 15:57.15 |
| tor8 | sheel: define PDF flattening | 15:58.23 |
| kens | What, precisely, do you mean by ... what tor8 said | 15:58.33 |
| sheel | The data for forms is stored in a separate area to the text on the page (which coincidentally makes it much easier to edit and extract). Forms flattening is the process of removing this separate form data and adding it as text, images and shapes into the actual PDF data stream. The PDF will still look the same but the user can no longer interact with the forms. | 16:00.28 |
| Robin_Watts | sheel: Right. Use gs with pdfwrite for that, I guess. MuPDF tries not to flatten. | 16:01.51 |
| tor8 | ghostscript with the pdfwrite device will create a new PDF which has the same visual appearance as the input file | 16:02.28 |
| | but it may also try to preserve form fields as form fields | 16:02.39 |
| kens | It will preserve forms as form XObjects | 16:02.52 |
| | Generally anyway | 16:03.15 |
| | I can't recall if that only works for PostScript input or not | 16:03.28 |
| tor8 | kens: it turns the form field annotation into a regular XObject called with a Do operator? | 16:03.32 |
| kens | tor8 are we referring to AcroForms, XFA Forms or Form XObjects ? | 16:03.55 |
| Robin_Watts | AcroForms. | 16:04.04 |
| tor8 | kens: From sheel's description, I was assuming AcroForms. | 16:04.18 |
| kens | I think it would be best if sheel was explicit about the forms he means | 16:04.34 |
| Robin_Watts | I don't see how you get get AcroForms in in a PS input. | 16:04.46 |
| tor8 | kens: only people like us who have actually read the PDF specification know about Form XObjects :) | 16:04.57 |
| kens | You don't, I@m referring there to PostScript Forms Robin_Watts | 16:05.06 |
| tor8 | I always assume AcroForms when someone mentions forms in a PDF context :) | 16:05.29 |
| kens | Well given that there are 3 kinds of 'form' n PDF, I think hsheel needs to say which kind he is talkign about | 16:05.42 |
| | sheel^^ | 16:06.04 |
| sheel | Basically, I would like to remove any editable objects but preserve the view. | 16:06.06 |
| kens | All klinds of objects are editable in a PDF file, nit just form elements | 16:06.37 |
| | However, I believe that if you take a 'filled in' saved AcroForm and run it through Ghostscript and pdfwite, then the result will look the same, but will no longer be an AcroForm | 16:07.45 |
| | If that's what you mean | 16:07.58 |
| sheel | Thanks! yes that is what I mean. I need to read more on PDF format specification to see whether any other elements i.e. XObjects needs to be removed. | 16:11.15 |
| Robin_Watts | sheel: You don't want to remove XObjects :) | 16:13.16 |
| | Cos Images etc are done as XObjects. | 16:13.30 |
| sheel | OK; thanks for that! However, I believe, from my limited knowledge on forms, XFA forms can not be removed directly without having an impact on the view. | 16:15.47 |
| | I am referring to this - http://blogs.adobe.com/an_tai/archives/172 | 16:16.08 |
| kens | XFA forms aren't really PDF, they are XML | 16:16.19 |
| | Ghostscript doresn't support them at all | 16:16.28 |
| tor8 | MuPDF doesn't support them either. | 16:16.58 |
| kens | And they are nuked in the PDF 2.0 draft standard (yay!) | 16:17.32 |
| sheel | Cool! Thanks a lot for your help! | 16:24.15 |
| Robin_Watts | tor8: Something like: http://pastebin.com/dST4rnsh | 16:31.47 |
| tor8 | depth is the indentation level in an otherwise flat list? | 16:37.11 |
| | and pdf_select_layer_config picks View or Design based on i? | 16:37.38 |
| Robin_Watts | depth is the indentation. | 16:40.31 |
| | We are not limited to just View or Design. | 16:40.45 |
| | Those are the "Intents". | 16:42.03 |
| | But we can have more configurations than just those 2 Intents. | 16:42.19 |
| tor8 | what does pdf_select_layer_config do? is this related or unrelated to the Intent? | 16:43.19 |
| Robin_Watts | tor8: That's "a user clicked on entry i" | 16:47.34 |
| | so that will update 'selected', while taking care of radio groups etc. | 16:47.51 |
| | (some entries are mutually exclusive to others) | 16:48.15 |
| | rather than trying to tell the caller which ones are mutually exclusive, we'll handle it for them. | 16:48.35 |
| | So pdf_layer_config is read only effectively. | 16:49.18 |
| tor8 | right, so that's the layer not the intent | 16:50.47 |
| Robin_Watts | yes, nothing in there handles intents. | 16:50.58 |
| | we already have 'pdf_run_page_with_usage' that takes an intent. | 16:51.16 |
| | I forsee us having 'pdf_run_page_with_config' that will take one of these. | 16:51.31 |
| tor8 | Robin_Watts: http://pastebin.com/p3pLU5VH something like that could work, and not require malloc/drop if we're just querying read-only data | 16:55.33 |
| | maybe just call them pdf_layer and pdf_count_layers, etc | 16:56.11 |
| | layer_configs sounds like more than just layers, which is what confused me about the name just above when I thought it referred to the intent | 16:56.32 |
| Robin_Watts | There are several possible configs in a file. | 16:56.42 |
| | Each one is made up of a list (or array) of entries? | 16:57.21 |
| tor8 | is there one per intent? | 16:57.32 |
| Robin_Watts | tor8: I'm not convinced Intents and configs map together. | 16:57.53 |
| | For instance in Raeds example file, we have just the default config 'D'. | 16:58.47 |
| tor8 | okay, trying to explain to myself how this works... | 17:01.41 |
| | there are three variables involved: | 17:01.46 |
| | the intent, the configuration, and the individual layers | 17:02.18 |
| Robin_Watts | AIUI, you can label each OCG with an intent (view or design), and when we render a page, in the absence of any layer controls having been set, we will trigger stuff on or off according to whether it matches the intent. | 17:02.30 |
| tor8 | the intent + configuration will define the initial state of the layer visibility | 17:02.35 |
| | and toggling a layer may ripple and toggle other layers due to the membership dictionaries | 17:02.53 |
| Robin_Watts | Yes. | 17:03.15 |
| | I think possibly we should think of there being 2 layers of filtering here. | 17:04.08 |
| | 'configuration' determines the first layer of filtering as to what gets plotted or not. | 17:04.23 |
| | 'intent' is then the second layer. | 17:04.33 |
| tor8 | http://pastebin.com/4JFq8Q2W perhaps? | 17:04.44 |
| Robin_Watts | so maybe we only *actually* plot stuff if it's enabled in both? | 17:04.45 |
| tor8 | and then remove pdf_run_page_with_usage; if you want a different view you select a different config before running. | 17:05.15 |
| Robin_Watts | I don't believe intent has any part to play in choosing the config. | 17:05.16 |
| tor8 | I suspect Intents came first, and then the whole configuration dictionaries came later | 17:05.51 |
| Robin_Watts | Yes. | 17:05.54 |
| | But nothing in the configuration dictionaries mentions intents. | 17:06.19 |
| | Also, I dislike the idea of having a 'current selected layer config' in the doc. | 17:06.47 |
| tor8 | nah. OCG intents and configuration dictionaries were both introduced with PDF 1.5 | 17:06.55 |
| | so that's just some photoshop insanity that wormed its way into the spec, just like transparency... | 17:07.14 |
| Robin_Watts | That's state in the pdf_document which isn't in the actual pdf file, and we're trying to remove that, right? | 17:07.15 |
| tor8 | Robin_Watts: yeah. it might make sense to have a pdf_layer_config struct which we mutate and pass to the pdf_run_page function | 17:07.57 |
| Robin_Watts | tor8: That was my intent (ha!) with my version. | 17:08.15 |
| tor8 | right. the prototypes to your functions use the doc not the layer config as arguments though, so that was not clear to me | 17:08.59 |
| | my initial thought was to add a fz_layers struct (an array of bits, for toggling each layer) and pass that to fz_run_page | 17:09.04 |
| Robin_Watts | D'OH. Just a mo. | 17:09.41 |
| tor8 | and have the fancy logic for selecting configs in pdf_document, and that would get passed the fz_layer array and manipulate that | 17:09.42 |
| | yeah, I'm not clear just how Intents and the configuration dictionaries intersect. | 17:10.05 |
| Robin_Watts | http://pastebin.com/W7esEWBR | 17:11.19 |
| tor8 | there is an /Intent in the configuration dictionary alongside the /ON and /OFF arrays | 17:11.31 |
| | pdf_select_layer_config will take config i and apply it to the pdf_layer_config 'selected' fields? | 17:12.05 |
| Robin_Watts | pdf_layer_configs are "an entire configuration". | 17:12.43 |
| | an array of 'n' entries. | 17:12.58 |
| | pdf_select_layer_config says "select the ith element of that config" | 17:13.12 |
| | (select may be a bad name. toggle might be better?) | 17:13.34 |
| tor8 | I'm not sure I follow. | 17:13.45 |
| Robin_Watts | OK. so suppose we have a file that contains OCGs 1 to 10. | 17:14.08 |
| tor8 | pdf_layer_config, is that an array of the layers and their state; or an array of the possible "OC Configuration Dictionaries" | 17:14.33 |
| | i.e. does entry 'n' correspond to an OCG or an OCGD | 17:14.47 |
| Robin_Watts | n corresponds to an OCG. | 17:14.59 |
| | The same PDF file might have 4 different configurations in it. | 17:15.32 |
| tor8 | right. so pdf_select_layer_config(4) would try to turn OCG #4 in our list of layers on | 17:15.47 |
| Robin_Watts | Yes. | 17:15.53 |
| tor8 | but how do you select OCGD #2 in your API? | 17:15.54 |
| Robin_Watts | oh, ffs. I'm a moron. hold on. | 17:16.11 |
| | http://pastebin.com/QZsTuEKi | 17:18.21 |
| | OK, so we can say "Which config should I use for intent "View" ?" | 17:18.42 |
| | And we can say "How many configs are there?" | 17:19.06 |
| | and "Get me config number i" | 17:19.16 |
| | and "turn layer j on in this config" etc. | 17:19.36 |
| | and "process the page with this config" | 17:19.44 |
| tor8 | Robin_Watts: reading page 376 (table 4.51) | 17:20.36 |
| | it looks like selecting a configuration can toggle some layers on and some off and leave others untouched | 17:20.59 |
| | BaseState can be Unchanged | 17:21.22 |
| Robin_Watts | untouched from their defaults in the file, AIUI. | 17:21.23 |
| | So when I first get a config, I populate selected according the basestate and the defaults given in the OCGs for each one. | 17:22.02 |
| tor8 | the description of the Intent entry in table 4.51 seems to contradict the idea of pdf_get_layer_config_for_intent | 17:22.20 |
| Robin_Watts | hmm. | 17:23.05 |
| | So, are we back to thinking that that is a second layer of filtering then? | 17:23.17 |
| tor8 | not sure what the hell to do, but I *believe* that a second layer of intent filter is probably what's required | 17:25.26 |
| | I'm not making heads or tails of this... | 17:25.34 |
| Robin_Watts | I'm going to need to construct an example. | 17:26.23 |
| | But if we ignore intents, I think that what I have is about right. | 17:26.31 |
| | Just have to figure out how intents fit into it. | 17:26.49 |
| tor8 | hm. "If one or more of a groupâs intents is | 17:26.55 |
| | contained in the current configurationâs set of intents, the group is used in deter- | 17:26.55 |
| | mining visibility. If there is no match, the group has no effect on visibility." | 17:26.55 |
| | so the Intents in the OCGD would be a second way (apart from ON, OFF, and BaseState) to toggle the individual OCGs | 17:27.22 |
| Robin_Watts | So if a group has no intents, it can never contribute to visibility? | 17:27.36 |
| tor8 | Robin_Watts: what do you think about having a pdf_layer_list that we load once, and then we can pick OCGDs to load into it? | 17:28.23 |
| Robin_Watts | Ah. default is View. So you'd have to have Intent = [] to make something that never contributed. | 17:28.30 |
| | That would be wrong, I believe. | 17:28.38 |
| tor8 | Robin_Watts: something like http://pastebin.com/1nbY4a9B | 17:29.10 |
| Robin_Watts | because 1) not all layers get used in all configs, so the number of entries would change. | 17:29.20 |
| | 2) That would suggest that I might get different results by loading the layer list, then loading OCGD A then OCGD B than I would get by just loading OCGD B in the first place. | 17:30.07 |
| | which I believe is absolutely not the case. | 17:30.16 |
| tor8 | Robin_Watts: No? I think each OCGD refers to ALL OCGs (via BaseState if unlisted in ON and OFF) | 17:30.22 |
| | then select rather than apply. I wrote that example thinking "Unchanged" would be unchanged from current state, not from default | 17:30.52 |
| | the OC Properties dictionary has an OCGs array, and a Configs array | 17:31.47 |
| | the OCG array would map to the pdf_layer_list | 17:32.05 |
| | and then you can select each Config to load a visibility state into the layer list | 17:32.31 |
| Robin_Watts | The Optional Content Properties Dictionary has an OCGs array that lists all the entries in the file. | 17:32.51 |
| | The Configs array then chooses how to present those to the user. | 17:33.05 |
| tor8 | the text in table 4.51 has the wording "when this configuration is applied" | 17:33.06 |
| Robin_Watts | The order can be different, with nesting etc. | 17:33.13 |
| tor8 | oh, right. page 2... | 17:33.26 |
| | I should've known there was something I missed... | 17:33.41 |
| | okay, then we'll need a different array for each config | 17:34.04 |
| Robin_Watts | Each config can, I think, only provide a subset of the possible OCGs. | 17:34.33 |
| | But I think ALL the OCGs are rendered (according to their default states). | 17:34.50 |
| tor8 | the default config is an empty array, so no layers at all | 17:34.52 |
| | if I read the Order entry correctly | 17:34.59 |
| | Any groups not listed in this array should not be presented in any UI that uses the configuration. | 17:35.20 |
| | Dosen't say whether they render or not... | 17:35.25 |
| Robin_Watts | so the pdf_layer_config is really just what we are offering in the UI. | 17:35.30 |
| | I believe they do render. | 17:35.36 |
| | according to their base state, and the intent. | 17:35.49 |
| tor8 | gah. this part of the spec is such a steaming pile.... | 17:35.56 |
| Robin_Watts | http://pastebin.com/mjPKP6YA | 17:37.10 |
| tor8 | personally, I think it'd be a better (less confusing) UI to just display all the OCGs than all this magic "click here and the list changes, but the layers in the previous list are magically invisibly turned on or off" | 17:37.16 |
| | pdf_select_layer_config and pdf_toggle_layer_config, duplicates? | 17:38.34 |
| | and they take an int layer argument, but also a linked list node | 17:38.52 |
| Robin_Watts | They always take the head of the linked list. | 17:39.05 |
| | I think I maybe prefer the idea of having it as an array rather than a linked list. | 17:39.37 |
| tor8 | yeah, I think an array would be clearer | 17:39.44 |
| | or an opaque struct with a 'read entry N' function | 17:40.00 |
| Robin_Watts | http://pastebin.com/P9bc6tiT | 17:41.49 |
| | balls, lose the 'next' entry. | 17:42.29 |
| tor8 | pdf_layer_config can have the config name in the struct now too | 17:42.48 |
| Robin_Watts | Yes.... but I can imagine people wanting to get names for *all* the configs without wanting to have to load each config in turn. | 17:43.50 |
| tor8 | that shape looks reasonable. question is how Intent is to be handled. I guess some experimentation is in order... | 17:43.54 |
| Robin_Watts | tor8: I think that intent is a second layer of filtering. | 17:44.10 |
| | hence an intent can be passed in to pdf_run_page_with_config_or_intent. | 17:44.36 |
| tor8 | yeah, so that can be passed to pdf_run_page_with_intent and pdf_run_page_with_intent_and_config as we do now | 17:44.52 |
| Robin_Watts | It doesn't affect the actual configuration stuff at all (I hope). | 17:45.04 |
| tor8 | and maybe add a Intent drop-down in the UI | 17:45.05 |
| Robin_Watts | tor8: yeah. | 17:45.11 |
| tor8 | the Intents array in table 4.51 bothers me | 17:45.25 |
| Robin_Watts | Why? | 17:46.08 |
| | We already have code in MuPDF that looks at each OCG when we meet it, and if it's enabled, and the intent matches, we use it, otherwise we don't. | 17:46.37 |
| | that should be unchanged (other than the 'if it's enabled' test being more complex now). | 17:47.03 |
| | The intent part doesn't change. | 17:47.09 |
| | So rendering a given config under intent "View" will potentially look different to rendering the same config under intent "Design". | 17:48.27 |
| tor8 | yes, that makes the most sense. | 17:48.38 |
| | but this is PDF we're talking about... | 17:48.42 |
| | Robin_Watts: page 369 (pdfref17.pdf) | 17:55.35 |
| | if the Intent of an OCG matches the current OCGDs Intent, the OCGD is used to determine visibility | 17:56.12 |
| Robin_Watts | Making Graphical Content Optional ? | 17:56.13 |
| tor8 | and, the rest of the sentence makes no sense to me... | 17:56.42 |
| Robin_Watts | Are you talking about this: Configuration dictionaries (see âOptional Content Configuration Dictionariesâ on page 375) also contain an Intent entry. If one or more of a groupâs intents is contained in the current configurationâs set of intents, the group is used in determining visibility. If there is no match, the group has no effect on visibility. | 17:57.42 |
| | ? | 17:57.45 |
| tor8 | | 17:58.08 |
| | | 17:58.09 |
| | | 17:58.09 |
| | | 17:58.09 |
| | | 17:58.09 |
| | | 17:58.12 |
| | | 17:58.14 |
| | for optional content groups within a single document. An interactive PDF processor can decide to use only | 17:58.16 |
| | groups that are of a specific intent. | 17:58.19 |
| | Configuration dictionaries (see 8.11.4.3, "Optional content configuration dictionaries") may also contain | 17:58.21 |
| | an Intent entry. If one or more of a groupâs intents is contained in the current configurationâs set of | 17:58.24 |
| | intents, the group shall be used in determining visibility. If there is no match, the group shall have no | 17:58.26 |
| | effect on visibility. | 17:58.29 |
| | If the configurationâs Intent is an empty array, no groups shall be used in determining visibility; | 17:58.32 |
| | therefore, all content shall be considered visible. | 17:58.34 |
| | visibility of what?! | 17:58.35 |
| | the OCG has an effect on the visibility of the OCG? | 17:58.40 |
| | stupid spec language... | 17:58.41 |
| | okay, enough griping. time for dinner. | 17:58.46 |
| Robin_Watts | Yeah, same text, different version. I'm looking at pdf_reference17.pdf, the Adobe published spec. | 17:59.37 |
| tor8 | if it had said "configuration shall be used in determining visibility" I would have understood it better. but it doesn't. | 18:00.39 |
| eyolf | any mupdf devs here? | 18:11.22 |
| | "The MuPDF developers hang out on IRC in the #ghostscript channel" :) | 18:12.06 |
| Robin_Watts | we are here. | 18:13.50 |
| eyolf | OK! | 18:14.00 |
| | Just wanted to ask: is there a way to enable continuous scrolling? | 18:14.22 |
| Robin_Watts | generally on irc, you should ask questions, rather than asking to ask. | 18:14.30 |
| | In what viewer? | 18:14.40 |
| eyolf | I would have if this was the #mupdf channel and not the #ghostscript | 18:15.14 |
| | in mupdf... or what do you mean? | 18:15.32 |
| | Linux | 18:15.34 |
| Robin_Watts | Linux, right. | 18:15.40 |
| | MUPDF is a C library. It ports to many platforms. | 18:16.11 |
| | We have many different example viewers. | 18:16.20 |
| | These are normally dead simple and just wrap the C libs functionality. | 18:16.44 |
| | The C lib is perfectly capable of doing continuous scrolling. | 18:16.55 |
| | The mupdf linux viewer is designed to be page based though. | 18:17.06 |
| eyolf | ok | 18:17.10 |
| | that's not good... | 18:17.19 |
| Robin_Watts | If you want a more fully featured thing, look at gsview 6. | 18:17.22 |
| eyolf | I hate that jumping behaviour | 18:17.28 |
| Robin_Watts | That uses mupdf to do the display. | 18:17.33 |
| eyolf | ok | 18:17.37 |
| | thx | 18:17.42 |
| Robin_Watts | but has the power of ghostscript behind the scenes to handle PS etc too. | 18:17.49 |
| sheel | I get this error when I use -sDevice=pdfwrite for a file - ./gs gs -sDEVICE=pdfwrite -o Rabate_Flattened.ptf /vagrant/Rebate.pdf GPL Ghostscript 9.20 (2016-09-26) Copyright (C) 2016 Artifex Software, Inc. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. Error: /undefined in ELF Operand stack: Execution stack: %interp_exit .runexec2 --nostringval-- --nostringval-- --nostring | 20:29.12 |
| | NVM; Please ignore the last message. I types gs 2 times | 20:30.12 |
| ray_laptop | sheel: yeah, ./gs tried to read the the "gs" executable and it stumbled across the "magic" header for an ELF format executable. At least that makes a bit of sense | 21:17.58 |
| | sheel: BTW, in your command line you have "-o Rabate_Flattened.ptf " -- did you mean "-o Rabate_Flattened.pdf" (pdf, not ptf) ? gs will write whatever filename you tell it to, but some readers use the "extension" to determine how to process the file (rather than examining the content) | 21:19.55 |
| | sheel: and also, I am not sure what you are trying to accomplish. Your output file name _Flattened is a hint, but gs will *not* flatten transparency to an image unless you tell it that you want something less than PDF 1.4 out. | 21:22.39 |
| sheel | ray_laptop, thanks for asking the question - Here is what I am trying to accomplish - PDFFlatening... we are looking at it from a security perspective. We want to preserve the "view" of the document, but essentially want to remove layers and anything as individual embedded objects. Essentially, "Rasterize" the document. | 21:25.28 |
| | can ghostscript do that? Can we tell ghostscript to output pdf 1.4? | 21:26.02 |
| | or less? | 21:26.08 |
| ray_laptop | oops. sheel left. | 21:48.50 |
| | sheel: if you come back, the answer is "that depends". gs can "flatten" a PDF 1.4 document to an image if it has transparency. In general, gs tries to keep the output PDF as close as possible to the original, so if a non-transparency PDF input has annotations or OCG (Optional Content Group) layers, they will, if possible, be reflected in the PDF that gs creates. | 21:52.10 |
| | sheel: there are LOTS of details that affect this, obviously. If what you are looking to do is to totally flatten the input PDF to an image in all cases, with *NO* information retained, then I recommend going to an image format that gs knows how to read, then writing a PDF from that image. BRUTE FORCE, but *very* secure in that only the image will make it through | 21:54.32 |
| | sheel: but note that this will *not* make a PDF with searchable text. | 21:54.58 |
| | sheel: for example: gs -q -sstdout=/dev/null -sDEVICE=ppmraw -o /tmp/xxx input.pdf ; gs -q -r300 -sDEVICE=pdfwrite -o output.pdf -r300 -dSCALE=1 -- lib/viewpbm.ps /tmp/xxx | 22:02.45 |
| | sheel: the above will render (rasterize) the page to a 300 dpi RGB (ppm) image with NO other information, and then the second gs will read the PPM with a gs utility "lib/viewpbm.pds" and will write the PDF that just contains the image | 22:04.46 |
| | we have discussed adding a 'pdfimage' (similar to now deprecated "psimage" formats) that would do this in one step, but haven't decided whether or not to do it (yet). It is pretty simple | 22:06.56 |
| | sheel: BTW, the above method will work with multipage inputs all concatenated into a single ppmraw file (if there is enough space on /tmp) | 22:09.08 |
| | Forward 1 day (to 2016/11/08)>>> | |
Log of #ghostscript at irc.freenode.net.