| | <<<Back 1 day (to 2016/08/04) | 20160805 |
| psmlbhor | hello, I want to use mudraw. I am using Ubuntu 16.04. If I build mupdf from source then to use mudraw I have to do "mutool draw" but if I install mupdf-tools from ubuntu repo, then I directly get to use "mudraw" on the command line. Why is the difference here? | 13:36.26 |
| kens | Check the versions of the two | 13:38.37 |
| | Robin_Watts : is thre any reason why I can't build MuPDF on WIndows ? I get 42 unresolved externals | 13:39.22 |
| | That's trying to build the whole solution... | 13:39.53 |
| psmlbhor | kens, from mupdf-tools I get version 1.7a and the latest source code is for version 1.9a | 13:41.11 |
| kens | Well there's your answer then. The different versions work differently | 13:41.56 |
| psmlbhor | kens, Oh! Thank you | 13:42.16 |
| kens | THe old code built mudraw separatley now its all bundled together | 13:42.21 |
| psmlbhor | ok. Thanks for that | 13:42.53 |
| kens | NP | 13:43.05 |
| Robin_Watts | kens: Urm... have you sit submodule update --init recently ? | 14:39.50 |
| | s/sit/git/ | 14:40.00 |
| | what externals are you missing? | 14:40.13 |
| kens | I did both and cleaned the solution and deleted generated | 14:40.16 |
| | But.... I just built the solution, instead of just MuPDF and it worked | 14:40.46 |
| | SO I guess I shold have built the solution instead of just MuPDF | 14:40.59 |
| sebras | Robin_Watts: do you mind taking a quick peek at 73e2c44 on sebras/wip and discuss it a bit? | 17:28.57 |
| | Robin_Watts: maybe it is too late for working in your timezone, I'm not sure. :) | 17:29.18 |
| Robin_Watts | sebras: Sure. | 17:29.20 |
| sebras | Robin_Watts: so I think there are document-level resource caches for fonts and images. | 17:29.39 |
| | Robin_Watts: AIUI they are meant to be used when _creating_ new documents | 17:30.12 |
| Robin_Watts | sebras: We have the store. | 17:30.20 |
| sebras | Robin_Watts: no, not that one. | 17:30.29 |
| | Robin_Watts: I'm talking aobut doc->resources->* that mvhrel introduced when adding document creating thingies a while back. | 17:30.52 |
| Robin_Watts | Oh, you mean the stuff that mvrhel added as part of his pdf create work... | 17:30.58 |
| sebras | Robin_Watts: yeah. | 17:31.04 |
| Robin_Watts | You type fast :) | 17:31.04 |
| sebras | Robin_Watts: I'm excited! because I think I understand something! :) | 17:31.24 |
| Robin_Watts | urgh, "yes" or "no", really? | 17:31.45 |
| | true or false if we have to. 0 or 1 by preference. | 17:32.05 |
| sebras | Robin_Watts: oh, the other patch? mmm, I just followed how writer-options work. | 17:33.27 |
| | Robin_Watts: but this is just an example so far. | 17:33.33 |
| Robin_Watts | and changing a given device hint resets all the hints :( | 17:33.42 |
| sebras | Robin_Watts: ok, so I believe that the resource hashtables have not been initialized so we crash. | 17:34.10 |
| | Robin_Watts: it is easy to make set_hints() incremental, but then I'd need another clear_hints() to remove the one's already set. | 17:34.55 |
| Robin_Watts | no, set_hint("blah=0") would remove blah, and leave the others unset. | 17:36.04 |
| sebras | Robin_Watts: doc->resources == NULL and when iterating over annotations on page 3 in the document one of the freetext annotaitons decides to update its appearance which means we add a (new?) CID font which the code then attempts to add to doc->resources->fonts, but doc->resources == NULL. | 17:36.12 |
| Robin_Watts | s/unset/unchanged/ | 17:36.14 |
| | IMAO, doc->resources should be initialised just in time. | 17:36.57 |
| sebras | Robin_Watts: no, unset, dev->hints = 0; at the top of the function. | 17:37.01 |
| Robin_Watts | sebras: Right. That's what I'm objecting to. | 17:37.14 |
| sebras | Robin_Watts: oh, you stated what you desire, not how it behaves. sorry. :) | 17:37.42 |
| Robin_Watts | I'm saying that if I create a device, I should be able to change a single hint bit by calling: set_hints("blah=1") and then reset just that bit by set_hints("blah=0") | 17:37.58 |
| | With yes/no etc, we need to be consistent, yes. I'll just complain to tor that yes/no are a stupid choice when he gets back. | 17:38.39 |
| | sebras: Did you follow what I meant by "just in time"? | 17:39.10 |
| | Rather than doing: | 17:39.16 |
| | if (doc->resources) add_something_to_resources(); | 17:39.28 |
| sebras | Robin_Watts: mm, so when we look up a font we initialize the hashtables. | 17:39.32 |
| | Robin_Watts: unless already done | 17:39.44 |
| Robin_Watts | it should be: if (!doc->resources) init_resources;\nadd_something_to_resources(); | 17:39.52 |
| sebras | aw crap... | 17:40.37 |
| Robin_Watts | Why? | 17:40.51 |
| sebras | I created my patch based on 1.9 (I was accidentally there when looking at another patch and forgot to reset to master) | 17:41.02 |
| Robin_Watts | git cherry-pick is your friend. | 17:41.15 |
| sebras | Robin_Watts: I know. | 17:41.21 |
| | Robin_Watts: but after that i did git revert HEAD just to verify that the issue is there without my patch. | 17:41.43 |
| | Robin_Watts: and now it works. :-/ | 17:41.49 |
| Robin_Watts | Sorry, I forgot you are one of the git cognoscenti. | 17:42.09 |
| sebras | Robin_Watts: I think the setup of the hashtables has been moved around recently. | 17:42.09 |
| Robin_Watts | Ah, quite possibly, yes. | 17:42.16 |
| sebras | Robin_Watts: np. :) | 17:42.20 |
| | Robin_Watts: anyway, the idea is that doc->resources should always be present as soon as we add a font/image for any reason. | 17:43.44 |
| | Robin_Watts: and these are document-level, not page-level. | 17:43.59 |
| Robin_Watts | Yes. | 17:44.58 |
| | I'm not sure how such things fit with tors desire that we should thin the structures down and work direct from pdf_obj's where possible. | 17:45.48 |
| | I suspect we'll need to have them for speed, and have some crufty code to invalidate/recreate them when document level stuff is changed. | 17:46.19 |
| sebras | it seems as if vhrel's idea was to have the hashtables to avoid adding the same resources repeatedly. | 17:47.24 |
| | for different pages I'm assuming. | 17:47.38 |
| Robin_Watts | sebras: Yes. | 17:48.06 |
| | It's vital, cos otherwise when you add an image, you have no easy way of knowing if it's there already. | 17:48.28 |
| | and you might bloat the file. | 17:48.33 |
| sebras | Robin_Watts: at first I thought doc->resources were the page-level resources and was quite confused. | 17:49.14 |
| Robin_Watts | sebras: Those were page->resources, though it's possible they have gone now. | 17:49.41 |
| sebras | Robin_Watts: on a related note: pdf_set_annot_appearance() seems to have a bug. | 17:50.06 |
| | Robin_Watts: it calls pdf_new_pdf_device() to create a new device for the annotation | 17:50.30 |
| | Robin_Watts: and sets the resource dict to /Resources from the appearance object. | 17:51.07 |
| | Robin_Watts: but as I read pdfref appearance objects never have /Resource | 17:51.28 |
| Robin_Watts | They have AR ? | 17:51.41 |
| sebras | Robin_Watts: /AcroForm in /Root has a /DR which appear to be the top-level resource dict for annotations. | 17:52.05 |
| Robin_Watts | Right. I thought I'd seen that in one of your bug reports. | 17:52.30 |
| | You may be right. | 17:52.48 |
| sebras | Robin_Watts: mmm, I thought that that was the reason for the problem, but it appears that it was the hashtables. | 17:53.19 |
| | Robin_Watts: I'm a bit uncertain about whether I've read the spec correctly though. these annotation dicts are _really_ hairy | 17:53.51 |
| | Robin_Watts: pdfref17.pdf page 678 I think | 17:54.01 |
| | there's a bullet list | 17:54.20 |
| Robin_Watts | It's listed as being optional, but then says "must contain Font entry" | 17:55.34 |
| sebras | Robin_Watts: optional if you don't have annotations? | 17:55.55 |
| Robin_Watts | I guess. | 17:56.04 |
| | So, yes, I'd try fixing that and then running a cluster test. | 17:56.27 |
| | We regenerate the annotations on loading files, I believe, so stuff might show up. | 17:56.50 |
| sebras | Robin_Watts: ok, then we read the spec the same way at least (that was the important part) | 17:57.05 |
| | Robin_Watts: btw, seems address sanitizer has been busy fuzzing our files. | 17:57.58 |
| | Robin_Watts: so we got ~10 bugs today. | 17:58.36 |
| | Robin_Watts: since they are likely all crashes I'm guessing I should take a look at those next. | 17:58.52 |
| | Robin_Watts: oh, and bisecting from 1.9 to origin/master revealed commit 21e1880 | 18:00.08 |
| Robin_Watts | sebras: Yeah, I saw. | 18:00.56 |
| sebras | Robin_Watts: oh..? | 18:01.05 |
| Robin_Watts | (about the 10 new bugs) | 18:01.14 |
| sebras | Robin_Watts: I didn't know until now. :) | 18:01.22 |
| Robin_Watts | I finished the SOT bug today, so thought I might look but.... just when I thought I was out, they pulled me back in</Godfather> | 18:02.16 |
| sebras | Robin_Watts: I'll close the bug (it was still open). | 18:02.18 |
| Robin_Watts | Ta. | 18:13.10 |
| sebras | Robin_Watts: at least I my debugging lead me to find the resource dict bug. | 18:16.30 |
| Robin_Watts | sebras: Yes. That could have been horrible. | 18:16.48 |
| | It's the kind of thing you hunt for for ages, not seeing the wood for the trees. | 18:17.23 |
| sebras | Robin_Watts: I'm surprised that this has ever worked. | 18:17.27 |
| Robin_Watts | Did you try a cluster test? | 18:18.15 |
| sebras | Robin_Watts: looking at the fallout. | 18:20.54 |
| | Robin_Watts: two regressions | 18:21.02 |
| | Robin_Watts: what happens in pdf_dev_alpha(), and why are we doing pdf_dict_putp(ctx, pdev->resource, ...)? | 18:28.40 |
| Robin_Watts | In order to set an alpha in PDF we need to create an ExtGState dictionary in the resources that sets the alpha value. | 18:29.53 |
| | so we look to see if we have such a dictionary already. If we do, we reuse it. If not, we create one. | 18:30.28 |
| sebras | Robin_Watts: why do we save it? | 18:30.36 |
| | Robin_Watts: is this also for document creation? | 18:30.44 |
| Robin_Watts | pdev->resource = the resource dictionary for this device. | 18:31.01 |
| | sebras: I don't quite get what you mean by save it. | 18:31.15 |
| sebras | Robin_Watts: why do we do pdf_dict_put() in the resource dictionary? | 18:31.34 |
| Robin_Watts | Suppose I want to make a 50% alpha for a fill. | 18:31.37 |
| sebras | Robin_Watts: there must be something fundamental I don't understand. why do we do this when reading a document? | 18:32.07 |
| Robin_Watts | All we can do in the operator stream is something like: /HalfAlpha gs | 18:32.10 |
| | Hence we need to create an entry in the resources /ExtGState dictionary to map from /HalfAlpha to << /ca 0.5 /Type/ExtGState >> | 18:33.36 |
| | (or something like that). | 18:33.40 |
| | Why do we need to do this when reading a document? That's cos PDF is mad. | 18:33.54 |
| | Annotations in PDF have an "appearance stream" which is "the list of pdf operations we perform to render the appearance of the annotation". | 18:34.35 |
| | They also tend to have the underlying information from which the annotation is constructed. | 18:34.56 |
| | PDF editors are allowed to set 1 and not the other. | 18:35.07 |
| | hence "dumb" renderers just load the PDF file and display the current appearance streams. | 18:35.32 |
| sebras | Robin_Watts: do changed states in annotations get saved when a document is closed? i.e. switching annotation states from normal to... whatever the other ones might be. | 18:35.33 |
| Robin_Watts | which means that they can either display out of date versions, or no versions at all. | 18:36.04 |
| | "smart" renderers synthesise the appearance stream from the underlying data every time the file is loaded. | 18:36.30 |
| | Hence when we load a file, we run through the annotations and synthesise the streams, thus refreshing the files appearance. | 18:37.02 |
| | sebras: Good question. Not sure. | 18:37.11 |
| | I can't remember exactly, but I think annotations with several states can have several appearance streams, and we use the appropriate one? I may be misremembering. | 18:37.57 |
| sebras | Robin_Watts: ok, so in this case the annotation for whatever reason is resynthesized and this is why the alpha needs to be updated and added to the resource dict. | 18:38.11 |
| | Robin_Watts: now the trailer has no such dict. | 18:38.20 |
| | Robin_Watts: well document catalog really. | 18:38.37 |
| Robin_Watts | what dict? | 18:39.03 |
| | annotations are attached to a page. | 18:39.11 |
| | and the alpha should be updated and added to the resources for the annotation. | 18:39.50 |
| sebras | Root/AcroForm/DR | 18:40.06 |
| | so the resource dict == NULL when we try to put the new alpha into it. | 18:40.25 |
| Robin_Watts | Right. So AcroForm is yet different. | 18:42.22 |
| sebras | Robin_Watts: how so? | 18:42.53 |
| Robin_Watts | DR is a document wide default resource dictionary for form fields. | 18:43.20 |
| | Do annotations not have their own resource dictionaries? | 18:44.00 |
| | yes, they do. | 18:45.14 |
| | See page 646. | 18:45.29 |
| | 8 0 obj is an appearance stream, and it has a /Resources | 18:45.59 |
| sebras | Robin_Watts: what about page 678? | 18:47.00 |
| Robin_Watts | I'm confused now. Didn't you say earlier that the spec said Annotations don't have one of those? | 18:47.07 |
| sebras | Robin_Watts: that's how I read it, but I might misunderstand (it _is_ 3am). | 18:47.29 |
| Robin_Watts | sebras: 678 is about text fields, I think? | 18:48.16 |
| | I think that's saying that the default for newly created ones comes from /DR | 18:48.38 |
| | So, the code is probably right as is? | 18:49.09 |
| sebras | And textfields are not annotations? | 18:49.18 |
| sebras | is utterly confused. | 18:49.39 |
| Robin_Watts | textfields are special cases of annotation I think. | 18:49.50 |
| | Where do you see the information that annotations don't have resources ? | 18:50.04 |
| sebras | Robin_Watts: that was the one at page 678 that I was reading, but you appear to be correct that it refers only to text fields. | 18:50.45 |
| Robin_Watts | sebras: Even for text fields, I don't read that as meaning "you can't have Resources". | 18:51.13 |
| sebras | Robin_Watts: I read it as you don't have Resources, go read in AcroForm/DR | 18:52.06 |
| Robin_Watts | sebras: I think the idea is you use Resources if it's there, and if not, fall back to AcroForm/DR | 18:52.45 |
| | And any new ones you create should start off with AcroForm/DR by default. | 18:53.07 |
| sebras | Robin_Watts: if that's the case then I agree with you there was no bug to being with. | 18:53.48 |
| | gorram it. :( | 18:54.35 |
| | so I basically just wasted mine and your time. | 18:54.43 |
| | Robin_Watts: sorry. ok. I'll go sleep now. | 18:56.26 |
| Robin_Watts | no worries, these things need to be thought about. | 19:02.15 |
| fredross-perry | sebras, robin - please take a look at e1cf593 on my mupdf master. This works, and is what I plan to use for hilighting. I still need to put in some error checking. But looking for feedback on the generalapproach. cheers. | 19:42.13 |
| | Forward 1 day (to 2016/08/06)>>> | |
IRC Logs