Log of #mupdf at irc.freenode.net.

 <<<Back 1 day (to 2020/03/24)Fwd 1 day (to 2020/03/26)>>>20200325 
Robin_Watts_ Hey scottschafer16:03.05 
scottschafer Hi Robin - I was replying to your email but we can try chatting here16:25.57 
  so I tried opening a PDF with an annotation using the pdf.js viewer app here: http://mozilla.github.io/pdf.js/web/viewer.html16:27.01 
  The pdf has a text annotation. I used https://tcpdf.org/examples/example_036/16:27.21 
Robin_Watts_ Hi scottschafer, go for it.16:27.43 
scottschafer As I suspected, the pdf.js getAnnotations() API returns an array of annotations, and the viewer application renders it16:27.48 
Robin_Watts_ ok, where an annotation is what?16:28.29 
scottschafer So, if I fully implement getAnnotations() in mupdf.js (again, it returns an empty array now), then the viewer application will render the annotations in the "annotation layer" in the DOM16:28.33 
  > where an annotation is what?16:28.41 
Robin_Watts_ It's a javascript object, presumably?16:29.13 
  with what methods/fields?16:29.24 
scottschafer An annotation is some pdf.js object with fields….the documentation doesn't help much, but I can reverse engineer it and give it a typescript definition16:29.29 
  anyway, if I implement getAnnotations(), then the viewer app will render the annotations. But also, mupdf will render the annotations through drawPageAsPNG()16:30.09 
Robin_Watts_ OK, so change drawPageAsPNG, so that instead of calling fz_run_page(), it calls fz_run_page_contents()16:30.51 
  How can the caller edit annotations? By tweaking the objects found in the getAnnotations array?16:31.54 
scottschafer OK. I guess that's one of my questions. IMO, things will go more easily for us if we have the viewer application render the annotations rather than mupdf16:32.08 
Robin_Watts_ scottschafer: Not sure I agree.16:32.22 
scottschafer I'm sure there are a lot of considerations here16:32.41 
Robin_Watts_ I think having the viewer responsible for calling the render function is fine.16:33.01 
scottschafer ok, at least as a first stab at this16:33.17 
Robin_Watts_ but the actual plotting of the pixels that make up the annotations can be done by mupdf, right?16:33.30 
ator scottschafer: some annotations need to render with a specific blend mode on top of the page (like highlights using 'multiply' blend)16:33.37 
scottschafer ok, this is really the question16:33.47 
Robin_Watts_ ator: Yeah, but I think we're happy to drop that, aren't we?16:33.55 
  If we're not happy to drop that, then we can't work to the pdf.js interface.16:34.17 
ator Robin_Watts_: we really need the blend mode for highlights. everything else can just use plain alpha compositing.16:34.28 
  the client can do the blending though, at the web canvas (or however you display them)16:34.47 
scottschafer so…if we were to render the annotations in mupdf, then we'd have to: 1) provide a modified version of the viewer application that did not use the annotation layer and 2) modify the page in mupdf and re-render it whenever annotations change16:35.07 
ator we can render annotations to separate images, but compositing them with the page needs care for the blend modes16:35.13 
Robin_Watts_ ator: Right. The way this stuff works, AIUI is that we have a <div> in the page for the contents and another for the annotations.16:36.18 
  We render the page to a bitmap, and pass that bitmap back as a data URL.16:36.38 
scottschafer one of my challenges is to make a library that uses mupdf but is compatible with the pdf.js API. But if I implement getAnnotations(), then the pdf.js viewer application will try to render them in a div16:36.46 
Robin_Watts_ That is then inserted into the div.16:36.50 
  scottschafer: Is that a fair description?16:36.58 
scottschafer yes robin16:37.03 
ator scottschafer: right, so the pdf.js viewer app draws the annotations using Javascript and HTML?16:37.13 
scottschafer yes16:37.18 
Robin_Watts_ I was thinking that we'd do annotations the same way, and get another bitmap (this time a transparent one) and put that in the annotation div.16:37.39 
ator change (or add a variant) drawPageAsPNG to call fz_new_pixmap_from_page_contents instead then, I think.16:37.44 
Robin_Watts_ but ator seems to be saying that's not possible.16:37.49 
  or not good enough.16:37.54 
ator Robin_Watts_: if the rendered annotations are PNG files with transparent areas, that might work to overlay them16:38.19 
  but we might have some difficulty telling it to do the multiply blend mode for highlights16:38.34 
Robin_Watts_ ator: Right, but we can't get funky blend modes that way.16:38.38 
ator depends, might be possible with CSS16:38.47 
Robin_Watts_ ator: We'd have a single annotation bitmap, AIUI.16:39.07 
  so even if we could apply blendmodes to the whole bitmap, that'd probably be wrong.16:39.25 
ator ah, then that would not work. we'd need one bitmap per annotation (or two, one for normal and one for multiply)16:39.28 
Robin_Watts_ ator: And even then we could have parts of an annotation using a blend mode, and parts not.16:39.49 
ator Robin_Watts_: not worth worrying about, IMO. the only ones that use blend modes in practice are highlight annots16:40.13 
  and they always use multiply16:40.16 
Robin_Watts_ (or maybe we couldn't, depends how complex they are).16:40.20 
  rendering highlights without multiply looks crap? or doesn't show up at all?16:40.46 
ator any other types would be non-standard annotation types with funky appearance streams, not sure we'd care enough about those cases16:40.56 
scottschafer if we went the route of mupdf rendering annotations, why not just have it render the page with annotations?16:40.59 
  and not bother with the layer?16:41.09 
ator Robin_Watts_: they show up as opaque rectangles blocking out the text they're supposed to highlight16:41.14 
  like a redaction, basically16:41.22 
  scottschafer: that's what mupdf does now isn't it? the drawPageAsPNG certainly renders annotations.16:42.03 
Robin_Watts_ scottschafer: That depends if callers will get confused expecting the annotations to not be in the page bitmap.16:42.03 
ator "background-blend-mode: multiply" is a thing in CSS16:42.31 
scottschafer it is, it doesn't work in IE though16:42.41 
  I had to fake it in another project with pixel manipulation in canvas16:42.56 
ator scottschafer: ah, not even in edge?16:42.59 
scottschafer nope16:43.04 
Robin_Watts_ scottschafer: For instance, I could imagine that callers might not expect to have to redraw the page bitmap once they edit an annotation.16:43.11 
ator then scratch that for now, or use canvas for everything?16:43.18 
  or WebGL (shudder)16:43.25 
Robin_Watts_ ator: That won't fit under the pdf.js interface that we're trying to work to.16:43.37 
scottschafer yeah16:43.55 
  so again….if I implement getAnnotations(), then the pdf.js viewer will attempt to render them in the annotations layer16:44.30 
Robin_Watts_ scottschafer: HOW will attempt to render them?16:44.44 
  It will call the 'render' method for each annotation?16:44.54 
ator is there a primer on pdf.js somewhere?16:45.13 
scottschafer it's just built into the viewer application http://localhost:8888/web/viewer.html. There's a file called "annotation_layer.js"16:45.33 
Robin_Watts_ So, the annotation rendering code is not actually under the pdf.js interface?16:46.03 
  i.e. every caller of pdf.js is expected to implement its own annotation rendering? (which presumably they all do by using the same annotation_layer.js)16:46.35 
scottschafer pdf.js is the low-level library providing APIs, but the viewer application provides the UI and builds a lot of functionality on top of pdf.js. It's what polar-bookshelf uses and I think many other pdf.js users16:46.38 
  > every caller of pdf.js is expected to implement its own annotation rendering16:46.56 
Robin_Watts_ OK. So there are a couple of ways we could go here that I can see.16:47.11 
scottschafer unless they use the viewer application directly in their code16:47.15 
Robin_Watts_ 1) We can return a list of annotations with duff types, so that the annotation_layer doesn't render them.16:47.50 
  2) We can imagine that annotation_layer.js is part of a combined "pdf.js + annotations" API and work to make mupdf.js implement all of that.16:48.30 
  How are annotations created?16:48.56 
  Is there an annotation_layer.js method like "createAnnotation"?16:49.12 
scottschafer currently pdf.js doesn't support modifying pdfs, so we'd have to add that16:49.24 
Robin_Watts_ but annotations can be created/edited, right? They just don't go into the PDF. They are faked as non-PDF things.16:50.02 
scottschafer no, that's something that polar-bookshelf hacked together. It tracks and renders its "annotations" separately (in a firebase DB)16:50.45 
Robin_Watts_ Ah, ok.16:50.53 
scottschafer So #1 wouldn't be strictly pdf.js API compatible. You could imagine a use case where you wanted to call getAnnotations() without rendering16:51.35 
Robin_Watts_ indeed. #1 doesn't appeal to me.16:51.58 
scottschafer maybe not a big deal16:51.59 
Robin_Watts_ cos at the very least I'd like to be able to say "what type of annotation is this?" in the caller.16:52.23 
scottschafer yeah16:52.29 
Robin_Watts_ I think, my favoured way to work would be for us to replace annotation_layer.js too.16:52.53 
  If we do that, we have more freedom.16:53.11 
  Suppose we do that, and we have mupdf rendering the annotations as we do now.16:54.05 
  The 'render' function for each individual annotation can do nothing.16:54.23 
scottschafer We could give the annotation layer 0 opacity. So the user could still click on elements, etc, but mupdf would render. Not sure about performance though.16:54.24 
Robin_Watts_ scottshafer: OOOOH, that's clever.16:54.48 
  I like that.16:54.54 
scottschafer Dragging annotations around would be very choppy I'd imagine. Is that something we'd want to do?16:55.06 
Robin_Watts_ As long as you can still click on an annotation even when it's got an alpha of 0.16:55.39 
  generally you don't drag the annotation live, you drag a bbox for it.16:55.54 
scottschafer ok, sounds like this approach is worth pursuing16:56.16 
Robin_Watts_ For editing text, it means that the whole doc gets redrawn on every change.16:56.40 
scottschafer yeah, I suppose we could try having a creation mode in which the viewer application renders the annotations while you were acting on them and then commits to the mupdf page16:57.24 
Robin_Watts_ So *maybe* we want to consider having 2 bitmaps for such cases.16:57.24 
scottschafer or yeah, do that16:57.40 
Robin_Watts_ scottschafer: Well, you want fonts to match.16:57.41 
scottschafer true16:57.53 
Robin_Watts_ So either you abandon the thought of doing inline editing for text, and pop up a simple modal window, OR you have to get mupdf to render.16:58.11 
  I believe we do the former in mupdf-gl, but I could be wrong.16:58.32 
  Using a modal window means you don't have to worry about matching fonts etc.16:59.04 
ator we do the former in mupdf-gl --getting inline editing to work well is a lot of work16:59.16 
scottschafer I'm happy to do the modal window16:59.33 
Robin_Watts_ ator: yeah. so much work that I don't think we should be ashamed about punting on it for now.16:59.41 
ator in HTML you could just drop an input field on the right place16:59.46 
Robin_Watts_ scottschafer: Fab. sounds like we're agreed on that then.16:59.53 
scottschafer sounds like I have some things to try. let's check in again after I've made some progress and go from there17:00.16 
Robin_Watts_ scottschafer: Fab.17:00.23 
scottschafer thanks to both you! ttyl17:00.34 
Robin_Watts_ Do you want to register your nickname?17:00.37 
  If you do I can get you into the private channel.17:00.53 
scottschafer sure…not sure how though17:01.08 
Robin_Watts_ just a tick...17:01.15 
  try: /msg nickserv register ?17:01.44 
scottschafer ok, thanks - think that did it17:02.59 
Robin_Watts_ ok, now I need to do some magic. just a mo.17:03.13 
  ok, try doing: /msg chanserv invite #artifex17:04.58 
scottschafer chanserv17:06.01 
  (notice) You are not authorized to perform this operation.17:06.01 
Robin_Watts_ ok, just a mo...17:06.20 
  Apparently you've not completed registration verification.17:06.46 
  that might be why.17:06.50 
  You should get a mail from nickserv with some magic runes in it.17:07.15 
scottschafer hm, nothing yet17:07.43 
  I'm going to have to go, let's figure this out later17:08.13 
Robin_Watts_ Sure. np.17:08.17 
scottschafer ttfn17:08.23 
Robin_Watts_ we're always here if you need to talk.17:08.25 
  too slow.17:08.29 
 <<<Back 1 day (to 2020/03/24)Forward 1 day (to 2020/03/26)>>> 
ghostscript.com #ghostscript