| | <<<Back 1 day (to 2018/01/24) | 20180125 |
| fredross-perry | no, I didn't dig that far yet. | 00:00.28 |
| paulgardiner | fredross-perry: another though. The fix you pulled in fixed mupdf but wasn't sufficient to make muso work. I was calling the functions above with append = 0. I had to make sure I set append = to opts.doincremental. Maybe you have the same problem in android. | 00:00.29 |
| | Bah! It's too late to do typing! | 00:01.05 |
| fredross-perry | I am setting the opts based on the return value of pdf_can_be_saved_incrementally() | 00:01.35 |
| paulgardiner | yeah, but then you also need to make sure you set append to the same value | 00:02.09 |
| | You may be doing that. I wasn't in iOS and needed to | 00:02.28 |
| fredross-perry | I am, I believe. (he said, checking...) | 00:02.34 |
| paulgardiner | Strange for wb to work and not wb+ | 00:02.54 |
| fredross-perry | pdf_save_document sets appand. | 00:03.20 |
| | yes, strange. if you try wb and it works in iOS, maybe that's what we do. | 00:03.44 |
| | yes, and, typing. | 00:03.53 |
| paulgardiner | wb not ideal because we may want to sign a doc we cannot save incrementally, although that would be an unusual case. | 00:04.35 |
| | Maybe open wb, close and then open again rb+. the first open would create or empty the file. | 00:06.44 |
| fredross-perry | that could do it. | 00:07.19 |
| | sorry to harsh your mellow. | 00:07.39 |
| paulgardiner | The other weird thing is, given pdf_save_document sets append, why did I need to change something in iOS muso | 00:08.02 |
| | I needed to pass opts.do_incremental to fz_new_output_with_path. I had been passing 1 | 00:09.13 |
| fredross-perry | yep | 00:12.59 |
| | tor8, sebras - reminder - want to get the streaming i/o pushed. Let me know, please? | 00:35.38 |
| Doof_ | Join | 01:54.03 |
| sebras | Doof_: ? | 01:55.52 |
| | Doof_: you're welcome to ask questions about mupdf here. it may take some time before you get an answer though. | 01:57.13 |
| | Doof_: the reason being that the developers are located in europe and so it is best to ask questions during european working hours. :) | 01:57.45 |
| paulgardiner | What is doc->crypt? Is it surprising for it to be non-NULL when a non-passworded file is loaded? | 10:06.35 |
| | I'm seeing a problem with adding annotations to some files. The non-NULLness of doc->crypt requires the file to be rewritten rather than incrementally updated, and the result is corrupt. | 10:08.10 |
| kens | Are you sure the file is not encrypted ? The lack of a password requirement doens't mean it isn't encrypted | 10:09.02 |
| | I'll look at the file you uploaded to the bug report (the original) | 10:09.34 |
| paulgardiner | That was sort of what I was asking. Might it be encrypted but not passworded. So I guess "yes" | 10:09.44 |
| kens | It is possible for a file to be encrytped, but have no user password (it will still have an owner password) | 10:10.14 |
| paulgardiner | In any case, should that prevent our saving new annotations? | 10:10.23 |
| kens | But you can open an encrytped file which has no user password without supplying a password | 10:10.30 |
| | I think it should yes. I don't think you should be able to edit an encrytped file withotu supplying the owner password | 10:11.03 |
| | There may be a flag for that in the permissions, give me a minute and I'll look | 10:11.22 |
| | Your web_cbt file is encrypted | 10:11.44 |
| | 'Changing the document' is 'not allowed' | 10:11.57 |
| | So I don't think you should be able to annotate that file without having teh owner password. Let me see what happens if I try annotating it in Acrobat | 10:12.36 |
| | Yeah the annotations are grayed out in Acrobat | 10:13.24 |
| | So I cannot add annotations to this file | 10:13.32 |
| paulgardiner | Okay thanks. I guess I need to use the presence of doc->crypt to disable editing | 10:14.15 |
| kens | Not exactly | 10:14.25 |
| | If doc-cryupt is present you need to check the permissions flags | 10:14.35 |
| | And if 'changing the document' is not allowed then you disable editing | 10:14.59 |
| paulgardiner | Is it still worrying that mupdf produces a corrupted file when we try, or is that expected? | 10:15.18 |
| kens | I'm not sure how you might detect an encrypted file where the user has supplied hte owner password, in that case they can edit it | 10:15.27 |
| paulgardiner | I'll see if I can find out what old SO (pre MuSO) did. Can't remember this issue coming up | 10:16.17 |
| kens | paulgardiner : I can't really answer that. Creating a corrupted PDF sounds wrong, perhaps MuPDF should not permit this for encrypted documents unless the password is supplied. But I don't know enough about MuPDF | 10:16.23 |
| | It certainly should not permit you to alter that file. | 10:16.37 |
| | Unless, of course, you supply the owner password | 10:16.54 |
| | OK you need to check bit 4 and 6 of the /P in the encryption dictionary | 10:19.25 |
| | Actually, just bit 6 it seems | 10:20.23 |
| | That allows 'commenting' (as described by Acrobat) which the spec says means 'Add or modify text annotations, fill in interactive form fields and if bit 4 is also set create or modify interactive form fields (including signature fields). | 10:21.34 |
| | That's the PDF 2.0 definition | 10:21.56 |
| paulgardiner | Old SO permits editing in that case and saves using incremental update. | 10:32.20 |
| kens | Well it shouldn't :-) | 10:36.44 |
| | But note that the enforcement of encryption and especially permissions, particularly when using the old security handlers and no user password is entirely enforced by the application | 10:37.23 |
| | So its pretty much open to abuse | 10:37.36 |
| paulgardiner | I'm surprised that an encrypted file with an incremental update add to the end of it is still valid. or valid enough that Acrobat doesn't moan | 10:43.50 |
| kens | So am I frankly, but I can't recall exactly how that works | 10:44.09 |
| | Can you mail me the edited file to look at ? | 10:44.24 |
| paulgardiner | Sure | 10:46.21 |
| kens | Thanks | 10:46.26 |
| | I'm curious what GS will make of it :) | 10:46.37 |
| | I suspect I know what's going on but I'd like to check | 10:47.20 |
| | Its incremental, but the trailer dictionary still maintains the Encrypt entry. Encryption only affects streams, and the Annotation doesn't have an Appearance (/AP) entry, so there's no stream to be encrypted. | 10:51.20 |
| | So basically its completely valid | 10:51.35 |
| | Its simply the case that SO should have refused to modify the document, because the permissions didn't allow it. Like I say, you are dependent on the applications to enforce these. SO is by no means hte only application which doesn't | 10:53.00 |
| | Oops actually I wasn't quite correct. | 10:53.53 |
| | The annotation *does* have an apperance stream, but its properly encrypted, so its still 100% valid. | 10:54.11 |
| paulgardiner | So, although the app should perhaps request the owner password, mupdf should be able to save the new annotation, given we now support doing so when there's a user password. | 10:58.27 |
| kens | Oh yes, it should always be *possible* to modify the document, because we *must* be able to decrypt it. | 10:58.58 |
| | If we can't decrypt it, we can't render it | 10:59.09 |
| | If we can decrypt it, then we can modify and re-encrypt it | 10:59.19 |
| | But the application should enforce the permissions. In this case it should insist on the owner password before allowing annotations to be added. Which is what Acrobat does, it prompts for the password. | 10:59.58 |
| paulgardiner | Yeah, so two independent problems. Saving is causing corruption in this case. We aren't requesting the owner password before permitting editing. | 11:01.58 |
| kens | Indeed. I can't really comment on either I'm afraid, I can only diagnose it, sorry. You'll need Robin's help probably for fixing the corruption, that one is just weird. | 11:02.42 |
| | Requesting the owner password is more of a UI thing really. | 11:03.06 |
| paulgardiner | Yeah | 11:03.12 |
| | I'm not so bothered about that for now because old SO didn't request it. | 11:03.35 |
| kens | Fair enough, not my problem :-) | 11:03.51 |
| paulgardiner | I've opened bug 698918 for this issue. Also opened bug 698917 for a crash that happens with several files | 11:59.41 |
| Robin_Watts | tor8, sebras: One of us (by which I mean, probably not me) should look at those as a matter of urgency. They are, I think, blocking a release. | 12:03.51 |
| tor8 | oh great, openjpeg crashes... | 12:05.20 |
| sebras | Robin_Watts: a release of what? | 12:10.26 |
| | Robin_Watts: mupdf? | 12:10.30 |
| | paulgardiner: can you only reproduce 698917 on iOS?! | 12:13.27 |
| Robin_Watts | sebras: MuSo | 12:13.42 |
| sebras | Robin_Watts: ah, that's why I can't see the Blocks-bug. | 12:14.11 |
| paulgardiner | sebras: I have not tried other than on iOS | 12:15.32 |
| sebras | paulgardiner: I ran the file successfully using openjpeg on x86_64 both in valgrind and ASAN. no assert(). | 12:16.04 |
| Robin_Watts | sebras: Is SSE enabled in openjpeg on desktop builds? | 12:16.24 |
| | (SSE won't be used in the ios build, so, could be a bug in the non SSE paths?) | 12:16.51 |
| sebras | Robin_Watts: __SSE__ is set when I compile on my machine, yes. | 12:18.01 |
| Robin_Watts | sebras: If you compile without __SSE__ does it crash? (or assert rather, so it'll need to be a debug build) | 12:19.44 |
| sebras | Robin_Watts: didn't know how to disable __SSE__ so I just deleted all the ifdefs, running now. | 12:20.16 |
| | Robin_Watts: nothing in valgrind. | 12:20.30 |
| | and nothing in ASAN. | 12:20.48 |
| sebras | is puzzled. | 12:20.55 |
| paulgardiner | sebras: do you know if your tests are taking execution through that particular assert? | 12:23.37 |
| sebras | paulgardiner: now assertion is printed. | 12:23.51 |
| paulgardiner | But is that because it always passes or because execution never makes it through there? | 12:24.26 |
| malc_ | sebras: -mno-sse ? | 12:24.43 |
| sebras | paulgardiner: ah, I see I misunderstood your question. | 12:24.45 |
| | malc_: ah, thanks. | 12:24.58 |
| | paulgardiner: I placed an abort() on the line after the assert() in opj_pi_create_decode() and now trying to render page 1 of the sample file fails. | 12:26.35 |
| | paulgardiner: and removing the abort() allows this to render fine again. | 12:27.27 |
| paulgardiner | Okay | 12:27.54 |
| | Could be iOS specific then | 12:28.24 |
| sebras | paulgardiner: or some other combination of CFLAGS that affects this. | 12:29.29 |
| paulgardiner | MuSO is using the Makefile unaltered. | 12:30.06 |
| sebras | malc_: -mno-sse was not a winner, it causes many "SSE register return with SSE disabled" warnings. not sure I understand why. | 12:32.14 |
| malc_ | sebras: because on x86_64 floating point is implemented atop of sse? ;) | 12:32.57 |
| | sebras: you can try -mno-sse[2..] just not a plain -mno-sse for kicks | 12:36.10 |
| sebras | malc_: #ifdef __SSE__ code would still be compiled in that case though. | 12:37.46 |
| malc_ | sebras: yeah, that said my grep for SSE inside of mupdf yields nothing interesting | 12:41.09 |
| sebras | malc_: it ought to in openjpeg. | 12:41.31 |
| malc_ | sebras: right you are mct.c et al | 12:42.23 |
| | and ominous comments like ( /* tile data shall be aligned on 16 bytes */) there | 12:43.44 |
| sebras | malc_: if __SSE__ is enabled, yes. | 12:45.48 |
| | paulgardiner: I wonder if thirdparty/openjpeg/src/lib/openjp2/opj_config_private.h possibly setting OPJ_BIG_ENDIAN could have something to do with this. | 12:48.14 |
| paulgardiner | sebras: do you have a change in mind I could try out? | 12:49.23 |
| sebras | paulgardiner: I'd like to know if the tests in thirdparty/openjpeg/src/lib/openjp2/opj_config_private.h causes a different state on iOS than it does on my machine. I'm not sure if this is the cause of course. | 12:50.06 |
| | paulgardiner: I added this: https://pastebin.com/raw/yz20HM69 and when compiling I (of course) see little printed repeatedly. | 12:52.21 |
| paulgardiner | :-) I read that as you don't see very much printed out!! | 12:53.56 |
| | I'll try it here once my vm has booted | 12:54.22 |
| sebras | paulgardiner: oh. here, have two quotes: "" | 12:54.27 |
| paulgardiner | Ah that's better. | 12:54.41 |
| Robin_Watts | IIRC, iOS is wierdly mixed endian. | 12:57.33 |
| sebras | Robin_Watts: not, UNIX, not XINU, but NUXI?! | 12:57.59 |
| Robin_Watts | sebras: Instructions are one endianness, data the other, or something? | 12:58.17 |
| | I remember it being bloody odd, anyway. | 12:58.42 |
| malc_ | instructions are always LE in ARM if my memory serves me | 12:59.05 |
| sebras | Robin_Watts: sounds like someone went wild on those configuration parameters for the ARM IP block... | 12:59.05 |
| Robin_Watts | sebras: Software switchable on ARM, I think (or maybe switchable by voltages on pins) | 12:59.37 |
| malc_ | page tables are always BE in power.. historic quirks like that is what makes life interesting | 12:59.40 |
| paulgardiner | Bah. I'm not seeing big or little | 13:18.10 |
| sebras | paulgardiner: oh. | 13:19.48 |
| paulgardiner | Could be to do with Xcode processing the build log. I need to run under Xcode to be sure of testing in the right environment. | 13:21.00 |
| sebras | paulgardiner: btw, I ran openjpeg's own tool opj_decompress on the JPX encoded images from page 1 inside valgrind. no complaint.s | 13:22.32 |
| | so I can't reproduce this. :-/ | 13:22.40 |
| paulgardiner | Not under linux you mean? | 13:23.53 |
| sebras | paulgardiner: exactly, this is my only platform. | 13:25.21 |
| kens | Excuses! :-) | 13:25.40 |
| paulgardiner | I guess either Artifex gets you an iOS device or someone else needs to look at it then. | 13:27.38 |
| sebras | kens: but a Good<tm> one! | 13:28.40 |
| kens | Well it might win you an iOS device to play with | 13:29.23 |
| malc_ | envies sebras... a new cisco router whoa.. | 13:30.33 |
| Robin_Watts | paulgardiner: You're not going to like it if I point out who in the MuPDF development team actually has an iOS device... | 15:36.43 |
| jogux | Robin_Watts: I was assuming the bugs Paul is raising aren't iOS specific. | 15:40.23 |
| kens | The OpenJPEG one won't exhibit on Linux | 15:40.40 |
| | and so may be iOS-specific | 15:40.47 |
| paulgardiner | I don't mind looking at it, provided no one minds that the thing I'm trying to finish off wont get done and in turn nor will the thing I'm actually supposed to be doing. | 15:42.30 |
| jogux | oh, urgh. not even in valgrind? | 15:42.34 |
| kens | sebras says not | 15:42.51 |
| | But there was some discussion about SSE as well | 15:43.02 |
| Robin_Watts | I floated the idea that the SSE code (used on desktops) might be working, and the C stuff not. | 15:43.25 |
| kens | Yep, I think its possible | 15:43.37 |
| Robin_Watts | but sebras disabled the SSE stuff, and still couldn't see the problem. | 15:43.38 |
| kens | Ah, I missed that | 15:43.45 |
| sebras | jogux: nope, I can't reproduce it. | 15:43.55 |
| | jogux: can't see the assert() being triggered on page one of that file (or any other page for that matter) | 15:44.20 |
| jogux | weird. did you try 32 or 64 bit build? | 15:44.31 |
| sebras | jogux: I tried to see if there was something to maket it undeterministic according to valgrind/ASAN, but they didn't trigger either. | 15:44.50 |
| | jogux: I tried on 64-bit with and without SSE, I also tried on 32-bit in a virtualbox. | 15:45.22 |
| jogux | pretty comprehensive then. thanks sebras :) | 15:45.38 |
| sebras | jogux: I only found one thing in openjpeg where they test __APPLE__ and this controls OBJ_BIG_ENDIAN | 15:45.58 |
| | for me I end up using little endian. | 15:46.04 |
| | jogux: comprehensive, but unhelpful. :-/ | 15:46.25 |
| jogux | paulgardiner: oh, are you sure it's not a symbol clash with sot, or did you fix all those? | 15:46.39 |
| | I guess we could try it in the mupdf ios app to rule that out. | 15:47.39 |
| paulgardiner | Could be I guess. I thought you looked at those | 15:47.40 |
| jogux | uh, I thought you did | 15:47.49 |
| paulgardiner | I may have added a define to the build prompted by you, but I've never looked at the problem in general | 15:49.50 |
| sebras | jogux: note that you'd probably need to build the mupdf ios app from scratch. I don't know how up to date the one on apple's store is. | 15:51.45 |
| jogux | sebras: mmm | 15:52.06 |
| | we use the same openjpeg library in sot afaicr so it's unlikely to be an issue in the actual library. | 15:52.29 |
| sebras | jogux: same version of it? I'm not sure, but perhaps we have local patches in openjpeg..? | 15:52.52 |
| malc_ | paulgardiner: all this crashing is related to https://bugs.ghostscript.com/show_bug.cgi?id=698917 ? | 15:55.34 |
| jogux | yeah, I think we took the one from mupdf. in October last year. | 15:55.47 |
| paulgardiner | malc_: yes, I believe so | 15:56.40 |
| malc_ | paulgardiner: it doesn't crash here... lots of openjpeg warnings in console after page 3, but no crashes | 15:57.29 |
| paulgardiner | malc_: what OS? | 15:57.54 |
| malc_ | paulgardiner: linux | 15:58.06 |
| paulgardiner | malc_: yes, no one has been able to reproduce it on linux | 15:58.25 |
| malc_ | paulgardiner: where does it crash? all this sse talk leaves only two practical choices windows and macos, which one? | 15:59.09 |
| paulgardiner | iOS | 15:59.26 |
| malc_ | paulgardiner: uhm.. then what sse has to do with all of this? | 16:00.36 |
| Robin_Watts | malc_: Reread the logs. | 16:00.46 |
| jogux | paulgardiner: nm suggests there are symbol clashes. | 16:00.56 |
| malc_ | Robin_Watts: aha, my bad | 16:01.38 |
| paulgardiner | jogux: does SO have a define for renaming the openjpg entry points? | 16:02.03 |
| jogux | I doubt it, given openjpg was a relatively recent addition. | 16:02.26 |
| paulgardiner | More than a little screwed then | 16:02.50 |
| jogux | I can have a prod at that probably. | 16:06.51 |
| paulgardiner | Wonderful. Thanks | 16:07.03 |
| | Apologies for dragging everyone into this bug if it is just an entry point clash. Never occurred to me | 16:33.10 |
| malc_ | paulgardiner: fwiw it doesn't reproduce without SSE either (tested by building m32 release version) | 16:39.48 |
| paulgardiner | malc_: okay. Thanks for looking | 16:40.12 |
| malc_ | paulgardiner: looks like something arm specific... | 16:41.05 |
| paulgardiner | malc_: we're currently thinking it may be because the project we are using it within has another library which pulls in a second copy of openjpg | 16:42.04 |
| malc_ | paulgardiner: funky | 16:43.28 |
| Dharmik | Hello, Mupdf how to get text which is not Hyperlink but it simple writter ashttp://www.google.com how to detect it ? | 18:09.14 |
| | Not as Hyperlinks, it simple writehttp://www.google.com | 18:09.35 |
| | How can we detect those text ? | 18:09.47 |
| | As link | 18:09.53 |
| sebras | Dharmik: if you want to copy text on the desktop version of mupdf, like mupdf-gl or mupdf-x11, you can right-click to copy text. | 18:10.26 |
| | Dharmik: mupdf for androi has a special button for copying text. | 18:10.29 |
| Dharmik | in Android is there any way that we use this function **MuPDFCore_getPageLinksInternal**? | 18:14.33 |
| | I need those link in this method | 18:14.54 |
| | is it possible / | 18:20.58 |
| | On singleTap I need to open that link in broweser | 18:21.41 |
| | anu one there ? | 18:31.48 |
| | I am not sure but I think I need to change something in this method xps_load_links_in_fixed_page | 18:42.25 |
| | I am not sure but I think I need to change something in this method **xps_load_links_in_fixed_page** | 18:42.34 |
| sebras | jogux: I see in thirdparty/openjpeg that there are indeed a number of patches that we have. so if muso is using plain openjpeg, perhaps they might be the cause of pauls crashes? | 19:12.42 |
| | Robin_Watts: I think I know what might be wrong with 692758 | 23:40.20 |
| | Robin_Watts: the annotation has blend mode multiply. | 23:40.43 |
| | Robin_Watts: in mupdf-x11 we clear a pixmap and give to a draw device, then we first render page contents and then annotations onto the same pixmap. | 23:41.19 |
| | Robin_Watts: in mupdf-gl we render the page contents to one pixmap using one device, and then render all annotations to a second pixmap using another device. | 23:41.44 |
| | then later on in platform/gl/gl-main.c::do_forms() we try to blend them together, but at the point we don't take the blend mode into account. | 23:42.22 |
| | perhaps if each annotation could signal what type of blending should be done we could call glBlendFunc() with something appropriate, but I'm not sure it covers all strange blend modes that each annotations might use. | 23:44.47 |
| | I'm not sure why tor8 decided to render to separate textures, perhaps there is some point in doing so, but if we don't I think the blend modes would be sorted out automatically (as they are in the -x11 case). | 23:51.19 |
| | Forward 1 day (to 2018/01/26)>>> | |
Log of #mupdf at irc.freenode.net.