| | <<<Back 1 day (to 2018/01/28) | 20180129 |
| paulgardiner | I want to test mupdf under linux, but my system's openssl is back on 1.0.1. Is there a clean way that I can build with a downloaded version of openssl (without messing up my server)? | 09:39.17 |
| sebras | paulgardiner: if you install openssl like so: ./Configure linux-x86_64 && make it will install in /usr/local/ssl (or you can give --prefix=/tmp/ssl to Configure and have it install there instead) | 10:19.29 |
| | paulgardiner: next you can find openssl's pkg-config file by e.g. find /usr/local/ssl -name '*.pc' | 10:19.51 |
| | paulgardiner: this file contains all paths to the libraries and cflags and ldflags that are necessary to link with that openssl. | 10:20.13 |
| | paulgardiner: mupdf's Makerules uses pkg-config to locate openssl. | 10:20.24 |
| | paulgardiner: therefore I believe that you would be able to run PKG_CONFIG_LIBDIR=/usr/local/ssl/lib/pkgconfig/ make -j10 to compile mupdf with the non-system installed openssl. | 10:21.07 |
| paulgardiner | And nothing else in my system will suddenly decide to use this new version? | 10:25.02 |
| sebras | if you install in /usr/local/ssl programs that look for openssl there will find it. | 10:25.28 |
| | if you install to /tmp/ssl instead nothing should be able to find it. | 10:25.47 |
| | but you'd need to supply --prefix to ./Configure. | 10:25.58 |
| | paulgardiner: also there is no need to install it as root. | 10:26.17 |
| paulgardiner | And in the rest of your instructions, I sub /tmp/ssl for/usr/local/ssl | 10:26.38 |
| sebras | yes. | 10:26.53 |
| paulgardiner | Great. That sounds like a plan | 10:26.56 |
| sebras | paulgardiner: good luck! :) | 10:27.04 |
| paulgardiner | If I have no more email, no more tv recordings and my websites disappear, I'll be blaming you! :-) | 10:28.12 |
| sebras | paulgardiner: if you are anxious, try it in a virtualbox..? | 10:28.45 |
| | vmware | 10:28.54 |
| | parallels, whatever... :) | 10:29.05 |
| paulgardiner | I feel fairly safe given I can do it without using root | 10:29.09 |
| | Does PKG_CONFIG_LIBDIR add new directories rather than overriding? I'm just thinking that the build still needs to find the other thirdparty configs | 10:31.04 |
| sebras | paulgardiner: I think it might be replacing the default searchdirs in /usr/lib and so on. | 10:31.31 |
| | paulgardiner: I tested briefly it and it appeard compiled fine here. I never ran the resulting binary though! :) | 10:31.56 |
| paulgardiner | Tested what? | 10:32.47 |
| | Oh I see | 10:33.00 |
| sebras | paulgardiner: and if I'm wrong somewhere, let me know. | 10:42.21 |
| paulgardiner | sebras: build/release/scripts/namedump.exe: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory | 10:51.29 |
| | Makefile:174: recipe for target 'include/mupdf/pdf/name-table.h' failed | 10:51.31 |
| | make: *** [include/mupdf/pdf/name-table.h] Error 127 | 10:51.32 |
| sebras | paulgardiner: ah, then you also need to set LD_LIBRARY_PATH when compiling mupdf | 10:54.48 |
| | like so: LD_LIBRARY_PATH=/tmp/ssl/lib/ make | 10:55.06 |
| | or wherever libcrypto.so.1.1 might be installed in /tmp/ssl | 10:55.17 |
| paulgardiner | Ah right. Thanks | 10:55.36 |
| | That did it. It now fails in the way I was expecting, because the restructuring I've done. So now I can update the make file to account for the restructuring. | 10:58.54 |
| sebras | paulgardiner: nice. | 10:59.05 |
| | paulgardiner: remember to remove the stuff from /tmp or wherever you installed it later. no need to keep it around. :) | 10:59.45 |
| | tor8: sebras/master | 11:23.46 |
| tor8 | sebras: both LGTM | 11:25.43 |
| sebras | both?! apparently I need to take a look at my branch myself. :) | 11:26.45 |
| | ah, yes. the stream one doesn't entirely fix 698830 though. | 11:27.21 |
| | now I end up in an O(n^2) thingy related to our error handling, but we don't crash in ASAN! :) | 11:27.56 |
| tor8 | sebras: in that case, "Bug 698942: Don't complain about external URLs being unknown." LGTM | 11:28.42 |
| sebras | tor8: well, the second patch is still valid, it just doesn't fix the entire thing. | 11:29.16 |
| | I think the other error handling thing might be in pdf-repair and is actually unrelated. | 11:29.31 |
| paulgardiner | sebras: how do I check a signature with the x11 app? Clicking on it doesn't seem to do anything | 12:01.49 |
| sebras | paulgardiner: it ought to. | 12:02.27 |
| | paulgardiner: but it prints the result to the console. | 12:02.40 |
| | on stderr..? | 12:02.47 |
| paulgardiner | Not seeing any messages | 12:02.55 |
| sebras | paulgardiner: see pdfapp_onmouse() line 1721 in platform/x11/pdfapp.c | 12:03.29 |
| | paulgardiner: then perhaps something is wrong. | 12:03.40 |
| | paulgardiner: I have gotten that part working. note that for some reason not all signatures are visible. | 12:04.07 |
| | but you probably already know that. | 12:04.17 |
| paulgardiner | yeah, it's a file I frequently use for testing. | 12:04.36 |
| | And also weird: I just added some debug to pdfapp.c and make is saying nothing to be done | 12:06.44 |
| sebras | paulgardiner: could it be that mupdf-x11 is not built..? | 12:08.12 |
| | paulgardiner: you should also be able to run mutool sign my-doc.pdf to have sigantures validated using the same code. | 12:08.31 |
| | the same API. | 12:08.48 |
| paulgardiner | mutool sign works | 12:11.09 |
| sebras | paulgardiner: great, then use that. :) | 12:11.23 |
| paulgardiner | Hmmm! I don't know. That would be too easy. :-) | 12:11.53 |
| sebras | paulgardiner: you do remember that you need to set LD_LIBRARY_PATH=/tmp/ssl/lib or whatever when you run mutool and mupdf-x11 too, right? | 12:12.40 |
| paulgardiner | I didn't but the first failure reminded me. | 12:13.19 |
| | mutool sign seems to be working fine. mupdf-x11 isn't but maybe that's my X setup, or because I'm using VNC | 12:14.12 |
| | The signature support reorganisation is done, I think. Now the main build doesn't include a pkcs7 implementation. Instead an object is passed to pdf_sign_signature. The openssl implementation is now a helper. There's also a helper verification function that can be called to get the old behaviour. | 13:15.31 |
| | The changes are on the branch signature-support-reorganisation. | 13:16.24 |
| tor8 | paulgardiner: looking. | 13:33.04 |
| paulgardiner | Thanks tor8 | 13:37.48 |
| tor8 | paulgardiner: I've now looked at all but the "decouple" commit | 13:38.31 |
| | a few minor nitpicks | 13:38.40 |
| | but a bigger question for you: does the pkcs7 stuff that is split out now depend on pdf at all? | 13:39.00 |
| | or is it completely pdf independent, in which case it may make sense to move that module to the fitz namespace | 13:39.14 |
| paulgardiner | Okay. Makes sense. I did wonder that. | 13:39.45 |
| tor8 | ah, you moved it to helpers/pkcs7 anyway later | 13:40.17 |
| sebras | tor8: can xps files also be digitally signed? | 13:40.29 |
| tor8 | sebras: not to my knowledge | 13:40.37 |
| sebras | tor8: I peeked into the openxps spec: "penXPS Documents also use these digital signature mechanisms, they have a specific | 13:42.29 |
| | agh! | 13:42.38 |
| paulgardiner | I guess the question is whether the pdf_pkcs7_signer objects should be move to fitz. The one function that takes one as an argument is definitely part of pdf. | 13:42.42 |
| tor8 | paulgardiner: I don't see it as crucial now that the majority of the functions moved into the helper library | 13:43.09 |
| sebras | tor8: The digital signature specification for OpenXPS Document is described int he OPC standard." | 13:43.11 |
| | tor8: my reasoning being that then it makes even more sense to move it out of pdf. | 13:43.33 |
| tor8 | but if pdf-pkcs7.c now doesn't depend on any pdf_document stuff it may still make sense to just move it into fitz | 13:43.52 |
| sebras | paulgardiner: in pk7_verify(): "were we able to find the cert in passed to us", I assume this should be "passed in"? | 13:44.54 |
| | paulgardiner: X509_STORE_CTX_new() may fail with NULL, but pk7_verify_cert() doesn't handle that. | 13:48.34 |
| paulgardiner | That chunk of code is partially cribbed from the openssl library. I think those comments are copied across. | 13:48.49 |
| | sebras: oh yes. Well spotted | 13:49.34 |
| sebras | paulgardiner: ah, possibly the openssl implementation you were inspired by also needs a few more if statements. :) | 13:50.07 |
| | paulgardiner: actually the same goes for the calls to X509_STORE_CTX_set_purpose() and X509_verify_cert() too. | 13:50.44 |
| | paulgardiner: I'm confused as to whether openssl depends on return codes to check errors or if they just don't bother and instead rely on function calls such as X509_STORE_CTX_get_error(). but if that is the case, why return error codes? | 13:51.43 |
| paulgardiner | It seems to be a crazy mix of the two approaches | 13:52.20 |
| sebras | paulgardiner: I hate that. :-P | 13:52.37 |
| | paulgardiner: perhaps you are meant to first get error/noerror as a return code and then use the function call to see which one? | 13:53.15 |
| paulgardiner | I think I tried that at one stage and it didn't work | 13:53.53 |
| | I have test files that produce each of the errors I'm attempting to report, although not many of each | 13:54.53 |
| sebras | paulgardiner: ok, I'm just reading the code and looking at function signatures. | 13:55.51 |
| | paulgardiner: and if there is no error checking I usually ask about it. not because I know it might fail, but because the developer needs to think about it and rule it out or add a check. like now. :) | 13:56.49 |
| paulgardiner | That's sounds like a good thing. I've certainly missed one case, as you just pointed out | 13:58.01 |
| sebras | paulgardiner: I have issues compiling commits 15ba61e3 and b72e1bda (undefined reference to `pdf_check_signature' in function `pdfapp_onmouse') as well as commit 7b7601dd (undefined reference to `pdf_check_signature', `winquery', `wingetcertpath', `pkcs7_openssl_read_pfx' in function `pdfapp_onmouse'( | 14:02.59 |
| | ) | 14:03.04 |
| paulgardiner | Are you sure that isn't 7b7601dd | 14:06.02 |
| | I thought I was introducing that only on my master branch, with no intention to push to golden. | 14:06.49 |
| sebras | paulgardiner: the first two sit on paul/signature-support-reorganisation and the last one is only on paul/master | 14:07.41 |
| | paulgardiner: none of it is on origin/master yet. :) | 14:07.54 |
| paulgardiner | Sorry. Misread your message | 14:08.02 |
| sebras | paulgardiner: let me nuke inbetween everything so it is not a dependency issue. | 14:08.55 |
| paulgardiner | The pdf_check_signature one is part of what I was sorting out under linux earlier (in pdfsign at least). Maybe I missed a warning fom pdfapp.c | 14:10.13 |
| sebras | running make nuke inbetween things helped, now it all compiles on every commit. but doesn't that imply makefile dependency issues? | 14:10.34 |
| paulgardiner | I saw similar problems earlier, but that was before I made any changes to the make file | 14:11.44 |
| | Should I be adding something explicit to the make file for the sake of pdfapp.h including two .h files from the helpers directory? | 14:14.34 |
| sebras | paulgardiner: so in Makefile you've added source/helpers/pkcs7/*.c to MUTOOL_SRC. | 14:15.51 |
| | paulgardiner: are those .c-files not necessary for mupdf-x11, mupdf-gl etc, as well..? | 14:16.07 |
| paulgardiner | Ah yes. Anything that uses pdfapp.c | 14:16.44 |
| | So it was working by luck just because mutool built first. | 14:17.12 |
| sebras | paulgardiner: I'm still confused as to how it ever worked. :) | 14:17.42 |
| paulgardiner | Yeah, actually, how come it linked those .o files | 14:20.53 |
| | Not sure how to handle this. Should I add a rule for pdfapp.o | 14:24.28 |
| sebras | paulgardiner: you would creat a library similar to libmuthreads.a perhaps? | 14:24.42 |
| paulgardiner | Yeah, I wondered about that. I struggled a bit to see all that's involved with that. | 14:25.25 |
| sebras | paulgardiner: and you actually have two object files rather than one so in the pkcs7 case it makes more sense. | 14:25.35 |
| | paulgardiner: seems to be confined to the makefile and platform/win32/libmuthreads.vcproj and platform/win32/mupdf.sln | 14:26.21 |
| | paulgardiner: git grep muthreads and git grep '\<THREAD_LIB\>' illustrates this. | 14:26.53 |
| paulgardiner | Seemed a bit odd to me to make a library out of it when I'm not sure we intend people to use this openssl-based solution, although obviously we are still using it in our apps | 14:30.36 |
| | Couldn't I just define something like SIGNATURE_OBJ and include that in each app that needs it? | 14:32.25 |
| | Is that the general consensus, that I should add something like libmupkcs7.a? | 15:06.36 |
| tor8 | paulgardiner: yes, I think that would be best. | 15:32.53 |
| paulgardiner | Okay, I'll have a go | 15:58.38 |
| sebras | tor8: "Update mupdf_native.h header." LGTM. | 16:05.15 |
| | tor8: "Add buffer to fz_output." LGTM. | 16:05.44 |
| | tor8: "Reduce size of pool allocator blocks to minimize waste." also LGTM | 16:06.38 |
| tor8 | sebras: there's a patch for "Add buffer to fz_output" on the top of tor/master | 16:07.59 |
| | if that looks good to you I'll squish it back down before I push | 16:08.09 |
| sebras | tor8: for "Return PDF_TOK_ERROR when encountering isolated '>'" what was the outcome of the cluster eventually? I remember we saw bmpcmp issues with annotations. | 16:08.53 |
| tor8 | I'm clustering anew to see if we get any diffs | 16:09.07 |
| | but if those three + the top one are good I'll push those in the meanwhile | 16:09.31 |
| | I rearranged the order so the string tokenizing ones are at the end | 16:10.28 |
| | so everything up to 'tofu' should be good AFAIK | 16:10.57 |
| sebras | tor8: that top commit looks reasonable, but that means that the gdb command now is: call pdf_print_obj(ctx, fz_stdout(ctx), pdf_resolve_indirect(ctx, ref), 0), fz_flush_stdout(ctx, fz_stdout(ctx)), fflush(0) | 16:10.59 |
| tor8 | sebras: yes. if we want that, we should just add a more convenient call | 16:11.29 |
| | like pdf_debug_obj(ctx, obj) | 16:11.34 |
| sebras | tor8: I use it often enought that it is warranted. I'll add it after your stuff is committed. | 16:11.56 |
| | tor8: you can squash the output ones. | 16:12.12 |
| tor8 | sebras: thanks. a WIP commit for pdf_debug_obj on tor/master now too. | 16:17.06 |
| | I don't know if you want tight or loose printouts by default | 16:17.42 |
| | personally I wouldn't mind tight being the default for pdf_debug_obj | 16:17.54 |
| | but I don't think it matters too much, so I'll leave the choice to you | 16:18.06 |
| | (but I don't think it warrants adding the 'tight' argument, you want it to be convenient) | 16:18.18 |
| sebras | yeah, but a single ,0 or ,1 I can survive. | 16:18.38 |
| | it's these chained function calls that get annoying. | 16:18.51 |
| tor8 | yeah, but you shouldn't have to remember to add a ,0 or ,1 :) | 16:19.23 |
| sebras | ok, so we just put it as the first argument. | 16:19.42 |
| | that way it won't be ,0 or ,1 | 16:19.49 |
| sebras | <--- brilliant! :) | 16:20.04 |
| | tor8: in "Return error token if strings are unterminated." should we also return PDF_TOK_ERROR when we fail to parse #hexhex in lex_name()? | 16:21.54 |
| | tor8: I think we should since we complain about bad numbers in lex_number() and now strange strings and hexstrings. | 16:27.07 |
| paulgardiner | There's a line "$(OUT)/source/helpers/%.o : source/helpers/%.c | $(ALL_DIR)" in Makefile and helpers does not directly contain any c files. It contains subdirectories which in turn contain c files. | 16:34.08 |
| sebras | paulgardiner: then I don't think that line will do anything for us. :-/ | 16:35.46 |
| paulgardiner | Okay, I'll delete it | 16:36.04 |
| sebras | paulgardiner: we probably need corresponding lines for the directories we do have though. | 16:36.21 |
| | pkcs7 and... threads..? | 16:36.26 |
| paulgardiner | yeah, although - for threads - the make file currently works somehow | 16:37.30 |
| | I'll see if I can work out by what means | 16:37.49 |
| sebras | paulgardiner: could be working of the implict rules, see make -p | 16:38.54 |
| paulgardiner | I think I can see how it should be | 16:43.21 |
| | THREAD_OBJ is defined unconditionally and then later there is THREAD_OBJ := $(THREAD_OBJ) (which I am interpreting as define it as "" if not already defined) | 16:47.08 |
| | Actually, it looks like that rule I mentioned earlier for helpers/%.c was there just to make sure $(THREADING_CFLAGS) were included for the compile of those files. I maybe don't need that for the pkcs7 files because LIBCRYPTO_CFLAGS is already added to CFLAGS. | 16:52.07 |
| sebras | hm.. but if there is no rule for the hlpers then that means that the DIRs will not be created correctly, nor will the objekt files properly depend on the .c-files. | 16:53.53 |
| | I might be missing something. | 16:54.09 |
| | perhaps those rules are only if there are special flags and are not meant for tracking dependencies. | 16:54.29 |
| paulgardiner | No that sounds right now you mention it. The rules end in | $(ALL_DIRS) | 16:54.46 |
| tor8 | sebras: I don't htink so, acrobat ignores bad escape sequences | 16:55.08 |
| sebras | paulgardiner: ah, at line 165 you have $(THREAD_OBJ) : $(THREAD_HDR) this is what keeps track of the deps. | 16:55.28 |
| | tor8: oh, does it? :-P | 16:55.44 |
| paulgardiner | sebras: what's the import of that? Does that relate to the rule I was questioning? | 16:56.11 |
| | The .o from .c one I mean | 16:56.35 |
| sebras | paulgardiner: the line THREAD_OBJ := $(THREAD_OBJ) just looks strange to me. | 16:58.12 |
| paulgardiner | I think delete it | 16:58.25 |
| | I might ask Robin in case he knows a reason for it | 16:58.56 |
| sebras | paulgardiner: it was added in cb44c9ae591ae5cdfda932f04bf321932a0e1d0f, but I can't see why. | 16:59.07 |
| Robin_Watts | hmm. | 16:59.09 |
| paulgardiner | It might have been that THREAD_OBJ was once defined conditionally and then this line made sure that it was at least defined to something. | 16:59.44 |
| Robin_Watts | That line seems redundant to me | 16:59.54 |
| | please feel free to remove it. | 17:00.32 |
| sebras | tor8: apart from my PDF_TOK_ERROR question and the WIP commit tor/master LGTM. | 17:20.05 |
| tor8 | sebras: I don't think a bad escape sequence should generate a PDF_TOK_ERROR | 17:32.16 |
| | it would not behave like adobe reader then | 17:32.23 |
| sebras | tor8: yeah, I got that. I meant that other than that I had no questions. | 17:35.57 |
| tor8 | sebras: ah, okay. | 17:36.36 |
| | just tracking down a cluster diff... | 17:36.51 |
| sebras | tor8: eep, jag kraschade nyss i fz_output. | 17:42.36 |
| | tor8: wrong language or wrong window. | 17:46.46 |
| | tor8: this crash happens consistently when I use mutool draw -F png -o out%02d.pnm doc.pdf | 17:47.09 |
| | seems to be related to the file mode. | 17:47.19 |
| | drawpage() calls fz_new_output_with_path() with append == 0 the first time, but with append == 1 the next time. | 17:47.40 |
| | not sure why. | 17:47.43 |
| tor8 | sebras: the output logic in mudraw is beyond my comprehension... | 18:01.14 |
| sebras | tor8: found it. | 18:01.19 |
| | output_append should go away. I'll send a patch in a minute. | 18:01.29 |
| | tor8: it's on sebras/master now. | 18:03.58 |
| | tor8: I found this while attempting to reproduce your regression. (I didn't find any diffs when I tried locally) | 18:05.28 |
| tor8 | sebras: seems fine to me, but best run it through Robin_Watts too. | 18:08.32 |
| sebras | Robin_Watts: _4_ line patch on sebras/master | 18:11.02 |
| Robin_Watts | sebras: Looks plausible. | 18:12.09 |
| sebras | tor8: I clustered it, not differences. | 18:14.07 |
| leviathon | is there a way to add a scrollbar to mupdf | 21:38.32 |
| sebras | leviathon: are you talking about the linux x11/gl desktop viewer? | 22:14.47 |
| | or android? | 22:15.44 |
| | or ios? | 22:16.12 |
| | or windows? :) | 22:16.18 |
| leviathon | sebras: linux x11/gl | 22:20.00 |
| | sebras: arch linux | 22:20.13 |
| sebras | leviathon: ok. in mupdf-x11 there is no scrollbars at all. but in mupdf-gl there is one for the outline. I'm assuming you want one for the page view..? | 22:21.01 |
| | leviathon: it would probably require you to look a bit at the mupdf-gl code to sort it out though. | 22:21.22 |
| leviathon | sebras: yes i want it for page view | 22:21.44 |
| | sebras: i could git clone it | 22:22.20 |
| sebras | leviathon: in do_outline is the implementation of the outline, there is an example of a scrollbar there. | 22:23.10 |
| | leviathon: http://git.ghostscript.com/?p=mupdf.git;a=blob;f=platform/gl/gl-main.c;h=620348c26e9328aca6fcde09b24173c7e6c78bec;hb=HEAD#l559 | 22:23.13 |
| | leviathon: I haven't delved into the detail myself, so you'd need to play around with that for a bit. | 22:23.45 |
| malc_ | leviathon: what sebras is trying to say, i speculate here, is that there's no such functionality in x11/gl viewers, you'd have to code it yourself | 22:25.43 |
| leviathon | alright well thanks for the help | 22:26.34 |
| sebras | malc_: leviathon: well in mupdf-gl there is a scrollbar implemented, it is just not used for the page view, but for the outline. so you'd have to understand how it works and then implement it yourself. | 22:26.36 |
| leviathon | ok thank you | 22:26.54 |
| | Forward 1 day (to 2018/01/30)>>> | |
Log of #mupdf at irc.freenode.net.