| | <<<Back 1 day (to 2014/08/14) | 2014/08/15 |
| Montreseur | question | 03:55.42 |
| | is there a similar function to "control + F" do search through a pdf with mupdf? | 03:56.07 |
| henrys | rayjj: I just reported a bug for their stuff⦠ugh | 05:50.53 |
| rayjj | henrys: I just saw it. I looked at the files w/ gs as well as mupdf. As reported to them previously, mudraw -T only looks at the DeviceRGB and calls the page color. gs looks at the pixels | 05:53.01 |
| henrys | I was told mupdf looks at pixels | 05:53.39 |
| rayjj | henrys: I was going to leave opening a bug on the other things for Kens when he sees my reply. (openjpeg noise and mudraw calling pages gray that clearly aren't) | 05:54.03 |
| henrys | rayjj: kens told them that in the emalil | 05:54.18 |
| rayjj | henrys: no, I looked at the code when tor's patch first went in | 05:54.23 |
| | henrys: I think you misread it. gs looks at the pixels, mudraw -T doesn't | 05:54.54 |
| | mudraw -T also requires an exact match of RGB | 05:55.34 |
| | for non-image colorants | 05:55.49 |
| | gs has a tolerance | 05:56.05 |
| henrys | rayjj: how can it require an exact match of rgb and not look at the pixels? | 05:56.43 |
| rayjj | even so, on the 32Mix file, some pages look like monochrome, but gs classifies them as color (eg. page 1) | 05:56.44 |
| henrys | I donât really care the problem is both of us are working on this at the same time, not good use of time. | 05:57.18 |
| rayjj | mudraw requires an exact match on the current color when painting other stuff (text, fills, strokes) | 05:57.36 |
| henrys | rayjj: kens asked me to download the files and look at them. | 05:58.00 |
| | rayjj: did he ask you too? | 05:58.10 |
| rayjj | henrys: I agree. I'm going to bed and was going to leave it for kens. I told Kens this AM that if the file came in, I'd have a look at it | 05:58.26 |
| henrys | rayjj: no problem it was an education for me. | 05:59.00 |
| rayjj | that was early this AM (yesterday's log) | 05:59.02 |
| | at least with luratch, I think gs works OK. The "noise" from openjpeg (even with a release build) make it unusable for 50Mix | 06:00.23 |
| | g'nite, henrys | 06:00.58 |
| henrys | rayjj: see ya tomorrow | 06:01.07 |
| jogux | anyone fancying upvoting http://stackoverflow.com/questions/25323575/compiling-a-32bit-binary-that-uses-ssl-on-a-64bit-debian-wheezy-multi-arch-host (or telling me what I'm doing wrong, either's good :) ) | 08:57.58 |
| kens | I can upvote it, butI do't really understand the issue :-) | 08:59.13 |
| jogux | me either. everything seems right, but the linker's not playing ball :-( | 08:59.30 |
| | it's SOT related if you're wondering, would be really handy to be able to run it's tests on a 64bit host :( | 08:59.47 |
| | thanks for the upvote kens :) | 09:00.06 |
| kens | :-) | 09:00.10 |
| kens | remembers Robin sending an email that said Paul would have Smart Office finished by the time Robin was back from holiday.... | 09:01.57 |
| jogux | he's a funny guy that Robin | 09:02.30 |
| chrisl | jogux: did you try adding an explicit -L to the gcc command line? | 09:04.07 |
| jogux | yeah, still no good :( | 09:04.30 |
| | the system is definitely adding the right -L | 09:04.38 |
| | I *think* it may be the missing libssl-dev that's the issue... but I can't see how I solve that without breaking 64bit builds. | 09:05.01 |
| chrisl | Well, you definitely need the dev package | 09:05.26 |
| jogux | well I have libssl-dev:amd64, just not libssl-dev:i386 (and those two seems to conflict with each other) | 09:06.10 |
| chrisl | You need the right one for the platform for which you're building | 09:06.44 |
| | It does look like it's a bug in the -dev packages | 09:07.17 |
| jogux | urgh :( | 09:07.37 |
| kens | Well, I now can't reproduce Ray's problems with the corrupted path to the ICC manager :-( For me, using current code, it works with 32 bit release and debug builds on WIndows 7. Somehow I did manage to reprodsuce it, once, last night, but I can't now. | 09:07.52 |
| chrisl | jogux: http://askubuntu.com/questions/252168/can-i-install-libssl-devi386-on-x86-64-system-without-losing-important-packag | 09:08.00 |
| kens | I am going to revert the patch that breaks quit though. | 09:08.08 |
| jogux | chrisl: argh! :( | 09:08.23 |
| kens | Well, at least now you know it can't work :-) | 09:08.53 |
| jogux | kens: that only makes me feel marginally better :-) | 09:09.46 |
| chrisl | jogux: you *might* be able to just extract the required files from the .deb package, put them somewhere non-standard and add -I and -L parameters to find them | 09:09.49 |
| jogux | chrisl: thanks for finding that. | 09:10.05 |
| kens | At least you know its not you doing something dumb, that always makes *me* feel better :-) | 09:10.29 |
| chrisl | So, doing "dpkg-deb -x <package> <directory>" will simply extract the contents of "package" into "directory" | 09:11.55 |
| jogux | kens: :-) | 09:14.12 |
| | I *think* maybe it's just two missing symlinks. although the bug report says the headers are actually platform dependent and in the wrong location and hence the conflict I think. | 09:15.13 |
| kens | The report certainly claims hte headers are device dependent | 09:15.33 |
| chrisl | No, I think the libraries conflict: both packages want to put them in /usr/lib | 09:15.56 |
| jogux | seems like it's fixed in unstable over a year ago. | 09:19.25 |
| chrisl | Welcome to Debian ;-) | 09:19.46 |
| jogux | yep :( | 09:20.09 |
| chrisl | Like I said, grabbing the i386 dev .deb and extracting the files (rather than installing) should be workable | 09:20.52 |
| jogux | chrisl: yeah, sounds like it should, though I don't like buggering about with systems in that way really :-S | 09:23.19 |
| | hmm. the fix may already be released in ubuntu. | 09:23.32 |
| chrisl | jogux: you're not (necessarily) buggering about with the system | 09:24.21 |
| | kens: I don't understand your comment on Bug694160 | 09:26.13 |
| kens | Which part ? | 09:26.36 |
| chrisl | You said: "If I do 'systemdict /quit get exec' then it works" - the patch changes it to do exactly that | 09:27.07 |
| kens | Maybe so, but the patch doesn't work, and the command line does (or did). Given that I now can't reproduce Ray's problem let me check it. | 09:27.43 |
| | Yes, I'm correct, give it a try | 09:28.15 |
| | Just run up GS, then type quit at the GS> prompt | 09:28.29 |
| | When that fails, try systemdict /quit get exec, that'll work | 09:28.48 |
| chrisl | Yes, so the patch changes it so it does what you see working | 09:29.15 |
| kens | THe patch causes that behaviour. If I revert the patch then quit works, as I would expect | 09:29.38 |
| chrisl | This suggests this bug is unfixable | 09:30.47 |
| kens | Possibly, I have only just started looking at it | 09:31.04 |
| | But the patch can't be used as is. It breaks '-o', throwing an error, which also causes our scripts to break (because they all use -o) and if you don't use -o then you have to use 'systemdict /quit get exec' in order to leave the interpreter, instead of simply 'quit'. | 09:32.22 |
| chrisl | It *seems* we're relying on the definition of /quit in userdict, but it's the (re-)definition of /eq in userdict that causes the problem | 09:32.43 |
| kens | Why do we redefine /eq in userdict ? Better yet, why can't we find the definition of quit in systemdict ? | 09:33.40 |
| | OK there is no definition of quit in userdict when that patch is in place | 09:34.57 |
| | So the patch disables the redefinitions in userdict ? | 09:35.46 |
| chrisl | Yes | 09:36.06 |
| | It pulls values from systemdict directly | 09:36.29 |
| kens | Well the problem seems to be that systemdict isn't open then. | 09:36.34 |
| | Hmm, no its not that# | 09:36.55 |
| chrisl | It can't be that, or *no* operator would work | 09:37.12 |
| kens | Well if I do 'systemdict begin quit' then I still get an undefined in quit | 09:37.35 |
| | But systemdict /quit get exec works | 09:37.58 |
| kens | boggles | 09:38.07 |
| chrisl | I can only assume that in some configuration we don't define /quit in systemdict - which seems insane | 09:38.53 |
| kens | No quit is *defintely* defined in systemdict, because "systemdict /quit get exec" works, and that wouldn't work at all if it wasn't defined there | 09:39.25 |
| | But just "quit" doesn't work | 09:39.44 |
| | chrisl have you tried this ? | 09:39.51 |
| chrisl | tried what? | 09:39.57 |
| kens | open gs, type quit, see what happens | 09:40.10 |
| | I'd like to know its not just me..... | 09:40.21 |
| chrisl | I've tried the patch, obviously, as it's been in git | 09:41.03 |
| kens | Yes, and I've been running with it too, clearly, but I'd never noticed this behaviour | 09:41.30 |
| chrisl | What behaviour? | 09:41.48 |
| kens | Typing quit at the GS> prompt throwing an error. | 09:42.00 |
| | And it looks like the error isn't from quit at all. | 09:42.12 |
| | Its actually from the code in gs_main_finit | 09:42.22 |
| | THe quit there is the one throwing the error. | 09:42.38 |
| chrisl | Yes, I know that | 09:42.42 |
| kens | Hmm, well I didn't, I only just got that far | 09:43.01 |
| chrisl | Erm, you reverted the patch | 09:43.14 |
| kens | Yes, because it breaks everything for me, but I'm running at the moment with it back in to investigate it | 09:43.34 |
| jogux | ahha. just adding two symlinks for libssl.so / libcrypto.so seems to "fix" it. maybe that's not too evil. | 09:43.51 |
| chrisl | kens: Clearly doesn't "break everything" otherwise it wouldn't pass the cluster | 09:45.10 |
| kens | Well, break everything I meant breaks -o, which is pretty commonly used (in all our scripts for example) | 09:45.49 |
| chrisl | kens: so the Postscript redefines /eq in the top dictionary, and the redefinition includes a call to showpage which causes a crash because we no longer have a valid device | 09:46.29 |
| kens | OK weird, its not the /systemdict invocation. It seems I need to run quit twice, teh first time throws an error, the second time works. Now I'm very puzzled | 09:46.36 |
| | chrisl yes, I understood the rationale for the fix. | 09:46.54 |
| | But breaking top level quit doesn't seem like a solution we can live with | 09:47.33 |
| chrisl | I can't see how it can break "-o" since that's just shorthand for options the cluster uses | 09:47.45 |
| kens | I'm presuming -o includes a 'quit' | 09:48.01 |
| | But seriously, give it a try. | 09:48.13 |
| chrisl | I've been using it | 09:48.30 |
| kens | It does exit, but it throws an error 'undefined on quit' | 09:48.36 |
| | Surely this isn't Windows-specific ? | 09:48.53 |
| | Actually it can't be, the chap who pointed it out was using Linux IIRC | 09:49.12 |
| chrisl | Does the same problem arise if you use "-sOutputFile=... -dBATCH -dNOPAUSE"? | 09:50.04 |
| kens | Not sure, let me see | 09:50.13 |
| | I normally use the windows windowed executable, and therefore don't see the problem | 09:50.36 |
| | chrisl, yes still get undefined in quit | 09:51.07 |
| | Although as noted it *does* exit | 09:51.17 |
| | and it performs correctly, ie all files created etc | 09:51.29 |
| chrisl | It seems what we have in systemdict called /quit throws an undefined error - WHAT?? | 09:53.13 |
| kens | THat could be what I'm seeing | 09:53.24 |
| | And yes, WTF ? | 09:53.30 |
| | BTW It seems that (on WIndows at least) this is returning an error code of 0 on exit. | 09:53.55 |
| | So the cluster will work because there is no error reported to the OS< and the files are all correctly created. It just throws an error message | 09:54.27 |
| chrisl | That's not ideal | 09:54.55 |
| kens | It may not be the case on Linux (I may even be wrong on Windows) but it explains the observed results. | 09:55.26 |
| | On Windows %ERRORLEVEL% is 0 on exit, even though we threw an error. | 09:55.43 |
| chrisl | Oh FFS, both gs_init.ps *and* gs_lev2.ps have definitions of /quit..... | 09:56.11 |
| kens | So we have a definition of /quit in terms of .quit. Why ???? | 09:56.12 |
| kens | screams in horror | 09:56.19 |
| | More insane startup code | 09:56.38 |
| chrisl | So, quit is not an operator...... | 09:56.46 |
| kens | Hmm, there's a .quit as well. | 09:57.11 |
| chrisl | However /.quit is an operator | 09:57.48 |
| kens | Yes that's the one | 09:57.54 |
| | So if I do 'quit' at the prompt, it goes to zquit() | 09:58.29 |
| | Which (OMG) returns error e_Quit | 09:58.41 |
| | With a comment 'Interpreter will do the exit' | 09:59.01 |
| | Then we special csae e_Quit in gs_call_interpret() | 10:00.05 |
| chrisl | And, despite what the PLRM says, we don't seem to have a definition of /quit in userdict <sigh> | 10:01.39 |
| kens | OK so having exited because of quit, we then go back and run the cleanup code in gs_main_finit, which (of course) includes a quit. THat causes us to run zquit() again. | 10:04.17 |
| | Oh, and then we run it *again*...... | 10:05.07 |
| chrisl | Again?? | 10:05.36 |
| kens | In gs_main_finit, we run quit twice | 10:05.51 |
| | Once at line 904 | 10:06.06 |
| | And again line 924 | 10:06.18 |
| chrisl | Do you know which one's failing? | 10:06.53 |
| kens | Its the final one that throws an error | 10:06.58 |
| | I'll have to go round again now to see more. | 10:07.07 |
| | Ah, and because the 3rd one doesn't find and execute 'quit', we end up stuck in the interpreter. Unlike all the prevous ones, which do execute quit, and exit the interpreter. | 10:09.10 |
| | Of course, when I then execute quit (any quit, notice), then it exits the interpreter, and goes back to line 924 and carries on..... | 10:10.09 |
| | WTF ? Even if I remove the 'quit' from the PostScript in line 924, it still gives me an 'undefined' in quit. Huh ??? | 10:13.08 |
| kens | guesses the actual quit causing the problem is in fact somewhere else. Oh god this is a mess | 10:14.18 |
| chrisl | kens: I think it's a simple typo: if you add spaces at the ends of the concatenated strings, it works | 10:16.33 |
| kens | Seriously ? | 10:16.47 |
| chrisl | I'm messing with it now | 10:17.00 |
| kens | Oh, I see the other lines have that. | 10:17.01 |
| chrisl | But obviously, I'd appreciate you confirming it! | 10:17.56 |
| kens | Indeed, with that in place zquit gets called | 10:17.58 |
| | Give me a minute | 10:18.04 |
| | Works perfectly. What a bizarre bug | 10:18.41 |
| chrisl | Also, I think there's a spurious call to .uninstallpagedevice in there, too | 10:19.13 |
| kens | I was wondering about that, but it doesn'[t seem to cause a problem | 10:19.27 |
| chrisl | I *suspect* that by that time the error handler is gone, and we're getting complete nonsense in the error message | 10:19.47 |
| kens | Also if PSI_INCLUDED isn't defined we won't have previously called it | 10:19.50 |
| | chrisl I htink you may be correct, certainly the error message for pdfwrite was barking | 10:20.05 |
| | I thikn you're right about uninstallpagedevice though, tht looks like it isn't needed | 10:20.57 |
| chrisl | I *thought* for PSI_INCLUDED we shouldn't call .uninstallpagedevice since the default device is created independently of Ghostscript | 10:21.16 |
| kens | All we're doing here is flushing stdout and stderr, and teh quit is there to exit the interpreter when completed | 10:21.23 |
| | chrisl Yes, we're in agreement there, the .uninstallpagedevice shouldn't be there | 10:21.47 |
| | And yes, may even be harmful | 10:22.07 |
| | So do you want to commit the corrected version ? | 10:22.33 |
| chrisl | kens: sure. But what do you think about changing it so it doesn't rely on the automatic string concatting? | 10:24.54 |
| kens | I'd be happy with that too, thar's very confusing syntax, its easy to miss the space | 10:25.16 |
| chrisl | OKay, I will make that change, too | 10:25.30 |
| kens | Thanks, and well spotted! | 10:25.41 |
| chrisl | More luck than anything else..... | 10:26.27 |
| kens | Well you probably just saved me a couple of hours, I was approaching it by reducing the POostScript it was executing there. It would heva been a while before I realised what was going on | 10:27.08 |
| hyper_ch | hi there, I use ghostscript to manipulate PDF files... like combining them and making an index. However I face a strange issue and I'm not sure what causes it. First I let couple of PDFs run through a script that makes a numbered stamp and stores every single page as seperate file. After that I combine them. When I look at the single pages the numbering is all fine. But once combined, every 14th pdf has issues | 11:03.13 |
| | is there a problem using when *pdf includes 14 or more pdfs? gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile="/tmp/test.pdf" *pdf | 11:04.36 |
| chrisl | hyper_ch: the first thing you need to understand is that you are not "manipulating PDF files", Ghostscript is interpreting PDFs into an internal representation and then creating brand new PDFs from that. | 11:09.41 |
| | Secondly, what version are you using? | 11:10.02 |
| hyper_ch | 9.10 | 11:10.17 |
| chrisl | I would urge you to try 9.14 which is the latest | 11:10.36 |
| hyper_ch | chrisl: could you test if you get the same result? | 11:10.55 |
| chrisl | hyper_ch: I'd have to see your files | 11:11.33 |
| hyper_ch | I can give you the single pdf files | 11:11.36 |
| kens | chrisl Ian just mailed me this link: | 13:17.55 |
| | http://valleywag.gawker.com/lawsuit-claims-google-wrote-down-plan-to-steal-idea-on-1621674163 | 13:17.55 |
| | Ooops :-) | 13:17.59 |
| chrisl | kens: couldn't happen to a nicer company :-) | 13:21.23 |
| kens | "Do no evil"...... | 13:21.37 |
| chrisl | "Be seen to do no evil....." | 13:22.01 |
| pedro_mac | âDestroy all evidence of evilâ | 13:24.19 |
| jogux | hehe | 13:25.09 |
| chrisl | The post-it guy clearly didn't get that memo..... | 13:25.27 |
| pedro_mac | thereâs another post-it saying âDestroy the post-itsâ | 13:27.36 |
| nemo | kens: so, I was following: http://bugs.ghostscript.com/show_bug.cgi?id=695420 and fwiw, that revision seems fine for me too. | 13:29.00 |
| kens | nemo, you no longer get the corrupted message ? | 13:29.17 |
| nemo | looks like | 13:29.22 |
| | running | 13:29.29 |
| | git/ghostpdl/gs$ bin/gs -o x.pdf -sDEVICE=pdfwrite -dPDFSETTINGS=/ebook -c "<< /ColorImageDict << /QFactor 0.95 /Blend 1 /HSamples [2 1 1 2] /VSamples [2 1 1 2] >> /AutoFilterColorImages false /ColorImageFilter /DCTEncode >> setdistillerparams { .93 div dup 1 gt { pop 1 } if } settransfer" -f /tmp/before.pdf | 13:29.30 |
| | which was the same thing I was using in the bisect | 13:29.36 |
| kens | OK then I guess that's fixed, but I'll leave it to Ray to close it, since I re-assigned it to him | 13:29.43 |
| nemo | aight | 13:29.47 |
| | just figured I'd mention | 13:29.51 |
| kens | Good to know, thank you | 13:29.59 |
| nemo | 'course, I already munged all the PDFs here, so.. | 13:30.00 |
| | n/p | 13:30.02 |
| | seeya | 13:30.05 |
| Kenneth | Hi guys | 13:30.46 |
| | I have a question | 13:30.49 |
| Guest2790 | I am using Linux, and have already made a build of MuPDF. Now I try to compile the example.c in docs, but I get this massive "undefined refenrence" respons. | 13:31.42 |
| Robin_Watts | Drop it into a pastebin | 13:32.02 |
| kens | Which version of MuPDF did you get, did you update teh thirdparty submodules ? | 13:32.14 |
| Guest2790 | The new one yes. All taken from git 15 min ago. I use this command: gcc -g -o build/debug/example -Iinclude docs/example.c build/debug/libmupdf.a build/debug/libfreetype.a build/debug/libjbig2dec.a build/debug/libjpeg.a build/debug/libopenjpeg.a build/debug/libz.a -lm | 13:33.10 |
| | and I get this: | 13:33.19 |
| kens | Not here, pastebin please | 13:33.28 |
| Guest2790 | ya 2 sec | 13:33.43 |
| kens | NP | 13:33.46 |
| Guest2790 | http://pastebin.com/aJBiWcFa | 13:34.22 |
| jogux | add -lssl -lcrypto I think | 13:35.13 |
| Guest2790 | at the end? | 13:35.24 |
| jogux | yeah | 13:35.28 |
| | and one more, the js library - not sure what that's called | 13:35.37 |
| kens | isn't sure that'll do anything for the js undefines | 13:35.42 |
| | Aren't we using Tor's JS engine now ? | 13:36.01 |
| | mujs | 13:36.14 |
| jogux | yeah, add build/debug/libmusjs.a too I reckon | 13:36.25 |
| Guest2790 | ok | 13:36.32 |
| | 2 sec | 13:36.33 |
| jogux | er, I typoed there | 13:37.05 |
| | debug/debug/libmujs.a | 13:37.11 |
| Guest2790 | ok 2 sec | 13:37.31 |
| jogux | okay, I'm not doing well. give me one more go. | 13:37.47 |
| | build/debug/libmujs.a :-) | 13:37.59 |
| | why I didn't C&P that I have no idea | 13:38.11 |
| tor8 | Guest2790: use gcc -o build/debug/example -Iinclude docs/example.c build/debug/*.a build/debug/*.a -lm -lcrypto | 13:38.48 |
| | (yes, that's build/debug/*.a twice, fixes the dependency order if you're lazy) | 13:39.17 |
| jogux | :-) | 13:39.26 |
| | tor8: don't you need a -lssl in there too? | 13:39.44 |
| tor8 | jogux: openssl is -lcrypto, IIRC | 13:39.52 |
| jogux | yeah. ignore me :) | 13:40.05 |
| Guest2790 | its working | 13:40.37 |
| | yaaaaa | 13:40.40 |
| tor8 | but yes, from the pastie you're missing libmujs.a (I ought to update the comment in docs/example.c) and openssl | 13:40.41 |
| henrys | kens, chrisl, ray (for the logs) and ps language folks: I donât see a nice solution to http://bugs.ghostscript.com/show_bug.cgi?id=695387 the image is setup with the 4 bit per component long before the filter code digs into the jpx header for the true bpc. I guess the image has to be torn down and rebuilt when the true bpc is read but that doesnât look pleasant at all. | 13:40.52 |
| kens | henrys it was horrible last time I had to implement this. PostScript interpreters always assume the image dict is gospel, and until JPX PDF consumers did too. | 13:41.39 |
| | Adding the ability for JPX to override the PDF dictioanry was totally insane | 13:42.02 |
| chrisl | I think we already do special stuff to get information from JPX, so we probably need to extend/augment that | 13:42.25 |
| tor8 | kens: yeah. JPX overriding the PDF dictionary is the only idea that's possibly more stupid than adding transparency to PDF... | 13:42.32 |
| Guest2790 | But guys, in the example.c then please include the other .a file | 13:43.04 |
| | it is missing | 13:43.06 |
| | so people cant compile it | 13:43.11 |
| | It should look something like: | 13:43.22 |
| | gcc -g -o build/debug/example -Iinclude docs/example.c build/debug/libmupdf.a build/debug/libfreetype.a build/debug/libjbig2dec.a build/debug/libjpeg.a build/debug/libopenjpeg.a build/debug/libz.a build/debug/libmujs.a -lm -lssl -lcrypto | 13:43.23 |
| kens | we must already get the colour space out of the JPX file, is htere a reason we can't do BPC at the same time ? | 13:43.28 |
| chrisl | I just said that..... | 13:43.39 |
| Guest2790 | Cya | 13:43.43 |
| kens | sorry, missed it chrisl | 13:43.47 |
| chrisl | *or* we could promote the BPC in the filter code | 13:44.01 |
| | But that's icky | 13:44.12 |
| kens | Well its all pretty horrible | 13:44.20 |
| chrisl | But there is a fundamental issue which is that JPX supports bit depths that PS and PDF don't | 13:44.55 |
| henrys | chrisl: thatâs not in play for this bug though | 13:45.17 |
| kens | Yeah, but I thought the spec said something about not supporting all of JPX> Only baseline or something like that | 13:45.19 |
| kens | opens the PDFRM | 13:45.34 |
| chrisl | henrys: so in this case the dictionary says 4bpc but the image data is, what 8? | 13:46.06 |
| henrys | chrisl: yes | 13:46.16 |
| chrisl | Okay, so we really don't want to reduce it | 13:46.32 |
| kens | Page 87 says XObjects 'should' be limited to the JPX basleine (there's that wonderful 'should' in the spec again) | 13:47.02 |
| henrys | chrisl: you can change the 4 to an 8 in the pdf and it looks dandy, but most customers donât like that approach | 13:47.06 |
| chrisl | Oh, we already parse a BPC out of the JPX stream | 13:48.02 |
| kens | And we don't use it ? O.O | 13:48.12 |
| henrys | chrisl: we do use it to read the data - see the comment | 13:48.24 |
| | ^^^ sorry kens not chrisl | 13:49.20 |
| kens | Hmm, I don't see anywhere in the spec that says we should use the BPC from the JPX, but I may just be missing it. | 13:49.30 |
| henrys | kens: did you read the BPC part ? ;-) | 13:50.05 |
| kens | In the PDF spec ? It mentions that they can be variable and have values 2->38 | 13:50.27 |
| | I haven't cracked open the refernce to see exactly what JPX baseline says yet | 13:50.43 |
| chrisl | So, we already get the BPC (before completing the image dict), but we don't currently use it in the dictionary | 13:50.45 |
| kens | SO that sounds like an easy fix | 13:50.53 |
| henrys | BitsPerComponent paragraph 3 | 13:51.02 |
| kens | We should probably also test the BPC and throw errors on ones we don't like | 13:51.11 |
| henrys | table 4.39 | 13:51.11 |
| kens | henrys, JPX spec or PDFRM ? | 13:51.34 |
| chrisl | kens: we that should happen automatically in the PS interpreter | 13:51.35 |
| henrys | chrisl: pdf ref 1.7 | 13:51.44 |
| kens | Hmm, yes fair enough | 13:51.50 |
| | henrsy if that was to me, table 4.29 is entries in a type 1 sahding dictionary | 13:52.19 |
| henrys | sorry not chrisl but kens Iâm mixing you up this morning just got up | 13:52.21 |
| kens | Oh 4.39 wait | 13:52.34 |
| henrys | yes 4.39 sorry page 340 | 13:52.53 |
| kens | No, I misread it. OK so we should always ignore BPC. From what chrisl says that looks not too bad, since we already extract it | 13:53.19 |
| henrys | kens: how you are going to open the filter early? | 13:53.54 |
| chrisl | get_jp2_csp - % Parse jp2 file format to get color space and image depth info. | 13:54.22 |
| | in pdf_draw.ps | 13:54.29 |
| kens | henrys, I don't know, I'm just looking at the PS now. But this can only occur in PDF files, therefore we can open the image twice if required | 13:54.35 |
| | chrisl so we do extract both, but only use the ColorSpace, seems odd. | 13:56.00 |
| henrys | kens: thatâs what didnât occur to me. I was thinking of the streaming in PS and thought oh no ⦠| 13:56.04 |
| kens | It wouldn't be possible in PostScript, sure. | 13:56.29 |
| chrisl | We don't use the filter: if I'm reading this right, we parse this stuff in Postscript - ick :-( | 13:56.47 |
| kens | Oooh, well I guess there's not much to grab out | 13:57.05 |
| chrisl | And we have a second procedure called /getu16 just for fun..... | 13:57.28 |
| kens | What, in addition to the one for TrueType ? :-) | 13:57.46 |
| | I'm sure there are probably more :-) | 13:57.53 |
| henrys | as chrisl said though we handle the ColorSpace right? | 13:57.56 |
| chrisl | Yes, and /getu32 | 13:57.58 |
| Robin_Watts | I have a horrible cold, and a headache. I suspect I am going to get very little done on SOT today. | 13:58.01 |
| kens | henrys, yes we do | 13:58.02 |
| henrys | Robin_Watts: are you home already? | 13:58.16 |
| kens | We just don't seem to use the BPC we get out, we just ignore it | 13:58.16 |
| Robin_Watts | henrys: Is it worth me making the test-device in mupdf respond to images properly? | 13:58.20 |
| | henrys: Yes, landed yesterday morning. | 13:58.26 |
| kens | Robin_Watts : I think that's going to be a requirement for that 'customer' if they are going to use MuPDF | 13:58.46 |
| Robin_Watts | It's pretty easy to do. | 13:58.56 |
| henrys | Robin_Watts: for gray - no Iâd rather push back, these yahoos have too many engineering hours | 13:58.59 |
| kens | I thought it woujld be | 13:59.02 |
| Robin_Watts | and I can do it without my head exploding. | 13:59.06 |
| chrisl | kens: if you're involved with something else, I can take a punt at this JPX stuff if you like | 13:59.26 |
| kens | chrisl, I'll take a quick peek, I'm just grabbing teh file | 13:59.43 |
| | I was looking at shading dictionaries with 'strange' results | 13:59.54 |
| Robin_Watts | ok, I shall keep bashing the SOT rocks together then. | 13:59.55 |
| henrys | kens: itâs a low priority enhancement now. Letâs see what they say | 13:59.56 |
| kens | the MuPDF thing ? I'm not worried either way | 14:00.19 |
| Robin_Watts | So, in other news, my eye cleared up within the first two days of the holiday. | 14:00.38 |
| henrys | Robin_Watts: I wish youâd take time off and get better⦠but | 14:00.45 |
| kens | D'oh! I already have this PDF file..... | 14:00.46 |
| chrisl | Robin_Watts: any thoughts on these downloads problems with GoDaddy? | 14:00.53 |
| Robin_Watts | And the NHS just gave me an appointment so they can do some tests. In October. | 14:00.59 |
| kens | Helpful :-) | 14:01.08 |
| Robin_Watts | chrisl: Other than shouting at them? | 14:01.12 |
| kens | Finding another provider would seem best | 14:01.24 |
| | But that could be tricky I guess | 14:01.31 |
| Robin_Watts | kens: Finding one that doesn't charge us a fortune for bandwidth, yes. | 14:01.52 |
| kens | Yeah that's what I meant | 14:02.00 |
| Robin_Watts | chrisl: I can phone them if you want. | 14:02.10 |
| | I sound like the voice over guy from movie trailers at the moment, so I might scare them. | 14:02.33 |
| henrys | chrisl: I wouldnât pull yourself off of the directory stuff unless you think you need a break. | 14:02.52 |
| chrisl | Robin_Watts: my problem is having something solid to report to them, because now, I can't reproduce a problem | 14:02.55 |
| Robin_Watts | ah, | 14:03.03 |
| chrisl | henrys: the directory stuff has been stopping and starting regularly as support work comes i | 14:03.48 |
| | n | 14:03.50 |
| Robin_Watts | no, it looks to be working for me. | 14:03.53 |
| chrisl | Robin_Watts: I had a script I ran for 4 days from the day after I called GoDaddy which downloaded a large file from the site, and checked the MD5 every 5 minutes - it never failed | 14:04.40 |
| henrys | chrisl, Robin_Watts do we have stats for number of downloads? | 14:04.57 |
| chrisl | Where I was able to see about 1 in 5 downloads fail before | 14:05.06 |
| | henrys: I thought we did, but marcosw said not.... | 14:05.20 |
| henrys | chrisl: so reassign the jpx to you? | 14:05.22 |
| jogux | chrisl : robin_watts : oh, that reminds me - http://www.xilo.net/shared_web_hosting/ claim to have unlimited bankwidth web hosting | 14:05.35 |
| chrisl | henrys: kens' going to take a look first | 14:05.37 |
| jogux | they have a good rep too | 14:05.41 |
| kens | I'm looking at it now. | 14:05.50 |
| chrisl | If we're going to change hosting, should we not punt it to Ron? | 14:06.23 |
| henrys | I got up out of bed last night remembering kens said to look at the hcl files. I fooled with that for a while and responded to the customer only to find ray was doing the same thing, our customer emails went out at the same time. | 14:08.27 |
| kens | henrys, I said "If you want to" :-) | 14:08.42 |
| henrys | kens: rayjj had a better understanding of the issue I didnât realize we didinât look at pixels for images in mupdf. | 14:10.36 |
| kens | henrys no, MuPDF doesn't do that. Ghostscript does | 14:10.51 |
| | GS also has some 'slack' whcih is, I think, definable, so that 'nearly gray' can be regarded as gray | 14:11.17 |
| | I'm sure its not hrd to add both if required to MuPDF | 14:11.38 |
| henrys | kens: why was generating output and looking at a histogram with another tool not considered? too slow? | 14:12.02 |
| kens | henrys, by us or by the customer ? | 14:12.17 |
| henrys | s/was/wasn't | 14:12.21 |
| | kens: was there a reason we did not suggest that to the customer? | 14:12.59 |
| kens | I have no idea. But it would probably be slower | 14:13.42 |
| | And not very amenable to automation | 14:13.56 |
| | Usually people want these things so they can charge differently for coloured/monochrome pages | 14:14.16 |
| | So picking up the gray/colour status while rendering is a win | 14:14.38 |
| | why this particular 'customer' want the information I have no idea. | 14:15.04 |
| | Hmm, it seems that if the image dicitonary has a ColorSpace and a BPC then we don't bother to check the actual JPX image. Presumably because its slow. Whereas in fact (according to the spec as Henry just pointed out) we should ignore teh BPC in the image dictionary, if present, and always use the one from the JPX file. | 14:17.47 |
| rayjj | henrys: generating output and downscaling will "collapse" pixels, and (at least with gs) may skip color pixels -- generating output at really high res is too expensive | 14:17.54 |
| kens | henrys, if I force teh code through the BPC detection then the output looks fine. Seems like its a simple fix, but it will hit performance with JPX files a little. I don't think we have a choice though. | 14:19.28 |
| rayjj | gs looks at the source image data, even if it's higher res than the output (which is why the technique to do the fast check args.gs I used does -r18 | 14:19.29 |
| | kens: is the JPX issue why gs called some pages wrong? | 14:20.44 |
| kens | rayjj, no its a reported bug, we aren't using the BPC from the JOX file, we're using the one from the image dictionary, which is incorrect. The spec says we should ignore that and *always* use the info from the JPX file (JPX is just crazy in the spec) | 14:21.39 |
| rayjj | kens: I agree that having the image data "change" the colorspace AND the BPC, etc. goes against having the parameters in the PDF image dict | 14:22.34 |
| | seems crazy | 14:22.42 |
| kens | It does indeed. BUt its the way its specified :-( | 14:22.53 |
| henrys | kens: oh great news. I thought this was going to be a pain in the arse | 14:23.20 |
| kens | No, its pretty easy, its just going to cost us a tiny bit in performance to scan the data. At the moment we skip it if we think we can get away with it. Sadly, we are wrong | 14:23.53 |
| chrisl | Good grief, henrys is translating freely! | 14:24.02 |
| rayjj | kens: if it was up to me, the spec would have required the parameters in the image dict to match, if present, and make them optional for JPX | 14:24.03 |
| kens | But given the performance of JPX decoding, I'm not inclined to worry | 14:24.09 |
| henrys | chrisl: I have brit friends here and you guys, itâs too much | 14:24.54 |
| kens | rayjj that is the case for colour spce, but BPC is the other way around, we are supposed to *always* use the JPX data | 14:24.59 |
| | We'll get him spelling colour properly soon too :-) | 14:25.23 |
| chrisl | :-) | 14:25.37 |
| rayjj | kens: thanks for reverting the patch. I'll test in a bit. | 14:29.10 |
| kens | rayjj nemo popped in and said it was OK for him now (with the latest code which chrisl added to put the patch back in with corrections) but I figured I'd let you tets it too and close when you're happy with it. | 14:30.04 |
| rayjj | kens: it still failed for me with a release build (not the unrecoverable error -- just the gsicc_ messages). Firing up a debugger with the same command line... | 14:40.52 |
| kens | rayjj I can't reproduce that now at all. | 14:41.21 |
| | The error shold be fixed, that was done in chrisl's commit | 14:41.37 |
| | But with that in place I can't get the corrupted string | 14:41.51 |
| henrys | rayjj, kens we payed shelly for 694160, not a big deal, but if you want me to ask him to look again or you guys want to talk to him go ahead. It seems like a bad bug for someone outside the staff. | 14:47.03 |
| | to work on | 14:47.16 |
| | s/payed/paid | 14:47.59 |
| rayjj | henrys: agreed -- It seems pretty hairy to have kens and chrisl pulling hair out | 14:48.08 |
| kens | I think chris and I looked at it and didn't see a problem. It was only hard to diagnose the problem because teh error handler is basically kaput at that point. We're closing down the interpreter (multiple times!) | 14:48.41 |
| | The way that the code is written with the strings on separate lines and having the compiler automagically concatenate them fools you into thinking that they are separate arguments, so its easy to miss the trailing space. | 14:49.51 |
| | Which is why Chris rewrote it so that we use the string continuation '\' instead. Its *much* more obvious what's going on. | 14:50.18 |
| | There's nothgin fundamentally wrong with the patch, just a typo or two | 14:50.44 |
| rayjj | kens: with the debugger, after the page is done, I get a call to putdeviceparams, and in gx_default_put_params the string being read from the plist for OutputICCProfile is coming back with the icc_pre.data contents 0xeefeeefe ... | 14:50.50 |
| chrisl | Yeh, the fix is actually correct, there was just a typo or oversight in the final commit | 14:50.53 |
| kens | rayjj, well like I say, it doesn't happen for me. | 14:51.10 |
| | So its kind of hard to fix it. | 14:51.27 |
| rayjj | kens: now, the strange thing is the ee fe ee fe ... is what the debugger shows in gx_default_put_params, but down in gx_default_put_icc the memcpy has filled in 'curv' -- screwy | 14:54.47 |
| kens | Wow, that JPX change caused a number of diffs :-( | 14:55.49 |
| chrisl | rayjj: did you try it with -Z@? | 14:57.56 |
| rayjj | disregard the ee fe ee fe -- must have been a debugger issue -- now I see 'curv' up in gx_default_put_params before calling gx_default_put_icc | 14:57.56 |
| | chrisl: not yet. Thanks for the reminder. I'll try that now. | 14:58.27 |
| kens | rayjj there's not a lot I can say, I don't see the problem, so its not really possible for me to work on it | 14:58.27 |
| rayjj | kens: np. It's very reproducible for me | 14:58.40 |
| kens | Its possible that some memory is being freed, but the device parameter code is pretty crazy. | 14:59.04 |
| | I'd have to guess its something to do with memory layout. | 15:00.04 |
| rayjj | chrisl: interesting -- no change | 15:00.52 |
| chrisl | Huh. Maybe memory corruption, then? memento might help | 15:01.18 |
| kens | I would suspect memory corruption anyway. Or possibly reading from memory after its freed | 15:01.44 |
| rayjj | I recall that the storage of the icc profile names was problematic for mvrhel to get working. | 15:02.01 |
| kens | :-( | 15:02.13 |
| rayjj | kens: well, reading after free will be affected (usually) by -Z@$? | 15:02.33 |
| | but reading a stale pointer after a GC reloc will point to the wrong place in a chunk | 15:03.18 |
| | OK, so somehow I am getting into zputdeviceparams with a "counttomark" of 369 ! Before running the job, the highest I ever saw was 130 (sort of expected during setup because we get all the params, then put them all) | 15:11.52 |
| | and otherwise I saw a count of 3 and a count of 15 | 15:12.19 |
| | so a PS stack issue ? Adding some PS debug next (pstack) before the putdeviceparams | 15:13.16 |
| | but first -- coffee | 15:13.27 |
| zeniko | tor8: Robin_Watts: (or FTL) there are 5 new commits on zeniko/MuPDF for review when you get some time to spare | 15:36.23 |
| Robin_Watts | hi zeniko. | 15:36.43 |
| | I'm currently neck deep in SOT, so I'll have to leave that for Tor. | 15:37.15 |
| | I'll try to remember to mention it on monday when he appears. | 15:37.24 |
| zeniko | Robin_Watts: thanks | 15:37.55 |
| | these changes aren't urgent (the overflow one should go into the next release, though) | 15:38.23 |
| rayjj | making progress on that strange condition. With /ColorConversionStrategy /Gray in a distillerparam dict, it rc_decrements the icc_struct which frees the entire profile array. Interestingly, the device_profile[0].namelen is 15 which is what I see after the page is done | 17:27.14 |
| | definitely a stale pointer. The memory that contained the name for the string "default_rgb.icc" that was freed contains the string "curv" | 17:44.24 |
| | mow I have to track down the stale pointer (probably a missing rc_increment somewhere when the device is copied) | 17:45.06 |
| | s/mow/now/ | 17:45.40 |
| | but how that gets into the paramlist is a mystery | 17:46.36 |
| Robin_Watts | rayjj: tried memento? | 17:54.43 |
| rayjj | Robin_Watts: memento won't (AFAIK) tell me where stale pointers are | 18:24.19 |
| | Robin_Watts: I was thinking about adding temp code to the GC trace (putting a conditional bp is too painfully slow) | 18:25.21 |
| | Robin_Watts: I'm going to let that go and continue on the company C project -- it's a lot more fun :-) | 18:25.56 |
| mvrhel_laptop | rayjj: sounds like it | 18:36.30 |
| | sort of like SOT | 18:36.40 |
| | brb | 19:22.40 |
| rayjj | It looks like I can subvert one of the company C reference_code apps to run gs or mudraw as the RIP | 23:09.32 |
| rayjj | hope mentioning "subvert" {twice} doesn't get me on the NSA list ;-) | 23:10.11 |
| rayjj | resolves to use "adapt" rather than THE OTHER WORD I DON'T WANT TO MENTION AGAIN ;-) | 23:11.01 |
| | or N S A | 23:12.57 |
| | if their software is any good at all, I haven't fooled them. But, then, it _is_ a government agency. Look at the Health Insurance registration ;-) | 23:14.21 |
| | I'll know next time I fly whether or not I get pre-check | 23:14.56 |
| | Forward 1 day (to 2014/08/16)>>> | |
IRC Logs