| | <<<Back 1 day (to 2017/08/01) | 20170802 |
| knielsen | tor8: I implemented X11 primary selection in GLFW, as you speculated it was straight-forward | 14:29.44 |
| | tor8: The patch against GLFW is here: https://github.com/knielsen/glfw/commit/3a6608e8ed6bd704e6c886843c8accd19d69da29 | 14:30.08 |
| | tor8: and the patch to use it in mupdf-gl: https://github.com/knielsen/mupdf/commit/1df5667b3ae9e93232dfdba193d4704fcc271a89 | 14:30.27 |
| | tor8: in the glfw patch I implement both get and set, and stubs for other platforms, docs etc. But if this is included as a modification to thirdparty/glfw, much of that could probably be ripped out, just keep the glfwSetSelectionString() that is needed | 14:40.50 |
| tor8 | knielsen: it would be nice if GLFW upstream could take on the patch | 14:52.17 |
| knielsen | tor8: yep. I added the patch to the issue to ask: https://github.com/glfw/glfw/issues/894 | 14:53.08 |
| sebras | tor8: opinions on the first three patches on sebras/master? | 15:01.40 |
| tor8 | sebras: check the logs :) | 15:02.18 |
| sebras | rechecks! | 15:02.27 |
| | tor8: oh. I missed it, sorry. :) | 15:02.59 |
| mvrhel_laptop | Morning Robin_Watts | 15:14.46 |
| | aha. found the source of the problem on this altona issuse | 15:22.31 |
| | Robin_Watts: you around? | 15:25.13 |
| Robin_Watts | morning. | 15:25.26 |
| | yeah, I found my problem too. | 15:25.35 |
| | Just updating lcms to 2.9 now. | 15:25.45 |
| sebras | tor8: no regressions I'll push these commits then. | 15:33.49 |
| | tor8: there are still a few warnings that show up when building largefile=yes because the printf-format doesn't match the 64-bit arguments | 15:37.51 |
| Robin_Watts | mvrhel_laptop: So, do you have a fix for me to pull in? | 15:40.27 |
| sebras | tor8: also if I change the proof of concept pdf so that the trailer has /Size 2147483647 and the xref section says 2147483612 35 then mupdf will allocate a 2G * sizeof (int) array in xref_index where each index seems to indicate in what subsection each object can be found. | 15:42.53 |
| | even for objects that are not allocated. | 15:43.03 |
| | I'm seeing an allocation of around 9Gbyte all of which are zeros (strangely the _virtual_ allocation is 20+Tbyte!). | 15:44.01 |
| mvrhel_laptop | Robin_Watts: so question on image decoding | 15:48.19 |
| | Page 4 of the altona test suite has some Lab images | 15:48.53 |
| | I always get myself turned around a bit with regard to if these images should be "decoded" | 15:50.04 |
| | If we are going through the icc logic they should not | 15:50.22 |
| Robin_Watts | By decoded you mean "have the Decode array applied" | 15:50.47 |
| mvrhel_laptop | yes | 15:50.52 |
| | sorry | 15:50.53 |
| Robin_Watts | I was not aware of any difference there. | 15:51.04 |
| mvrhel_laptop | I am a little surprised too as I thought there were FTS tests that we were passing | 15:51.39 |
| | maybe I should grab one of those files and check this more closelyl | 15:51.53 |
| | and maybe I should read the spec there again. | 15:52.48 |
| | oh baby deer jumping in the back yard | 15:53.01 |
| Robin_Watts | OK, I've updated robin/spots will all the latest shininess - yours, mine and martis. | 15:53.37 |
| | If you can point me at an image decode problem, I'll happily look. | 15:53.57 |
| | I was about to look at the image problems on page 9 of that file. | 15:54.05 |
| mvrhel_laptop | ha she just ran back and forth across the yard about 10 times | 15:54.21 |
| | Robin_Watts: ok. let me hand this simple file to you | 15:54.41 |
| | It is a cut down version | 15:54.47 |
| Robin_Watts | I have no deer. I merely have a pint sized marauder. | 15:54.51 |
| mvrhel_laptop | I filled the image with constant values of D0 00 D0 which is a green color | 15:56.20 |
| Robin_Watts | No sign of the email yet. | 16:01.40 |
| mvrhel_laptop | sorry | 16:03.20 |
| | had to video deer for kids as they are still sleeping | 16:03.32 |
| | mamma showed up and gave baby a bath in the back yard | 16:03.42 |
| | but back to work | 16:03.48 |
| | Robin_Watts: ok email sent | 16:04.33 |
| | Robin_Watts: so if I skip the array decode I get the correct result | 16:05.43 |
| Robin_Watts | The problem being the hard edge on the triangle ? | 16:07.21 |
| mvrhel_laptop | The triangle is an image fill | 16:08.16 |
| | the image is pushed through a luminosity blend with the gray backdrop | 16:08.39 |
| | the image is actually green. and I thought the issue was in the blending logic but that is all correct. The issue is the source image values are wrong. When I looked at the image data values prior to decode those are correct | 16:09.42 |
| Robin_Watts | ok, finally up to speed. | 16:27.50 |
| | so the image gives a /Decode array of [0 100 -128 127 -128 127] | 16:28.11 |
| | which is the default for LAB. | 16:28.17 |
| | The question is, should we completely ignore that Decode array if we have an ICC space, or should we make allowances for it? | 16:29.02 |
| | i.e. if I set that to be [0 200 -128 127 -128 127], and run it through Acrobat, would I expect to see the image change? | 16:29.30 |
| | Right using 10 as the second value DOES make a difference to acrobat. | 16:30.18 |
| | so ignoring the Decode array is not correct. | 16:30.26 |
| | mvrhel_laptop: ^ | 16:30.29 |
| | I suspect what we need to do is to bend the decode array if it's an icc space to allow for the fact that [0 100 -128 127 -128 127] is default. | 16:33.01 |
| | i.e. compose the "inverse" of [0 100 -128 127 -128 127] onto the given array. | 16:33.47 |
| | Does that make sense? | 16:33.51 |
| | (if it's LAB and icc, that is) | 16:34.10 |
| mvrhel_laptop | ok back | 16:34.21 |
| | sorry | 16:34.23 |
| | Robin_Watts: yes. that makes sense | 16:35.04 |
| Robin_Watts | so I need to figure out how to compose the inverse of a decode array... | 16:35.33 |
| mvrhel_laptop | Robin_Watts: if you want me to fool with that I can | 16:36.03 |
| Robin_Watts | mvrhel_laptop: Hmm. fz_colorspace_is_lab detects to_ccs being lab_to_rgb | 16:37.20 |
| | which it will not be for icc spaces. | 16:37.35 |
| | I think we need a bit in our colorspace flags word for "is_lab" | 16:38.45 |
| | and maybe another for "is_icc" | 16:39.10 |
| | oh, we have is_lab_icc :) | 16:39.35 |
| mvrhel_laptop | ah | 16:40.20 |
| | brb | 16:41.54 |
| Robin_Watts | mvrhel_laptop: Yell when you're bacl. | 17:03.31 |
| mvrhel_laptop | I am back | 17:06.48 |
| | Robin_Watts: ^^ | 17:06.52 |
| Robin_Watts | I'm drawing a blank on how to invert the decode array. | 17:07.10 |
| | Given a decode array [a b c d e f] I need to find a new one [A B C D E F] s.t. values fed through that later array and then fed through [0 100 -128 127 -128 127] would given the same result as if the value was just fed through the first array. | 17:08.24 |
| | Actually, stating it like that may give me what I need. | 17:10.22 |
| mvrhel_laptop | good deal | 17:10.49 |
| Robin_Watts | Bah. I've fixed that and I still get bad rendering. | 17:15.34 |
| mvrhel_laptop | Robin_Watts: I need to run to shop to get car. $$$ . Will be back on line in about 50 minutes | 17:16.08 |
| Robin_Watts | ok. | 17:16.15 |
| mvrhel_laptop | ok back a few $$ lighter | 18:21.19 |
| Robin_Watts | mvrhel_laptop: OK, so I fixed the decode stuff. | 18:21.43 |
| | Makes no difference :) | 18:21.46 |
| mvrhel_laptop | That can't be | 18:21.53 |
| Robin_Watts | The problem is in fz_blend_nonseparable_nonisolated | 18:22.10 |
| | Why can't that be? | 18:22.17 |
| mvrhel_laptop | at the end of the day, the image source data should be D0 00 D0 | 18:22.35 |
| | that gets mapped to CMYK values | 18:22.44 |
| | via LAB to the output intent profile | 18:23.05 |
| Robin_Watts | yeah, I get a green triangle rendered in the debug output. | 18:23.17 |
| mvrhel_laptop | are you seeing D0 00 D0 values? | 18:23.26 |
| | prior to color conversion? | 18:23.34 |
| Robin_Watts | The black edges appear during the group blend. | 18:23.35 |
| mvrhel_laptop | no black edge | 18:23.41 |
| | I am not looking at any black edges | 18:23.58 |
| Robin_Watts | Then I am confused. | 18:24.13 |
| mvrhel_laptop | For me the resulting image pixels were to light | 18:24.37 |
| | I am running with -A 0 also | 18:24.44 |
| Robin_Watts | Ah, that's why. | 18:24.54 |
| mvrhel_laptop | not sure what black edge you are talking about | 18:24.57 |
| Robin_Watts | If you run without -A 0 you'll get black edges. | 18:25.09 |
| | So it's entirely possible that my fix has solved your problem. | 18:25.26 |
| mvrhel_laptop | ok lets ignore those for a moment | 18:25.26 |
| Robin_Watts | ok, we can stick a pin in the black edges, and we'll come back to them in a mo. | 18:26.06 |
| mvrhel_laptop | ok, let me grab your fix | 18:26.13 |
| | ok that fixed the issue I was seeting | 18:29.28 |
| | seeing | 18:29.30 |
| Robin_Watts | ok, so lets pull the pin out... :) | 18:29.53 |
| | in fz_blend_nonseparable_nonisolated... | 18:30.13 |
| | lines 855-857 | 18:30.31 |
| mvrhel_laptop | And that fixed all the issues on that page (with -A 0) | 18:30.40 |
| | ok | 18:30.42 |
| | let me get there | 18:30.44 |
| Robin_Watts | You're combining bp[0-3] into the mix here. | 18:30.53 |
| | sr and rr are both inverted-if-required so definitely additive. | 18:31.15 |
| | bp[0-3] are not. This seems wrong to me. | 18:31.25 |
| mvrhel_laptop | ok let me get caught up | 18:31.47 |
| Robin_Watts | My best guess is that you should be using br, bg, bb there. | 18:31.50 |
| mvrhel_laptop | ok. bp[x] was used in the original code. I would think that we should have these values inverted though when we are doing cmyk | 18:41.13 |
| Robin_Watts | inverted and unpremultiplied, I would think. | 18:41.31 |
| | cos you redo the premultiplication a few lines later on (after the reinversion) | 18:41.49 |
| | which makes me think br, bg, bb | 18:42.06 |
| mvrhel_laptop | hehe. This was me just following what was there already | 18:42.18 |
| | The old code did that but with RGB | 18:42.34 |
| Robin_Watts | mvrhel_laptop: Not seeking to ascribe blame :) | 18:42.40 |
| mvrhel_laptop | well before doing anything I want do read one thing and check some math | 18:43.05 |
| | s/do/to/ | 18:43.11 |
| Robin_Watts | but... that doesn't solve the problem :( | 18:43.13 |
| | please, go for it. | 18:43.17 |
| mvrhel_laptop | Robin_Watts: so if I drop -A 0 I will see the problem? | 18:43.37 |
| | rendering wise? | 18:43.41 |
| Robin_Watts | Yes. | 18:43.43 |
| mvrhel_laptop | whoa | 18:44.07 |
| | ok | 18:44.09 |
| | hehe | 18:44.11 |
| Robin_Watts | I think the key thing here is that ha ceases to be 0 and 255. | 18:44.40 |
| | when ha is 0, none of this code runs. | 18:44.52 |
| | when it's 255, the bit I was just wibbling about has no effect. | 18:45.11 |
| | hence -A 0 will neatly hide the problem. | 18:45.27 |
| mvrhel_laptop | ok | 18:45.34 |
| Robin_Watts | I don't understand the /* Black */ section at all. | 18:46.09 |
| mvrhel_laptop | ok. give me a few minutes to step through all this | 18:47.07 |
| | So the black section does the same uncompositing that the cmy values do. Then we pick the value based upon the blendmode, | 18:52.11 |
| Robin_Watts | I have a solution. | 18:52.28 |
| mvrhel_laptop | hmm | 18:52.34 |
| | oh good | 18:52.40 |
| | I have gotten myself confused | 18:52.53 |
| | so a solution is most welcome | 18:53.01 |
| Robin_Watts | sk = fz_clampi(sk, 0, 255); | 18:53.15 |
| | sk was ending up negative, and hence going wrong. | 18:54.46 |
| mvrhel_laptop | I need to step through all this carefully. I am not feeling very confident about all of this | 19:00.08 |
| Robin_Watts | We have some edge effects on the edge of the text on page 3. That may be expected though. | 19:04.26 |
| | Some of the other blends on page 4 still have black edges. | 19:04.52 |
| mvrhel_laptop | I am worried that the clamping is masking some issue | 19:05.16 |
| | was the negative value off by a lot? | 19:05.40 |
| | or like -1 | 19:05.42 |
| Robin_Watts | ffffffffff3 | 19:05.48 |
| | for an ha of 0x13 | 19:05.55 |
| | so that seems plausible to me. | 19:06.26 |
| | edge effects on 7 too. | 19:07.04 |
| | is looking good now (I hope never to have to look at that page again :) ) | 19:07.33 |
| | 8 is looking good now (I hope never to have to look at that page again :) ) | 19:07.39 |
| | 9 is still screwy. | 19:07.47 |
| mvrhel_laptop | so I am still wrapping my head around shape having alpha values. | 19:08.40 |
| | I need to step through a couple things | 19:08.48 |
| Robin_Watts | oh, right, yes, you'd suggested that it shouldn't really, and my head was still spinning from that. | 19:09.29 |
| mvrhel_laptop | It is also applied in the underlying pixmap yes? | 19:09.53 |
| Robin_Watts | You may be right, but it worries me. | 19:09.59 |
| | yes. | 19:10.01 |
| mvrhel_laptop | as it is assumed to be in the non-compositing operation | 19:10.09 |
| | ok. I am going to step carefully through this simple image that we were just working on that has the black edges | 19:10.43 |
| | to understand how this is occuring | 19:10.52 |
| Robin_Watts | I have a more simplified version if that will help. | 19:11.18 |
| mvrhel_laptop | This is going to take me a bit of time... | 19:11.19 |
| | oh please | 19:11.21 |
| | Robin_Watts: ^^ | 19:11.33 |
| Robin_Watts | tri2.pdf in my casper homedir. | 19:11.59 |
| mvrhel_laptop | great. thank you | 19:12.34 |
| | Robin_Watts: other than these edges and the optional content do you know of any other issues? | 19:13.37 |
| Robin_Watts | there are issues all over this file. | 19:14.00 |
| | but we're hunting them down. | 19:14.15 |
| | ok, I've got to go. I'll look at page 9 tomorrow, assuming I can find a way to edit the damn file :( | 19:17.54 |
| mvrhel_laptop | oh that one | 19:20.34 |
| | It has to be an image decoder issue. It has two images there. one a sep and one a dev n | 19:21.52 |
| | I can see if I can chop it down some if you would like Robin_Watts | 19:22.03 |
| | these dumped images that you get for debugging transparency are great | 19:23.19 |
| | bbiab | 19:48.54 |
| Robin_Watts | mvrhel_laptop: The problem is that as soon as I load it into any of my usual editors and change it, mupdf won't read it any more. | 22:42.06 |
| | emacs, MSVC and Ultra-Edit all have problems with it. | 22:42.20 |
| mvrhel_laptop | Robin_Watts: ok. let me have a go at it | 22:42.23 |
| | you should try notepad++ | 22:42.30 |
| Robin_Watts | Ah, I have that. | 22:42.37 |
| mvrhel_laptop | I like it. It shows me how many characters there are | 22:42.58 |
| | so when I hack pdf files I can keep the stream lengths correct easily | 22:43.10 |
| Robin_Watts | Nope, same problem with that. | 22:44.29 |
| mvrhel_laptop | hmm | 22:44.39 |
| Robin_Watts | It's probably pointing to a problem in our repair code :( | 22:44.42 |
| mvrhel_laptop | ok black areas on page 3 gone | 22:46.53 |
| | so page 8, page 13 page 14 page 16 have issues | 22:49.29 |
| | Let me see if I can cut page 8 down to something for you | 22:49.49 |
| | i mean page 8 | 22:50.03 |
| | page 9 | 22:50.05 |
| | good grief | 22:50.07 |
| Robin_Watts | S'OK, I'm getting there now. | 22:50.23 |
| mvrhel_laptop | ok | 22:50.30 |
| | I am going to cut down page 13 then | 22:50.46 |
| | Robin_Watts: so my branch has a commit on it | 22:53.13 |
| | if you want to pull that in and have a look | 22:53.20 |
| Robin_Watts | Urm... fz_clampi needs to be assigned to something. | 22:54.48 |
| | line 760ish. | 22:54.55 |
| | should be sc = fz_clampi... | 22:55.11 |
| mvrhel_laptop | oops | 22:55.40 |
| | thanks | 22:55.41 |
| Robin_Watts | this ignores the stuff in my commit, right? | 22:55.46 |
| mvrhel_laptop | yes | 22:56.20 |
| Robin_Watts | ok, so if you update your commit, I'll pull that in, and redo the optimisations on top of yours. | 22:56.54 |
| mvrhel_laptop | ok sounds good | 22:57.20 |
| | ok it is there | 22:58.17 |
| Robin_Watts | I still don't think your commit is right :( | 23:01.16 |
| | oh, wait... | 23:01.39 |
| | yeah, I'm still suspicious. | 23:02.24 |
| | Line 861... | 23:02.33 |
| mvrhel_laptop | ok | 23:02.50 |
| Robin_Watts | sr and rr are non premultiplied values. | 23:02.57 |
| | We read sr and br above and unpremultiply them. | 23:03.21 |
| | Then we blend as appropriate to get rr. | 23:03.31 |
| | Then we reassemble it here, then we remultiply as we store. | 23:03.50 |
| | so how can we be throwing bp[0] into the mix? | 23:04.02 |
| | which is premultiplied. | 23:04.09 |
| | We should be using br/bg/bb instead of bp[0-3], I reckon. | 23:04.28 |
| mvrhel_laptop | so lets back up for a second | 23:04.39 |
| | do you think lines 887-889 are correct? | 23:04.48 |
| | the additive case? | 23:04.52 |
| | which is what we had always had? | 23:05.00 |
| Robin_Watts | no. same objection. | 23:05.04 |
| mvrhel_laptop | ok | 23:05.06 |
| Robin_Watts | Like I say, I'm not saying "*you've* got this wrong", just "this still doesn't look right to me". | 23:05.38 |
| mvrhel_laptop | That is where you have been confusing me | 23:05.39 |
| Robin_Watts | It's quite possibly my code originally. | 23:05.52 |
| mvrhel_laptop | let me spend some time on this then going through the math to make sure | 23:06.03 |
| Robin_Watts | I reckon that if we change bp[0] to br etc in all those lines, we'd be golden. | 23:06.22 |
| mvrhel_laptop | oh due to the pre-multiplied | 23:06.47 |
| Robin_Watts | yes. | 23:06.55 |
| | also, br has already been complemented above. | 23:07.08 |
| | so the 2 cases collapse together nicely. | 23:07.16 |
| mvrhel_laptop | So I am trying to compare the non-sep compositing | 23:10.12 |
| | rc = bc + fz_mul255(sa, fz_mul255(255 - ba, sc) + fz_mul255(ba, rc) - bc); | 23:10.14 |
| | oops | 23:10.28 |
| | that was sep case | 23:10.30 |
| | with what we have in the non-sep case | 23:10.35 |
| | as they really should be the same shouldnt they? | 23:10.45 |
| Robin_Watts | Just to be clear, what I am suggesting is: | 23:11.00 |
| | rr = fz_mul255(255 - haa, br) + fz_mul255(fz_mul255(255 - ba, sr), haa) + fz_mul255(baha, rr); | 23:11.02 |
| mvrhel_laptop | rc = ( 1 - (ha/ra)) * bc + (ha/ra) * ((1-ba)*sc + ba * rc) | 23:12.27 |
| | is what we want | 23:13.13 |
| Robin_Watts | Is it? | 23:13.41 |
| mvrhel_laptop | comment on line 636... | 23:13.49 |
| | Did I write that comment? | 23:14.10 |
| Robin_Watts | No, I suspect I did :) | 23:14.26 |
| | but regardless, it's a good one. | 23:14.39 |
| mvrhel_laptop | yes it is a good comment | 23:14.45 |
| | very clear | 23:14.49 |
| | brb | 23:15.22 |
| | ok back | 23:19.23 |
| | so I like the formula after the comment | 23:19.38 |
| Robin_Watts | OK, so my proposed form is: | 23:20.02 |
| | rr = fz_mul255(255 - haa, br) + fz_mul255(fz_mul255(255 - ba, sr), haa) + fz_mul255(baha, rr) | 23:20.14 |
| | Which is: | 23:20.17 |
| | rc = (1-ha).bc + ha.(1-ba).sc + ba.ha.rc | 23:20.49 |
| | = (1-ha).bc + ha[(1-bc).sc + ba.rc] | 23:21.28 |
| mvrhel_laptop | so does give rc * ra? | 23:21.46 |
| Robin_Watts | If you multiply the original equation through by ra, you get: | 23:23.16 |
| | ra.rc = (ra-ha).bc + ha[(1-ba).sc + ba.rc] | 23:23.55 |
| | which is sadly, not quite what I had. | 23:24.10 |
| mvrhel_laptop | hmm | 23:24.43 |
| | I have to run Ethan to a baseball tryout. | 23:24.58 |
| Robin_Watts | I have to go to bed. | 23:25.06 |
| mvrhel_laptop | If you feel you have a handle on this, I will bow to you. I may step through this math tonight though | 23:25.34 |
| | and send you and email | 23:25.37 |
| | an email | 23:25.40 |
| Robin_Watts | Please feel free. | 23:25.41 |
| mvrhel_laptop | ok | 23:25.49 |
| | have a good night | 23:25.51 |
| Robin_Watts | I have something that I think is better than what we have, but I wouldn't like to swear it's correct. (probably because of the shape not being just 0 or 255). | 23:26.33 |
| | talk to you tomorrow. | 23:26.55 |
| | mvrhel_laptop: My proposed commits are on robin/spots | 23:34.06 |
| | Forward 1 day (to 2017/08/03)>>> | |
Log of #mupdf at irc.freenode.net.