| | <<<Back 1 day (to 2017/04/30) | 20170501 |
| brettk80 | @Sebras @salzh is working with me on the issue too, but we dont have permission to to submodules on the repo.. | 14:32.07 |
| kens | brettk80 : you shouldn't need special permission, its all in the same Git repository | 14:32.30 |
| sebras | brettk80: I'm not sure what that means? do you have an error message? | 14:32.32 |
| kens | did you do a submodule update ? | 14:32.42 |
| sebras | brettk80: oh, and a corresponding command you are using. | 14:33.12 |
| brettk80 | Im relaying.. let me see if salzh is around.. | 14:33.33 |
| | fatal: remote error: access denied or repository not exported: /user/sebras/thirdparty-curl.git | 14:33.34 |
| sebras | brettk80: oh! I see the problem. | 14:34.30 |
| kens | Oh, using a sebras repository | 14:34.46 |
| | I had assumed the canonical MuPDF repo | 14:35.04 |
| brettk80 | we were trying to test the build/patch you recommended | 14:35.13 |
| sebras | kens: I wasn't aware that the relative links would result in the thirdparty repos would also be assume to belong to me. | 14:35.24 |
| kens | :-) | 14:35.31 |
| | I don't htink it would have occured to me | 14:35.43 |
| | Probably the easiest thing is to clone MuPDF then cherry-pick your path(es) ? | 14:36.02 |
| sebras | kens: yes, after adding my repo as a remote and fetching it. | 14:36.38 |
| kens | Yep makes sense | 14:37.04 |
| Robin_Watts | sebras: Either clone, or make symlinks :) | 14:37.25 |
| sebras | brettk80: ok, let's try this: git clone --recursive git://git.ghostscript.com/mupdf.git which ought to give you the official developer version without my patch. | 14:37.32 |
| Robin_Watts | symlinks means you don't have to work to keep them in sync. | 14:37.48 |
| brettk80 | Thanks. Ill have Zhong (@salzh) give it a shot. He's heading up this issue now. Im more of the grunt researcher :) | 14:38.49 |
| sebras | brettk80: then cd mupdf; git add remote sebras git://git.ghostscript.com/user/sebras/mupdf.git; git fetch sebras; git cherry-pick b2c820e11bdec61fc38dde8d7d0fd3b35fc7ccfa | 14:38.59 |
| brettk80 | I dont mind being the grunt | 14:39.07 |
| sebras | brettk80: this would add my git repo as an additional remote and then you'll cherry-pick the patch that I belive fixes your problem. | 14:39.30 |
| brettk80 | awesome.. thanks everyone. | 14:39.44 |
| sebras | Robin_Watts: where do I create the symlinks? in my repo on the server? | 14:39.55 |
| brettk80 | There hasnt been much documentation on google about this issue, other than working with massive tiffs which isnt the issue in this case. | 14:40.12 |
| | Just kind of funny it was trying to assign a TB of memory for a dumb fax tiff. | 14:40.34 |
| Robin_Watts | on casper: cd /home/sebras/repos && ln -s /home/git/thirdparty-jpeg.gig thirdparty-jpeg.git | 14:40.38 |
| | or something like that I'd guess. | 14:40.43 |
| | (once you fix the deliberate typos :) ) | 14:41.05 |
| brettk80 | You all have helped me out in the past so I figured I check here... Mucho appreciated. | 14:41.25 |
| sebras | Robin_Watts: these show up in gitweb though. that might be annoying to people..? | 14:43.08 |
| Robin_Watts | sebras: So do creating normal ones, right? | 14:43.29 |
| | Possibly we could hack gitweb to avoid showing symlinked ones. | 14:44.01 |
| sebras | Robin_Watts: I wonder what happens if the gitweb config in each repo is set to hid the repo and you have a symlink though... ;) | 14:44.50 |
| Robin_Watts | Is there a gitweb config in each repo ? | 14:45.17 |
| sebras | Robin_Watts: you have to edit somethign to allow gitweb to export the repo, yes. | 14:45.42 |
| Robin_Watts | sebras: You have to edit something to allow *git* to export the repo. | 14:46.07 |
| | not gitweb. | 14:46.12 |
| | git daemon-export-ok or something like that. | 14:46.21 |
| sebras | Robin_Watts: git-daemon-export-ok, yes. I though this was related to gitweb. | 14:46.36 |
| Robin_Watts | sebras: I believe it's related just to people being able to get to the repos using git:// or http:// | 14:47.06 |
| sebras | Robin_Watts: though apparently you can $GITWEBDIR/gitweb_config.perl, not sure what it is used for yet though. | 14:47.23 |
| | Robin_Watts: if you have time, do you mind taking a look at sebras/master? only ~35 lines changed. | 14:52.38 |
| Robin_Watts | sebras: In the 2nd one... Are you assuming MSB tiffs? | 14:55.34 |
| | oh, gawd, you do that for all the samples after having already byteswapped them. | 14:56.57 |
| | Can't we do that all a bit more simply? | 14:58.25 |
| | For the 16 bit case, it's buf[2] ^= 128; buf[3] ^= 128; | 14:59.16 |
| | oops, 2 and 4. | 14:59.24 |
| | For the 8 bit case, it's buf[1] ^= 128; buf[2] ^= 128; | 14:59.57 |
| | 1 and 3 lgtm (assuming the cluster is happy). 2 can be simpler. | 15:00.55 |
| sebras | Robin_Watts: I do, but then again these tiffs are apparently rare. | 15:00.56 |
| | Robin_Watts: I need to think about why xor is the same as subtraction a bit. | 15:01.27 |
| Robin_Watts | sebras: We byte swap tiffs before this code is called, so assuming MSB tiffs is fine. | 15:01.35 |
| | sebras: (x + 128) & 0xff == x ^ 128 (for 0 <= x < 256) | 15:02.10 |
| sebras | Robin_Watts: I guess we could do all of it in one go, but I'm chasing the accuracy first. | 15:02.14 |
| Robin_Watts | (x + 32768) & 0xffff == x ^ 32768 (for (0 <= x < 65536) | 15:02.46 |
| sebras | Robin_Watts: right so for the 16-bit case I'd still have to combine the bytes to form x, and then xor ok. | 15:03.54 |
| Robin_Watts | sebras: Why!? | 15:04.04 |
| | 32768 = 0x8000. | 15:04.19 |
| | Eorring with 0x8000 doesn't do anything to the lower bits :) | 15:04.36 |
| sebras | Robin_Watts: oh... it is 32678, not 128. /me is not paying attention. | 15:04.42 |
| | Robin_Watts: I'll adress this, and get back to you. | 15:05.23 |
| | Robin_Watts: if we're really unlucky it might be a tiled 16 bit byte swapped Lab image with all three color components compressed using lzw and having a horizontal differencing predictor. in this case we'd iterate over each sample four times after having decompressed it. | 16:47.22 |
| Robin_Watts | sebras: Yes. | 16:47.37 |
| | I think I pulled MuPDFs tiff compression into SOT. | 16:48.02 |
| | decompression./ | 16:48.06 |
| | and I altered stuff like this so that it happened through a stream. | 16:48.19 |
| sebras | Robin_Watts: yeah, but it hasn't made it back into mupdf yet. :) | 16:48.44 |
| Robin_Watts | so we ended up with a composition of streams, thus avoiding the need to hold the entire thing uncompressed. | 16:48.52 |
| | sebras: Indeed not. | 16:49.00 |
| sebras | Robin_Watts: right, that makes sense I belive. | 16:49.05 |
| Robin_Watts | I think we just live with the fact we're running over it multiple times for now. | 16:49.21 |
| sebras | Robin_Watts: agreed, if tiff files were more prevalent (or if someone has a good case where performance is abysmal) then I'd look into moving the code over. | 16:50.36 |
| | Robin_Watts: I've updated sebras/master. | 16:52.40 |
| Robin_Watts | lgtm. | 16:53.30 |
| | For performance it's always almost better to do: for (i = n; i > 0; i -= 6) | 16:53.51 |
| | but that's a small point. | 16:54.17 |
| sebras | Robin_Watts: compared to iterating over each sample four times as the worst case? absolutely. ;) | 16:54.35 |
| | Robin_Watts: I'd rather look into moving the SOT code over. | 16:54.48 |
| Robin_Watts | I'm not sure the SOT case wins for speed :) | 16:55.03 |
| | It wins for memory use. | 16:55.08 |
| | On ARM it's a 9 cycle inner loop, using 1 extra register, rather than an 8 cycle inner loop. | 16:59.34 |
| | but that get swamped my memory timings of course. | 16:59.42 |
| | but that gets swamped by memory timings of course. | 17:00.07 |
| sebras | Robin_Watts: of course. does this mean you implemented the inner loop in ARM assembler? or you just looked at the output from the compiler afterwards? | 17:02.11 |
| Robin_Watts | sebras: I implemented it. | 17:16.55 |
| | (not in SOT, just in a scratch buffer, just now) | 17:17.16 |
| | In ARM assembler you'd unroll the loop and work in ints. | 17:17.42 |
| sebras | I see. | 17:20.32 |
| | Robin_Watts: the LGTMed commits clustered ok, so I'll push it. | 17:20.55 |
| | Forward 1 day (to 2017/05/02)>>> | |
Log of #ghostscript at irc.freenode.net.