| | <<<Back 1 day (to 2013/04/16) | 2013/04/17 |
| mvrhel_laptop | tkamppeter_ are you there? | 05:08.25 |
| | ok. so the color monitoring stuff is written except for high level c-list images. that is going to be a little ugly so will tackle that in the morning | 05:40.53 |
| Robin_Watts | paulgardiner: So, you're coming to Miami. | 08:28.30 |
| paulgardiner | Right. Great | 08:28.56 |
| Robin_Watts | We are on VS005 out on the 7th, and VS006 back on the 8th | 08:29.26 |
| | 9th! | 08:29.31 |
| kens | Robin_Watts : re your comment on bug #693917 | 08:29.42 |
| | It really is the downsampling we want for pdfwrite, not downscaling | 08:29.56 |
| | I waqnt to reduce the effective resolution of the image data | 08:30.10 |
| Robin_Watts | kens: Right. The terms downsampling and downscaling are really interchangable in general. | 08:31.13 |
| | but in gs, we have 2 completely different implementations. | 08:31.30 |
| kens | Hmm, well... | 08:31.44 |
| Robin_Watts | the downsampler takes 'groups' of pixels and turns them into small groups of pixels. | 08:31.55 |
| | typically 4:1, 9:1, 16:1 etc, but also 9:4 etc. | 08:32.15 |
| | and the way that's hooked up it's for the tiffscaled device. | 08:32.25 |
| kens | Sounds like what I want though | 08:32.47 |
| Robin_Watts | The downscaler takes a stream of image data at any size, and bilinearly interpolates that into a stream of image data at any other size. | 08:32.54 |
| | That sounds more to me like what you want. | 08:33.03 |
| kens | I don't want just bilinear, I need bicubic, average and subsample | 08:33.21 |
| | Say I have an image 300x300 samples, scled so that it covers 1 inch on the page | 08:33.46 |
| | THat is an effective resolution of 300 dpi. | 08:33.56 |
| Robin_Watts | right. And if you are working at 72dpi, then you want to change that to be a 72x72 image, right? | 08:34.18 |
| kens | I want to reduce the number of samples (giving a smaller image) so that there are only 150x150 | 08:34.22 |
| | Giving a 2:1 ratio | 08:34.31 |
| | THat works. | 08:34.35 |
| | But a non-integer ratio doesn't | 08:34.44 |
| Robin_Watts | Why not? | 08:34.54 |
| kens | Because the ciode doesn't support non-integer downsampling | 08:35.08 |
| Robin_Watts | kens: whose code? | 08:35.23 |
| kens | I don't know, its not mine :-( | 08:35.33 |
| Robin_Watts | which code then? | 08:35.48 |
| | pdfwrite? | 08:35.51 |
| kens | Its called from pdfwrite, but I was under the impression it was in the graphics library | 08:36.05 |
| | I've never really looked into it | 08:36.13 |
| | Possibly siiscale.c | 08:36.31 |
| | judging by comments | 08:36.38 |
| | give me a minute and I'll find the filter it uses | 08:37.19 |
| Robin_Watts | If you're calling siscale.c then you are not restricted to integers. | 08:38.35 |
| | "interp" = bilinear, "mitchell" = "bicubic" | 08:38.59 |
| kens | Not sure, let me find the code first, its in the general case, not pdfwrite per se | 08:39.01 |
| | Its in the stream code iut seems | 08:40.57 |
| | Yes its the stream code that does all the work in gdevpsds.c | 08:41.38 |
| | For example s_Subsample_process | 08:41.49 |
| | So essentially ps2write and pdfwrite are using their own code. Of course, why would we use code that already existed when we can write a new implementation..... | 08:42.25 |
| Robin_Watts | right, so the choice of just subsampling by a given integer amount rather than scaling properly by an arbitrary amount is done in gdevpsds.c | 08:44.11 |
| kens | Yes. Presumably I could get bicubic (and arbitrary factor) by using sisscale.c, but not for average or subsample | 08:44.45 |
| Robin_Watts | I *assume* that subsample = nearest neighbour? | 08:45.23 |
| | and average is what I think of as simple subsampling. | 08:45.41 |
| kens | No it just means 'throw away samples' | 08:45.42 |
| | average I would guess is take a group of nxn pixels and replace them with a single pixel of the averag value | 08:46.15 |
| Robin_Watts | kens: Right, so if you're 'subsample' by a factor of 2 in each direction, you throw away 3 of every 4 pixels, and just take the 'nearest' sample. That's nearest neighbour. | 08:46.40 |
| kens | OK fair enough | 08:46.52 |
| Robin_Watts | 'average' is what I do in the subsampler. | 08:47.41 |
| kens | Hmm, this is all too much for a bug, I'm going to change it to an enhancement so I can ignore it more efficiently. | 08:48.15 |
| Robin_Watts | So in gdevpsds.c, you have an s_Average | 08:49.49 |
| kens | Indeed and an s_Subsample | 08:50.00 |
| | THere's a 'process' for each type | 08:50.50 |
| Robin_Watts | right. | 08:51.18 |
| | so those implementations are hardwired to integer amounts. | 08:51.30 |
| kens | Yep | 08:51.35 |
| | That's what the user is complaining about | 08:51.43 |
| | The implementations are pretty noddy by the look of them | 08:51.56 |
| Robin_Watts | s_Average would be hard to do in a non-integer amount. | 08:52.15 |
| | and s_Subsample too. | 08:52.24 |
| kens | Hmm, I wonder what Acrobat does iwth those. | 08:52.40 |
| Robin_Watts | but s_Bilinear and s_Bicubic need not have the same restriction. | 08:52.41 |
| kens | Well we don't support bicubic at the moment, that's a different bug (by the same user) | 08:53.08 |
| | When I get the PXL code finished I'll try and come up with an example PostScript image to play with Acrobat a bit. Possibly I can close the bug if Acrobat doesn't do average or subsample at non-integer resolutions. | 08:54.27 |
| | err factror, not resoution | 08:54.57 |
| Robin_Watts | kens: yeah, the "customer" doesn't seem slavishly keen on us being identical to acrobat. | 09:11.45 |
| kens | I'm not totally averse, but its not a paying customer, I've no idea who he repesents but could easily be an infringer, and its a lot of work for little result. | 09:12.26 |
| | I'll do the Acrobat research and then change it to an enhancement | 09:13.00 |
| Robin_Watts | I got the cluster logfile searching from the dashboard working last night. | 09:13.39 |
| kens | nice, and I see Henry's links in the bmpcmp result this morning too | 09:13.58 |
| Robin_Watts | Click on "search" next to a result, then enter a search term, and if we match it, it'll show the logs for those jobs. | 09:14.18 |
| | kens: yeah | 09:14.21 |
| | If you search with the empty string, you get all the logs. | 09:14.36 |
| | I need to add the search links to the User links section. Will do that when I get back. | 09:14.57 |
| kens | ah, I had ot refresh Chrome to get the search links | 09:15.27 |
| Robin_Watts | paulgardiner: This may be useful for you, as you can search for "Alert" in a mujstest and see all the js alerts. | 09:15.30 |
| | kens: yeah. you'll need to reload, as that's in the html, not the json. | 09:15.54 |
| Robin_Watts | runs | 09:16.38 |
| paulgardiner | Robin_Watts: may well be | 09:16.59 |
| kens | TIMe for me to go too, I won't be in much this afternoon either. | 09:20.38 |
| Robin_Watts | chrisl: dashboard looks suspect... | 10:35.05 |
| | same commits different revisions? | 10:35.14 |
| chrisl | Robin_Watts: I'm having trouble - clusterpush push works, commit run fails - hence the commit fiddling | 10:35.47 |
| Robin_Watts | ok. if you see the cluster getting into trouble, let me know and I can screw it up worse. I mean fix it. | 10:38.49 |
| chrisl | Do you know what the differences is between a clusterpush and a commit run? | 10:38.49 |
| | it shouldn't matter - we should never be running tiff configure in the tiff directory with that commit | 10:48.10 |
| Robin_Watts | So the most recent thing the cluster ran... the gs build command is: | 10:51.00 |
| | cd __gsSource__ ; make clean >/dev/null 2>&1 ; make -j 1 >__temp__/gs_build.log 2>&1 | 10:51.07 |
| chrisl | My best guess is that the commit cluster run does not start with a "clean" repo, but I don't know where the repo is that the commit run uses | 10:54.07 |
| Robin_Watts | chrisl: /home/marcos/cluster/ghostpdl | 10:54.24 |
| chrisl | Of course it is, why would I think it lived in /home/regression somewhere - silly me! | 10:55.03 |
| Robin_Watts | sorry! | 10:55.16 |
| | /home/regression/cluster/ | 10:55.30 |
| | /home/regression/cluster/ghostpdl | 10:55.37 |
| | On the remote machines (such as peeves) it's often /home/marcos/cluster/ghostpdl | 10:56.05 |
| | ok, user commit searching from the dashboard is up and working. | 10:56.41 |
| chrisl | But on remote machines, it doesn't use git, does it? | 10:57.13 |
| | Balls! It looks like for commit runs, the cluster isn't cleaning out the tiff directory between runs - I'm shocked that hasn't fallen over previously! | 11:00.02 |
| | Okay, I *think* I can temporarily work around this in the Ghostscript and GhostPDL configure scripts - it's just a pain that I can only test it by doing a commit :-( | 11:03.35 |
| Robin_Watts | chrisl: The directory is rsynced from casper to the hosts. | 11:06.19 |
| | hence if you clear out /home/regression/cluster/ghostpdl, you should then see the hosts cleared out too. | 11:06.40 |
| chrisl | No, there must be more to it than that - the directory on casper is clean, but the one on peeves still has the config.status file in it | 11:08.25 |
| Robin_Watts | cd users/marcos ; rsync -cvvlogDtprxe.iLsz --timeout=60 --progress --delete -e "ssh -l regression -i $HOME/.ssh/cluster_key" regression@casper.ghostscript.com:/home/regression/cluster/users/marcos/ghostpdl . >rsync.log 2>&1 | 11:11.27 |
| chrisl | Er, that's not the path you mentioned above..... | 11:12.44 |
| Robin_Watts | I picked the wrong one from the log. | 11:13.41 |
| | Oh! | 11:14.04 |
| chrisl | And worse, /home/regression/cluster/users/marcos/ghostpdl isn't a git repo | 11:14.11 |
| Robin_Watts | cd /home/marcos/cluster/ghostpdl/gs ; touch base/gscdef.c ; rm -f base/gscdef.c ; cd /home/marcos/cluster/ghostpdl ; git fetch -q ; git reset --hard -q master ; git reset --hard -q origin/master ; git reset --hard -q 28acb04f77ba58b1507c32d52402e2ff8eaa2e40 | 11:14.22 |
| | So commit runs are done using git now. That's a change. | 11:14.46 |
| | So we probably want a git clean in there too? | 11:15.01 |
| | Are the libtiff config products in .gitignore ? | 11:16.12 |
| chrisl | And that's a different path again - what the hell.....?? | 11:16.22 |
| | Robin_Watts: I've no idea | 11:16.37 |
| Robin_Watts | chrisl: On the remotes, as I said, the cluster tends to be installed in /home/marcos/cluster | 11:16.58 |
| | and hence the ghostpdl tree is in /home/marcos/cluster/ghostpdl | 11:17.13 |
| chrisl | Argh, yes the tiff derived files are in .gitignore | 11:17.25 |
| Robin_Watts | chrisl: That's good. It means I can do: git clean -X -f in the cluster scripts. | 11:17.53 |
| chrisl | So I should remove them - I wonder if that would solve the problem | 11:18.05 |
| Robin_Watts | no, don't remove them! | 11:18.14 |
| chrisl | Why not? They won't exist any more - at least, in the stated paths | 11:18.40 |
| Robin_Watts | We want them ignored, lest people see them in git status and add them. | 11:19.00 |
| | and if I "git clean -X" then it will remove all .gitignored files. | 11:19.18 |
| chrisl | But the stuff in .gitignore is not correct for the new way of running tiff configure, so it won't work | 11:19.59 |
| Robin_Watts | chrisl: I've updated run.pl to clean. Try pushing your commits again now. | 11:20.53 |
| | We can correct .gitignore later. | 11:21.03 |
| chrisl | It's on it's way now | 11:22.20 |
| | okay, so now, instead of a whole list of explicit files to ignore for tiff, we can just have a single "tiff-config" entry | 11:25.18 |
| | Robin_Watts: Yay! Adding the git clean step seems to have work :-) Thanks! | 11:26.01 |
| Robin_Watts | chrisl: We should have both, at least until every cluster node has cleaned :) | 11:26.04 |
| | The aws ones will still be dirty at the moment. | 11:26.19 |
| chrisl | I don't like the fact that appear to ignore the entire tiff directory just now | 11:27.19 |
| tor8 | Robin_Watts: ping | 11:37.04 |
| Robin_Watts | pong | 11:38.04 |
| | tor8: I'm being called for lunch... | 11:39.58 |
| tor8 | Robin_Watts: then ping me when you're back | 11:50.33 |
| Robin_Watts | tor8: back, sorry. | 12:33.36 |
| | tor8, paulgardiner: OK, so http://ghostscript.com/~robin/MuPDF-12.apk is the new build that I plan to put on google play. | 13:41.29 |
| paulgardiner | Robin_Watts: text selection is broken | 13:46.59 |
| Robin_Watts | so it is. | 13:48.21 |
| | Is it broken on your builds too? | 13:48.36 |
| paulgardiner | Don't think so | 13:49.31 |
| | It was working last time I built here. | 13:49.44 |
| Robin_Watts | Can you double check please? This is a clean build from a clean tree. | 13:50.04 |
| | oh, actually, there is a change. | 13:50.25 |
| | Let me rebuild... | 13:50.30 |
| paulgardiner | My last build was at 6ddde649c594c254ffbb101ef8b8ee8b005992a8 | 13:51.39 |
| Robin_Watts | It's probably my change here to do with the text device and images. | 13:52.18 |
| | rebuilding now. | 13:52.40 |
| | I should probably do separate armeabi/armeabi-v7a builds too. | 13:54.04 |
| | http://ghostscript.com/~robin/MuPDF-12-armeabi.apk | 14:03.08 |
| | Text selection works in that, forms are ignored. | 14:04.45 |
| | http://ghostscript.com/~robin/MuPDF-12-armeabi-v7a.apk | 14:04.51 |
| paulgardiner | Robin_Watts: I believe I'm testing the v7a one and calc.pdf isn't working | 14:10.19 |
| Robin_Watts | paulgardiner: Yeah, I'm looking into that now. | 14:10.33 |
| | Does it look ok otherwise. | 14:10.37 |
| paulgardiner | Yes, I think so | 14:11.33 |
| Robin_Watts | ok, I see the mistake. | 14:15.37 |
| | http://ghostscript.com/~robin/MuPDF-12-armeabi-v7a.apk <- New version | 14:20.36 |
| | done. | 14:22.02 |
| | that seems to work. | 14:22.08 |
| | Gah. I need to make the armeabi-v7a version number 13, so v7a devices will get that rather than the armeabi version. | 14:28.42 |
| | OK, updated version are published to google play. just have to wait for them to propagate around. | 14:41.45 |
| sebras | Robin_Watts: not here yet... :( | 14:45.09 |
| Robin_Watts | 87% of installed users of MuPDF are Android 4 or above. And we have 2204 users currently. | 15:09.24 |
| mvrhel_laptop | bbiab | 15:36.01 |
| sebras | Robin_Watts: you have at least one user which is android 2.3 though! | 16:06.47 |
| Robin_Watts | Me! | 16:07.07 |
| sebras | Robin_Watts: there is a slight chance that I will upgrade in october since I might be visiting .tw again then... | 16:07.10 |
| | Robin_Watts: then we are two! :) | 16:07.16 |
| Robin_Watts | but I don't generally install off google play :) | 16:07.28 |
| | sebras: New version has reached me. | 16:08.40 |
| sebras | Robin_Watts: nope, not yet. | 16:09.37 |
| mvrhel_laptop | hi ray_laptop: just tried to call you a bit ago | 16:29.52 |
| atmos4 | hi | 16:30.33 |
| | I'm trying to wrap jpeg images in PDF using gs and viewjpeg.ps, however the images always get recompressed. Is there a way to skip this recompression? | 16:33.36 |
| | I've tried setting -dJPEGQ=100 but that had no effect | 16:34.57 |
| | neither did -dAutoFilterColorImages=false | 16:35.39 |
| chrisl | atmos4: there is no way to avoid decompressing the JPEG image data | 16:37.53 |
| atmos4 | hmm theoretically there is | 16:38.20 |
| | if I embed the jpeg data with a hexeditor into the opdf it's fine =) | 16:38.33 |
| | so you mean this can't be done with gs? | 16:39.13 |
| chrisl | Well, you were asking about gs, weren't you? | 16:39.32 |
| atmos4 | yes | 16:39.37 |
| ray_laptop | mvrhel_laptop: sorry -- I was on a quick errand | 16:39.59 |
| atmos4 | chrisl: do you care to explain what's the limitation for that? I can represent a jpeg image in PDF by an xobject w/wubtype image and filter dctdecode which is followed by the raw jpeg data. Is it because that is not possible in PD as opposed to PDF? | 16:45.05 |
| | s/PD/PS/ | 16:45.15 |
| chrisl | atmos4: there is no way for pdfwrite to see anything other than the decompressed jpeg data | 16:45.36 |
| atmos4 | ok | 16:47.30 |
| Robin_Watts | atmos4: gs runs through the PDF file interpreting the data it finds to a set of graphics primitives sent to the current device. | 16:47.39 |
| | pdfwrite is one such device. All it sees is these primitives coming in. When it gets a bitmap in, it is passed the decompressed form. It has no way of knowing if it's come from a jpeg or a png or a fax... | 16:48.31 |
| atmos4 | Robin_Watts: you mean the PS file? Given that my source is PS not PDF. | 16:48.32 |
| Robin_Watts | ok, then for PDF read PS. | 16:48.49 |
| atmos4 | yeah, seems like I'll have to either write my own pdf generation code or look for another tool. | 16:50.31 |
| | guess the Developing with PDF book by O'Reilly will finally be read ;-) | 16:51.35 |
| | ok, afk for a while, thanks for the explanation! | 16:52.08 |
| ray_laptop | OK. Ready to start debugging background print changes done, and clean build. First to test foreground printing with NumRenderingThreads > 0 (since I refactored much of that) | 18:05.18 |
| | this is the enhancement for cut #801, issue #10 | 18:05.39 |
| | bbiaw... | 18:06.02 |
| Robin_Watts | ray_laptop: Nice. | 18:06.09 |
| mvrhel_laptop | whew. ok indexed case written. now for the real ugly part of going through the high level c-list image data | 21:25.13 |
| | then testing the many variations.... | 21:25.39 |
| | bbiaw | 22:27.51 |
| | Forward 1 day (to 2013/04/18)>>> | |
IRC Logs