| | <<<Back 1 day (to 2012/07/09) | 2012/07/10 |
| tkamppeter | Anyone has an idea how to fix bug 693176? It is about wrong color output in RGBW color space. | 08:22.24 |
| Robin_Watts | tkamppeter: It could be a devcups bug, or it could be a bug elsewhere in the system. | 08:24.39 |
| | Is this a regression? Has it ever worked? | 08:24.51 |
| tkamppeter | Robin_Watts, I am not sure whether it worked before, the first report of it is the one mentioned in this bug report, from yesterday. | 08:28.47 |
| chrisl | Can we do RGBW with tiffsep? | 08:34.21 |
| tkamppeter | Robin_Watts, I am trying with older Ubuntu versions now: Lucid (10.04 LTS) with GS 8.71 does not work at all. The command line with color space 17 errors out due to an old bug. | 08:44.59 |
| Robin_Watts | chrisl: not afaik | 08:45.58 |
| | ok, so it's quite likely a bug in cups. | 08:46.11 |
| | I've never looked in cups personally - I think kens or mvrhel will be the people to talk to first. | 08:46.41 |
| chrisl | Robin_Watts: tiffsep should handle arbitrary separations, otherwise it's a bit pointless....... | 08:46.44 |
| Robin_Watts | chrisl: OK, let me rephrase: I have no idea how you'd tickle tiffsep into trying for rgbw. | 08:47.20 |
| | Going for a run. bbs. | 08:48.11 |
| chrisl | Robin_Watts: it should take separation names and sepration order in the PS world - what I don't know is whether the PDF interp will override that. | 08:48.20 |
| tkamppeter | Robin_Watts, Ghostscript 9.04 also shows the problem. | 08:49.05 |
| chrisl | tkamppeter: how can I change the resolution in devcups - "-r" doesn't seem to work | 08:53.38 |
| tkamppeter | chrisl, for me it works. The following line gives me 300 dpi: | 08:58.14 |
| | gs -sDEVICE=cups -r300x300 -dcupsColorSpace=17 -sOutputFile=out17.raster firefox-google-com-lp1022516.pdf | 08:58.17 |
| chrisl | Well, I'm getting *extremely* course halftoning, to the extent I can't tell what the "color" is | 09:00.12 |
| tkamppeter | chrisl, the correct color is a very light gray and the wrong color is an intense yellow, easy to distinguish also with coarse halftoning. | 09:25.01 |
| | chrisl, the objects with the wrong color are the two buttons directly under the input field. | 09:25.45 |
| | chrisl, you should raise color depth via "-dcupsBitsPerColor=8". | 09:28.58 |
| chrisl | tkamppeter: With -dcupsBitsPerColor=8 I get light gray for those buttons with both 9.05 and the current code | 09:33.42 |
| | tkamppeter: but with 9.05 packaged on Ubuntu 12.04 they come out yellow...... | 09:35.25 |
| tkamppeter | chrisl, the CUPS backend in the Ubuntu package is not patched. We have only two patches backported from GIT for ps2write and pdfwrite. | 09:41.36 |
| chrisl | tkamppeter: the point is 9.05 with a default build gives the correct output - did you even bother to test that?? | 09:42.37 |
| tkamppeter | chrisl, not yet. Now I tested Quantal and there the buttons are only yellow at their borders. So it seems to highly depend on how the code gets compiles, GCC version and GCC options. | 09:47.30 |
| | chrisl, did you test with 32-bit or 64-bit? | 09:47.50 |
| chrisl | 64 bit right now | 09:47.59 |
| tkamppeter | chrisl, tested Precise 32-bit and there the same full yellow buttons as with Precise 64-bit come out. | 09:51.56 |
| chrisl | tkamppeter: that doesn't surprise me..... of course, being a cairo PDF, Acrobat can't edit it, so that makes like harder...... | 09:52.46 |
| | bbiab | 10:07.03 |
| ghostser | hello | 10:21.23 |
| | anyone? | 10:21.33 |
| Robin_Watts | impatientser :) | 10:21.43 |
| ghostser | i am having a problem using ghostscript ... can anyone help? | 10:22.09 |
| Robin_Watts | That depends on the question. | 10:23.37 |
| | by which I mean, "Ask the question" | 10:34.18 |
| | Gah. How did the cups bug land on my plate? | 11:54.36 |
| Robin_Watts | pops out for a bit. back before the 4pm forms meeting. | 12:39.41 |
| | Morning mvrhel_laptop. Got a mo ? | 14:17.03 |
| | How familiar are you with the internals of gx_image_enum *penum ? | 14:17.50 |
| | I've decided that penum->rect is not so much the 'subrectangle to be rendered' as the 'subrectangle for which data is being sent'. Can anyone confirm/deny this offhand? | 14:18.39 |
| mvrhel_laptop | hi Robin_Watts | 14:21.25 |
| | I am not that familiar with it. But from what I saw that you mentioned earlier, it sounds like we need a rect of what is to be rendered | 14:22.01 |
| Robin_Watts | Yes. | 14:22.12 |
| mvrhel_laptop | This seems like a major oversight | 14:22.26 |
| Robin_Watts | I've got local changes here that calculate an additional rectangle. | 14:22.28 |
| | I was hoping that I could just chop bits of 'rect' and have it work, but it seems we do need a new 'render rectangle' as well. | 14:23.19 |
| mvrhel_laptop | ok. is the path that the image is filling in the graphic state at the time? | 14:28.35 |
| | and does the clip device come in to play at all with this? | 14:28.55 |
| Robin_Watts | I get a pcpath argument when I'm setting up the image. | 14:29.02 |
| | so calculating the correct box is all done (I think). | 14:29.28 |
| mvrhel_laptop | thats good | 14:29.35 |
| | and what if we are stroking a path vs. filling? | 14:29.50 |
| Robin_Watts | The problem is juggling stuff so that the inner loop only interpolates the 4 pixels it needs to per row :) | 14:29.52 |
| | At this level, stroking == filling, I hope. | 14:30.05 |
| mvrhel_laptop | ok | 14:30.10 |
| Robin_Watts | (i.e. the path to be stroked will have been converted to one to be filled) | 14:30.24 |
| mvrhel_laptop | right | 14:30.28 |
| | so, now do you need to make changes in the interpolation renderer to make use of the rect? | 14:31.24 |
| | i imagine that is what you mean above | 14:31.38 |
| Robin_Watts | yes. | 14:31.49 |
| mvrhel_laptop | that will be the fun part :0 | 14:31.58 |
| alexcher | GS uses a pointer to a local variable to pass back GC requests. This pointer is saved in the allocators and used when it's no longer valid. This causes SEGV's in 34_all.ps. Changing the GC signal variable to a static one cures all SEGVs. Any better idea? | 14:36.38 |
| Robin_Watts | alexcher: Using statics sounds bad to me. | 14:38.23 |
| | Can the lifespan of that variable be shortened so it's safe? | 14:38.40 |
| | (i.e. can we arrange for it to be set to NULL at the start of every gc operation? | 14:39.00 |
| chrisl | Given that we're *supposed* to be thread safe, using a static isn't really going to fly....... doesn't GC have it's own "state"? | 14:39.56 |
| alexcher | Robin_Watts: that's what I cannot figure out. | 14:40.23 |
| Robin_Watts | Presumably the value of that variable is something that is used during the next gc operation, and then never again? | 14:40.23 |
| ray_work | hi, all | 14:40.24 |
| Robin_Watts | alexcher: Can you give a pointer to the variable/code in question? | 14:40.45 |
| alexcher | gc_signal and ticks_left in interp.c | 14:42.10 |
| | I can use 1 variable instead of 2 but it doesn't help. | 14:43.07 |
| Robin_Watts | so, if I'm following this correctly, gs_call_interp has a 'gc_signal' variable on the stack. We call set_gc_signal with the address of that, which has the effect of pushing a pointer to that into every memory space. | 14:48.57 |
| alexcher | yes, for the current context. | 14:49.50 |
| chrisl | Shouldn't gc_signal be part of the interpreter context, then? | 14:50.09 |
| Robin_Watts | So presumably something then indicates that a garbage collection is required by writing through one of those memory pointers into gc_signal. | 14:50.32 |
| | The while loop at the top of gs_call_interp then does an interp_reclaim. | 14:51.05 |
| alexcher | chrisl: the context is movable. | 14:51.40 |
| Robin_Watts | and, if I'm understanding correctly, you're saying that that reclaim can result in the gc_signal pointer stored in one of the spaces becoming invalid? | 14:52.01 |
| | I'm not sure I see how that is possible. gc_signal will exist on the stack, and therefore cannot be moved by gc, right? | 14:52.29 |
| chrisl | Don't we do a GC sweep when shutting down? | 14:53.12 |
| alexcher | Robin_Watts: yes. The call tree above can change depending whether we are running a string or a file. | 14:53.23 |
| chrisl | alexcher: is the graphics library context movable? | 14:54.08 |
| Robin_Watts | alexcher: Right, but gs_call_interp is only called from one place. | 14:54.11 |
| | and that place always does set_gc_signal(NULL) afterwards. | 14:54.25 |
| | So no space should keep a pointer to gc_signal outside of gs_call_interp. | 14:54.50 |
| | (well, gs_interpret, more accurately) | 14:54.58 |
| alexcher | Robin_Watts: unfortunately, the context can change before the cleaning up. | 14:55.09 |
| ray_work | chrisl: I don't think we do a GC when shutting down | 14:55.28 |
| Robin_Watts | alexcher: A different i_ctx_p might be used to set NULL than was used to set gc_signal initially? | 14:55.52 |
| alexcher | chrisl: yes | 14:56.08 |
| chrisl | ray_work: does the end of job restore happen while still in gs_call_interp()? | 14:57.32 |
| ray_work | chrisl: are you referring to the alloc_restore_all ? | 14:58.53 |
| | chrisl: sorry for the distractedness -- I am on the phone to scott | 14:59.39 |
| chrisl | ray_work: could be - I'm not sure. At the end of PS job, all VM should be returned to its original state - I assume a GC sweep happens at that point..... | 15:00.05 |
| Robin_Watts | alexcher: I can see that the value of i_ctx_p may change as this code runs. Are you saying that we might set_gc_signal(i_ctx_p, gc_signal) and have it push a pointer to gc_signal into memory spaces M1,M2,M3 say, and then change i_ctx_p in such a way that a subsequent set_gc_signal(i_ctx_p, NULL) will only clear it out of M1 and M2 say? | 15:01.25 |
| alexcher | chrisl: this file exits the server loop | 15:01.28 |
| | Robin_Watts: exactly | 15:01.57 |
| Robin_Watts | OK. | 15:02.23 |
| chrisl | alexcher: does the context really change *during* gs_call_interp()? | 15:02.40 |
| Robin_Watts | chrisl: I think so, yes. | 15:02.51 |
| | Look at gs_interpret. | 15:02.56 |
| | Note the repeated rereading of i_ctx_p = *pi_ctx_p; | 15:03.12 |
| chrisl | Robin_Watts: I know it can *move*, but it should be same context in a different location | 15:03.22 |
| Robin_Watts | So, can we change set_gc_signal to be more thorough about what spaces it writes to? | 15:03.34 |
| henrs | paulgardiner:is a meeting needed this time? The report looks pretty complete. | 15:03.49 |
| Robin_Watts | Or can we arrange to set_gc_signal(NULL) before we change the i_ctx_p so that the spaces change? | 15:04.16 |
| | henrs: I have nothing to add to paulgardiner's report (and no questions etc). Looks great to me. | 15:04.41 |
| paulgardiner | henrs: Can't think of anything I was intending to bring up. | 15:04.42 |
| henrs | paulgardiner:I am sort of interested in anything we can move to Robin_Watts or tor8 to accelerate the schedule but I don't want to sound like a broken record. | 15:05.03 |
| alexcher | Robin_Watts: current set_gc_signal has no access to other contexts. | 15:05.10 |
| henrs | tor8:did you have anything. | 15:05.29 |
| Robin_Watts | Right, but can we call set_gc_signal *before* the context changes, then again to reset it afterwards ? | 15:05.36 |
| henrs | where is that thunder god when you need him. | 15:05.48 |
| chrisl | alexcher: can't it access other contexts through the "top_of_system" pointer? | 15:05.56 |
| alexcher | chrisl: I need to check. | 15:06.43 |
| henrs | paulgardiner:excellent progress, thanks. | 15:06.47 |
| paulgardiner | ok cheers henrs | 15:07.07 |
| henrs | tor8:anything for the meeting, we've all decided we didn't have anything? | 15:07.25 |
| | oh my nick is wrong. | 15:07.37 |
| Robin_Watts | Oh, D'Oh. I have something. | 15:07.39 |
| chrisl | alexcher: I'm still bemused as to how the context can change (rather than just move) inside gs_call_interp()...... | 15:07.40 |
| tor8 | henrs: no, nothing new to report | 15:07.57 |
| Robin_Watts | We are presumably intending to do a release for mupdf alongside gs in August? | 15:08.10 |
| henrys | I hope | 15:08.24 |
| Robin_Watts | Are we intending to roll the forms branch back to master before that? So we can do a normal mupdf release and say "pre-alpha level forms support to give a taster of things to come" | 15:09.04 |
| tor8 | Robin_Watts: I don't think the new viewer will be in good enough shape by next release to do that though | 15:10.13 |
| Robin_Watts | The forms stuff is to a stage where people can actually see it working (albeit only in a limited way), and while there is stuff it doesn't do, it doesn't crash (touch wood). | 15:10.15 |
| henrys | are we risking open security issues? | 15:10.15 |
| Robin_Watts | henrys: Not in the existing binaries. | 15:10.31 |
| | Maybe in the new '+v8' binaries. | 15:10.41 |
| henrys | tor8:is this something sumatra will pick up? | 15:11.00 |
| tor8 | henrys: it might, or they could just say "screw it, if insane people want forms, just use acrobat" | 15:11.27 |
| Robin_Watts | Even if no one ever uses the forms stuff in this release, it's nice to have there for scott to start to use as a marketing tool. | 15:12.00 |
| tor8 | Robin_Watts: I think we can merge forms into master before the release, but I don't know if we want to build binaries with it unless they're in a usable form for end users | 15:12.49 |
| henrys | I'm always in favor of getting stuff early but I do see talk of indeterms and segv's so I'm not so sure there aren't security issues - I mean a buffer overflow is a security problem. | 15:12.51 |
| Robin_Watts | tor8, paulgardiner: The existing viewer (mupdf-v8) does forms though right? (On windows at least, and in a cheap'n'cheezy way on linux too?) | 15:12.52 |
| | henrys: We have 1 indeterminism, and no SEGVs. | 15:13.11 |
| tor8 | gtk+ and gui programming is demotivating :( but I do hope to make some visible progress soon... | 15:13.48 |
| Robin_Watts | but like I say, that's only for the forms specific builds. | 15:13.50 |
| | (i.e. us rolling the forms stuff back to master doesn't suddenly pollute all the existing binaries with security issues) | 15:14.27 |
| paulgardiner | Robin_Watts: I'm sure the linux app opens a dialog when you click on a field. | 15:15.29 |
| | "not sure" I mean :-) | 15:15.40 |
| Robin_Watts | I thought it asked at the command line ? | 15:15.45 |
| | I thought sebras/tor had made that work (no dialogue box, but you can enter text in the shell you launched the program from) | 15:16.33 |
| paulgardiner | Robin_Watts: Possibly. I've made no changes to the linux app, and I did need to make changes to the windows one | 15:16.35 |
| | Oh right. I was just basing my worry on my not having added anything. That's good then | 15:17.09 |
| Robin_Watts | Even to give early visibility on 1 platform would be worthwhile IMHO. | 15:17.24 |
| henrys | I guess I'm for having the source available in August. Fodder for the news letter. | 15:18.02 |
| paulgardiner | The forms branch does change code that is built even in the non-forms build, so might be an idea for someone to cast an eye over it. | 15:18.38 |
| Robin_Watts | paulgardiner: That could happen fairly naturally as someone merges it down to the trunk. | 15:20.00 |
| henrys | tor8:I assumed you would have asked the sumatra folks if they were going to use it. It would be cool if they had it working in there popular reader. | 15:21.35 |
| | are we all good then with Robin_Watts suggestion of rolling back to master and me adding something to the August newsletter? | 15:23.21 |
| Robin_Watts | I'd like to see us at least try for it (with the option to bale out, if people spot stuff that wants refactoring a bit before it gets merged down) | 15:25.56 |
| paulgardiner | Robin_Watts: after this, would we still need to keep a separate forms branch? | 15:25.58 |
| Robin_Watts | paulgardiner: Possibly not. Unless you felt it was easier to hack on that (less worries about changing interfaces etc?) | 15:26.43 |
| paulgardiner | Yeah. Might be better. We have changed are minds about some of the interfaces during development. | 15:27.21 |
| | I guess we'll want to put warnings on the current interfaces that they may be liable to change. | 15:28.20 |
| Robin_Watts | paulgardiner: Yeah. | 15:29.04 |
| | No comment from tor8 about how he feels about trying to get the forms stuff into a release... | 15:32.11 |
| tor8 | how much bigger is the -v8 binaries? | 15:33.08 |
| Robin_Watts | tor8: I was imagining them as a separate download (if supplied as binaries at all) | 15:35.47 |
| | Debug: mupdf.exe = 9Meg, mupdf-v8.exe = 12 Meg | 15:36.36 |
| tor8 | as I think I mentioned earlier, I do want us to merge to master. but whether or not to build binaries for relesae is a different question. | 15:36.47 |
| Robin_Watts | sorry, I missed that comment completely. | 15:37.15 |
| henrys | I didn't think we were considering binaries at this point. | 15:38.42 |
| | with that sort of size difference and the infrequent use of forms we should consider a shared library setup, no? | 15:39.44 |
| chrisl | How many different (binary incompatible) versions of v8 are there out there? | 15:41.19 |
| tor8 | another problem with v8 is we can't use it on iOS (and maybe not on android either, depends on how sandboxed android is w.r.t. running JIT code) | 15:44.27 |
| Robin_Watts | henrys: I think shared libs are an issue for any potential customer. | 15:46.50 |
| | tor8: For iOS we could potentially try to link into the nitro engine that's already there. | 15:47.44 |
| henrys | where an issue == a problem? I don't follow. | 15:48.04 |
| Robin_Watts | henrys: I mean that we could jump through hoops now to make shared lib builds work, but as soon as a customer turns up, they'll be using a different toolchain, on a different system, and we'll have a whole new set of hoops to jump through | 15:50.04 |
| | I don't see it as something that we gain by worrying about now. | 15:50.36 |
| henrys | fair enough. | 15:51.20 |
| chrisl | Robin_Watts: I think that cups bug (693176) should be mvrhel_laptop's (as it was originally, I think) - especially based on my findings...... | 15:56.10 |
| mvrhel_laptop | hmm... | 15:56.36 |
| chrisl | mvrhel_laptop: sorry about that..... | 15:56.48 |
| Robin_Watts | chrisl: woo hoo! :) | 15:57.36 |
| ray_laptop | our old friend RGBW space ? | 15:57.40 |
| mvrhel_laptop | so specifying -IResource/Init causes it to not work correctly? | 15:58.27 |
| chrisl | mvrhel_laptop: yes, for me anyway..... | 15:58.47 |
| mvrhel_laptop | oh but depending upon the clist too | 15:59.14 |
| chrisl | mvrhel_laptop: yes, if you then push the bitmap memory up, so there's no clist, it works again | 15:59.43 |
| henrys | meeting time. | 16:00.24 |
| mvrhel_laptop | sounds like the icc dir must be confused | 16:00.25 |
| chrisl | mvrhel_laptop: I couldn't see how that would affect the clist. | 16:00.41 |
| mvrhel_laptop | I agree. I don't think I am going to be able to get this fix in before the freeze if I have any hope of getting my stuff done for customer 330 | 16:01.44 |
| Robin_Watts | mvrhel_laptop: If I dig myself out from the interpolated image slowness thing I can try to triage if it you want (unless chrisl has completely done that) | 16:02.54 |
| henrys | alexcher when will we have a branch? | 16:03.48 |
| chrisl | mvrhel_laptop, Robin_Watts: I'll take another look at it tomorrow, see if I can narrow things down a bit. At least I'm moderately familiar with devcups | 16:03.48 |
| henrys | chrisl:I am sort of thinking we should have a freetype branch also. What do you think? | 16:04.26 |
| alexcher | henrys: we do | 16:04.30 |
| chrisl | henrys: I have a branch for the FAPI work, yes | 16:05.03 |
| henrys | alexcher:right mooscript - but I mean with all your updates. | 16:05.04 |
| marcosw_ | sorry I'm late | 16:05.13 |
| henrys | chrisl:public? | 16:05.20 |
| chrisl | henrys: no, not yet | 16:05.31 |
| henrys | sorry I though public was assumed. | 16:05.56 |
| | s/though/thought | 16:06.03 |
| | I was just looking back at the project list from the tech agenda. If we could get some of those projects out in the open we could spare discussing them at the meetings. | 16:07.29 |
| chrisl | Oh, okay - so no, not public. I haven't been in any rush to "go public" since I'm sure I'm not going to be ready for 9.06 | 16:08.05 |
| henrys | I was thinking public for colleague feedback only. | 16:08.46 |
| Robin_Watts | Having a public branch does not necessarily mean locking yourself down to not being able to rebase etc. | 16:08.47 |
| | You can easily have your own gs repo on casper (just clone the normal one into ~chrisl/repos/) and then push your stuff to that. | 16:09.28 |
| | People can get visibility of what's going on either by pulling from that, or by looking on the gitweb interface. | 16:09.56 |
| | but you aren't locked down to not being able to rebase/change history etc. | 16:10.13 |
| chrisl | Robin_Watts: I already have a private repo on casper, I just haven't been that bothered about using it for this project | 16:10.22 |
| henrys | IMHO, either you finish the project in a month or less or it should be public. Anything over a month is significant and should be visible and tracked but just my opinion as I said. | 16:10.39 |
| Robin_Watts | chrisl: Sorry, that wasn't aimed just at you - more as information for everyone. | 16:11.11 |
| chrisl | henrys: I'm not too fussed - I can make it public shortly | 16:11.34 |
| henrys | okay great. | 16:12.10 |
| chrisl | henrys: BTW, I ran the pcl5efts through with GS_FAPI/UFST (the "new" interface code), and it very nearly worked - some missing glyphs I need to investigate | 16:13.21 |
| henrys | other opinions on that issue? I think it would help us to make a policy: significant changes, public branches? | 16:13.53 |
| mvrhel_laptop | sounds reasonable to me. it makes things safer from a disk crash etc on large projects | 16:15.02 |
| Robin_Watts | but not necessarily on the main casper repo. | 16:15.14 |
| | (There are lots of advantages to not 'freezing' a branch, especially when you're still figuring out the best way to approach a problem, and git lets you rewrite history in some very clever ways) | 16:16.05 |
| henrys | my question to the git experts are we entering some sort of git hell with this policy. I don't think we are but git surprises me. | 16:17.10 |
| Robin_Watts | henrys: I don't see why we should be. | 16:17.24 |
| | If you change your mind about something in a public branch you can always force update it. | 16:18.12 |
| chrisl | henrys: as Robin_Watts mentioned, I think it would be preferable for these branches to be in individual repositories, rather than all hanging off the "central" one | 16:18.41 |
| henrys | so I'd say ray_work smart clist tile cache would be a candidate for a branch also. | 16:18.47 |
| tor8 | public for colleague feedback is what the ~/repos/ directory is set up for on casper | 16:18.47 |
| Robin_Watts | We choose not to do that on the main casper repo (unless there is a cockup) as a matter of policy, but there is no reason why people can't do what they want to their own casper repos. | 16:18.55 |
| ray_laptop | Robin_Watts: yes, I agree. | 16:19.07 |
| mvrhel_laptop | and does cluster push testing work fine with this work flow? | 16:20.13 |
| Robin_Watts | If you cluster push something on a branch then it will compare it to the last trunk run. | 16:20.39 |
| | (The cluster doesn't care what branch you are on) | 16:21.04 |
| mvrhel_laptop | ok | 16:21.09 |
| ray_laptop | right. I clusterpush from local branches all the time. The only difference, AFAICT is where it goes when I git push the branch (goes to the published branch repo), right ? | 16:22.16 |
| henrys | I knew 0 about ~/repos was I sleeping? | 16:22.16 |
| Robin_Watts | henrys: Look at: http://git.ghostscript.com/ | 16:22.51 |
| | You'll see that various people have created a ~/repos dir, and anything there appears (by the magic of softlinks) on the main git page. | 16:23.38 |
| henrys | so I use ~/repos and how do I say rebase changes onto ghostpdl.git when I'm ready? | 16:23.45 |
| Robin_Watts | henrys: mkdir ~/repos. Then git clone whatever ~/repos/ghostpdl.git | 16:24.35 |
| | Then on your local machine: git add remote henrys@ghostscript.com:/home/henrys/repos/ghostpdl.git | 16:25.07 |
| | oh, urm... | 16:25.17 |
| | git add remote personal henrys@ghostscript.com:/home/henrys/repos/ghostpdl.git | 16:25.25 |
| | then you can: git push personal blah just like you would git push to origin. | 16:25.53 |
| | I find it a useful tool for moving commits from my windows box to my mac/linux for testing - push the commit onto my personal repo, then git pull onto the target machines. | 16:26.31 |
| ray_laptop | Robin_Watts: what's the fastest way to clone on casper (for "Then git clone whatever ~/repos/ghostpdl.git") ? | 16:27.02 |
| henrys | but the merging across repos how does that go? | 16:27.11 |
| Robin_Watts | You don't merge across repos. | 16:27.26 |
| | You pull changes into your repo, merge there, and push the merged thing out. | 16:27.50 |
| ray_laptop | don't you just push to 'origin' when you are ready ? | 16:27.50 |
| mvrhel_laptop | that is what I would think | 16:28.19 |
| Robin_Watts | ray_laptop: 'origin' tends (but doesn't need) to be the master repo. | 16:28.26 |
| henrys | oh I see what you are doing. | 16:28.31 |
| ray_laptop | Robin_Watts: right -- once you've completed work on a branch, you push to origin | 16:28.54 |
| henrys | so you do have to do quite a bit on casper which might be a bit awkward for the windows folks. | 16:28.56 |
| Robin_Watts | ray_laptop: Indeed. | 16:29.00 |
| | henrys: Not really. | 16:29.11 |
| | Suppose michael has a local branch on his windows box, on which he is working. | 16:29.33 |
| henrys | you have to login to casper and rebase you repo before pushing right? | 16:29.44 |
| Robin_Watts | all we are asking is that periodically he does: git push personal branchname | 16:29.53 |
| | Or, if he's done history modification: git push -f personal branchname | 16:30.15 |
| | That will send all the history of branchname up onto his personal repo on casper. | 16:30.35 |
| | No need for him to ever do anything on casper. | 16:30.42 |
| | (other than the initial setup) | 16:31.01 |
| ray_laptop | henrys: why ? If you've done pull --rebase on your local branch, pushing that to the casper ~/repos/ git takes care of it without logging into casper | 16:31.02 |
| mvrhel_laptop | If I keep my branch updated with the commits from origin when I push to the personal it should keep that up to date yes? | 16:31.16 |
| Robin_Watts | Whenever you push, it will attempt to send exactly what you have on your machine out. | 16:31.41 |
| chrisl | Robin_Watts: can you create the remote repo without logging into casper? Or is that the one thing that does need done on there? | 16:31.51 |
| henrys | ray_laptop:okay I see, I thought you'd have manually rebase the personal report against ghostpdl.git but I guess that is not the case. | 16:32.01 |
| Robin_Watts | It may object if you've done a rebase locally, because it can't trivially tell that you won't be losing commits by on the remote by pushing to it. | 16:32.17 |
| | That's what the -f is for. | 16:32.21 |
| ray_laptop | is there a fast way to clone the repo on casper ? | 16:32.32 |
| Robin_Watts | ray_laptop: yes, hold on... | 16:32.40 |
| ray_laptop | Robin_Watts: OK. I just thought my ques. earlier got missed | 16:33.01 |
| chrisl | henrys: the thing to remember is you still only have *one* local repos, and all your doing is telling git to push and/or pull from different remote repositories. | 16:33.19 |
| | s/your/you're | 16:33.26 |
| mvrhel_laptop | just so I understand. If we need to have a branch on a large project, it should be in ~/repos not on the origin like we did a few times when we first moved to git | 16:33.54 |
| Robin_Watts | mkdir ~/repos; cd ~/repos; git clone --bare /home/git/ghostpdl.git ghostpdl.git | 16:34.00 |
| henrys | can't -f mask other problems? | 16:34.01 |
| Robin_Watts | henrys: Like what? | 16:34.12 |
| ray_laptop | Robin_Watts: thanks | 16:34.48 |
| chrisl | mvrhel_laptop: that's what we're talking about, yes. I'm not sure if henrys agrees (yet), though | 16:35.17 |
| Robin_Watts | "git push personal branchname" means "push branch branchname onto the personal remote". | 16:35.18 |
| | "git push -f personal branchname" means "Trust me, I'm a doctor. Just push branch branchname onto the personal remote". | 16:35.30 |
| henrys | yes exactly now say branchname is damaged. | 16:36.00 |
| Robin_Watts | Right, if you push a broken branch (one on which you've screwed a lot up) onto the remote, then that end will be broken too. | 16:37.02 |
| | You've still got the standard git escape clauses of being able to git ref-log and recover, but yes, you need to be sure that what you're pushing is sane before you push it. | 16:37.50 |
| tor8 | and correspondingly, if you push a fixed branch (one on which you've fixed a screw up) onto the remote, then that end will be fixed too. | 16:37.56 |
| ray_laptop | henrys: but you'll know your local branch is honked up, right ? | 16:37.58 |
| Robin_Watts | You can do things like: git push personal branchname:testbranch | 16:38.39 |
| ray_laptop | having the security of an offline personal repo appeals to me | 16:38.39 |
| Robin_Watts | to push 'branchname' up to 'testbranch'. | 16:38.53 |
| henrys | it's not offline. | 16:39.08 |
| kens | Hmm, well I think I'm off, nothing for me tonight ;-) | 16:39.23 |
| henrys | or you mean just a separate repo | 16:39.24 |
| | bye kens | 16:39.31 |
| ray_laptop | bye, kens | 16:39.37 |
| Robin_Watts | "a repo on a different machine" , I think he means :) | 16:39.44 |
| henrys | not convinced -f is good for those of us that have not reached "doctor" status. | 16:39.56 |
| Robin_Watts | You should only use -f if you know what you're doing, yes. | 16:40.16 |
| henrys | but I'll just get Robin_Watts to fix it when I screw it up ;-) | 16:40.32 |
| Robin_Watts | But if you've rebased your branch, then you'll need to use it. | 16:40.34 |
| | One day, I'll completely lose someone a couple of months work, and that way people will bug tor instead :) | 16:41.32 |
| chrisl | henrys: we should endeavor to never use "-f" on the main repos, but breaking your own repository is just a little embarrassing..... | 16:41.41 |
| Robin_Watts | Indeed. No -f on the main repo. Unless you're me, fixing an embarrassing mistake you made earlier :) | 16:42.10 |
| henrys | okay best to give it a go before judging, I have the xps driver code change I'll try to get that set up. | 16:42.24 |
| chrisl | Robin_Watts: that's why I said endeavor - in the real world, we know it needs doing occasionally | 16:42.52 |
| Robin_Watts | FWIW: The mupdf developers all have personal mupdf repos. Whenever we have a commit, we test it, and push it to our personal repos first. | 16:43.15 |
| henrys | I see alexcher has something. | 16:43.36 |
| Robin_Watts | We then get one of the other devs to look it over (review it) before we push to the main repo. | 16:43.36 |
| chrisl | henrys: gs_fapi branch is up on my repos on casper - sans the latest the changes for PCL/FAPI/UFST, which are still in progress | 16:43.46 |
| Robin_Watts | It seems to work well, as it prevents a lot of the instant "fix silly typo in previous commit" commits. | 16:44.32 |
| alexcher | henrys: my repository is still just a clone. | 16:44.43 |
| henrys | makes sense | 16:44.50 |
| chrisl | henrys: do you happen to know if the pcl5efts have any downloaded TTFs in them? | 16:46.08 |
| henrys | and why would we want to have multiple repos per user, instead of branches within the personal repo? | 16:46.31 |
| Robin_Watts | henrys: Who suggested that? | 16:46.55 |
| henrys | chrisl:yes they have all flavors of TTF downloads. | 16:46.59 |
| Robin_Watts | I have a mupdf repo and a ghostpdl one and a v8 one. | 16:47.35 |
| | but I can't see why I'd need more than one of each. | 16:47.47 |
| henrys | Robin_Watts:ah I see | 16:47.49 |
| chrisl | henrys: Oh, cool, in that case, I've also been testing TTF fonts with PCL/FAPI/UFST - with UFST rendering the TTF glyphs :-) | 16:47.52 |
| ray_laptop | henrys: if they are different trees, you need different repos, but other than that.... | 16:48.05 |
| henrys | Robin_Watts:okay I see I forgot v8 was outside mupdf. | 16:48.54 |
| Robin_Watts | On another topic: Shelly got a patch to me for the openjpeg stuff. He's tested it, and it looks fine to me. | 16:49.09 |
| | It seems that OpenJPEG doesn't ever check that a malloc worked. | 16:49.20 |
| ray_laptop | Robin_Watts: when I try the git add remote personal ... I get fatal: pathspec 'remote' did not match any files | 16:49.20 |
| henrys | Robin_Watts:if we were setting things up gs style with libraries in gs then v8 would be a branch inside mupdf.git, yes? | 16:49.31 |
| Robin_Watts | git remote add, not git add remote, sorry. | 16:49.33 |
| | henrys: Yes. | 16:49.42 |
| ray_laptop | Robin_Watts: OK, I was just cutting and pasting from above | 16:49.53 |
| Robin_Watts | v8 is a clone of googles v8 repo with an 'artifex' branch added with our changes. | 16:50.14 |
| ray_laptop | Hurray. it worked | 16:50.14 |
| Robin_Watts | ray_laptop: Yeah, my bad. | 16:50.20 |
| | So we'll get SEGVS if we ever run out of memory while decoding a jpeg 2k. | 16:50.56 |
| ray_laptop | and it even shows up at http://git.ghostscript.com | 16:51.00 |
| henrys | I really do like this much better, transparency during project development is something I think we lack and this does seem to fix it nicely. | 16:51.26 |
| Robin_Watts | He's looked at the OpenJPEG 2.0 code, and he reckons that's much better. Reading between the lines he reckons on a release at the end of the year. | 16:51.33 |
| henrys | alexcher:are yo going to integrate the 2.0 stuff? | 16:52.24 |
| | s/yo/you | 16:52.30 |
| alexcher | henrys: yes. | 16:52.54 |
| Robin_Watts | Shelly's indicated that he'd be interested in try to integrate the openJPEG 2.0 alpha, run it through testing etc if we wanted to avoid further burdening alex. | 16:53.09 |
| alexcher | henrys: do we need it for the upcoming release? | 16:53.23 |
| chrisl | When I spoke to him last, Shelly reckoned that the OpenJPEG 2.x API is *quite* different to the 1.5.x API | 16:53.41 |
| | alexcher: no, definitely not for 9.06 - we're not going to heap that on you! | 16:54.07 |
| alexcher | Shelly's help is always welcome. | 16:54.11 |
| henrys | Robin_Watts:it's hard for me "bounty" that. Is he proposing we pay him hourly? | 16:54.31 |
| | or chrisl? | 16:54.40 |
| Robin_Watts | henrys: pass. | 16:54.40 |
| | he hasn't mentioned money - that would be an ecumenical problem. | 16:54.58 |
| | (Do you guys get Father Ted?) | 16:55.18 |
| chrisl | henrys: he originally mentioned it as a bounty, but he may have revised his thoughts given the extent of the API change. | 16:55.24 |
| henrys | alexcher:if we do get shelley to do it you would still own it. | 16:55.32 |
| alexcher | henrys: OK | 16:55.54 |
| henrys | I'll talk to him offline about $ | 16:56.04 |
| Robin_Watts | alexcher: Any progress on bug 693115? | 16:56.30 |
| henrys | anyway I don't want to hold folks up longer. marcosw did we need to have a bug meeting soon? | 16:56.39 |
| | or do you think we are good? | 16:56.46 |
| Robin_Watts | That's a blocker for a customer bug. | 16:56.48 |
| | (oh, I've marked it as a customer bug anyway) | 16:57.00 |
| mvrhel_laptop | ok so I created a GhostPDL/mvrhel on casper in my repos directory. Now I guess I need to make it available from windows. Should I do a git clone from that repository or can I do a switch from one of my current checkouts of origin? | 16:57.19 |
| chrisl | henrys: given our experience the last time we outsourced the OpenJPEG stuff, at least with Shelly, I can drive up the road, and bang on his door ;-) | 16:57.20 |
| Robin_Watts | mvrhel_laptop: On your windows box... | 16:57.38 |
| alexcher | Robin_Watts: not yet. I've spent most of the week looking for the SEGV causes. | 16:58.04 |
| henrys | chrisl:right. | 16:58.12 |
| chrisl | henrys: are we a UFST customer? I mean, what level of tech support are we entitled to? | 16:58.30 |
| Robin_Watts | git remote add mvrhel@ghostscript.com:/home/mvrhel/repos/ghostpdl.git | 16:58.37 |
| | bah. | 16:58.48 |
| | git remote add personal mvrhel@ghostscript.com:/home/mvrhel/repos/ghostpdl.git | 16:58.53 |
| | (or whatever the series of clicks you need to do that in tortoise git is) | 16:59.22 |
| chrisl | For TortoiseGit: http://stackoverflow.com/questions/959365/push-to-remote-url-with-tortoisegit | 17:00.33 |
| henrys | chrisl:I've always gotten questions answered we've had some feature requests done others rejected - mixed... just ask them. | 17:01.44 |
| Robin_Watts | OK, if we're done, I have to pop out and reassemble a church... | 17:02.30 |
| chrisl | henrys: okay, thanks, can you e-mail me your contact, please? This isn't urgent - it's not blocking progress on this project | 17:02.48 |
| mvrhel_laptop | thanks for the help Robin_Watts | 17:02.56 |
| henrys | chrisl:will do. | 17:03.00 |
| mvrhel_laptop | and chrisl | 17:03.12 |
| chrisl | mvrhel_laptop: I just hid behind Robin_Watts :-) | 17:03.26 |
| | henrys: thanks | 17:03.32 |
| | mvrhel_laptop: just to confirm: you're going to leave the cups bug for me to have a closer look at tomorrow? | 17:04.28 |
| mvrhel_laptop | chrisl: if you have the time to look at it that would be great. I am working on this customer stuff which is going to take some time and I am out Wed-Fri this week | 17:05.05 |
| henrys | not up on Father Ted. Looks like it does play on one of my network channels though. | 17:05.17 |
| ray_laptop | I have to run... bbiaw. | 17:05.42 |
| chrisl | mvrhel_laptop: sure, np. I just didn't want to risk us both looking at it. | 17:05.47 |
| karl_heinz | hi there. I'm looking for some insight on licensing fees for use in a commercial product | 17:06.08 |
| mvrhel_laptop | thanks chrisl | 17:07.01 |
| henrys | chrisl:I forwarded the names I have to you on 10/1/2010 | 17:07.24 |
| | let me know if I should send again. | 17:07.46 |
| chrisl | henrys: yes, so you did - it went to my spamcop address, that's why I didn't find it before. Thanks! | 17:09.15 |
| | karl_heinz: the guy to contact is: Scott Sackett sales@artifex.com - see http://www.artifex.com/ | 17:09.21 |
| mvrhel_laptop | ok. I think I now have my repos set up on casper and can push to it if needed. my current project should take less than a month though, so no branches yet | 17:13.14 |
| karl_heinz | OK, I have one more question. I'm interested in ghostscript in conjunction with a web app that gathers data from user, fills the PDF and convert that PDF to images (currently done with PDFBox, but there are numerous font issues). Do I need a license? | 17:14.25 |
| | This is a commercial product | 17:14.44 |
| chrisl | karl_heinz: I'm not a lawyer, nor a GPL specialist, so I really wouldn't want to give an opinion...... sorry | 17:16.57 |
| henrys | karl_heinz:best to talk with Scott and he'll guide you to the best solution. | 17:17.20 |
| karl_heinz | alrighty, I'll do that | 17:17.28 |
| henrys | mvrhel_laptop:I didn't mean we should require a branch for any project I just wanted them to be available and thought the public branch was one way to do it. I didn't know about the public repos | 17:20.41 |
| mvrhel_laptop | henrys: no worry. I just wanted to get this thing set up now, while everyone was talking about it. rather than 3 months from now when I would be scratching my head trying to figure it out | 17:21.41 |
| | I still need to get set up to do the cluster pushes directly from windows. I still do the trick that ken does which is to bring a patch over to my linux vmware checkout | 17:22.46 |
| | which is fine in a way as it makes me keep the linux checkout up to date | 17:23.05 |
| | and sometimes things dont' build or have problems there and not on windoze | 17:23.23 |
| henrys | I can't remember what prevents us from having a simple windows batch file, is it rsync that doesn't work? | 17:24.31 |
| mvrhel_laptop | henrys: I am not sure what was done to make it all work, but I know ray does it | 17:25.41 |
| | that is cluster pushes directly from msys | 17:26.05 |
| | marcosw: you there? | 17:27.24 |
| | why did you change 692892 to a blocker? | 17:27.57 |
| | it should be resolved now | 17:28.03 |
| chrisl | henrys: yes, there isn't a fully functional, Windows native rsync | 17:28.54 |
| mvrhel_laptop | not sure about 693085 though | 17:29.08 |
| | henrys: have you looked at 693085 by chance? | 17:29.19 |
| | since it is the pxlcolor device | 17:29.36 |
| chrisl | henrys: FWIW, I clusterpush from cygwin quite regularly....... | 17:29.46 |
| henrys | chrisl:yeah that worked for me too, but other have had problems. | 17:30.15 |
| mvrhel_laptop | cygwin: ick | 17:30.29 |
| chrisl | cygwin is better than msys - at least cygwin is a "full" Unix style environment | 17:31.29 |
| henrys | mvrhel_laptop:with cygwin installed you should be able to run clusterpush.pl from the dos command line. | 17:31.46 |
| | should never have to deal with cygwin directly. | 17:32.10 |
| chrisl | it might need some fancy footwork with paths for that to work. | 17:32.43 |
| henrys | mvrhel_laptop:we just need to lower the priority on that as I recall. | 17:32.59 |
| mvrhel_laptop | ok. marcos just set it up as a p2 blocker | 17:33.20 |
| chrisl | right, off to pretend I can play squash for while. 'Nite folks | 17:33.42 |
| henrys | mvrhel_laptop:probably not something suited for bounty? | 17:35.25 |
| | although hin tak might be able to come up with something. | 17:36.26 |
| mvrhel_laptop | I should at least see if I can replicate it | 17:38.04 |
| | I need to run and errand right now. I will take a quick look at it this afternoon | 17:38.25 |
| ray_work | henrys: mvrhel_laptop: with the msysgit installed you can also run toolbin/gitpush.sh from the command line (I do all the time) | 17:42.17 |
| | henrys: iirc, I've asked mvrhel and he has msysgit | 17:42.36 |
| | chrisl_away: I don't agree that cygwin is better. I had many problems trying to get it installed -- kept having to go an add stuff. With msysgit the only other thing I got was an X server | 17:48.00 |
| henrys | ray_work:Robin told me the same thing about cygwin - for me it has worked perfectly. | 17:56.57 |
| | granted I don't use windows frequently. | 17:57.45 |
| scottb_ | exit | 18:33.12 |
| Robin_Watts | back. | 18:55.28 |
| | mvrhel_laptop: (For the logs) There is an easy way to do clusterpushes from windows. I set up a git based 'bridge'. | 18:56.10 |
| | Let me know when it's convenient and I'll talk you through setting it up. It is really easy. | 18:56.32 |
| karl_heinz | Can I expect to pay a lot for the commercial license? | 19:30.48 |
| henrys | karl_heinz:really you are going to have to speak with scott - everyone here works in engineering/development and we try not to get mixed up in that stuff. | 19:43.14 |
| mvrhel_laptop | Robin_Watts: I am back now | 19:43.35 |
| | if you want to walk me through the push from windows | 19:43.46 |
| | oops need to head out again for a bit | 19:52.23 |
| scott_b | I wrote a viewer two years ago which used an old version of mupdf to support viewing pdf files. I hacked in a routine so that when an image had excessive dpi, it would read the image data a band at a time and scale it down. This made some edge cases where the program would consume close to 2 GBs of ram work much better. I'm in the process of updating to mupdf 1.0 and need to recreate this modification. I think I've found a place to perform this in the fl | 20:17.48 |
| | If there is interest, I would like to work with someone on this so the patch can be considered for merging. The patch would involve adding support for a scale factor to PDF_IMAGE_FAX type of stream. (see pdf_open_image_decomp_stream in pdf_stream.c) | 20:17.54 |
| tor8 | scott_b: Robin_Watts is the man to talk to about the image handling in mupdf, he's the one most knowledgeable about it | 20:21.49 |
| ray_work | karl_heinz: I assume you know that the "Scott" you need to contact is Scott.Sackett@Artifex.com | 20:33.40 |
| | karl_heinz: and just so you know, GPL allows for commercial use (as long as you meet the GPL v3 terms) | 20:35.12 |
| | karl_heinz: We _do_ have customers that use the GPL version, but still get support contracts from us. It all depends on what you are doing and what you need which is what Scott is there for (to try and figure it out) | 20:36.37 |
| scott_b | Robin_Watts: It looks like the factor param during the process of reading an image stream is a new feature and is only implemented for images of type = PDF_IMAGE_JPEG. I would like to help add that support for type = PDF_IMAGE_FAX. | 20:41.20 |
| Robin_Watts | scott_b: Hi. Just got back. Let me read the logs. | 21:33.48 |
| | scott_b: Right. We've added, as you've seen a downsampling factor for images already to the 'core' of mupdf. | 21:35.02 |
| | Each of the image decoders is free to implement it or not as it sees fit. | 21:35.14 |
| | Currently the only one that uses it, again as you've spotted, is JPEG. | 21:35.38 |
| | If you'd like to make the modifications for other image formats then we'll gladly take them on. | 21:36.01 |
| | One possibility would be to implement a generic downscaler filter and to hook it into pdf_open_decomp_stream. | 21:38.57 |
| | (so if factor 'f' is requested, and the image decoder type can only supply factor 'g' (where g < f) we could stick a 'downsample by factor of f-g' downsampling filter on the stream. | 21:40.01 |
| scott_b | So that would not be a catchall but a tool so that modifying a decoder to support scaling would be easier (less duplicated code)? | 21:40.24 |
| Robin_Watts | no, it would be a catchall. | 21:40.44 |
| scott_b | Will it still accomplish the goal of conserving RAM? | 21:41.03 |
| | Shrink as the data is being read? | 21:41.23 |
| Robin_Watts | Assuming all the image formats don't create a full image size buffer internally, sure. | 21:41.37 |
| scott_b | instead of after the entire image is loaded? | 21:41.47 |
| Robin_Watts | (And only jpx does that, and that's a special case anyway) | 21:41.49 |
| | The whole compressed image data will still be loaded. | 21:42.03 |
| scott_b | That's what it's doing now. Perhaps it's easily avoidable. | 21:42.06 |
| | The current fax code does it. | 21:42.23 |
| Robin_Watts | but as the filters return it a line at a time, we'd buffer just enough lines and subsample them. | 21:42.26 |
| | scott_b: Does it? | 21:42.33 |
| scott_b | Great. That would work. We'll both have to understand the current flow to see what it would take. | 21:42.58 |
| Robin_Watts | I didn't write the fax decoder, but it looks to me like it only allocates 2 line buffers. | 21:43.27 |
| | and it decodes a line at a time, and passes the pixels from that out as a stream. | 21:43.46 |
| scott_b | Yes, that is the nature of group4 decompression. Reference line. | 21:43.57 |
| Robin_Watts | The core of mupdf then allocates a full size bitmap for it to be read into, and that's where the whole bitmap comes into it. | 21:44.09 |
| scott_b | I'm sure you are right about the filter. You understand the flow way better than I. | 21:44.23 |
| Robin_Watts | What we'd do is still decode the full lines, and pass them out. | 21:44.34 |
| | We'd then have another filter that reads in (say) 4 lines, and shrinks it to a single line and passes that out. | 21:44.58 |
| | That way the full buffer that the core holds will be 1/16th the size. | 21:45.14 |
| scott_b | Yes. a max of 8 (like jpg) sounds good to me. I can only forsee the need from 1 to 4. | 21:46.03 |
| Robin_Watts | My experience of doing exactly this for another project is that either we need to be very careful with subsampling fax decoding (as thin black on white lines very quickly disappear) or we need to limit how far we shrink them. | 21:46.36 |
| | We may need to consider a special downsampler for fax that avoids dropouts. | 21:47.00 |
| | but that's a refinement we can worry about later. | 21:47.10 |
| scott_b | Yes. I've done it three ways: | 21:47.24 |
| Robin_Watts | mvrhel_laptop: Hi. I've just got back, and I'm pretty knackered. | 21:47.25 |
| | I sent an email to tech the other day about the windows cluster push stuff. | 21:47.38 |
| | If you want to have a look at that to get you started, can we talk through any problems you hit tomorrow ? | 21:47.53 |
| mvrhel_laptop | Robin_Watts: no worries. I will look it over | 21:48.03 |
| Robin_Watts | mvrhel_laptop: Fab. | 21:48.11 |
| mvrhel_laptop | no the rest of the week I am out | 21:48.12 |
| Robin_Watts | oh, right, well when you get back then. | 21:48.24 |
| mvrhel_laptop | sounds good | 21:48.30 |
| | thanks | 21:48.32 |
| Robin_Watts | no worries. | 21:48.36 |
| | have fun in NC. | 21:48.41 |
| | (What happens in NC, stays in NC etc) | 21:48.53 |
| | scott_b: Sorry, carry on. We get very used to having multiple conversations going on at once on irc :) | 21:49.27 |
| mvrhel_laptop | yes. off to get a beer in a bit | 21:50.32 |
| Robin_Watts | scott_b: I'm in the UK, and it's 11pm here, so I'm going to head off. | 21:52.22 |
| scott_b | Monochrome downscaling techniques: OR black pixels (Did that exclusively for 20 years, don't use that one anymore) #2 consider the pixels grayscale and use a box filter to reduce the res (fastest and highest quality. It's what I currently use for display.) #3 when resaving to group 4 format, I perform #2 and then use floyd-steinberg dithering to return back to 1-bit data | 21:52.50 |
| Robin_Watts | Feel free to type here and I'll read it later, or drop me an email: Robin.Watts at artifex.com or check back in with me tomorrow. | 21:53.20 |
| scott_b | If the data is being considered grayscale or color by mupdf then we should use #2. | 21:53.34 |
| | OK, goodnight. | 21:53.39 |
| Robin_Watts | scott_b: Right. 2) is the obvious one that I used at my last job. | 21:53.40 |
| | but it means that the moment you are scaling down by more than a factor of 2 in each direction, text and single pixel lines are almost unreadable. | 21:54.24 |
| scott_b | Whenever it's being restored, #3 gives consistent good results. It's pretty fast too. But if it's being considered grayscale then it's a waste. | 21:54.54 |
| Robin_Watts | scott_b: It's being considered greyscale. | 21:55.07 |
| | (In gs, in the tiffscaled device I use FS when going to 1bpp, and it does indeed give nice results) | 21:55.36 |
| scott_b | The only time I run into the need to scale down is when the res is way higher than needed. Which means that line start thick and fonts are rather huge (typically). | 21:56.09 |
| Robin_Watts | Ah. I've seen cases (particularly of electronics diagrams) where boxes are all 1 pixel thick. | 21:56.39 |
| | and text is still small. | 21:56.46 |
| | Genuine faxes tend to be fine. | 21:57.09 |
| | but fax compressed artifical images often suffer horribly through downscaling. | 21:57.31 |
| | But, we can experiment when we have the basic infrastructure in place. | 21:57.47 |
| | I really have to go though, so ttyl! | 21:57.57 |
| scott_b | Good night. (for morning reading) The only way to handle one pixel lines when the res is way too high would be to OR the black. That way one pixel will be the minimum. That should not be the default though because of cases: a large scanned drawing has background noise, significant areas use a dot pattern fill: ORing the black makes these cases too dark or go solid black when they shouldn't. | 22:05.35 |
| | But the question is, what is the code currently doing? Right now, it will read in the fill res image, and later in the process, it will be scaled down to the output rendering resolution. Whether the image fills the page or is just part of the output pdf page, when it's placed, it's shrunk down. It seems that it's doing a greyscale box filter (based on not seeing any signs or ORing black or dithering in the current output). If the current flow is using a | 22:10.00 |
| | I meant it will read in the *full res image... | 22:14.41 |
| | If I'm right, then we have only one choice but to just use a box filter. Unless a larger directive is added such as a parameter to fz_new_draw_device ( ctx, image ) at a high level so that an app using mupdf will have the ability to specify 'ORing black pixels when downsampling monochrome' or 'we want monochrome' (in which case it FS just like GS does). Adding such a parameter would be a separate project but the filter we add for this should respect such | 22:19.56 |
| | Another enhancement I need to add is to detect when the page consist of nothing more that a single image, what the original dpi is, and is it monochrome. (since my app re-saves an image file routinely.) Having this info will allow my app to know when I should use FS and save as a group4, when I should reduce the target dpi (because it's not available in the source) and even when I can grab the raw group4 data and short-circuit the rendering process, grea | 22:30.46 |
| | But my first priority is to get this to not hog memory so I can release an updated product (by the way, my whole viewer is GPL and open source). | 22:31.56 |
| Robin_Watts | scott_b: back again, just on my way to bed. | 23:05.41 |
| | At the moment, we take the full resolution image, and then feed it through a scaler. It's not a box filter, but rather a mitchell filter (see Graphics Gems 3, Chapter I.2 for exact details) | 23:06.49 |
| | What I am proposing is that we should first 'subsample to a nearby factor' then still scale like that as we do now. | 23:07.28 |
| | The 'subsample to a nearby factor' is free to be any mechanism we want. For JPEGs we can use the inbuilt reduction from the jpeg lib (effectively ignoring the higher order DCT components and simplifying the DCT). | 23:08.19 |
| scott_b | The schematics with super thin lines, is that bi-level or color? | 23:08.23 |
| Robin_Watts | Every image in mupdf ends up as being 8 bits per component. | 23:08.47 |
| scott_b | If there was a schematic that you want to render to 200 dpi and then further shrink to the screen for display, if it was 800 dpi and had single pixel lines, you have a challenge. | 23:09.58 |
| Robin_Watts | scott_b: indeed. | 23:10.08 |
| | The best we could hope is that we'd go from 800 -> 200 with the black lines staying black. | 23:10.34 |
| scott_b | ORing the black handles this well but only for bi-level input data. if you had a background color and different color lines, you need something trickier than ORing black | 23:10.46 |
| | That's why I asked if the input was bi-level. | 23:11.16 |
| Robin_Watts | Then the scale to display would make the black grey. | 23:11.20 |
| | Well, all fax output is bi-level. | 23:11.28 |
| scott_b | Yes, a very light gray. | 23:11.40 |
| Robin_Watts | Ideally, the arbitrary scale phase would never be more than 1/2 in size. | 23:12.12 |
| | so black would go to mid-grey at worst. | 23:12.20 |
| scott_b | true. but since this is pdf input, it could be other than fax. and the filter idea is to try to cover more than fax | 23:12.22 |
| Robin_Watts | right. | 23:12.29 |
| | but I'm suggesting that once we get the arbitrary filter working, we might do a modified version just for fax. | 23:12.47 |
| scott_b | I did a side project recently which used ImageMagick to scale down different images. and I tested all available filters | 23:13.23 |
| | found that box filtering fastest, and suprisingly the highest quality. | 23:13.47 |
| Robin_Watts | I did a similar imagemagick vs mupdf comparison when I implemented the mitchell filter. | 23:14.08 |
| scott_b | I concluded that all the other filters were designed to maximize quality while enlarging but that box filtering was the best for shrinking | 23:14.24 |
| Robin_Watts | and I found that mupdf beats or at least matches imagemagick. | 23:15.42 |
| | certainly for downscaling, the difference between box and mitchell is much smaller. | 23:16.03 |
| | The key thing is that every input pixel should count somewhere. | 23:16.17 |
| scott_b | Have you used ORing black before? | 23:16.45 |
| Robin_Watts | not exactly. | 23:16.58 |
| | and I think we can do better than simply orring black. | 23:17.17 |
| | If we are shrinking by a factor of 'f' in each direction, there are f*f pixels within each block we are subsampling. | 23:18.31 |
| scott_b | I only use box filtering but If I had very high res bi-level with single pixel lines, ORing would be exactly what you want (as long as they weren't using absurdly small fonts) | 23:18.38 |
| Robin_Watts | We count the number of black pixels within the block (call that x) and if x >= f it's black. If x < f then we output a pixel value of x/f. | 23:19.42 |
| | (or if you'd prefer, we always output (min(1, x/f)) | 23:20.00 |
| | That way a black line across a block is sufficient to keep it black. | 23:20.27 |
| | It should give an effect halfway between orring black and doing a naive downsample (which would be to output x/(f*f)) | 23:21.48 |
| scott_b | Interesting approach. effectively a variable threshold ( 50% for f=2, 25% for f=4, etc ) | 23:21.50 |
| Robin_Watts | Yeah, that's my first thought at least. We may need to tune it. | 23:22.15 |
| scott_b | Would you use this even though the output is greyscale? | 23:23.00 |
| Robin_Watts | The output from the fax stream is a series of 0 or 255 bytes. | 23:23.26 |
| | The output from the subsample filter would be a series of bytes with values from 0 to 255. | 23:23.47 |
| scott_b | One pixel per byte or Eight Pixels per byte? | 23:23.58 |
| Robin_Watts | One pixel per byte, | 23:24.08 |
| scott_b | So greyscale. | 23:24.23 |
| Robin_Watts | All mupdf image filters give out 1 byte per component. | 23:24.25 |
| | The fax output stream is 'greyscale' in the sense that it's 8 bit, but only 0 and 255 are ever used, so it's really mono. | 23:25.04 |
| | The downsampled fax output would be proper greyscale though. | 23:25.20 |
| scott_b | Does it return a one component image because it came from bi-level or does it return three component rgb because that's what the caller expects? | 23:25.50 |
| Robin_Watts | It returns 1 component image because it came from bi-level. | 23:26.11 |
| scott_b | Well that's a nice optimization to start with. | 23:26.31 |
| Robin_Watts | When the stream gets painted, it gets converted to how ever many components are needed to plot it then. | 23:26.42 |
| scott_b | Which ever downsampling filter we use, 8-bit greyscale will be fine. | 23:27.04 |
| Robin_Watts | scott_b: We can write one generic downsampling filter to start with (that just does the naive x/(f*f) subsampling) and that will work for everything. | 23:28.16 |
| | but then we can optionally do a tweaked one just for fax that uses min(1,x/f) | 23:28.35 |
| | but I'm really going to bed now :) | 23:29.51 |
| | Night. | 23:29.54 |
| scott_b | Good night. | 23:30.02 |
| | Forward 1 day (to 2012/07/11)>>> | |
IRC Logs