| | <<<Back 1 day (to 2014/06/03) | 2014/06/04 |
| marcosw | mvrhel_laptop: made it home? | 04:09.39 |
| andremedeiros | Robin_Watts: Hi! You work for Artifex, right? | 08:36.49 |
| | Anyone from Artifex around? | 08:41.42 |
| pedro_pc | andremedeiros: the Artifex guys had a staff meeting in SF over the past few days and soem will still be on their way home | 08:47.23 |
| | Robin is probably flying just now | 08:47.49 |
| superdealloc | Ah, gotcha. | 08:51.24 |
| | Nothing urgent anyway, my company wants to sponsor a feature on MuPDF | 08:51.33 |
| pedro_pc | cool - if you want to raise it here then it'll be logged and the guys do check back through the logs. I suspect the US-based guys will be on later today in any case and maybe Robin too dependin how his jetlag goes ;) | 08:55.24 |
| superdealloc | Essentially, we want to sponsor the color management feature on MuPDF. | 08:56.36 |
| | We wanted to implement it ourselves, and got as far as plugging in LCMS2 on the predefined colorspaces, but that's not nearly enough (and, TBH, I couldn't make my way through the ghostscript codebase well enough to understand what goes on in there) | 08:57.24 |
| Zaister | Any mupdf developer present? | 09:56.35 |
| rayjj | Zaister: the mupdf developers are probably jetlagged to the max. They are in europe and just got home (at best) a couple of hours ago | 14:12.56 |
| | superdealloc: color management is quite tricky, and the first part is to determine what you *really* need. Ghostscript has "full blown" color management with proofing profiles supported as well as object type (text, image, vector graphics) specific device profiles, and uses ICC based color spaces even when painting transparency stack buffers | 14:15.08 |
| superdealloc | Yeah, I got that. | 14:15.43 |
| | Essentially I want to be able to render PDFs with MuPDF looking the same as Acrobat / Preview. | 14:16.04 |
| rayjj | superdealloc: so, what exactly does your company need ? | 14:16.06 |
| | superdealloc: OK, so for the most part, you DON'T need object specific device profiles, or proofing profiles | 14:16.54 |
| superdealloc | No, I don't think so. | 14:17.03 |
| | Essentially what happens is that the rasterized PDFs that mupdf produces are too saturated compared with Preview/Acrobat. | 14:18.17 |
| rayjj | but you DO need input color conversion of CMYK colors. I think you will be mostly there with that. I suspect that even transparency in the "correct" color space (instead of output colorspace) won't be needed on many files | 14:18.27 |
| superdealloc | Can I show you what I have? | 14:18.40 |
| rayjj | superdealloc: sure. The other thing is that mupdf (AFAIK) doesn't pay attention to transfer functions and that can muck up the colors. | 14:19.24 |
| superdealloc | https://gist.github.com/andremedeiros/75979fff98f5a8af85f7 | 14:19.29 |
| | I hijack ctx->colorspace->cmyk->to_rgb with this function. | 14:20.00 |
| rayjj | superdealloc: OK. And how close does just doing that get you ? | 14:20.40 |
| superdealloc | Very very close. | 14:20.48 |
| | But the reds aren't quite there. | 14:20.58 |
| | And that's a problem when there are pictures of people. | 14:21.04 |
| | (as in, the difference is enough to make it look like they've just been crying!) | 14:21.16 |
| rayjj | superdealloc: As I suspected, the main thing is that the CMYK colors with Adobe are quite different. | 14:21.37 |
| | superdealloc: there are MANY different CMYK ICC profiles. Picking the "right" one is key. | 14:22.24 |
| superdealloc | So, going back to my initial question. | 14:22.30 |
| | How would I go about sponsoring this? | 14:22.38 |
| rayjj | superdealloc: so having a clear spec is the first thing. And example files that can be compared to Acrobat Reader (and specifying what the Color Management preferences in Acrobat are the target) | 14:24.55 |
| superdealloc | I've got plenty of samples I can provide. | 14:25.14 |
| | Is there someone specific at artifex I should get in touch with? | 14:25.23 |
| | Or get it in bugzilla? | 14:25.26 |
| | Also, how does Ghostscript do this exactly? | 14:26.25 |
| rayjj | I think starting with an enhancement bug in bugzilla is a good starting place. Then we can have a look and scope it out. Maybe even find some people we've worked with that would want to take it on. | 14:26.37 |
| | superdealloc: well, first, ghostscript with the default CMYK profiles we use doesn't match Acrobat Reader all that closely, since that isn't a concern of ours. | 14:27.32 |
| superdealloc | Incidently it does. | 14:27.51 |
| rayjj | superdealloc: secondly ALL colors get converted from the input colorsace (Gray, RGB, CMYK, ...) to device colorspace (or transparency blending colorspace) using the CMS (which may be lcms2 or "nocms" for our UseFastColor option) | 14:30.25 |
| | superdealloc: and gs does transparency buffers in the correct colorspace (which I don't think mupdf bothers with) | 14:31.41 |
| | superdealloc: for the most part the transparency colorspace is not that important, but can be for our really picky customers that are making plates for printing presses | 14:32.49 |
| superdealloc | We're not doing stuff that is that advanced. | 14:33.06 |
| rayjj | superdealloc: which is why I pointed out that a clear spec is an important first step. Set the goals, otherwise our idea of 'color management' may not match what you want and can vary the scope of the project immensly | 14:34.18 |
| | immensely, even | 14:34.28 |
| superdealloc | Alright | 14:35.06 |
| | I'll work on that. | 14:35.09 |
| rayjj | superdealloc: and if the target is a printer, having the correct device profile is key. | 14:35.16 |
| superdealloc | Target is a screen. | 14:35.26 |
| rayjj | superdealloc: it doesn't need to be at all formal, and we'll probably ask questions via email or in the bug tracker | 14:35.56 |
| | superdealloc: in fact informal description is preferred :-) | 14:36.32 |
| superdealloc | I like that ;-) | 14:36.39 |
| rayjj | superdealloc: one point that we're sensitive to is the performance. For instance, gs goes to great lengths to cache link profiles (the transform) and also has an API to allow for the CMS to be pluggable | 14:38.39 |
| superdealloc | I understand that better than you'd think. | 14:38.55 |
| rayjj | superdealloc: and gs attempts to transform more than a single color at once since that can make the CMS more efficient | 14:39.39 |
| superdealloc | yeah, the API here isn't prepared for that | 14:39.55 |
| | it's quite easy to "hijack" colorspaces | 14:40.02 |
| rayjj | superdealloc: I'm glad that you understand the issues, and apologize if I'm being too basic | 14:40.13 |
| superdealloc | Not at all, assume nothing :-) | 14:40.22 |
| | My colleague and I wanted to contribute this, but we just couldn't wrap our heads around what GS does to get it right. | 14:40.44 |
| | In fact, we understand the MuPDF codebase (great job on that, it's so nice and clean!) | 14:40.57 |
| rayjj | superdealloc: thanks to starting with a clean slate and a clean goal, and having developers that had dealt with the mess of gs helps. Now it's gotten a bit polluted with stuff that's been added in that wasn't even considered at first, | 14:42.29 |
| | but overall the developers chat about the methods to do new things to try and keep it clean, and also they've been willing to go back and seriously alter the design a few times | 14:43.34 |
| marcosw_ | rayjj: good morning. | 14:43.39 |
| rayjj | marcosw: morning, marcosw | 14:43.49 |
| marcosw_ | rayjj: what's the command line option to specify the unprintable margins so that -dPDFFitPage works as expected with the ps2write device? | 14:44.24 |
| rayjj | superdealloc: well, I'll go back to playing with gs performance for a paying customer and look for the enhancement request to come along | 14:44.47 |
| superdealloc | please do | 14:44.59 |
| | and God help you | 14:45.02 |
| rayjj | marcosw: let me have a look | 14:45.03 |
| superdealloc | doesn't sound like an easy task ;-) | 14:45.26 |
| rayjj | marcosw: phone call. Just a sec | 14:45.58 |
| marcosw_ | marcosw: no hurry. | 14:46.41 |
| rayjj | marcosw: OK, so the PDFFitPage logic takes .HWMargins into account. That's good. | 14:47.07 |
| jhabjan | rayy: about the bug #695196 | 14:53.17 |
| | rayy -> rajj ( sorry ) | 14:53.33 |
| | it seems it's not my day... named you wrong twice ... :) | 14:54.59 |
| rayjj | marcosw: this works for me: -dPDFFitPage -dFIXEDMEDIA -c "<< /.HWMargins [ 72.0 72.0 96.0 96.0 ] >> setpagedevice" -f ... | 14:55.36 |
| | marcosw: the .HWMargins array items are offsets in from the page edges in points | 14:56.11 |
| | jhabjan: np. | 14:56.34 |
| jhabjan | anyway... the only way I could make piped output is by using a '%handle%' | 14:57.32 |
| | everything else on windows fails | 14:58.17 |
| rayjj | jhabjan: OK, that wasnt' clear from the bug. You are dealing with OutputFile (%handle%...) then, correct ? | 14:58.58 |
| jhabjan | the only sample I could write you is a C# sample as I dont do C | 14:59.13 |
| | rayjj: yep | 14:59.32 |
| | rayjj: so, the arg looks like: -o%handle%240 | 15:00.57 |
| rayjj | jhabjan: it is somwhat different depending on the type of device. That also isn't in the bug. What output device types are you working with ? | 15:05.58 |
| jhabjan | rayjj: in this case pdfwrite device | 15:06.55 |
| rayjj | jhabjan: for example the 'printer' class of devices doesn't close the OutputFile unless you set ReopenPerPage true (default is false) | 15:07.17 |
| | jhabjan: OK, pdfwrite is different. | 15:07.31 |
| jhabjan | rayjj: I did mentioned couple of months ago that I could only use pipes with %handle% logic... | 15:07.54 |
| | rayjj: search for %handle% in: http://ghostscript.com/irclogs/2014/01/31.html | 15:08.08 |
| | rayjj: where you did some tests with kens I think... | 15:08.47 |
| rayjj | jhabjan: OK, thanks for the link. That info probably should be in the bug. | 15:15.29 |
| | jhabjan: so, gsprint is NOT using pdfwrite, that's the first difference. pdfwrite _does_ close the OutputFile (it calls gdev_vector_close_file, which does the deed) | 15:16.59 |
| | gsprint uses a printer type device, and it works differently | 15:18.35 |
| jhabjan | rayjj: I think that at that time I was asking something about the inkcov device and it's output to a pipe or stdout rather than to the file | 15:24.21 |
| | rayjj: what I'm doing now is: 1. I Create an input inheritable pipe. 2. I get the client pipe handle string 3. Pass that handle to gs | 15:27.11 |
| | rayjj: after that, when GS is done, I should be able to close client pipe handle passed to the gs | 15:27.34 |
| | rayjj: but, when I call 'CloseHandle', it says that the handle is not valid | 15:28.35 |
| mvrhel_laptop | morning | 15:36.44 |
| rayjj | morning, mvrhel_laptop | 15:39.15 |
| mvrhel_laptop | hi rayjj | 15:39.39 |
| rayjj | jhabjan: so if we open a FILE* on the underlying fd we get from _open_osfhandle we have to close it, right ? | 15:40.31 |
| | jhabjan: but you are saying that it is also closing the handle ? | 15:41.28 |
| | jhabjan: we have to call fclose to make the OS's buffer get flushed | 15:46.10 |
| jhabjan | rayjj: i think so | 15:49.00 |
| henrys | mvrhel_laptop: I have your mouse, do you want me to ship it to you? Might be easier to just get another. | 15:56.17 |
| mvrhel_laptop | henrys: oh I was wondering where it went | 15:57.05 |
| | I had to steal my son's this morning | 15:57.16 |
| | henrys: yes. I will pick up a new one | 15:57.30 |
| | henrys: bring the mouse to the next meeting though | 15:57.57 |
| | I have the dongle | 15:58.00 |
| henrys | mvrhel_laptop: okay | 15:58.18 |
| mvrhel_laptop | and the way I work, I will probably lose another one at some point | 15:58.23 |
| | hi kens | 16:00.03 |
| henrys | kens: mission stay away now? | 16:00.28 |
| kens | Hi Michael | 16:00.30 |
| | awake ? | 16:00.38 |
| henrys | s/away/awake | 16:00.41 |
| kens | Yeah need to stay awake another 5 hours :-( | 16:00.59 |
| mvrhel_laptop | Robin_Watts: I sent you the file that had the threading issues on gsview. I will be curious if you see any issues | 16:01.26 |
| henrys | kens:go swimming, can't fall asleep doing that ;-) | 16:01.33 |
| mvrhel_laptop | yes. sitting around is not so good | 16:01.45 |
| kens | Henrys, true... | 16:01.57 |
| | Actually I will have to go cook dinner shortly | 16:02.05 |
| | Melanie and Stella at the yard | 16:02.15 |
| rayjj | jhabjan: now, since %handle% types are never created by the graphics library, it may be that mswin_handle_fclose should only fflush, but it will leave the FILE * structure (and it's buffer) allocated until the process exits, which isn't good | 16:07.38 |
| | jhabjan: and whatever is reading the %handle% needs to know when the EOF is reached | 16:08.36 |
| jhabjan | rayjj: ok, so, what do you suggest ? | 16:10.37 |
| | rayjj: hm....or should I DuplicateHandle to protect myself..... | 16:12.40 |
| | rayjj: or could you check if the handle owner is a current process gs is running in....and if yes, then you can DuplicateHandle.. | 16:14.51 |
| | rayjj: which will (I think) leave my initial client pipe handle valid | 16:15.38 |
| rayjj | jhabjan: I'm not sure why you need the handle after gs has finished writing to it | 16:25.44 |
| | it's not like anything else is going to show up from it | 16:26.24 |
| | jhabjan: why does it matter whether the process that creates the handle is the same as the gs process or not ? | 16:27.24 |
| jhabjan | rayjj: the thing is that I use .NET framework built-in 'AnonymousPipeServerStream' class | 16:29.26 |
| | rayjj: and when I dispose that class instance, Garbage collector wants to close client pipe handle... and it fails as it's already closed... which results with the .NET framework exception ... | 16:30.43 |
| rayjj | jhabjan: sounds like the expectations of the 'AnonymousPipeServerStream' class are not consistent with how gs expects to work. That doesn't mean that DuplicateHandle is the solution. It is just as easy to imagine a use that does expect gs to close the handle | 16:40.18 |
| | (as it does now) | 16:40.29 |
| | jhabjan: if I understand it, if your app does the DuplicateHandle this will prevent the GC from seeing an invalid handle, right ? | 16:42.14 |
| jhabjan | rayjj: yep | 16:43.09 |
| | rayjj: that's one workaround (from my side) | 16:43.24 |
| | rayjj: the other one is to manually 'flag' the client pipe handle as invalid before disposing an instance (since AnonymousPipeServerStream uses SafeHandle's which are simple wrappers around the native handles ).. which will tell Garbage collector not to try to close the handle... | 16:46.49 |
| rayjj | jhabjan: did you call DisposeLocalCopyOfClientHandle() ? That says that "If this method is not called, the AnonymousPipeServerStream object will not receive notice when the client disposes of its PipeStream object." | 16:54.42 |
| | if not, the GC may not know that the client (gs) has already done the close | 16:55.24 |
| jhabjan | rayjj: well, If I call it too early, gs will report an error, If I call it too late I get invalid client handle exception | 16:55.46 |
| | rayjj: to call DisposeLocalCopyOfClientHandle, there would need to be one more API callback which will be triggered once you get the handle.... | 16:56.27 |
| rayjj | jhabjan: just grasping at straws. The examples in the docs assume that the client is using AnonymousPipeClientStream AFAICT | 16:57.02 |
| jhabjan | rayjj: I think I know what you are trying to say.... and.. could be you are right | 16:57.57 |
| rayjj | jhabjan: once you read the EOF from the stream you can do the Dispose (count from Read is 0) | 17:01.51 |
| jhabjan | rayjj: ok, will do that | 17:03.29 |
| | rayjj: thanks for your time | 17:03.40 |
| rayjj | jhabjan: np. Please let me know what you find so that I know what to tell others. | 17:04.30 |
| | mvrhel_laptop: you actually know C# better -- does this make any sense ? | 17:04.50 |
| jhabjan | rayjj: sure, will do | 17:04.56 |
| mvrhel_laptop | sorry I was not following | 17:05.31 |
| | when I have had issues with native code getting stuff GCd from managed code I have either had to pin the item in the managed code or make the object a member variable in the managed code | 17:06.32 |
| rayjj | mvrhel_laptop: np. jhabjan was having problems with communicating from his C# app getting the pdfwrite data from gs via a handle that is an anonymousPipe | 17:06.52 |
| | jhabjan was saying that the GC was trying to CloseHandle and it was no longer valid | 17:07.51 |
| | jhabjan: and he thought that was because gs was doing fclose on the FILE * that we use to write to the handle | 17:08.54 |
| mvrhel_laptop | hmm that sounds like a messy case | 17:09.23 |
| | how is he getting the pdfwrite out? through stdout? | 17:09.49 |
| rayjj | mvrhel_laptop: but I don't see any way we can NOT do the flclose (we have to so that the reading end gets the EOF) | 17:09.57 |
| mvrhel_laptop | I am just wondering what the API call to gs is in this case | 17:10.31 |
| rayjj | mvrhel_laptop: he is giving the %handle%XXXXX to gs as the OutputFile (HandleAsString) | 17:10.35 |
| mvrhel_laptop | I see | 17:11.00 |
| rayjj | mvrhel_laptop: he's using "-o%handle%1234" | 17:11.07 |
| jhabjan | mvrhel_laptop: if it makes sense to you, here is the pipe reader implementation: https://github.com/jhabjan/Ghostscript.NET/blob/master/Ghostscript.NET/GhostscriptPipedOutput.cs | 17:13.20 |
| | mvrhel_laptop: and here usage implementation: https://github.com/jhabjan/Ghostscript.NET/blob/master/Ghostscript.NET.Samples/Samples/PipedOutputSample.cs | 17:13.40 |
| | rayjj: I just did try to dispose after EOF, but a same thing happens (invalid handle) | 17:15.58 |
| mvrhel_laptop | I don't see anything obviously wrong. However, in my work with C# + GS I had several GC issues that were not obvious and hard to track down. | 17:19.09 |
| | does it have the problem in debug and release mode? | 17:19.50 |
| jhabjan | I did try it with 9.14 (release) and the I build the latest source code a day a go in debug mode... both had a same problem | 17:20.49 |
| mvrhel_laptop | I mean your c# project | 17:21.29 |
| jhabjan | yep, a same for C# debug and release... | 17:22.04 |
| mvrhel_laptop | ok. I need to sign off now. at coffee shop. no battery or power supply | 17:22.26 |
| jhabjan | although, this is not exactly a GC problem.. as GC simply calls AnonymousPipeServerStream Dispose method.. which then tries to close the handle.. | 17:23.01 |
| | n/p | 17:23.02 |
| mvrhel_laptop | bbiaw | 17:23.29 |
| rayjj | jhabjan: there is a IsConnected that tells you "If this property returns false, the pipe may be waiting to connect, or may be disconnected, closed, or broken." | 17:31.25 |
| jhabjan | rayjj: already checked that.. it always returns 'connected' = true | 17:33.31 |
| rayjj | jhabjan: OK, and can't you just catch the IOException when you try the Dispose after reading the EOF ? | 17:34.37 |
| jhabjan | rayjj: I mean, at the point I check the "IsConnected" GS already starts writing to the pipe | 17:34.38 |
| | rayjj: I can, but... | 17:35.33 |
| | rayjj: that's one more solution... and I thought that maybe it would be better (if this is a bug) to be fixed in the GS.. | 17:36.15 |
| rayjj | jhabjan: I mean after the 'Read' returns readCount == 0 -- from: while ((readCount = reader.Read(buffer, 0, buffer.Length)) > 0) | 17:36.46 |
| jhabjan | rayjj: got it | 17:39.30 |
| rayjj | jhabjan: I can't see that it is a bug in gs. It would be a bug if we didn't do the fclose since that would leak resources | 17:39.40 |
| jhabjan | rayjj: ok | 17:46.27 |
| marcosw_ | rayjj: thanks for the .HWMargins help. | 19:00.02 |
| Robin_Watts | mvhrel_laptop: For the logs. Thanks. Will look tomorrow. | 22:31.26 |
| | Forward 1 day (to 2014/06/05)>>> | |
IRC Logs