| | <<<Back 1 day (to 2012/06/06) | 2012/06/07 |
| mvrhel | Robin_Watts: yes please commit that | 05:20.00 |
| d3c | can anyone figure out what's going on here? http://pastebin.com/raw.php?i=69efhHFG | 06:44.05 |
| | I've been converting ~4000 PDFs and suddenly that error shows up | 06:44.32 |
| | converting to PNG, that is | 06:44.51 |
| chrisl | d3c: that's memory corruption or other similar problem detected by glibc's memory manager - if you can identify a specific file that causes the error, you can report it as a bug | 06:47.07 |
| d3c | chrisl: sure, will do in a sec. | 06:48.49 |
| | chrisl: works pretty good though. converted all 4000 PDFs, only three failed. I'll attach those to the bug report | 06:49.49 |
| Robin_Watts | Hmm. All of a sudden part way during this run, a load of data structures suddenly move in memory. | 12:12.32 |
| | That smells like GC to me. | 12:12.41 |
| kens | sure does | 12:12.45 |
| Robin_Watts | so I've run with -dNOGC, and it still happens. | 12:12.50 |
| kens | Was about to suggest that | 12:12.51 |
| | Bizarre. | 12:12.56 |
| | But as chrisl said before, -dNOGC doesn't actually disable it | 12:13.08 |
| Robin_Watts | so I've put a breakpoint on case e_VMreclaim: in interp.c and that never gets hit. | 12:13.21 |
| kens | Beats me :( | 12:13.59 |
| Robin_Watts | Ah. let me try breakpointing interp_reclaim itself. | 12:14.13 |
| kens | That might be better | 12:14.31 |
| | I'm not cler on how it wodl get there though | 12:14.41 |
| Robin_Watts | Gottit. | 12:14.52 |
| kens | Ah, so how does it get there ? | 12:15.07 |
| Robin_Watts | From another call from the interpreter (the interpreter calls automatically after so many iterations). | 12:15.28 |
| kens | AH, didn't know that | 12:15.45 |
| | Busy fighting through transparency at the moment.... | 12:15.55 |
| Robin_Watts | If I was to add a new debug flag: --debug=gc-disable that actually nobbled gc by just returning from interp_reclaim would anyone object? | 12:16.10 |
| | kens: Welcome to my world. | 12:16.22 |
| chrisl | Robin_Watts: being able to fully disable GC might be useful in theory. In practice, the explosion of memory use might limit its utility....... | 12:42.56 |
| Robin_Watts | chrisl: For this particular example (stepping through with data breakpoints) it will be immensely helpful :) | 13:05.44 |
| chrisl | Robin_Watts: Yeh, I wasn't saying not to do it, just that, the time I tried it, the memory got out of hand pretty quickly. Another (possibly more useful) option would be to prevent memory compaction - so addresses remain stable. | 13:07.36 |
| Robin_Watts | chrisl: That would require far more work, I fear. | 13:08.04 |
| chrisl | Which is why I gave up on it when I was looking ;-) | 13:08.27 |
| | Actually, I wonder if we could do just by initialising the PS memory manager to use the "stable_memory" allocator........ | 13:09.29 |
| d3c | chrisl: http://bugs.ghostscript.com/show_bug.cgi?id=693097 | 13:27.35 |
| | chrisl: just filed the bug we spoke about earlier | 13:27.46 |
| Robin_Watts | d3c: If you're interested in pursuing that bug further, you could try retesting with a memento build. | 13:34.38 |
| ray_laptop | chrisl: stable_memory is _still_ subject to GC and reloc -- it is 'stable' only in that it is NOT subject to save/restore | 14:10.54 |
| | I also looked into hobbling the GC's "reloc" a few years ago and ran into the usual -- I didn't understand it well enough and my attempts without lots more digging failed. | 14:12.39 |
| | chrisl: in theory, we would do the GC mark and scan part, then rather than moving the remaining items up, just add the items no longer referenced to the free_lists | 14:13.50 |
| | I think when I made my attempt, I just skipped the reloc, but hadn't actually done anything to free the objects no longer referenced (marked, I think) | 14:15.03 |
| | we even have an open enhancement (I think) to try getting rid of the reloc as a performance improvement (although it _might_ result in more memory usage). | 14:16.38 |
| | maybe, I'll have another stab at doing it. As Robin_Watts said, it would immensely help debugging sometimes. | 14:17.59 |
| Robin_Watts | ray_laptop: It's something I wouldn't mind looking at at some point. | 14:18.54 |
| ray_laptop | for MUCH greater performance improvement, using non-gc memory for more stuff would be the biggest win. | 14:18.55 |
| Robin_Watts | But it's not important now - the --debug=gc-disable that I added this morning is doing what I need it to. | 14:19.13 |
| ray_laptop | Robin_Watts: if I start looking at it, I'll be glad to discuss my findings, ideas, progress, etc. here (with you or any interested parties). If you dive into it, please do the same | 14:19.59 |
| Robin_Watts | ray_laptop: Of course. | 14:20.08 |
| ray_laptop | Robin_Watts: I'm glad the gc-disable is working OK. As chrisl found, I saw surprisingly quick memory growth and couldn't use it | 14:21.01 |
| | time for breakfast... | 14:21.13 |
| chrisl | ray_laptop: Oh, okay - I *thought* we a memory pool that was subject to garbage collection, but not to reloc - no idea where I (wrongly) picked that up....... | 14:27.11 |
| Robin_Watts | I *think* I understand what's going wrong with the transparency stuff now. | 14:31.01 |
| | We do indeed leave the transparency masks defined for longer than we should. | 14:31.32 |
| | if anyone uses a transparency group in the middle of a transparency mask definition, we'll get bad results. | 14:32.03 |
| kens | I'm seeing that happen generated by our own code (I believe) | 14:35.06 |
| Robin_Watts | right. | 14:35.34 |
| | so pdfwrite is making pdf files that gs renders badly, but acrobat renders fine ? | 14:35.50 |
| kens | No. | 14:36.02 |
| | What I mean is that our own transparency compositor is doing something like that internally | 14:36.18 |
| | I'm curretnly fixing pdfwite's handling of such | 14:36.41 |
| Robin_Watts | right. | 14:37.38 |
| | I'm going to try to add some push/pops to the pdf interpreter so that it behaves properly. | 14:38.11 |
| kens | Oh god. That may break my fix... | 14:38.24 |
| Robin_Watts | I'd hope not. | 14:39.02 |
| | patching the symptom is bad (M'kay). | 14:39.22 |
| kens | I'm not patching a symptom | 14:39.35 |
| | The problem is to do with maintaining a flag when a new group comes long | 14:39.56 |
| | The flag says (in essence) this group was inserted by our transparency code, so we *don't* want to write it to the output | 14:40.35 |
| | But if we get a new group inside that one, it resets the flag | 14:40.47 |
| Robin_Watts | OK, that sounds like a different problem. | 14:41.04 |
| | The problem that I have here, is that the pdf interpreter is leaving softmasks in effect for longer than it should. | 14:41.09 |
| kens | Ye, my problem is limited to pdfwrite | 14:41.28 |
| | But the order of things is 'surprising' | 14:41.41 |
| | Eg I see: | 14:42.27 |
| | Push_smask_color | 14:42.27 |
| | Begin_trans_mask | 14:42.27 |
| | Pop_smask_color | 14:42.27 |
| | End_trans_mask | 14:42.27 |
| Robin_Watts | That does sound odd. | 14:42.46 |
| kens | It seems to work | 14:42.59 |
| mvrhel | Robin_Watts: I know we have test files in place that include soft masks that have groups. Do you have an example where we render wrong? | 15:36.52 |
| Robin_Watts | mvrhel: Nope. | 15:37.01 |
| mvrhel | ok. We also have examples with softmasks inside softmasks | 15:37.36 |
| | that render correctly | 15:37.40 |
| Robin_Watts | You know how we repeatedly render the SMask for each successive object that uses it? | 15:39.43 |
| mvrhel | Yes. that is def. wrong | 15:39.57 |
| | each group you mean | 15:40.04 |
| Robin_Watts | each successive 'Do' in this case. | 15:40.18 |
| | each successive xobject. | 15:40.27 |
| mvrhel | yes | 15:40.33 |
| Robin_Watts | After each one of those, we don't take the smask off the stack. | 15:41.05 |
| mvrhel | I would think we would leave it until we had a Q occur | 15:41.55 |
| Robin_Watts | I think if we wrap the code so we: pushextendedgstate render_mask use_mask popextendedgstate | 15:42.09 |
| | each time, then we'd be fine. | 15:42.13 |
| mvrhel | the solution should be that the interpreter should not resend the soft mask if we are still in the same extended graphic state | 15:42.44 |
| Robin_Watts | That'd be even better. | 15:42.54 |
| mvrhel | I see the bug as being there | 15:42.57 |
| | not in the C code | 15:43.02 |
| Robin_Watts | Yes. | 15:43.09 |
| | My debugging is showing me the effects of the pdf interpreter code within the C. | 15:43.40 |
| mvrhel | right. but I would like to fix the issue in the interpreter. bending over backwards in the C code to fix this is not ideal | 15:44.14 |
| Robin_Watts | Right. I'm not suggesting we change the C. | 15:44.24 |
| mvrhel | I realize you want to pop the smask though | 15:44.38 |
| Robin_Watts | I'm talking about making the pdf interpreter do some extra push/pops. | 15:44.54 |
| mvrhel | right. that sounds a bit like masking a problem ;) | 15:45.13 |
| Robin_Watts | Does it? | 15:45.39 |
| | The problem is that pdf interpreter code is pushing more and more onto the stack each time. | 15:46.01 |
| mvrhel | well yes. the interpreter should not be sending these extra begin trans mask | 15:46.02 |
| | that is the problem | 15:46.16 |
| Robin_Watts | by putting the q/Q's in, the pdf interpreter would be being well behaved. | 15:46.17 |
| mvrhel | inefficient | 15:46.42 |
| Robin_Watts | A fix where it doesn't resend unnecessary ones would be an even better fix. | 15:46.43 |
| mvrhel | but well behaved | 15:46.48 |
| | yes | 15:46.54 |
| | that is the fix I would like to see | 15:46.58 |
| | we would need to drag in ray or alexcher to help with this | 15:47.22 |
| Robin_Watts | indeed. I'm shooting for something that works, and wouldn't be slower than what we currently have. | 15:47.33 |
| mvrhel | yes. I would like us to be faster | 15:47.43 |
| | and working | 15:47.46 |
| Robin_Watts | and the 'do it properly' fix can come afterwards. | 15:47.53 |
| kens | Robin_Watts : got a second ? | 15:48.18 |
| Robin_Watts | kens: sure. | 15:48.24 |
| kens | Go some funny errors in my last bmpcmp | 15:48.35 |
| | "rm: 0*.pam: No such file or directory" | 15:48.46 |
| | repeated many times | 15:48.57 |
| Robin_Watts | ooh. | 15:49.07 |
| kens | "rm: cannot remove `0*.pam': No such file or directory" | 15:49.12 |
| | also repeated many times | 15:49.17 |
| | And a number of other very similar messages | 15:49.40 |
| | Doens't seem to be a problem though | 15:49.49 |
| | This is under 'misc errors' | 15:50.05 |
| Robin_Watts | I suspect marcosw has been fiddling. | 15:51.26 |
| kens | I couldn't possibly comment :0) | 15:51.44 |
| mvrhel | hehe | 15:51.49 |
| | I smell smoke | 15:51.59 |
| Robin_Watts | run.pl has the offending lines in. | 15:52.41 |
| | Oh! Ha. I know why :) | 15:52.52 |
| kens | Well it seems harmless | 15:53.03 |
| | Just surprised me | 15:53.09 |
| Robin_Watts | I accidently pushed a test the other day with the RAW_DUMP stuff enabled. | 15:53.24 |
| mvrhel | Robin_Watts: I understand what you want to do to make sure at least that we pop the mask we have before we push another one while in the same extended graphic state | 15:53.30 |
| | I guess to solve the real problem we need to have a uid or something for the current soft mask in the interpreter so it can know that it already has the mask set | 15:54.15 |
| Robin_Watts | Or we have to have a flag for "the softmask has changed since it was last rendered". | 15:54.59 |
| mvrhel | iirc there is something like this for some of the PDF cal color spaces | 15:55.00 |
| | so that we avoid regenerating what used to be the joint CIE cache but is now the ICC profiel | 15:55.29 |
| | or a flag would be fine | 15:56.11 |
| | back to the copy alpha salt mine.... | 15:57.36 |
| Robin_Watts | mvrhel: Thanks again. | 15:57.44 |
| mvrhel | Robin_Watts: thank you for finding and fixing all of this | 15:57.55 |
| | I believe when you are done, we should open a bug to fix the multiple pushes with each Do though | 15:58.19 |
| Robin_Watts | absolutely. | 15:58.28 |
| mvrhel | Oh. Did Miles decide on a date for the meeting?] | 15:59.17 |
| | I had not heard anymore since kens through out the one date | 15:59.41 |
| Robin_Watts | yes. 9th and 10th. | 15:59.41 |
| mvrhel | oh | 15:59.45 |
| Robin_Watts | miles rejected kens request to move it. | 15:59.58 |
| mvrhel | ok | 16:00.03 |
| | I missed that email | 16:00.07 |
| Robin_Watts | miles emailed ken direct, so we all missed it. | 16:00.23 |
| mvrhel | Last email I have in that thread is from Ken | 16:00.30 |
| | oh ok | 16:00.32 |
| | thanks | 16:00.41 |
| Robin_Watts | np. | 16:00.45 |
| henrys`` | kens or anyone: looks like this high level device parameter only works for pdfwrite and pswrite2 - I need something to detect a vector device. Shouldn't all vector devices be "high level"? | 16:09.37 |
| kens | henrys yes thery should | 16:09.51 |
| | Which one(s0 are not ? | 16:09.57 |
| henrys`` | pxlcolor and pxl, svgwrite, ... | 16:10.33 |
| kens | Then they should be updated | 16:10.45 |
| | I cna do that tomorrow if you like | 16:10.51 |
| henrys | okay sounds good. | 16:11.11 |
| | Isn't there just some way to do it for the vector super class? | 16:11.43 |
| kens | Maybe, I'm not certain | 16:11.55 |
| | But I think not | 16:12.01 |
| | Its a device parameter | 16:12.10 |
| henrys | all the devices that are gx_device_vector - few more than I listed but not many. | 16:16.39 |
| kens | I'll have to look at it, will do it tomorrow, am about to go tonight | 16:17.01 |
| henrys | right see you tomorrow. | 16:17.24 |
| kens | Night all | 16:17.30 |
| ray_laptop | for the logs: kens -- there are 'gdev_vector_{get|put}_params' functions in base/gdevvec.c -- that's a good place to move the HighLevelDevice handling | 16:26.09 |
| | Robin_Watts: I am looking at the igc.c code. I think I see a way to disable relocation -- | 16:45.15 |
| Robin_Watts | ray_laptop: I'm in pdf interpreter hell. | 16:46.44 |
| ray_laptop | Robin_Watts: OK. I'm going to twiddle some stuff and look with the debugger for some simple cases | 16:47.33 |
| | Robin_Watts: can I help with the pdf_interp ? (alexcher is also a resource) | 16:48.02 |
| Robin_Watts | ray_laptop: Well, I'll gladly take any help I can get. | 16:48.24 |
| | Let me describe the problem. | 16:48.30 |
| | At the moment, before we do any fill operation we do 'setfillstate' | 16:49.08 |
| ray_laptop | Robin_Watts: OK. I did read the logs yesterday and today w.r.t. the SMask being applied twice | 16:49.16 |
| Robin_Watts | And that checks to see if an smask should be present - and if it should be, it sets the smask. | 16:49.40 |
| | If we do a series of operations all with the smask in effect, this means that we repeatedly reset the same smask again and again. | 16:50.09 |
| ray_laptop | Robin_Watts: right -- I see the code (and comment) in .settransparencymask | 16:51.24 |
| Robin_Watts | This is a) inefficient (because it means the xobject is repeatedly redrawn), and b) incorrect, as it leaves the previous smasks kicking around the transparency stack until we hit a .popextendedgstate | 16:51.57 |
| ray_laptop | Robin_Watts: makes sense. | 16:52.43 |
| Robin_Watts | The ideal solution would be that we .pushextendedgstate before rendering the smask and then never draw it again until it stops being in force - whereupon we should pop the gstate. | 16:53.27 |
| ray_laptop | Robin_Watts: so you want to detect i the SMask has already been executed and not do it again. And have the .popexterndedgstate go back to the SMask (if any) previously in effect | 16:53.46 |
| Robin_Watts | For me, correctness is more important than efficiency. | 16:54.02 |
| | So, I was considering trying for a "correct, but still inefficient" solution, whereby I surround each rendering/use of the smask with a push/pop pair. | 16:55.09 |
| | Sadly, even that is not straightforward. | 16:55.44 |
| ray_laptop | Robin_Watts: so does saving the 'mask' object and comparing with 'eq' or 'ne' and then if 'ne' set a 'currentSMask' to that object (for later compares) not work ? | 16:56.17 |
| | Robin_Watts: obviously if the SMask comes in 'null' that needs to get set as well | 16:56.46 |
| | s/that/the saved 'currentSMask'/ | 16:57.10 |
| Robin_Watts | ray_laptop: I don't know. But (if I'm understanding you correctly) that would only affect the efficiency side - it wouldn't affect the correctness. | 16:57.21 |
| ray_laptop | Robin_Watts: since popping the ExtGState does an 'end' on the dict, it'll take care of reverting to the previous SMask | 16:57.47 |
| | Robin_Watts: I don't understand the 'correctness' issue | 16:58.26 |
| | what does HEAD do incorrectly ? | 16:59.00 |
| Robin_Watts | I have an example file that does: /smask gs /redsquare Do /greensquare Do | 16:59.40 |
| | And the smask involves drawing a solid white square in a group with a group alpha of 0.33 | 17:00.30 |
| ray_laptop | where the 'smask' Resource is an ExtGstate that sets an SMask into the current transparency group ? | 17:01.01 |
| Robin_Watts | so the sequence of operations called by the pdf interpreter looks something like: | 17:01.02 |
| | ray_laptop: Yes. | 17:01.13 |
| ray_laptop | pulls up the PDF Ref manual ... | 17:02.46 |
| Robin_Watts | 1 begin_transparency_mask | 17:02.53 |
| | 2 draw_white_square | 17:02.55 |
| | 3 end_transparency_mask | 17:02.57 |
| | 4 begin_transparency_group | 17:02.59 |
| | 5 draw_red_square | 17:03.00 |
| | 6 end_transparency_group | 17:03.02 |
| | 7 begin_transparency_mask | 17:03.03 |
| | 8 draw_white_square | 17:03.05 |
| | 9 end_transparency_mask | 17:03.06 |
| | A begin_transparency_group | 17:03.08 |
| | B draw_green_square | 17:03.09 |
| | C end_transparency_group | 17:03.11 |
| | ray_laptop: follow that? | 17:04.55 |
| ray_laptop | did my connection die ? | 17:06.23 |
| Robin_Watts | I don't think so. The logs are complete. | 17:07.37 |
| ray_laptop | Robin_Watts: OK -- connection glitch, but I see it on the logs | 17:08.02 |
| Robin_Watts | OK. | 17:08.08 |
| ray_laptop | ahh... it all just came in | 17:08.14 |
| Robin_Watts | So the mask that is rendered in lines 1-3... call that M1. | 17:08.27 |
| | the problem is that when we render M2 (lines 7-9), M1 is still in effect. | 17:08.48 |
| | Now, for the simple case I've outlined above, that doesn't matter. | 17:09.01 |
| | but if 7-9 had involved a group, it WOULD matter, because the output of that group would end up being masked by M1. | 17:09.44 |
| mvrhel | ray_laptop: the best solution for all of this is that the interpreter does not resent the begin transparency mask with every do, if it is in the same graphic state with a mask that was already rendered with a btm | 17:09.53 |
| ray_laptop | Robin_Watts: so the correct thing would be for the SMask to only be applied when the ExtGstate changed the SMask or did a 'Q', right ? | 17:09.57 |
| mvrhel | right | 17:10.06 |
| Robin_Watts | ray_laptop: That would be the ideal solution, yes. | 17:10.12 |
| | BUT even then... we'd need to be sure to REMOVE an SMask early enough. | 17:10.31 |
| mvrhel | well when the Q occurs it is removed | 17:10.58 |
| Robin_Watts | Right, but consider this case: /smask1 gs /red Do /green Do /smask2 /blue | 17:11.28 |
| | Do | 17:11.35 |
| mvrhel | right. when it changes in the same state it needs to be replace | 17:11.48 |
| | replaced | 17:11.51 |
| Robin_Watts | We need to ensure that /smask1 is removed before /smask2 starts to be rendered. | 17:12.00 |
| mvrhel | right | 17:12.04 |
| | so we need to send the pop command | 17:12.17 |
| | in that case | 17:12.19 |
| Robin_Watts | Right. | 17:12.25 |
| | Ideally we'd do: push draw_smask1 red green pop push smask2 blue pop | 17:12.54 |
| | At the moment the smasks are sent 'lazily' at the last possible minute, so popping there may be too late. | 17:13.42 |
| ray_laptop | Robin_Watts: my IRC connection is having problems... | 17:17.08 |
| Robin_Watts | reconnect ? | 17:17.29 |
| | (try turning it off and on again) | 17:17.41 |
| | </TheITCrowd> | 17:17.48 |
| ray_laptop | once we 'end_transparency_mask' it is retained in the pdf14 'ctx->maskbuf' so we should not do it again -- so if we do the check for 'already has been set' in .settransparencymask and not 'Draw' it again, won't that make it correct as well as more efficient ? | 17:19.17 |
| Robin_Watts | In order for it to be correct, we need to pop the extended gstate after the last use. | 17:20.26 |
| | and before we start to render any subsequent smask. | 17:20.46 |
| ray_laptop | if the 'push' and 'pop' in your sequence are transparency groups, then all is well. | 17:20.52 |
| Robin_Watts | as long as that is achieved, I'm happy. | 17:20.55 |
| | No. | 17:20.57 |
| | They aren't transparency groups. | 17:21.01 |
| ray_laptop | Robin_Watts: the problem comes if there are two different SMask in the _same_ tranparency group | 17:21.56 |
| Robin_Watts | no. | 17:22.03 |
| | the problem comes if there are 2 different smasks within the same zpushextendedgstate/zpopextendedgstate | 17:22.26 |
| | or q/Q if you prefer. | 17:22.48 |
| ray_laptop | Robin_Watts: to accomodate your sequence (which I _think_ is legal) we'd need to push a transparency group after each SMask is set (with a null SMask) then pop that group if the SMask is set again | 17:23.37 |
| Robin_Watts | My sequence is legal (TTBOMK) | 17:24.18 |
| | but I disagree. | 17:24.27 |
| ray_laptop | For efficiency, we'd only to the pop/push of the transparency group when the SMask actually changed from what was set to in the ctx->maskbuf | 17:24.28 |
| mvrhel | yes. it is legal but rare | 17:24.30 |
| | right | 17:24.47 |
| Robin_Watts | hold on. | 17:24.59 |
| | I'm still not happy with ray's proposed solution. | 17:25.09 |
| | Actually... maybe I am. | 17:25.56 |
| ray_laptop | reconnecting... | 17:26.23 |
| mvrhel | What needs to be done is the following1) Avoid doing begin transparency soft mask for every Do within a qQ 2) Make sure to pop any existing mask if we have another mask within a qQ | 17:26.44 |
| Robin_Watts | ray's solution doesn't quite work. | 17:27.01 |
| | mvrhel: I'm not disgreeing with that. | 17:27.10 |
| | If we define an SMask, then push a group, then pop it, the SMask is still in existence. | 17:27.29 |
| mvrhel | ray just called he is moving locations | 17:27.54 |
| | he will be back on in 10 minutes | 17:28.15 |
| Robin_Watts | We need to push before defining the SMask, and then use it, then pop just before we start to do anything with the next one. | 17:28.21 |
| | Fair enough. I'll shut up for 10 mins then :) | 17:28.31 |
| mvrhel | I dont understand what you mean by "We need to push before defining the SMask" | 17:28.48 |
| | what are you pushing | 17:28.53 |
| Robin_Watts | .pushextendedgstate I think. | 17:29.11 |
| mvrhel | why | 17:29.16 |
| Robin_Watts | OK. Ray asserts that we can solve it by doing the following: | 17:29.34 |
| | btm draw_mask1 etm btg red green etg btm draw_mask2 etm btg blue etg | 17:30.35 |
| | Is that a correct statement as you understand it? | 17:31.12 |
| mvrhel | is this with a source document that has q , soft mask set, trans group with red and green, another soft mask set, a trans group with blue Q ? | 17:33.20 |
| Gigs- | bmpcmp seems to be segfaulting on all the separation pngs (converted from tiff) | 17:34.11 |
| Robin_Watts | It's for my earlier example: /smask1 gs /red Do /green Do /smask2 Do gs /blue Do | 17:34.28 |
| mvrhel | right. so what we need to make sure to do is to have the popextendedgstate occur before smask2 is drawn | 17:35.26 |
| Robin_Watts | mvrhel. Yes. | 17:35.37 |
| | I claim that the right thing to aim for is: | 17:35.45 |
| mvrhel | we also would like to avoid having smask1 occur twice | 17:35.50 |
| ray_laptop | OK, back (in the office with agood connection) | 17:35.52 |
| mvrhel | if we can | 17:35.53 |
| | ok I will let you and ray_laptop duke this out. | 17:36.05 |
| Robin_Watts | pushegs btm draw_mask1 etm red green popegs pushegs btm draw_mask2 etm blue popegs | 17:36.14 |
| mvrhel | I am just causing trouble probably | 17:36.21 |
| ray_laptop | mvrhel: Robin_Watts: The issue is that the pdf14 device only applies the SMask when it does the 'endtransparencygroup' | 17:36.26 |
| | mvrhel: is that correct ? | 17:36.34 |
| mvrhel | yes that is correct | 17:36.39 |
| Robin_Watts | ray_laptop: ish. | 17:36.40 |
| mvrhel | That is not the issue though | 17:37.14 |
| | but simply when the mask is used | 17:37.26 |
| ray_laptop | so any drawing operations BEFORE the SMask is set should be painted directly (just using the alpha), then WE NEED A NEW GROUP when the SMask is set | 17:37.37 |
| Robin_Watts | ray_laptop: Can we rewind a bit? | 17:37.52 |
| | Before you went away, you suggested a possible scheme for a solution. | 17:38.14 |
| ray_laptop | because when the SMask is changed or we end the outermost group, we need to apply the SMask to all things that were painted while that SMask (smask1) was in effect | 17:38.33 |
| Robin_Watts | For my earlier example: /smask1 gs /red Do /green Do /smask2 Do gs /blue Do | 17:38.40 |
| ray_laptop | Robin_Watts: let's use the example: | 17:38.55 |
| Robin_Watts | i believe that your scheme would give us: btm draw_mask1 etm btg red green etg btm draw_mask2 etm btg blue etg | 17:38.58 |
| mvrhel | ray_laptop: I believe we fixed that long ago | 17:38.59 |
| | as far as objects outside a group getting the softmask applied | 17:39.11 |
| | you fixed it in fact | 17:39.18 |
| | years ago | 17:39.21 |
| Robin_Watts | Yes, I don't believe we have that problem. | 17:39.23 |
| ray_laptop | /yellow Do /smask1 gs /red Do /green Do /smask2 gs /blue Do | 17:39.50 |
| Robin_Watts | ok, that's a good example, I can live with that. | 17:40.10 |
| ray_laptop | mvrhel: I fixed that ? | 17:40.15 |
| mvrhel | ray_laptop: What needs to be done is the following1) Avoid doing begin transparency soft mask for every Do within a qQ 2) Make sure to pop any existing mask if we have another mask within a qQ | 17:40.28 |
| | ray_laptop: that was one of the first things thrown at me when I started and you helped me a lot with it | 17:41.01 |
| | iirc | 17:41.07 |
| Robin_Watts | ray_laptop: So, I believe the correct way to implement that example is to do the following: yellow .pushegs btm smask1 etm red green .popegs .pushegs btm smask2 etm blue .popegs | 17:42.58 |
| | (Am I right in thinking that the extended graphic state is just the transparency gubbins?) | 17:44.00 |
| ray_laptop | mvrhel: right. For my example I think we need: 1) push group (no smask in effect yet) 2) draw the yellow thingy 3) push smask1 4) PUSH GROUP (with NO SMask) 5) draw green 6) draw red 7) POP GROUP (due to SMask changing to smask2, 8) push smask2 9) PUSH GROUP 10) draw blue ... | 17:44.15 |
| | that "smiley" was supposed to be ten) | 17:44.41 |
| Robin_Watts | ray_laptop: That won't work. | 17:44.41 |
| | It won't work for exactly the reason the current code does not work. | 17:45.08 |
| | When you pop the group at 7) the smask from 3 is still in effect. | 17:45.41 |
| ray_laptop | oops. do I have the order wrong. It s/b 3) PUSH GROUP 4) push smask1 ... 7) POP GROUP | 17:46.36 |
| Robin_Watts | Right. | 17:46.44 |
| ray_laptop | sorry, but same idea | 17:46.52 |
| Robin_Watts | With the pushes BEFORE the drawing of the smasks we are much closer together. | 17:47.09 |
| | The difference now is that you're using transparency groups, and I'm suggesting using pushegs and popegs. | 17:47.48 |
| ray_laptop | Robin_Watts: if by 'the pushes' you mean 'transparency group pushes', right | 17:48.01 |
| Robin_Watts | sorry, yes. | 17:48.18 |
| ray_laptop | Robin_Watts: there is no pushegs popegs | 17:48.20 |
| Robin_Watts | .pushextendedgstate | 17:48.33 |
| | .popextendedgstate | 17:48.43 |
| | If we use transparency groups, thats a whole new buffer, and lots of compositing etc. | 17:49.27 |
| | If we .pushextendedgstate and .popextendedgstate we get exactly what we need with none of that. | 17:49.46 |
| | (I think) | 17:49.51 |
| mvrhel | popextendedgstate and pushextendedgstate are *only* to keep the soft mask in sync with the qQ | 17:50.58 |
| | it does nothing else | 17:51.02 |
| Robin_Watts | Right. | 17:51.08 |
| | We can't use q/Q itself though. | 17:51.22 |
| ray_laptop | Robin_Watts: does pushing/popping the push transparency groups IFF needed ? | 17:51.23 |
| Robin_Watts | because changes in the other parts of the graphics state MUST be maintained. | 17:51.46 |
| | we want just to store/restore the transparency stuff - and that's exactly what .pushextendedgstate etc do, right? | 17:52.15 |
| | ray_laptop: Too many missing words in that... | 17:52.31 |
| | ray_laptop: Can you ask your last question again please just rephrased a bit? | 17:53.35 |
| ray_laptop | Robin_Watts: I think that the problem is that PUSH_TRANS_STATE does not push a buffer that we can draw into. We need begintransparencygroup to do that | 17:55.26 |
| Robin_Watts | push_trans_state? | 17:56.35 |
| | is that a name for what pushegs calls internally? | 17:56.55 |
| | (sorry, I just can't bear typing .pushextendedgstate each time) | 17:57.10 |
| Gigs- | Robin_Watts: did you write bmpcmp | 17:58.02 |
| Robin_Watts | Gigs-: I did. | 17:58.12 |
| Gigs- | Robin_Watts: it's setfaulting on the separation pngs at *isrc++ = 0xFF0000; /* Red */ | 17:58.31 |
| Robin_Watts | but not the png reading code - that was rillian. | 17:58.43 |
| ray_laptop | Robin_Watts: pushegs (adopting your shorthand for .pushextendedgstate) just calls gs_push_transparency_state which conditionally does a PDF14_PUSH_TRANS_STATE | 17:58.53 |
| Gigs- | Robin_Watts: looks like the actual diff is what is segfaulting, that pointer math | 17:59.05 |
| Robin_Watts | but if you can mail me a png, I'll look at it. | 17:59.05 |
| Gigs- | what's your email | 17:59.17 |
| Robin_Watts | ray_laptop: Right. | 17:59.20 |
| | robin at wss.co.uk is the shortest one. | 17:59.31 |
| | or robin.watts at artifex.com | 17:59.43 |
| ray_laptop | Robin_Watts: (mvrhel) from what I can tell, doesn't do much | 17:59.57 |
| Robin_Watts | ray_laptop: I'm pretty sure that it does enough. I can't see how it can't do really. | 18:00.19 |
| | but, regardless, the differences between a solution that uses .pushegs and transparency groups is going to be very small, right? | 18:00.53 |
| | I mean, it'd be easy to code it one way and then change it if required. | 18:01.14 |
| ray_laptop | Robin_Watts: right -- except on the visible output :-) | 18:01.21 |
| Robin_Watts | right :) | 18:01.38 |
| Gigs- | Robin_Watts: gave you the gdb too | 18:01.53 |
| mvrhel | I dont do very much? | 18:01.57 |
| Robin_Watts | mvrhel: ray was addressing his comment both to me and you. | 18:02.38 |
| | He was saying that .pushegs doesn't do very much, not that you were idle :) | 18:02.55 |
| ray_laptop | Robin_Watts: pdf14_push/pop_transparency_state do mask stacking, but DON"T actually ever do the 'blend' of stuff that was drawn | 18:03.05 |
| Robin_Watts | ray_laptop: Right. And isn't that what we want ? | 18:03.23 |
| ray_laptop | Robin_Watts: so the visible effect I would expect of push/pop egs would be that smask 2 would be ignored since when the group was popped, smask1 would be in effect (and drawing would have been to the single group level buffer) | 18:04.35 |
| Robin_Watts | No. | 18:04.50 |
| ray_laptop | wheras what we want is the drawing with a specific SMask to use that maskbuf and be composited to the group level when the SMask changes | 18:05.36 |
| | so that red and green use smask1 and blue uses smask 2 (and yellow from my example does not use an smask) | 18:06.10 |
| Robin_Watts | AIUI, we ended up with: 1) yellow 2) PUSH 3) btm 4) smask1 5) etm 6) red 7) green 8) PULL 9) PUSH 10) btm 11) smask2 12) etm 13) blue 14) PULL | 18:06.27 |
| | where PUSH/PULL will either be btg/etg (in rays world) or pushegs/popegs (in mine). Agreed? | 18:06.58 |
| ray_laptop | Robin_Watts: I think I follow your abbrevs, but yes | 18:07.06 |
| | Robin_Watts: and I think they need to be btg/et | 18:07.25 |
| | etg | 18:07.27 |
| Robin_Watts | yes, I said that. | 18:07.56 |
| | PULL? PULL? I meant POP. It's been a bad day, sorry... | 18:08.24 |
| ray_laptop | because without the btg, the ctx->buf would be the outermost group's buffers which has no SMask) and the SMask ONLY gets applied when the GROUP is popped (etg) | 18:08.49 |
| | Robin_Watts: either is opposite of PUSH, so that was the least of my confusion :-) | 18:09.14 |
| Robin_Watts | ray_laptop: You're right. | 18:09.18 |
| | I fold. | 18:09.21 |
| ray_laptop | Robin_Watts: OK. | 18:09.39 |
| Robin_Watts | Sorry. In my tests here, red and green have both been themselves groups, hence I was seeing the blending happening at a different time. | 18:09.59 |
| | but you're absolutely right. | 18:10.08 |
| | sorry. | 18:10.14 |
| ray_laptop | Robin_Watts: the key is that the maskbuf is totally ignored during painting operations and is only used during the 'blend' of the popgroup | 18:10.22 |
| Robin_Watts | yes. | 18:10.28 |
| | So, if we're all agreed (and I think we are now), the question is how do we change the pdf interpreter to actually make this happen? | 18:10.51 |
| ray_laptop | Robin_Watts: right -- try it with just rectfills | 18:10.58 |
| | Robin_Watts: exercise for the student ??? | 18:11.15 |
| | ;-) | 18:11.22 |
| | Robin_Watts: if you can send me a couple of simple examples (with the 'Do' as well as rectfills in between), then I'll do the PS mods | 18:12.21 |
| Robin_Watts | I have such an example here :) | 18:13.01 |
| ray_laptop | Robin_Watts: hopefully, the current code will render them wrong w.r.t. Adobe | 18:13.10 |
| | Robin_Watts: with the rectfills as well as the 'Do' of a Form ? | 18:13.36 |
| Robin_Watts | no, the current code gets away with it by dint of not using groups at the wrong point. | 18:13.44 |
| | I'll do you a file that is exactly the example we've been talking about. | 18:14.28 |
| ray_laptop | Robin_Watts: GREAT! (feel free to change the colors to make it easier to see how we get it wrong) | 18:14.59 |
| | or the shapes and overlap | 18:15.11 |
| | go wild, dude ! | 18:15.26 |
| Robin_Watts | It's 7pm here, so I only have so much wildless left in me for the day. | 18:15.57 |
| ray_laptop | Robin_Watts: no rush. | 18:16.12 |
| Robin_Watts | wildness, even. | 18:16.14 |
| ray_laptop | BTW, my changes _should_ address the efficiency as well since I want to tolerate: /smask1 gs .... painting ... /smask1 gs ... more painting ... /smask2 gs ... painting with different smask ... | 18:17.36 |
| | so that smask1 only gets rendered to the maskbuf once for the two first painting sequences | 18:18.12 |
| | Robin_Watts: if you want to add that to your test sequence, fine | 18:18.37 |
| Robin_Watts | ok. | 18:18.45 |
| | I have to go cook, bbs. | 18:20.09 |
| ray_laptop | Robin_Watts: although if you send the test file too soon, I may not make progress on disabling reloc ;-) | 18:21.24 |
| | bbl8r | 18:21.35 |
| | for the logs -- I think it may not be correct to use "HighLevelDevice == present/true" to infer that ROPS are not supported. The pxl devices _are_ high level, but _can_ support rops | 18:30.32 |
| henrys | ray_laptop:I'm not - I'm using it to detect if the device will render paths - Robin_Watts' new path code does not yet work with vector devices. | 18:38.47 |
| ray_laptop | henrys: OK. that makes sense for paths (for now) but we may want more specificity in the future. | 18:40.22 |
| ianneub | hey all, I'm working with MuPDF's mudraw command, and wondering is it possible to output a png image to stdout instead of a file? | 18:49.41 |
| Robin_Watts | nope. | 18:51.44 |
| ianneub | Robin_Watts: thanks | 18:55.01 |
| Robin_Watts | shouldn't be hard to change the code though. | 18:55.14 |
| ianneub | Robin_Watts: awesome, i was just going to ask that. thanks! | 18:55.36 |
| Robin_Watts | You could make it take '-' as the filename to mean stdout ? | 18:55.38 |
| ianneub | Robin_Watts: yeah thats what i would want | 18:55.48 |
| | Robin_Watts: guess I'd have to give it a default image type too | 18:56.05 |
| Robin_Watts | yeah. | 18:56.34 |
| henrys | for logs, ray_laptop, yes I agree HighLevelDevice does not have fine enough granularity. | 19:01.21 |
| Robin_Watts | For the logs,and for anyone other than ray who is interested: http://ghostscript.com/~robin/ray.pdf | 19:13.20 |
| tor8 | ianneub, Robin_Watts: if on linux (or any other unix) mudraw -o /dev/stdout should work | 20:04.14 |
| ianneub | tor8: thanks, i'll give it a shot | 20:04.35 |
| | tor8: i don't think that will work because it doesn't know what format to output in, it uses the -o parameter string to determine the output | 20:05.37 |
| | tor8: at least thats what my 5 minutes of looking at the source told me :) | 20:05.52 |
| mvrhel | hmm Robin_Watts you around? | 20:54.55 |
| | actually never mind | 20:55.42 |
| tor8 | ianneub: d'oh. you're probably right there :/ | 21:59.33 |
| ianneub | tor8: i got it to work using your suggestion, just had to tweak the code a bit | 22:01.01 |
| mvrhel | bbiaw | 22:04.35 |
| ianneub | tor8, Robin_Watts: thanks for your help! | 22:05.30 |
| Robin_Watts | mvrhel: yeah, I'm here. | 23:20.32 |
| mvrhel | hi Robin_Watts: I got it figured out. thanks. just working on copy_alpha fun | 23:21.00 |
| Robin_Watts | ah, cool. | 23:21.10 |
| | mvrhel: In gdevp14.c there is a #define DUMP_MASK_STACK | 23:37.07 |
| | If enabled, it makes -Zv dump the mask stack at various points. | 23:37.26 |
| | Would it be worth having that as a separate debug option? | 23:37.58 |
| | --debug=compositors-masks ? | 23:38.49 |
| | Forward 1 day (to 2012/06/08)>>> | |
IRC Logs