| | <<<Back 1 day (to 2012/07/16) | 2012/07/17 |
| jvervier | Hi all | 07:45.08 |
| saper | hi jvervier | 07:45.17 |
| jvervier | I'm just coming back to notify that I get a (huge) difference of performance for PDF compress with ghostscript between the 9.04 and 9.05 | 07:45.53 |
| | 572 PDF's take 1 hour and 5 minutes with 9.05 and 14 minutes with the 9.04 for the same result | 07:46.22 |
| kens | jvervier; if you think you have encountered a bug please raise a bug report | 07:46.52 |
| jvervier | the problem has been arised on windows 2003 server and 2008 server | 07:46.52 |
| kens | anecdotes with no supporting documents and non information about the command line etc, are not especially useful | 07:48.39 |
| chrisl | jvervier: you said before you were testing with the *same* Ghostscript version on all the machines | 07:49.11 |
| kens | If you think you have found a performance problem, then it would ideally be best to find a single file whcih exhibits the most dramatic problem, and report that as a bug along with the command line being used. | 07:49.43 |
| | chrisl I must have missed this prior conversation, where can I look in the logs ? | 07:50.15 |
| chrisl | kens: much of it was in a private chat :-( | 07:51.01 |
| kens | Ah, well that would explain why I missed it.... | 07:51.14 |
| jvervier | In fact, the initial problem comes from our sys_admin... | 07:51.42 |
| | He installed ghostscript on the server saying me it was the 9.04 | 07:52.00 |
| | but it was the 9.05 | 07:52.06 |
| kens | jvervier there is really very little we can do without something concrete to look at. | 07:52.20 |
| jvervier | so I was thinking both were equal (my local execution and the server) | 07:52.23 |
| | (I just give the information back to Chrisl who help me to look at the problem before) | 07:52.46 |
| | I'll open a bug report with a concrete example today | 07:53.03 |
| kens | That would help, thank you | 07:53.11 |
| jvervier | so chrisl, the version were not the same --> explain why the problem arise only on the server.... | 07:53.46 |
| chrisl | jvervier: did you try 9.05 on your workstation? | 07:54.15 |
| jvervier | I only test (just 10 minutes ago on the server), I've planed to test it on my workstation too | 07:54.47 |
| | chrisl: I confirm, it's happenning on my working station too with the 9.05. I'll prepare a bug report with another PDF because it's private content. | 07:59.10 |
| kens | Well, I've convinced myself there's no realistic way to get pdfwrite to output a Linearised PDF file in one step. I think I can do it by adding a post-process step that operates on the PDF file we've just written, without having to parse that file. | 08:21.20 |
| | Though to be honest, I'm not certain I want to | 08:23.46 |
| Robin_Watts | kens: In pdfclean, I effectively write the file twice. | 09:40.33 |
| kens | Robin_Watts : that's what I'm coming back to | 09:40.44 |
| Robin_Watts | Write it once without the hint streams to get the offsets. | 09:40.53 |
| | Then make the hint streams, seek back and write again. | 09:41.11 |
| kens | The problem is that pdfwrite writes all the page content streams in one file, and everything else in another file, then copies the second file into the first. | 09:41.21 |
| | So there's no realistic way to write page 1,. then the page 1 resources, then everything else | 09:41.36 |
| | I think I may end up writing the file 3 times :-( | 09:42.11 |
| | But I haven't got as far as that yet, I'm figring out where pdfwrite stores all the information I need, then I'm going to work on rewriting the file with all the objects renumbered and where necesary reordered (and generating a new xref of course) | 09:43.32 |
| | After that I'll look at the hint stream rubbish | 09:43.56 |
| Robin_Watts | The hint stream stuff really is rubbish. | 09:44.16 |
| kens | I know :-( | 09:44.22 |
| Robin_Watts | It's hugely poorly documented. | 09:44.26 |
| | I had to resort to taking files to bits to understand it. | 09:44.39 |
| kens | Well I may come back and bug you when I get that far | 09:44.44 |
| | (if) | 09:44.47 |
| | I did understnad it once upon a long time ago though, it might come back | 09:45.12 |
| | I might also steal the logic from pdfopt.ps ;-) | 09:47.20 |
| Robin_Watts | paulgardiner: OK. I finally get to look at your patches... assuming tor8 hasn't beaten me to them. | 10:54.36 |
| paulgardiner | Robin_Watts: sorted | 11:58.53 |
| Robin_Watts | fab. | 11:58.57 |
| | Pushed. | 12:00.37 |
| | That'll keep the cluster busy for a while :) | 12:01.23 |
| paulgardiner | Great. Thanks Robin. | 12:07.05 |
| chrisl | kens: ping | 12:33.18 |
| kens | chrisl pong | 12:33.25 |
| chrisl | Got another ps2write problem from Ubuntu :-( | 12:33.41 |
| kens | Oh boy :( | 12:33.48 |
| chrisl | It's another Kyocera printer! | 12:34.07 |
| | In this one, the ps2write output has LZW compressed inline images | 12:36.33 |
| kens | Is that a problem ? | 12:36.41 |
| | Though I would have expected Flate | 12:36.50 |
| chrisl | I don't know, but it's just about the only filter left! | 12:36.58 |
| | I was wondering if there would be a way to disable compression for inline images in ps2write? | 12:37.34 |
| kens | not as such | 12:39.26 |
| | You could try -dCompressPages=false | 12:39.33 |
| chrisl | That's already used | 12:39.45 |
| kens | Then I guess not | 12:39.55 |
| | could add it of course | 12:40.10 |
| chrisl | I'll manually hack the file and ask the user to run it, see if it is the problem | 12:41.09 |
| kens | OK well let me know. | 12:42.34 |
| | I cna try different filters too would have expected Flate there | 12:42.45 |
| chrisl | Yes, I was surprised - in fact, I was doubly surprised because the image is 8 bpc RGB | 12:43.18 |
| kens | could be a bug | 12:43.47 |
| chrisl | If you want to take a quick peak, I can send you the PDF source file...... | 12:44.49 |
| kens | sure I can ignore linearisation for longer that way :-) | 12:46.45 |
| | crazy bugged printers thugh Batman | 12:47.10 |
| Robin_Watts | s/crazy/holy/ | 12:47.51 |
| chrisl | kens: pdf winging its way to you now...... | 12:48.42 |
| kens | ok thanks | 12:48.51 |
| chrisl | kens: should have added the command line for this case: -q -dNOPAUSE -dBATCH -dSAFER -sDEVICE=ps2write -sOUTPUTFILE=%stdout -dLanguageLevel=3 -r600 -dCompressPages=false -dCompressFonts=false -dNoT3CCITT -c save pop -f | 12:49.57 |
| kens | OK lets see what I come up with, will eb a while, I have pdfwrite in pieces | 12:50.17 |
| | (ie I've broken it again) | 12:53.15 |
| | OK file is here | 12:53.38 |
| | BTW LanguageLevel=3 does nothing | 12:57.00 |
| chrisl | kens: yes, it's removed in the latest (but not released) version | 12:57.27 |
| kens | chrisl can I reduce this file at all ? 2 pages and lots of content | 13:02.58 |
| chrisl | I haven't tried - I don't see any reason why not, though | 13:03.28 |
| kens | Let me look at the toutput | 13:03.37 |
| chrisl | Obviously, leave the images in! | 13:03.46 |
| kens | :) | 13:03.53 |
| | I wonder if its the pattern using an image causing a problem.... | 13:09.33 |
| chrisl | Well, I can't convince it to read ASCII85 data without the LZW, so..... | 13:12.09 |
| kens | Give me a few minutes | 13:12.23 |
| | Its setting up Flate at the moment, so it must get overridden later | 13:12.35 |
| | Yes, its overriding it with 'lossless_template' which specifically uses LZW | 13:14.20 |
| | Oh because we can't use Flate in level 2 | 13:14.46 |
| chrisl | Actually, I was expecting DCT | 13:15.29 |
| kens | No we are specifying lossless ocmpression because its inline | 13:15.45 |
| | OK I have a version which only uses Ascii85, want to look ? | 13:17.55 |
| chrisl | kens: please | 13:18.19 |
| kens | 2MB on its way to you | 13:19.04 |
| chrisl | Robin_Watts: Looks like another interpolation problem: http://bugs.ghostscript.com/show_bug.cgi?id=693183 - -dNOINTERPOLATE it takes 2 seconds | 13:19.15 |
| kens | I guess I should have zipped it first. | 13:19.22 |
| chrisl | Pah, proper broadband! We don't need no stinking compression :-) | 13:19.52 |
| kens | I want fibre! | 13:20.04 |
| | (then I can complain how slow WiFi is) | 13:20.16 |
| | Coffee | 13:20.52 |
| chrisl | I can't believe that going from 2 seconds without interpolation, to (an alleged) 13 hours with interpolation can be considered "correct" behaviour! | 13:21.10 |
| kens | It really takes that long ? Seems 'odd' | 13:28.32 |
| chrisl | Well, I don't intend to leave it running for that long! | 13:28.55 |
| kens | I guess not | 13:29.01 |
| | Must be some kind of crazy stupid file | 13:29.11 |
| chrisl | I think there is transparency in the mix, too. | 13:29.36 |
| kens | I was wondering that, that alwqays makes it worse | 13:29.45 |
| chrisl | I still feel we're being over-zealous in the level of interpolation we're using | 13:30.38 |
| kens | Disable it! :-) | 13:30.49 |
| | Well, decompressing the file blows it up to 7Mb | 13:31.24 |
| | Hmm, PowerPoint, I wonder if there's some silly pattern | 13:32.01 |
| | Lot of images in ICCBased colour spaces that are interpolated | 13:34.24 |
| chrisl | So, do we interpolate in the source space, and then convert? | 13:34.54 |
| kens | I have no idea | 13:35.02 |
| | have to ask Robin/Michael | 13:35.10 |
| | But all the images seem to be in ICCBased spaces and all are interpolated | 13:35.23 |
| | One of the is Indexed with an ICC base :-) | 13:35.49 |
| | I take ti back one of them is in DeviceGray, its an SMask | 13:36.25 |
| chrisl | No other rip I've found suffers a similar performance differential between interpolated and not | 13:36.39 |
| Robin_Watts | The interpolation code is smart about whether to color convert then interpolate, or do it the other way. | 13:36.47 |
| kens | Only one of the images has an SMask, the others seem to be just images | 13:37.15 |
| Robin_Watts | chrisl: I'm (hopefully) very close to fixing (some) interpolation slowdowns. | 13:37.17 |
| chrisl | Robin_Watts: I thought that was only for halftoned output? | 13:38.08 |
| Robin_Watts | no, halftones don't come into it. | 13:38.18 |
| kens | Im26 and Im27 get used many times in the file | 13:38.23 |
| Robin_Watts | The example file happened to be halftoned. | 13:38.31 |
| | It's more to do with clipping. | 13:38.38 |
| | If we have a 256x256 image scaled up to 17067x17067 and only the middle 128x128 is actually displayed, we would still do the interpolation for all 17067x17067 pixels. | 13:39.21 |
| chrisl | Ah, I doubt that will affect this file (much, anyway) | 13:39.48 |
| kens | I don't think it will affect it at all | 13:40.03 |
| | I can't see any real reason why this would take so long. Well, over to Robin_Watts | 13:40.51 |
| Robin_Watts | joy. | 13:41.03 |
| kens | Hmm I wonder if UseFastColor will make a difference. | 13:41.36 |
| chrisl | Not if they are already ICC color spaces..... | 13:42.01 |
| Robin_Watts | what the hell happened to the cluster? | 13:42.11 |
| kens | cluster master had a problem perhaps | 13:42.45 |
| | lost contact wiht all the nodes ? | 13:42.57 |
| Robin_Watts | yeah, network outage or something. pain in the bum. | 13:43.12 |
| kens | well I can confirm that its much faster when run with NOINTERPOLATE | 13:43.36 |
| chrisl | Well, I just killed the Bug 693183 job at 12+ minutes - definitely an unreasonable performance differential, I reckon...... | 13:43.45 |
| kens | Yes, very much so | 13:43.54 |
| | I just don't see why | 13:44.06 |
| chrisl | At this rate we're going to have to make -dNOINTERPOLATE the default! | 13:45.37 |
| kens | jusrt running very sleepy on the file quickly, I won't let it run to completion though | 13:46.54 |
| | teh process spent 76.5% of its time in zoom_y which is part of the interpolation | 13:49.03 |
| | Called from image_render_interpolate_icc and s_IScale_process | 13:49.53 |
| | THe majority of that time was spent in a loop calculating 'wiehgt' | 13:50.40 |
| Robin_Watts | kens: yes, that's consistent with interpolation. It implies that we are generating a LOT of interpolated pixels. | 13:53.59 |
| kens | That does seem to be the case, though I cannot for the life of me see why. | 13:54.19 |
| | I'm not going to look at it any more though, better if you do it, I'll just be wasting my time :-) | 13:54.39 |
| Robin_Watts | I'll add it to the list :( | 13:54.57 |
| | If this bmpcmp ever finishes... | 13:55.10 |
| kens | you killed the cluster again ? | 13:55.31 |
| | Looks nearly done | 13:55.55 |
| Robin_Watts | I think the job timed out and started again. | 13:56.30 |
| kens | its all on'stnadby' now | 13:56.47 |
| | Oh, but 2 nodes not completed yet | 13:57.04 |
| chrisl | Robin_Watts: I had a go at implementing something like your suggestion for the gc_signal stuff - it *seems* to be working okay | 13:57.47 |
| Robin_Watts | chrisl: Ah, nice. | 13:58.09 |
| chrisl | I still think we should sit on it until after 9.06, though.... | 13:58.32 |
| Robin_Watts | Yes. It can go in with the big ref stuff and then we can all run away and hide. | 13:59.02 |
| kens | I'll chip in with linearised PDF otuptu :-) | 13:59.27 |
| | Might as well break everything | 13:59.40 |
| chrisl | Robin_Watts: if you're interested: http://git.ghostscript.com/?p=user/chrisl/ghostpdl.git;a=commitdiff;h=bec8828d | 13:59.55 |
| Robin_Watts | chrisl: It'll kill some time waiting for the cluster :) | 14:00.11 |
| chrisl | That's why I check the interpolation bug - waiting for my UFST test to finish. | 14:00.54 |
| Robin_Watts | Ok, that wasn't quite what I suggested. | 14:02.39 |
| | You're still keeping the 'signal_value' in each space. | 14:03.37 |
| | and hence set_gc_signal has to do lots of dancing through the spaces to set it in the relavant ones. | 14:04.11 |
| | I was hoping we could keep signal_value in the gs_lib_ctx too. | 14:04.34 |
| chrisl | At the moment, that would mean no way to spot gc'ed spaces, so we'd end up doing a lot more gc runs. | 14:05.15 |
| Robin_Watts | If we have some way of knowing from a space whether it would have been found by the search (equivalently, I believe, "whether it's gcable") then we can avoid the dancing entirely. | 14:05.55 |
| chrisl | I couldn't find annother way to tell unequivocally whether a memory space was gc-able or not | 14:07.14 |
| Robin_Watts | How does the dancing do it? Doesn't it give up when it finds that mem = mem->stable ? | 14:15.40 |
| chrisl | No, in the PS interpreter, we keep an array gc'ed spaces, and we walk the array | 14:17.08 |
| Robin_Watts | I thought there was an array of 'indexed' spaces, and we walked down the 'stability' chain from each one until we hit stable? | 14:21.04 |
| chrisl | Oh, yeh, we do - but I still don't think it will work. IIRC, the chunk allocator (for example) uses that for it's "parent" allocator | 14:23.44 |
| Robin_Watts | the chunk allocator has itself as it's stable allocator? | 14:24.19 |
| | Then the chunk allocator can't be gc'd, right? | 14:24.43 |
| | Failing that, we could put a 'is_gcable' boolean in every space when it's created. | 14:25.21 |
| chrisl | There are non-gc memory spaces that do not have themselves as their "stable_memory" parent. | 14:26.55 |
| Robin_Watts | chrisl; Right. But we'll never meet them during the 'dancing' process, right? | 14:27.24 |
| chrisl | Robin_Watts: No, we won't, but that's not the problem, the problem is when we write the value *back* the pointer - we must only do that for gc'ed spaces. | 14:28.13 |
| Robin_Watts | chrisl: Right. At the moment the 'dancing' process serves to identify those spaces which can trigger gcs. | 14:29.48 |
| chrisl | Robin_Watts: yes. | 14:30.08 |
| Robin_Watts | i.e. if we want to move the value to write back out of there, we need some other way to do that determination. | 14:30.29 |
| chrisl | Yes. | 14:30.50 |
| Robin_Watts | Balls. The cluster looks like it's queuing too many mujstest jobs again. I thought I'd solved that :( | 14:31.03 |
| kens | THey've all vanished suddenly | 14:34.05 |
| Robin_Watts | I edited the list :) | 14:34.40 |
| kens | cheat ! | 14:34.54 |
| Robin_Watts | The command from that bug completes in under 10 seconds for me. | 14:42.35 |
| chrisl | With your revisions, or with master? | 14:42.59 |
| Robin_Watts | With my revisions. | 14:43.05 |
| | I'm about to try without. | 14:43.15 |
| chrisl | Well, that is excellent! | 14:43.20 |
| kens | absolutely | 14:43.33 |
| Robin_Watts | Without, it's taking longer. | 14:44.56 |
| kens | sounds like a fix then | 14:45.29 |
| Robin_Watts | 3 minutes to forms meeting. | 14:58.51 |
| | paulgardiner, tor8, henrys: ping | 14:59.07 |
| henrys | as usual form stuff seems to be progressing along? Do we need a meeting this week? | 14:59.08 |
| Robin_Watts | paulgardiner has no access to a 64bit linux box. | 14:59.25 |
| | Can we get him an account on peeves? | 14:59.30 |
| | (or one of the others) | 14:59.39 |
| henrys | sure I thought he had one. | 14:59.57 |
| Robin_Watts | Other than that, I have nothing for the meeting. | 15:00.05 |
| | He's on casper, but not on peeves, AIUI. | 15:00.13 |
| paulgardiner | On the other hand, all the segv's, that seemed to be 64bit specific have disappeared of their own volition | 15:00.42 |
| henrys | oh right | 15:00.45 |
| tor8 | I have one (very) minor gripe. javascript/util.js should probably live in pdf/pdf_util.js instead | 15:00.47 |
| Robin_Watts | No /home/paulg on peeves that I can see. | 15:00.51 |
| henrys | we'll ask ray_laptop at the meeting. | 15:01.34 |
| paulgardiner | tor8: No probs. I'll move it | 15:01.35 |
| Robin_Watts | (casper is 64bit linux, but using casper for dev work is likely to overload it) | 15:01.41 |
| tor8 | paulgardiner: I know we keep the font and cmap directories on the top level, but who said I have to be consistent? ;) | 15:02.32 |
| paulgardiner | tor8: I usually feel I have no idea where anything should go, so I'm grateful of anyone with an opinion. :-) | 15:03.31 |
| | Just a reminder. I'm away for a week from sat, so wont be able to make the meeting next week. | 15:04.50 |
| henrys | paulgardiner:okay | 15:05.12 |
| Robin_Watts | paulgardiner: The cluster tests are showing a couple of segvs (in different files each time) on the last few tests. | 15:07.26 |
| | Normally on i7. | 15:07.51 |
| | and a couple on meters. | 15:07.58 |
| | oh, then a couple on miles. | 15:08.19 |
| henrys | tor8:on a different subject how's the viewer? | 15:08.38 |
| paulgardiner | Robin_Watts: Access to a 64bit machine would certainly be handy. Sonds like I need to run valgrind too, as you'd suggested before. | 15:09.27 |
| tor8 | henrys: no progress since last meeting... :( | 15:09.53 |
| | henrys: but I did fix a bug from 2007! | 15:10.52 |
| henrys | I'm excited about getting a decent viewer out that can be used with all the languages. Maybe I'm too easily excited. | 15:12.02 |
| tor8 | I'm excited too, but UI programming has a way to kill brain cells... :( | 15:12.46 |
| | it's all that digging around in crufty frameworks and bad documentation trying to fit the pieces together | 15:13.10 |
| henrys | this is gtk3 based? | 15:13.14 |
| tor8 | gtk2 for now, but trying to stay away from deprecated functions | 15:13.35 |
| paulgardiner | tor8: is the finding out what pieces are needed the worse bit, or is it still pretty horrid once you know? | 15:15.45 |
| henrys | casper is a pair on Xeon 2.33 G, you paulgardiner had performance problems? | 15:16.24 |
| | s/you/ | 15:16.33 |
| | s/you// | 15:16.37 |
| Robin_Watts | henrys: No. I know that previously casper has struggled to cope with the weight of mailing lists/git/cluster etc. | 15:17.07 |
| | but that may have been before we moved to a bigger instance. | 15:17.17 |
| henrys | no big deal either way we can talk to ray at the next meeting. | 15:18.29 |
| tor8 | paulgardiner: I've seen worse, but it's still pretty awkward. I've been fairly pleasantly surprised how easy some things were, once I understood which signals connected where. | 15:20.57 |
| mvrhel | good morning | 15:26.59 |
| Robin_Watts | mvrhel: Don't jinx it :) | 15:27.12 |
| kens | Hi mvrhel | 15:27.15 |
| mvrhel | uhoh | 15:27.24 |
| Robin_Watts | Tea! | 15:27.42 |
| mvrhel | my blocker bug is driving me crazy | 15:27.53 |
| | Hopefully I can figure out what is going on with it today | 15:28.12 |
| | looks like there were some discussions about interpolation and the color space in which they are occuring? | 15:29.57 |
| | I thought we set it up so that we always do the color management step at the spot that will give us the fewest pixels to transform | 15:30.58 |
| | i.e. do it first when enlarging and after when shrinking | 15:31.13 |
| kens | mvrhel : Robin_Watts fix seems to solve the problem nicely | 15:31.35 |
| | We were just talking round the problem at the time | 15:31.53 |
| mvrhel | good deal | 15:33.07 |
| | Gemma's reply to marcos' didn't really answer his question | 15:36.42 |
| | back to mine a built more salt before the meeting | 15:39.15 |
| Robin_Watts | mvrhel: bug 694173 (rendering takes ages unless no transparency set) | 15:39.30 |
| | That looks like it's stroke path calls that are hurting. | 15:39.50 |
| mvrhel | I cant find that bug | 15:40.25 |
| kens | 693173 | 15:40.47 |
| Robin_Watts | sorry. | 15:40.59 |
| mvrhel | ah ok | 15:40.59 |
| | isnt that what your one fix took care of Robin_Watts ? | 15:41.39 |
| | or is this different? | 15:41.51 |
| Robin_Watts | The 'knockout group' stuff? | 15:41.56 |
| kens | I think differnet | 15:41.59 |
| mvrhel | yes | 15:42.00 |
| Robin_Watts | That's not gone in yet. It's blocked on alexcher fixing smasks. | 15:42.12 |
| mvrhel | oh | 15:42.18 |
| | does it solve the problem? | 15:42.27 |
| Robin_Watts | I will try. | 15:42.38 |
| mvrhel | I had hoped alexcher was going to get that in place before the release | 15:42.47 |
| Robin_Watts | I am still hoping that :) | 15:42.56 |
| mvrhel | we can bring it up at the meeting | 15:43.07 |
| | in 15 minutes | 15:43.15 |
| | let me know if you patch solves the issue | 15:43.57 |
| | s/you/your/ | 15:44.04 |
| | Robin_Watts: ^^ | 15:44.09 |
| Robin_Watts | just rebased it. building now. | 15:44.30 |
| mvrhel | so on my blocking bug, there is something weird going on with some copy_planes that are going through the clist | 15:45.08 |
| | I have mask fills with patterns | 15:45.33 |
| | that end up as copy_planes in the clist | 15:45.47 |
| | then the are eventually read back as copy_planes for the mem planar device | 15:46.12 |
| | and it is odd since the Yellow and Black planes end up being the same as the Magenta plane | 15:46.51 |
| | in the final output | 15:46.55 |
| | in non-clist mode all is fine | 15:47.05 |
| Robin_Watts | Either the data must be being put into the clist wrong, or it's being read out wrong. | 15:48.45 |
| Robin_Watts | states the obvious. | 15:48.55 |
| mvrhel | yes. I stepped through it and didnt see anything obvious. | 15:49.18 |
| | now I need to look for the in-obvious... | 15:49.38 |
| | at least there is not any transparency... | 15:49.56 |
| | to add to the fun | 15:50.06 |
| henrys | yikes another regression for mvrhel | 15:53.00 |
| Robin_Watts | might that be related to the thing ray found with the hl_color stuff ? | 15:54.00 |
| | or has that fix gone in? | 15:54.04 |
| mvrhel | who what where | 15:54.48 |
| henrys | 693184 | 15:54.59 |
| mvrhel | I had a conversation with ray about that yesterday | 15:55.08 |
| | the hl_color fixed rect stuff I mean | 15:55.21 |
| Robin_Watts | I can't see a commit relating to it. | 15:55.37 |
| mvrhel | there is not one | 15:55.44 |
| | we discussed a solution | 15:55.48 |
| | he is working on it | 15:55.53 |
| Robin_Watts | right. | 15:55.54 |
| mvrhel | 693184 is new | 15:55.59 |
| | and related to AA is appears | 15:56.06 |
| Robin_Watts | I was just saying that maybe that will explain bug 693184 ? | 15:56.11 |
| mvrhel | I would be surprised | 15:56.22 |
| | likely some mistake in my default implementation of copy_alpha hl_color | 15:56.46 |
| | that was a tough out to QA | 15:57.05 |
| | s/out/one/ | 15:57.10 |
| | I ran it to make sure there were no segvs | 15:57.23 |
| | and scrolled through the diffs | 15:57.34 |
| | oh | 15:58.00 |
| | this one is like the whole thing is missing | 15:58.09 |
| | maybe you are right Robin_Watts | 15:58.15 |
| | should be easy to figure out at least compared to this thing I am working on now | 15:58.56 |
| henrys | so do we make that a blocker? | 16:00.10 |
| mvrhel | I would think so | 16:00.15 |
| | the odds of my color changes for customer 330 getting into the release are fading fast :( | 16:00.57 |
| henrys | done | 16:00.59 |
| | yeah I would just abandon that for now and focus on the blockers. | 16:01.18 |
| mvrhel | ok | 16:01.22 |
| | sounds like a wise idea | 16:01.39 |
| kens | meeting time ? | 16:01.47 |
| henrys | we call them blockers but the release is going hell or high water before august is over. | 16:02.00 |
| | kens:that was my meeting opening. | 16:02.14 |
| kens | :-) | 16:02.24 |
| mvrhel | these should be easy to resolve compared to the initial commits for the fixes that led to the regressions | 16:02.35 |
| Robin_Watts | ray_laptop: Can paulgardiner get an account on peeves please? | 16:02.40 |
| henrys | chrisl:so it looks like the release will happen when mvrhel unblocks I don't see anything else stopping it. Do you? | 16:02.57 |
| mvrhel | it would be nice to get the fix from alexcher in | 16:03.22 |
| | so that Robin_Watts can get his transparency fix in | 16:03.30 |
| ray_laptop | Robin_Watts: yes. just a minute | 16:03.30 |
| chrisl | henrys: I thought ray_laptop had a blocker, too..... | 16:03.31 |
| Robin_Watts | feeds mvrhel some prunes. | 16:03.34 |
| ray_laptop | Robin_Watts: oops. Did paul send me his public key | 16:03.57 |
| mvrhel | ray is being kind enough to work on a blocker that should be mine too | 16:04.09 |
| henrys | oh let's do a blocker search. | 16:04.16 |
| alexcher | mvrhel: yes, I'm working on the bug 693115. | 16:04.16 |
| ray_laptop | chrisl: yes. The segv with psdcmyk | 16:04.18 |
| chrisl | ray_laptop: the one I was thinking about was Bug 692542 | 16:04.46 |
| ray_laptop | oops. My wife has to go with her mom to yet another bank to lock it down | 16:04.52 |
| | chrisl: yes -- it's a blocker because of the segv | 16:05.08 |
| chrisl | Right, I was just seeing that in the comments | 16:05.27 |
| ray_laptop | I have to run back home -- be back online in about 5 min. | 16:05.29 |
| mvrhel | Robin_Watts: are you trying to get me to pipe down :) | 16:05.35 |
| Robin_Watts | :) | 16:05.50 |
| | ray_laptop: He says "can you grab it from casper"? | 16:06.00 |
| henrys | which bug of alexcher did you want in? | 16:07.42 |
| kens | 693115 | 16:07.56 |
| | THe smask mishandling | 16:08.05 |
| mvrhel | right | 16:08.16 |
| henrys | so we'll block on that one too? | 16:09.13 |
| mvrhel | well I don't know about that. it would be nice to get it in though | 16:09.43 |
| henrys | okay so everyone knows blocker's are priority, yes? | 16:09.45 |
| | I already changed it. | 16:10.18 |
| mvrhel | ok sounds good to me | 16:10.24 |
| chrisl | Do we want to block on performance bugs? | 16:10.47 |
| henrys | alexcher:good? | 16:10.49 |
| mvrhel | everyone needs a blocker on their plate | 16:10.50 |
| alexcher | henrys: yes | 16:11.05 |
| chrisl | mvrhel: can I be exempt from that since I have to do the release? ;-) | 16:11.22 |
| henrys | chrisl:what were you thinking of? | 16:11.30 |
| | the interp stuff? | 16:11.38 |
| kens | having a blocker :-) | 16:11.48 |
| chrisl | henrys: no the mishandled smask problem was prompted by a performance issue, IIRC | 16:12.13 |
| mvrhel | yes. it was a significant one | 16:12.28 |
| | though | 16:12.36 |
| henrys | the other thing for the meeting is to review the tech agenda and see that you make some progress on projects before our september get together. | 16:12.41 |
| | other than that I didn't have anything else for this meeting. | 16:13.22 |
| ray_laptop | OK. I'm back | 16:13.34 |
| Robin_Watts | mvrhel: With the fix in, it still runs slowly. but 60% of the time is now spent in gc. | 16:13.38 |
| ray_laptop | Robin_Watts: yes, I'll grab the key from casper | 16:13.48 |
| henrys | ray_laptop:do you have any money left? | 16:13.50 |
| ray_laptop | henrys: last I checked, yes | 16:14.08 |
| henrys | I can spot you 20.00 if you need it. | 16:14.09 |
| Robin_Watts | If so, I have a nigerian friend who needs help getting a few million $ out of the country... | 16:14.25 |
| mvrhel | Robin_Watts: well I would say that your "fix" makes a difference. not sure what to make of the time spent in GC | 16:14.36 |
| henrys | ray_laptop:just check out the logs for the meeting, I think we finished early. | 16:15.57 |
| | ray_laptop:do you expect to have your blocker wrapped up soon? | 16:16.32 |
| kens | If we are done, I will run off, got guests visiting tonight | 16:16.45 |
| henrys | kens:goodnight | 16:17.26 |
| mvrhel | oh I had something for marcosw | 16:17.29 |
| Robin_Watts | Damn. I have just a few differences left with this interpolation stuff. Mostly in xps. | 16:17.32 |
| kens | Bye all | 16:17.35 |
| marcosw | sorry I'm late | 16:17.37 |
| Robin_Watts | night kens | 16:17.40 |
| tor8 | Robin_Watts: sebras: paulgardiner: a handful of patches on tor/master that need a second set of eyeballs | 16:17.40 |
| Robin_Watts | tor8: I'll look. | 16:17.51 |
| mvrhel | just in time for me marcosw | 16:17.53 |
| | a few questions about 693061 | 16:18.34 |
| | marcosw: did you see my comment 5? | 16:19.02 |
| | I am working on the issue with the initial file but I didn't see issues in the files from Comment 4 | 16:19.59 |
| marcosw | yes, I was going to recheck to see if all of those files broke at the same time, but I'm traveling (in Long Beach @ ISMB) and the internets at the conference sucks | 16:20.21 |
| mvrhel | marcosw: ok. when ever you get the time then. I have plenty to keep me busy..... | 16:20.48 |
| | back in 2 minutes | 16:22.14 |
| marcosw | you don't see any issues with those files or just not any caused by the commit referenced in 693061? | 16:22.32 |
| ray_laptop | paulgardiner: please try ssh to peeves now | 16:23.42 |
| sebras | tor8: oh, ok. let me return home and have some food and then I'll get back to you. | 16:23.48 |
| paulgardiner | Yep. I'm in, thanks Ray. | 16:25.26 |
| ray_laptop | henrys: et al.: I discovered a small worm nest w.r.t. fill_rectangle_hl_color that I am fixing, that _should_ clean up the segv for the blocker. Hopefully finish today (if I don't waste too much time again at the banks) | 16:25.51 |
| | paulgardiner: great | 16:25.57 |
| henrys | ray_laptop:okay | 16:26.50 |
| Robin_Watts | tor8: I see 2 patches. both look fine to me. | 16:27.48 |
| mvrhel | marcosw: no there are diffs | 16:28.08 |
| | but they are progressions | 16:28.14 |
| tor8 | there should be 4 patches | 16:28.25 |
| mvrhel | except for the ones that I mentioned | 16:28.25 |
| ray_laptop | henrys: that's the issue that mvrhel alluded to earlier. He and I discussed it earlier. We decided I would add a new fill_rect_hl_color that was like fill_rectangle (dev, x, y, w, h, pdcolor) to avoid the messy confusion with the fixed vs. int coordinates | 16:28.31 |
| Robin_Watts | The one thing that occurs to me, is that we should have a call that marks there as being a render error in the cookie. | 16:28.41 |
| tor8 | font style symbol voodoo, xps font synthesis tweaks, and two for drawing fonts as outlines | 16:28.53 |
| Robin_Watts | tor8: Oh, then I've only looked at the last 2. Let me look at the others too. | 16:28.55 |
| tor8 | I'm unsure if I've got all the lock nesting with try/catch correct for the outline font stuff | 16:29.39 |
| marcosw | mvrhel: I just took a look at one of the files that I listed in comment #4 and don''t see any differences. I checked the logs and the md5sum for many of the psdcmyk files changed a few weeks ago, perhaps the problem that I originally saw with those files was fixed. I'll recheck all of the files when I'm home tonight. | 16:29.46 |
| mvrhel | marcosw: ok cool. thanks | 16:30.14 |
| ray_laptop | henrys: we will deprecate the old fill_rectangle_hl_color. We were tempted to violate the 'rules' and just change the calling sequence on the old method, but were concerned that a customer device might have implemented the old method | 16:30.34 |
| Robin_Watts | tor8: The other 2 patches look fine. As to locking... if you test a debug build, you should get FITZ_DEBUG_LOCKING defined | 16:34.12 |
| | and that should check that there are no locking order violations. | 16:34.29 |
| marcosw | alexcher: there seems to be an issue with the alex_x6 cluster node. It's stuck "Updating mupdf". Do I have an account on that machine? | 16:34.31 |
| ray_laptop | I wanted to ask what others thought -- can I just violate the rules and change the calling sequence to fill_rectangle_hl_color ? | 16:35.16 |
| alexcher | marcosw: yes | 16:35.17 |
| Robin_Watts | ray_laptop: If you do so, will people get an obvious error when they compile? | 16:35.45 |
| marcosw | okay, I'll take a look. Is the cluster code in my account? | 16:36.00 |
| Robin_Watts | (i.e. are you changing the number of params or from float -> int or anything?) | 16:36.04 |
| alexcher | marcosw: the cluster directory is full with files like 00)Device_Push.pam | 16:36.26 |
| ray_laptop | a customer device that has the old fill_rectangle_hl_color would not compile, so it would be easy enough for them to fix | 16:36.26 |
| Robin_Watts | AIUI, fill_rectangle_hl_color was one of igors, right? So it's been around a long time. | 16:36.30 |
| ray_laptop | Robin_Watts: changing the number of params AND the types | 16:36.47 |
| Robin_Watts | ray_laptop: Right, so there is little danger of someone not realising there is a problem. | 16:37.08 |
| marcosw | alexcher: yeah, that's a problem. anyone here want to admit to writing out those files as debugging? | 16:37.13 |
| Robin_Watts | I personally think it's past time we had a big cleanout of the device procs, but... | 16:37.31 |
| ray_laptop | Robin_Watts: it's been around a long time, but the only thing (until after 9.05) it was used for was fillpage | 16:37.34 |
| marcosw | as a hack I'll fix run.pl to delete those files when it starts, but you'll need to make a bit of space so the update of run.pl works. | 16:37.51 |
| Robin_Watts | marcosw: That was me, ages ago. | 16:38.08 |
| | Just delete *.pam and *.raw from the cluster dir. | 16:38.30 |
| | sorry. | 16:38.37 |
| ray_laptop | so Robin_Watts is in favor of letting me just change the params. Anyone else want to weigh in (or decline to be involved) | 16:39.00 |
| marcosw | Robin_Watts: not ages ago, the files are dated July 13. | 16:39.02 |
| Robin_Watts | oh, well, not me then. | 16:39.28 |
| ray_laptop | mvrhel: as you can tell -- after sleeping (fitfully) on it, I've sort of changed my thinking | 16:39.38 |
| marcosw | Robin_Watt: that doesn't quite work, you get an "Argument list too long" error. You have to delete them in batches. | 16:40.05 |
| | I've enabled the .pam/.raw deletion code in run.pl, so they'll go away the next cluster run. | 16:40.45 |
| ray_laptop | I've never understood why linux hasn't fixed that arg list too long. | 16:40.49 |
| mvrhel | ray_laptop: great. I think we should just do the change too. | 16:41.41 |
| ray_laptop | mvrhel: OK. thanks | 16:41.49 |
| chrisl | ray_laptop: you can change the allowed size of the command line | 16:42.05 |
| ray_laptop | alexcher: chrisl: henrys: ? | 16:42.11 |
| mvrhel | I would be *very* surprised to see someone using the current one | 16:42.24 |
| | and it would be an easy thing for them to fix | 16:42.33 |
| chrisl | ray_laptop: yes? | 16:42.49 |
| ray_laptop | chrisl: about changing fill_rectangle_hl_color calling sequence to make it like fill_rectangle | 16:43.29 |
| chrisl | ray_laptop: I think the chances of a customer having implemented their own version are minute, and if your changes break the build, then go for it! They can just ask | 16:44.28 |
| ray_laptop | chrisl: thanks | 16:47.05 |
| alexcher | marcosw: Is it possible to add 32-bit user regression against 64-bit master? | 16:48.52 |
| marcosw | you mean a cluterpush in 32 bit mode? | 16:49.15 |
| alexcher | marcosw: yes. | 16:50.17 |
| marcosw | shouldn't be a problem⦠(famous last words) | 16:50.57 |
| | oops, battery about to die, will be back after I find an outlet | 16:51.22 |
| alexcher | marcosw: We've alsoe seen differences between real 32-bit hosts and -m32 builds. | 16:51.26 |
| marcosw | alexcher: not sure I can do anything about that, unless we want to setup a second cluster of 32 bit machines :-) | 16:51.56 |
| chrisl | alexcher, marcosw: It will depend on all the nodes having 32 bit libraries, which may or may not be the case | 16:52.00 |
| alexcher | chrisl: the libraries can be installed. | 16:54.04 |
| chrisl | alexcher: yes, it isn't a fundamental issue, I just thought I'd mention it since it might stop it working right now | 16:54.43 |
| apineda | "Unrecoverable error: rangecheck in showpage Operand stack: 1 true" Anyone can point me to the most likely cause of error? It seems to be at the end of the file. Its a 7MB pdf, 40 separation colors, graphic stack nesting is like 6 max. I've tested a smaller file with 12 separation colors and that worked no problem. | 16:58.12 |
| | using 9.05 | 16:58.43 |
| | tiffsep | 16:58.51 |
| chrisl | apineda: probably the 40 separations colors is the problem - I think the limit is 16 or 32 in 9.05 | 17:00.12 |
| ray_laptop | apineda: probably rangecheck is due to non-encodeable colors | 17:00.59 |
| Robin_Watts | apineda: It would be worth trying again with the latest version from git as that ups the number of spots to 60 or so. | 17:01.14 |
| apineda | sorry but what does non-encodeable mean? | 17:01.26 |
| | the doc says "Generally Ghostscript will support a maximum of 64 process and spot colors. However the maximum may be smaller if the C compiler that was used to build Ghostscript does not support a 64 bit integer type. MSVC, gcc, and most current compilers do support a 64 integer type. For those compilers that have 32 bits as the largest integer type, then only 4 separations can be produced... | 17:01.30 |
| | ...per Ghostscript pass." | 17:01.31 |
| ray_laptop | Robin_Watts: it also gets rid of the 'compressed color encoding' | 17:01.40 |
| Robin_Watts | ray_laptop: Indeed. | 17:01.55 |
| ray_laptop | apineda: were you using the tiffsep device ? | 17:02.19 |
| | (or the psdcmyk device) ? | 17:02.30 |
| apineda | Robin_Watts: I'll try that... I might need help getting it to 64 release config | 17:02.41 |
| | tiffsep | 17:02.44 |
| Robin_Watts | apineda: What platform ? | 17:02.54 |
| ray_laptop | compressed color encoding was used by default, but now mvrhel has fixed that so we can handle spot colorants correctly | 17:03.28 |
| | the previous spot color limit was 14 iirc | 17:03.50 |
| apineda | x64 | 17:04.09 |
| Robin_Watts | apineda: Linux, presumably then? | 17:04.22 |
| apineda | win 7 | 17:04.27 |
| ray_laptop | but even with that, we had some files from one of our customers that would get the rangecheck in showpage because we couldn't pack all of the colors in 64 bits | 17:04.39 |
| apineda | msvc then :D | 17:04.43 |
| Robin_Watts | Ah, well the stanard 32bit build will be fine for you. | 17:04.44 |
| ray_laptop | apineda: but you still need the 'latest and greatest' git version to avoid the compressed encoding | 17:05.52 |
| apineda | Robin_Watts: thank you | 17:05.53 |
| | ray_laptop: I can do that I think. Anonymous pulls are pretty simple no? :D I use svn here :( | 17:06.48 |
| Robin_Watts | apineda: Or you can download a snapshot from the git web interface. | 17:07.28 |
| ray_laptop | Robin_Watts: mvrhel: GS_CLIENT_COLOR_MAX_COMPONENTS is still 14 in the current code (base/gsccolor.h) | 17:07.36 |
| | mvrhel: weren't we going to change that ? | 17:08.22 |
| Robin_Watts | apineda: Click on the top right "snapshot" link. | 17:08.26 |
| | apineda: And as ray says, you may need to edit base/gscolor.h to make GS_CLIENT_COLOR_MAX_COMPONENTS higher. | 17:08.56 |
| apineda | Robin_Watts: but I'll make it 1000 | 17:09.26 |
| Robin_Watts | apineda: http://git.ghostscript.com/?p=ghostpdl.git;a=summary | 17:09.27 |
| | apineda: No. | 17:09.32 |
| | Make it no bigger than 64. | 17:09.40 |
| ray_laptop | Robin_Watts: with the planar code, aren't we limited to 64 ? | 17:09.51 |
| Robin_Watts | ray_laptop: We are limited to 64 as the rest of the code assumes that we need 1bpc at least. | 17:10.26 |
| apineda | ok 64 then | 17:10.35 |
| | thanks | 17:10.46 |
| | more thanks | 17:10.55 |
| Robin_Watts | apineda: Let us know how you get on | 17:10.56 |
| ray_laptop | Robin_Watts: we probably need a comment in gsccolor.h to identify the limit, and maybe even a compiler fail if it is too big (via a /DGS_CLIENT_COLOR_MAX_COMPONENTS=99) | 17:11.04 |
| | apineda: if you can send us the 40 spot color file, we'd appreciate having one that's "real world" for our testing | 17:12.31 |
| apineda | ray_laptop: yeah I think I can do that | 17:15.05 |
| ray_laptop | apineda: thanks. we'll put it in our 'tests_private' unless you tell us it can be made public | 17:16.25 |
| | mvrhel: I can't recall why we decided to leave the default limit at 14 -- do you recall ? | 17:17.26 |
| apineda | ray_laptop: It will have to stay private for now, I'll talk to my boss later. Please let me know if there is anything of note. https://www.box.com/s/46164d6cd29515b95332 | 17:27.44 |
| ray_laptop | yuck. another lurking gotcha. Why in the world does mem_abuf_fill_rectangle_hl_color NOT limit the height ? | 17:28.06 |
| | apineda: umm... by posting a URL here, it ain't private -- this is a PUBLIC forum with logs. | 17:28.46 |
| apineda | yeah but im deleting it | 17:28.53 |
| | I guess you're right though | 17:29.16 |
| ray_laptop | apineda: I'm downloading it, I'll let you know as soon as I have it. | 17:29.31 |
| apineda | I just didn't think anyone else would bother :D, so get it now! | 17:29.34 |
| ray_laptop | apineda: OK. go ahead and delete it | 17:29.53 |
| apineda | ok | 17:30.07 |
| Robin_Watts | tor8: ping | 17:53.51 |
| alexcher | marcosw: alex_x6 is still marked as -down- . How can I find the cause? | 18:24.05 |
| marcosw | I know the cause. There was a bug in run.pl that caused the code to exit with an error and not remove the run.pid file. I've fixed the bug and alex_x6 will reset the lock file at 120 minutes, so you don't need to do anything. | 18:25.06 |
| | alexcher: could you run 'sudo apt-get install g++-multilib' on your cluster machines? This will install the 32 libraries. | 18:30.31 |
| alexcher | marcosw: doing now | 18:31.27 |
| marcosw | alexcher: sorry, could you also: sudo apt-get install lib32z1-dev | 18:50.12 |
| | henrys: could you 'sudo apt-get install lib32z1-devg++-multilib' | 18:54.43 |
| | henrys: sorry, I meant 'sudo apt-get install lib32z1-dev g++-multilib' on henrysx6 | 18:55.21 |
| alexcher | marcosw: alex_x6 is done. I need to find a working mirror for i7a and i7b | 18:56.27 |
| henrys | will do | 19:11.58 |
| | done | 19:13.51 |
| tor8 | Robin_Watts: yes? | 19:14.52 |
| Robin_Watts | tor8: False alarm, sorry. | 19:15.07 |
| | Do you want to push your patches, or should I ? | 19:15.23 |
| tor8 | Robin_Watts: I see one regression in sane, but that may be another patch | 19:16.41 |
| Robin_Watts | Sounds like I should leave it to you. | 19:17.01 |
| tor8 | 1522*.pdf from sumatras tests is missing a diagram | 19:17.25 |
| | and I have a lot of 'invalid key length 128' errors too, that I need to bisect | 19:17.47 |
| | but I don't think they're related to my changes | 19:18.08 |
| | Robin_Watts: okay, tested, the problems are earlier regressions so I'll go ahead and push the patches now | 19:29.59 |
| | and figure out what's gone wrong tomorrow | 19:30.08 |
| mvrhel | aha! found the problem | 19:41.01 |
| | clist copy planes had a bug | 19:42.54 |
| | ok that fixed it | 19:43.50 |
| | now to clean up all my debug stuff and do a cluster push | 19:44.02 |
| Robin_Watts | mvrhel: Nice. | 19:48.41 |
| mvrhel | somehow I did not have the index in the plane loop in the computation to update to the next plane :0 | 19:49.17 |
| | not sure how that got by | 19:49.24 |
| | so it was doing the first plane with the command | 19:49.33 |
| | and then the magenta (or second plane) was getting written for all the subsequent planes | 19:49.49 |
| | hopefully that will take care of one of the blockers | 19:55.42 |
| | now onto 693184 | 19:56.24 |
| sebras | tor8: the invalid key length sounds more like I might have messed up the crypto stuff somehow? | 20:03.19 |
| | tor8: the invalid key length sounds more like I might have messed up the crypto stuff somehow? | 20:05.44 |
| mvrhel | bbiaw | 20:12.33 |
| sebras | tor8: yeah, e401113 is the offender. | 20:21.48 |
| | tor8 Robin_Watts: d'oh, patch over at sebras/master fixing "invalid keys length"s... | 20:37.24 |
| ianneub | Howdy all. I've been working with the mudraw.exe -ttt flag to get XML about the text in a PDF file. I'm having trouble parsing the bbox attribute. Should those be measured in pixels? Or PDF units? | 21:11.06 |
| sebras | ianneub: normally mupdf never uses pixels anywhere. my best bet is userspace units, and I'd assume 72dpi. however, most of the other people here are better equipped to answer this kind of questions than I am. | 21:22.47 |
| ianneub | sebras: thanks, i'll see if i can use that to make these bbox's align better | 21:23.24 |
| sebras | ianneub: sure, good luck. | 21:48.23 |
| marcosw | alexcher and henrys: one more set of 32 bit libraries: sudo apt-get install ia32-libs | 22:23.43 |
| fm__ | chrisl_away, we will talk here soon regarding to https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1025061 | 22:53.46 |
| mvrhel | oh I think this new issue with the missing objects from Gemma has to do with a clear varnish spot colorant | 23:39.54 |
| | that is not coming out so clear.... | 23:40.01 |
| | oh actually no, it looks to be an overprint problem | 23:42.38 |
| | the varnish drawing which should be doing an overprint is blowing away the other colorants | 23:42.55 |
| | ok off to make dinner | 23:44.44 |
| | Forward 1 day (to 2012/07/18)>>> | |
IRC Logs