| <<<Back 1 day (to 2013/04/15) | 2013/04/16 |
mvrhel_laptop | ick in adding in the monitoring for color in the cm I see an MT issue that could have come up when we are doing the no color management case and the replace color stuff | 04:58.47 |
| ok. all the changing of the color management procs are in place now for color monitoring. now to add in the monitoring | 05:19.37 |
| ugh some issues to think about when the profile is identiy | 05:31.16 |
| when the link is the identity | 05:31.26 |
| we need to monitor but we have special paths to avoid cm | 05:31.43 |
| I guess I will need to turn of the is_identity flag, forcing color management, and then deal with the data conversions that may exist | 05:32.31 |
| but not do any color managment | 05:32.43 |
| sigh. it looks like the link is going to need to have some information about the source color space packed into it. | 05:53.05 |
| oh actually I think it already is in there with the buffer description | 05:55.15 |
| nice VS crash.. | 05:59.16 |
| had not had that in a while | 05:59.33 |
| henrys: so under $200 for me to fly to the open printing meeting so not too bad | 06:41.40 |
| good night all | 06:53.06 |
kens | chrisl ping | 08:44.39 |
chrisl | kens: pong | 08:46.59 |
kens | got a build error, not sure why. WHen I do male clean I get '../common/ugcc_top.mak:68 @FAPIUFST_MAK@: No such file or directory | 08:47.51 |
| THis is Linux | 08:47.57 |
| I think I did a git pull -- rebase so I did a / | 08:48.21 |
| ./confgure first. GS builds OK | 08:48.27 |
chrisl | You need to run autogen.sh | 08:48.37 |
kens | ah, OK thanks | 08:48.42 |
kens | can't keep configure and autogen straight | 08:49.04 |
chrisl | In both ghostpdl (for pcl and xps) and ghostpdl/gs (for gs) | 08:49.08 |
kens | OK make clean works now thanks | 08:49.47 |
chrisl | kens: the best thing to do, I find, is (assuming you have no important non-tracked files), after you do git pull, then do "git clean -x -f -d" and that will remove *everything* that didn't come from git. Then there's only once script left that you can run. | 08:50.52 |
kens | :-) | 08:51.08 |
| I'll try and bear that in mind | 08:51.14 |
chrisl | Or, in the near future: make maintainer-clean will also work | 08:51.39 |
kens | Some new target you're working on ? | 08:51.57 |
chrisl | The target is already there in gs, but it's incomplete, and I'm adding it to ghostpdl for consistency | 08:52.37 |
kens | makes sense | 08:52.49 |
chrisl | Basically, "make clean" removes the files created by "make", "make distclean" gets you back to what's in a distributed archive, and "make maintainer-clean" gets you back to what comes out of git. | 08:54.00 |
kens | OK sounds useful | 08:54.16 |
chrisl | Like I say, all there, but bit rotted | 08:54.36 |
| kens: I've got a public domain implementation of sprintf() integrated here (no localization!). Unsurprisingly it causes quite a few diffs in pdfwrite...... | 08:57.20 |
kens | Oh great.... | 08:57.35 |
| How bad is it ? | 08:57.58 |
chrisl | So far, all pixel level differences | 08:58.07 |
kens | OKso probably precision then | 08:58.35 |
chrisl | Yeh, the floating point rounding is very slightly different, which isn't surprising | 08:59.05 |
kens | No, that's not at all surprising, unfortunate, but.... | 08:59.22 |
chrisl | I may look into whether upping the precision would help, but I suspect not - it's a quite difference implementation to the glibc/bsd libc versions | 09:00.41 |
kens | Well we probably get slightly different results on Windows as well, so if its just pixels I doubt its worth worrying about | 09:01.28 |
chrisl | I actually grabbed it from SQLite: it says it's a public domain version from the '80's which they need because (shock horror!) they need a version without localization! | 09:02.56 |
kens | :-) | 09:03.05 |
chrisl | Given that, I find it even more astonishing that glibc doesn't provide a simple way to ignore the locale for a given call into the lib :-( | 09:04.08 |
kens | Yes, looks like its a problem a lot of people have struggled with. | 09:04.33 |
| Hmm, I cannot reproduce the cluster results on my Linux system | 09:04.49 |
| I suppose that means trying it on a node | 09:05.42 |
chrisl | Or run it under valgrind first? | 09:06.23 |
kens | Hmm, no this may be a GSproblem. | 09:06.35 |
| I was looking at the resulting PDF file with Acrobat/MuPDF and they show the correct result, GS complains about an error | 09:07.02 |
| What's the display device in Linux ? | 09:07.33 |
chrisl | x11 | 09:07.41 |
kens | aha, thanks | 09:07.45 |
chrisl | or x11alpha | 09:07.55 |
kens | don't work, I guess i DON'T HAVE X INSTALLED | 09:08.07 |
chrisl | Probably don't have the x libs. | 09:08.23 |
kens | Its OK, it gives tghe same result under Windows | 09:08.37 |
| So I can at least debug it now | 09:08.44 |
chrisl | What's the error? | 09:08.49 |
kens | Just 'Error reading a content stream' | 09:09.02 |
| ie 'something is wrong' | 09:09.08 |
| Its OK, this is new output from PXL+pdfwrite, so I'm happy to see an error | 09:09.30 |
chrisl | Ah, well, that's not so bad | 09:09.59 |
kens | and the GS output dfoes look like the cluster output, so I'msure that's the problem. I shouldn't have trusted Acrobat.... | 09:10.01 |
| Right, its a limitcheck in setdash | 09:11.01 |
| Which is the problemI was wrestling with yesterday | 09:11.15 |
| Looks like GS has an artifical limit on setdash parameters | 09:11.34 |
| I guess I need to read the POOLRM some more | 09:11.45 |
| PLRM | 09:11.49 |
| Ah, 11 elements in a dash is listed as a 'memory limit' in PostScript. In level 1.... | 09:16.21 |
chrisl | I've got to go out for a while.... | 09:19.52 |
tor8 | chrisl: the magic invocation to printf & co is %.9g to get all 32-bits losslessly formatted for round-trips. maybe that could help test the public domain version of sprintf you got with the glibc one | 09:32.35 |
Robin_Watts | tor8: Did you see my comments yesterday? | 09:41.18 |
| On robin/master there are various reviews. | 09:41.38 |
| http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=cca5a8508f628035c41c969edfffdd9c9a3b5748 | 09:41.48 |
| You may have looked at an earlier version of that one before. | 09:42.11 |
tor8 | ah yes, the don't-copy-verts patch? | 09:42.37 |
Robin_Watts | yeah. | 09:43.09 |
tor8 | you also said there was no measurable benefit, in which case I'd prefer the simpler to understand code | 09:43.27 |
Robin_Watts | So I then tried a version which further reduced the copying which had no measurable benefit... ok. | 09:43.52 |
tor8 | my hunch would be that the copying is on the stack in hot cache | 09:45.32 |
Robin_Watts | I do wonder about switching to fixed point for this stuff. | 09:45.56 |
tor8 | fixed point will be slower on non-arm today, and possibly slower on arm tomorrow as well | 09:46.47 |
Robin_Watts | All the patch decomposition could be done in fixed point, and it won't be slower on anything! | 09:46.59 |
tor8 | the triangle rasterization is all done in fixed point already? | 09:47.25 |
Robin_Watts | The drawing routines end up wanting results in fp (we convert everything by multiplying by 65536 to put stuff into the gel. | 09:47.42 |
| Looks like x and the colors all end up fixed point, but y stays fp. | 09:57.07 |
| s/fp/floating/ | 09:57.15 |
tor8 | y is integerizet | 10:03.42 |
Robin_Watts | tor8: is it? | 10:03.53 |
tor8 | y in the gel is an int (which subscanline) | 10:04.15 |
Robin_Watts | I'm looking in draw/draw_mesh.c in fz_paint_triangle. | 10:04.55 |
tor8 | sorry, which scanline, not subscanline | 10:05.11 |
Robin_Watts | oh, I see, sorry. | 10:05.14 |
| so my contention is that we could convert the values to int and/or fixedpoint earlier. | 10:05.52 |
| and do the recursive splitting on the fixed point versions. | 10:06.29 |
tor8 | we could fairly easily clip in fixed point, but then we'd need to have y as fixed point as well | 10:06.34 |
Robin_Watts | tor8: going from fixedpoint to int is a cheap operation. | 10:07.01 |
tor8 | yeah, just pointing out where it needs to be done lest we forget | 10:07.19 |
Robin_Watts | So, there are 2 commits ready to go on robin/master then. | 10:08.29 |
Robin_Watts | runs for a bit. | 10:09.37 |
tor8 | Robin_Watts: those look good | 10:09.47 |
Robin_Watts | Ta. | 10:11.50 |
| tor8: I'm going to abandon this fixed pointing for now. | 13:57.37 |
| The tensor patch stuff works OK, but the other mesh types, like the radial ones use huge values that will clip badly. | 13:59.13 |
henrys | kens:just a few things:update the makefile with new includes, the comment explaining the code < 0 return can be omitted - I really hate changing the parsing code but I guess we're stuck with that as things stand. BTW great job getting the knack of PXL code so quickly. | 14:15.33 |
kens | henrys, no problem, I will do thosae things. Can't say I understand the parser though :-) | 14:16.03 |
| I oculd have changed the underlying code instead of the parser, but it would have meant *much* more extensive changes | 14:16.42 |
| For XPS I did it wihtout touching the parser, buit it only needed changes in 2 places. | 14:16.59 |
henrys | kens:so pcl is next? | 14:23.58 |
kens | Yes, that's the plan. DId you get a chance to look at the diffs ? | 14:23.59 |
henrys | I've only looked at XL diffs. | 14:24.05 |
| kens:did you send others? | 14:24.29 |
kens | It was the XL ones I meant, I'm a bit concerned about the missing lines in C424.bin and the other test suite files like it | 14:24.57 |
| The rendering of the 'G' looks to be resolution dependent (its ok at 75 dpi) andthe 'grid effect' I'm not too concerned about, but missing content worries me | 14:25.48 |
sojic | gs -sDEVICE=pdfwrite -o page_003-t.pdf -c "[/CropBox [0.00 0.00 623.62 807.87] /PAGES pdfmark" -f page_003.pdf -c quit // causing rotate pdf. | 14:27.03 |
henrys | kens:I assume the missing lines were resolution dependencies | 14:27.15 |
kens | henrys thsoe are missing at both 75 and 600 dpi | 14:27.30 |
sojic | http://butterfly.venikom.com/pdf/sourcePdf/VOGUE_/123/distribute/3/page_003.pdf - original | 14:27.32 |
| http://butterfly.venikom.com/pdf/sourcePdf/VOGUE_/123/distribute/3/page_003-t.pdf - rotated | 14:27.36 |
kens | sojic -dAutoRotatePages=false | 14:27.45 |
sojic | Thank. I'll try right now | 14:27.56 |
kens | come back if that doesn't work | 14:27.59 |
henrys | kens:arg I guess I should simplify it for you. | 14:28.13 |
| kens:you are sure pdfwrite is being run at 600 on the cluster? | 14:28.54 |
kens | henrys, according to the logs marcos runs pdfwrite at 75 and 600 then runs GS at 75 and 600 on the resulting PDF files | 14:29.18 |
| Many other files do show resolution dependent differences | 14:29.38 |
henrys | kens:well I'll break down the file after the meetings today but I wouldn't hold up a patch for that. | 14:30.03 |
sojic | kens : gs -sDEVICE=pdfwrite -o page_003-t.pdf -c "[/CropBox [0.00 0.00 807.87 623.62] /PAGES pdfmark" -dAutoRotatePages=false -f page_003.pdf -c quit // Same problem | 14:30.04 |
kens | What particularly puzzles me about the missing lines is that the other lines on the same page don't have a problem, and I assumed they were constructed the same way | 14:30.20 |
henrys | kens:have you looked at both resolutions non pdfwrite? | 14:30.55 |
sojic | Fixed | 14:31.01 |
kens | sojic put the -dAutoRotatePages *before* the pdfmark (starting at -c) , order matters | 14:31.03 |
sojic | Yes | 14:31.06 |
Robin_Watts | ooh, factor of 4 speedup! I win! | 14:31.08 |
sojic | Figured out | 14:31.11 |
| Thanks | 14:31.13 |
kens | NP | 14:31.15 |
| henrys I did check pcl6.exe at 600, the lines are present. I thought I tried 75, I'll do it agian to be sure | 14:31.59 |
Robin_Watts | tor8: ping | 14:32.50 |
kens | henrys, yes the lines are there at 75 dpi as well | 14:33.16 |
| THe 'G' fill is not clipped at 75 dpi though, so that is obviously resolution dependent (one reason why I was less worried about it) | 14:34.00 |
henrys | hmm maybe touched vs centered rendering but I'll break it down and we'll see for sure. | 14:35.14 |
Robin_Watts | tor8: 2 reviews on robin/master for you. | 14:35.40 |
kens | OK thanks, like I said, the fact that the other lines exhibit (and the lines were present in the old pattern handling) is a slight worry | 14:35.41 |
kens | wonders if this missing line might be a ROP-related thing | 14:42.07 |
henrys | kens:now that I'm thinking about this high level patterns should break a number of things in PXL with pdfwrite - any non trivial raster op between source and pattern would have been previously handled when rendering to the pattern cache. | 14:42.22 |
| yes exactly what I was just writing... | 14:42.31 |
kens | :-) | 14:42.42 |
| Mostly the files seem to be OK | 14:42.53 |
tor8 | Robin_Watts: okay (re fixed pointing gradients) | 14:44.07 |
| Robin_Watts: 2 patches LGTM | 14:44.16 |
Robin_Watts | tor8: We *could* change the code to use floating point for x/y and ints for colors. | 14:47.27 |
| but I can't be bothered at the moment. | 14:47.47 |
henrys | kens, Robin_Watts: what is the upper limit on compares for bmpcmp, are we looking at everything? I am expecting to see a particular rop test from the customer_files directory but I don't see any files from there. | 14:49.49 |
kens | 1000 | 14:49.57 |
Robin_Watts | 1000 jobs. | 14:49.57 |
kens | do you know hte name henrys ? THre were not that many files that showed diffs | 14:50.30 |
henrys | Feature request a list of files as html links could be the first page of bmpcmp output. | 14:50.35 |
Robin_Watts | If kens is generating more diffs than that, the best approach is to retest with -filter set to get you a subset (say all the ppmraw 600dpi ones). | 14:50.44 |
| Then bmpcmp that subset which will hopefully be less than 1000. | 14:50.54 |
kens | There were not 1000 diffs | 14:50.55 |
| 197 jobs showed differences | 14:51.26 |
Robin_Watts | henrys: You want an easy way to find what page a given diff is on? | 14:51.45 |
henrys | yes but you have at least 5 diffs per job right? | 14:51.54 |
Robin_Watts | I'd rather put the list at the end of the bmpcmp output, but it's doable, certainly. | 14:52.07 |
kens | AIUI its the number of jobs, not the number of diffs | 14:52.31 |
Robin_Watts | It is the number of jobs, yes. | 14:52.46 |
kens | Sp lots of diffs per job just gets you lots of pages of output | 14:53.08 |
henrys | Robin_Watts: I meant the first thing we should see on the web is a list links that say like 150: tests_private/xl/pcl6cet/c427.bin.pdf.ppmraw.600.0 (600dpi) and we can click on that and look at that output. | 14:53.36 |
kens | NB each resolution or 'device' is a job | 14:53.37 |
Robin_Watts | henrys: Right, and I'm saying I'd rather have that list on the last page. | 14:54.29 |
henrys | kens:I was hoping to see all_rops.pxl | 14:54.30 |
kens | one moment henrys | 14:54.38 |
henrys | Robin_Watts: that's fine for me too. | 14:54.54 |
kens | THat file doesn't come up in the list of diffs | 14:55.08 |
| Either its not different, or (possibly) Marcos doesn't test it | 14:55.30 |
| I guess maybe its not tested, because it soudns like it wouldn't work with pdfwrite anyway ;-) | 14:56.51 |
henrys | kens:well it would exhibit the type of fallout I'm expecting from the pattern cache not handling the rop. | 14:57.37 |
kens | Yes, I understand | 14:57.48 |
henrys | anyway we are 3 minutes of a meeting and I'm completely unready | 14:57.57 |
kens | coffeee quick! :-) | 14:58.19 |
Robin_Watts | is boiling the kettle for tea already :) | 14:58.44 |
henrys | kens:we'll ask marcos what the coverage is I don't feel like log excavating. | 15:00.19 |
kens | OK no problem henrys, I'll sort out the other thigns now and put it to one side | 15:00.44 |
henrys | paulgardiner: I can't wait to have signature at the show - I'll demo our support contract ;-) | 15:01.15 |
paulgardiner | :-) How many do you think you can catch? | 15:02.27 |
mvrhel_laptop | sorry I am late | 15:03.22 |
Robin_Watts | henrys: which show? | 15:03.53 |
henrys | chicago | 15:04.10 |
Robin_Watts | ah, ok. | 15:04.26 |
paulgardiner | henrys: this is assuming I can figure out how to use certs from the OS within openssl, which isn't so far going well. :-( | 15:04.50 |
henrys | paulgardiner: you know the pdfcreator guy might have researched this already and we could pay him something if he had something interesting. | 15:06.09 |
| http://www.pdfforge.org | 15:07.02 |
| you can also look at the code but be careful not to infringe. | 15:08.29 |
paulgardiner | Ok, that may well help. | 15:08.46 |
henrys | Robin_Watts: good to hear from a mupdf customer. | 15:09.32 |
paulgardiner | The verify call I'm using takes as an argument a store of root certificates. Possibly I need to enumerate the OS store and place them in the openssl store. | 15:09.42 |
Robin_Watts | henrys: Christophe? | 15:09.52 |
henrys | right | 15:10.10 |
Robin_Watts | That company talks to me a fair amount, normally text messaging via skype. | 15:10.17 |
| I'm trying to steer them towards copying support using bugzilla. | 15:10.32 |
henrys | so it looks like paulgardiner will be in signature hell for a bit longer - anything to discuss schedule wise? | 15:11.32 |
Robin_Watts | I've been distracted away from image extraction. I hope to get back to that soon. | 15:12.38 |
paulgardiner | Difficult to know at the moment. There's several things that could turn out to be awkward. | 15:12.44 |
mvrhel_laptop | henrys: I am going to be slowing down a bit on the windows stuff this week due to ghostscript work, but I am going to pull out all my mupdf calls and wrap them up so that they can be called from the other languages so that is it a true winRT set up | 15:12.56 |
| the viewer has a couple glitches but I have links, contents, search all working | 15:13.25 |
Robin_Watts | The same customer is interested in text reordering etc, so that might cause us to revisit/rework some of the existing 'done' stuff on the text extraction roadmap. | 15:13.40 |
henrys | mvrhel_laptop: okay sounds good. Have you posted all the viewer code for feedback? | 15:13.50 |
| mvrhel_laptop: I wish we could keep you going on this but the gs project is uniquely yours. | 15:14.32 |
Robin_Watts | mvrhel_laptop: Did you look at reflow? ISTR you saying that you were going to look at that. | 15:14.36 |
mvrhel_laptop | Robin_Watts: yes. just typing about reflow | 15:14.48 |
Robin_Watts | marcosw: Has the cluster jammed? | 15:14.58 |
marcosw | Robin_Watts: let me check | 15:15.12 |
mvrhel_laptop | I have it working, but the html webview object blows in windows 8 ui and intercepts all the manipulations making zooming difficult | 15:15.28 |
| right now, I have it display at a rich text output and you can scale that. I need to look a bit more at what my options are | 15:16.10 |
Robin_Watts | mvrhel_laptop: We could use onscreen icons for zoom +/-, but it's a shame that the UI is different. | 15:16.35 |
mvrhel_laptop | Robin_Watts: yes | 15:16.51 |
henrys | tor8:ping | 15:16.57 |
mvrhel_laptop | anyway, my big thing will be to pull out the mupdf stuff and add in a couple simple sample apps in other languages | 15:17.30 |
| that is pull it out into a dll that can be shared amongst the languages | 15:17.52 |
Robin_Watts | mvrhel_laptop: Presumably, someone has already done that for the thing that David used to write the article ? | 15:17.58 |
tor8 | henrys: yes? | 15:18.27 |
henrys | how is localization? | 15:18.53 |
tor8 | mvrhel_laptop: did you see the link that robin pasted the other day, the blog post on msdn? | 15:18.55 |
| henrys: I've got the string resources in the android app all cleaned up. just need to set up the oneskyapp account and figure out how that works. | 15:19.21 |
| do we have a specific list of languages we want to support? | 15:19.43 |
Robin_Watts | All of them! | 15:19.58 |
tor8 | that'll put a dent in my wallet... | 15:20.08 |
Robin_Watts | http://email.oneskyapp.com/wf/click?upn=R-2B1hm6oTz-2FyAhd1Dtw8ZZkhELWqcMtWHUIwXk-2Bm2L4ITZsoNHiAV6LyrTLGQs4FU8G-2BfycZsWcTfHxktuAkU7cTt7Q2NsEAxiIbovh2ejUQxaB9MJKRlzQ9UJMqPXHP9vBKZISNcLOUR63hoXutGr8omhEEfhRJOW66xukANGLLeIheU3JHB9nXFyOM0W9ezjKwefYDLmnkSvY6eD3igPTJzyX8T1QLmit5rzrFpUpVMSFEYaP2FQgRFcWsJd879YEuszRmkh8KZkI9PSmDUlg-3D-3D_6xWBv-2B8To-2Bx7VnI0-2BJY-2FHF6hzuUwn6Y34xjriiq51ybcJeyOew-2B8Qe5qY | 15:20.10 |
| 6XCkU8tOy5o7ch6MIUh8643UCPeoVjpnqeju1VD82htsxBVzByWXPcg-2F4SKUYmQi6f4XfjhMeko40HHypxKnlFb0ga5cuAoP0cx-2FGsceSynE-2FnwHfzFnYGq0AW93DNgzfloxbvzub-2FBn4rrtNdMginEekPcbQ-3D-3D | 15:20.12 |
henrys | we might as well support all | 15:20.18 |
Robin_Watts | It's like $400, right ? I have a US $ card if that helps. | 15:20.45 |
henrys | Robin_Watts: overflows my irc clients line buffer | 15:20.45 |
mvrhel_laptop | tor8: yes. I was looking at the code/project. they guy did a nice job on it | 15:21.07 |
tor8 | the dollar is pretty worthless compared to the swedish currency, and I don't have a visa attached to my USD account | 15:21.14 |
Robin_Watts | http://tinyurl.com/cjlzfxq | 15:21.25 |
marcosw | Robin_Watts: I'm not sure what the cluster is doing. I could just restart it but would rather understand what's going on. | 15:21.43 |
Robin_Watts | mvrhel_laptop: Is it worth us just paying him $$$ and using that? | 15:21.49 |
| Cost of mvrhel vs cost of paying out etc. | 15:22.03 |
mvrhel_laptop | Robin_Watts: well, we could do that. | 15:22.19 |
| what do you think henrys? | 15:22.48 |
Robin_Watts | mvrhel_laptop: I can imagine there might be arguments both ways (he might not have done it in quite a general enough way etc). | 15:23.02 |
mvrhel_laptop | Robin_Watts: there will still be work to do. He does not have a viewer here, just wrapped up the calls needed into mupdf so render pages | 15:23.50 |
| s/so/to/ | 15:23.56 |
henrys | mvrhel_laptop: I am fine with it but you should look at it carefully, if you know over time your are going to do it differently it doesn't really make sense. CLA of course. | 15:24.08 |
mvrhel_laptop | henrys: yes, I think it will not be too hard for me to take the few calls that I have into mupdf, wrap them up into a new class that can then be shared amongst the different languages | 15:24.47 |
Robin_Watts | mvrhel_laptop: Right. Were you planning to just expose the calls you need for your viewer? Or all the calls? | 15:24.55 |
mvrhel_laptop | to me this makes more sense, since these are the things that I know we need for a viewer | 15:25.06 |
| just the viewer ones for a demonstration | 15:25.19 |
Robin_Watts | mvrhel_laptop: Do you handle forms etc? | 15:25.29 |
mvrhel_laptop | Robin_Watts: I have no special stuff for form handling | 15:25.54 |
Robin_Watts | ok. | 15:26.08 |
henrys | mvrhel_laptop: also once you have something workable in git you can point him to it and ask if he's interested then post bountiable enhancements, right? | 15:26.11 |
mvrhel_laptop | I have reflow, text search, links, content display | 15:26.14 |
| henrys: true | 15:26.30 |
| let me at least make an effort at it as I will feel like I wasted a bunch of time otherwise | 15:26.54 |
henrys | mvrhel_laptop: seems like you'd have better control over direction that way. | 15:26.57 |
| anything else we should discuss here? 3 minutes to go. | 15:27.56 |
| tier 3 is 208.58 right? | 15:28.29 |
Robin_Watts | it is. | 15:29.09 |
| It's $467.72 for all 3 tiers for the app description. | 15:29.29 |
| Plus some other as yet unknown sum for the resource strings. | 15:29.47 |
henrys | oh I see I though lower tiers were subsets. | 15:30.07 |
| s/though/thought | 15:30.14 |
| peculiar how they are split up whey would swedish be a different tier than norway? | 15:31.04 |
Robin_Watts | henrys: number of people that speak that language I guess. | 15:31.29 |
henrys | then chinese would be tier 1 | 15:31.50 |
tor8 | henrys: twice as many swedes as norwegians | 15:32.02 |
Robin_Watts | ...and have smartphones. | 15:32.02 |
| 4.7M swedes with smartphones. 2M Norweigies. | 15:32.26 |
| Hindi has 38Million, but then they all speak english too anyway. | 15:33.13 |
tor8 | odd that they have spanish (spain) and latin american spanish as different... I'd have guessed the difference there was like between US and UK english | 15:33.48 |
Robin_Watts | tor8: It's different when spoken (very different accents) | 15:34.17 |
| but yes, it's all a bit arbitrary I suspect. | 15:34.37 |
henrys | anyway let's just finish that - it really could matter sales wise and it takes little effort. | 15:34.45 |
Robin_Watts | We can cherry pick individual languages. | 15:34.51 |
tor8 | I'll set up the account and prep the files then maybe robin can whip out his USD visa? | 15:35.08 |
Robin_Watts | tor8: Sure. | 15:35.15 |
tor8 | and check the price diffs between the tiers to see if it's actually worth spending or not for the flavours of same langugae | 15:35.43 |
| Robin_Watts: tor/android has one commit that I see hasn't been merged yet | 15:36.26 |
Robin_Watts | latin american spanish = $16 | 15:36.36 |
marcosw | Robin_Watts: figured out in what state the cluster is stuck and will add code to automatically recover from this condition the next time it happens. the cluster would have recovered 2 hours after your job started, but the new check will cut that to five minutes. however, I'm still not sure how it got into this state. 3 of the nodes never received the command to start the run but they cluster master is convinced it sent the comma | 15:36.45 |
| I could believe if one node had this happen due to some internet glitch, but it's odd that 3 nodes are in this condition. | 15:36.46 |
Robin_Watts | tor8: Looking now. | 15:37.25 |
| Looks fine. Will push. 1 on robin/master for you. | 15:39.19 |
henrys | wow 5k running race today in a foot of snow, races don't get canceled here for anything. | 15:39.43 |
Robin_Watts | henrys: Pays to not be leading that one I guess. | 15:40.24 |
| marcosw: odd. | 15:40.57 |
mvrhel_laptop | henrys: so I am going to be out for a good part of the day. I have to chaperone a field trip at my daughters class | 15:41.02 |
henrys | mvrhel_laptop: okay | 15:41.23 |
| where you off to? | 15:41.33 |
Robin_Watts | The cardboard box factory again? | 15:41.45 |
tor8 | Robin_Watts: yours also looks good | 15:41.58 |
Robin_Watts | tor8: Thanks. It caused diffs in the last test. trying to figure out why :( | 15:42.24 |
henrys | I remember chaperoning to the butterfly pavilion and I kept stepping on the damn things and all the kids would yell at me. | 15:42.48 |
Robin_Watts | henrys: And 1) you were never asked back, and 2) it's all the kids remember about that day, right? | 15:43.19 |
paulgardiner | so a thoroughly good result really. | 15:44.09 |
henrys | yes I volunteered for all sorts of stuff and in hindsight ⦠well I don't know what I'd do. | 15:44.24 |
tor8 | henrys, Robin_Watts selecting absolutely everything on the UI strings resource for android runs up at $552 | 15:44.41 |
| and that's just 52 strings | 15:44.48 |
Robin_Watts | tor8: That includes tiers 1-3 ? or is that in addition? | 15:45.05 |
tor8 | Robin_Watts: there's a different ui for picking once logged in and doing a proper "project" in oneskyapp | 15:45.54 |
mvrhel_laptop | henrys: hehe. we are off to a boat | 15:46.23 |
tor8 | picking the "the european biggies" and "the roaring east asians" tiers is $132 and gets us basically tier 1 and 2 | 15:46.24 |
mvrhel_laptop | http://www.argosycruises.com/ | 15:46.24 |
| I pick only the fun field trips to help out on | 15:46.48 |
marcosw | bbitft9m (be back in time for the 9:00 meeting) | 15:47.22 |
Robin_Watts | mvrhel_laptop: kids on a boat. nausea, vomiting and drowning. Is it too late to back out? | 15:47.29 |
mvrhel_laptop | ha. I had not thought of that | 15:47.39 |
henrys | mvrhel_laptop: so the mission is to keep them out of the water. | 15:48.18 |
ray_laptop | and stay clear when they throw up their fruit loops and milk | 15:48.56 |
Robin_Watts | mvrhel_laptop: So, on the bus on the way there, pick the roudiest ones, and appoint them "Vomit Monitors" as a punishment. | 15:49.38 |
mvrhel_laptop | good idea | 15:50.00 |
ray_laptop | my wife said there was just an earthquake, but I didn't feel anything | 15:52.41 |
| I'm only .25 mi away. Must have been pretty small | 15:53.10 |
| nothing shows on the usgs map. Probably some construction local to our house | 15:57.43 |
henrys | mvrhel_laptop: last meeting I was going to ask about the NRE project but I think that's on the way and 689057 which was felt worthy of fixing at the meeting. | 15:58.08 |
Robin_Watts | ray_laptop: will get home to find a tree embedded in his roof. | 15:58.10 |
chrisl | Robin_Watts: or a roof embedded in his tree........ | 15:58.49 |
Robin_Watts | There's no place like home. There's no place like home... | 15:59.10 |
mvrhel_laptop | henrys: yes. I made some good progress on this project yesterday | 15:59.31 |
chrisl | thinks Robin_Watts would not suit ruby slippers | 15:59.48 |
mvrhel_laptop | hoping I can get it clean up middle of the week and then pass off to ray_laptop for him to finish up | 15:59.54 |
henrys | marcosw:kens and I wanted to verify you run all the tests with pdfwrite - he made a change that should have triggered failures but didn't. | 16:00.33 |
ray_laptop | mvrhel_laptop: sounds fine. I should have the #10 issue soon as well. | 16:01.03 |
henrys | actually we don't know for sure it should have | 16:01.04 |
marcosw | henrys: yes, all the usual test files are run with both pdfwrite and ps2write. | 16:01.18 |
| I can check the logs, was it a clusterpush or commit regression run? | 16:01.48 |
henrys | marcosw:a local push by kens associated with ~regression/ken bmpcmp's | 16:02.42 |
kens | My guess is it was wrong before and worngnow. | 16:03.16 |
| I will get the file and look | 16:03.22 |
henrys | I find it odd thought that not a single file from the customer_tests directory shows up in your bmpcmp | 16:04.57 |
| kens:given the errors I do see. | 16:05.22 |
| alexcher:anything for the meeting? | 16:05.49 |
kens | Umm, possibly, I have the file now, give me a moment | 16:05.52 |
alexcher | marcosw: did you do any new testing for the fuzzing bugs? | 16:05.53 |
marcosw | henrys: ? The first file in <http://www.ghostscript.com/~regression/ken/> is "tests_private/customer_tests/A1TEST_CHART.hp2.pdf.ppmraw.600.0 " | 16:06.21 |
| alexcher: no, I just moved the issues from the two bugs to the new, individual bugs. | 16:06.49 |
henrys | marcosw:ah okay I am not sure how I did that started on page 1. | 16:07.15 |
alexcher | marcosw: OK, I'll close the bugs that have been fixed already. | 16:08.29 |
kens | henrys looks to me like none of the reops are applied, sothe result is always incorrect | 16:08.37 |
| sonow its the same incorrect | 16:08.44 |
| Need to do a rebuild to check though | 16:08.58 |
marcosw | alexcher: thanks. I should have done that but haven't had time. | 16:09.07 |
henrys | kens:okay so I should apply your patch and find the missing lines part in isolation correct? | 16:09.45 |
kens | henrys I can apply the patch iof you like, the missing lines thing could just be rops, but I would like to see for sure so it would be nice to check it out before comitting | 16:10.24 |
henrys | chrisl, ray_laptop : anything for the meeting? | 16:10.27 |
marcosw | chrisl: The customer who reported a problem with ufst some weeks ago but didn't give a version has come back with UFST 6.2 and GhostPCL 9.05. However I can't get that combination to compile. Okay if I assign the bug to you to have a look at? | 16:10.32 |
henrys | kens:that's fine I'll do it just after the meeting. | 16:10.55 |
Robin_Watts | marcosw: 6.2? I thought they said 6.3 ? | 16:11.07 |
kens | henrys did a rebuild, checked old PDF vs new PDF< thery are identical. Wrong, but identically wrong | 16:11.10 |
henrys | kens:apply the patch from your email that is. | 16:11.12 |
chrisl | marcosw: as far as I'm aware, we only support PCL withe UFST that we supply, and we've only ever supplied 5.x | 16:11.16 |
marcosw | kens: right, 6.3 | 16:11.25 |
kens | me ? 6.3 ? confused... | 16:11.49 |
Robin_Watts | kens: marcosw meant me :) | 16:11.59 |
henrys | hopefully ray is still on terra firma heard they had a 7.8 in iran. | 16:12.14 |
marcosw | kens: I meant Robin_Watts, all you brits look alike | 16:12.16 |
kens | :-) | 16:12.21 |
mvrhel_laptop | hehe | 16:12.27 |
chrisl | henrys: I did want to mention that I'll be committing the tiff configure changes soon, so everyone that uses the configure build will need to "make maintainer-clean" and autogen.sh | 16:12.37 |
Robin_Watts | It's our matching bowler hats, umbrellas and moustaches. | 16:12.43 |
ray_laptop | henrys: nothing for the meeting from me (I already mentioned cust #801's enhancement is in progress) | 16:12.43 |
marcosw | pasty with bad teeth :-) | 16:12.45 |
kens | henrys I didn't send you quite all the patches as diffs, there's a change also needed in pdfwrite, I was happy to review that one myself though :-) | 16:12.54 |
| Also I need to do the makefile changes and remove the comment | 16:13.22 |
| So I'm happy to commit it tomorrow instead. | 16:13.37 |
| Unless you would prefer to do so. | 16:13.44 |
henrys | kens:at this rate I'd say just commit it and we'll have an open bug. | 16:13.46 |
marcosw | could be worse, at least you aren't fat germans in lederhosen. | 16:13.50 |
Robin_Watts | was in the middle of preparing a german bashing joke :) | 16:14.13 |
kens | henrys I do need to do those changes anyway, so I'm happy to wait and look at a reduced problem tomorrow | 16:14.18 |
marcosw | henrys: what is our policy regarding ufst 6.3? do I tell the customer it's not supported by us? We have discussed supporting ufst 6.x | 16:15.16 |
henrys | kens:I am concerned I'll be able to find the problem without the completed patch set. Do I have everything I need to get the missing lines? | 16:15.19 |
kens | henrys I htink so yes, the change in pdfwrite prevents a seg fault in C318.bin, so don't run that one | 16:15.51 |
henrys | kens:okay | 16:16.00 |
kens | I believe all the other files run to completion with what you have | 16:16.13 |
Robin_Watts | henrys: kens changes are shown at the end of the cluster test report, aren't they? | 16:16.31 |
kens | yes | 16:16.37 |
chrisl | marcosw: as we don't have UFST 6.3, and UFST needs our custom patches to work, I would double check they don't actually mean 5.3..... | 16:16.46 |
kens | but because I wasn't using a totaslly up to date source, there are some other there too Robin_Watts | 16:16.59 |
marcosw | chrisl: will do. | 16:17.08 |
kens | ones that chris had pushed | 16:17.09 |
henrys | marcosw:what chrisl said, oh this is the folks mixing and matching fcos? | 16:17.26 |
kens | henrys looking at the all_rops file, the pdfwrite output is just totally wrong, but at least its consistent | 16:17.37 |
marcosw | henrys: yes. | 16:17.53 |
chrisl | Why would they not use the FCOs supplied? | 16:18.16 |
henrys | marcosw:so I think the first answer should be we don't know anything about compatibility beyond the stock fco's that come with a particular version and the first question should be how did they think to do something so daft? | 16:20.16 |
chrisl | It makes me very wary since the FCOs are shipped with the UFST releases | 16:21.04 |
henrys | anything else meeting wise I can't think of anything but I didn't try very hard this week ;-) | 16:24.33 |
alexcher | marcosw: Sometimes debug build of gs generates warnings about GC and memory problems. How difficult is to screen the whole test suite for such warnings? | 16:25.38 |
marcosw | alexcher: I run the debug build as part of the weekly regression tests (on Tuesday, so it's running as we speak). I should be able to just write a simple script to check the log files. | 16:26.52 |
| should I be using a particular -Z option to see the issues? | 16:27.18 |
| we run with -Z@ currently. | 16:28.03 |
Robin_Watts | I had a random thought earlier today, when I wanted to find which test files used a type 6 shading - I put exit(1) in the code to make it die and looked for the ones that died in my clusterpush. | 16:28.12 |
alexcher | marcosw: I have gases when all Valgrind warnings are fixed but gs warnings persist. I didn't use any -Z options. | 16:28.26 |
chrisl | alexcher: I think a bunch of those that relate to strings are when we try to gc scan string constants - so we get a "not in chunk" warning. | 16:28.43 |
Robin_Watts | I was wondering about maybe having the cluster automatically grep the logs etc for lines starting with a given string, and collating them into a log file reachable from the dashboard. | 16:29.04 |
chrisl | Robin_Watts: that would be useful - I ended up adding code that would segfault for that purpose | 16:29.36 |
Robin_Watts | So people could printf("CLUSTER: whatever\n"); and cluster test and automatically get a list of what files emitted that. | 16:29.49 |
alexcher | chrisl: such strings should be in the foreigh memory, souldn't they? | 16:30.03 |
chrisl | alexcher: the problem is we don't know that because "string allocations" (*not* string refs!) don't have memory management headers - we reach them through enum_ptrs calls. | 16:31.21 |
henrys | I've always wanted to try cross referencing code coverage lines with test files. you'd have some sort of html output of the code click on a line and see which files exercise that line. | 16:32.21 |
chrisl | alexcher: I'm not suggesting that's all the problems, though, we probably do need to investigate the warnings | 16:32.32 |
alexcher | chrisl marcosw: but little is done without a tracking number. | 16:34.05 |
chrisl | alexcher: yes, I am in agreement with you - a list of files, and a bug would be a good thing to have | 16:34.58 |
| alexcher: I've only ever looked into any of them in-passing when working on another problem | 16:35.34 |
marcosw | Robin_Watts: I can do that pretty easily, can you add some output and do a clusterpush so that I can test the code when I add it later today? | 16:36.05 |
Robin_Watts | marcosw: I will. | 16:36.22 |
marcosw | henrys: I'm surprised such a thing doesn't already exists. I should be able to hack something together... | 16:38.35 |
henrys | marcosw:I don't think it can be just brute force - you are going to need some way to keep the number of files under control. | 16:39.19 |
chrisl | marcosw: is the clusterpush.pl in ghostpdl/gs/toolbin/localcluster the "golden" version, or does it reside elsewhere, too? | 16:40.37 |
marcosw | chrisl: that's the "golden" version. | 16:41.07 |
chrisl | Cool, thanks | 16:41.16 |
Robin_Watts | henrys: The number of lines more than the number of files, I suspect. | 16:41.17 |
henrys | well if you click on one line of the code is that going to show a list of 10,000 files, then the next line the same 10,000 files on another page? | 16:42.19 |
| presumably the same files. | 16:42.41 |
Robin_Watts | henrys: Right. You'd want to identify lines in the source that were called 'together'. | 16:43.43 |
marcosw | henrys: I'm guessing lines that are covered by 10,000 files aren't interesting; we could just report the first 10 matches and a ... | 16:43.44 |
henrys | marcosw:well I also wanted to say give me all the test that use this line. | 16:44.45 |
marcosw | Robin_Watts: yeah, I was going to do that. Generate a md5sum of the files that affect each line and treat groups of lines that have the same md5sum as a set. | 16:44.47 |
henrys | Robin_Watts:exactly. | 16:45.09 |
Robin_Watts | marcosw: What tool would you use to generate the coverage ? | 16:45.13 |
marcosw | henrys: even if there are 10,000 of them? | 16:45.19 |
| Robin_Watts: gcov. Or is there something more better? | 16:45.40 |
Robin_Watts | marcosw: No, I just wasn't aware of gcov :) | 16:45.54 |
| You've just halved the problem size :) | 16:46.12 |
marcosw | I think it's gcov (it's some gcc related coverage tool). | 16:46.28 |
henrys | marcosw:If you go to the trouble of implementing "together" I don't think it would be a problem to report all files. | 16:47.48 |
marcosw | the biggest problem I foresee in this coverage tools is that every time a source file is edited the results are invalidated. And as I recall coverage testing isn't fast, so regenerating the results every commit isn't possible. | 16:48.01 |
mvrhel_laptop | bbiaw . | 16:49.02 |
henrys | marcosw:I think that is okay for the use cases we are imagining. once a week it runs and the database is available online somewhere. | 16:50.07 |
| we discussed gcov cluster stuff before that would certainly help, merging gcov is suppose to be easy ... | 16:50.56 |
marcosw | henrys: I will probably have to involve the cluster machines, as I recall coverage analysis on a single machine takes many days. | 16:51.48 |
kens | OK have to go make dinner goodnight all | 16:51.56 |
marcosw | Robin_Watts: is there a way to have the memento memory squeeze feature to stop at some point? I.e. specify a starting and an ending allocation count? | 16:55.26 |
Robin_Watts | marcosw: Not currently. | 16:55.41 |
henrys | marcosw:you also have to identify all the parameters with the test file which makes it quite nasty. pbmraw uses different lines than ppmraw etc. | 16:55.47 |
Robin_Watts | henrys: You just collect coverage data based on the filename_options not just the filename. | 16:57.06 |
marcosw | henrys: what Robin_Watts said | 16:57.21 |
Robin_Watts | i.e. tests_private__pdf__foo__bar__baz__200_0_ppmraw or whatever it is. | 16:57.39 |
| marcosw: Is there a particular reason you want that ? | 16:58.01 |
| The initial allocations are probably the same for many files (the postscript setup is identical for any given device call). | 16:59.21 |
| The latter ones are where it's interesting. | 16:59.28 |
henrys | yes I understand that, I was just thinking of the ballooning number of files. | 16:59.37 |
marcosw | Robin_Watts: makes it easier to run memory squeezing on idle cluster nodes. Right now even a simple job such as tiger.eps runs for many hours and other than killing the job there isn't any way to stop it. | 17:00.23 |
Robin_Watts | marcosw: Can you not suspend processes under unix ? | 17:01.25 |
| kill -STOP pid then kill -CONT pid ? | 17:02.06 |
Robin_Watts | wonders if the pid is a moving target with all the forking...) | 17:02.40 |
| or... nice the process ? | 17:03.04 |
henrys | marcosw:something else I forgot from the meeting: you were talking about more machines for miles' office - should we be getting heavier duty hardware and having those nodes do more. It sounded like we were getting into diminishing returns problems from communication. | 17:03.05 |
| marcosw:you've been purchasing fairly low end stuff node wise. | 17:03.57 |
marcosw | Robin_Watts: nice doesn't really work, I run some jobs at nice level 19 and they still slow down the normal cluster jobs (which run at nice level 10). | 17:04.04 |
Robin_Watts | marcosw: OK. so -stop, -cont ? | 17:04.21 |
| I can change memento so that the 'main' app doesn't run up the pid table if that's wrong currently. | 17:04.49 |
| (i.e. have the one that forks be the one that dies) | 17:05.04 |
| right. it's always the child that dies, so if you -stop the parent (the one that you initially started), that will stop the whole process (within as long as it takes the child to die). | 17:06.35 |
marcosw | Robin_Watts: I understand how to do this; I wasn't looking for advice or suggestions. I just asked if the ability already existing in memsqueezing. I can work around the lack of such a feature,. | 17:10.25 |
Robin_Watts | marcosw: right, sorry. There isn't code in memento to do that, though I could add it if required. I can't see why we'd be better off with code in Memento than using -stop/-cont though. | 17:11.27 |
marcosw | Robin_Watts: because stopping/continuing jobs isn't currently an ability of the cluster code and before I added it I wanted to check if it was needed. It's not better or worse than adding a feature to memento. | 17:17.31 |
Robin_Watts | restarting memento from a given point takes time though (we rerun through the first n allocations). | 17:18.06 |
| and presumably if you wanted to start/stop memento using 'MEMENTO_SQUEEZEAT'/'MEMENTO_SQUEEZETO' you'd have to decide those limits before starting. | 17:18.54 |
marcosw | henrys: I've been buying 6 and 8 core AMD machines, I'm not sure I'd call these low end. They are cheap, but that's probably because they have really low end graphics and not much expansion capabilities (i.e. few PCI slows and usually only two memory slots). | 17:18.58 |
| henrys: I did look at higher end machines (i.e. multiple 6 core xeons, but they cost thousand and I'm not sure how much faster they would be). | 17:19.52 |
henrys | marcosw:I'm not sure either I just looked here http://www.cpubenchmark.net/high_end_cpus.html and noticed my henrysx6 chip is pretty far down the list. | 17:22.30 |
| and the other cluster machines are of similar performance | 17:25.45 |
marcosw | one of the latest nodes uses a AMD FX-8150 Eight-Core, which gets a score of 7,747, which isn't as good as the Intel Xeon E5-4650 @ 2.70GHz which get a score of ~14,000 but otoh, that chip costs $3,899.99 (odd how cpu manufacturers use $0.99 pricing, like it's an impulse purchase in the supermarket check out lane). | 17:26.11 |
| what cpu does your henrysx6 node use? | 17:26.38 |
henrys | phenom 1055T | 17:27.27 |
marcosw | we could look into upgrading the cpu. It's a AM3 socket cpu and there aren't many faster AM3 chips, but if you mother board supports AM3+ chips there are a lot of options. | 17:32.27 |
Robin_Watts | aws nodes that are down are now hidden on the dashboard. | 18:08.14 |
| henrys: And bmpcmp's have an index on the last page. | 18:08.50 |
henrys | thanks Robin_Watts | 18:09.17 |
Robin_Watts | np. | 18:09.33 |
| marcosw: I'm fiddling with the log searching stuff now, to avoid us both doing it. | 18:21.02 |
| marcosw: Do the .log.gz files get both stdout and stderr? | 18:56.49 |
| oh, he's not here. | 18:56.59 |
henrys | paulgardiner: still about? | 19:13.04 |
Robin_Watts | I think he'll be gone. He has a regular tuesday night activity. | 19:13.52 |
henrys | Robin_Watts:okay no problem, if you see him before I do tell him to buy a ticket to miami - he'll be coming regularly now so I assume you or the other UK'ers will try and hook up with him for travel. | 19:16.39 |
Robin_Watts | will do. Cool. | 19:16.58 |
| henrys: Yeah, we can share Arnie to and from the airport :) | 19:17.19 |
| hi marcosw | 19:17.48 |
| Just to say, I'm playing with the log searching stuff now. | 19:18.00 |
marcosw | Robin_Watts: uh oh. What's fell over now? | 19:18.00 |
Robin_Watts | Do .log.gz files contain both stderr and stdout ? | 19:18.13 |
marcosw | I think so, let me check. | 19:18.39 |
| it turns no, just stdout, but trivial to chagne. | 19:19.51 |
Robin_Watts | ah, ok. Where does stderr go? hyperspace? | 19:20.08 |
| or is it in another file? | 19:20.14 |
marcosw | hold on, it's more complicated... | 19:20.38 |
| never mind, it's not complicated. The .log.gz files contain both stdout and stderr. I thought perhaps they only included stderr and stdout for the pdfwrite/ps2write portion of those jobs, but it should be everything. | 19:22.46 |
Robin_Watts | OK, I have the searching stuff almost working. Just need to sort out a webby front end for it. will do that tomorrow. | 19:28.29 |
| OK, new search links are active on the dashboard. Let me know what people think. | 23:48.08 |
| Forward 1 day (to 2013/04/17)>>> | |