IRC Logs

Log of #ghostscript at irc.freenode.net.

Search:
 <<<Back 1 day (to 2012/08/20)2012/08/21 
tor8 henrys: as far as I know, he's well aware that his contributions would be part of the commercial license under artifex copyright00:17.36 
  bed time for me too. night all.00:17.43 
sebras tor8 henrys Robin_Watts: I believe I'm on good terms with him, and he's been encouraging me to look through their bugzilla and their updated diff.07:45.20 
  I wasn't aware that there were underlying discussions about compensation and copyright assignments that were at issue.07:46.17 
  so I guess it's back to harvesting their bugzilla for bugs and ignoring their fixes then. alright I can work with that.07:47.09 
  let me know if I can assist in anyway. also I'll let you guys disuss this with him as it is a matter between artifex and him, not really involving me.07:48.46 
  oh, and by harvesting I thinking about finding .pdfs to use as testcases.07:51.59 
  tor8: ok, so no harvesting either. noted.09:09.37 
tor8 sebras: at least until this all blows over. I'll let you know when.09:21.41 
sebras tor8: sure. how annoying. :-P09:24.57 
tor8 paulgardiner: Robin_Watts: git clone --recursive clones submodules too11:27.44 
paulgardiner Ah, nice.11:28.03 
tor8 I tried out a git-subtree based approach. even with --squash it doubles the mupdf git size11:29.57 
paulgardiner tor8: Yeah, I think subtree is brilliant for libraries that you are developing as part of your main project, but submodules sounds right for true thirdparty stuff.11:31.10 
tor8 yeah, that what I'm thinking as well11:31.40 
  the thirdparty libs for mupdf really are external projects, that we rarely if ever develop in sync so submodules are a good fit11:32.15 
Robin_Watts tor8: git clone --recursive - nice.11:32.51 
  Now, if only there was a way to avoid the git submodule update... like a hook script or something.11:33.08 
paulgardiner I've never used either, so it's the discussions have been really handy for me: feel like I have a pretty good understanding now.11:33.17 
  Do any other commands support --recursive?11:36.32 
tor8 Robin_Watts: .git/hooks/post-checkout may be the place11:39.45 
Robin_Watts http://blog.chaitanyagupta.com/2009/08/couple-of-hooks-to-make-life-easy-with.html11:40.24 
tor8 Robin_Watts: sticking a "git submodule update --init" in .git/hooks/post-merge and post-checkout seems to fix most cases (both switching branches and doing merge and rebase)11:46.03 
  the post-checkout hook is run on git-bisect11:47.43 
Robin_Watts aargh. garbage collection.11:52.10 
kens is in that particular heell right now11:52.25 
Robin_Watts I'm getting a segv due to an illegal halftone, but can I see where that halftone is created? Can I hell.11:53.14 
  It won't go wrong on windows, or with memento.11:53.23 
kens Naturally11:53.31 
Robin_Watts Do we have a debug switch that lists the relocations that garbage collection does?11:54.39 
kens has no idea11:55.05 
  I suspect one of the -Z flags will do that, and bury it in so much other studff, you won't be able to find it11:55.41 
tor8 hmm, I didn't know about git sparse-checkout 12:25.28 
alexcher Where's marcosw ? A customer wants to submit a large file.13:17.01 
kens Yeah I leave those for marcosw to answer13:30.21 
Robin_Watts OK. I need a sanity check.13:42.32 
  Where in the code can blah->dev_ht ever get set ?13:42.46 
kens Presumably atfer a sethalftone13:43.03 
Robin_Watts file and line, please.13:43.15 
kens Have you tried setting a halftone and following the code ?13:43.15 
  No idea, sorry.13:43.23 
  I would break on zsethalftone and follow the code13:43.35 
Robin_Watts gsht.c line 1112 ish does some rc_unshare_struct voodoo that can result it in being set.13:44.39 
kens OOh... strange13:44.55 
  it appears to use rc_alloc_struct_array to make the halftone13:46.04 
Robin_Watts But that is NOT the location that sets dev_ht to this particular value.13:46.41 
  so I'm trying to figure out where else it's set.13:46.54 
kens I'm not at al lsure about dev_ht13:46.56 
Robin_Watts I'm grepping the code, and failing to find anything. Maybe I'm just being blind.13:47.20 
kens It might be more complicated than that13:47.36 
  LIke I said, zsethalftone5 puts the whole lot into a struct_array...13:48.02 
Robin_Watts I have it SEGVing in gx_ht_write with pdht=0x1bb163813:50.46 
  and if I trace the garbage collection, I can see that address being marked.13:51.16 
  but I cannot see where that address ever comes from in the first place.13:51.35 
kens gx_imager_dev_ht_install a possibility ?13:51.48 
Robin_Watts I've got printfs in various places that I hoped would capture all the creations, but... no luck.13:51.56 
kens gx_imager_dev_ht_install seems to copy halftone stuff around13:52.22 
  Its quite complicated, I can't grok it on a quick scan13:53.05 
Robin_Watts That creates a dht on the stack.13:53.17 
  Then the unshare voodoo is used later on to turn it into a stack based thing.13:53.59 
kens OK is that similar to gx_ht_install ?13:54.19 
Robin_Watts gx_ht_install calls gx_imager_dev_ht_install13:54.52 
kens OK I hadn't got that far yet13:55.04 
  TO be honest, I've never looked at any of this stuff13:55.33 
  So you undoubtedly know more than I do13:55.50 
Robin_Watts kens: I know nothing about this code, so any insight gained from a second set of (fresher) eyes has got to be good.13:56.24 
  I literally cannot see anywhere in the code that assigns to blah->dev_ht except this unshare thing.13:57.05 
kens Hang on a moment, VS having a knicker fit13:57.34 
  My own GC problem :-(13:57.54 
mvrhel morning Robin_Watts 14:02.55 
Robin_Watts mvrhel: Morning.14:03.02 
kens Someone else to look at dev_ht :-)14:03.13 
Robin_Watts I'm sorry, I haven't managed to look at either of the bugs you mentioned in the email.14:03.24 
mvrhel So I made some progress on the crash I mentioned to you via email14:03.26 
  there is an issue with the copy_planes in the clist14:03.37 
  again...14:03.41 
Robin_Watts I had 2 regressions thrown back at me last week,and I've been trying to sort them.14:03.50 
mvrhel the big pain is that I can't get the error in windows only linux14:04.06 
Robin_Watts I think I've fixed one, but I'm in linux-only non-memento gc hell with the other.14:04.16 
mvrhel so debugging is very slow as I try to see where the clist goes off the rail14:04.21 
Robin_Watts Sounds like you're in a similar place to me.14:04.30 
mvrhel oh you are having fun then too14:04.31 
  hehe14:04.33 
Robin_Watts 'fun'.14:04.50 
mvrhel yes14:05.00 
kens Robin_Watts : gs_image_state_pre_assign copies dev_ht from one imager state to another14:05.02 
  well its called before copying14:05.18 
  can't see anywehre its used though14:06.49 
  In gx_dc_ht_colored_read:14:08.19 
  devc.colors.colored.c_ht = pis->dev_ht;14:08.19 
  Which probably isn't relevant either14:08.39 
Robin_Watts Ok. I can see that the pre_assign is called with the relevant value of dev_ht14:09.02 
kens Oh really ? I couldn't find anmywhere that used that...14:09.15 
Robin_Watts so let me see if I can trace through there to see where it gets corrupted.14:09.46 
kens OK let me know if not and I'll keep looking14:09.57 
Robin_Watts kens: Thanks for that. You've got me moving again.14:10.25 
kens No problem, back to figuring out why may adresses are wrong14:10.43 
  aha! I think I found my problem. Now for a cluster push14:18.27 
  alexcher : the customer has sent a link to that file, hosted on their own ftp server14:19.02 
Robin_Watts Right. So I've caught the thing being overwritten in a gc. gc_objects_compact14:21.49 
kens Hmm chrisl's bmpcmp i s sick it seems14:24.54 
chrisl yeh, I just killed it - can't work out why the normal push would be fine, but bmpcmp wouldn't......14:25.37 
henrys hmm, power failure last night and both macpro and henrysx6 had to be started manually ...14:27.15 
kens Hmm henrysx6 is idle, has it been removed from cluster ?14:35.23 
Robin_Watts kens: Nope.14:35.37 
  At least, not in the usual way.14:35.46 
kens Oh well, its not important really14:36.02 
henrys kens:huh it was just working and now it's idle14:37.10 
kens henrys, no idea :-)14:38.12 
  But like I said, its not important14:38.20 
  Robin_Watts : your mail to Gemma, is that he problem you're looking at ?14:39.06 
Robin_Watts adeo.pdf was yesterdays problem.14:39.16 
kens Oh, too many problems ;-)14:39.27 
Robin_Watts You sound like my psychiatrist.14:39.49 
kens I probably don't charge as much though14:40.02 
alexcher kens: I've got the file. The file fails only on v. 9.01. Any other version works just fine.14:44.37 
kens Well, I guess it was a regression14:44.53 
  I'd leave it to Marcos to bisect14:45.06 
  But its up to you14:45.13 
henrys kens:well neptune labs is a useful one to follow up on. We'd like to know why they're switching, I'll send it on to Scott14:51.43 
kens OK14:51.52 
henrys 5 of a meeting Robin_Watts, tor8, and paulgardiner14:54.49 
Robin_Watts oh, yes.14:55.09 
paulgardiner Yep. I'm here.14:55.18 
kens suspects Robin_Watts will be glad to step away from gc problem14:55.23 
paulgardiner henrys: I'm still having problems with bugzilla. Maybe I'm doing something wrong my end.14:57.01 
henrys paulgardiner:more likely I screwed it up, what's wrong?14:57.20 
paulgardiner Still can't edit the assignee14:58.05 
  Or close bugs, but I assume that's because I'm not the assignee.14:58.46 
Robin_Watts paulgardiner: Random idea; log out, then back in again?14:59.05 
henrys I did make a mistake hang on.14:59.45 
paulgardiner But Robin_Watts's solution seems to have worked!!15:00.32 
henrys I just changed so you have the proper permissions also, it was certainly wrong before.15:00.59 
  anyway sounds like we're goo15:01.12 
  d15:01.14 
paulgardiner Looks ok now. Great. Thanks henrys 15:01.17 
henrys just reading the report I'll make sure to get to it night before in the future.15:02.42 
  so it looks like the schedule for mid october or so is still good.15:06.10 
paulgardiner Yes, I think so.15:06.27 
  That's based on my guess as to what constitutes an acceptable alpha.15:06.55 
henrys I'm thinking it would be useful to see what foxit is doing with all this stuff but I don't know if that would slow you down. That's who we lost the chrome deal to and keeping an eye on how we do relative to them is probably worthwhile15:10.25 
paulgardiner I think I have it installed here.15:11.00 
tor8 henrys: I've heard people gripe about chrome's pdf form support saying it's worthless and doesn't work often enough so that you have to use acrobat anyway... but let's not derail the project just yet :)15:11.32 
paulgardiner At least we are in the position of being able to react to similar criticisms by targetting what people moan about most.15:13.07 
henrys tor8:I am surprised you know a person who uses an adobe form. Haven't met one myself15:13.27 
  now I'm really derailing15:13.43 
tor8 henrys: some tax form or post office form I think15:13.55 
paulgardiner Don't tell me I've wasted all this time. :-)15:13.58 
tor8 shuts up.15:14.04 
paulgardiner My other half used a pdf form to apply for a job the other day.15:14.45 
Robin_Watts And do we support that form? :)15:15.01 
henrys paulgardiner:was it a server submission?15:15.41 
paulgardiner I found this out when she rang me up saying she thught she must have hit the back button accidently and is there any way to get back all the info she'd been typing in over the last hour. :-(15:16.03 
Robin_Watts ouch.15:16.26 
  Robins second law of computing: It never takes as long the second time.15:16.54 
henrys I still think it is an important project, if nothing else it has been noted as a deficiency by a potential game changing customer so we continue...15:17.03 
paulgardiner henrys: Oh. That's an interesting question. I think is was, because it had facilty for a digital signature.15:17.30 
henrys Robin_Watts:oh for computing right ;-)15:17.54 
Robin_Watts :)15:18.10 
henrys anyway progress looks great any specifics to discuss with (2) and (8)15:19.11 
  ?15:19.12 
Robin_Watts 2) is mostly finished now?15:19.40 
  s/mostly// ?15:19.50 
paulgardiner I think 2 may be at a stage acceptable for the alpha15:19.51 
Robin_Watts and 8 is "how much low hanging fruit can we gather before a release", right ?15:20.09 
paulgardiner It's a bit clunky in several ways, but usable15:20.15 
henrys usable is alpha15:20.46 
paulgardiner Robin_Watts: yes, but the cluster test look to be telling me I'm slowly winning.15:21.16 
  If I ignore the app and display object, there isn't much unimplemented that our test files use.15:21.44 
Robin_Watts paulgardiner: Cool.15:21.44 
henrys Robin_Watts:I guess we shoud ask you the first law of computing.15:22.12 
paulgardiner But I may get a shock when I add empty app and display objects and find out how many used props/methods they have.15:22.13 
Robin_Watts "All compilers suck"15:22.22 
henrys is there a brief write up for the how exactly the test suite works, where the test are etc.15:23.39 
paulgardiner One way we differ greatly from Reader/Chrome/Foxit is that we use a dialog for text capture, whereas they all do inline entry15:23.42 
henrys Adobe uses a dialog don't they?15:24.44 
paulgardiner not that ive noticed15:25.45 
henrys paulgardiner:why are we distinguishing ourselves?15:26.06 
tor8 henrys: because our gtk+ viewer isn't ready yet15:26.28 
Robin_Watts There are advantages/disadvantages to both ways of working.15:26.37 
  inline is clearly better if you can manage it, but for devices like phones, native may be preferred as the phone has a standard UI for keyboard + text entry etc.15:27.15 
  The plan is that we will offer both, but native was done first for simplicity.15:27.38 
tor8 Robin_Watts: we're discussing "dialog" input rather than native/inline (or at least I thought we were)15:27.50 
henrys are we targeting the gtk+ viewer to be ready for alpha?15:27.51 
paulgardiner henrys: that was an early dicision. I think I was the only one keem on inline early on15:28.07 
Robin_Watts tor8: Not sure I see the distinction between dialog and native.15:28.32 
  Whether your native input field is in the same window as the display, or in a pop up dialog, it's still under the control of the app.15:29.19 
henrys certainly not something to hold up the alpha.15:29.57 
paulgardiner Robin_Watts: it's whether mupdf generates the image of the text as you type15:30.41 
henrys it is half past now, anything else we need to discuss. I would like to get the regression test suite we have if someone can point me to it.15:30.57 
Robin_Watts henrys: It's in svn.15:31.07 
  tests_private/pdf/forms/15:31.12 
henrys thanks15:31.15 
  kens:I think henryx6 is okay now at least I saw it running on the last job.15:33.23 
kens OK np henrys15:33.32 
Robin_Watts I wonder if marcosw is working levers behind the curtain.15:34.06 
chrisl I'm getting very confusing bmpcmp results :-(15:35.00 
henrys I do need to talk to him at the meeting we had another mysterious compile fial on henrysx615:35.11 
Robin_Watts chrisl: Are you entirely up to date?15:35.37 
chrisl Robin_Watts: up to date with what?15:35.54 
Robin_Watts the trunk15:35.59 
  What's confusing about the results?15:36.25 
chrisl Not quite, no. But I'm getting far fewer diffs in the bmpcmp than the preceding push would suggest15:36.34 
Robin_Watts Ah, right.15:36.41 
  It's all high res banded things.15:37.34 
chrisl the clusterpush says ~1800 files with differences, and bmpcmp gives me one page of diffs.......15:37.38 
  I filtered the tests down to 600dpi pbmraw output15:38.29 
henrys tor8, Robin_Watts:I haven't heard back from Miles but I don't think I need to get started, I'll propose zeniko work with tor8 and Robin_Watts to create retroactive bug to get him reimbursed from his work, once that is done he can sign the transfer of copyright and we can talk about future stuff. okay?15:38.45 
Robin_Watts OK.15:39.04 
  I don't feel I entirely understand where the copyright assignment discussions fell off the rails.15:39.27 
chrisl D'you think we'll ever get him to report bugs in a sensible fashion?15:39.51 
Robin_Watts Last I heard, I'd thought he was happy to sign some sort of copyright assignment, with just the wording to be agreed.15:40.09 
  Was it perhaps that he disagreed with the wording in Miles' standard form?15:40.32 
tor8 chrisl: maybe if we completely ignore their own bugs and SumatraPDF.patch he'll start submitting patches or open bugs in our bugzilla...15:40.47 
  sebras is playing along too nicely ;)15:41.04 
chrisl tor8: maybe - I just don't don't like trying to wade through multiple bugs/patches all dumped into one bug report :-(15:41.50 
Robin_Watts chrisl: Indeed. That's the big problem with sumatraPDF.patch.15:42.07 
tor8 chrisl: he sometimes opens bugs, and then gripes about us not taking on bugfixes from sumatrapdf he hasn't entered, or bugs that depend on fixes in there that we haven't taken on, etc.15:42.44 
henrys my feeling is a lot will come to light during the retroactive bug bounty session ;-)15:42.52 
tor8 I'd be a lot happier if he could use git or send us patches so we can review individual contributions15:43.21 
henrys for now we are just trying to make him whole so he can sign. I think we should just focus on that.15:44.15 
chrisl I've closed a few gs bugs because they had multiple patches for multiple problems attached.......15:44.28 
sebras Robin_Watts: chrisl: he also pointed me here: http://code.google.com/p/sumatrapdf/issues/list?can=1&q=label:MuPDF%20status=Fixed15:48.20 
  where you can find their fixed bugs with pointers to each revision which translates to a patch fixing each issue.15:48.55 
Robin_Watts sebras: Well, that's a lot more use, but it would still be better if he opened bugs in our bugzilla, even if they just pointed to that.15:49.34 
sebras I was just about to try to wade through this list and see if something was sensible. as it stands now, I'll just wait.15:49.38 
Robin_Watts That way there would be at least an implicit "I'm happy for you to take on this patch".15:49.56 
sebras Robin_Watts: yeah, I'm not sure what the difference is between the .patch-file and the list of bugs though.15:50.04 
chrisl I dislike being referred to other bug trackers - too often I find test cases disappear or similar issues arise15:50.52 
sebras tor8: but yes, I guess I'm partly to blame for the fact that they have never really opened bugzilla bugs as I was eager to find pdfs causing problems and fix them asap.15:51.28 
  s/eager/too eager/ perhaps15:51.51 
Robin_Watts chrisl: True, but having pointers to somewhere is better than being told "well, you should be keeping an eye on our bug tracker too".15:52.12 
tor8 if we offer bounties then I guess he'll be more incentivised to open bugs in our bugzilla15:52.18 
chrisl Robin_Watts: yes, true - I can still b*tch about it, though....15:52.56 
sebras chrisl: you can, but it doesn't improve the quality of the code. ;)15:53.25 
chrisl sebras: it improves my quality of life ;-) Actually, it's more because marcosw often trawls our bugzilla to add interesting test cases to our regression tests, and if tests are attached to other bug trackers, they may not be included in our test suite15:54.52 
sebras chrisl: another issue with bugtrackers (which gs bugzilla might have as well?) is that they run out of space for attachments. I have come across many bugs (not from gs) where they refer to filehosting sites due to out of attachment space.15:55.56 
kens I htink we do have a limit too15:56.17 
chrisl sebras: yeh, we do have a limit on attachment size - it's not *that* common that people fall over it, though15:57.26 
sebras chrisl: which makes it even more probable that the pdfs disappear.15:57.29 
Robin_Watts Morning marcosw. Which continent are you on today?15:58.02 
marcosw Robin_Watts: I'm in california. You?15:58.17 
Robin_Watts back at home.15:58.28 
henrys marcosw:I was wondering if we should move this bug meeting to 9:30 Tuesday so other can participate if they want to.15:59.57 
  the friday bug meeting that is.16:00.25 
marcosw works for me.16:00.52 
henrys that was my meeting opener ...16:00.54 
  never buy turtles, did I tell you that? Foul creatures are laying eggs and it stinks in here. Then they eat them - both females.16:01.57 
Robin_Watts craigslist :)16:02.17 
henrys okay so we will now have the bug meeting at 9:30 Tuesday if anybody wants to stick around and weigh in great if not that's fine too.16:03.19 
Robin_Watts After the meeting I have some gc/halftone/graphics state questions for ray and mvrhel, if possible.16:04.01 
henrys marcosw:another thing, I saw another compile fail doing the romfs on henrysx616:04.06 
Robin_Watts s/meeting/meeting(s)/16:04.09 
mvrhel Robin_Watts sounds good16:04.58 
henrys Robin_Watts:I suspect we might wrap up early, I didn't see much to talk about.16:05.03 
  anybody have anything?16:05.15 
Robin_Watts henrys: A compile fail occurred on henrysx6 with a git commit from alexcher the other day (dunno if that's the same one you're talking about)16:05.31 
kens I can't think of anything16:05.35 
Robin_Watts I reran it and it went away.16:05.39 
henrys alexcher:what is the future of change the ref size?16:05.40 
marcosw henrys: that sucks. I guess I'll add code to the cluster to detect when the romfs build fails and do a retry. Shouldn't add too much time.16:05.42 
henrys Robin_Watts:yes that is what I meant.16:05.52 
mvrhel henrys: I have a couple segvs that I am working on after I turn on all the threshold stuff16:05.58 
  the issue is in the clist copy_planes command16:06.09 
alexcher henrys: I need some help. Please look at the bug 69326816:06.18 
chrisl marcosw: any idea why the romfs fails to build?16:06.24 
marcosw chrisl: feels like a hardware issue. I ran 2000 test builds using the same commands as the cluster uses and none of them failed.16:07.19 
chrisl marcosw: hmm, it's just that Solaris/SPARC build problem a couple of weeks ago was something in the PS compaction code going awry16:08.13 
henrys mvrhel:this is about halftoning before clist?16:08.45 
Robin_Watts alexcher: sizeof(ref) is 12 on 64 bit windows, but 16 on 64bit linux, right?16:09.30 
  The sizeof(union v) is presumably 8 on both.16:10.04 
alexcher No it's 16 on any 64-bit platform.16:10.11 
Robin_Watts oh, so the problem is with 32 bit windows ?16:10.24 
alexcher Yes16:10.32 
Robin_Watts Right, so can we put a __int64 in the union to solve it ?16:10.47 
  __int64 unused;16:11.09 
alexcher 16-byte ref works everywhere.16:11.26 
mvrhel henrys: yes16:11.47 
alexcher 16-byte do we want to save some memory on 32-bit platforms?16:11.56 
Robin_Watts right, and putting a 64bit in in the union will make it a 16 byte ref everywhere?16:11.57 
mvrhel henrys: as I got to working on it, I realized we were not doing the fast thresholding when we could be16:12.09 
  I needed to add in all the cases when we can do it in the clist_begin_image code16:12.27 
  to make sure we go off and do it16:12.33 
  when it makes sense (size wise)16:12.46 
ray_work for embedded systems it does result in somewhat larger memory consumption, but compared to other stuff, I doubt the refs will stress much of anything16:12.48 
henrys somewhat clunky the code requires that sort of hack.16:14.09 
ray_work mvrhel: what's the decision based on for using the thresholding ?16:14.31 
henrys but I guess Robin_Watts' solution is the simplest fix.16:15.04 
ray_work assumes that the high-level data is larger than the thresholded monochrome bitmap16:15.11 
alexcher The problem is that windows tas 8-byte allocation alignment. Can it libe with 4-byte one?16:15.14 
  The problem is that windows has 8-byte allocation alignment. Can it libe with 4-byte one?16:15.23 
mvrhel ray_work yes16:15.30 
  ray_work: we sometimes are going to cmyk halftone16:15.59 
ray_work mvrhel: thanks. So that means that near full res images will be thresholded down16:16.11 
mvrhel ray_work : yes16:16.23 
alexcher In fact, PC doesn't care about misaligned data at all.16:16.53 
chrisl alexcher: does this apply only to 32 bit windows, or to all 32 bit platforms?16:17.25 
henrys alexcher:I don't understand why that matters - we'd like the code to work on machines with either aligment and Robin_Watts' fix will allow that, it would be nice to see the underlying issue though - where in the code is the dependency?16:17.39 
alexcher Windows compiler alighes dounles to 8 bytes.16:18.25 
  GCC alignes to 6.16:18.37 
  GCC alignes to 4.16:18.41 
Robin_Watts obj_align_mod is the minimum alignment that we impose on objects we allocate 'aligned'. I think it's important that obj_align_mod shouldn't be SMALLER than it actually needs to be, but having it larger is no problem.16:18.58 
alexcher We detect this and use for allocator granularity.16:19.06 
mvrhel ray_work: question is, do you always want to pre-threshold to save memory in clist or is it sometimes better to let it happen post clist to take advantage of the multithreaded rendering16:19.41 
ray_work alexcher: but other 32-bit platforms _do_ care about alignment16:19.45 
Robin_Watts Embedded platforms means "ARM" these days, in any sane world, and modern ARMs insist on 64bit alignment to enable the use of LDRD etc.16:20.04 
mvrhel ray_work: perhaps I should include a #ifdef to turn it off and on?16:20.09 
henrys mvrhel, ray_work:I'd like a command line option for that, what do you think?16:20.15 
mvrhel that would be better16:20.22 
Robin_Watts So for embedded things, we're going to be wasting the memory anyway.16:20.41 
  So for simplicity, I say waste it everywhere.16:20.51 
henrys mvrhel:a device parameter would be better than a user parameter for me.16:21.05 
chrisl alexcher: I'd go with the padding to 16 byte alignment - presumably when we port to an embedded 32 bit platform, we can tailor the padding based on the platform/compiler etc.16:21.09 
ray_work mvrhel: good question. For embedded systems with memory based clist, then clist size is important. For host based (usually disk based clist) the clist size doesn't matter, and performance is better with MT rendering16:21.18 
mvrhel henrys: ray_work, any suggestions for a command name?16:21.49 
ray_work chrisl: I agree. I don't think embedded systems will care -- it's not like we often have thousands of refs around16:22.03 
  mvrhel: command name ?16:22.21 
henrys PreBandThreshold=true16:22.26 
mvrhel ray_work: command line option name16:22.53 
  in long hand16:22.56 
ray_work mvrhel: oh, did you mean a prn_device parameter ?16:23.10 
mvrhel ray_work: yes. henrys just said a device parameter would work best for him16:23.36 
chrisl Robin_Watts: FWIW, our biggest embedded Ghostscript customer right now is using PPC, not ARM......16:23.43 
Robin_Watts chrisl: See the "sane world" bit of my comment :)16:23.57 
ray_work mvrhel: henrys: I think henry's suggestion is fine, or 'EnablePreBandThresholding' (more descriptive, but a lot longer)16:24.06 
Robin_Watts EnablePreClistThresholdingOfImagesIfWouldBeSmaller16:24.39 
mvrhel hehe16:24.43 
chrisl Robin_Watts: just pointing out that there is an embedded world outside of ARM, even nowadays16:24.49 
Robin_Watts chrisl: Yes, but I don't have to approve of it.16:25.02 
mvrhel ok I am going with PreBandThreshold16:25.05 
henrys ray_work, mvrhel:either of the two is good for me.16:25.24 
chrisl Robin_Watts: okay, you enjoy your walled garden :-)16:25.28 
henrys I don't know if we want to even get into checking if it is more optimal to do it or not. A 4 bit cmyk customer probably wants to put the option on the command line ... and that's it.16:26.41 
alexcher Robin_Watts: we need an ARM box to run tests.16:26.48 
Robin_Watts I have a beagleboard here.16:27.01 
henrys I guess I was wrong about the meeting ending early.16:27.36 
  so chrisl, Robin_Watts and alexcher will work on 693268 until we have something? Let's get rid of this one once and for all.16:28.38 
ray_work henrys: I don't think we should ever threshold pre-clist if it results in size explosion (as well as single threaded performance)16:29.16 
mvrhel right. I think the check is required16:29.35 
  it is easy enough to do16:29.44 
Robin_Watts alexcher: I can probably poke a hole in my firewall if you want to use the beagleboard. Or I can run tests.16:29.54 
ray_work but we may want to make it the default and allow customers that use NumRenderingThreads > 1 to disable it16:30.08 
henrys mvrhel, ray_work:fair enough16:30.39 
ray_work mvrhel: or just base it on NumRenderingThreads > 1 (disable then, and otherwise use it and we don't need another parameter)16:30.55 
Robin_Watts or we could break the bank and buy alex a raspberry pi :)16:31.21 
henrys marcosw:did you want to do a bug report this week?16:31.45 
ray_work that's about $99, right ?16:31.49 
chrisl raspberry pi is about $45 isn't it?16:32.25 
henrys I see 25.0016:32.49 
ray_work yes, I just saw that. And you can get an enclosure for $1516:33.06 
chrisl Oh, I thought it was 25ukp.....16:33.18 
henrys But we should be able to knock down this bug with chrisl, Robin_Watts and alexcher machines right?16:33.52 
Robin_Watts henrys: yeah16:34.16 
  model A = $25, model B = $3516:34.27 
  You want the model B as it has an ethernet port.16:34.42 
alexcher At this price we can have a cluster.16:35.21 
Robin_Watts IF we want a cluster of ARM based things: http://www.baserock.com/16:35.56 
henrys anyway 5 past the meeting finished, if marcosw wants a meeting we'll just talk across the halftone discussion.16:36.39 
marcosw henrys: I'm just looking at the bug report, that was just emailed out, to see if there is anything that needs dicussion.16:37.13 
ray_work Robin_Watts: that looks pricey. 8 mSata SSD 30Gb drives16:37.24 
  the CPU is probably the cheap part of it16:37.44 
chrisl Robin_Watts: RS are quoting 14 weeks for raspberry pi delivery :-(16:37.50 
Robin_Watts ray_work: No idea. I have a friend who is involved.16:37.56 
  ray_work 30*8 = 240Gig SSD = 120 quid these days.16:38.34 
ray_work Robin_Watts: that's for a single 256 Gb drive, not 8 little ones, right ?16:39.18 
henrys marcosw:down to 5 customer bugs nice16:39.23 
Robin_Watts right. It has a single SSD, per slab, not one per CPU16:39.39 
  oh, wait, you may be right.16:40.02 
kens OK I'm off, will be back later to see what my regression test shows up16:40.31 
ray_work Robin_Watts: one per CPU (each CPU is a quad core, however)16:40.37 
  Robin_Watts: Kingston SSDNow mS100 32GB is $5416:41.10 
Robin_Watts yeah.16:41.28 
marcosw henrys: I think we do not need to meet this week.16:41.29 
henrys marcosw:agreed16:41.47 
Robin_Watts Anyway.... ray/mvrhel. I've been looking into bug 69327816:42.08 
  It's a bug that's shown up since we pushed the number of client colors from 32 to 64.16:42.36 
ray_work Robin_Watts: I'm sure as an OEM they get them for less. SanDisk mini SSD 32Gb is $4216:42.47 
Robin_Watts but I don't think that's the direct cause (at least, I have found no evidence that that is the case)16:43.03 
  I think it's just that it happens to move stuff in memory and the bug is triggered.16:43.29 
ray_work Robin_Watts: OK.16:43.43 
  was this what you were talking with kens about earlier ?16:44.14 
Robin_Watts The problem is that in clist writing it tries to write a device_color that happens to be a halftone, and the halftone pointer it's finding has actually been moved due to garbage collection already.16:44.33 
ray_work Robin_Watts: only fails on linux, right ?16:44.35 
Robin_Watts Yes.16:44.35 
  Only fails on linux, only with a very specific configure invocation.16:44.56 
  but it does at least fail in the debug build (but not memento)16:45.10 
ray_work so what struct has the stale pointer to the ht ?16:45.12 
Robin_Watts hold on.16:45.41 
ray_work sounds like a struct descriptor is missing that pointer16:45.58 
Robin_Watts In clist_fill_path at line 793, we call cmd_put_drawing_color16:46.20 
  The pdcolor there is incorrect.16:46.41 
ray_work (it needs to be in the RELOC list). Have you tried a run with -Z@$? (with shell you need -Z@\$\? )16:47.04 
  in particular -Z? does extra checking16:47.30 
Robin_Watts ray_work: I can see where the halftone is relocated during garbage collection.16:47.59 
ray_work Robin_Watts: this is during clist writing, correct ?16:48.08 
Robin_Watts Yes.16:48.22 
  I've run with -Z@716:48.38 
ray_work Robin_Watts: I meant, what structure has the stale pointer 16:48.39 
  -Z@ just does the filling with pattern. -Z? is the extra sanity checking and often shows a problem early16:49.31 
Robin_Watts The pgs16:49.32 
  Let me try rerunning with -Z@$?16:49.50 
ray_work Robin_Watts: let me switch to peeves.16:49.57 
rayjj OK. Here I am on peeves16:50.27 
Robin_Watts ok, that fails in the same place.16:50.35 
  ~robin/sauce/ghostpdl.git/gs/log16:50.46 
mvrhel Robin_Watts sorry I had stepped out someone was at the door16:50.48 
rayjj Robin_Watts: so no extra messages.16:50.52 
mvrhel it looks like ray has you taken care of though16:51.04 
Robin_Watts mvrhel: Yeah, unless you have any experience in this area...16:51.28 
rayjj I think most of us have visited GC hell 16:51.51 
Robin_Watts In this run, the pdht is 0x1bb1638 when it crashes.16:53.03 
rayjj grr... it is taking forever to open the bug from peeves :-(16:53.34 
  Robin_Watts: is that log on casper or peeves ?16:54.00 
Robin_Watts peeves.16:54.42 
  peeves is my 64bit linux debugging machine of choice.16:55.00 
  In the gc the dev_ht is found through the gs_state structure.16:55.49 
  [7]relocating ptrs in gx_device_halftone(224) 0x1bb162816:57.09 
rayjj Robin_Watts: right -- I see that location being marked frequently in the log with index == 2516:57.20 
Robin_Watts Yes.16:57.29 
  That's the index within the gs_state enumeration.16:57.50 
  [7]compacting gx_device_halftone 0x1bb1628(224) to 0x1bb131816:58.34 
rayjj Robin_Watts: is the problem that the pointer is a pgs is stale (still 1bb1628) or something else ?16:58.47 
Robin_Watts but then the pgs device color still points to a stale location (1bb1638)16:59.18 
rayjj what's the address of the pgs when it fails ?16:59.46 
Robin_Watts pgs=0x1865cb817:00.19 
rayjj you should be interested in the relocating ptrs in gs_state(1928) steps I think17:00.32 
Robin_Watts pgs->dev_ht = 0x1bb132817:00.53 
  pgs->color[0].dev_color = 0x1bf159817:01.52 
rayjj something strange -- I see that same address for a gs_context_t AND gs_state17:02.27 
Robin_Watts pgs->color[0].dev_color->colors.colored.c_ht = 0x1bb163817:03.03 
rayjj Robin_Watts: that's the 1865cb8 addres17:03.05 
  Robin_Watts: OK. so the pointer pgs->dev_ht = 0x1bb1328 is correct and got reloc'ed correctly17:03.58 
Robin_Watts Yes, the dev_ht moved right. The device color pointer did not.17:04.20 
rayjj but the pointer in pgs->color[0].dev_color->colors.colored.c_ht did not17:04.25 
  so the colors.colored struct doesn't have the c_ht enumerated so it can be traced and reloced17:05.42 
  since it is a union, it needs a 'smart' ENUM to trace pointers depending on type17:07.15 
  yuck.17:07.24 
Robin_Watts Is that because st_dc_colored_masked and st_dc_devn_masked share the same enumerators I wonder...17:08.11 
  in fact, st_dc_ht_binary , st_dc_colored_masked and st_dc_devn_masked are all the same.17:09.23 
henrys tor8, Robin_Watts:okay here we go17:10.18 
  flame on17:10.28 
Robin_Watts Ok, so as expected he's effectively pointed at everything where we've credited him at all. Presumably he's not expecting every one of those to result in a bountiable bug, but they are a reasonable place to start.17:14.12 
rayjj AFAIK, bountiable has to be presented in a bug on bugs.ghostscript.com in order to qualify17:15.24 
Robin_Watts I'd imagine the AESv3 stuff should be a bountiable bug by itself. and possibly the XPS mitering mode.17:15.32 
  rayjj: Yes. The plan here is to generate 'retroactive' bountiable bugs that he can qualify for and thus sign off his work.17:16.02 
rayjj and should be set by us BEFORE someone develops the expectation of a bounty17:16.16 
Robin_Watts rayjj: Yes. This is an unusual usage. See the logs for the past day or so.17:17.02 
rayjj Robin_Watts: I understand. The problem is that the developer doesn't want to let us have the copyright (as we require), right ?17:17.44 
Robin_Watts rayjj: I'm not quite sure where it all ran off the rails, but basically, I think he's unhappy to sign Miles' standard copyright assignment form as it is.17:18.24 
rayjj I guess we need to spell out somewhere that submission of bug fixes / patches implicitly assigns copyright to Artifex Software17:19.04 
henrys Robin_Watts:I expect the contributor and we will value the work differently and we're going to have problems. I don't know what to do about it other than let it happen.17:19.10 
sebras +rayjj: that would be a _very_ good idea.17:19.23 
henrys rayjj we should do that I agree17:19.58 
rayjj henrys: once we deal with these horses that are out of the barn ;-)17:20.35 
henrys and we've got some wild horses17:22.34 
Robin_Watts henrys: I could do a similar thing to bugzilla on attachments to make people accept a copyright assignment statement before uploading a patch as I did with the "origin" stuff.17:23.41 
henrys Robin_Watts:that would solve it as long as all patches come through bugzilla. None of this business sebras is doing with the mupdf patch.17:24.49 
  for example.17:24.58 
Robin_Watts henrys: Right.17:25.01 
chrisl Robin_Watts: not everyone uses the patch flag, either17:25.13 
Robin_Watts But I like to think that once we have paid him through the bug bounty program once, we can reasonable say "in order for anything to be considered for payment in future, it needs to go through this route"17:25.56 
rayjj Robin_Watts: that seems reasonable17:26.31 
sebras henrys: true, on the other hand there is no implicit copyright assignment on bugzilla today as +rayjj pointed out, so I guess the difference between the two is rather minute.17:28.13 
henrys sebras:right but there will be shortly17:29.20 
sebras henrys: I know. and I do understand the consequences of that.17:30.17 
chrisl I think we also need to add appropriate words to the gs/docs files and the FAQ, and the MuPDF web site, and README etc....17:30.31 
rayjj chrisl: and mention the bounty program (link to it) in those same places. The bounty program description can be made clear that we pay only if we have marked a bug bountiable, 17:32.09 
chrisl rayjj: I thought the bug bounty scheme was already covered - we should probably review that, anyway17:33.07 
rayjj we can mention that if someone has an improvement that they'd like to be paid for to contact us 17:33.10 
  but once a patch is submitted we own the copyright and are not obligated to pay anything17:34.05 
sebras henrys: ok, so I'll ignore the .patch-file and their source code repo. what about picking out the .pdf-files attached or linked to from their bugzilla, run them through mupdf/gs/whatever and fix whatever bugs I discover by myself without looking at their code/fixes/whatever. I.e. getting testcases from other's bugzillas.17:34.17 
rayjj sebras: the pdf files are public, so I don't see there is any harm there. Same with the info in their bug tracker17:35.08 
chrisl rayjj, henrys: I'll see about reviewing the gs/docs files and the bug bounty stuff tomorrow, and I'll e-mail tech with any concerns or revisions for consideration17:36.02 
rayjj unless there is a notice in their bug tracker that a PDF is restricted use17:36.05 
henrys sebras I need to talk with tor8 about the status with other sumatra pdf developers.17:36.39 
rayjj sebras: even the bug report is just 'information' on how to reproduce the bug and is not copyrighted17:36.57 
henrys but for now yes do not look at their solution everything else is fair game.17:37.35 
rayjj (in fact if there was a copyright to the bug report, it would be owned by the submitter)17:37.49 
chrisl I have to go - 'nite all17:38.31 
rayjj Robin_Watts: back to the GC problem ?17:38.34 
henrys bye chrisl17:38.37 
Robin_Watts rayjj: Yeah, I'm writing new enumerators/relocators now.17:38.53 
  night chrisl_away 17:38.57 
sebras henrys rayjj: looking at their site as it is now, there is nothing indicating any restricted use of the contents of their issue tracker17:40.57 
rayjj Robin_Watts: OK. I'll be working on other stuff, so text me if you need me.17:49.23 
Robin_Watts rayjj: Thanks. I'm struggling to find where stuff is declared...17:49.45 
  ./base/gsptype1.c:1070: error: âdc_colored_masked_enum_ptrsâ undeclared here (not in a function)17:50.05 
  ./base/gsptype1.c:1070: error: âdc_colored_masked_reloc_ptrsâ undeclared here (not in a function)17:50.08 
  Those are my new ones, based on dc_masked_enum_ptrs and dc_masked_reloc_ptrs17:50.29 
  but I can't see where the original 2 are declared.17:50.47 
rayjj Robin_Watts: BTW, I think we need a comment in gx_device_color_s to point out that the c_ht and b_ht MUST be first since the ENUM/RELOC relies on being able to use b_ht for both types (binary and colored)17:51.27 
  AFAICT17:51.37 
Robin_Watts I'm hoping to avoid that by doing this properly.17:52.06 
rayjj original two ???17:52.08 
Robin_Watts dc_masked_enum_ptrs and dc_masked_reloc_ptrs - where are they declared?17:52.47 
  I can see where they are defined, but not where they are declared.17:52.58 
rayjj Robin_Watts: line 1125 ?17:55.11 
Robin_Watts That's the definition.17:55.37 
rayjj Robin_Watts: and the declaration -- it is static17:55.59 
  or what am I not understanding ?17:56.47 
Robin_Watts (sorry, phone!)17:56.56 
rayjj in gsstruct.h there the procs are declared using gs__st_composite with "static struct_proc_enum_ptrs(penum);" (and reloc, too)18:02.16 
kens Can I butt in for a moment ?18:05.15 
rayjj kens: sure18:05.21 
  Robin_Watts is on the phone18:05.28 
kens Is anyone else seeing seg faults on 09-47n.ps and 09-50.ps at 300 dpi ?18:05.45 
  The last 2 cluster pushes I've run have had those18:05.56 
  But these are regular rendering, not pdfwrite, and all my changes are in pdfwrite.18:06.10 
rayjj kens: 09-50.PS is the bug Robin_Watts is looking at 69327818:06.19 
kens Oh, then its probably the case tht teh other is related18:06.39 
  Thanks ray_work18:06.52 
rayjj kens: that is rendering to the pkmraw device, but it may happen elsewhere (depends on when GC happens)18:07.07 
kens Or rayjj :-)18:07.09 
rayjj kens: either (my highlight is just set for 'ray')18:07.34 
kens I'm seeing 09-50 with the pkmraw device18:08.00 
rayjj kens: right. That's exactly the bug 693278 case18:08.24 
kens but09-47n.ps is with psdcmyk, I dont; think that can be a halftone problem18:08.24 
rayjj kens: Robin_Watts noted that the same (bogus) enum/reloc is also used for devn color type18:08.59 
kens Ah....18:09.06 
  Then maybe it is related after all18:09.12 
  OK So I'll ignore them for now, I don't thin my code can be causing this, thanks ray18:09.24 
Robin_Watts ok, back.18:14.04 
  Yes, 09-50 dying with the pkm device is definitely this bug.18:14.18 
  Ok, rayjj. On peeves, in ~robin/sauce/ghostpdl.git/gs/18:15.29 
  do: make debug18:15.38 
rayjj Robin_Watts: so you are trying to add dc_colored_masked_enum_ptrs and getting the error ?18:15.39 
Robin_Watts and tell me what I've done wrong :)18:15.46 
  Yes.18:15.48 
  I've got gs_private_st_composite_only(....., dc_colored_masked_enum_ptrs, dc_colored_masked_reloc_ptrs); at line 1170 in gsptype1.c18:16.48 
  and that's complaining that they haven't been declared beforehand.18:17.06 
  oops. 1070, sorry.18:18.01 
rayjj Robin_Watts: looks like you put your ENUM_PTRS_WITH inside another RELOC_PTRS_WITH18:19.33 
  Robin_Watts: it should be AFTER the RELOC_PTRS_END18:19.51 
Robin_Watts ok, that solves lots of problems, but not the one I was originally talking about.18:21.25 
  try make debug again.18:21.29 
rayjj strange -- how did it work for dc_masked_enum_ptrs ???18:22.31 
Robin_Watts SOMEWHERE in this code, something must be declaring dc_masked_enum_ptrs elsewhere.18:23.05 
  probably in a nest of macros.18:23.16 
  but I can't see where :(18:23.24 
rayjj Robin_Watts: grep only shows it in base/gsptype1.c 18:23.54 
Robin_Watts rayjj: yeah, I know :(18:24.30 
  Got it.18:27.59 
  It was a macro problem.18:28.13 
  Well, it doesn't SEGV any more.18:28.42 
rayjj I see composite, not composite_only18:29.40 
  Robin_Watts: are you going to fix devn too ?18:29.55 
Robin_Watts Yeah. I had composite_only,and changing it to composite solved it.18:30.04 
  Just looking at that now.18:30.07 
  As far as I can tell, devn is simpler; doesn't need anything enumerated/relocated.18:30.34 
rayjj so kens' problem with psdcmyk (and 09-47n.PS) may be something else ?18:36.26 
Robin_Watts Maybe.18:36.37 
  Well, I think the current code is enumerating something that isn't a pointer as a pointer.18:37.11 
  So it could well be related.18:37.15 
  I'll cluster push a test in a mo.18:37.27 
rayjj oh, the b_ht ?18:38.39 
Robin_Watts yeah.18:38.43 
  I have some fixes here that shut valgrind up, that I want to separate out into a different commit/test.18:39.24 
rayjj Robin_Watts: OK, I'll be away now (see me back as ray_laptop) -- SMS me if you want to wake me up 18:46.15 
Robin_Watts Thanks rayjj - you've really helped me out.18:46.31 
ray_work just a listening ear mostly. My first programming course (WATFOR in 1968) had guys that we could go to for help when something wouldn't compile or didn't work. My favorite 'helper' was a guy that would just listen while I explained everything (up until I saw my error)18:48.37 
mvrhel ray_work: I may drag you into my fun one a bit later18:55.49 
Robin_Watts OK, cluster test running.18:56.59 
  mvrhel: If you get to quitting time tonight and don't have it working, feel free to send me an email cos I should hopefully be free to look into it tomorrow if you want.18:57.31 
Robin_Watts walks dogs. bbs.18:57.43 
  ouch. One of those baserock slabs is about 5K each.19:37.04 
marcosw henrys: I was just about to give paulgardiner the ability to close bugs on bugzilla, but I see you beat me to it :-)20:06.57 
mvrhel hi ray_laptop20:21.52 
rayjj BTW. I'm back here in front of peeves21:15.37 
  gs_debug['I'] = 121:22.05 
  gxclutil.c: cmd_put_list_op21:31.33 
  Robin_Watts: well, it looks pretty well engineered, but only 32 1.3GHz ARM cores isn't competitive in raw compute power (unless you have to have ARM)21:33.40 
  mvrhel: bbiab21:42.48 
  mvrhel: let me know here when you've sent the info or opened a bug21:43.10 
mvrhel rayjj: I did a bit more analysis and just sent you everything22:15.09 
  it appears to me that either 1) the reader is eating an extra 20 bytes or 2) the command (which is written after the 0th plane data) is written in the wrong location22:16.01 
  the subsequent command that is22:16.12 
  rayjj: I need to run a quick errand22:16.27 
  bbiab22:16.30 
rayjj mvrhel: got it. I'll have a look22:19.49 
mvrhel thanks22:30.47 
Robin_Watts rayjj: The attraction, I suspect, is the sheer number of them you can get into a single rack with no cooling/power supply problems.22:57.12 
rayjj Robin_Watts: maybe so. a 1U Dual Xeon (each with 8 cores) is about $2500, so maybe not that bad a price23:17.51 
  although I suspect that each 3GHz Xeon core is quite a bit faster than the 1.3GHz ARM core ;-)23:18.43 
alexcher rayjj: take a look at the bug 692769. I think gs works as designed and the bug can be closed. The bug is yours now.23:33.29 
 Forward 1 day (to 2012/08/22)>>> 
ghostscript.com
Search: