IRC Logs

Log of #ghostscript at irc.freenode.net.

Search:
 <<<Back 1 day (to 2015/11/12)20151113 
mjkr what's the supposed benefit of using mupdf-gl over mupdf on windows?10:07.35 
  (alternatively, what does opengl adds to the picture?)10:07.48 
tor8 mjkr: it's got a different UI, keybindings differ somewhat, but most importantly you can view the outline. press 'o' to toggle the outline.10:20.29 
mjkr ah, neat10:21.47 
  how does it compare with latest adobe acrobat reader for speed though?10:22.01 
tor8 mjkr: speed should be very similar to the old mupdf viewer; possibly a bit faster on linux.10:46.18 
mjkr ah. great.11:06.07 
kens tor8 in case you missed it in the logs I did look at your XPS commits last night13:18.48 
tor8 kens: yes, I saw. thanks.13:47.31 
kens Are you going to commit them ? THere's a few gradient bugs just might be affectd13:48.07 
tor8 kens: That was the plan. The gradient bug fix is one from zeniko.14:00.07 
kens I guess I could try gxps, or just pull from your repository. I'll just wait till you commit them though I htnk and test the 3 or 4 gradient bugs then.14:01.06 
  I meant MuXPS of course....14:01.18 
tor8 there are significant differences in how gradients are rendered between MuXPS and gxps due to the underlying graphics library14:02.01 
kens Well that's fair enough, but these bug reports look (from what I remember) just wrong (and I thnk in some cases, they vanish)14:02.40 
tor8 bah, git log --follow only works for single files, not directories :(14:03.02 
kens Didn't we have this discussion already ? :-D14:03.18 
tor8 well, I can solve it by checkout out the commit just before moving the directory :)14:04.16 
  it's just annoying... and I'm sure we've griped about it before14:04.27 
kens I seem to remember seeing it.14:04.52 
  Hmm the fist one I tried is wrong in MuPDF 1,7 as well as GhostXPS14:05.09 
tor8 right, the OpenXPS fix to MuXPS was from february 201314:05.18 
kens Well, wrong as compared to the MS viewer14:05.21 
tor8 yeah, we get a lot of gradients wrong I fear :(14:05.34 
  in both muxps and gxps14:05.41 
kens We have 4 or 5 bug reports open against GXPS and gradients14:06.00 
  One of the reporsts seems OK in MuXPS so I guess that's going to be closable14:07.36 
tor8 kens: I guess I can go ahead and push the current batch of commits (sans the WIP one)14:10.25 
kens THere's a bug report opened by robin that usggests an alpha problem with GhostXPS that MuXPS doesn't suffer from, that's one worth checking14:10.28 
tor8 I'm still working on porting some older bugfixes as well14:10.42 
kens tor8 Would be a good idea I thnk, I can quicky run through some of these reports and maybe close a couple14:10.49 
  Oh and I worked out the implication of the initial gradient sorting bug fix too14:11.52 
  OK I'll pull and rebuild then test14:12.37 
chrisl kens: when you have a moment, would you cast your eye over the top four commits please? http://git.ghostscript.com/?p=user/chrisl/ghostpdl.git14:25.02 
kens OK just a momment14:25.14 
chrisl I'm going to grab a drink.... back in a sec14:25.30 
henrys kens: he's starting early ;-)14:26.24 
kens Hmm hte pattern one is a cunning problem :-) Nice fix14:26.39 
chrisl I think the pattern one is the most important of the fuzzing ones - that could happen "for real"14:28.27 
kens Yes, that's a good one14:28.41 
chrisl And the fontmap ones is just fixing the code so it does what was originally intended14:29.21 
  s/ones/one14:29.29 
kens <quibble> the log for the incomplete TTF glyph lengths the 1st para should be incomplete *or* missing, not 'of' </quibble>14:29.31 
  Yes the fontmap one is trivial more or less, good to have the fix14:29.58 
chrisl Log message fixed, thanks14:30.47 
kens Just reading the code for that one, gie me a sec14:31.02 
  Yeah that looks reasonable to me14:32.13 
  some nice fixes there14:32.20 
chrisl Useful things, I hope.....14:32.40 
kens <quibble> the log for the incompeltThe pattern one is a definite good one, distinctly possible to hit that in the wild, the others defintely nice to have14:33.06 
henrys chrisl: is that the right bug number in the Pattern segfault?14:33.28 
kens didn't check the bug numbers, more concerned wiht the code....14:33.48 
chrisl henrys: it's part of the fix for that - the full fix for that bug is in two separate commits14:34.02 
henrys ah gcc 4.4 does not allow duplicate typedefs but gcc 4.9 and Visual Studio do. I fear as we upgrade we are going to get compile fails from the field, it a common problem in our include file situation.14:55.36 
  that's why my compile was failing on peeves and not elsewhere on the cluster run14:57.35 
chrisl henrys: I do test builds on older gcc's as part of the release process14:58.29 
henrys chrisl: that's good.14:59.37 
chrisl The linux binaries I build are done in old Ubuntu releases, for a decent chance of compatibility15:00.18 
Robin_Watts henrys: #include "standard_rant_about_the_stupidity_of_ghostscript_headers"15:02.11 
henrys chrisl: I wish there we a gcc flag for the old behavior though, to me a duplicate typedef is just wrong.15:02.21 
  Robin_Watts: I suppose I go on about refactoring a lot, but I'd love to see that spaghetti unbraided.15:03.55 
Robin_Watts henrys: Yes.15:04.02 
  Every header should include the others headers it depends on.15:04.12 
  Every typedef should only ever happen in one place.15:04.27 
  It's not brain surgery.15:04.37 
chrisl One problem we have is that there a few headers with circular dependencies15:05.11 
Robin_Watts chrisl: So that should be fixed.15:07.30 
  I can't believe there is a good reason for it.15:07.58 
chrisl Yes, I wasn't defending it, just observing - I hit it when trying to tidy up some stuff a while back15:08.24 
  There does not appear to be an option for gcc to go back to the old behaviour, except maybe to do "-pedantic -Werror" - and then we'd spend three months getting Ghostscript to build again.... maybe15:10.39 
henrys chrisl: yea, off for coffee15:11.36 
mvrhel_laptop henrys: so bug 696316 segv is occurring during the set up for the png long jump error handler. Line 102 in xpspng.c if (setjmp(png_jmpbuf(png))). It is making it through png_jmpbuf() but the call to setjmp() is doing the segv. Win32 ok just in x64. I can't see any obvious issues.16:59.22 
Robin_Watts mvrhel_laptop: It's the alignment of the jmp buffer.17:03.34 
mvrhel_laptop ok is there a fix for it17:03.48 
Robin_Watts jmp_buf needs to be 16 (or is it 32?) bit aligned on 64 bit builds.17:04.01 
  Just a tick, let me look at the source.17:04.12 
  png_jmpbuf(x) = x->png_jmpbuf17:05.52 
  jmp_buf is the first thing in png_struct17:06.56 
  so when we allocate png_ptr we need to make sure it's aligned properly.17:07.10 
mvrhel_laptop I see17:07.17 
Robin_Watts googles for jmp_buf alignment x64 and first hit is: http://sourceforge.net/p/libpng/bugs/216/ :017:08.43 
  OK, so we need a 16 *byte* alignment guarantee.17:09.26 
mvrhel_laptop and there you are.....17:09.32 
Robin_Watts malloc on 64bit windows has such a guarantee.17:09.43 
  our allocators do not. And really, we can't afford them too, cos it would cost huge amounts of memory.17:10.01 
mvrhel_laptop right17:10.10 
  so where exactly do I do this alignment. x is an opaque pointer isnt it17:10.55 
  or is this going to be handled in xps_png_malloc17:12.15 
Robin_Watts Ok, so do_png_print_page calls png_create_write_stuct.17:13.00 
mvrhel_laptop hmm we are in the png reading of xps17:13.35 
Robin_Watts How does it know where to allocate?17:13.55 
  mvrhel_laptop: Can you step through the call to png_create_write_struct to see what allocator it ends up calling ?17:16.07 
  and where :)17:16.11 
mvrhel_laptop yes. I am doing that know17:16.17 
  now17:16.19 
  hold on a sec17:16.23 
  so it looks like in xpspng.c we call png = png_create_read_struct_2(PNG_LIBPNG_VER_STRING,17:18.55 
  NULL, NULL, NULL,17:18.57 
  ctx->memory, xps_png_malloc, xps_png_free);17:18.58 
  which leads us into xps_png_malloc17:19.22 
  I am guessing some where in there I can do the alignment force?17:19.38 
  that gets me my png_ptr17:20.50 
  hmm no thats not right17:21.29 
Robin_Watts mvrhel_laptop: Ah, right.17:22.00 
mvrhel_laptop oh yes it is17:22.04 
Robin_Watts I'm looking at the usage of it in gdevpng, and that ends up calling malloc, so it is aligned.17:22.20 
  So, yes, I reckon you need to frob it in xps_png_malloc17:22.37 
  Now, we only have malloc and free, right? No realloc.17:23.05 
mvrhel_laptop correct17:23.26 
  png_create_png_struct,(png_const_charp user_png_ver, png_voidp error_ptr,17:23.43 
  png_error_ptr error_fn, png_error_ptr warn_fn, png_voidp mem_ptr,17:23.44 
  png_malloc_ptr malloc_fn, png_free_ptr free_fn),PNG_ALLOCATED)17:23.46 
Robin_Watts So alloc a bigger block, return the aligned pointer, and then 'magically' know how to get the original pointer back at free time.17:24.30 
  (by stashing some magic beans just before the aligned pointer).17:25.09 
mvrhel_laptop ok. sounds like a marvelous hack. 17:25.11 
Robin_Watts The alternative (making our allocators all return aligned blocks) is a lot more work, and would cost a lot more runtime memory.17:26.05 
mvrhel_laptop so seriously, I will I know if I need to back up17:26.07 
  s/I/how17:26.19 
Robin_Watts You want a 16 byte aligned block, right?17:26.36 
mvrhel_laptop well I know how to get the pointer to the alighned block17:26.55 
henrys won't ask why were are jumping in the first place.17:26.56 
mvrhel_laptop it is the free that I am worriied about17:27.05 
  henrys: it is to catch the errors in png lib17:27.19 
Robin_Watts The magic happens in the malloc.17:27.25 
mvrhel_laptop ok17:27.33 
Robin_Watts So whenever you allocate a block of size x, actually allocate a block of x+17.17:27.47 
mvrhel_laptop I see17:28.00 
henrys we did just fix this elsewhere, but please be sure it *absolutely* necessary before hacking something up17:28.16 
Robin_Watts Then you can do: void *block = malloc(size+17);17:28.21 
  void aligned_block = (int)((intptr_t)(block+16)) & ~15);17:29.16 
  That gives you an aligned block that's guaranteed to have at least 1 byte between block and aligned_block.17:29.40 
mvrhel_laptop ah ok. How will I know to do the free though on block17:29.53 
Robin_Watts Then you can do: ptrdiff_t d = (char *)aligned_block - (char *)block;17:30.17 
  That gives you the difference in bytes.17:30.24 
mvrhel_laptop right17:30.29 
  henrys: and spoil all this fun....17:30.48 
Robin_Watts Then... ((char *)(aligned_block))[-1] = (char)d;17:31.09 
  So you store the number of bytes you need to backtrack in the byte before the aligned block.17:31.28 
  So on the free, you can find that byte, backtrack, and Roberts your fathers male sibling.17:31.54 
mvrhel_laptop I have to admit, that is rather impressive.17:32.22 
Robin_Watts actually... only needs to be size+16.17:32.28 
  Hmm. block+16 in the above will fail cos block was a void *.17:33.16 
  Might be simpler to make block and aligned_block both be char *'s and just cast to void * when you return.17:33.32 
  You should be careful to ensure that malloc(0) can still return NULL and free(NULL) should not try to access memory position -1 :)17:34.28 
mvrhel_laptop ah17:34.35 
  ok. I am going to write all this out, to make sure I have it all understood. 17:35.14 
  Thank you for the clever trick17:35.30 
Robin_Watts No worries.17:35.49 
mvrhel_laptop There is going to need to be a comment or two. Otherwise henrys is going to look at this in a couple years and wonder what was being smoked that day17:36.15 
henrys stop the nonsens: If you would rather avoid the complexity of setjmp/longjmp issues, you can compile libpng with PNG_SETJMP_NOT_SUPPORTED, in which case errors will result in a call to PNG_ABORT() which defaults to abort().17:38.24 
Robin_Watts henrys: ick.17:39.25 
mvrhel_laptop I suppose in one case the image would just be missing and in the other case, the page/doc would not render17:40.18 
henrys mvrhel_laptop: no you'd lose the stream and create a big mess, likely hit a buffer overflow or something parsing garbage. expat is not safe.17:41.18 
  but who cares about security and bricking a printer, it's too icky17:41.46 
  I'm amazed the png developers would leave us with these choices, really sucks17:42.53 
Robin_Watts henrys: No, the png developers are entirely in the right here.17:43.17 
  It's reasonable to assume that malloc will return blocks 'appropriately aligned' for the platform.17:43.39 
henrys I don't want to argue about setjmp I'm not a believer sorry17:43.40 
Robin_Watts It's our fault that we're not honouring that.17:43.56 
mvrhel_laptop And is it a lot of work for us to have an allocator that provided some alignment?17:45.44 
  When needed17:45.51 
  i.e. some special gs_malloc_aligned17:46.11 
henrys what I meant was they shouldn't force us to use setjmp. The alignment thing is completely separater.17:46.12 
Robin_Watts mvrhel_laptop: Well, the code you're writing proves it's not.17:46.24 
mvrhel_laptop well it proves that I can't do it....17:46.42 
Robin_Watts henrys: Yeah, cos setjmp is such a strange beast. No one else uses that.17:46.42 
  <cough>jpeg</cough>17:46.54 
  <cough>mupdf</cough>17:46.57 
mvrhel_laptop it is a weird c macro you have to admit17:47.08 
Robin_Watts not a macro.17:47.50 
  but yes, it's strange, but standard.17:48.02 
mvrhel_laptop oh ok 17:48.09 
henrys Robin_Watts: I understand you've added it to mupdf and I know why. I am somewhat surprised tor8 did not veto but... anyway I'd like to avoid it when possible.17:49.14 
Robin_Watts http://pastebin.com/BZ4YNXkW17:49.48 
mvrhel_laptop ok that doesn't look bad really17:51.17 
Robin_Watts henrys: Look at the gs source code. Note that we throw away the ability of C to return values from functions, because everywhere we are passing error codes, which are 99% of the time 'OK!', and the rest of the time are unchecked.17:51.28 
mvrhel_laptop especially with a comment on what is going on 17:51.30 
Robin_Watts If you think gs is a bad example, look at the epage source tree. That also relies on passing errors.17:52.27 
henrys Robin_Watts: I understand the tradeoffs17:52.50 
mvrhel_laptop so how do we want to proceed? I am prepared to go ahead and add the alignment code. It really looks pretty reasonable18:00.48 
  bbiab. change of venue18:02.19 
Robin_Watts mvrhel_laptop: I think that's the only sensible solution.18:02.31 
henrys whatever you guys want to do, can Robin_Watts review for volatile problems?18:03.36 
chrisl I guess we'll steal whatever you end up with and do the same in gdevpng.c18:06.24 
Robin_Watts henrys: This change here will cause no setjmp issues of the type that people tend to work around with volatile.18:07.09 
  Actually... to pick up on something michael said... maybe we should turn this stuff into a generic aligned block handler for gs?18:10.00 
  gs_alloc_aligned(gs_memory_t *, size_t, int alignment)18:10.28 
  gs_free_aligned(gs_memory_t *, void *)18:10.47 
  Repeating for mvrhel:18:11.09 
  Actually... to pick up on something michael said... maybe we should turn this stuff into a generic aligned block handler for gs?18:11.11 
  gs_alloc_aligned(gs_memory_t *, size_t, int alignment)18:11.13 
  gs_free_aligned(gs_memory_t *, void *)18:11.14 
chrisl Is it something we need enough for that?18:11.16 
Robin_Watts chrisl: Well, we need it in 2 cases (xps and gdevpng)18:11.35 
  who's to say there won't be more in future?18:11.53 
mvrhel_laptop If we find in the future18:11.58 
  yes18:12.00 
chrisl And I'd prefer if we could avoid a gs_free_aligned() and just have it work with gs_free()18:12.05 
mvrhel_laptop that would be tricky I think18:12.29 
Robin_Watts Ah, well, that's hard.18:12.32 
  Cos that would require us to put the standard gs block header on it, right?18:12.51 
  Actually, that might be doable.18:13.05 
chrisl We already allocate at aligned offsets into chunks18:13.30 
  It just means using a variable alignment rather than the hard coded one18:14.14 
Robin_Watts oh, yuk.18:14.20 
chrisl Huh?18:14.40 
Robin_Watts to do it with gs_free, we'd need to make an 'aligned' memory handler.18:14.43 
chrisl No, I don't think so18:15.02 
  As I say, we already enforce alignment in the memory manager18:15.39 
Robin_Watts gs_free maps to gs_free_object18:15.44 
  gs_free_object(mem, ....) calls mem->procs.free_object18:16.06 
henrys has anyone quantified the memory increase with 16 byte alignment recently?18:16.19 
chrisl It would be considerable - we allocate a lot of small blocks18:16.41 
Robin_Watts So to cope with the unaligning, we need to have mem->procs.free_object know about the unaligning.18:16.58 
chrisl No we don't18:17.06 
Robin_Watts chrisl: go on...18:17.13 
chrisl As I said, we already allocate on aligned addresses, so we just do the same thing18:17.47 
Robin_Watts chrisl: For *some* gs_memory_t's we allocate on aligned addresses, not for all.18:18.48 
chrisl No, *all* - we have to18:19.14 
Robin_Watts Not sufficiently aligned.18:19.27 
chrisl They align to that required by the platform18:19.46 
Robin_Watts No, they don't.18:20.02 
  If they did, I could get the last hour of my life back.18:20.15 
chrisl They have to, otherwise SPARC/MIPS etc won't work....18:20.24 
Robin_Watts chrisl: All x64 builds (windows and linux) are specced to return 16 byte aligned addresses from malloc.18:21.16 
  We return 8 byte aligned addresses.18:21.27 
  Hence, we do NOT return sufficiently aligned blocks.18:21.55 
  We return blocks aligned enough for pointers.18:22.05 
  But not for 128 bit registers.18:22.38 
chrisl Right, but that is set by genarch (or in the platform pre-defined arch.h)18:22.42 
  So, we could just change windows-x64-msvc.h18:23.04 
mvrhel_laptop I have to step out for a bit. I am going to let you guys battle this one out18:23.18 
Robin_Watts We could. At the cost of increasing our small block overheads.18:23.30 
  as I said, right at the very beginning of this discussion.18:23.53 
chrisl But the point is, we *are* applying an alignment always, so if we leverage the same method, we can allow for variable alignment, without needing a special free method18:24.26 
Robin_Watts chrisl: Urm... no.18:25.00 
chrisl I don't have time to argue this now......18:25.50 
Robin_Watts We get the current alignment because either we allocate a block which comes at the required alignment already from the underlying allocator, OR we allocate a large chunk and return areas of that which happen to be aligned because every sublock is a multiple of the required alignment.18:27.12 
  Hence at no point does free have to say "ah, well, I need to adjust this pointer somehow before actually freeing it"18:27.40 
chrisl Yes, well, I was thinking we'd have it work like the alloc_imoveable method18:28.19 
Robin_Watts i_alloc_bytes_immovable ?18:29.55 
chrisl Yes18:30.58 
  Or each chunk contains only allocations at a specific alignment18:31.50 
  only contains18:32.01 
Robin_Watts That's a function of one gx_memory_t type, not all of them though.18:32.06 
  And I don't see what's special about i_alloc_bytes_immovable.18:32.23 
  And if we want chunks to contain only allocations at a specific alignment, doesn't that require a new gx_memory_t ?18:32.48 
chrisl I'm really out of time - can we pick this up on Monday?18:33.48 
Robin_Watts chrisl: sure.18:34.11 
chrisl Have a good weekend!18:34.18 
Robin_Watts You too.18:34.21 
henrys chrisl, Robin_Watts : just so you guys know mvrhel_laptop is using the chunk allocator for his problem.18:34.27 
Robin_Watts Does the chunk allocator guarantee 16 byte alignment ?18:34.49 
  And, talk about a sledgehammer to crack a nut.18:35.06 
henrys haven't lookes18:35.07 
  haven't looked18:35.15 
Robin_Watts It would not surprise me to find that the chunk allocator allocates at pointer size granularity and alignment.18:35.47 
  so it won't solve the problem.18:35.53 
henrys I just didn't want you to fix in ialloc and puzzle over why the change wasn't working18:35.57 
Robin_Watts Oh, right, you mean gxps is already using the chunk allocator.18:36.22 
henrys Robin_Watts: yes18:36.39 
Robin_Watts Honestly the simplest solution is a gx_alloc_aligned and gx_free_unaligned.18:36.58 
  That's easy for Michael to use, and if chrisl can come up with a simple solution so we can remove gx_free_unaligned later, great.18:37.22 
  We only need to align a single block after all.18:39.38 
mvrhel_laptop ok sounds like a plan19:32.20 
  so I feel like it should be gx_alloc_aligned and gx_free_aligned rather than gx_alloc_aligned and gx_free_unaligned20:56.24 
  I do understand that the free does the unaligned, but the name non-symmetry bothers me20:57.06 
  Robin_Watts: you are probably gone for the day. henrys, any thoughts?20:57.21 
henrys mvrhel_laptop: I think I missed something, whey do we need a free aligned at all whey not the regular free?21:56.45 
  s/whey/why21:56.51 
  mvrhel_laptop: you can probably leave this to chrisl sounded like he wanted to weigh in on it more...22:05.07 
mvrhel_laptop henrys: ok, I will pass the bug off to him then22:35.52 
  henrys: essentially we need a special free, since the pointer we get given to us to free may or may not need to be offset by a byte (its not the "real" allocation)22:36.39 
henrys mvrhel_laptop: okay I thought you just needed a special allocation call that would return something on 16 byte boundary, the header is just behind the allocation as always.23:05.14 
  mvrhel_laptop: beating the dead horse here is why I don't like setjmp, a bug reproduced using the same api we are using but in imagemagick http://patrakov.blogspot.com/2009/07/dangers-of-setjmplongjmp.html. I understand the it cleaner to have a nice mechanism for errors but the choice is not black and white. There are clear downsides with setjmp.23:25.24 
mvrhel_laptop henrys: doing that is going to be beyond my comfort zone and best left to someone else23:56.06 
Robin_Watts mvrhel_laptop: Sorry, I missed your earlier stuff.23:57.08 
  It should absolutely be alloc_ and free_ aligned. If I typed it wrongly before it was a typo.23:57.30 
 Forward 1 day (to 2015/11/14)>>> 
ghostscript.com
Search: