IRC Logs

Log of #ghostscript at irc.freenode.net.

Search:
 <<<Back 1 day (to 2016/03/20)20160321 
tor8 Robin_Watts: argh. I've figured out why annotations disappeared.11:25.51 
  transformation madness...11:25.57 
  I removed the beginPage call, and moved the initial 'cm' that sets up the page space to match mupdf's top-down coord system11:26.39 
  but for annots, the initial 'cm' was never emitted, so this flips the drawing outside the bbox :(11:27.16 
kens tor8 welcoem to my world :-)11:27.39 
Robin_Watts tor8: Ah.13:31.22 
tor8 Robin_Watts: double argh!14:44.59 
  java dynamically loads libfreetype from the system14:45.11 
  and that collides with our built-in thirdparty freetype which is also loaded as a shared library14:45.27 
Robin_Watts tor8: urgh.15:03.22 
  Can we insert a shim header to define all the FT_ stuff as MUFT_ ?15:04.02 
tor8 I'm thinking we could just use the system freetype for desktop java builds.15:06.41 
  the android build builds the native so using its ndk-build right, so the platform/java/Makefile is only used for desktop?15:10.16 
  ... and possibly to build the .jar?15:10.27 
  or is it used for more?15:10.37 
  Robin_Watts: the freetype include system is a nightmare ... and it uses its own module system, etc. so I think trying to mess with redefining as MUFT_ is going to be more trouble than its worth15:13.00 
  it's, even15:13.08 
  Robin_Watts: a bunch of commits on tor/master for review (ignore the URW font one)15:14.05 
Robin_Watts The android ndk-build is currently used for all the C stuff (libs, core, and jni layer)15:15.57 
  but fredross-perry is fiddling with this stuff.15:16.10 
  The "Fix unintialized value errors" one. Those are all spurious compiler warnings :(15:48.15 
tor8 Robin_Watts: actually, no.15:48.30 
  in the case of an exception stack overflow, they are real uninitialized errors15:48.41 
  because then the fz_push_try part will fail, we'll drop into catch and then the jni_rethrow (which just sets the exception flag), and then return the unitilialized value to java15:49.22 
Robin_Watts Ah, yes, good catch.15:49.54 
  Though, hopefully java doesn't do anything with those values as there is an exception flagged.15:50.21 
tor8 and oddly enough the compiler didn't spot it when we had the old formulation of the try macro15:50.33 
  I suspect it turns off a bunch of warnings when it sees a setjmp rather than get confused :(15:50.47 
  so having the first if-test separate from the if(setjmp) has shown a number of possible bugs due to more warnings appearing15:51.16 
Robin_Watts tor8: I'm wondering if the new formulation of setjmp is what is causing the slowdown on the pi.15:51.26 
  I had the idea that we could do:15:51.37 
  { jmpbuf_t *FZ_BUF; if ((FZ_BUF = fz_try_push()) != NULL) { if (setjmp(FZ_BUF) { ...15:52.30 
  Which we might hope would compile down to something closer to what we had before.15:52.59 
tor8 that's troublesome because of the type of jmp_buf is defined as an array type15:53.15 
  the only way to do it sanely is to use void *FZ_BUF.15:53.28 
  it's an array type of an implementation-defined and hidden magic type :(15:54.05 
Robin_Watts Whatever type was returned from fz_try_push before, I mean.15:54.11 
  Your commits lgtm then.15:54.21 
tor8 I believe that was void* :)15:54.23 
  Robin_Watts: thanks.15:54.27 
  I'm confused to how that would generate any better code than what we do now... it's still two branches15:55.05 
Robin_Watts We called setjmp with a void?15:55.09 
tor8 void pointer15:55.15 
Robin_Watts We can avoid the extra ctx->error->top dereference.15:55.35 
tor8 ah, no, we returned a fz_errro_stack_slot that had a jmpbuf in it15:55.45 
Robin_Watts Ah, right. That then.15:55.58 
  but let's wait to see where the time has gone.15:56.08 
tor8 I'd be annoyed if it turns out to be this15:56.25 
Robin_Watts The other possibility is the glyph plotting changes.15:56.34 
  but I can't really see how I can have made it slower (famous last words)15:56.54 
tor8 yeah, that was a mechanical change (the #include voodoo shouldn't change implementations) that only added cmyk functions right?15:57.35 
Robin_Watts tor8: Yes.15:59.32 
  and added optimised rgb/greyscale functions for glyph plotting.15:59.50 
tor8 does the pi have any fancy branch prediction?16:00.12 
Robin_Watts I believe ray did timings at 4b6c0ab659b3feb5f8e3683e393d61ccfdc09c3c before.16:00.15 
  tor8: Nope.16:00.21 
  Hmm. building noto.o kills the pi.16:03.36 
tor8 Robin_Watts: $(OUT)/fitz/noto.o : $(FONT_GEN)16:07.55 
  + $(CC) $(CFLAGS) -O0 -o $@ -c $<16:07.55 
  it might help if we turn off optimizations completely when building noto.o16:08.21 
rayjj tor8: When building on the Pi, I had to use NOTO_SMALL16:11.15 
  but building with -O0 would probably speed it up as well16:11.41 
  Robin_Watts: I did apt-get upgrade, but my gcc is still at 4.6.3 :-(16:12.43 
tor8 + $(CC) $(CFLAGS) -O0 -o $@ -c source/fitz/noto.c16:12.46 
Robin_Watts rayjj: You may need to dist-upgrade16:13.04 
rayjj tor8: yeah, I saw that on the logs16:13.05 
  Robin_Watts: OK, trying that...16:14.05 
tor8 rayjj: you may also want to try using clang rather than gcc16:14.17 
rayjj the upgrade loaded 77Mb, the dist-upgrade after that calculated needing 243Mb16:14.53 
Robin_Watts I'm seeing the speed difference on windows even more than on the pi.16:15.00 
rayjj tor8: I'll try that after re-running with gcc upraded16:15.26 
Robin_Watts Waas 16.5 seconds. Now is 23 seconds.16:15.28 
rayjj Robin_Watts: slowing down ?16:15.39 
Robin_Watts Yes.16:15.47 
rayjj :-(16:15.51 
Robin_Watts rayjj: bisecting will be MUCH faster on the PC :)16:16.06 
rayjj no kidding 16:16.16 
  Robin_Watts: and you are correct about the version I tested with previously being 4b6c0ab659b3feb5f8e3683e393d61ccfdc09c3c16:18.34 
Robin_Watts Ok, so it was a0960a716:41.24 
  Presumably cos I close and reopen the output file now. Let me see if I can work around that.16:42.31 
rayjj Robin_Watts: close and re-open how often ?16:46.42 
  (more than once per page?)16:46.53 
Robin_Watts once a page. Seems an odd result. Let me double check it.16:46.57 
rayjj I *really* need to get a bigger sd card on the Pi. The Pi 2 I got has a 8Gb, but the older Pi B only has a 4Gb. With multiple trees of GS and mupdf, it ran out of space trying to dist-upgrade :-(16:51.54 
  I think I'll get 32Gb chips for both.16:52.16 
Robin_Watts rayjj: Yeah, I got a 32gb card off amazon on saturday for exactly that reason :)16:52.51 
  ok. Previous best time was 16.6 seconds.17:19.40 
  I just had a 15.8 second one.17:19.54 
  with the latest code + a fix.17:20.10 
rayjj hmm... even after dist-upgrade my gcc is _still_ 4.6.317:22.35 
  Robin_Watts: which file, etc. and which machine ?17:23.14 
Robin_Watts rayjj: Windows.17:23.38 
  My point is that I'm back down to the proper timings rather than being up in the 23s.17:23.57 
rayjj oooh, that is quite a difference17:25.04 
  and *MUCH* worse than I saw on the Pi17:25.23 
  Robin_Watts: BTW, the CMYK 8-bit (contone) numbers for mupdf look reasonable now, some faster than gs, some slower. As with the other modes, gs is still much faster on the PLRM17:33.37 
  Robin_Watts: and the 'mono' (1-bit) is faster than it was with 4b6c0a (PLRM is up by 7%, although it is still only 30% of gs throughput on that file)17:36.43 
Robin_Watts tor8: (For the logs) 1 commit on robin/master to get ray his speed back.18:04.24 
  Hmm. On windows, doing the first 100 pages of pdf_reference17.pdf at 600dpi, I still get 1/3 of the time wasted in the file writing code even when going to NULL.18:07.54 
lotsoftuna Hi, I'm wondering if it's possible to enable constant page scrolling in mupdf18:14.52 
  as in, to have one page flow to the next, instead of having the previous page disappear18:15.09 
Robin_Watts On Android, presumably.18:15.09 
lotsoftuna How about desktop?18:15.15 
Robin_Watts ok, let me back up a bit.18:15.36 
  MuPDF has a core C library that does PDF opening/rendering/manipulation etc.18:15.52 
lotsoftuna mhm18:16.06 
Robin_Watts That bit will happily run multiple pages at a time.18:16.10 
lotsoftuna so..18:16.56 
Robin_Watts The simple example viewers that we've done don't make use of that flexibility though.18:18.46 
lotsoftuna Ah18:18.52 
rayjj Robin_Watts: oops. I did the dist-upgrade and it did eat up almost as much space as it estimated, so now I don't have enough left to build mupdf :-(18:19.06 
lotsoftuna Would it be possible to... or is this the right process, to recompile it with an option like that?18:19.09 
rayjj at least until I get the 32Gb chip18:19.27 
Robin_Watts (Sorry, am on hold on phone...)18:19.58 
  ok, back.18:21.09 
  lotsoftuna: You could try using gsview.18:21.35 
  That's a viewer based on mupdf that does continuous page scrolling.18:21.51 
mvrhel_laptop ok. I seem to have the page grafting stuff working now18:21.54 
Robin_Watts nice one.18:22.01 
mvrhel_laptop error leaks fixed. So I like how memento does the little stack trace print out18:22.15 
Robin_Watts mvrhel_laptop: Yeah, pleased with that :)18:22.44 
mvrhel_laptop so I added a mutool graft that lets you specify a source document with a page number and a destination document with a page number. It will then copy the page object and all the dependencies from one to the other. pdf_insert_page is used to do the actual insertion of the created object. The only trick is that the PDF_NAME_Parent entry in the page dictionary has to be deleted from the...18:26.09 
  ...source page dictionary prior to the copy. This avoids the copying of the entire page tree from the source18:26.10 
  I wonder if I we should look into issues with internal links etc that are going to occur18:26.45 
  or contents too18:26.52 
lotsoftuna ah geez18:27.35 
Robin_Watts mvrhel_laptop: I suspect it would be safer if rather than copying the page dictionary, we make an empty dictionary, and then copy all the known entries from the page dictionary into it.18:27.38 
lotsoftuna I gotta compile gsview from aur :/18:27.43 
Robin_Watts lotsoftuna: For what OS?18:27.52 
mvrhel_laptop Robin_Watts: actually a new dictionary is created18:28.02 
lotsoftuna Arch >.>18:28.03 
Robin_Watts ArchLinux, right.18:28.18 
lotsoftuna oh geez, Ghostscript is quite large18:28.31 
Robin_Watts But you like compiling things, right? otherwise you wouldn't be a linux user.18:28.35 
lotsoftuna Yeah18:28.47 
mvrhel_laptop Robin_Watts: as we encounter new dictionaries or arrays we create new ones that are populated 18:28.55 
lotsoftuna I think I'll stick with MuPDF for now, but I'll note that gsview does continuous scrolling18:28.59 
  thanks :)18:29.00 
Robin_Watts np.18:29.12 
  mvrhel_laptop: I mean, instead of taking the page dictionary P from file 1 and putting it into file 2, we should make a new dict Q.18:29.49 
mvrhel_laptop That is what I do18:30.07 
  I create a new Dict Q18:30.20 
Robin_Watts and then put Q.blah = P.blah for all blah in X, where X is the set of names that we know we need to copy over (and that are safe to copy over) 18:30.45 
mvrhel_laptop yes. that is what I do18:31.17 
  you can look at the code in my branch18:31.24 
  pdf-graft.c18:31.32 
  it is a recursive calling function that creates new arrays and dicts as it goes into the object18:32.15 
Robin_Watts So, if in the original document, I have a page tree entry P, that contains a /RobinRules entry, would that get copied across?18:33.19 
mvrhel_laptop yes18:34.35 
Robin_Watts Right, that's what I'm saying shouldn't happen.18:34.46 
  We should only copy *known* entries across at the top level.18:35.01 
mvrhel_laptop Robin_Watts: I am not sure I am quite following you. Originally I was creating new objects for everything. And tor told me not to create new objects for names, null, ints, reals, and strings18:36.40 
  so I am not only creating new objects for arrays, and dicts18:36.58 
  s/not/now18:37.03 
Robin_Watts We'd copy, Type, LastModified, Resources, MediaBox, CropBox, BleedBox, TrimBox, ArtBox, BoxColorInfo, Contents, Rotate, Group, Thumb, B, Dur, Trans, Annots, AA, Metadata, PieceInfo, ID, PZ, SeparationInfo, Tabs, TemplateInstantiated, PresSteps, UserUnit, VP.18:38.06 
mvrhel_laptop everything is copied18:38.21 
Robin_Watts mvrhel_laptop: It sounds like you've written a generic 'deep copy' routine.18:38.28 
  And that's great, and a useful thing to have.18:38.37 
mvrhel_laptop I suppose I could strip out the unknown name entries18:38.53 
  if that is what you want18:38.56 
Robin_Watts BUT for copying a page tree P to put into a new document, I reckon you should not deep copy P.18:39.16 
  You should do Q = { deep_copy(P.Type), deep_copy(P.LastModified), deep_copy(P.Resources), .... }18:39.43 
malc_ Robin_Watts: your inner haskellite is trying to break free18:40.24 
Robin_Watts Which would be the same net effect as deep copying P and then deleting the unwanted entries, yes.18:40.56 
  I also wonder whether deep_copy should have special knowledge about Keys with 'Parent' in the names.18:41.37 
  :)18:41.40 
mvrhel_laptop Robin_Watts: I am not saying that we delete unwanted entries. Just avoid doing the copy of them.18:41.41 
  Robin_Watts: no 18:41.45 
  I had that earlier18:41.50 
  and tor told me to take at it18:41.57 
  it was up to the caller to do any "special" stuff18:42.08 
Robin_Watts ha. OK.18:42.10 
mvrhel_laptop s/at it/ it out/18:42.22 
Robin_Watts So, I think we're at the same place now.18:42.31 
mvrhel_laptop I will let him look this over and we call all chat about it tomorrow18:42.58 
  you can have a look to if you have a chance18:43.07 
Robin_Watts deep_copy P but restricted to just the known set of children.18:43.08 
  I will try to.18:43.11 
mvrhel_laptop Perhaps the restriction of not copying unknown children should be an option18:43.56 
  there may be applications where we want to do it18:44.12 
Robin_Watts mvrhel_laptop: We could recast deep_copy(P) to be deep_copy(P,X)18:45.15 
  which would mean deep_copy P. If X is non NULL, restrict the copy to the names in X.18:46.05 
mvrhel_laptop yes18:46.23 
lotsoftuna ...18:46.59 
  So now my pages fade in/out...18:47.05 
Robin_Watts malc_: It's not haskell as such. I was tortured in a mathematics department for 3 years, so sometimes I have painful flashbacks.18:47.07 
lotsoftuna I don't know what button I pressed...18:47.08 
  also they seem to be changing on their own18:47.20 
  like a slideshow18:47.27 
Robin_Watts Sounds like you hit presentation mode.18:47.28 
lotsoftuna waa18:47.31 
  oh, it's just p18:47.44 
  lmao18:47.44 
  why isn't that documented :S18:47.53 
Robin_Watts It is.18:48.01 
malc_ Robin_Watts: I only lasted 2...18:51.30 
Robin_Watts tor8: (For the logs) I think the noto.o line in the Makefile is wrong, cos it never actually passes noto.c to the C compiler.20:18.47 
tor8 mvrhel_laptop: Robin_Watts: I want to keep the pdf_graft_object simple and predictable without magic stuff happening, hence me wishing that the user would deep copy only the stuff he wants.20:23.54 
Robin_Watts tor8: Right.20:24.10 
tor8 in page_graft, do what robin suggests and create a new dictionary and deep copy the known entries to that dictionary, then insert the new dictionary into the page tree20:24.26 
  the current page_graft code *destroys* the source document by deleting the parent entry20:24.51 
  new_page_obj = pdf_new_dict(); pdf_dict_put(new_page_obj, "Resources", pdf_graft_object(pdf_dict_get(pageref, "Resources")));20:25.58 
  etc.20:26.03 
  safer to copy what we know, than to filter out what we can only hope is complete20:26.44 
  new versions of PDF may add other references that we don't know about20:27.00 
  Robin_Watts: yes, the noto.o line I pasted was wrong. I did paste an updated line a bit later, that you may have missed.20:28.35 
Robin_Watts I did miss it.20:28.42 
  Another fix robin/master20:33.47 
tor8 I don't like adding another branch to each fz_write call :(20:46.53 
  that feels like it should just make every other non-benchmark use slower20:47.52 
Robin_Watts tor8: The alternative is to force all fz_write callers to check for NULL20:50.28 
tor8 the null device seemed like a decent compromise20:50.50 
  if you don't want output, don't call the function to create it...20:51.08 
Robin_Watts tor8: OK... I could have a: fz_is_output_null() function that returns true if it's a NULL device then?20:51.24 
tor8 this looks like benchmark wankery, with no real life applications20:51.40 
Robin_Watts excellent use of the term wankery, BTW.20:51.56 
tor8 the other two commits LGTM (l2factor and avoiding reopening)20:52.27 
Robin_Watts tor8: OK, I'll take another run at the NULL one.20:52.52 
tor8 can you benchmark actual (inefficiently written) output that writes stuff in smallish chunks?20:53.42 
Robin_Watts tor8: pgm is an example.20:54.13 
tor8 it may be that the extra if test doesn't hurt us measurably, in which case I can live with it. but I want numbers for real files not just dumping to /dev/null which is really just cheating :)20:54.16 
  the benchmark will be faster, but is it fair? does what we compare with also bail out early of writing data.20:55.08 
Robin_Watts gs? Yes.20:55.22 
  Essentially the point is that we generate data into a buffer, and then write it out.20:55.41 
  printer manufacturers will use the data from the buffer.20:55.50 
  Hence they aren't interested in the time taken to write the buffer to disc.20:56.01 
  We are making the same decision/optimisation here that gs does.20:56.23 
  OK, so on my windows PC: platform/win32/Release/mudraw.exe -o nul: -r600 -F pam -c gray pdf_reference17.pdf 1-30020:59.44 
tor8 do we care about formatting the output as an image file? or just the rendering time?21:00.03 
Robin_Watts Just the rendering time.21:00.15 
tor8 -F benchmark ?21:00.38 
  like the default OUT_NONE but trigger a render21:00.57 
Robin_Watts With the nul: special case removed so we actually call the output functions... 10 pages takes 10.250 seconds.21:02.16 
malc_ Robin_Watts: starting with page 1131 is fairer i think... lots of pretty pictures ahead21:02.31 
Robin_Watts malc_: On the contrary. I am looking for pages where rendering time is minimal, so as to get the most emphasis on the output times.21:03.30 
  tor8: With the if (out) removed from fz_write...21:04.00 
  10.352s :)21:04.12 
malc_ Robin_Watts: the range you cited 0-300 contains quite a lot of vectors graphics (44, 91, etc)21:04.33 
  so uhm21:04.35 
Robin_Watts so the if (out) in fz_write is actually saving us time if you believe that.21:04.40 
tor8 repeat the job 100 times :)21:04.42 
malc_ s;vectors;vector21:04.44 
Robin_Watts malc_: I cut it back to just 50 pages.21:04.53 
  tor8: I've repeated the timings 5 times each.21:05.01 
  As far as I can tell the if (out) doesn't cost us anything.21:05.19 
tor8 Robin_Watts: you're probably right.21:05.31 
  Robin_Watts: still, -F benchmark might be worth doing anyway21:05.41 
Robin_Watts and that's the only real objection to the proposed solution, I think.21:05.44 
  tor8: Except we need to say 1bpp or 8bpp as well as colorspace.21:06.07 
tor8 Robin_Watts: we should make sure all fz_output functions can handle null as well21:06.13 
Robin_Watts and currently that's done using -F21:06.14 
  tor8: I did :)21:06.18 
  fz_write is the only one that wasn't testing anyway.21:06.31 
tor8 Robin_Watts: ah. well in that case :)21:06.56 
  Robin_Watts: we have -c for colorspace, mono for 1-bit grayscale21:07.05 
  might want cmykmono for 1-bit cmyk21:07.14 
Robin_Watts we have no way to say... yeah, that.21:07.20 
  if you're happy, I'll push this stuff, and then ray can retime at his leisure.21:07.48 
tor8 Robin_Watts: go for it.21:08.00 
Robin_Watts I want to do some profiles to see why gs beats us for text only output.21:08.07 
  tor8: Thanks.21:08.40 
tor8 Robin_Watts: the runlength coded font cache, the branching may be slower than brute force bit pushing?21:09.16 
Robin_Watts tor8: yeah. It may be that we can write an optimised glyph cacher/plotter for 0 bits of AA.21:09.58 
Robin_Watts is called for food.21:10.04 
rayjj Robin_Watts: tor: We used to have a -b # option to let us control the output bit depth. My objection to overloading the -F is that currently (AFAIK) the only way to get 1-bit mono is 23:16.24 
  -F pbm23:16.34 
  and we've had interest from customers in 2-bit and 4-bit output (with or without AA), so IMHO, keeping bit depth as a separate parameter makes life easier. 23:18.47 
  Robin_Watts: tor: I have no problem with using something different to -o /dev/null to just render buffers and skip any writing. In fact, if you implement something reasonable (that doesn't conflict with some gs parameter), I'd be glad to switch gs over23:20.22 
  but good luck on side-stepping the myriad gs command line parameters ;-)23:20.48 
  the thing is that MANY important gs customers take data directly from our buffers23:21.33 
  (or the clist, for one top-ten customer)23:22.12 
  obviously, mupdf isn't going to provide clist output, but since that just goes to their "GRS" graphics library calls now, maybe someday ???23:23.17 
  BTW, can we *PLEASE* alphabetize the options in the usage ??? If you don't object, I'll be glad to do it23:24.58 
 Forward 1 day (to 2016/03/22)>>> 
ghostscript.com
Search: