| | <<<Back 1 day (to 2016/03/21) | 20160322 |
| Robin_Watts | With MuPDF we *always* render to 8bpp contone. | 00:00.41 |
| | And anything else is a post processing step. | 00:00.49 |
| | I'm not sure about alphabetizing the options. I'd rather group them logically. | 00:01.39 |
| | But they could certainly be better than they are now, yes. | 00:02.30 |
| rayjj | Robin_Watts: I have a clean pull of mupdf and cannot build on the Pi -- getting a *bunch* of undefined references: murun.c:(.text+0x91c): undefined reference to `fz_lookup_base14_font' | 00:15.15 |
| | I did git submodule update --init | 00:15.49 |
| Robin_Watts | rayjj: Yeah. Did you use tor's updated makefile line from earlier? :) | 00:18.04 |
| rayjj | Robin_Watts: I have added: | 00:19.10 |
| | $(OUT)/fitz/noto.o : $(FONT_GEN) | 00:19.11 |
| | + $(CC) $(CFLAGS) -O0 -o $@ -c $< | 00:19.13 |
| Robin_Watts | In front of $(FONT_GEN) you need to add source/fitz/noto.c | 00:19.37 |
| rayjj | Now I get: Assembler messages: | 00:22.03 |
| | Fatal error: can't create build/release/fitz/noto.o: No such file or directory | 00:22.04 |
| Robin_Watts | You didn't actually add the '+' did you? | 00:22.31 |
| rayjj | no, that's from my git diff | 00:22.49 |
| | duh | 00:23.24 |
| Robin_Watts | I've killed my pi again. | 00:24.00 |
| rayjj | so, now my git diff has: | 00:24.12 |
| | -$(OUT)/fitz/noto.o : $(FONT_GEN) | 00:24.13 |
| | +$(OUT)/fitz/noto.o : source/fitz/noto.c $(FONT_GEN) | 00:24.15 |
| | + $(CC) $(CFLAGS) -O0 -o $@ -c $< | 00:24.16 |
| Robin_Watts | even with 1.5Gig swap, I can't compile noto.c with anything other that -DTOFO -DNOTO_SMALL -DNOCJK | 00:24.49 |
| | What's the actual build command it ends up running? | 00:25.21 |
| rayjj | Robin_Watts: AFAIK, NOTO_SMALL takes care of -DTOFU and -dNOCJK (or it used to) | 00:25.46 |
| | Robin_Watts: and for some reason, it is still running :-/ | 00:26.06 |
| | hmm.. I have: 28896 28894 0 0 0 0 17:21 pts/0 00:00:00 [as] <defunct> | 00:27.04 |
| | as a process that won't go away | 00:27.16 |
| Robin_Watts | Odd. | 00:27.18 |
| rayjj | I may have to reboot | 00:27.25 |
| Robin_Watts | I'm off to bed, going to leave it building. | 00:29.54 |
| rayjj | hmm... I had done: make clean | 00:29.58 |
| Robin_Watts | How much memory does your pi have? | 00:30.00 |
| rayjj | and then I did mkdir build/release/fitz and it is building without the strange message from as | 00:30.26 |
| Robin_Watts | They doubled the memory at some point. | 00:30.30 |
| rayjj | Robin_Watts: standard Pi Model B -- 512 Mb | 00:30.44 |
| Robin_Watts | Right. THe original model b's had 256Meg, I think. | 00:31.10 |
| | Then they doubled it. | 00:31.15 |
| | I suspect I have the original one. | 00:31.19 |
| rayjj | but note that I have (locally) patched my noto.c to have +#define NOTO_SMALL | 00:31.44 |
| | It seems to be going along OK now that I have (manually) created the build/release/fitz directory | 00:33.04 |
| | Robin_Watts: g'nite -- I | 00:33.18 |
| | I will submit a patch if it is just not making the directory | 00:33.41 |
| | OK, so from the make log, it tries to compile noto.c (with Tor's patch for $(CC) ... *before* it does all of the MKDIR actions | 00:55.58 |
| | so missing a dependency. I have to go get dinner for the family. bbl8r | 00:57.40 |
| tor8 | Robin_Watts: to build noto.o on pi, it may help to add -DHAVE_INCBIN to speed up compiles | 08:56.32 |
| | we normally only use .incbin on debug builds; on release builds we use a not incbin approach in order to allow the compiler to strip out the unused fonts | 08:57.11 |
| malc_ | tor8: objcopy -I binary font -O elf[32|64] font.o doesn't work? | 09:11.41 |
| tor8 | malc_: doesn't work on windows or macosx | 09:20.07 |
| | nor on non-gnu tool chains | 09:20.23 |
| malc_ | tor8: eh? | 09:20.37 |
| | i can only read it as objcopy is not present on windows | 09:20.47 |
| | which is fair | 09:20.54 |
| | but the rest, uhm | 09:20.59 |
| tor8 | we need a fallback anyway, even if using objcopy would be a *lot* faster | 09:21.19 |
| malc_ | it is a lot faster. as for fallback.. yeah if extreme form of portability is required then uh.. yeah | 09:24.12 |
| tor8 | malc_: ld -r -b binary -o font.o resources/fonts/urw/*.cff is even easier | 09:27.05 |
| malc_ | tor8: det er bra | 09:34.43 |
| brock | Question: I have a PDF that contains a Link to a Video, how do I capture the 'click' ? | 11:40.22 |
| | FYI. This is in the Android version of MuPDF | 11:40.57 |
| kens | Your application captures the 'click' surely | 11:42.33 |
| | Determining if a mouse event is within the bounding boc of an annotatoin requires you to check the bounding boxes of all annotaions on the page I would think. NB I am not a MuPDF developer | 11:43.43 |
| jogux | brock: You may be looking for 'visitExternal' in MuPDFReaderView.java | 11:45.17 |
| brock | Jogux, I will look again at visitExternal, I might be on a older version that does not cleanly expose notifications, I will pull the latest and try it. Thanks | 11:46.44 |
| tor8 | Robin_Watts: rayjj: some changes on tor/master that should speed up font compilation | 12:29.34 |
| Robin_Watts | tor8: HAVE_INCBIN did indeed solve the problem. | 12:35.36 |
| | I've just got a profile out of the pi on the first 100 pages of pdf-reference at 600dpi. | 12:35.56 |
| | 68.5% of runtime is spent in fz_paint_glyph | 12:36.11 |
| tor8 | Robin_Watts: I have to split them into separate files though, so the TOFU etc defines let the linker omit the unused object files | 12:36.19 |
| Robin_Watts | 26.5% in the actual function, 42% in memset. | 12:36.40 |
| | which is interesting, cos that function NEVER CALLS MEMSET. | 12:37.34 |
| tor8 | sounds like an interesting problem :) | 12:42.19 |
| malc_ | Robin_Watts: sampling profiler? | 12:53.36 |
| Mulover | Hey there. Anybody here can tell me, why my recently commited bug-report concerning "mutool: source/fitz/image.c:352: fz_image_get_pixmap: Assertion `l2factor_remaining >= 0 && l2factor_remaining < 8' failed." was removed? I have open and unconfirmed bugs over an year old, which is one thing. But why do reports get just removed without any feedback? | 12:59.49 |
| Robin_Watts | malc_: google perf tools, yes. | 13:00.02 |
| | Mulover: Urm... I fixed that bug yesterday ? | 13:00.19 |
| | It wasn't removed, it was closed. | 13:00.35 |
| malc_ | Robin_Watts: i wouldn't trust it too much then, cachegrind would produce much more believable results | 13:00.36 |
| Robin_Watts | Bug 696668 | 13:01.02 |
| Mulover | Great to hear! :-) I just thought that some mail would tell me about. So soryy, my fault! | 13:01.05 |
| Robin_Watts | Mulover: If you were CC'd on the bug, you should have had the email. | 13:01.27 |
| | malc_: I have some experience of profilers :) | 13:01.35 |
| malc_ | Robin_Watts: your "interesting" "memset" remark fooled me into thinking that you are profiling noob | 13:02.24 |
| tor8 | malc_: cachegrind is useful, but IIRC, it treats all instructions as taking the same number of cycles | 13:07.04 |
| Mulover | So even the missing mail was my fault :-( Sorry for disturbing ;-) | 13:07.29 |
| malc_ | tor8: erm.. no | 13:10.04 |
| Robin_Watts | Ok, so having said that I understand profilers, I was misreading this ones results. | 13:23.00 |
| | The 42% of time spent in memset is not as part of fz_paint_glyph. | 13:23.28 |
| malc_ | Robin_Watts: perf? | 13:25.51 |
| | vtune? | 13:26.02 |
| Robin_Watts | google-perf-tools | 13:26.13 |
| malc_ | oh yeah | 13:26.24 |
| Robin_Watts | On windows I use Very Sleepy. | 13:26.28 |
| malc_ | my brain is disintegrating as we speak | 13:26.34 |
| Robin_Watts | I've used oprofile in the past, but that seems to not be supported on the pi. And it now relies on perf, and perf isn't available for the lastest Raspbian. | 13:28.06 |
| | so google-perf-tools it is. | 13:28.14 |
| | Which looks an awful lot like the profiler I wrote for the NDS :) | 13:28.38 |
| malc_ | Robin_Watts: Nintendo Dual Screen? :) | 13:30.24 |
| Robin_Watts | Yeah. | 13:36.47 |
| malc_ | reminds me that i must buy/play spirit tracks | 13:37.49 |
| Robin_Watts | tor8: That font commit looks vaguely plausible to me. | 13:40.58 |
| | The only thing that struck me is the change from unsigned char to char throughout. | 13:41.13 |
| tor8 | Robin_Watts: I've been reading too much about undefined behaviours and crappy optimizing compilers... | 13:44.13 |
| Robin_Watts | It bothers me that memset is a higher CPU user than fz_paint_glyph. Clearing memory should be cheaper wholesale than setting it retail. | 13:44.14 |
| tor8 | and aliasing rules, etc | 13:44.19 |
| Robin_Watts | unsigned char vs char should give a speed up? | 13:44.39 |
| | or the separation of the fonts into separate files should enable unused ones to be dropped? | 13:45.17 |
| tor8 | it should not matter, but non-signed char is a better type for opaque 'blob of data' | 13:45.20 |
| Robin_Watts | (The latter makes perfect sense) | 13:45.27 |
| tor8 | the separation of fonts into separate files is the main purpose of the commit, to allow the linker to drop unused ones | 13:45.43 |
| malc_ | tor8: void * is better :) | 13:45.46 |
| Robin_Watts | tor8: I can't see that that matters really, but as long as we are consistent. | 13:45.50 |
| | malc_: Not when you want to have a size associated with it. | 13:46.09 |
| tor8 | Robin_Watts: so I'm going to spend some time post release trying to fix all the signed/unsigned UB insanities | 13:46.12 |
| malc_ | Robin_Watts: huh? | 13:46.17 |
| Robin_Watts | These are "blocks of data of a given size". | 13:46.33 |
| | As such they are blocks of n bytes. | 13:46.43 |
| | so char (or unsigned char or uint8_t or whatever) is the right type. | 13:47.03 |
| malc_ | Robin_Watts: and who pray tell guarantees you that char is a byte? | 13:47.05 |
| Robin_Watts | void * is not. | 13:47.08 |
| malc_ | Robin_Watts: uint8_t - maybe | 13:47.14 |
| | the rest not so much | 13:47.17 |
| Robin_Watts | We already assume char == byte everywhere in mupdf. That assumption is not going to be lifted any time soon. :) | 13:47.46 |
| malc_ | that position i can live with ;) | 13:48.03 |
| Robin_Watts | but the point is that the size should reflect the type. We could use uint32_t *'s if we gave the size in multiples of that for all I care. | 13:48.33 |
| Robin_Watts | goes to get lunch. | 13:49.06 |
| malc_ | Robin_Watts: that's inconsistent with C... think memset and it's ilk | 13:49.09 |
| mrev1l | hi everyone. can anyone tell me what will happen if within a single pdf file there will be two objects with equal object numbers? | 13:59.35 |
| | like | 13:59.38 |
| | like 12 0 obj someobj decription endobj | 13:59.41 |
| | ........... | 13:59.43 |
| | 12 0 obj someobj decription endobj | 13:59.55 |
| kens | If they have hte same generatiopn number then the PDF is illegal. The ehaviour of a PDF consumer with an invalid file is undefined. However, provided the xref is valid it will only point to a siongle one of the objects, and so the other will be ignored. | 14:00.34 |
| | So its safe but very highly not recommended | 14:00.47 |
| | Ths will usually only be a concern if the xref is damaged, forcing the consumer to rebuild the file, whcih it will have to do by scanning to find all the objects. If it finds two with the same number and generatoin no consumer can tell which one is the 'correct' one to use. | 14:02.06 |
| mrev1l | gotcha | 14:10.22 |
| | thank you | 14:10.25 |
| | is there a way to generate unique number in mupdf? | 14:11.59 |
| kens | Can't help you there | 14:12.15 |
| mrev1l | ok, thank you | 14:12.26 |
| tor8 | mrev1l: what are you trying to do? | 14:12.28 |
| mrev1l | tor8: well, i'm working with annotations and some of them behave wrong. i reckon this is because of they are missing P entry in annotations object's dictionary. some i'm figuring out a way to construct it | 14:15.41 |
| tor8 | mrev1l: if you want to add a new object to the PDF, pdf_create_object will add a new entry and return the object number | 14:18.32 |
| rayjj | Robin_Watts: I started re-running the mupdf timings (just now), but if the results hold up past j9 rgb contone, the performance is a LOT better. from 14.6->27.9 ppm !!! | 14:18.55 |
| tor8 | pdf_add_object will do the same, but also insert the pdf_obj into the document and return a pdf_obj indirect reference to it (the 12 0 R thing) | 14:19.00 |
| rayjj | I just hope it is really still working ;-) | 14:19.20 |
| mrev1l | yeah, cool, thanks tor8 | 14:20.04 |
| rayjj | Robin_Watts: j11 up as well: 12.1->17.5, j12 14.3->26.8 | 14:21.24 |
| | Robin_Watts: Are we sure it is actually working (rendering and everything ?) | 14:22.43 |
| tor8 | Robin_Watts: god I hate compilers. so, error->top = error->stack - 1 is undefined behaviour. | 14:22.44 |
| | you can't have pointers to anything outside an 'object' except to one past the end. | 14:23.03 |
| | Robin_Watts: the int32_t to float pointer casting in your harfbuzz code runs afoul of the strict aliasing rules | 14:26.21 |
| rayjj | Robin_Watts: Now I _am_ suspicious. PLRM went from 37->285.3 ppm | 14:27.21 |
| | Unless something was totally broken before | 14:28.42 |
| Robin_Watts | rayjj: Writing times would have been slow before. That's why I said to retest. | 14:29.00 |
| rayjj | Robin_Watts: so you are sure that it is actually rendering the page when you detect the -o /dev/null ? | 14:30.13 |
| Robin_Watts | rayjj: Yes. The /dev/null only takes effect in the output routines. | 14:30.34 |
| rayjj | because the times with gs writing (to the SD card) vs. skipping writing are not that dramatic | 14:31.01 |
| malc_ | tor8: trying UBSAN or something? | 14:31.42 |
| Robin_Watts | tor8: You mean "the int32_t to float pointer casting that's commented in the code as being horrible and hacky" ? | 14:31.44 |
| rayjj | Robin_Watts: well, at least the timing tests will complete faster :-) | 14:31.55 |
| tor8 | Robin_Watts: yeah. I'm going to move that to use fixed point instead... | 14:32.21 |
| malc_ | tor8, Robin_Watts: officially blessed way to do that is actually memcpy believe it or not (if i understood you correctly, casting ints to floats or vice versa) | 14:32.27 |
| Robin_Watts | tor8: Eh, what? | 14:32.37 |
| tor8 | malc_: change node_scale to node->em / walker.scale * 1000 and keep the x_offset as int | 14:33.55 |
| | and divide by 1000 to put it back into the matrix | 14:34.01 |
| | Robin_Watts: ^ | 14:34.06 |
| Robin_Watts | tor8: But we have to convert the number back to float eventually. | 14:34.14 |
| | So that feels really bad. | 14:34.18 |
| tor8 | malc_: yes, memcpy or 'char*' | 14:34.21 |
| malc_ | tor8: only memcpy nowadays according to the most recent stuff _I_ have read | 14:35.13 |
| tor8 | using plain char* (not unsigned char* or signed char*, plain unadorned char*) is always treated as aliasing everything | 14:36.01 |
| malc_ | tor8: thing is - you then you run into the signedness problem immediately, (even forgetting the whole madness with trying to figure out the, uhm, default signedness of plain unadorned char), you have to manipulate the value of this char leading to promotions and that information will be used by the iveel campajler to screw you over | 14:48.05 |
| mvrhel_laptop | tor8: did you get a chance to have a look at my branch? | 15:14.00 |
| tor8 | mvrhel_laptop: yes, there should be some comments in the log | 15:15.54 |
| mvrhel_laptop | hmm let me look again | 15:16.36 |
| | oh I see | 15:16.58 |
| | tor8: ok I got it | 15:17.56 |
| | I will fix that up | 15:17.59 |
| | thanks | 15:18.01 |
| Robin_Watts | tor8: OK, so I had to resort to profiling the code on windows to get a sensible callstack out. | 15:55.10 |
| | The 42% of the time that is being spent in memset is all from fz_clear_pixmap_with_value, and that's the blanking of each page. | 15:56.24 |
| | so I don't think we can hope to beat that. | 15:56.32 |
| tor8 | Robin_Watts: yeah, sort of unavoidable that | 15:59.16 |
| Robin_Watts | tor8: Couple of commits on robin/master | 16:10.16 |
| malc_ | tor8, Robin_Watts: clear_pixmap is what i've optimized when i needed speed the most actually.. (back on my previous mac mini, the ppc one)... | 16:15.18 |
| | actually it was the routine that sidestepped fz_clear_pixmap that was optimized, not fz_clear_pixmap per se | 16:15.51 |
| | and for image heavy documents (or just images) i had a special device that avoided clearing the pixmap altogether if the image covered whole page and was non transparent (that code is gone now though) | 16:18.01 |
| tor8 | Robin_Watts: #LDFLAGS += -all-static ? | 16:52.48 |
| Robin_Watts | tor8: oops. Let me remove that line. | 16:54.48 |
| | The docs said to use it, but it didn't work with my linker, and we don't use dynamic linking anyway. | 16:55.18 |
| | Updated commits online. | 16:58.27 |
| | Removed that line, plus some dead code I'd forgotten. | 16:58.37 |
| tor8 | Robin_Watts: LGTM ... but, I worry about strict aliasing with the type casting of pointers :/ | 17:04.09 |
| Robin_Watts | tor8: Any suggestions for improving it? | 17:09.15 |
| | Presumably you mean the*(uint16_t *)ddp = color; and similar lines? | 17:10.12 |
| | Would *(uint16_t * restrict )ddp = color; work ? | 17:10.29 |
| malc_ | Robin_Watts: that makes no sense | 17:14.13 |
| tor8 | uint16_t *ddp = dp maybe? | 17:17.31 |
| | Robin_Watts: it's probably fine as is | 17:17.56 |
| pingtux | Hey everyone. Im trying to do a font substitution in cidfmap for a font with a space in the name but it doesn't work. Escaping the space doesn't work as well. Is this possible? | 19:12.47 |
| rayjj | pingtux: you should be able to enclose the name in ( ) to make a string, then use the 'cvn' operator, for example: (Font Name) cvn | 20:18.26 |
| | pingtux: you might have been trying Font\ Name but that won't work | 20:19.03 |
| pingtux | rayjj: Yeah, tried Font\ Name. Thanks, it works this way! | 20:19.44 |
| rayjj | pingtux: glad to help | 20:19.59 |
| | Forward 1 day (to 2016/03/23)>>> | |
IRC Logs