| <<<Back 1 day (to 2016/11/21) | 20161122 |
sebras | sh4rm4^bnc: make nuke && make XCFLAGS='-DTOFU_CJK_LANG' # http://git.ghostscript.com/?p=mupdf.git;a=commit;h=954e3bb173c3302bf458875e86c49c712f0789d4 | 03:36.10 |
| sh4rm4^bnc: http://git.ghostscript.com/?p=mupdf.git;a=blob;f=include/mupdf/fitz/config.h;h=557633a338274dea0d7353d8ae3417298ba00d2a;hb=HEAD#l38 | 03:36.39 |
Robin_Watts | tor8: Morning. | 10:49.36 |
sebras | Robin_Watts: morning! | 10:54.25 |
Robin_Watts | morning. | 10:54.38 |
tor8 | Robin_Watts: morning. | 11:07.04 |
Robin_Watts | tor8: Morning. | 11:07.19 |
tor8 | so I guess you want to talk about the muso thing? | 11:07.37 |
Robin_Watts | sh4rm4^bnc found a problem with the release Makefile last night - dunno if you saw the logs. | 11:07.41 |
| also, the muso thing, yeah. | 11:07.46 |
tor8 | sh4rm4^bnc: make generate on host, then make CROSSCOMPILE=yes CC=whatever doesn't work? | 11:08.49 |
sebras | tor8: also TOFU_CJK_LANG appears to be missing from include/fitz/mupdf/config.h | 11:09.14 |
tor8 | oh, right. I keep forgetting there is that config.h file... | 11:10.08 |
sebras | tor8: I didn't know of it until i git grepped yesterday when looking for that define. | 11:11.04 |
| Robin_Watts: since the muso files do not affect the existing files (apart from the .vcproj files apparently), is it controversial in some way? | 11:14.21 |
Robin_Watts | sebras: Well, the threading changes do. | 11:14.47 |
sebras | Robin_Watts: seems like that layer conforms more to so than mu conding style wise, but I guess that is deliberate. | 11:14.49 |
Robin_Watts | but other than that, no. | 11:15.00 |
| sebras: The point of mu-office-lib is that it offers an almost identical interface to smart-office-lib.h | 11:15.26 |
| hence yes, the naming style etc is in keeping with the other one. | 11:15.40 |
sebras | Robin_Watts: yeah, I think that makes sense, as the people potentially looking at these functions are the ones that know so. | 11:16.05 |
tor8 | Robin_Watts: that's a lot of extra directories added | 11:17.17 |
Robin_Watts | tor8: Is it? | 11:17.39 |
tor8 | source/helper, source/tests, source/helper/mu-office-lib | 11:17.50 |
Robin_Watts | I add "helper" in include and source for the mu-threads.{c,h} stuff. | 11:18.15 |
| Right. | 11:18.25 |
| Which ones do you feel are superfluous ? | 11:18.36 |
tor8 | do we anticipate more tests? | 11:18.58 |
| maybe we should move docs/examples to that directory as well | 11:19.06 |
Robin_Watts | I really don't want the threading stuff to go into the mupdf lib. | 11:19.11 |
| I have a layertest.c here. | 11:19.19 |
tor8 | platform/mu-office-lib for the mu-office-lib stuff instead of as a subdirectory of helpers? | 11:19.25 |
Robin_Watts | tor8: But it's not a platform specific thing. | 11:19.50 |
tor8 | source/mu-office-lib then | 11:20.08 |
| I have a dislike for deeply nested directory structures :) | 11:20.19 |
| I wonder if we should reconsider our stance of being threading library agnostic though | 11:20.22 |
Robin_Watts | Noooo! | 11:20.32 |
tor8 | okay, I'll wait another year until I say that again :) | 11:21.32 |
Robin_Watts | I guess the question is "do we forsee ourselves having more helpers in the future" ? | 11:21.50 |
sebras | naming is not my forte, but it seems to me like a bit of a misc-directory..? | 11:22.55 |
Robin_Watts | sebras: helper is more directed than misc I feel. | 11:23.24 |
tor8 | these helpers, they are for the mol code only now? I'm fuzzy on the details of how this ties into fitz. | 11:23.30 |
| when would someone using fitz need these helpers and how would they use them? | 11:23.57 |
sebras | tor8: it is also for the threading implementation which is used in mudraw/muraster. | 11:24.09 |
Robin_Watts | The idea is a "helper" lib is a lib that wraps libmupdf (or some other part of our system) to provide new (or alternative) APIs. | 11:24.16 |
tor8 | is this to provide implementations for the callbacks for the fitz locks only? | 11:24.20 |
Robin_Watts | tor8: There are 2 parts to this stuff. | 11:25.08 |
| The first part is libmuthreads. That's a simple thread abstraction that provides a simple mutex/semaphore/thread API based on windows and pthreads systems. | 11:26.12 |
| We need those features for mudraw and muraster (when used withb bgprint etc) | 11:26.48 |
| They also use that lib to provide the locks, but that's just one part of it. | 11:27.07 |
| so libmuthreads is a helper lib for tools that might need cross platform threading. | 11:27.50 |
tor8 | right. so mudraw needs a threading api (which we with your branch provide with the helper library, previously with #ifdef'd code) | 11:28.00 |
Robin_Watts | Previously I had the code for this hardwired into mudraw/muraster. | 11:28.09 |
| Yes. | 11:28.11 |
tor8 | how much would it pain you to make that API part of mupdf core, with the actual implementation provided by the helper library? | 11:28.42 |
Robin_Watts | Massively. | 11:28.53 |
tor8 | sort of like how we provide malloc callbacks | 11:29.01 |
Robin_Watts | Cos that breaks the threading agnosticism that I like so much. | 11:29.02 |
tor8 | okay. I'm happy to let you keep this stuff in a separate helper library, that mudraw and muraster depend on. | 11:29.38 |
| and now I understand why you put mol in helpers/ | 11:30.25 |
Robin_Watts | I am absolutely prepared to rethink directory naming though if we have a better plan. | 11:30.59 |
tor8 | source/help/threads and source/help/mu-office-lib perhaps? | 11:31.01 |
sebras | tor8: do we need the -lib suffix? source/help/muoffice? | 11:31.30 |
tor8 | I think 'util' is a more common name, but I like the idea of helpers | 11:31.32 |
Robin_Watts | source/help sounds like a help system. | 11:31.39 |
tor8 | source/helpers (or source/helper, I can never decide on singular or plural names for directories) | 11:32.06 |
Robin_Watts | sebras: It's called mu-office-lib to match smart-office-lib | 11:32.15 |
tor8 | oh. I need to pop out for lunch. be back later. | 11:32.25 |
Robin_Watts | The mupdf convention is to call things libmuoffice etc. | 11:32.28 |
| tor8: Have fun. | 11:32.36 |
tor8 | it's foreign, mu-office-lib doesn't offend me | 11:32.44 |
Robin_Watts | but again, the aim is to have this level be as close to smart-office-lib as possible. | 11:33.03 |
| partly cos I want to be able to take freds code, and search and replace it :) | 11:33.20 |
sebras | Robin_Watts: ok, so-lib is in flux now as well? | 11:36.23 |
| Robin_Watts: I expected that it was a stable API already. | 11:36.35 |
Robin_Watts | sebras: smart-office-lib.h is broadly stable, but expanding. Paul is adding stuff to it all the time. | 11:36.57 |
| We are just supporting the core of it. | 11:37.06 |
sebras | but then again I'm so-illterate (I'm happy that I used a dash there...) | 11:37.13 |
Robin_Watts | fred is writing android classes that wrap it to make an app. | 11:37.37 |
| I want to be able to take those classes, do some renaming in them (and other minor tweaks) and hence have an app that calls mu-office-lib instead. | 11:38.14 |
sebras | Robin_Watts: right, I see. | 11:38.38 |
Robin_Watts | and ultimately we want to have a single app that will call either smart-office-lib or mu-office-lib as appropriate for the doc type. | 11:38.53 |
sebras | Robin_Watts: the tests require the so framework right? | 12:03.54 |
| Robin_Watts: oh, and windows. that's means that we plan to run these tests manually and not in the cluster..? | 12:06.19 |
Robin_Watts | mu-office-test is standalone - nothing from so required. | 12:06.52 |
| and I don't think it's windows only. | 12:07.00 |
| gotta run H to station. | 12:07.09 |
sebras | Robin_Watts: but it includes windows.h and calls CreateSemaphore()..? | 12:14.19 |
| tor8: figuring out what to do about the new unicdoe scripts in noto.c was not as easy as expected. | 12:47.23 |
| tor8: noto has declared future support for adlam, but newa seems to be an active language of nepal. | 12:48.53 |
| s/of/in/ | 12:49.07 |
tor8 | sebras: do they have the fonts yet? | 12:52.55 |
sebras | tor8: as far as i can tell from the noto repo over at github, no. | 12:57.12 |
| tor8: why is BRAILLE and MIAO not handled in the same way? | 12:59.24 |
tor8 | sebras: have you got the ucdn update on a branch somewhere? if so, I can look at what we need to do with noto.c | 12:59.25 |
sebras | github/wip2 | 12:59.32 |
| tor8: and if you check now you can see my noto proposal at the tip of the branch | 13:00.55 |
tor8 | sebras: you mean the comment? | 13:03.10 |
| I don't know if NotoSansSymbols covers MIAO | 13:03.34 |
sebras | tor8: but will not the BRAILLE case just be a fallthrough to MIao? | 13:05.32 |
tor8 | sebras: the BRAILLE case should just have a break; as well | 13:05.48 |
sebras | tor8: and a comment, ok. | 13:06.06 |
tor8 | there's no real reason for it not to have one | 13:06.12 |
| the comment is just there to note that there is a font, but it's not handled there | 13:06.26 |
sebras | tor8: ok, where is it handled though? | 13:06.35 |
malc_ | "6 files changed, 5133 insertions(+), 5067 deletions(-)" oh the perils of automatic indentation :( | 13:06.36 |
tor8 | sebras: "fallback to NotoSansSymbols will cover this" | 13:06.55 |
| i.e. when you ask for a font for Braille, you'll get no font back (due to the break;) | 13:07.12 |
| but then we do a last resort using a symbol font, and there we have braille | 13:07.29 |
sebras | ok. | 13:08.06 |
tor8 | fz_lookup_noto_font(braille) returns NULL so then we try fz_lookup_noto_symbol_font | 13:08.21 |
| it's to avoid loading the symbol font twice | 13:08.35 |
| otherwise we'd load it for braille *and* random symbols | 13:08.51 |
| now we just load it once | 13:08.57 |
sebras | tor8: ok. updated commit online. | 13:09.20 |
malc_ | tor8: i've found that notosansymbbol coverage is lacking (compared to Symbola.ttf) | 13:09.23 |
Robin_Watts | sebras: oops. | 13:11.24 |
sebras | Robin_Watts: ok, I wasn't aware this wasn't intentional. :) | 13:12.36 |
| tor8: ok, so my proposed patch for noto.c looks plausible in that case? | 13:18.47 |
tor8 | sebras: yes. | 13:19.01 |
| malc_: okay. | 13:19.13 |
sebras | tor8: I'll merge it with the unicode fix and then that commit ought to be ready. | 13:19.35 |
malc_ | tor8: and symbola is significantly larger (ttf file size wise).. oh well | 13:20.03 |
tor8 | malc_: it is, which is why I'm not thrilled about taking that instead. | 13:20.22 |
| that, and the fact that the glyphs aren't very pretty, are serif, and don't match weight or styles with the other noto fonts | 13:21.12 |
malc_ | tor8: replace it with code (like stadium drawing example you presented yesterday) :) | 13:21.20 |
tor8 | and it has a lot of extra glyphs outside of the symbol block- | 13:21.33 |
malc_ | true that | 13:24.26 |
Robin_Watts | ok, so that's customer stuff put back to bed. | 13:42.06 |
| tor8: So, did we settle on naming/dir locations etc ? | 13:42.23 |
tor8 | Robin_Watts: just one more thing. do we put the helper library headers in with the public headers or with the helper source? | 13:42.59 |
Robin_Watts | tor8: mupdf/helper/mu-office-lib/mu-office-lib.h currently, I think. | 13:43.28 |
| I could live with mupdf/helper/mu-office-lib.h I guess. | 13:43.45 |
tor8 | Robin_Watts: in include/ not source/ then? | 13:44.01 |
| either one of those is fine by me. | 13:44.12 |
Robin_Watts | Yeah. APIs in include. | 13:44.13 |
tor8 | helper or helpers? | 13:44.28 |
Robin_Watts | I have helpers at the moment, but I could be persuaded otherwise. | 13:44.49 |
| We use 'docs' not 'doc'. | 13:45.01 |
tor8 | if we don't anticipate multiple headers per helper, I'd probably go with include/mupdf/helpers/mu-office-lib.h | 13:45.08 |
Robin_Watts | and scripts, not script. | 13:45.17 |
tor8 | I think I'm also on the side of preferring plural | 13:45.18 |
Robin_Watts | (and resources) | 13:45.29 |
| Ok. | 13:45.31 |
tor8 | yeah, so source/helpers/* and include/mupdf/helpers/* then | 13:46.03 |
| and one sub-directory per helper in both source and include is probably best | 13:46.27 |
Robin_Watts | tor8: Seems reasonable. | 13:46.40 |
| Will adjust commits accordingly. | 13:46.45 |
tor8 | so source/helpers/mu-office-lib/*.c and include/mupdf/helpers/mu-office-lib/mu-office-lib.h | 13:46.52 |
Robin_Watts | Are you OK with a 'tests' directory too? | 13:46.58 |
tor8 | yes. | 13:47.01 |
Robin_Watts | cool. | 13:47.07 |
| I shall do that after lunch then. | 13:47.20 |
tor8 | let me know if you want me to cook up some Makefile voodoo for muso and the tests | 13:48.01 |
Robin_Watts | tor8: yeah, need to think about that. | 13:48.32 |
tor8 | you can leave the makefile stuff to me | 13:49.54 |
sebras | tor8: I'm clustering sebras/master, want to LGTM them..? | 14:44.42 |
tor8 | sebras: the 4 commits on sebras/master LGTM | 14:46.08 |
sebras | tor8: clustered ok, so I'll push in that case. | 14:53.36 |
| done. | 14:57.06 |
Robin_Watts | tor8: muso commits updated. | 15:04.05 |
| I'm in two minds about the include paths. | 15:04.14 |
tor8 | Robin_Watts: okay? | 15:04.37 |
Robin_Watts | #include "mupdf/helpers/mu-threads/mu-threads.h" seems good if we forsee ourselves having lots of different header files for each helper. | 15:04.48 |
| but #include "mupdf/helpers/mu-threads.h" seems better if we think our helpers are going to be small things. | 15:05.06 |
tor8 | yes. | 15:05.14 |
Robin_Watts | The current commit uses the former. | 15:05.20 |
| I think I would prefer the latter. | 15:05.25 |
tor8 | and I do anticipate them to be small things; with possibly big header files. | 15:05.31 |
Robin_Watts | Shall I change to the latter? | 15:05.40 |
tor8 | I think I'd prefer the latter, even if it means having big header files | 15:05.48 |
Robin_Watts | We are in agreement. Will fix. Ta. | 15:05.55 |
tor8 | if they become too big, the helper is probably going to be in need of splitting up | 15:06.00 |
| Robin_Watts: the #ifdef guard has HELPER in singular | 15:06.32 |
Robin_Watts | OK, all fixed in the new commits I hope. | 15:18.31 |
sebras | Robin_Watts: hm... I have found a regression due to 37f95f87bdfb2e0b01d649afae5fffe60bf99d3a | 16:18.39 |
Robin_Watts | ooh. | 16:19.08 |
| tell me more. | 16:19.34 |
sebras | Robin_Watts: http://pastebin.com/dsKK6HWN | 16:19.54 |
| Robin_Watts: this is a murun script called wrapjpx.js that tor8 helped me make | 16:20.07 |
| Robin_Watts: using that I can run it like so: build/debug/mutool run wrapjpx.js test.jpx | 16:20.34 |
| after that I get an out.pdf which I can view perfectly fine. | 16:20.43 |
Robin_Watts | That just slides off my brain. | 16:20.53 |
| Ok. | 16:21.01 |
sebras | Robin_Watts: the details of the js are not important (yet). | 16:21.10 |
| after your commit I see: | 16:21.28 |
| error: bad data in ahxd: '�' | 16:21.28 |
| warning: read error; treating as end of file | 16:21.29 |
| ... | 16:21.31 |
| so somewthing is awry somewhere. I haven't yet localized where, tor8 suspected pdfwrite, but that is all handwaving so far. | 16:21.57 |
Robin_Watts | Breakset fz_buffer_storage, fz_buffer_as_string (or is it fz_string_from_buffer), and fz_buffer_extract | 16:22.40 |
sebras | evince cannot view the pdf either, so the output from mutool is likely broken. | 16:23.30 |
| Robin_Watts: six calls to fz_buffer_storage(), then the output is done. | 16:24.28 |
Robin_Watts | sebras: Right, so look at where those calls are from. I probably screwed one of them up. | 16:25.11 |
sebras | pdf_update_stream, copystream, isbinarystream | 16:25.44 |
| I can't fault those functions. | 16:29.16 |
Robin_Watts | sebras: I just rebuilt with the commit before that function, and it gives me the same errors. | 17:05.36 |
| Oh, wait, fingers... | 17:06.05 |
sebras | Robin_Watts: I'm seeing in my file that the length entry of one of the objects differ in the output that works and the one that doesn't work. | 17:09.41 |
| the length is bigger in the one that works. | 17:10.02 |
| Robin_Watts: oh... I see a problem. | 17:10.33 |
| Robin_Watts: the non-working one claims to use ascihexdecode but the data is never really encoded. | 17:10.56 |
| Robin_Watts: localizes it to copystream() right? | 17:11.05 |
Robin_Watts | probably, yes. | 17:11.16 |
| but a clean build of 37f95f8~1 is still failing for me. | 17:12.11 |
sebras | retests. | 17:12.45 |
| Robin_Watts: make nuke? | 17:14.02 |
Robin_Watts | windows :) | 17:14.10 |
sebras | Robin_Watts: I can still reproduce the issue. | 17:14.10 |
Robin_Watts | sebras: Can you point me at a jpx file that works as an arg? | 17:14.58 |
sebras | Robin_Watts: ok. find the project pane, find the project build listbox, choose rebuild all, press F9 | 17:15.02 |
Robin_Watts | sebras: Yeah, I know how to rebuild all :) | 17:15.18 |
sebras | Robin_Watts: I know. :) | 17:15.23 |
| Robin_Watts: Rome.jp2 in ~sebras on casper | 17:15.31 |
Robin_Watts | Ok, rome.jp2 was what I was testing with :) | 17:15.48 |
sebras | Robin_Watts: 4636fb9cf8d5c219cd388e0442fedc73 <-- md5 | 17:16.07 |
| Robin_Watts: to confirm it's the same. | 17:16.13 |
| seems plausibnle though. | 17:16.20 |
Robin_Watts | platform/win32/Debug/mutool.exe run wrapjpx.js Rome.jp2 | 17:16.57 |
| Rome.jp2 matches that md5sum. | 17:17.15 |
| out.2pdf has an md5sum of 385da5fff06d880735e74d9cac41cfc9 | 17:17.35 |
| out.pdf even | 17:17.41 |
sebras | Robin_Watts: ok, so the bug is in hexbuf() | 17:19.27 |
| Robin_Watts: you construct a fz_new_buffer_from_data() with len, | 17:19.41 |
| but you never adjust buf->len to whatever value n ends up being. | 17:19.56 |
| Robin_Watts: just shy of half of that I think. | 17:20.09 |
| half of the value len is originally. | 17:20.19 |
Robin_Watts | Oh! | 17:20.35 |
| The calculation of len is supposed to be exact. | 17:20.42 |
| This function takes data starting at p for n bytes, and returns a new buffer. | 17:21.19 |
| len = n*2 + (n/32) + 2 is supposed to be the final length. | 17:21.39 |
| and we return the buffer as buf. | 17:22.00 |
| That looks correct to me. | 17:22.04 |
sebras | Robin_Watts: in the original code we get called with n == 388051 then computes buf->cap == 788230 and buf->len == 0 | 17:22.12 |
Robin_Watts | sebras: AH! | 17:22.17 |
| The problem is where we *call* hexbuf. | 17:22.27 |
sebras | how so? | 17:22.40 |
Robin_Watts | On return, we should do len = fz_buffer_storage(ctx, &data, len); | 17:22.44 |
sebras | Robin_Watts: yeah, that seems correct. | 17:23.39 |
| Robin_Watts: well.. on tmp that is. | 17:23.50 |
Robin_Watts | http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=cbdbb2beb81c596855661507ea004c3a0b4f8263 | 17:25.56 |
sebras | Robin_Watts: NAK. | 17:26.24 |
| Robin_Watts: you need to stop messing about with thirdparty... ;) | 17:26.39 |
| Robin_Watts: the code changes seem correct though. | 17:26.55 |
Robin_Watts | http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=5065a3dca4fd9c3b20d29c7f72d25d5213959cb1 | 17:28.21 |
| Thanks. | 17:28.23 |
| I wonder if that justifies 1.10.1 ? | 17:28.33 |
sebras | Robin_Watts: in a few days perhaps? | 17:28.51 |
Robin_Watts | yeah. | 17:28.58 |
sebras | Robin_Watts: something else might crop up now that more people are testing and doing a 1.10.2 for those things is unnecessary. | 17:29.25 |
| Robin_Watts: thansk for helping out in debugging this. | 17:29.38 |
Robin_Watts | sebras: No worries. Sorry I broke it :( | 17:31.45 |
sebras | Robin_Watts: np. I'm happy we found it quickly! | 17:32.08 |
| Robin_Watts: but we might want to test pdfwrite output in the cluster somehow? | 17:32.33 |
Robin_Watts | mmm. | 17:33.11 |
sebras | Robin_Watts: it is interesting that we appear to decode the image data in this case even though we _strictly_ don't need it. | 17:39.58 |
Robin_Watts | do we call get_pixmap ? | 17:40.48 |
sebras | Robin_Watts: fz_get_pixmap_from_image()? no. | 17:41.24 |
Robin_Watts | I wonder where we decode the data then... | 17:41.43 |
sebras | Robin_Watts: oh! fz_load_jpx_info() -> jpx_read_image(onlymeta==1) -> opj_decode() which appears to decode it all. | 17:43.01 |
| maybe openjpeg doesn't allow for decoding all metadata but not the samples themselves. | 17:46.38 |
malc_ | Robin_Watts: making it not decode the image, when one is, for example, only interested in dimensions would be great | 17:53.27 |
sebras | malc_: I'm not sure whether openjpeg allows for that. I hope it does, but it was not immediately obvious. | 17:54.47 |
malc_ | sebras: my main problem is jpg/png mainly | 17:55.24 |
| when one has .cbz with a bunch of humongous images and a continuous scroll things get ugly (you end up decoding all of them) | 17:56.15 |
sebras | malc_: right, thanks for explaining the use case. I was just going to ask. :) | 17:56.42 |
malc_ | det er ingenting... (i hope you can understand my broken norwegian (even though a swede. if Bron/Broen have thought me anything it's that you guys are pretty good at decoding each others tongues :) | 17:59.22 |
| ) | 17:59.34 |
sebras | sleeps. | 18:54.15 |
sk_ | does gsview 6.0 have auto redisplay? | 18:59.30 |
Robin_Watts | sk_: You mean "reload when the file changes" ? | 19:08.19 |
| No, I don't believe it does. | 19:08.24 |
sk_ | Too bad. This is a very important feature I believe. | 19:09.33 |
Robin_Watts | sk_: Feel free to open an enhancement request on bugs.ghostscript.com | 19:10.26 |
| sk_: Linux or windows? | 19:12.50 |
sk_ | Well, either would be OK. But Windows for now. Thanks. I appreciate your consideration. | 19:13.59 |
mvrhel_laptop | sk_: I will look into this. I am hoping to take a bit of time next month to spend on gsview. There are a few outstanding issues that have appeared | 19:17.20 |
sk_ | I would be happy to test a version. skokoska@bloomu.edu Many thanks. | 19:21.03 |
Robin_Watts | sk_: I guarantee your email will be lost if you just leave it on here. Leave it on a bug, and you stand a much better chance :) | 19:27.14 |
malc_ | Robin_Watts: I guarantee that he will not, having left 3 minutes before your message | 19:31.06 |
Robin_Watts | malc_: Ha! | 19:31.18 |
| :) | 19:31.21 |
mvrhel_laptop | argh | 20:14.44 |
| Forward 1 day (to 2016/11/23)>>> | |