| | <<<Back 1 day (to 2016/04/18) | 20160419 |
| Maori | to tor8: http://pan.baidu.com/s/1pKYfXV1 The pdf file and the raw document.Thanks | 01:13.00 |
| | Hi,there.I'm trying to get the font information of the special character by using the Mupdf.Using the fz_text_char->style->font->name.But the result seemed to be wrong.Can someone tell me how to get it? http://pan.baidu.com/s/1coLoYU tor8: http://pan.baidu.com/s/1pKYfXV1 The pdf file and the raw document.Thanks | 07:16.56 |
| tor8 | Maori: hi. which special character is getting the wrong font for you? | 08:11.57 |
| kens | tor8 as far as I can see he wants to identify the 'Qt' text, hence the ping. I'm assuming this is the text he has a problem with | 08:12.29 |
| | The Qt is in a non-=embedded bsae 14 font, times roman | 08:12.38 |
| tor8 | well, the 'Qt' text (and all other latin text) in the document has the font "Times New Roman" | 08:13.13 |
| | and the CJK text is using a subset font "ABCDEE+å®ä½" | 08:13.15 |
| kens | Yeha that's a Chinese font name | 08:13.29 |
| | Two glyphs | 08:13.31 |
| | I ought ot be able to read them, but memory filas | 08:13.49 |
| | or even fails | 08:13.55 |
| Maori_ | So,how can i make the Qt words highlight? | 08:15.14 |
| | @kens | 08:15.14 |
| kens | YOu need to do that yourself, clearly | 08:15.15 |
| Maori_ | E | 08:15.30 |
| tor8 | Maori_: what do you mean with "make the Qt words highlight"? | 08:17.36 |
| Maori_ | make it colored | 08:17.36 |
| tor8 | the font name is "Times New Roman", which is true. The PDF file uses Times new roman for the "Qt" text. | 08:17.36 |
| kens | Grr freenode hates me today | 08:17.37 |
| tor8 | do you want to change the PDF file to make the Qt text colored? | 08:17.37 |
| Maori_ | no ,do not change the file | 08:17.37 |
| | just colored the text | 08:17.37 |
| | that's what i want to do | 08:17.45 |
| tor8 | Maori_: which operating system? windows? linux? android? ios? | 08:18.01 |
| Maori_ | windows | 08:18.06 |
| kens | waits for ghostbot's logs to catch up | 08:18.10 |
| tor8 | searching for "Qt" can highlight the text in the windows viewer | 08:18.30 |
| | use '/' key to open search text field | 08:18.39 |
| Maori_ | oh ,no. | 08:19.50 |
| tor8 | Maori_: mupdf-gl is better; the original mupdf.exe does not support unicode in the search input box | 08:19.55 |
| | Maori_: you can use the same text information as the search highlighting does | 08:20.52 |
| Maori_ | the search highlights is just draw a black rect in the background ,Bro | 08:22.10 |
| tor8 | how you draw it is up to you; the information about which area to color is there. | 08:24.01 |
| Maori_ | Not like the words which has a link ,such as the "Download" words | 08:24.39 |
| tor8 | Maori_: changing the color of the text from black is going to be difficult | 08:25.14 |
| Maori_ | yeah | 08:25.22 |
| tor8 | because the PDF specifies the color for all text drawing | 08:25.23 |
| Maori_ | that is whta i waht | 08:25.31 |
| | That's what I want to do | 08:25.51 |
| tor8 | mupdf.exe inverts the rectangle; mupdf-gl blends a transparent red triangle on top | 08:26.09 |
| | Maori_: there are two ways -- 1) change the PDF file, or intercept drawing commands to modify the color | 08:26.39 |
| | 2) cheat. | 08:26.41 |
| | draw a rectangle with additive blending to lighten black to the color you want | 08:27.06 |
| | pixel.red = max(pixel.red + highlight.red, 255) | 08:27.48 |
| Maori_ | alright,thanks very much ,tor8 | 08:28.45 |
| tor8 | or use one of the photoshop blend modes | 08:28.46 |
| | you can look in source/fitz/draw-blend.c for blending math | 08:29.06 |
| Robin_Watts_ | Maori_: Is the idea to read in a PDF file, then change the color of the Qt text within the PDF file and save it back out again? | 08:31.12 |
| | Or do you want to change it just within the display? | 08:31.25 |
| Maori_ | yes | 08:33.55 |
| | That's it | 08:34.11 |
| | Robin: Do you have some ideas? | 08:36.05 |
| Robin_Watts_ | Maori_: So, it seems to me like you want to be using a modified version of the pdf filter stuff. | 08:41.08 |
| | If you use mutool clean with the -s option, we run the content streams in the page through a series of filters. | 08:41.52 |
| | How much do you know about the inside of PDF files? | 08:42.03 |
| tor8 | Robin_Watts_: we don't have a way to expose the pdf op filtering nicely yet | 08:43.12 |
| Robin_Watts_ | Each page has it's contents as a sequence of operators. Saying things like: let the transformation matrix to this, save the state, move here, set the color, set the font, write this text etc. | 08:43.31 |
| tor8 | Robin_Watts_: so faking it with funky blend modes is probably going to be easier. | 08:43.39 |
| | Robin_Watts_: but do go on :) | 08:43.44 |
| Robin_Watts_ | tor8: Funky blend modes is never going to let him save a revised file out. | 08:44.22 |
| tor8 | Robin_Watts_: Yes, but I was under the impression that's not what he wants. | 08:44.46 |
| Robin_Watts_ | That was the direct question I asked above, and he said yes. | 08:45.01 |
| Maori_ | I do not want to chang the file. | 08:45.01 |
| Robin_Watts_ | Oh. | 08:45.07 |
| Maori_ | Just change the display is alright ,bro | 08:45.41 |
| | I think I should to understand what the tor8 tell me now | 08:48.49 |
| | Thanks ,everyone | 08:49.06 |
| tor8 | Robin_Watts_: gah, security researchers... opening bugs for issues fixed four years ago. | 08:52.01 |
| kens | I saw that one, and he'll expect you to fix it 'because it can still be downloaded' | 08:52.32 |
| tor8 | kens: I know. hence my "gah, security researchers" :) | 08:52.53 |
| chrisl | But you change it, it won't be v0.9 any more, so....... | 08:53.05 |
| kens | That's not the way that the security people's 'minds' work | 08:53.24 |
| chrisl | The words "tough sh*t" spring to mind | 08:53.34 |
| tor8 | kens: speaking of bugs, 696711 has an ancient latex document with type3 fonts that use /0, /2, /8, etc as glyph names for the digits | 08:54.23 |
| kens | I saw your reply | 08:54.33 |
| tor8 | neither mupdf nor gs gets that right | 08:54.35 |
| kens | We certainly won't no | 08:54.44 |
| tor8 | oddly enough, evince manages to... | 08:54.53 |
| kens | I guess it decides they are numerals | 08:55.06 |
| tor8 | yeah. I can only imagine that going wrong with fonts that are truly symbolic and use numbers for the glyph names | 08:55.53 |
| kens | Its bound to, there's no way thatr's going to be reliable | 08:56.07 |
| | Of course, falling back to glyph names is never entirely reliable anyway | 08:56.19 |
| tor8 | kens: Well, with type3 fonts you really don't have much choice | 08:56.32 |
| kens | True. I'm fairly sure I've seen such fonts where the glyph names are /1, /2 etc and are not numerals, they are just the index in the CharStrings | 08:57.03 |
| tor8 | kens: Yeah. This font uses /: as a glyph name as well | 08:57.21 |
| kens | So those would be wrong too if you treat the name as a numeral | 08:57.27 |
| chrisl | pdfwrite output from pcl would probably end up like that | 08:57.55 |
| tor8 | I suspect evince just falls back to using the index in the charstrings as the encoding rather | 08:58.30 |
| | chrisl: I've uploaded the mupdf 1.9 release windows binary and source tarballs | 09:04.24 |
| | Robin_Watts_: can you do the android builds, or are you busy with SOT? | 09:04.42 |
| Robin_Watts_ | gs, but yes, I can do the builds. | 09:04.55 |
| chrisl | tor8: thanks - I'll sort the commercial release later today..... | 09:05.28 |
| Robin_Watts_ | tor8: Urgh. android builds are failing.... | 09:14.12 |
| | No, my bad. | 09:18.27 |
| tor8 | sebras: so, your pdf- | 09:43.52 |
| | device fixes | 09:43.55 |
| | all LGTM | 09:45.27 |
| sebras | tor8: I think I may have found one more: http://git.ghostscript.com/?p=mupdf.git;a=blob;f=source/pdf/pdf-write.c;h=243e83eb5d945e168e3922812ac44a08c2a86411;hb=HEAD#l1150 | 09:47.36 |
| | tor8: in there, if pdf_dict_put_drop() fails on line 1170 fz_always() will attempt to pdf_drop_obj(o) a second time, right? | 09:48.34 |
| | tor8: because the pdf_dict_put_drop() will drop its val argument if the put throws. | 09:49.00 |
| | tor8: and if that happens o = NULL is never executed. | 09:49.15 |
| tor8 | sebras: correct. if you use put_drop, it's important not to do it in the always/catch block as well | 09:49.22 |
| sebras | tor8: I think. | 09:49.23 |
| | tor8: there may be a similar situation in pdf_new_xobject(). | 09:49.54 |
| | tor8: do you want me to make both of those part of that pdf-device commit? | 09:50.08 |
| tor8 | sebras: so that code should just use dict_put, never set o=NULL and free in the always block | 09:51.04 |
| sebras | tor8: also I think it is good form to do something like: o = pdf_new_array(...); pdf_dict_put_drop(..., dict, ..., o); and then do pdf_array_push(o, ...); as many times as you want. that ought to be safe, right? | 09:51.26 |
| | tor8: i.e. keep the pointer o around, but do not keep the reference. | 09:52.26 |
| tor8 | sebras: that's also a possibility. but that gets you into scary territory with borrowed references :) | 09:52.27 |
| sebras | tor8: is it really? | 09:52.35 |
| tor8 | but we do that all the time with pdf_dict_get etc | 09:52.41 |
| sebras | tor8: scary? | 09:52.41 |
| | tor8: exactly. | 09:52.47 |
| tor8 | so we guarantee that a pdf_obj is valid for the lifetime of the current function | 09:52.54 |
| | if you have a borrowed reference from pdf_dict_get tec | 09:53.14 |
| | etc | 09:53.16 |
| sebras | tor8: would that be true even if we modify the object or the parent of the object? | 09:53.35 |
| tor8 | so that's probably a better form to make sure we don't leak | 09:53.37 |
| sebras | tor8: I'm unsure what operations may cause a borrowed pointer to become stale. | 09:54.07 |
| tor8 | if you overwrite the dictionary slot that holds the borrowed object, then you're going to be holding a stale pointer | 09:54.09 |
| | sebras: two things can happen to invalidate a borrowed pdf_obj pointer | 09:54.41 |
| | something causes the original holder to drop it (such as overwriting the slot in the array/dict) | 09:55.05 |
| | f.ex: a = pdf_new_int(); pdf_dict_puts_drop(dict, "foo", a); [a is on borrowed time here] pdf_dict_puts_drop(dict, "foo", somethingelse); [a is pointing to freed memory here] | 09:55.51 |
| | the second thing that can happen is, if you free the container itself all its children will be dropped as well | 09:56.27 |
| sebras | the second thing is kind of obvious. | 09:56.57 |
| | but in the first one, what happens is then that pdf_dict_puts*() reallocates the dict array and therefore the pointer becomes stale? | 09:57.41 |
| | tor8: remember that this is likely the pointer to the value, not the key. | 09:57.52 |
| tor8 | and *that* can happen if pdf_clear_xref is called, which will flush all cached pdf_objs that we have parsed from the xref | 09:57.52 |
| | sebras: the pointer is to the original object, not the slot in the dictionary array | 09:58.11 |
| sebras | tor8: now I see it! you wrote "foo" twice above! so of course the dict drops the reference to the value of "foo". d'oh! | 09:59.07 |
| | tor8: I was reading "foo" as pdf_dict_puts_drop(foodict, ...) :) | 09:59.30 |
| tor8 | yes, "foo" is the key into the dictionary | 10:00.09 |
| sebras | tor8: naturally. | 10:00.16 |
| | tor8: so which approach would you encourage? keeping a reference around or borrowing a pointer? | 10:00.46 |
| tor8 | for stuff where you create objects like here in pdf-device etc | 10:01.13 |
| | I'd do what requires the least amount of try/catch messing about | 10:01.23 |
| | which is allocate, put_drop, then use | 10:01.28 |
| sh4rm4^bnc | i'm seeing this in my mupdf 1.9 build log: | 10:02.35 |
| | -Dhb_malloc_impl=hb_malloc -Dhb_calloc_impl=hb_calloc \ | 10:02.41 |
| | -Dhb_free_impl=hb_free -Dhb_realloc_impl=hb_realloc \ | 10:02.41 |
| | which gives me the feeling that someone is trying to use a replacement malloc | 10:03.05 |
| Robin_Watts_ | sh4rm4^bnc: Yes... | 10:03.08 |
| sh4rm4^bnc | can i disable that ? | 10:03.18 |
| Robin_Watts_ | sh4rm4^bnc: No. | 10:03.21 |
| sh4rm4^bnc | using replacement mallocs is broken | 10:03.35 |
| | let me find the link... | 10:03.41 |
| Robin_Watts_ | All MuPDF allocations go through fz_malloc. | 10:03.42 |
| sh4rm4^bnc | well if it uses the system malloc everything is fine | 10:04.04 |
| Robin_Watts_ | fz_malloc goes through whatever the mupdf caller specifies. | 10:04.08 |
| | All our apps currently have fz_malloc call malloc, but the point is this is up to the caller. | 10:04.51 |
| | Why is using replacement mallocs broken? | 10:05.01 |
| sh4rm4^bnc | there was a discussion recently on the musl mailing list about that | 10:06.17 |
| Robin_Watts_ | Having a library that is hardwired to call malloc and free is what is broken. | 10:06.48 |
| | harfbuzz is VERY poor in this regard. | 10:06.55 |
| sh4rm4^bnc | how so ? | 10:07.03 |
| Robin_Watts_ | It cannot function as a shared library in any sensible way. | 10:07.22 |
| sh4rm4^bnc | malloc and free are provided by the C library | 10:08.23 |
| Robin_Watts_ | sh4rm4^bnc: Yes, obviously. | 10:08.44 |
| sh4rm4^bnc | and it's not just that; there are also strdup, realloc, getaddrinfo and other APIs that allocate memory | 10:09.09 |
| Robin_Watts_ | Now consider an embedded system where you can't use vanilla malloc and free. | 10:09.25 |
| sebras | tor8: back to that link that I sent you. doing pdf_dict_put_drop() early and borrowing the pointer ought to mean that I no longer need fz_var(o) outside of fz_try since I no longer need to initialize or nor will it be used in fz_always and I need to highlight to the compiler that the value may have changed. | 10:09.56 |
| sh4rm4^bnc | now if you override the malloc symbol, libc internal stuff calls a malloc that not acts like the one libc was coded against | 10:10.05 |
| Robin_Watts_ | sh4rm4^bnc: realloc is similarly redirected, and we avoid th others. | 10:10.13 |
| | sh4rm4^bnc: Right. We never override the malloc symbol. | 10:10.27 |
| tor8 | sebras: correct. since you don't read the value in the always block, you can safely remove the fz_var | 10:10.29 |
| sh4rm4^bnc | also on musl brk() and sbrk() are noops | 10:10.55 |
| sebras | tor8: right. | 10:10.58 |
| Robin_Watts_ | We specifically avoid using methods that call malloc underneath (or if we do call them, we copy the data and then free immediately). | 10:11.10 |
| | So all persistent data in the system is handled by our own malloc system. | 10:11.27 |
| | Relying on a crap #define like harfbuzz does is REALLY poor. | 10:11.41 |
| sh4rm4^bnc | since musl's malloc controls the heap, and other implementations calling brk() would mess up the internal allocators state | 10:11.42 |
| Robin_Watts_ | sh4rm4^bnc: Yes. In most cases, this stuff is either used so that allocations/frees goes through an allocator that wraps malloc (in which case only malloc is calling brk), or it's on a system where brk is not being called. | 10:12.49 |
| tor8 | sh4rm4^bnc: yes, but that's not what's happening here. We are patching harfbuzz to use our hooks, so that if a user installs a custom allocator at runtime with mupdf, we don't mix pointers from one allocator with another. | 10:13.12 |
| Robin_Watts_ | I'll say again; having a lib hardwired to call malloc/free is very poor practise. | 10:13.35 |
| tor8 | if you don't install a custom allocator with mupdf (none of the default apps do) then we use the system malloc | 10:13.41 |
| sh4rm4^bnc | malloc is defined by the C standard, what's so bad about assuming it exists | 10:14.46 |
| | if you code in C | 10:14.53 |
| tor8 | sh4rm4^bnc: just because it's there doesn't mean you want to use it. and libraries shouldn't be making those decisions. | 10:15.26 |
| sh4rm4^bnc | if your libc doesnt have malloc, its not a real libc | 10:15.42 |
| sebras | tor8: Robin_Watts_: sh4rm4^bnc: actually shared libraries have a similar situation with logging through printf(). you don't want libraries do to that, instead you want to have an interface in the library so you can hook the logging into your programs logging interface. same thing, different area. | 10:16.19 |
| sh4rm4^bnc | what would the objection to using it be ? speed considerations ? | 10:16.21 |
| sebras | lest you want to end up with the GNOME-like programs whose libraries spew a ton of loggging to stderr when executed in a terminal. :-/ | 10:17.41 |
| sh4rm4^bnc | yeah of course it's bad practice to mess with stdout/stderr from a library | 10:18.59 |
| sebras | sh4rm4^bnc: same thing with malloc. | 10:19.07 |
| sh4rm4^bnc | but i don't see the relation to malloc | 10:19.07 |
| sebras | sh4rm4^bnc: if the program needs to use a different allocator for some reason, you don't want any library to override the application developers decision. | 10:19.39 |
| kens | People using embedded systems have requirements to be able to control memory usage themselves. THey specifically do not want lilbraries to allocate fomr the system pool of memory | 10:19.45 |
| sebras | sh4rm4^bnc: one of the reasons could be speed for small allocations. or having to allocate memory in a certain location to make it possible to do DMA (granted to you rarely need to do that from userspace ;) ). | 10:20.41 |
| chrisl | Memory pools dedicated to specific "subsystems" are common on embedded systems | 10:20.55 |
| Robin_Watts_ | Android builds done. | 10:22.54 |
| tor8 | Robin_Watts_: thanks! | 10:23.15 |
| sebras | tor8: I've updated sebras/master with the leak fixes in pdf-write and pdf-xobject too. can I trouble you with a review of those? | 10:35.25 |
| sebras | learned that git cluster will test even uncommitted changes. d'oh! | 10:41.50 |
| sh4rm4^bnc | wonders why the mupdf tarball grew from 14 to 20 MB | 10:42.52 |
| sh4rm4^bnc | suspects it's docs | 10:43.10 |
| kens | sebras, yes you need to stash anything you have lying around if you only want to test commits | 10:50.39 |
| | because we (well me anyway) often want to test uncommited changes | 10:51.02 |
| sebras | kens: this is not a problem, I just wasn't expecting it. | 10:51.21 |
| tor8 | sebras: LGTM. | 10:59.55 |
| Robin_Watts_ | chrisl: here? | 11:13.00 |
| chrisl | Yep | 11:13.04 |
| Robin_Watts_ | I've been looking into why biros bug is so slow (before my change) | 11:13.23 |
| | and it's down to chunk_locate_ptr running backwards and forwards along the chunk history chain. | 11:13.43 |
| | 50000 steps in some cases. | 11:13.56 |
| | I was pondering converting it from a chain into a splay tree. | 11:14.21 |
| | so we get log2 searching on average. | 11:15.19 |
| | what do you reckon? | 11:15.23 |
| chrisl | Is it always doing that? I thought the heavy use of chunk_locate_ptr was only on debug builds | 11:15.33 |
| | The problem I've always found with using binary trees for memory management stuff is that the memory locations aren't random, so what you gain in search time, you tend to lose in re-balancing/re-organising the tree. But I haven't tried splay trees and I haven't experimented with the gs memory manager(s). | 11:20.14 |
| Robin_Watts_ | chrisl: splay trees do a 'move to root' operation on each locate, that has the effect of (on average) balancing the tree. | 11:33.26 |
| | The SO heap manager is based on splay trees and was WAY faster than the alternatives when it was first written. | 11:33.48 |
| | chrisl: Yes, debug builds use it more heavily, but it's still used on every free operation at least once, as far as I can tell. | 11:34.22 |
| | though repeatedly locating the same thing is very cheap. | 11:34.38 |
| tor8 | Robin_Watts_: AA-trees are a decent but very simple to implement self balancing tree structure. | 11:35.02 |
| chrisl | AFAICT, it should only be called when freeing a chunk containing only one object - I'm surprised I path triggers that | 11:35.51 |
| tor8 | Robin_Watts_: in mujs/jsproperty.c if you want to copy some working code | 11:36.37 |
| | Robin_Watts_: source/fitz/tree.c is a simpler variant that doesn't have delete | 11:38.41 |
| Robin_Watts_ | chrisl: It's triggered for every segment in a path. | 11:41.06 |
| chrisl | Robin_Watts_: is this using immovable memory? | 11:41.28 |
| Robin_Watts_ | line 940 of gsalloc.c | 11:41.41 |
| | in i_free_object | 11:41.55 |
| | 'stable memory' I believe. | 11:42.00 |
| chrisl | <sigh> chunk_locate() rather than chunk_locate_ptr()...... | 11:43.26 |
| Robin_Watts_ | yeah, the one calls the other. | 11:43.46 |
| chrisl | macro fun :-( | 11:44.36 |
| | Robin_Watts_: Yeh, okay, that makes more sense. I'm not sure how big a win it will be overall, but I don't have a problem with the idea | 11:46.54 |
| Robin_Watts_ | I wish I could understand why. | 11:47.37 |
| chrisl | Why what? | 11:47.48 |
| Robin_Watts_ | I mean, we're allocating new segments one after another and putting them into a list. | 11:47.56 |
| | How do we suddenly end up with them being at opposite ends of the chunk list? | 11:48.23 |
| | Why this situation arises. | 11:49.08 |
| | I would have expected us to be freeing blocks that were allocated one after another (or one before the other) | 11:49.44 |
| | Having to jump around in the history seems really odd. | 11:49.57 |
| chrisl | Is it because we are always starting the search from cfirst? | 11:51.06 |
| Robin_Watts_ | We always start the search from clp->cp | 11:51.39 |
| | which is "where the last one was found". | 11:51.46 |
| chrisl | Yeh, but clp->cp can be null | 11:52.00 |
| Robin_Watts_ | when it's first called, I think. | 11:52.09 |
| | I'm not hitting a breakpoint in that if (cp == 0) branch. | 11:52.44 |
| | so no, that's not the problem. | 11:52.51 |
| chrisl | Are we allocating each segment individually? | 11:53.59 |
| Robin_Watts_ | yes. | 11:54.04 |
| chrisl | Hmm, could we switch allocating several at a time? | 11:55.14 |
| | I can't actually remember the form that path segments take.... | 11:56.20 |
| Robin_Watts_ | chrisl: not easily. | 11:56.41 |
| Robin_Watts_ | has to grab lunch. will keep poking this with a stick afterwards. | 11:57.11 |
| Van_ | +Robin_Watts_: Hi it's me again :) I think that muPDF NDK compile works fine with AS2 because it has the NDK version Pkg.Revision = 12.0.2763256-beta1 and google downloads only offer Pkg.Revision = 11.2.2725575 | 13:52.43 |
| Robin_Watts_ | Until very recently I was using r10e with no ill effects. | 13:54.46 |
| Van_ | I have now this problem filed here: http://bugs.ghostscript.com/show_bug.cgi?id=695488 | 13:55.18 |
| | +Robin_Watts_: Alright, I will try to download the r10e version | 13:55.42 |
| Robin_Watts_ | Van_: Wait.... | 13:56.01 |
| | Fundamentally, that problem should only occur if you're using a 64bit ndk building for 32bit targets. | 13:56.38 |
| | Are you using the 64bit ndk? | 13:56.46 |
| Van_ | I assume so, I don't know wich version the AS2 downloads | 13:57.06 |
| Robin_Watts_ | I would assume not. | 13:57.21 |
| | I mean, my local.properties says: | 13:57.41 |
| | ndk.dir=C\:\\Users\\Robin\\AppData\\Local\\Android\\sdk\\ndk-bundle | 13:57.43 |
| | sdk.dir=C\:\\Users\\Robin\\AppData\\Local\\Android\\sdk | 13:57.44 |
| | Actually... I appear to be using r10e (64-bit_, according to /c/users/Robin/AppData/Local/Android/sdk/ndk-bundle/RELEASE.TXT | 13:58.29 |
| | and that works for me. | 13:58.38 |
| | Let me see if that's the latest. | 13:58.49 |
| sebras | Robin_Watts_: I think r11 was released a while back (but that may be the sdk otoh) | 13:59.13 |
| sebras | can't really differ between the two. | 13:59.28 |
| Robin_Watts_ | Yeah, r12 of the NDK is available. | 13:59.30 |
| | I'll try to remember to download that tonight. | 13:59.42 |
| sebras | Robin_Watts_: http://developer.android.com/ndk/downloads/index.html I see r11c here, where are you looking? | 14:00.19 |
| Robin_Watts_ | In the AS2.0 "Check for Updates" window :) | 14:00.40 |
| Van_ | Android Studio downloads Pkg.Revision = 12.0.2763256-beta1 | 14:00.48 |
| | And it works fine, I assume its 64bit and thats why I'm getting the "java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "rand" referenced by "libmupdf.so"..." on the test application | 14:01.49 |
| Robin_Watts_ | what does RELEASE.TXT say for you? | 14:02.35 |
| Van_ | Tehre are no RELEASE.TXT in the folder, either for AS ndk-bundle or the google downloads | 14:03.14 |
| | ndk-build.cmd -v does not help either | 14:03.51 |
| tor8 | Van_: cannot locate symbol "rand" referenced by "libmupdf.so" is EXACTLY the problem we used to be seeing when you download the 64-bit target NDK and use it to build a 32-bit target | 14:05.02 |
| Robin_Watts_ | tor8: It's a pain, because the NDK etc that comes with AS2.0 appears to be the 64bit one :( | 14:05.35 |
| | Which is an ass, cos that's the way I'd like to tell people to build. | 14:05.52 |
| | Cos it's much simpler. | 14:06.04 |
| tor8 | they have over 60 thousand employees ... and still can't fix this problem in, what, the two or three years it's existed? | 14:06.26 |
| Van_ | So my only option is to find a 32bit version of ndk r12 or r10 | 14:07.44 |
| Robin_Watts_ | Van_: look at thirdparty/mujs/jsmath.c | 14:08.28 |
| | and search for rand() in there. | 14:08.34 |
| | Replace it with 0, and rebuild. | 14:08.45 |
| tor8 | Robin_Watts_: there are a dozen other standard libc symbols that it links against, but the device libc doesn't have because they rely on the compiler inlining them | 14:09.21 |
| | it's down to a header/library mismatch... | 14:09.31 |
| Robin_Watts_ | tor8: Oh, so it's not just rand ? ass. | 14:09.36 |
| tor8 | it just bails on the first symbol it can't find | 14:09.52 |
| Robin_Watts_ | Van_: You could try downloading r10e and installing that in place of the existing ndk-bundle. | 14:10.25 |
| jogux | Van_: what android version is your device running? | 14:10.44 |
| Van_ | jogux: It's 4.4.2 Samsung | 14:11.42 |
| | +Robin_Watts_: I will try that, searching the web for downloads | 14:12.04 |
| Robin_Watts_ | The top link on this page, SHOULD work: http://developer.android.com/ndk/downloads/index.html | 14:14.49 |
| Van_ | +Robin_Watts_: I tried both versions yesterday. Always ending in __isinf not found reference or something | 14:17.02 |
| | Found the r10e download. It's an .exe instead of .zip which I searched... https://web.archive.org/web/20151209045413/https://developer.android.com/ndk/downloads/index.html | 14:20.02 |
| jogux | robin: I may have missed something, but is this the issue because if you build with the latest ndk you can only run on the latest Android OS? if so putting android.ndk { | 14:22.35 |
| | platformVersion = "19" | 14:22.35 |
| | } | 14:22.36 |
| | in the gradle config should solve? | 14:22.43 |
| Robin_Watts_ | jogux: We're using ndk-build, not gradle. | 14:23.00 |
| jogux | oh, APP_PLATFORM = 19 then? | 14:23.12 |
| Robin_Watts_ | Yes, we already have that. | 14:23.19 |
| jogux | oh :( | 14:23.23 |
| Robin_Watts_ | This is androids ndk being broken. | 14:23.38 |
| jogux | fair enough | 14:23.50 |
| Robin_Watts_ | If you download the 64bit ndk you can only target 64bit platforms. | 14:23.57 |
| | which is no problem, you just download the 32bit one - except AS seems to download the 64bit version by default. | 14:24.14 |
| | (on 64bit OS at least) | 14:24.20 |
| Van_ | NDK r10e 32bit builds it without errors (something is up with r11...) and the test app works now without a problem | 14:27.09 |
| jogux | Robin_Watts_: urgh | 14:27.29 |
| Van_ | Google messed up the experience with their ndk r11. Otherwise I would've been a breeze. Just download everything, make, modify property file, ndk-build :) Thank you guys for your input and and help | 14:34.38 |
| Robin_Watts_ | no worries. Hopefully google will fix it at some point. | 14:35.10 |
| mvrhel_laptop | tor8: so the talk is this next Tuesday. I am leaving Sunday and hoping to finish up the slides today :) | 14:44.29 |
| | I have a place holder where I was going to talk about mutool run | 14:44.44 |
| | if you could throw something my way by Thursday that should be fine | 14:44.59 |
| chrisl | kens: http://git.ghostscript.com/?p=user/chrisl/ghostpdl.git;a=commitdiff;h=2809d9ef | 15:16.58 |
| tor8 | mvrhel_laptop: I have a stale commit on tor/wip with a few mutool run examples | 15:18.45 |
| mvrhel_laptop | tor8: ah ok. Thanks | 15:19.12 |
| tor8 | mvrhel_laptop: http://ghostscript.com/~tor/stuff/pdfmerge.js but that's a bit more intricate and maybe not such a good example | 15:19.57 |
| mvrhel_laptop | tor8: maybe two. One simple and one that is complex might be nice | 15:20.33 |
| tor8 | mvrhel_laptop: so on the wip branch, the "draw.js" example is how to use the high level document stuff to open documents and render to PNGs and save those out | 15:22.26 |
| | trace.js is an example of how to write a custom device for processing graphics | 15:22.42 |
| | dev.js is how to use the device interface as a graphics library to draw stuff from javascript | 15:23.19 |
| kens | chrisl that seems to make sense | 15:24.09 |
| tor8 | create.js is creating a new pdf document from scratch (and doesn't use the page or resource stuff, just the raw pdf objects) | 15:24.39 |
| | b.js does something similar but using the resource and page APIs you helped write | 15:25.06 |
| chrisl | kens: it's a bit icky, but supporting broken stuff tends to get that way :-( | 15:25.07 |
| mvrhel_laptop | tor8: these are great. I will play around with them some | 15:25.29 |
| kens | I'm afraid so, yes. I like your solution better than mine anyway :) | 15:25.37 |
| | Tyupical Bitstream fonts, mind you, shoddy..... | 15:26.25 |
| tor8 | mvrhel_laptop: so, the pdf_obj's that get wrapped and exposed to javascript have some fancy js->pdf and pdf->js automatic mapping between types | 15:26.33 |
| kens | supposes I should try the file with MuPDF, just a minute | 15:26.52 |
| tor8 | mvrhel_laptop: a javascript string gets turned into a pdf name (so "foo" becomes /foo, and vice versa) | 15:27.14 |
| chrisl | kens: It'll work because freetype handles the fonts | 15:27.19 |
| tor8 | javascript strings with parens get turned into pdf strings, and vice versla | 15:27.30 |
| kens | Oh yeah, never thought of that | 15:27.30 |
| mvrhel_laptop | ok | 15:27.39 |
| tor8 | so "(foo)" becomes (foo) | 15:27.40 |
| | dictionaries and arrays get mapped between PDF and JS objects and arrays | 15:28.00 |
| | and numbers and booleans, etc | 15:28.08 |
| | mvrhel_laptop: so in b.js at the bottom you have a line: trailer.Info = { Author: "(JavaScript)" } | 15:28.41 |
| mvrhel_laptop | yes | 15:28.57 |
| tor8 | that invokes all of the magic. you could do it with functions as well trailer. | 15:29.36 |
| | like: dict = doc.newDictionary(); trailer.put("Info", dict); etc | 15:30.06 |
| | mvrhel_laptop: mutool run without arguments drops you into a repl, so you can easily try things out on the command line | 15:31.52 |
| mvrhel_laptop | tor8: I am going to play around with it some. I probably should read a bit about Javascript. But if I understand this, you have defined some bindings for pdf types which allows one to do this wild west manipulation | 15:34.16 |
| HenryStiles | mvrhel_laptop: be agile! | 15:35.09 |
| tor8 | mvrhel_laptop: yes. all the magic is in source/tools/murun.c | 15:35.46 |
| mvrhel_laptop | tor8: ok I think I have enough to get myself in trouble | 15:37.06 |
| tor8 | mvrhel_laptop: the real magic happens in ffi_toobj, should you feel inclined to go spelunking (I don't expect you to) | 15:37.22 |
| sebras | tor8: I believe I have a fix for 696687 ready over at sebras/master should you feel like reviewing a bit more. | 15:40.28 |
| tor8 | sebras: s/fz_stdout/fz_stdout_global (or something like that)/; s/fz_output_stdout/fz_stdout/ | 15:43.09 |
| sebras | tor8: ok, other than that? | 15:43.28 |
| tor8 | sebras: robin would probably want to see it done as another thing in the fz_context, but we don't have reference counting fz_outputs (and it makes no sense to) so what you have there is fine by me | 15:44.12 |
| sebras | tor8: the api would allow us for putting it inside fz_context at some later point, should the need arise. | 15:44.47 |
| tor8 | sebras: the crypt stuff looks reasonable, but I'm a bit too hungry to review it properly now | 15:45.05 |
| sebras | :) | 15:45.13 |
| tor8 | sebras: indeed. | 15:45.19 |
| Robin_Watts_ | sebras: Just looking at that commit... | 15:47.26 |
| sebras | Robin_Watts_: the pdfposter one? ignore that one. | 15:47.37 |
| Robin_Watts_ | The fz_stdout one. | 15:47.48 |
| sebras | Robin_Watts_: ok. | 15:47.52 |
| Robin_Watts_ | I would indeed like fz_stdout and fz_stderr to be in fz_context. | 15:48.09 |
| sebras | Robin_Watts_: why? | 15:48.16 |
| kens | sebras that file also didn't work on Ghostscript as I recall, how did you fix it ? | 15:48.35 |
| sebras | Robin_Watts_: the reason I'm asking is that maybe there is something more I need to implement. | 15:48.42 |
| kens | That's #696687 | 15:48.51 |
| sebras | kens: the file specifies that the cryptfilter should be /Identity, which doesn't have to be in the /CF dictionary. | 15:49.26 |
| Robin_Watts_ | fz_context should have an fz_output_context which is a reference counted thing. | 15:49.33 |
| kens | sebras, I guess someone will have to look at that for GS as well. | 15:49.47 |
| sebras | kens: so if I spot /Identity then I simply skip investigating if the /CF dict contains it. | 15:49.51 |
| kens | Right | 15:49.56 |
| Robin_Watts_ | That would contain 2 fz_output pointers, which by default would point to your fz_stdout and fz_stderr structures. | 15:50.12 |
| kens | Can you reassign it to me after you're done please and note that GS needs fixing too. | 15:50.24 |
| | sebras^^ | 15:50.34 |
| Robin_Watts_ | We'd need a new fz_context_set_std{out,err} API that just replaces those values in the context. | 15:50.35 |
| sebras | kens: the more worrying thing is /EFF which seems to indicate that there are embedded files (as that filter is not /Identity). | 15:50.42 |
| Robin_Watts_ | That way, a caller can fz_context_set_stdout and mupdf will send stuff where he wants. Make sense? | 15:51.28 |
| kens | sebras I'm out of touch with PDF security handlers :-( | 15:51.54 |
| sebras | kens: oh. there is not way to split a ticket to another product in bugzilla? hrm. :-/ | 15:51.54 |
| Robin_Watts_ | fz_output_stdout(fz_context *ctx) would then be return ctx->output->stdout etc. | 15:52.23 |
| kens | You can't split it no. Just add a note to the report and reassign it to me. You can even close it if you like | 15:52.24 |
| Robin_Watts_ | sebras: But I like what you've done. | 15:53.32 |
| sebras | Robin_Watts_: yes, that makes sense. an application would then be able to do fz_new_output_with_path() e.g. | 15:53.36 |
| Robin_Watts_ | sebras: Right. | 15:54.07 |
| sebras | Robin_Watts_: I simply needed it because pdfshow couldn't print the encryption dict from the file that I spoke with kens about. | 15:54.22 |
| Robin_Watts_ | chrisl:, rayjj: Something is dropping... it might be a penny, it might be large weight onto my head... | 15:56.40 |
| | When we free an object in a chunk, it goes back on a free list right? | 15:57.01 |
| | Does that mean it might get allocated again? | 15:57.13 |
| chrisl | Yes | 15:57.23 |
| | We'll also amalgamate adjacent free blocks | 15:57.58 |
| Robin_Watts_ | Is it possible that we just happen to be getting into a situation where we end up allocating block A, then allocating block B (both from free lists in wildly different chunks), then dropping A, then dropping B, then allocating those 2 again etc ? | 15:58.23 |
| | i.e. do we exhaustively search the free list in all the chunks we're holding for a possible match when allocating ? | 15:59.04 |
| chrisl | Sort of. There are restrictions to do with "inner chunks" (related PS VM) - but I don't think that should apply for stable memory | 16:00.34 |
| Robin_Watts_ | Right, so I was being very confused how this could be happening, but if it's just that "we happen to have 2 small blocks on the freelists separated by miles", then that seems quite plausible. | 16:01.24 |
| | And the splay trees solution should solve this. | 16:01.45 |
| | So I'm going to give that a whirl. | 16:01.54 |
| | Thanks. | 16:02.02 |
| chrisl | One possible issue with splay trees is that chunk_locate_ptr can be called from the garbage collector for potentially very disparate pointers each time: we could end up "thrashing" the tree organising code quite a lot | 16:04.09 |
| Robin_Watts_ | sebras, tor8: I reckon all but the last 3 commits on robin/master are good to go. | 16:04.09 |
| | (Some of those are sebras commits, some are tor8's) | 16:04.40 |
| sebras | Robin_Watts_: but the openjpeg patch has been updated. | 16:07.49 |
| | Robin_Watts_: I attempted to address the issues you mentioned and also removed .c-files that weren't used. | 16:08.19 |
| Robin_Watts_ | OK, updated version. | 16:09.05 |
| chrisl | sebras: did you get around the openjpeg headers issue? | 16:10.44 |
| sebras | chrisl: yes, I specify USE_JPIP and... ehm.. the other flag on the command line like gs does. | 16:11.15 |
| | chrisl: that way we could avoid the mupdf local header file that wasn't being used anyway. | 16:11.30 |
| chrisl | Ah, okay. As I said, with things like that, we can always discuss gs doing something different to make the sharing easier | 16:12.17 |
| | sebras: ^^ | 16:12.24 |
| sebras | chrisl: mm, but tor8 wanted the local header removed if possible so this looked like a good solution. | 16:12.49 |
| chrisl | sebras: yep, I agree. My point was just that you shouldn't be shy about asking if there's a genuine clash - the gs build can be quite flexible that way | 16:13.57 |
| sebras | chrisl: yup, got it. I'm not shy around here, I know you guys too well. :) | 16:14.45 |
| chrisl | :-) | 16:14.54 |
| Robin_Watts_ | mvrhel_laptop: I've squashed the graft branch to a single commit, and I'm looking at that. | 16:29.33 |
| mvrhel_laptop | Robin_Watts_: ok | 16:29.41 |
| Robin_Watts_ | The indentation in the vcproj stuff looks odd, but that's not a showstopper. | 16:29.48 |
| | pdf_graft_object potentially leaks if we have problems, I think. | 16:33.55 |
| | We (might) allocate drop_map at the top. | 16:34.18 |
| | actually, ignore me for now. | 16:34.49 |
| | mvrhel_laptop: All looks good to me. | 16:38.55 |
| mvrhel_laptop | Robin_Watts_: ok great. Thanks for reviewing | 16:39.05 |
| Robin_Watts_ | no worries. | 16:39.12 |
| mvrhel_laptop | Robin_Watts_: and thanks for the suggestions about what to add in the talk for the open print meeting | 16:39.32 |
| | Robin_Watts_: I am not sure how best to commit this branch | 16:40.06 |
| Robin_Watts_ | mvrhel_laptop: Oh, we also do multi-threaded rendering now. | 16:40.09 |
| mvrhel_laptop | Robin_Watts_: yes I did have that added | 16:40.21 |
| Robin_Watts_ | mvrhel_laptop: It's small enough that I might be tempted to squash it all to a single commit. | 16:40.37 |
| | But that's something that you and tor8 need to be happy with. | 16:40.55 |
| mvrhel_laptop | Robin_Watts_: I think that makes sense. What do you think tor8? | 16:41.15 |
| Robin_Watts_ | I think tor8 is currently thinking "nom nom nom". | 16:41.30 |
| tor8 | I'm fine with a squished commit; the intermediate ones do have memory leak and double free issues so having those would be bad for bisecting. | 16:41.38 |
| mvrhel_laptop | right | 16:41.46 |
| tor8 | (waiting for the cooking to finish...) | 16:41.51 |
| mvrhel_laptop | Robin_Watts_: ok. So I am not sure how to do this properly | 16:46.14 |
| | this being, squashing the branch and then bringing it into my master | 16:47.00 |
| Robin_Watts_ | mvrhel_laptop: git checkout graft && git branch graft2 && git checkout graft2 | 16:47.09 |
| | That makes a new branch called graft2 and changes to it. | 16:47.22 |
| | git rebase -i HEAD~20 | 16:47.27 |
| | Then edit every commit except "Initial work" to be 's' rather than 'pick'. Save the file and close. | 16:47.45 |
| | That squashes all of graft2 to a single commit, and will ask you to edit the commit message. | 16:48.10 |
| | When that's done, git pull --rebase golden master | 16:48.31 |
| | and you'll be left with your single commit graft2 branch sat on the end of master. | 16:48.45 |
| mvrhel_laptop | ok. let me give it a go | 16:49.06 |
| | ok. so I now have the single commit in a graft2 branch at the end of master | 17:08.50 |
| | Robin_Watts_: ^^ | 17:08.55 |
| | now do I do a merge or do I make a patch and pull that into master | 17:10.57 |
| Robin_Watts_ | mvrhel_laptop: git push golden graft2:master | 17:20.05 |
| | That says "push graft2 onto golden as master" | 17:20.20 |
| mvrhel_laptop | ah. ok | 17:20.30 |
| Robin_Watts_ | Then: git checkout master | 17:20.31 |
| | git pull --rebase golden master | 17:20.36 |
| | and you pull it back in to your local master branch. | 17:20.49 |
| mvrhel_laptop | I already did it a little different | 17:20.53 |
| Robin_Watts_ | ok, there are lots of ways :) | 17:21.04 |
| mvrhel_laptop | and it faild | 17:21.05 |
| | failed | 17:21.09 |
| | crap | 17:21.11 |
| | Now I will really need your help to clean this up | 17:21.28 |
| | should have done a cluster push | 17:21.40 |
| Robin_Watts_ | OK, so what's the problem ? | 17:22.10 |
| mvrhel_laptop | hmm thats weird | 17:22.17 |
| | looking at the fail log | 17:22.44 |
| Robin_Watts_ | oh, I see, you've pushed. Let me have a look. | 17:22.47 |
| | I suspect you've reverted the mujs thirdparty changes. | 17:23.35 |
| | mvrhel_laptop: Oh, no. I bet you just didn't update the makefiles. | 17:24.57 |
| mvrhel_laptop | Robin_Watts_: ok so what do I need to do to clean up my mess | 17:24.58 |
| Robin_Watts_ | Let me revert your push to golden... | 17:25.09 |
| mvrhel_laptop | I thought the make files look at what *.c files are present | 17:25.16 |
| | I don't recall having to do anything when I did the create stuff | 17:25.31 |
| | Robin_Watts_: thanks | 17:25.45 |
| | maybe I should just fool with this after I get back from the open print stuff | 17:26.01 |
| | I was hoping this would take 5 minutes | 17:26.08 |
| Robin_Watts_ | So your new file is mumerge? | 17:29.18 |
| mvrhel_laptop | There should be 2 new files | 17:29.38 |
| | graft.h, pdf-graft.c, pdfmerge.c | 17:30.29 |
| Robin_Watts_ | pdfmerge is the one I need to add to the mutool make line. | 17:30.55 |
| | Just testing that now. | 17:31.01 |
| mvrhel_laptop | oh. I dont recall doing that for pdfcreate.c | 17:31.15 |
| Robin_Watts_ | pdfcreate is in there, so someone did :) | 17:31.59 |
| | And it's Starting jobs, so it obviously built. | 17:32.15 |
| mvrhel_laptop | You might have done it | 17:32.18 |
| | or maybe I did and I forgot | 17:32.32 |
| Robin_Watts_ | Ok, I've pushed the fix. | 17:32.37 |
| mvrhel_laptop | Robin_Watts_: thanks | 17:32.47 |
| Robin_Watts_ | no worries. | 17:32.52 |
| mvrhel_laptop | of course this is the guy who put away the ice cream in the cupboard | 17:33.01 |
| | and had a plate full of food that I put in the plate stack in the cupboard | 17:33.19 |
| | mind was elsewhere | 17:33.30 |
| Robin_Watts_ | that sounds so like me. | 17:34.07 |
| | except for the ice cream. That's unforgivable. | 17:34.18 |
| mvrhel_laptop | that was a mess | 17:34.28 |
| | bbiaw | 18:18.09 |
| rayjj | Robin_Watts_: or chrisl: could you have look at the "max_used" patch for gs that is on my repo ? I'd like to have this so I can get memory usage even when not in a DEBUG build from -Z: | 19:12.17 |
| Robin_Watts_ | looking | 19:13.04 |
| rayjj | it's more like mudraw this way, where -s m is always available | 19:13.15 |
| Robin_Watts_ | rayjj: Should be size_t not ulong, surely ? | 19:13.42 |
| | long is still 32bits on 64bit windows. | 19:14.09 |
| rayjj | Robin_Watts_: that has a lot of side effects. I looked at that and it was more of a rabbit hole | 19:14.38 |
| | gets into the max_vm parameter setting and other stuff | 19:15.09 |
| Robin_Watts_ | looks vaguely plausible to me otherwise. | 19:15.59 |
| rayjj | Robin_Watts_: I realize that we might want to make all the changes to allow 64-bit windows to allocate more RAM, but that's much more involved. | 19:18.17 |
| Asuran | if i would like to convert a pdf/x.4 to a pdf/a-2b would this embed the colorprofile too? | 19:43.21 |
| rayjj | Asuran: I'd say try it and see, but kens is the pdfwrite expert. I am pretty sure it embeds everything required by the spec | 19:57.12 |
| | ken sharp (kens) is in the UK, so he won't be around for several hours (12+ or so) | 20:00.47 |
| Asuran | thanks | 20:02.28 |
| rayjj | marcosw: the fans on peeved (Dell PowerEdge 2950) are running at high speed even though the load average is low (0.1) and the cpu temps (shown with lm-sensors) are low (~50C) | 23:33.38 |
| | marcosw: do you have any hints to find out why ? | 23:33.52 |
| | i.e., is it just firing them up to see if they work? | 23:35.43 |
| | Forward 1 day (to 2016/04/20)>>> | |
IRC Logs