| | <<<Back 1 day (to 2012/07/04) | 2012/07/05 |
| Robin_Watts | tor8: OK, about to start rejigging MIN/MAX/CLAMP/abs | 12:01.42 |
| | I'm going to put fz_min/fz_max/fz_clamp/fz_abs as inline statics in fitz.h, right? All operating on floats. | 12:02.07 |
| | If I need int versions I'll do fz_mini/fz_maxi etc. | 12:02.22 |
| tor8 | Robin_Watts: okay. | 12:02.31 |
| | sounds like what we discussed y'day :) | 12:02.39 |
| Robin_Watts | Fab. | 12:02.43 |
| tor8 | Robin_Watts: I can't see a diff with normal_498 on the huge glyphs :( | 12:03.19 |
| Robin_Watts | Very bottom, with the squiggly line? | 12:04.01 |
| | Left hand end of the squiggly line ? | 12:04.15 |
| | I could be wrong, but that was the difference between us/acrobat. | 12:04.30 |
| tor8 | oh, right, the white noise pattern thingy | 12:04.55 |
| Robin_Watts | yeah, that was all I could see. | 12:05.05 |
| | Yeah with -r50, we draw it. | 12:05.47 |
| | without that, we miss it out and give font size warnings. | 12:05.59 |
| tor8 | well, it's easy enough to reduce the max glyph size to something ridiculously small for testing at least :) | 12:06.12 |
| Robin_Watts | true. | 12:06.27 |
| | Also, normal_130.pdf | 12:07.01 |
| | that one tries to do stroked text with a dash style. | 12:07.14 |
| | so it's a fix in the same area if you feel like it. | 12:07.33 |
| | random thought; is it worth us providing both function and macro versions of those things? and having a #ifdef PREFER_MACROS ? | 12:22.25 |
| | (for compilers that are crap with inlines) | 12:22.35 |
| tor8 | Robin_Watts: argh! and it's a type 3 font too... :( | 12:22.36 |
| | which we can't easily turn into type 3 fonts. time for recursive device calls :( | 12:23.03 |
| Robin_Watts | tor8: Why are type3 fonts harder? | 12:23.30 |
| tor8 | Robin_Watts: no. if the compilers are crap at inlines, they can buy a better compiler :) | 12:23.36 |
| Robin_Watts | assuming such a compiler exists for their target system. | 12:23.56 |
| tor8 | Robin_Watts: for non-type3 fonts we can just turn it into a path to fill | 12:24.01 |
| Robin_Watts | (See Robins first law of compilers) | 12:24.08 |
| | tor8: type3 fonts can just render direct onto the output, right? | 12:24.35 |
| tor8 | Robin_Watts: if they are that desperate, they can live with the performance or rewrite the macros themselves. imo. | 12:24.44 |
| Robin_Watts | I think we already do that for some classes of type3 glyphs (out of spec ones that use color, but don't define the color of the glyph within the instructions) | 12:25.23 |
| tor8 | Robin_Watts: yeah, but to do that they need to call the device from within a device call. reentrancy and crap. | 12:25.30 |
| | Robin_Watts: yeah, but those (render_t3_direct) are done from the interpreter | 12:25.46 |
| Robin_Watts | Yeah, but as I say, I think we already do it. | 12:25.47 |
| | OOOOH. | 12:25.55 |
| | So, can we return something from the device call that means "do this again, direct" ? | 12:26.35 |
| | I should look at the code rather than relying on my crap memory... | 12:26.47 |
| tor8 | I'm thinking of ways to pass the scissor rect, so we can render to a subset pixmap for huge type 3 glyphs | 12:27.25 |
| | or just remove the hard limit for type 3 and suffer the consequences | 12:27.34 |
| | returning the path for freetype fonts was such an elegant solution. damn pdf spec! | 12:27.57 |
| | well, we could put a hard limit on font sizes for type 3 fonts in the interpreter, and render direct there so we don't hit the same case. but that's getting into fragile interdependent territory. | 12:28.46 |
| Robin_Watts | So... we currently say "render_direct = !fz_glyph_cacheable(....)" | 12:29.02 |
| | IF we had the font too large detection done within fz_glyph_cacheable, we'd be fine, right? | 12:30.19 |
| | as we'd go into the direct rendering thing already? | 12:30.43 |
| | or simply move it into pdf_show_char? size = ...; if (size > FONT_MAX) render_direct = 1; ? | 12:32.50 |
| | The trick would be to avoid calculating size twice. | 12:33.01 |
| | but I can't immediately see why the type3 case needs to be hard. | 12:33.19 |
| | if (font->t3procs && !render_direct) { size = fz_matrix_expansion(...); if (size > FONT_MAX) render_direct = 1; } is better | 12:34.57 |
| | Pushing that test into fz_glyph_cacheable seems nicer though - it avoids (as you mention above) having to have the pdf interpreter know of the FONT_MAX limit. | 12:36.30 |
| | It's just a question of passing a pointer to the matrix into fz_glyph_cacheable too, right? | 12:37.03 |
| tor8 | Robin_Watts: yeah, something like that. but it makes the font size limit device independent, not based on the actual resolution. | 12:49.36 |
| | well, when making a display list anyway | 12:49.45 |
| Robin_Watts | Oh, yes, for a displaylist, I guess it does :( | 12:50.14 |
| tor8 | I think passing in the scissor rect and making sure the type 3 glyph pixmap doesn't exceed it is better overall | 12:50.54 |
| | and for freetype fonts, return a path and draw that with our normal path filling | 12:51.08 |
| Robin_Watts | tor8: We should avoid trying to cache bitmaps that exceed the MAX_SIZE though. | 12:51.45 |
| | (I mean, draw them as if we were going to cache them, use them then throw them away) | 12:52.04 |
| tor8 | Robin_Watts: yeah, that's a different limit and different constant though | 12:52.06 |
| Robin_Watts | ok. | 12:52.14 |
| tor8 | MAX_GLYPH_SIZE | 12:52.17 |
| | or CACHE_SIZE or something like that | 12:52.24 |
| Robin_Watts | Urm... in dev_text.c, append_char line 132 | 12:54.05 |
| | Ignore me. | 12:54.23 |
| | oh, crap. I just had a really nasty thought. | 12:58.24 |
| | Some (possibly all) ARM compilers (including the one that customer394 is using) promote floats to doubles on all function calls. | 12:59.03 |
| tor8 | oh... that is a nasty thought indeed. | 12:59.24 |
| Robin_Watts | so using a static inline that takes floats may cause lots of type coercions. | 12:59.37 |
| | Bah. Will ponder that over lunch. | 12:59.54 |
| tor8 | Robin_Watts: being inlined, the coercions may be folded away though. only one way to find out. | 13:00.15 |
| Robin_Watts | I don't have the 5 grand to buy a test copy though. | 13:00.39 |
| tor8 | well, you can at least test what gcc and clang do :) | 13:00.56 |
| Robin_Watts | I'll send them a test file and get them to compile it and send me the disassembly. | 13:01.04 |
| paulgardiner | Robin_Watts: just got a weird cluster push result. | 15:00.06 |
| Robin_Watts | go on... | 15:01.00 |
| | SEGVtastic. | 15:01.22 |
| paulgardiner | Diffs at the end of the report include stuff from ages ago - some it not even mine | 15:01.53 |
| Robin_Watts | The diffs at the end of the report may be wrong. | 15:02.07 |
| paulgardiner | Also SECVtastic too, but given the diffs... | 15:02.09 |
| Robin_Watts | It's possible that it's comparing from forms -> master ? | 15:02.23 |
| paulgardiner | Oh ok. SEGVs could easilt be my fault | 15:02.56 |
| | although my first attempt to repeat them here failed. | 15:03.12 |
| Robin_Watts | I'll have a look at that in a mo. | 15:03.28 |
| paulgardiner | I have had sensibe diffs before | 15:03.29 |
| | Thanks. | 15:03.49 |
| | Actuall my previous cluster push had the correct diffs and sensible resultes | 15:04.11 |
| | Robin_Watts: any reason I shouldn't just try again? | 15:04.49 |
| Robin_Watts | none at all. | 15:05.06 |
| sebras | Robin_Watts: didn't ARM use to have an eval version of SDT? do they have one of their current suite? | 15:09.03 |
| Robin_Watts | SDT was replaced by ADS (which is what the customer uses). ADS was replaced by RealView. | 15:09.28 |
| | And there have been at least 2 versions of RealView. | 15:09.40 |
| | so they are at least 2 generations out of date. | 15:09.50 |
| sebras | Robin_Watts: right, and there is no eval version to make test-compilations with? | 15:10.10 |
| Robin_Watts | not afaik. Certainly it's not something they would offer to download - I don't think ADS is even for sale any more. | 15:10.37 |
| | ADS docs for v1.2 say "Copyright (C) 1999-2001" | 15:12.34 |
| sebras | Robin_Watts: oh, an realview? | 15:14.13 |
| | Robin_Watts: could be worthwhile to try if available. also wouldn't a proper license make quite a bit of sense for artifex given the embedded market? | 15:15.02 |
| Robin_Watts | sebras: I can't see the benefit. | 15:15.30 |
| | If everyone was using one particular toolchain, then maybe. | 15:15.56 |
| sebras | Robin_Watts: are most of your customers running android and thereby using gcc? | 15:15.57 |
| Robin_Watts | I suspect the vast majority of our customers are gcc based, yes. | 15:16.16 |
| | either by using linux directly or by using something linux derived (like android). | 15:16.44 |
| | or by using their home-rolled OS to which they have ported gcc. | 15:17.05 |
| | It used to be that gcc's code quality on ARMs was shockingly poor. | 15:17.19 |
| | Hence why people paid the small fortune for ADS etc. | 15:17.30 |
| | but it's now not-significantly more shockingly poor than the commercial ones are :) | 15:17.51 |
| sebras | Robin_Watts: yes I know. the dumbphones tend to use realview a lot... | 15:18.03 |
| Robin_Watts | dumbphones ? | 15:18.24 |
| sebras | Robin_Watts: non-smartphones. | 15:20.04 |
| | Robin_Watts: phones without touch-screens. | 15:20.25 |
| | Robin_Watts: that's what I started working with in 2004 and we used ARM and realview a lot. | 15:20.42 |
| | Robin_Watts: until it became ARM and gcc because of android which caused lots of grief among the senior developers becuase of generated code quality. | 15:21.28 |
| Robin_Watts | right. | 15:21.35 |
| | At Picsel we assumed all compilers were crap and ARM coded all the speed critical bits. | 15:33.24 |
| sebras | Robin_Watts: oh, well I guess it's different if you basically share offices with ARM... | 15:36.58 |
| | Robin_Watts: working in lund.se is a bit too far away (though ARM eventually bought an ASIC company in lund but they have no SW knowledge). | 15:37.40 |
| Robin_Watts | Really? ARM had a presence in SE? | 15:37.53 |
| mvrhel | good morning | 15:50.55 |
| chrisl | Okay, in the grand scheme of gcc warnings, this one has me bemused: "note: expected âstruct gs_font *â but argument is of type âstruct gs_font *â" | 15:51.57 |
| Robin_Watts | mvrhel: morning. Good 4th July? | 15:52.08 |
| | chrisl: 2 different structures defined with the same name (but at different points in the source) ? | 15:52.56 |
| | Nah, that would have given an error. | 15:53.15 |
| mvrhel | Robin_Watts: yes it was. Summer weather started finally here in seattle | 15:53.25 |
| Robin_Watts | yeah, we've had the first rainless day here for ages (so far) | 15:53.44 |
| mvrhel | this morning there is the smell of Sulfur in the air due to all the fireworks | 15:53.44 |
| chrisl | Robin_Watts: exactly, I'll look into this a bit more when I actually get things building, and functioning again! | 15:54.06 |
| henrys | mvrhel:of your blockers I would think only the xps one matters customer-wise. I don't think we have any customers using those devices, not to say we shouldn't understand the issues. | 15:54.12 |
| mvrhel | ok that one I will have a fix for shortly I think henrys | 15:54.41 |
| | I noticed a pile of progressions with my current patch for fixing the slow issue | 15:56.36 |
| | there was a goof up related to lingering soft masks | 15:56.52 |
| | I have one funny file to look into today | 15:57.10 |
| | henrys: so next week I am going to be in North Carolina. Wed-Fri I am helping out with a short course at NC State University based on my book. | 16:06.36 |
| henrys | mvrhel:great glad to hear the book is getting use. | 16:09.00 |
| Robin_Watts | tor8, sebras: New commits on my casper repo for your consideration. | 16:12.56 |
| | 1 is the static inlines instead of macros, another is the clamp01 one (without clamp01), and the last is the "returns at the start of a jpeg stream" thing done better. | 16:13.50 |
| ray_laptop | Robin_Watts: I was surprised at cust 394's comments about manually taking constants and doing loop unrolling -- I thought many compilers did that (as long as they weren't in -O0 mode) | 16:40.00 |
| Robin_Watts | ray_laptop: Indeed. But their compiler is over a decade old. | 16:40.39 |
| | (2001ish) | 16:41.00 |
| sebras | Robin_Watts: ack, will take a look later. | 16:43.36 |
| Robin_Watts | oops: http://latimesblogs.latimes.com/lanow/2012/07/entire-san-diego-fireworks-show-exploded-in-15-seconds-ruining-show.html | 17:08.46 |
| | tor8, sebras, paulgardiner: The merge of the latest master into forms is now sat on the forms branch on my casper repo. | 17:21.56 |
| mvrhel | ok. find my hopefully last xps trans issue | 17:22.33 |
| | brb | 17:22.37 |
| Robin_Watts | The diffs are expected, except for tests_private/pdf/forms/v1.7/nonemployeetravel.mjs.ppmraw.72.0 which is the same file that paul and I were just looking at in his bmpcmp. It looks like an indeterminism | 17:23.47 |
| mvrhel | ok. yet another push to test the xps transparency fixes. | 19:48.58 |
| | bbiab | 19:49.04 |
| | henrys: so the regression xps blocker is fixed and the customer's xps bug also | 21:51.34 |
| henrys | ah good news. | 22:01.13 |
| | I finally got my pcl problem fixed a bit ago. Very frustrating ... lots of work for a corner case at best. | 22:01.48 |
| | well lots of analysis. | 22:02.05 |
| | Robin_Watts:I'll hold off on the obligatory "premature" joke for the fireworks. | 22:04.43 |
| sebras | Robin_Watts: checked your patches and they look good to me. | 22:12.44 |
| | Robin_Watts tor8: also I thought about Robin's patch for negative stream lengths. the same issue arises in lots of places. e.g. I just managed to get a valgrind complaint on an encrypted pdf where the length field in the encryption dict is negative. is there every a situation where a length can be < 0? I'm thinking about clamping it to [0,INT_MAX] or so. | 22:14.42 |
| | nope, INT_MAX is not apt either. it will crash on this as well. | 22:19.55 |
| | Robin_Watts tor8: I rebased my old patches on top of origin/master, and added three preparation patches befor the last one that actually does all the work -- namely checking that the encryption key length is within bounds. | 23:10.06 |
| | I believe that each patch is correct, but do review it. | 23:10.18 |
| | Robin_Watts: also your patch for negative stream length would apply to pdf_load_image_stream() and pdf_load_raw_renumbered_stream() as well had it not been for check for initial < 1024 in fz_read_all(). this is guaranteed to save us in this case, but do you think that it's obvious enough? | 23:15.02 |
| | I'm unsure. :) | 23:15.32 |
| mvrhel | henrys: I have identified the issu in 693159 and it is in the interpreter. Ray is going to have a look at this. | 23:32.22 |
| | I am now going to work on 693138 and the other stuff that this customer wants | 23:32.44 |
| | to try to get that in before the release | 23:32.52 |
| sebras | hm... I'm growing a bit unsure of whether my patches go against the laissezz-faire attitude we have elsewhere in mupdf. | 23:45.09 |
| | the patches for images and page count _will_ be caught by fz_malloc() eventually (becuase either we get an overflow, e.g. when allocating an image of -16 x -18 pixels, or if we have -99 pages in the page tree). | 23:47.01 |
| | there seems to be similar issues with predictor/columns/ in build_filter() and possible concerning object streams in pdf_repair() as well. in this case I believe that the latter may end up in a very long loop, but I haven't tested this yet. | 23:53.28 |
| henrys | mvrhel:okay sounds right to me. | 23:55.33 |
| | Forward 1 day (to 2012/07/06)>>> | |
IRC Logs