| | <<<Back 1 day (to 2014/06/16) | 2014/06/17 |
| mattchz | morning | 09:25.09 |
| kens | Hi Matt | 09:25.14 |
| Robin_Watts | morning | 09:25.57 |
| sebras | wiihooo!! :) | 09:30.23 |
| sebras | 's bugfix since 5 months was finally merged today! | 09:30.54 |
| jogux_mac | sebras: woot, congratulation! :) | 09:33.30 |
| kens | A MuPDF fix ? | 09:34.03 |
| pedro_pc | sebras> cool :) | 09:37.09 |
| pedro_pc2 | hmm - irc connections have been a bit droppy over the past day or so | 09:50.51 |
| sebras | no, in proprietary code with my employer. :) | 10:04.03 |
| | but still! | 10:04.07 |
| kens | I thought it seemed a longtime for a MuPDF patch | 10:04.20 |
| sebras | this means I free up more time to play around with mupdf. :) | 10:04.22 |
| | unfortunately it is not a _very_ long time for a patch here... | 10:04.40 |
| mattchz | Hi, does anyone fancy doing a quick android review? Thanks! http://git.ghostscript.com/?p=user/matt/mupdf.git;a=commit;h=d5a22b7c76c7c0f67f9350b2a7a5dc030b212421 | 10:16.04 |
| paulgardiner | I'll take a look | 10:20.38 |
| mattchz | thanks paul | 10:21.01 |
| paulgardiner | So is this pressing the back button quickly after making some other input that provokes rendering? | 10:22.23 |
| mattchz | yeah, basically, it looks like âgloâ is getting NULLed (as a guess, Iâm thinking this probably happens when the core is shutting down, although I donât know enough about the code to say), and the async tasks (I think) are still running | 10:24.29 |
| | Robin recommended checking for NULL and bailing out if so. | 10:24.52 |
| paulgardiner | Yeah makes sense, but is that the scenario that triggers it? | 10:25.28 |
| mattchz | It seems to fix the problem. I donât know if we should be trying harder to shut these async tasks down, or at least wait for them to complete. That would presumably block the UI thread though. | 10:26.12 |
| paulgardiner | Sorry. I haven't looked at the bug. What was the condition that caused the crash? | 10:26.45 |
| | Or was it just known that the crash sometimes happened in those bits of code? | 10:27.28 |
| mattchz | Yeah; flip through the pages of a specific PDF (although I suspect itâs nothing to do with the actual PDF), and press âbackâ while it is rendering | 10:27.31 |
| | http://bugs.ghostscript.com/show_bug.cgi?id=694967 | 10:27.42 |
| paulgardiner | Right | 10:27.44 |
| mattchz | The PDF seems slow to render, so maybe is more likely to trigger the condition than faster PDFs. | 10:27.59 |
| paulgardiner | Just suprised because I thought I tested that. | 10:28.08 |
| mattchz | itâs an intermittent thing; didnât crash every time. | 10:28.17 |
| paulgardiner | yeah might depend on the difficulty of rendering | 10:28.20 |
| | patch looks fine anyway. | 10:28.26 |
| mattchz | thanks. so ok to push to golden | 10:28.41 |
| paulgardiner | Sure | 10:28.46 |
| mattchz | thanks Paul! Thatâs pushed! I presume I should now close the bug (but I canât due to permissionsness, I think) | 10:30.21 |
| paulgardiner | I wasn't aware of AsyncTask.get. That does look like it would do the trick. | 10:32.01 |
| | Might be worth looking at sometime | 10:32.13 |
| mattchz | Yeah, although it would also block the UI thread. | 10:32.25 |
| paulgardiner | I'll close the bug | 10:32.35 |
| mattchz | I can add the AsyncTask.get if you like? | 10:32.50 |
| paulgardiner | I wouldn't worry for now if the current commit fixes the problem. | 10:34.16 |
| mattchz | it seems to. | 10:34.55 |
| | thanks for checking that. | 10:35.00 |
| | hmm. cool. If I put a git sha-1 id in bugzilla, it turns it into a link, which is great. | 10:37.32 |
| | but to the wrong repoâ¦.is there any way to specify the repo? | 10:37.39 |
| kens | thinks not | 10:38.16 |
| mattchz | aww, well | 10:38.25 |
| kens | I think that's somekind of bugzilla magic, so possibly it could look at the product to determine the repo | 10:39.34 |
| mattchz | ah right | 10:43.10 |
| Robin_Watts | yes, it's bugzilla magic that I added. | 10:52.16 |
| | but the point at which I added it, it's impossible to check for what repo. | 10:52.37 |
| mattchz | looks like Paul posted the commit message anyway (unless that was also automatic magic!) | 10:52.54 |
| | robin> nod | 10:53.00 |
| Robin_Watts | So I just guess at bugzilla, and mupdf folks can edit 'ghostpdl.git' to be 'mupdf.git' in the URL :0 | 10:53.05 |
| mattchz | :) | 10:53.23 |
| Robin_Watts | mattchz: We try to post the commit messages in the bug as it saves a lookup later :) | 10:53.45 |
| mattchz | cool | 10:54.04 |
| chrisl | I usually post a "full" link to the commit: http://git.ghostscript.com/?p=mupdf.git;a=commitdiff;h=d5a22b7c | 10:57.53 |
| mattchz | right; that makes sense. | 11:00.42 |
| kens | chrisl do you know where marcos keeps the fuzzing test files ? | 11:16.47 |
| chrisl | kens: you can retrieve them from the regression dashboard | 11:17.53 |
| kens | The fuzzing files ? really ? | 11:18.05 |
| | I can't see them there anywhere | 11:18.56 |
| chrisl | if you look in the log.txt it will give you the path for the file, and then you can use the Test Files link on the dashboard | 11:18.57 |
| kens | Hmm, OK thanks | 11:19.05 |
| Robin_Watts | kens: They are part of the test files (all in svn, either in tests, or tests_private) | 11:23.44 |
| kens | OK, thanks also | 11:23.56 |
| Robin_Watts | but the easiest way to get them is, as chrisl says, to either use the testdocs link on the dashboard or... by the power of crufty cgi scripts, by clicking on the links in the regression reports when viewed on the web. | 11:24.39 |
| mattchz | lunch, bbiab | 11:28.23 |
| | Does anyone have any thoughts on this, please? http://bugs.ghostscript.com/show_bug.cgi?id=694405 | 13:15.48 |
| | Iâm guessing the guy was using an older device (e.g. an iPad 1, with only 256Mb RAM) | 13:16.01 |
| | I couldnât get it to crash on our iPad 3, and I couldnât test with the iPad1 because it can only run iOS 5 | 13:16.30 |
| | Does muPDF have any kind of tuning for the different variants to avoid using too much memory on older devices? | 13:16.55 |
| | [although if it truly is specific to iPad 1, then maybe we donât care anymore as we canât run on 5.1 and the device cannot be upgraded to 6] | 13:17.37 |
| henrys | tkamppeter: weâve been discussing removing the X11 default device and making the display device default - or some printer device default if gtk is not available. Any idea how the ubuntu community would respond to that change? | 13:22.36 |
| Robin_Watts | mattchz: MuPDF has an adaptive memory strategy. | 13:38.20 |
| | Lots of our objects are reference counted. | 13:38.47 |
| mattchz | [btw, since I posted that, Iâve managed to find an iPad 2, and I can reproduce the crash on that] | 13:39.01 |
| Robin_Watts | And as we work, we make various objects (like for instance decompressed images) | 13:39.49 |
| | we put these in the fz_store, which is like a cache. | 13:40.03 |
| | When we first create an fz_context, we give it a maximum size for the cache. | 13:40.38 |
| | s/cache/store/. | 13:40.47 |
| | Whenever we hit that maximum, we do LRU removal of stuff from the store. | 13:41.11 |
| | Also, whenever we do a malloc and it fails, we try to bin stuff from the store and retry. | 13:41.42 |
| mattchz | like cacheman? | 13:41.59 |
| Robin_Watts | Any flashbacks you are getting to ImageCache and CacheMan are purely bad acid. | 13:42.05 |
| mattchz | :-D | 13:42.09 |
| | So, I guess the fz cache size is not getting set low enough in this particular case? | 13:42.53 |
| Robin_Watts | Well, the fz_store size being set too large shouldn't ever affect MuPDF itself. | 13:43.24 |
| mattchz | we are being killed by the OS, I think. | 13:43.39 |
| Robin_Watts | cos any allocation MuPDF does will bin stuff from the cache and retrying. | 13:43.44 |
| mattchz | (after it has warned us a couple of times) | 13:43.46 |
| Robin_Watts | Ah, well, that does indeed sound like the problem then, yes. | 13:44.01 |
| | if the OS warns us, we should probably run an eviction pass through the store. | 13:44.29 |
| mattchz | i.e. iOS doesnât fail mallocs iirc (usually). It just kills the process | 13:44.33 |
| Robin_Watts | How does the OS warn us? | 13:44.51 |
| mattchz | Is there a function we can when we do get a warning from the OS? | 13:45.07 |
| Robin_Watts | Is there some magic iOS message saying "Oy, MuPDF, No!" | 13:45.08 |
| | fz_store_scavenge ? | 13:46.09 |
| mattchz | Yeah. It calls our applicationDidReceiveMemoryWarning: method in the app delegate | 13:46.10 |
| | (which currently just contains a printf) | 13:46.22 |
| | cool - and I can call scavenge() from the main thread safely? | 13:47.37 |
| Robin_Watts | possibly fz_store_scavenge could be wrapped up more nicely for what you need. | 13:47.37 |
| | It's current form is dictated by its use within fz_malloc. | 13:47.59 |
| | Have a look at do_scavenging_malloc in source/fitz/memory.c | 13:48.27 |
| | You probably want to do fz_store_shrink(ctx, size) | 13:49.22 |
| | and have a similar lock/while/scavenge/unlock thing there. | 13:49.36 |
| | (I mean implement an fz_store_shrink) | 13:49.55 |
| mattchz | ok, Iâll try that. Thanks | 13:50.30 |
| Robin_Watts | The only lock required is the FZ_LOCK_ALLOC one, and that is safe to take from the main thread, yes. | 13:50.31 |
| mattchz | k | 13:51.43 |
| henrys | mattchz: are you are regular employee at emobix or is this consulting? | 14:01.35 |
| mattchz | iâm a regular employee | 14:01.48 |
| henrys | mattchz: we have a meeting in a 1/2 hour and I thought Iâd tell everyone who you are ⦠then it became clear I didnât really know ;-) | 14:03.05 |
| mattchz | heh ;) | 14:03.15 |
| Robin_Watts | mattchz is another ex-picselite. | 14:03.31 |
| henrys | how much do they owe him? | 14:03.58 |
| mattchz | I was one of the âluckyâ ones who got everything I was owed | 14:04.14 |
| | [eventually] | 14:04.26 |
| jogux_mac | chz: didn't they owe you a fiver in expenses? :-) | 14:04.30 |
| mattchz | I thought that was vark | 14:04.39 |
| jogux_mac | ah, you could be right | 14:04.46 |
| mattchz | he was a creditor, iirc, I wasnât (well, I wasnât once I got paid) | 14:04.56 |
| jogux_mac | nods | 14:05.02 |
| | we 'rescued' matt from picsel 3 or so years ago :) | 14:05.10 |
| Robin_Watts | jogux_mac: I should send him the picsel beagleboard for him to ebay :) | 14:05.16 |
| jogux_mac | hehe | 14:05.23 |
| pedro_pc2 | Robin: can stash it with our others :) | 14:06.09 |
| jogux_mac | all our employees we adopted from picsel ;-) | 14:06.09 |
| Robin_Watts | jogux_mac also rehomes cats. | 14:06.34 |
| mattchz | lol | 14:08.42 |
| Robin_Watts | poor starving mistreated creatures looking for a better life... | 14:09.35 |
| jogux_mac | we have a pregnant cat in at the moment that someone found on the street, due to give birth in ~2 weeks | 14:09.54 |
| mattchz | one thing we *werenât* was starving, at least on a Friday afternoon⦠;) | 14:09.55 |
| kens | Artifex worked that way for me and Chris :-) | 14:10.02 |
| mattchz | jogu> are you doing the midwife duties this time? | 14:10.24 |
| jogux_mac | matt: we'll see :-) | 14:10.51 |
| mattchz | OOI, where are all of you guys based? Iâm guessing youâre mostly in the CA, US? | 14:14.16 |
| kens | 'mostly' would be in the UK these days :-) | 14:14.31 |
| mattchz | ah, right :) | 14:14.40 |
| kens | Marcos ray and miles are in california | 14:14.46 |
| | henry in Boulder | 14:14.51 |
| Robin_Watts | Kens - near gatwick, chrisl - andover, henrys - denver | 14:14.55 |
| kens | Michael in Seattle | 14:15.01 |
| | Tor in Lund | 14:15.12 |
| Robin_Watts | Tor- Lund, Sweden. | 14:15.13 |
| kens | Takane somewhere in Japan | 14:15.32 |
| Robin_Watts | and Scott (our Sales guy) is in Texas. | 14:15.33 |
| mattchz | Quite spread out then⦠| 14:15.34 |
| kens | :-) | 14:15.46 |
| | Once upon a time it was just Tor in Europe | 14:15.59 |
| | everyone else in the US | 14:16.15 |
| henrys | mattchz: we need a lot of space between us. | 14:16.22 |
| mattchz | hehe | 14:16.31 |
| henrys | meeting in 5 minutes | 14:24.27 |
| | oops I texted ray then realized heâs on vacation. | 14:28.00 |
| kens | :-) | 14:28.13 |
| Robin_Watts | I think we're all here (not expecting Tor, right?) | 14:28.50 |
| henrys | is mvrhel_laptop here? | 14:29.45 |
| | sometimes he checks in and then does some stuff. | 14:30.13 |
| | chrisl: I forgot to ask tor to check the glyph list. Do you think I can just go ahead with the URW order anyway? | 14:30.59 |
| chrisl | henrys: I'd rather Tor checked it - baring in mind I only worked against the UFST, not the Adobe PDF fonts | 14:31.40 |
| henrys | chrisl: okay | 14:31.55 |
| | mattchz: we have a meeting this time each week for a half hour for ghostscript and mupdf. | 14:32.37 |
| mattchz | ok, cool | 14:32.56 |
| sebras | kens: who is Takane? or rather, what nick? | 14:33.03 |
| kens | sebras, he doesn't come to IRC< just the staff meetings | 14:33.15 |
| Robin_Watts | sebras: Takane is not here. He's our 'technical sales guy' in Japan. | 14:33.32 |
| henrys | most items I have are for ray and michael | 14:33.33 |
| kens | I was just being complete. | 14:33.37 |
| henrys | there were a number of items on the agenda that went unassigned. | 14:34.14 |
| | votes for who should write a âjump startâ guide for mupdf embedded - rayâs got gs. | 14:34.58 |
| | I thought Iâd get crickets for that one. | 14:35.37 |
| Robin_Watts | Me | 14:35.40 |
| | (as long as I don't have to write it in Smart Office) | 14:35.49 |
| pedro_pc2 | aww | 14:36.05 |
| jogux_mac | you're only allowed to use the types of diagrams that SO supports :-) | 14:36.35 |
| henrys | Robin_Watts: you have to write it in smart office on an iphone | 14:36.43 |
| Robin_Watts | jogux_mac: I think you mean "a text file". | 14:36.56 |
| jogux_mac | aw | 14:37.12 |
| sebras | Robin_Watts: Kens: aha. :-) | 14:37.24 |
| Robin_Watts | I could be persuaded to use simple HTML, but I'm not using any of this new fangled office filth. | 14:37.42 |
| kens | text files, 7-bit ASCII only, anything else is the work of Satan | 14:38.05 |
| mattchz | Thereâs always this funky new MarkDown thing | 14:38.06 |
| henrys | okay I put your name on it Robin_Watts I would think a pdf would be best. | 14:38.09 |
| tkamppeter | henrys, I am not sure how much the X11 device is still used. Usually nowadays one uses frontends like evince or Okular, but I do not know whether these use the X11 device or another. | 14:38.18 |
| Robin_Watts | HTML can be printed to pdf. | 14:38.23 |
| | Or... in the twiki... | 14:38.30 |
| henrys | Robin_Watts: yes that is fine. | 14:38.46 |
| | rayjj: sorry about the text. | 14:39.00 |
| | mattchz: how are the bugs coming along? Anything you need? | 14:40.37 |
| mattchz | not too bad thanks | 14:40.55 |
| | some Iâm having trouble reproducing, although I think they may be memory related. | 14:41.15 |
| jogux_mac | mattchz: did you get buzilla perms to close bugs? :) | 14:41.19 |
| henrys | mattchz: if you need a reasonably priced device for reproducing something just buy it a bill us. | 14:41.29 |
| mattchz | henrys> thanks! | 14:41.53 |
| | I think weâre okay for now | 14:42.05 |
| rayjj | chrisl: I just realized that an email I was going to send last night didn't go out. | 14:42.08 |
| mattchz | jogu> no, not got the perms yet. | 14:42.09 |
| | I fixed one bug and committed the fix. | 14:42.16 |
| mvrhel_laptop | henrys: sorry I am a bit late | 14:42.17 |
| jogux_mac | we have a reasonably complete set of iOS kit, but there's 4 billion android devices :( | 14:42.19 |
| henrys | mvrhel_laptop: no problem | 14:42.28 |
| chrisl | rayjj: is that something I need to worry about? | 14:42.36 |
| mattchz | I managed to reproduce the latest bug Iâm looking at on an iPad 2 | 14:42.44 |
| | (which has less RAM than the newer models) | 14:42.50 |
| jogux_mac | henrys: who can give matt more bugzilla permissions so he can close bugs etc? | 14:42.57 |
| Robin_Watts | jogux_mac: 6178 | 14:43.05 |
| henrys | jogux_mac: yup right after the meeting | 14:43.19 |
| Robin_Watts | (at least, according to google play, that's the number of devices that MuPDF can run on) | 14:43.48 |
| henrys | jogux_mac: I always take a few minutes to fumble through that ;-) | 14:43.49 |
| rayjj | chrisl: no, it's just been sitting while I was working on the performance issue, and it should be easy enough to look into (maybe with help from mvrhel). I picked on you since you have experience with their simulators | 14:43.53 |
| henrys | how about that US win yesterday? Are you guys folling the cup? | 14:44.25 |
| | s/folling/followin | 14:44.41 |
| chrisl | rayjj: I just saw it - I can take a look if you like, but a) it's not an area I know anything about, and b) I think my sim is rather out of date and c) it will probably have to wait until tomorrow...... | 14:44.42 |
| henrys | g | 14:44.43 |
| rayjj | chrisl: and, AFAIK, Robin is the only other person with experience with cust 532's sim and he's having too much "fun" with SOT | 14:45.03 |
| mattchz | SOT=? | 14:45.21 |
| Robin_Watts | henrys: No, not following. | 14:45.26 |
| | SOT = Smart Office (Technologies or Two depending on context) | 14:45.49 |
| henrys | mvrhel_laptop: I did want to ask about a gsview alpha - just throw it out there and see if we get some feedback now that we have the domain? | 14:45.52 |
| mattchz | ah, of course! | 14:45.57 |
| Robin_Watts | SOG = Smart office for Good. | 14:45.57 |
| tkamppeter | henrys, a problem of the display/GTK alternative is, that it is not modularized, as X11. The X11 driver is put into a separate binary package in distros, so that one can install GS on a headless server without installing tons of X libraries. | 14:46.12 |
| rayjj | chrisl: I understand that it didn't get there when I intended. Thanks. If I have airport/airplane time and finish the performance improvements, I may look into it, and I'll let you know. | 14:46.22 |
| mvrhel_laptop | henrys: we could do that. I would prefer if any windows users here give it a try. I also want to add it as an option into the bug tracker if we could. there are a couple things I stumble upon over the last couple weeks using it that I want to get fixed | 14:46.58 |
| chrisl | rayjj: tbh, this looks like something that will need addressed in the master code? | 14:47.18 |
| rayjj | chrisl: agreed. | 14:47.31 |
| mvrhel_laptop | but I am not working on it now. working on v2 icc stuff for kens | 14:47.34 |
| | hope to have that wrapped up this week | 14:47.41 |
| chrisl | rayjj: I'll have a look at it on master before I finish today - that's easier/quicker than using the sim | 14:48.09 |
| Robin_Watts | rayjj: Was your (x0, y0, x1, y1) -> (x0, y0, w, h) change reverted in the end? | 14:48.12 |
| henrys | tkamppeter: and we canât do that with gtk? | 14:48.16 |
| mvrhel_laptop | is adding it to the bug tracker something that marcosw can do henrys? I don't know where/how that is updated | 14:48.28 |
| rayjj | Robin_Watts: ? I don't know what you mean | 14:48.48 |
| henrys | mvrhel_laptop: Iâll do it after the meeting | 14:49.02 |
| mvrhel_laptop | ok thanks henrys | 14:49.11 |
| Robin_Watts | rayjj: You changed a tile_by_steps call to improve performance in J10. | 14:49.12 |
| | But the customer pointed out that it then rendered incorrectly. | 14:49.27 |
| henrys | mvrhel_laptop, kens: are we wrapping up color pdfwrite improvements? | 14:49.51 |
| chrisl | henrys: do we really need another gs project right now (fixing-up the gtk display exe)? | 14:50.27 |
| kens | henrys there's a couple new bug reports recently, not had time to look at them. In terms of new functionality, there'll be an ongoing project for me to add more 'stuff', like PDF/X-1 | 14:50.33 |
| tkamppeter | henrys, if someone of you will do this modularization, it would be OK, but we also need to find out whether evince and Okular need the x11 device. | 14:50.50 |
| rayjj | Robin_Watts: it renders correctly on their sim, so the problem is somewhere else. | 14:50.52 |
| Robin_Watts | rayjj: Right. | 14:51.07 |
| henrys | chrisl: probably not I just consider X11 to be such a bad first impression. | 14:51.18 |
| rayjj | Robin_Watts: it's clearly a bug and it works fine on the master code (commit 8269dd2b) | 14:51.40 |
| Robin_Watts | rayjj: Ah, OK, I'd not seen that bit of information in the email exchanges. | 14:52.07 |
| henrys | chrisl: but youâre right we have to much going on right now, Iâll leave it in the agenda though | 14:52.10 |
| Robin_Watts | (I probably just missed it) | 14:52.14 |
| chrisl | henrys: we couldn't remove x11 and co anyway since at least gv uses it, IIRC | 14:52.33 |
| tkamppeter | henrys, evince seems to use the x11 device, see bugs.ghostscript.com/show_bug.cgi?id=694979. | 14:52.58 |
| rayjj | Robin_Watts: I am waiting for an updated simulator from Len | 14:53.18 |
| mvrhel_laptop | kens: I am making the v2 color stuff very easy for you. you will just need to do a single call to get the v2 buffer of data and its size. no need to do any allocations for cleanups. Ideally you will just be able to write out that buffer. | 14:53.27 |
| kens | mvrhel_laptop : sounds good to me :-) | 14:53.43 |
| kens | likes easy | 14:53.50 |
| henrys | tkamppeter: ugh not likely X11 doesnât even have scrollbars | 14:53.58 |
| mvrhel_laptop | the v2 data will be maintained in the profile and cleaned up when the profile is released | 14:53.59 |
| kens | mvrhel_laptop : that sounds great, nice clean memory management. I assume if the v2 profile is already there it just gets handed back, no re-converted | 14:54.32 |
| mvrhel_laptop | yes | 14:54.38 |
| kens | :-) | 14:54.46 |
| chrisl | henrys: given that gv uses the x11 device, there's no reason why other apps aren't using it the same way | 14:54.53 |
| henrys | gv is another terrible introduction to ghostscript | 14:55.28 |
| tkamppeter | henrys, that seems all to be done by evince, and if the user moves in the document, evince runs GS to gwnerate the needed piece of the page to display. | 14:55.35 |
| kens | rayjj : looking at the rotate pages bug, it appears that if a page has /Rotate other than 0 then we rotate it with PDFFitPage, and do it again to apply the Rotation. I'll play with that at some point | 14:56.04 |
| chrisl | henrys: it all stems from the original, long dead "Ghostview" | 14:56.05 |
| rayjj | the changes I am working on for cust 532 to use nbands > 1 for pattern-clist is close, and gives a 3x performance improvement on the J10 test file (on the simulator). | 14:56.18 |
| mvrhel_laptop | another reason to think about getting gsview up and running in linux... | 14:56.27 |
| rayjj | kens: OK. Thanks. | 14:56.33 |
| henrys | tkamppeter: hmm Iâll have to loook at that. | 14:56.36 |
| chrisl | mvrhel_laptop: I suspect most Unix users will want a GUI for Ghostscript..... | 14:57.10 |
| henrys | I really do think it is important that we have modern viewing experiences on all the platforms. Windows now has mvrhel_laptopâs gsview. What to do about linux and mac is the next problem. | 14:57.46 |
| mvrhel_laptop | I would not mind tackling the one of those other platforms but perhaps after the SOT release | 14:58.24 |
| | s/the one/one/ | 14:58.31 |
| Robin_Watts | mvrhel_laptop: or before it, after you've seen the code :) | 14:58.44 |
| mvrhel_laptop | :) | 14:58.49 |
| | henrys: I need to step out for a bit. today is my daughters birthday. need to help get her out the door to school | 14:59.24 |
| henrys | thatâs all I had for gs and mupdf. | 14:59.32 |
| mattchz | This will be the first 2.0 release of SO, right? Picsel got to about 1.9 iirc | 14:59.34 |
| mvrhel_laptop | bbiab | 14:59.49 |
| Robin_Watts | mattchz: Smart office 2.0 was in the app stores. | 14:59.58 |
| rayjj | kens: that segfault with Bug688845.eps.pkmraw.300.1 that showed up with my commits yesterday seems to have gone away. I couldn't reproduce it on linux or windows | 15:00.02 |
| chrisl | The problem is, anything we do "better" than x11 and co will bring more dependencies and more hassle..... | 15:00.06 |
| kens | rayjj just a weird indeterminism then | 15:00.15 |
| pedro_pc2 | mattcz> 2.1.28 | 15:00.22 |
| rayjj | mvrhel_laptop: when is school out ? | 15:00.27 |
| henrys | do we want to have an SOT meeting? If so Iâd like to take about 5 minutes before starting? | 15:00.43 |
| jogux_mac | would be keen to do a better mac UI for mupdf btw. | 15:00.49 |
| Robin_Watts | jogux_mac: gsview uses mupdf and gs together. | 15:01.12 |
| jogux_mac | oh right, interesting. | 15:01.25 |
| kens | THat's the *new* gsview, not the old one :-) | 15:01.39 |
| Robin_Watts | specifically PDF viewing is done using mupdf, PS is distilled to PDF using gs and then viewed with MuPDF (I think that's right) | 15:01.41 |
| kens | thinks so | 15:01.52 |
| Robin_Watts | yes, sorry, this is mvrhel's new gsview. | 15:01.54 |
| jogux_mac | wasn't really volunteering to go near gs, but isn't fundamentally adverse to doing so :) | 15:02.00 |
| tkamppeter | henrys, one could perhaps proceed as following: First, bring display/GTK into a distro-acceptable form (modularization), second, deprecate (but not remove) x11 and tell the users (evince, Okular, ...) about this, give them enough time to switch over, only after that remove x11. | 15:02.02 |
| Robin_Watts | In an ideal world we'd just make gsview work on linux and mac. | 15:02.12 |
| | but sadly, that's not trivial. | 15:02.23 |
| tkamppeter | henrys, another problem is how well it will work with Qt-based Okular using GTK-based GS display device. | 15:02.56 |
| chrisl | tkamppeter: what do you mean by "modularisation"? | 15:03.12 |
| Robin_Watts | gsview uses C# (which is not in itself a problem), but the underlying WPF (windows presentation framework) stuff that it uses doesn't run on mac or linux. | 15:03.18 |
| | BUT... Silverlight provides the WPF functions. | 15:03.31 |
| | and so does Moonlight, apparently. | 15:03.47 |
| | and moonlight runs on macs and linux. | 15:03.56 |
| | so there is potential for us making gsview cross platform using that. | 15:04.09 |
| henrys | the only item I had for SOT is try to keep anything non proprietary on IRC (not skype). Itâs logged and more avaiable to other staff members. | 15:04.18 |
| Robin_Watts | (if you say it quickly, it sounds easy :) ) | 15:04.25 |
| chrisl | Robin_Watts: like I said - a boat load more dependencies :-( | 15:04.51 |
| Robin_Watts | henrys: yes, we're bad about that. We should try harder. | 15:04.58 |
| | chrisl: Indeed. but for gsview not for gs itself. | 15:05.10 |
| henrys | Iâll be back in 5. | 15:05.24 |
| chrisl | Robin_Watts: but that's not replacing the "first impression of gs" then | 15:05.35 |
| Robin_Watts | chrisl: It is if we can get it on steam. | 15:05.48 |
| rayjj | chrisl: can't make an omelette without dependencies on eggs ;-) | 15:05.49 |
| Robin_Watts | cos then people can have a one click download/install/run. | 15:06.04 |
| jogux_mac | henrys: the only comment I'd have is that if some of you guys are full flow in a gs problem, and we're full flow in a SOT stuff, it starts to get difficult to keep up / pick out the right bits with irc :-) | 15:06.24 |
| chrisl | Robin_Watts: I doubt we'll get Ubuntu/Debian/Fedora to start pulling core packages from Steam...... | 15:06.28 |
| pedro_pc2 | henrys> I have one outstanding crash bug to sort with the Android SOG - a pre-existing issue, and I've been testing the app against the Good pre-Veracode checklist to make sure there's nothing obvious | 15:06.43 |
| Robin_Watts | chrisl: Indeed not, the steam gsview would be self contained. Death to shared libs! :) | 15:06.54 |
| tkamppeter | mvrhel_laptop, hi | 15:07.07 |
| chrisl | Robin_Watts: and that leaves us with x11 for most LInux/Unix users...... | 15:07.17 |
| jogux_mac | robin_watts: moonlight I think presents via x11 on mac os x, if so it's not a good option :-( | 15:07.19 |
| pedro_pc2 | there are some differences in spec between our iOS and Android builds though, so we should really synchronise functionality when we release android (or constrain the android one a bit for now) | 15:07.37 |
| Robin_Watts | jogux_mac: It's a better option than we have now. | 15:07.50 |
| pedro_pc2 | henrys> have you had anything back from Good about Veracode? I haven't seen anything | 15:08.13 |
| jogux_mac | robin_watts: only just ;-) or at least native mac would be so so much nicer / more usable. | 15:08.17 |
| rayjj | mvrhel_laptop: (for the logs) you probably have already heard from everyone else, but gsview installs and works properly for me now. | 15:08.54 |
| Robin_Watts | jogux_mac: Yes. The trick would be to try and reuse as much as possible between platforms without going down the TGV-like rabbithole. | 15:09.08 |
| tkamppeter | chrisl, modularization is having an output device (or any other component of GS) in a separate binary file (usually a shared library), so that this module can have extra dependencies which the main part of GS does not have. | 15:09.26 |
| jogux_mac | robin_watts: nod. I did wonder if anything might be shareable between ios & mac. | 15:09.40 |
| rayjj | anything for me ? I have to run some errands and finish packing (head to the airport in 6 hrs) | 15:09.58 |
| | henrys: ^ (or anyone else) | 15:10.20 |
| Robin_Watts | rayjj: I think henrys has gone for 5 mins, and had hence closed that part of the meeting. | 15:10.28 |
| | Have a good holiday! | 15:10.31 |
| kens | enjoy vacation rayjj | 15:10.36 |
| chrisl | tkamppeter: I don't think that's necessary for the gtk display - the gtk dependencies is in the executable, not libgs.so | 15:11.03 |
| rayjj | Robin_Watts: thanks. I'll be connected (and working some) while there. | 15:11.04 |
| | kens: thanks | 15:11.19 |
| pedro_pc2 | rayjj: yep, have fun ! | 15:11.56 |
| rayjj | thanks, pedro_pc2 | 15:12.07 |
| mattchz | enjoy | 15:12.19 |
| rayjj | we are ALL looking forward to it. Thanks, matt | 15:12.35 |
| tkamppeter | chrisl, OK, then the main package will contain libgs.so and gsc and the extra package for display support will contain gsx. Am I correct? | 15:12.46 |
| chrisl | tkamppeter: indeed, yes. | 15:12.59 |
| tkamppeter | chrisl, henrys, this reduces the problem to the current viewers (evince and Okular) having to move over to display. | 15:13.27 |
| chrisl | tkamppeter: the remaining problem is that the gtk display uses deprecated gtk functions, is missing some functionality it ought to have, and really needs time spent "tidying up" - time which I haven't had, as yet :-( | 15:14.39 |
| Robin_Watts | Could we provide gsx that uses X and gsg that uses gtk ? | 15:15.30 |
| | and then tell people to use gsg rather than gsx? | 15:15.43 |
| kens | And what will prompt them to change ? | 15:15.55 |
| Robin_Watts | packaging. | 15:16.01 |
| kens | Hmm..... | 15:16.09 |
| | I doubt they ever will change | 15:16.15 |
| tkamppeter | chrisl, then we have to go the plan I mentioned, but "making distro-ready" is polishing up display to use up-to-date GTK API and bring it to the needed feature completeness. | 15:16.17 |
| henrys | tkamppeter: Iâm really curious how okular or evince uses x11 and wonder if it is not just a mistaken dependency. | 15:16.25 |
| chrisl | henrys: look in the devx11 code, there's stuff in there about using existing X11 windows | 15:17.05 |
| tkamppeter | henrys, you can see the actual GS command lines which evince uses (or libspectre which is used by evince) in bugs.ghostscript.com/show_bug.cgi?id=694979. | 15:17.27 |
| mvrhel_laptop | back | 15:18.47 |
| tkamppeter | mvrhel_laptop, will you (or someone else of the GS team) come to the OpenPrinting Summit? | 15:20.52 |
| chrisl | henrys: if you look in gdevxini.c around line 117 you'll see how applications can "hook" the x11 device window for their own purposes | 15:21.59 |
| henrys | chrisl: okay I had no idea this was an issue. | 15:22.21 |
| mvrhel_laptop | tkamppeter: we have not talked about it yet. | 15:23.15 |
| chrisl | henrys: it's a stupid, horrible hack presumably because whoever did it couldn't be bother/wasn't able to write their own device - and I assume it predates the display device | 15:23.23 |
| henrys | chrisl: obviously these open source developers have a stronger stomach than I | 15:23.49 |
| mvrhel_laptop | it is in august this year. I really dislike traveling then since the weather is so nice here at that time of year | 15:23.50 |
| chrisl | henrys: the problem is, once "GhostView" did it, everyone else probably just copy'n'pasted the code from there..... | 15:24.36 |
| henrys | tkamppeter: Iâll ask the rest of the staff but that is prime vacation time. | 15:24.53 |
| | tkamppeter: have you been keeping an eye on mopria? Do you think that will go anywhere? | 15:26.52 |
| tkamppeter | henrys, mvrhel_laptop, the move to August is hopefully an exception for only this year. It is because I got banned from entering the US and so had to move the Summit to the PWG Meeting in Canada, which happens to be in August. | 15:27.30 |
| henrys | tkamppeter: I remember you having problems once but I didnât know you were banned. How does that happen? | 15:28.44 |
| tkamppeter | henrys, I have no idea. They never told me why. Currently a specialized lawyer is trying to get information. | 15:31.58 |
| mattchz | robin> Iâve been thinking about this memory thing (and looking at the details of fz_store()) | 15:36.49 |
| | We donât get told by iOS how much memory we should free. | 15:37.03 |
| | One option would be to just release say 50% of the store size | 15:37.28 |
| Robin_Watts | mattchz: Ah. We could try to free half the store then? | 15:37.30 |
| mattchz | :) | 15:38.01 |
| Robin_Watts | Or we could be smarter about setting up the maximum size of the store to start with. | 15:38.16 |
| mattchz | thing is, presumably as soon as we start flipping pages, weâll start consuming memory again. | 15:38.25 |
| Robin_Watts | the problem is, not everything goes in the store. | 15:38.29 |
| jogux_mac | chz> I'd say 50% for level 1 warning, everything for level 2 warning? | 15:38.41 |
| mattchz | are there multiple levels? | 15:38.56 |
| | Iâm just wondering what will happen if we very quickly end up allocating the memory again. | 15:39.14 |
| | Will the OS even notice that weâve freed it, or will it kill us? | 15:39.27 |
| jogux_mac | chz: hm, I thought there were multiple levels. can't find the reference now. | 15:39.43 |
| | chz: yes, OS should notice (assuming our frees results in some whole pages being freed) | 15:39.55 |
| mattchz | and if so, might we be better also adjusting the maximum store size too, so that it doesnât get larger in the future? | 15:39.58 |
| Robin_Watts | surely the OS will check the apps usage when it returns? | 15:40.03 |
| | mattchz: That is a possibility, yes. | 15:40.11 |
| | but as I say, changing the max on the store, does not set a max for the app as a whole. | 15:40.35 |
| UAjim | Hello. I am trying to download gs914w64.exe (http://downloads.ghostscript.com/public/gs914w64.exe) but the connection keeps timing out. Is it okay to report that in this room or should I go elsewhere? Thanks! | 15:40.53 |
| Robin_Watts | If you wanted to keep track of the apps total usage, then you'd need to look at the malloc/free/realloc functions passed into fz_new_context | 15:41.16 |
| | UAjim: This is the right place. | 15:41.32 |
| UAjim | Okay, great. | 15:41.41 |
| mattchz | Iâm not sure if would make any difference. Iâm just wondering if we should half the store size each time we get a warning | 15:41.46 |
| Robin_Watts | UAjim: Works for me :( | 15:42.09 |
| mattchz | so that it works dynamically if the app has a lot of non-store data in RAM too. | 15:42.43 |
| jogux_mac | matt: I'm utterly wrong about the different levels | 15:42.54 |
| UAjim | Hmm. Okay thanks. I did not work for me or my boss. Perhaps it has something to do with the Unverisity that I work at. Thank you your assistance Robin)Watts | 15:43.01 |
| mattchz | jogux> ah :) | 15:43.05 |
| jogux_mac | matt: the documentation says "Your implementation of this method should free up as much memory as possible by purging cached data objects that can be recreated (or reloaded from disk) later" | 15:43.06 |
| | matt: it does display different levels in the console log though, I believe. | 15:43.20 |
| mattchz | indeed; that would be bad for performance. | 15:43.23 |
| Robin_Watts | jogux_mac: Oh, that sounds like "purge the entire store". | 15:43.32 |
| mattchz | to throw everything out. | 15:43.34 |
| | iâd assume weâd be much better limiting our store size so that we donât have to throw anything out | 15:43.57 |
| Robin_Watts | mattchz: The problem is, what can trigger this? | 15:43.57 |
| jogux_mac | matt: getting killed by the OS is /really/ bad for performance :-) | 15:44.07 |
| mattchz | oh, indeed. | 15:44.16 |
| Robin_Watts | If it can be triggered by the app going into the background, then we have no idea if the cache size was unreasonable. | 15:44.38 |
| mattchz | Iâm just wondering if throwing it all away is the best solution (rather than limtiting the size) | 15:44.55 |
| jogux_mac | the memory warning can be triggered in the background - if we're in the background we should definitely throw everything away, no question. | 15:45.03 |
| Robin_Watts | i.e. we might be using a small fraction of available memory, and some other task might get foregrounded. | 15:45.17 |
| mattchz | robin> in this case, itâs triggered by panning through the pages of a document | 15:45.18 |
| Robin_Watts | in that case, we should NOT update the store size. | 15:45.26 |
| jogux_mac | I don't think there's any sane solution involving guessing at a store size tbh | 15:45.43 |
| mattchz | Does mupdf actually run in the background? | 15:46.05 |
| jogux_mac | I think so | 15:46.15 |
| | open a pdf, put it into the background, pull it back into foreground, and see if the same document is on the display | 15:46.35 |
| | then try same again killing the app inbetween | 15:46.41 |
| Robin_Watts | Possibly we should always throw away the whole store, and in the case where we are in the foreground as we do it, set the maximum store size to min(max_store_size, current_store_size)- 1Meg. | 15:46.41 |
| jogux_mac | is the purge done on a last used last out basis? | 15:47.39 |
| Robin_Watts | If there is a reasonable way to read the total memory in the device when we start up, we should set the maximum store size to 3/4 of that. | 15:47.50 |
| | it is. | 15:47.53 |
| | (Least Recently Used. This is just like ImageCache and CacheMan.) | 15:48.29 |
| jogux_mac | would just say purge all in background, purge 50% in foreground. | 15:48.44 |
| mattchz | at the moment, the store size is hardcoded to 128Mb | 15:48.50 |
| jogux_mac | it's absolutely expected that apps should get memory warnings, the system also caches all sorts of stuff which it throws away on a memory warning. | 15:49.42 |
| Robin_Watts | jogux_mac: I'd favour at least *trying* to learn a good cache size from the evictions in the foreground. | 15:49.47 |
| | Do we ever get 2 warning calls in a row? | 15:50.18 |
| jogux_mac | robin_watts: potentially, yes. | 15:50.27 |
| Robin_Watts | i.e. could we bin 50% on the first, and then if we get another one before we've done anything else, bin all of it ? | 15:50.38 |
| jogux_mac | [potentially you could get 2 warning calls in a row] | 15:50.38 |
| Robin_Watts | where "anything else" = "any more allocations/deallocations" | 15:51.18 |
| jogux_mac | robin_Watts: the issue I have with learning is that there's all kinds of ways we can learn wrong. | 15:51.26 |
| Robin_Watts | jogux_mac: True. | 15:51.41 |
| jogux_mac | eg. if the user has a background app that uses a lot of memory temporarily whilst we're running, we could then get stuck in a 'crap performance' mode unpredictably | 15:51.54 |
| Robin_Watts | I'd be tempted by a strategy where we can learn upwards as well as downwards. | 15:52.11 |
| | but that's harder. | 15:52.24 |
| | OK, skip learning for now. | 15:52.28 |
| jogux_mac | :-) | 15:52.32 |
| | you've saved me bringing out my KISS flag | 15:52.38 |
| Robin_Watts | jogux_mac: That's pretty much the mupdf mantra :) | 15:52.53 |
| mattchz | :) | 15:53.39 |
| | This is bringing horrible flashbacks back. | 15:53.46 |
| Robin_Watts | Day #2 with MuPDF and mattchz is playing with locks already :) | 15:54.52 |
| pedro_pc | lets not go there... | 15:55.38 |
| mattchz | I remember trying to deal with exactly this problem in ePAGE, not on iOS though ;) | 15:56.59 |
| Robin_Watts | mattchz: Cacheman told you how much to free though, I think. | 15:57.28 |
| mattchz | it looks like we donât take any UIBackgroundTask handles. So Iâm guessing we donât run in the background, but we do stay resident when in the background | 15:57.50 |
| | robin> yeah, I think this was pre-Cacheman | 15:57.57 |
| | or possibly not, itâs all a haze ;) | 15:58.11 |
| | so, presumably that means we get woken up when in the background but only if memory is low. | 15:58.35 |
| paulgardiner | We use 4 screen-sized bitmaps. I wonder what the percentage split is between memory used there and in the core. | 15:58.55 |
| mattchz | so, 50% when in the foreground, 100% when in the background? | 15:59.53 |
| Robin_Watts | seems a reasonable guess. | 16:00.04 |
| paulgardiner | We could cut down the resolution of 3 of those bitmaps. They are used while waiting for the hq render. | 16:00.34 |
| Robin_Watts | and (if it can be done easily), 100% in the foreground if we get called twice without an intervening malloc/realloc | 16:00.37 |
| mattchz | Usually thereâs a fair gap between the calls, so I donât think thatâs very likely. | 16:01.20 |
| Robin_Watts | paulgardiner: So you'd keep the thumbnail bitmaps at a lower resolution and scale them up? | 16:01.23 |
| mattchz | but possible, I guess | 16:01.40 |
| paulgardiner | Yes. They get scaled up at the moment other than when the page fills the screen | 16:01.45 |
| | But it all depends on whether the core or these bitmaps are the biggest memory hogs | 16:02.18 |
| | And I have no idea about that | 16:02.36 |
| Robin_Watts | mattchz: Are we getting this callback from the OS "within" a malloc call? | 16:03.17 |
| mattchz | highly unlikely. | 16:03.27 |
| paulgardiner | It could be that the core always has to run in a tiny bit of memory left after the bitmap allocations, or it could be that the bitmaps are only a small proportion and have little effect. | 16:03.35 |
| Robin_Watts | ok. | 16:03.37 |
| mattchz | I suspect it could be in response to a malloc, but possibly one from a different process. | 16:04.00 |
| | paul> are the bitmaps a fixed size? | 16:05.26 |
| Robin_Watts | The worry I had was that MuPDF might try to do a large malloc, and rather than being told "no", and then doing it's own purge, the OS might immediately be calling us back to say "purge". | 16:05.48 |
| | mattchz: Can you resize an ipad's screen? :) | 16:06.03 |
| mattchz | err, what I mean, I guess, is are they statically allocated? | 16:06.24 |
| | I canât imagine we can free them even if we want to | 16:06.40 |
| Robin_Watts | They are allocated on the heap. | 16:06.48 |
| | but we keep them around forever. | 16:07.13 |
| | actually, that can't be right. | 16:07.51 |
| | AIUI, for each page, we keep a bitmap that represents that page rendered so it 'just' fills the screen. | 16:08.21 |
| | so according to the aspect ratio of the screen vs the page, the bitmaps might be a tad smaller than the screen. | 16:08.50 |
| | And we then have 1 bitmap that we use for the HQ rendering of the current page. | 16:09.29 |
| | We hold thumbnails for current page, previous page and next page (i.e. 3) | 16:09.53 |
| | and full for just current page. hence 4 bitmaps in total. | 16:10.06 |
| Robin_Watts | is explaining this really badly, I fear. | 16:10.14 |
| mattchz | I think I know what you mean :) | 16:10.35 |
| Robin_Watts | hence when we flip page, we destroy/recreate a thumbnail and the HQ. | 16:11.00 |
| | It's possible paulgardiner has been smart and avoided the churn there by reusing the existing bitmaps. | 16:11.22 |
| paulgardiner | We are reusing them, or intending to at least | 16:11.46 |
| Robin_Watts | Is that "We reuse them if the size of the page is the same"? Or "We always reuse them"? | 16:12.15 |
| paulgardiner | For that reason we don't account for the page aspect ratio (which would sometimes allow slightly smaller bitmaps) and instead always make them screen sized (always big enough) | 16:12.26 |
| mattchz | perhaps the safest thing to do might be just to throw away the entire store when we get a warning. | 16:12.30 |
| Robin_Watts | Presumably for the latter, they would need to always be allocated to be the size of the screen. | 16:12.31 |
| | OK. | 16:12.33 |
| | mattchz: Possibly that should be the first thing you test :) | 16:12.44 |
| mattchz | :) | 16:12.59 |
| paulgardiner | I think knowing the percentage of screen to core memory would be very useful | 16:13.10 |
| Robin_Watts | mattchz: fz_empty_store(ctx) then. | 16:13.26 |
| | no locking or anything to worry about. it's all done for you. | 16:13.37 |
| paulgardiner | ... by core meaning mupdf library | 16:13.50 |
| mattchz | bonus :) | 16:13.50 |
| | I presume thereâs no bound to the overall amount of heap allocated? | 16:14.07 |
| Robin_Watts | By MuPDF? no. | 16:14.20 |
| mattchz | because, if so, there wonât be a solution which prevents us ever getting killed | 16:14.27 |
| Robin_Watts | But... you could implement one. | 16:14.29 |
| | When you call fz_new_context, you pass in a set of allocation pointers. | 16:14.47 |
| | and all calls to fz_malloc/fz_free etc, end up going through those pointers. | 16:15.03 |
| | You can therefore keep track of allocations/deallocations there, and impose a total limit if you feel it's required. | 16:15.34 |
| mattchz | Iâm not sure it would help overall; we canât necessarily predict what is a sensible maximum, especially if weâre in the background | 16:15.59 |
| Robin_Watts | The general approach to MuPDF is to keep implementation decisions like this out of the core. | 16:16.12 |
| mattchz | nod. | 16:16.33 |
| Robin_Watts | It would help you to gather pauls allocated/screen ratio. | 16:16.33 |
| mattchz | sorry, Iâm slightly confused now⦠| 16:17.41 |
| | is this total memory allocated? | 16:17.49 |
| | and is the store involved here? | 16:18.17 |
| Robin_Watts | mattchz: Paul is interested in total memory allocated vs memory allocated just in those bitmaps, I think. | 16:18.25 |
| | If the bitmaps are a huge contributor, then we should take steps to reduce them. | 16:18.48 |
| | But I think regardless of whether they are or not, we need to cope with the messages from the OS. | 16:19.07 |
| | so trying a 'empty the store on a warning from the OS' seems a reasonable first step. | 16:19.30 |
| | certainly that should solve the crashes. | 16:19.35 |
| mattchz | ok, cool. | 16:19.40 |
| paulgardiner | Makes sense to me | 16:19.54 |
| Robin_Watts | The objects that go into the store are allocated through the normal malloc/free functions. | 16:20.06 |
| | i.e. the store is a subset of the 'core' memory use. | 16:20.20 |
| | fz_malloc/fz_free I mean. | 16:20.39 |
| | marcosw: Greetings. Can you give mattchz permissions to close bugs on bugzilla etc please? | 16:21.01 |
| kens | heads off to cook | 16:21.19 |
| marcosw | sure | 16:21.21 |
| kens | goodnight all | 16:21.23 |
| Robin_Watts | marcosw: mattchz is another emobixian doing mupdf stuff for us. | 16:21.41 |
| paulgardiner | Oh iOS! Some of my comments were about the Android app. I'm not sure of the bitmap allocation strategy in the iOS app. Possibly we reuse, possibly not. | 16:21.43 |
| mattchz | hi marcosw | 16:21.55 |
| Robin_Watts | mattchz: marcosw is the guy that handles most of our support stuff for us. | 16:22.00 |
| marcosw | hello mattchz | 16:22.09 |
| Robin_Watts | He also does lots of the unix admin stuff, and looks after the cluster. | 16:22.20 |
| mattchz | paul> Iâm even more worried about the android app. One of the crashes there involved recycled java Bitmaps :) | 16:22.25 |
| Robin_Watts | He has the most hi-tech power hungry garage in california :) | 16:22.32 |
| mattchz | cool :) | 16:22.52 |
| pedro_pc | you can see the glow from space ;) | 16:22.53 |
| mattchz | whatâs in it? | 16:22.53 |
| | heh | 16:23.15 |
| Robin_Watts | grow lights. No, sorry, urm... servers, yes, servers. | 16:23.19 |
| mattchz | hehe | 16:23.25 |
| paulgardiner | mattchz: that might be a bug in an earlier version that we haven't closed | 16:23.31 |
| mattchz | ah right. I know in one of my previous projects, I had some real problems with bitmaps, and making sure they were recycled properly. | 16:24.11 |
| | I think the improved it massively in later Android versions | 16:24.21 |
| Robin_Watts | mattchz: Yes. That bit of the code has been tweaked many times. We *think* we're OK at the moment. | 16:24.51 |
| mattchz | perhaps the bug was just an outdated one | 16:25.16 |
| Robin_Watts | The problems of making a java object have the same lifespan as a native object are interesting ones. | 16:25.44 |
| mattchz | I *think* in later android versions they changed the bitmaps to be allocated on the Java heap, which solved many of the issues. | 16:26.08 |
| Robin_Watts | That's exactly the problem I have with MuPDFCore and the underlying globals object. | 16:26.13 |
| mattchz | is this because finalizers can be asynchronous? | 16:26.26 |
| Robin_Watts | mattchz: Partly, yes. | 16:26.37 |
| | Significantly, I think. | 16:26.50 |
| | I believe I've solved that in MuPDF (except for the need to check for glo being NULL etc). | 16:27.08 |
| mattchz | and presumably before they changed Bitmaps to be on the java heap, native allocations didnât trigger a GC :( | 16:27.09 |
| marcosw | it might take a bit for me to give mattchz bug closing abilities on bugzilla, the internet connectivity sucks at this hotel (which is odd, since miles didn't choose it). | 16:27.17 |
| Robin_Watts | There are similar problems with the more general JNI reflection, but I think those are solved in my branch. | 16:27.36 |
| mattchz | robin> at some point, Iâd love to hear how you solved. | 16:27.42 |
| | I never managed to find a perfect solution. | 16:27.57 |
| Robin_Watts | mattchz: I will look forward to taking the time away from SOT to try to explain. But I shall fear you finding a schoolboy error in my logic :) | 16:28.31 |
| mattchz | unlikely ;) | 16:28.48 |
| jogux_mac | robin_watts : mattchz : forget who asked, but ios low memory notifications are delivered asyncronously to the application's main thread. so nominally could happen whilst another thread is inside malloc, but there's some grace period to deal with the notification. | 16:31.12 |
| mattchz | ah, neat. Yes, I put a breakpoint there, and it certaintly wasnât in a malloc call (not that I expected it to be; pretty much everything like that is pushed onto the run loop) | 16:32.45 |
| Robin_Watts | jogux_mac: Right. I was worried it would be an synchronous callback from the OS when we did a malloc. | 16:34.03 |
| jogux_mac | yeah, definitely not. there's a reasonable buffer it keeps free so it has time to deal with the situation asyncronously. | 16:34.32 |
| Robin_Watts | Cool. Better than linux then. | 16:34.48 |
| jogux_mac | yeah. I like the design tbh. It seems to work well. | 16:35.02 |
| Robin_Watts | supposedly linux will overcommit itself, and just kill the app when you actually try to use the memory. | 16:35.15 |
| jogux_mac | yeah. though I think they've frigged with that on android now. | 16:35.25 |
| Robin_Watts | pedro_pc: So... stupid epage questions that I should know about... | 16:37.21 |
| | (or jogux_mac or mattchz...)( | 16:37.35 |
| | in Edr, each page has a stylesheet. | 16:37.45 |
| jogux_mac | it does? | 16:37.55 |
| Robin_Watts | Is that one stylesheet per page, or one stylesheet per doc ? | 16:38.00 |
| mattchz | I honestly donât remember :(, sorry | 16:38.20 |
| jogux_mac | multiple stylesheets per document; though for office I think one per doc. | 16:38.20 |
| | there's no page level stylesheet afaicr. | 16:38.39 |
| pedro_pc | should be per doc for most of them | 16:38.41 |
| Robin_Watts | pedro_pc: oh, ass. | 16:38.50 |
| | I was hoping per page :( | 16:38.56 |
| pedro_pc | nods - for most formats we just amalgamate all styles into a single stylesheet | 16:39.03 |
| jogux_mac | you /could/ create one stylesheet per page in the DA if you wanted. | 16:39.10 |
| pedro_pc | jogu: html was the only place we used ultiple sytylesheets per doc wasn't it? | 16:39.33 |
| Robin_Watts | We get one set of "10 basic page styles" in the presentation.xml | 16:39.34 |
| jogux_mac | ie. multiple stylesheets in the document, hm. actually, no, I don't know how you'd do that. | 16:39.38 |
| Robin_Watts | and we then get another "10 basic page styles" in the layout. | 16:40.09 |
| | where there are multiple layouts in a document. | 16:40.28 |
| jogux_mac | pedro_pc: think so, yep | 16:40.33 |
| pedro_pc | yup, for the ooxml I'd say we'd be better creating multiple | 16:40.48 |
| | since that's the way its structured | 16:41.01 |
| Robin_Watts | if we create one stylesheet per doc, I think we'd be better off. | 16:41.12 |
| | if we create one stylesheet per *page*, I think we'd be better off. | 16:41.23 |
| jogux | I don't think Edr has a current mechanism where you could type a stylesheet to a page | 16:41.45 |
| | you'd have to do something icky like adding a prefix to all the rule names of 'pageXXXX' | 16:42.02 |
| | I'm not entirely convinced even that would work :( | 16:42.24 |
| Robin_Watts | At the moment we have style rule "TypeSelectors" for pptx that run from Pptx_Dictionary_TxLevel1 to 9 (plus a default) | 16:43.45 |
| | and I need to allow for at least ones defined in presentation.xml and master. So at least 2 sets. | 16:44.12 |
| | but then they can differ for every page. | 16:44.23 |
| | If those 'TypeSelectors' are just ints, then I could just define each combination as a set of 10 rules. | 16:45.12 |
| | And then have a table that says which set to use for each page. | 16:45.32 |
| | And then the entries on each page would use the appropriate set of rules. | 16:46.21 |
| jogux | I think you'd be able to tag the page group with a unique id, then set style selectors that say "group tagged txlevel1 inside a group tagged page1". | 16:46.30 |
| Robin_Watts | Oh, God. And we've got to be able to save this out again :( | 16:47.19 |
| pedro_pc | mm | 16:47.37 |
| jogux | mmhmm :-S | 16:47.39 |
| pedro_pc | its not trivial | 16:47.42 |
| jogux | has never had to poke hard at the export part of the DAs. | 16:48.15 |
| Robin_Watts | All my experience with Edr was with the PDF stuff, so staticly styled, absolutely positioned. | 16:48.58 |
| mattchz | so, the calling fz_empty_store() seems to work nicely: | 16:50.36 |
| | http://cl.ly/image/2L0j112g2k10 | 16:50.36 |
| jogux | matt: cool :-) | 16:51.12 |
| mattchz | obviously, blowing away everything is less than ideal in a case like that. | 16:51.25 |
| | it kinda defeats the point of having a cache. | 16:51.37 |
| jogux | I thought we'd settled on 50% in the foreground? | 16:51.49 |
| Robin_Watts | We said he should see if just dumping it all solved the crashes. | 16:52.05 |
| mattchz | I think we agreed on 100% everywhere in the end. | 16:52.07 |
| | Iâll try it with 50% | 16:52.26 |
| Robin_Watts | but this gives us confidence to try the 50% in the foreground version I think. | 16:52.27 |
| mattchz | it should work well, provided the store memory is the lions share of the overall allocation | 16:52.57 |
| Robin_Watts | mattchz: Ooh. Another thing to try... | 16:53.22 |
| | we could try ditching 1Meg on the first warning, then if we get another ditch 2, then 4 etc? | 16:54.05 |
| | (obviously resetting if we do a malloc or free). | 16:54.27 |
| mattchz | thereâs quite a gap in between warnings, so we might need to be a bit more aggressive than that, e.g. 10Meg, then 20Meg | 16:54.46 |
| Robin_Watts | mattchz: There is a bit gap between warnings if we ditch the whole lot, sure :) | 16:55.09 |
| mattchz | no, there is even if we donât ditch anything | 16:55.20 |
| jogux | robin: I don't think you'll ever get more than two warnings. "it's try". "really try". "cya". | 16:55.27 |
| Robin_Watts | jogux: And we can't know which one of those it is ? | 16:55.41 |
| pedro_pc | I'm not sure how 'quick' a fix we could do for the style hierarchy. My gut feel is that we'd ideally need multiple style selectors for each edr object (or possibly wrap them in groups - icky) and add create stylesheets mapping to each of the page masters/layouts, but I'm just thinking out loud. Preserving the source info is a bit tricky | 16:55.46 |
| jogux | apparently not :( | 16:55.46 |
| | I don't think that algorythm is precisely documented tbh. | 16:56.01 |
| | in certain cases I think you'll only get one. | 16:56.07 |
| mattchz | like most of things, itâs a black box :( | 16:56.20 |
| | http://stackoverflow.com/questions/3414390/memory-warning-level-in-applicationdidreceivememorywarning | 16:57.00 |
| jogux | ped> is my idea of using naming the pages and uses the (existing mechanism) for making style selectors match only groups that are children of certain groups not workable? | 16:57.03 |
| pedro_pc | possibly - haven't figured if that preserves the object/master style independence when we edit. Maybe does, just haven't got my head round it yet ;) | 16:58.11 |
| jogux | yeah, I don't know if it solves the editting problem. I /think/ it's encoding all the information into the edr, which should mean it's okay... | 16:58.41 |
| | of course that doesn't mean it's in any way compatible with the way the ppt agent currently deals with styles :-S | 16:59.10 |
| pedro_pc | mm | 16:59.24 |
| pedro_pc | has to head back now - bfn | 17:01.45 |
| mattchz | wave | 17:01.54 |
| Robin_Watts | Can Edr stylerules refer to other style rules? | 17:09.07 |
| jogux | I don't think so, but you can perhaps achieve that with careful use of selectors. | 17:09.41 |
| Robin_Watts | how so? | 17:09.59 |
| | The editing problem goes away, because we don't offer editing of style rules currently, just editing of individual styles, AIUI. | 17:10.32 |
| jogux | hrm. partly depends how the selectors are currently setup. but, if, say the id field isn't currently being used, then you could setup matches for 'groupname' and 'groupname.id', and both rules would be applied. | 17:11.05 |
| Robin_Watts | jogux: I think we can get definitions in presentation.xml, masters, and layouts | 17:12.01 |
| | slides inherit from layouts which inherit from masters. | 17:13.08 |
| jogux | I think I should probably understand how pptx currently uses stylerules / selectors before commenting further | 17:13.43 |
| Robin_Watts | Any entry on a ppt slide can either be a 'placeholder' entry, or not. | 17:14.15 |
| | 'placeholder' entries essentially just mean "inherit from the layout/master" styles. | 17:14.38 |
| | The "Artifex 2014.pptx" doc is a classic example of this. | 17:15.43 |
| jogux | perhaps you can share an edr dump for the appropriate bit of that, with a <--- this bit needs to do <x> instead type annotation? | 17:16.21 |
| Robin_Watts | We have a 'master' layout that is a PNG across the background of the top of the slide with an artifex logo etc. | 17:16.22 |
| jogux | sadly I don't have time to try and understand it today | 17:16.38 |
| Robin_Watts | sorry, I'll go back to pondering this to make sure I understand it :) | 17:17.04 |
| jogux | :) | 17:17.25 |
| | if you can share edr dump, I'll try and get my head around how pptx currently uses edr styling in the morning | 17:17.39 |
| jogux | needs to head home now; night everyone. | 17:19.37 |
| Robin_Watts | night. | 17:19.43 |
| marcosw | mattchz: you should now be able to close, etc. bugs | 18:06.20 |
| mattchz | coo, thanks marcosw! | 18:06.31 |
| SpNg | can ghostscript extract font files from EPS files? | 18:08.44 |
| | *embedded fonts | 18:08.50 |
| mvrhel_laptop | ok gray and rgb display v2 profiles getting created. now I just need to do the ones with the MLUTs and then do a pile of testing | 18:42.53 |
| | lunch time now | 18:43.39 |
| marcosw | breakfast time now | 18:44.32 |
| mattchz | robin_watts: please would you mind reviewing: http://git.ghostscript.com/?p=user/matt/mupdf.git;a=commitdiff;h=75a6ed82bd6eecd666af224900a7415d89a58c83;hp=d5a22b7c76c7c0f67f9350b2a7a5dc030b212421 | 18:54.06 |
| | [in retrospect, Iâm actuall wondering if I should have reduced the store to 50% of its current size, rather than 50% of the max size - that might work better if there are lots of non-store bits in memory] | 18:54.49 |
| | nice graph showing new behaviour: http://cl.ly/image/3N17443X2y2B | 18:55.27 |
| Robin_Watts | I was thinking 50% of current size. | 18:57.31 |
| mattchz | ok, no problem. Iâll change that tomorrow then. | 18:57.51 |
| Robin_Watts | but graph looks pretty. | 18:58.18 |
| | mattchz: mupdf uses tabs, not spaces. | 18:59.06 |
| | MuAppDelegate.h looks like spaces in the indent to me. | 18:59.28 |
| mattchz | ah right, Iâll fix that too, sorry. | 18:59.30 |
| Robin_Watts | np. Otherwise it looks great. | 18:59.53 |
| henrys | mattchz: did someone already fix your bugzilla perms? looks okay to me. | 18:59.55 |
| mattchz | henrys: yes, marcosw sorted it thanks! | 19:00.58 |
| henrys | great | 19:01.07 |
| mattchz | ok, folks, itâs hometime. | 19:01.34 |
| | cya tomrrow | 19:01.37 |
| Gigs- | I've got a malicious PDF that everyone here just tried to open | 20:31.18 |
| | all the data is in a flate stream, any ideas how to get it out using mutool or gs or something? | 20:31.33 |
| kens | Gigs- : you can use Mupdf to decompress the tnire file, or you can extract the Flate as a binary file and use GS to decomrpess it to a new file. | 20:40.02 |
| henrys | Gigs-: most exploits are through JS, is the problem seen in gs? | 20:41.08 |
| kens | SpNg GS can extract the fonts from an EPS< clearly, since we need to do that to turn it into a PDF file. We don't (AFAIK) supply any tools to do so. Apart form anythign else that would come close to supplying tools to circumvent copyright protection | 20:41.26 |
| Gigs- | I found it, it's under mutool clean to decompress them | 20:41.40 |
| | for some reason I thought that would be under extract | 20:41.48 |
| kens | Gigs- : yes, that decompresses the whole file | 20:41.52 |
| Gigs- | that's fine I can vim it from there | 20:42.00 |
| kens | WIth -d | 20:42.01 |
| Gigs- | it's javascript, somewhat obfuscated | 20:43.37 |
| kens | That's the usual way to put malware in a PDF | 20:43.51 |
| | THough there was a fad for JPEG at one point | 20:44.04 |
| henrys | kens:how do you get jpeg to do anything malicious? | 20:45.38 |
| kens | There was an exploit years ago | 20:45.50 |
| | buffer overrun leading to code execution I think | 20:46.03 |
| henrys | kens: oh you mean a buffer overflow | 20:46.04 |
| | right | 20:46.11 |
| kens | It got wrapped up into PDFs for quite a while | 20:46.32 |
| henrys | kens: up late | 20:50.19 |
| kens | yeah, nothing but football on TV | 20:50.44 |
| | THat's 'soccer' to you :-) | 20:51.02 |
| Gigs- | there's a massive section of what looks like shellcode or something | 20:51.29 |
| henrys | kens: I imagine. My stepson - Dan (sabrinaâs son) is at the games | 20:51.42 |
| kens | really ? In Brazil ? | 20:52.01 |
| kens | knows littel about Javascript | 20:52.45 |
| henrys | kens: yes he plays club soccer now and has played since high school. | 20:52.45 |
| kens | I expect being there is pretty exciting | 20:53.13 |
| henrys | henrys: heâs wearing all this USA clothing/costumes Iâm afraid heâs going to attacked. | 20:53.31 |
| kens | Quite an experience | 20:53.32 |
| | Umm.... I expect there are plenty of other tourists | 20:53.50 |
| | I believe the advice here is if you get mugged, don't put up a fight, give them what they want, and don't shout or scream.... | 20:54.28 |
| henrys | kens: yes probably being a paranoid parent | 20:54.44 |
| kens | Well, it can happen I guess :-( | 20:55.00 |
| | But if he's sensible then he should be fine and have a greaqt time | 20:55.17 |
| henrys | the US win a big deal here | 20:55.29 |
| kens | Did the US win ? I've nopt been following it | 20:55.42 |
| henrys | s/a/was a | 20:55.45 |
| | yeah they beat the team that has knocked them out the last 2x - Guiana, the US is not a football powerhouse as you probably know | 20:56.27 |
| kens | Well you have enough national games of your own | 20:56.48 |
| henrys | yeah like ârealâ football ;-) | 20:57.09 |
| kens | AH, that would be Aussie rules, right ? :-) | 20:57.36 |
| | Or are we talking the original game where the men of two villages formed teams and tried to ge the ball fomr one end of the village ot the other, fights optional ? | 20:58.23 |
| henrys | kens: that sounds like rugby | 20:58.47 |
| kens | Dear me no, Rugby has rules, its a school you know | 20:59.03 |
| | Though I used to play rugby for the school, I always preferred it to football | 20:59.50 |
| | THink I'd better go put the recycling bin out and look for the cat, night all | 21:01.19 |
| henrys | kens: everyone I know who has been playing for a while has been injured in some deforming way one guy is missing part of his ear. | 21:01.33 |
| kens | That does tend to happen, I guess its the lack of armour and helmets | 21:01.56 |
| Gigs- | I have narrowed it down to the Adobe Reader BMP/RLE heap corruption vulnerability CVE-2013-2729 | 21:06.55 |
| | Forward 1 day (to 2014/06/18)>>> | |
IRC Logs