| | <<<Back 1 day (to 2012/02/06) | 2012/02/07 |
| Robin_Watts | tor8: I've had the mupdf customer on skype asking questions all morning. | 12:26.09 |
| | All successfully sorted, except for 2 issues. | 12:26.20 |
| | 1) fz_clone_context is returning the old context, not the new one. Oops. My bad. Will fix. | 12:26.33 |
| | 2) The lock/unlock functions take ctx as an arg. | 12:26.56 |
| | Should they instead take ctx->alloc->user ? | 12:27.10 |
| | I can't think why they'd want ctx except to get to that. | 12:27.55 |
| tor8 | Robin_Watts: I don't understand question 2 | 12:47.09 |
| Robin_Watts | tor8: We have a lock and an unlock function in fz_alloc_context, right? | 12:48.44 |
| tor8 | yes. | 12:48.59 |
| Robin_Watts | Those are currently called with ctx as an arg. | 12:48.59 |
| tor8 | yes. again. | 12:49.05 |
| kens | lunches | 12:49.13 |
| Robin_Watts | I suspect that we should be calling them with ctx->alloc->user | 12:49.19 |
| tor8 | possibly, but that's annoying from a user perspective and exposes struct internals | 12:49.53 |
| Robin_Watts | Sorry? I'd argue exactly the opposite. | 12:50.05 |
| | The user provides the alloc struct, including lock/unlock/user. | 12:50.25 |
| | when those functions get called back, it's user he wants. | 12:50.40 |
| tor8 | fz_lock(ctx) { fz_lock_imp(ctx->alloc->user); } | 12:50.48 |
| Robin_Watts | he doesn't want to have to go hunting through ctx to find it. | 12:50.49 |
| | OK, so I'm a fool who can't read his own code :) | 12:51.25 |
| tor8 | um, yeah, I just read the code :) | 12:51.48 |
| | you do pass the alloc->user to the function pointer already :) | 12:52.04 |
| Robin_Watts | yeah, so the change I was saying we should make was how it was done already. | 12:52.29 |
| dxp2532 | is it possible to file a bug via the bugzilla without making it public in order to prevent disclosure of a vulnerability? | 13:15.53 |
| kens | No. | 13:16.00 |
| | But if you look through Bugzilla you will find other examples of vulnersability reports | 13:16.22 |
| | In general we do not consider Ghostscript to be a major target for exploits | 13:16.42 |
| | And given the tardiness of many distributions in upgrading to less than ancient versions, they seem to agree. | 13:17.17 |
| dxp2532 | is it safe to assume then that GS developers/maintainers are ok with having vulnerabilities as public bugs regardless of patch status? | 13:22.15 |
| kens | We already do. | 13:32.38 |
| | ping Robin_Watts | 13:33.13 |
| | Well duh. A 'vulnerability report' which doesn't actually disclose (even to us) details of the vulnerability..... | 13:40.26 |
| | THey want 'contact details fo the security team'. I'm inclined to mail them back and tell them to post it to Bugzilla. | 13:41.18 |
| | Or just wait and let them release it on the 2nd February | 13:41.35 |
| | 22nd | 13:41.38 |
| Robin_Watts | kens: pong | 13:43.31 |
| kens | ah Robin | 13:43.40 |
| | How can I tell if something is allocated in local or global VM in GS ? | 13:43.53 |
| | Perhaps I should say how can I tell if a struct is allocated in local or global VM | 13:45.24 |
| Robin_Watts | pfft. not a clue. | 13:46.39 |
| kens | Oh, OK thought your memory work might have helped. | 13:46.55 |
| | I'll wait and ask Ray | 13:47.02 |
| Robin_Watts | I haven't hit anything to do with that. | 13:47.15 |
| | I think that's at a higher level than Memento works, sorry. | 13:47.34 |
| kens | I have a weird bug assigned to me 'always allocate gx_show_enum in local VM' | 13:47.45 |
| | There is no specimen file, and I have no idea how to know which VM mode is in use | 13:48.01 |
| | Nor how to affect it :-) | 13:48.11 |
| | I suppose I could see what setglobal does. | 13:48.27 |
| | Well that might work | 13:49.01 |
| | Its a simple enough routine, I could set it to false before the allocation, and true afterwards. | 13:49.27 |
| | Huh, its all buried i nested macros, just like usual | 13:51.38 |
| Robin_Watts | tor8: Apparently, we have problems doing multiband rendering with mupdf when both bands try ti render glyphs at once. | 13:53.14 |
| | I suspect we need some locking around either freetype use or the glyph cache access. | 13:53.32 |
| | I'm going to have lunch now, but if you have any thoughts, I'd love to hear them. | 13:53.46 |
| | tor8: ping ? | 14:28.57 |
| kens | Hmm raspberry Pi ship date slipping | 14:30.25 |
| Robin_Watts | kens: Yeah, but at least they are open and honest, and feeding clear information. | 14:41.31 |
| | They've never promised a date or taken anyones money. | 14:41.42 |
| kens | Not complaining, just observing | 14:42.00 |
| Robin_Watts | Gah. Windows builds of mupdf are broken at the moment. | 14:45.33 |
| tor8 | Robin_Watts: non-shared glyph cache and lock around freetype access sounds best to me. it did strike me that we may not be mt-safe when rendering type3 glyphs thanks to the charproc running | 14:47.09 |
| Robin_Watts | tor8: We want a shared glyph cache, surely? | 14:47.34 |
| tor8 | that could be fixed by pre-parsing non-direct-rendered type3 glyphs to a display list | 14:47.38 |
| | lock contention for each glyph vs some extra work rendering twice, we'd have to do measurements | 14:48.10 |
| | we don't have a shared/exclusive locking mechanism with fz_lock | 14:48.33 |
| Robin_Watts | Locking around freetype access; so we can only call freetype from one thread at a time? | 14:48.51 |
| | Indeed not. | 14:48.57 |
| | We have problems in that if we lock, and then later try to do a malloc, we deadlock. | 14:49.09 |
| | Unless we specify that fz_lock must give us a recursive lock. | 14:49.22 |
| tor8 | I think locking as late as possible (i.e. at the freetype access level) will give us the least trouble, but that's just my wild guess | 14:50.55 |
| Robin_Watts | When freetype mallocs/frees does it do so using our malloc/free functions ? | 14:51.13 |
| | Or does it just call malloc/free directly? | 14:51.35 |
| henrys | wow more snow. I flew in yesterday just before the storm. | 14:51.47 |
| tor8 | not at the moment, we don't set up a custom allocator for it | 14:51.49 |
| Robin_Watts | But we should really. | 14:51.58 |
| henrys | Robin_Watts, tor8:looks like paul is done can you each review and test drive what we have? | 14:52.51 |
| Robin_Watts | Maybe we need 2 locks; one for malloc/free and one for 'other stuff'. | 14:52.58 |
| | henrys: Sure, but it won't be for a few days. | 14:53.28 |
| | Have pressure from 532 and the mupdf company at the moment | 14:53.44 |
| henrys | okay no problem | 14:54.19 |
| | I guess tor8 doesn't have a device. | 14:54.39 |
| Robin_Watts | tor8 has an HTC desire. | 14:55.00 |
| | I have an HTC desire HD. | 14:55.05 |
| | Paul developed it on an HTC Desire. | 14:55.30 |
| henrys | tor8:you'll probably want to look at the code right? | 14:55.44 |
| Robin_Watts | Getting other people with android devices to test it would be good. | 14:55.53 |
| | henrys: We'll need to look at the code to make it use fz_document rather than pdf_document. | 14:56.23 |
| sebras | Robin_watts: I have a HTC Sensation, if you want me to try it out... | 14:56.33 |
| Robin_Watts | sebras: ah, cool. | 14:56.41 |
| | tor8: Why is type3 glyph running not thread safe ? | 14:57.46 |
| tor8 | it invokes the pdf interpreter | 14:59.02 |
| Robin_Watts | oh, and reads from the file. Right. | 14:59.19 |
| malc_ | type3 fonts are just PDF commands according to pdf-ref.. question is why do they look so jagged? | 14:59.34 |
| tor8 | Robin_Watts: yeah, if we want shared writeable datastructures we'll want separate mutexes | 14:59.56 |
| Robin_Watts | malc_: Often each glyph is a bitmap. | 15:00.25 |
| tor8 | henrys: yes, I'd like to see the code (and have it committed to git) | 15:00.32 |
| henrys | sebras:here is his link http://intranet.glidos.net/~paul/MuPDF20120205.apk | 15:00.34 |
| Robin_Watts | tor8: He's working from git, and his intention is to give us a branch to commit back. | 15:01.05 |
| henrys | for the build | 15:01.07 |
| malc_ | Robin_Watts: so pdf commands are not curveto/lineto and such like, but rather put this bitmap other-there-stretching? | 15:01.08 |
| tor8 | Robin_Watts: cool | 15:01.12 |
| Robin_Watts | malc_: They can be both. | 15:01.32 |
| malc_ | tor8: i see, that explains it, thanks | 15:01.38 |
| tor8 | malc_: the type3 fonts can be anything, but most commonly they're just bitmap images | 15:01.41 |
| malc_ | tor8: that blows in a way | 15:01.54 |
| | tor8: btw. seen my late privmsg yesterday? | 15:02.01 |
| tor8 | Robin_Watts, henrys: I also have a galaxy tab | 15:02.19 |
| henrys | tor8:well we do want it tested like mupdf and ghostscript also - get in the hands as as many users as possible. We can post it on our web site, other than that where do folks get android apps? I googled a few places but I don't know if they are the main sources. | 15:04.11 |
| Robin_Watts | henrys: The Android Market. | 15:04.57 |
| kens | android app marketplace, and various phone suppliers home pages | 15:05.04 |
| | depending if device is locked | 15:05.15 |
| tor8 | and from any web site, android (unlike apple) usually allows third party apps to be installed | 15:05.38 |
| | and if you buy a locked device where that isn't possible, you've got a hole in your head | 15:05.52 |
| Robin_Watts | Yes, we can put the apk on the mupdf site. | 15:06.00 |
| tor8 | with a QR-code with the url for direct download | 15:06.25 |
| henrys | so paul should just publicize his branch for code review, yes? | 15:07.02 |
| tor8 | Robin_Watts: he's missed the tap-on-side-of-screen-to-flip-pages gesture | 15:08.23 |
| henrys | tor8:do you have paul's contact info? | 15:10.08 |
| | 532 and mupdf company are p1 or higher. | 15:10.39 |
| tor8 | no, but I have seen him around here a few times | 15:10.46 |
| henrys | let me send you his email. | 15:10.57 |
| Robin_Watts | I just spoke to Paul. He's going to get me a git format-patch of it all. | 15:13.24 |
| sebras | henrys: thanks for the link, I'll give it a whirl later today. | 15:13.34 |
| Robin_Watts | tor8: he says he'd spotted the tap-on-side-of-screen-to-flip thing, but had ignored it because swiping works so well. | 15:14.16 |
| | but he can add it if required. | 15:14.32 |
| henrys | Robin_Watts:I asked him for a public branch. | 15:14.53 |
| Robin_Watts | henrys: Right. He'll give me the git-format patch, which I'll add as a branch and push up. | 15:15.18 |
| henrys | okay | 15:15.24 |
| | so that will preserve his name on the commits? | 15:15.41 |
| Robin_Watts | I suspect it'll be a branch with just one commit in it. Yes. | 15:15.46 |
| | tor8: If I add new locks, then I need a new name; lock2/unlock2 ? don't like that much. | 15:16.58 |
| | make the new ones lock/unlock and the old ones lock_alloc/unlock_alloc ? | 15:17.18 |
| tor8 | lock_alloc, lock_cache ? | 15:18.46 |
| | Robin_Watts: swiping works better on a phone than a tablet | 15:19.31 |
| Robin_Watts | It's not just locking the cache. It's locking freetype, and (initially at least) the interpreter. | 15:19.58 |
| | Ultimately hopefully we can make it just lock file access calls rather than the whole interpreter. | 15:20.14 |
| tor8 | the android isn't responsive enough to fast swiping flicks :( | 15:20.48 |
| Robin_Watts | tor8: fair enough. | 15:21.00 |
| tor8 | maybe that'll work better if we can get the framerate up in the 30s or 60s | 15:21.17 |
| | Robin_Watts: or a new datastructure, fz_mutex or fz_lock | 15:22.40 |
| | but before going too far with locks, I think we better take a step back and think about building a proper mt-architecture with channels, queues and worker threads | 15:23.21 |
| Robin_Watts | I'm still very much in favour of trying to keep such considerations out of the mupdf lib. | 15:28.11 |
| | Having 2 locks should be enough for everything we ever want to do I think. (Famous last words) | 15:28.36 |
| tor8 | well, as long as they are internal I won't object too much | 15:30.11 |
| Robin_Watts | tor8: I like the idea of having the user provide the locking functions for us, as they do now. | 15:30.53 |
| | tor8: Are you planning to fix the pdfapp stuff soon? | 15:38.07 |
| | tor8: ping | 16:12.46 |
| | If I'm reading the code correctly, then we read the contents streams for a page into a buffer in pdf_load_page. | 16:14.44 |
| | Interpreting that buffer just runs the stream. We don't actually hit the file (except for XObjects, maybe). | 16:15.25 |
| tor8 | Robin_Watts: xobjects are also loaded into buffers | 16:18.00 |
| | so we're probably safe | 16:18.30 |
| Robin_Watts | I'm just trying to get my head around this. | 16:18.33 |
| | How about fonts? | 16:18.44 |
| tor8 | type3 charprocs are also loaded into buffers | 16:18.47 |
| Robin_Watts | or patterns, or colorspaces etc ? | 16:18.52 |
| tor8 | ah, no. the resources are loaded during parsing once we come across it in the content stream | 16:19.29 |
| | that's why it's unsafe | 16:19.35 |
| Robin_Watts | Right. But if we arrange the interpreter to take the prime lock before reading such resources, we'll be OK ? | 16:20.04 |
| | (I'm currently leaving the existing lock as lock/unlock and having a new one as lock_prime/unlock_prime - can't think of a better name at the moment) | 16:20.49 |
| tor8 | if you're concerned about type 3 fonts we should be fine if we turn the charprocs into display lists | 16:20.50 |
| Robin_Watts | I'd rather not have to do that if we don't have to. | 16:21.04 |
| tor8 | which pdfapp stuff did you talk about? | 16:21.12 |
| Robin_Watts | Building for windows fails. | 16:21.23 |
| tor8 | oh. I thought I fixed that :( | 16:21.32 |
| henrys | tor8:how are you profiling on lion? I had a nice command line script for shark now I'm stuck with Instruments it seems. | 16:21.34 |
| Robin_Watts | but I believe building mupdf for anything will fail. | 16:21.37 |
| tor8 | henrys: I don't use Lion | 16:21.41 |
| | I "upgraded" back to snow leopard | 16:22.00 |
| henrys | I am getting annoyed with it. | 16:22.05 |
| | kens:I don't know if I can keep you away from hpgl/2 -> pdf much longer ;-( | 16:42.04 |
| kens | OK now is as good a time as any I expect | 16:42.54 |
| | Does anyone have any idea what the function 'gs_show_n_init' ios for ? I can't find any calls to it, though it exists. | 16:43.39 |
| ray_laptop | kens: I saw that you wanted to know if an object is in local or global VM. If the object has a gs_ref_memory_t or gs_dual_memory_t *memory element then you can compare it to the gs_dual_memory vmspaces memories.named | 16:44.47 |
| kens | Hi ray_laptop I've moved on from there a bit :-) | 16:45.04 |
| ray_laptop | kens: OK -- I didn't scan the entire logs | 16:45.19 |
| kens | I know how to force the enumerator into local VM, I'm not sure about freeing it agina though. I'm assuming I don;t need to, teh GC will take care of it. | 16:45.33 |
| | Do you know anything about gs_show_n_init ray_laptop ? | 16:45.52 |
| | Its another place I would have to modify, but I cannot see any way for it to be called. | 16:46.08 |
| ray_laptop | kens: nope, sorry -- I've stayed away from text as much as possible :-) | 16:47.25 |
| kens | Yes, I think it may be vestigial. I'm hopinh so | 16:47.47 |
| Robin_Watts | fetches tea | 16:48.18 |
| henrys | time for the meeting... | 17:00.55 |
| chrisl | ray_laptop: any idea why Len would be coming to me with Type 1 pattern problems? | 17:00.59 |
| mvrhel | I am here | 17:01.01 |
| kens | Folks, I will be on holiday next week, ski-ing in Austria, back on Saturday 18th Feb | 17:01.35 |
| chrisl | Have fun :-) | 17:01.59 |
| Robin_Watts | chrisl: Because you're lucky? | 17:02.01 |
| kens | thanks chrisl, I hope to :-) | 17:02.09 |
| Robin_Watts | kens: yeah, enjoy yourself. | 17:02.10 |
| henrys | kens:great have a good time. | 17:02.32 |
| chrisl | Robin_Watts: is that 'lucky' like in Dirty Harry? | 17:02.38 |
| henrys | should we discuss 532 or is it as under control as it's going to get. They seem ansi. | 17:03.00 |
| | ansy | 17:03.04 |
| Robin_Watts | AIUI, they want the 6th gen to be at least as fast as the 5th gen. | 17:04.17 |
| | and there are some files on which it isn't. | 17:04.31 |
| | AIUI 5th gen was non gs based. | 17:04.39 |
| henrys | Robin_Watts:well that's fair right? | 17:04.42 |
| | right | 17:04.50 |
| mvrhel | can they update to 9.05... | 17:04.59 |
| Robin_Watts | And there are some files where we aren't. | 17:04.59 |
| | mvrhel: No. | 17:05.06 |
| | At least, not in the timescales. | 17:05.12 |
| ray_laptop | chrisl: no, Len is probably confused about Type 1 -- thinking it is Type 1 font | 17:05.28 |
| chrisl | ray_laptop: he definitely notes it as a problem with patterns....... | 17:05.58 |
| Robin_Watts | Last figures we have from them, on the PWTTQ1CC file, we are between 18 and 88% off. | 17:06.09 |
| ray_laptop | mvrhel: besides, I'm not sure 9.05 would help | 17:06.20 |
| mvrhel | ok | 17:06.29 |
| henrys | can we emulate the 5th gen in the simulator? | 17:06.40 |
| chrisl | henrys: it's only fair if they remember that we handle a lot more PDF features than their interpreter did - more features takes more time...... | 17:06.44 |
| henrys | is it worthwhile to profile each. | 17:06.54 |
| ray_laptop | Robin_Watts: right, and the one that is 88% over used to be 189% over | 17:06.58 |
| Robin_Watts | ray_laptop: Yes, it's a huge improvement. | 17:07.13 |
| mvrhel | did the 5th gen render it properly? | 17:07.32 |
| Robin_Watts | My memory is crap, and I've got my mupdf head on at the moment, but I think this is the file where we may be able to get that time back by writing some smarter code in image_render_mono. | 17:07.43 |
| ray_laptop | mvrhel: these files are all non-transparency cases | 17:08.03 |
| mvrhel | is this the 1 x N image chunks in landscape Robin_Watts? | 17:08.17 |
| Robin_Watts | We are called to landscape plot a 1x2047 image. | 17:08.24 |
| | yeah. | 17:08.28 |
| mvrhel | it would probably be a huge help to do a special case for those | 17:08.55 |
| Robin_Watts | If we can spot that and change it to a portrait 2047x1 image, we'd hit the performance target, I think. | 17:08.57 |
| henrys | Robin_Watts:are you doing timing on the current code also or just their corresponding release? | 17:09.47 |
| Robin_Watts | I have failed dismally to get any meaningful timings out. | 17:10.08 |
| | henrys: All we have is the code for the 6th gen. | 17:10.19 |
| | (AIUI). | 17:10.27 |
| | And we can only do Debug builds without diving into copying files around. | 17:10.46 |
| chrisl | It used to be possible to build the sim without GS, is that no longer the case? | 17:10.52 |
| henrys | I wonder if profiles of 5th gen would shed some light on what's going on. Maybe I'm reaching. | 17:11.11 |
| Robin_Watts | Hence I'm using their profiles to point me at interesting areas, then stepping through them in the debug. | 17:11.13 |
| | henrys: I suspect profiles that tell us where we are doing badly help us more than being able to run their code and trying to see where they are doing well. | 17:11.59 |
| henrys | right | 17:12.58 |
| Robin_Watts | I could dive into trying this image rotation thing, but I'm getting hassled from the mupdf customer. | 17:12.58 |
| | They've found a problem with the way we proposed to do banded rendering. | 17:13.17 |
| | It would be nice to get a fix for that out before 1.0 | 17:13.38 |
| ray_laptop | Robin_Watts: I'll have a look at that -- not sure how much it'll help, but only one way to know. | 17:13.51 |
| Robin_Watts | (and they want a fix NOW! :( ) | 17:13.52 |
| ray_laptop | Robin_Watts: I think you should do the mupdf -- if I need advice on the image rotation I can bug mvrhel | 17:14.23 |
| Robin_Watts | ray_laptop: That would be a huge help, thanks. | 17:14.36 |
| | I can certainly point you at the section of code and show you what my aborted test was doing. | 17:15.02 |
| henrys | is the image rotation problem the same as the enhancement assigned to me | 17:15.17 |
| | the patch Robin_Watts did which didn't quite work. | 17:15.31 |
| Robin_Watts | No. | 17:15.40 |
| | The patch I did that didn't quite work was to spot that an image is entirely within the inner_bbox of the clipping path; and if we are then don't bother inserting the clipping device. | 17:16.38 |
| henrys | I was think tor8 would take over mupdf and Robin_Watts would do more with 532 that would make more sense to me, but I don't feel active enough in the mix to decide. | 17:17.03 |
| Robin_Watts | 4 files had problems with that. 1 of them was down to the fact that something was trying to fill a 0x0 device. | 17:17.16 |
| | a trivial fix for that, and I have 3 files reporting as different, but none showing in bmpcmp. | 17:17.34 |
| | So I need to try to reproduce the difference on my local machine. | 17:17.51 |
| ray_laptop | Robin_Watts: I thought it was trying to file w=0, h=0 -- was the dev == NULL ? | 17:17.55 |
| | s/file/fill/ | 17:18.06 |
| Robin_Watts | I meant 0 by 0 (w=0, h = 0). | 17:18.19 |
| ray_laptop | Robin_Watts: right -- that was what I thought you said yesterday | 17:18.45 |
| Robin_Watts | with the clipping device in place, the clipping device handles the 0 by 0 case without error. Without it, the underlying device throws a wobbly, and the propogation of the error code prevents the image being flushed. | 17:19.15 |
| | ray_laptop: Yeah, I was unclear, sorry. | 17:19.31 |
| | henrys: Where is this enhancement? | 17:19.56 |
| henrys | oh it was the interpolation thing. | 17:21.15 |
| | 6902456 | 17:21.23 |
| | 690246 | 17:21.28 |
| Robin_Watts | Ah, right. Different thing. | 17:21.53 |
| henrys | alexcher:anything for the meeting? | 17:23.15 |
| | chrisl:release? | 17:23.23 |
| alexcher | henrys: no | 17:23.26 |
| chrisl | henrys: unless anyone has any objections, I'm planning the release for tomorrow - nothing of note has come up with the RC | 17:23.55 |
| ray_laptop | I haven't seen any feedback on the RC -- I don't know why we bother | 17:23.58 |
| alexcher | henrys: we have one more case of poor performance with multiple small images. | 17:24.23 |
| Robin_Watts | IS 12-07B.PS known to be indeterministic? | 17:24.24 |
| kens | Yes I think so | 17:24.41 |
| henrys | chrisl:so I don't know what the release words have about the pdf server but like the "alpha" should be inserted. | 17:24.46 |
| Robin_Watts | Ah, well, that's one of the fils. | 17:24.50 |
| kens | Its halftoning If I remember corectly | 17:24.50 |
| Robin_Watts | files. | 17:24.52 |
| henrys | s/like/likely | 17:24.58 |
| chrisl | ray_laptop: this is the first time I've had nothing back at all on the RC - I usually get at least a couple of replies. | 17:25.08 |
| Robin_Watts | How about 32-01.BIN ? | 17:25.11 |
| ray_laptop | chrisl: I think that it is broken enough that we shouldn't mention it yet | 17:25.15 |
| kens | Robin_Watts : yes I think so too | 17:25.20 |
| | I agree with ray_laptop | 17:25.30 |
| | Don't mention it | 17:25.37 |
| chrisl | henrys: I didn't put anything in about the pdfwrite server thing, it looked to flaky right now | 17:25.54 |
| Robin_Watts | ray_laptop: tkamppeter found and reported a problem with the rc, right? That chris fixed? | 17:26.00 |
| henrys | great | 17:26.02 |
| Robin_Watts | So there was some feedback. | 17:26.11 |
| ray_laptop | Robin_Watts: that's right -- I forgot | 17:26.25 |
| Robin_Watts | kens: Thanks. | 17:26.30 |
| | OK, then I'll forward the fix for the clipping thing along to 532. | 17:26.45 |
| chrisl | ray_laptop: anyway, doing the RC is a bit of "moral high ground", if some comes back and bitches about a build problem on a weird system, we can "you should have tested it"! | 17:27.40 |
| henrys | sorry to go all broken record: but tor on mupdf and robin_watts on 532 sounds like a good use of resources but your call. | 17:27.41 |
| kens | suspects Robin would rather work on MuPDF ;-) | 17:28.20 |
| Robin_Watts | henrys: It's been me that's been pushing the multithreaded stuff for mupdf, so I'd like to be the one to fix it. But I'll do 532 if you'd rather. | 17:28.42 |
| | (and it's my skype address that the mupdf customer has... :/ ) | 17:29.20 |
| henrys | Robin_Watts and tor8: I think you and tor8 can make a better decision that I, they are both P1's | 17:29.33 |
| | s/that/than | 17:29.39 |
| | I don't know how mvrhel's desk is either. | 17:30.01 |
| mvrhel | I am trying to keep a low profile during this conversation | 17:30.19 |
| | quietly exiting out the back door | 17:30.41 |
| Robin_Watts | I was going to appeal to mvrhel for help with the image rotation stuff, probably, so ray may be better positioned than me for it. | 17:30.41 |
| kens | henrys, do you need me ? I would like to slink off. | 17:30.47 |
| henrys | kens:nope I'll catch up with you latter in hpgl->pdf land ;-) | 17:31.10 |
| mvrhel | but I will pitch in as needed | 17:31.12 |
| chrisl | kens: before you go - should the fix(es?) you committed this morning go in 9.05? | 17:31.30 |
| kens | OK thanks henrys, I will read hte logs when I get back, so stick anything I should look at in irc. | 17:31.33 |
| | chrisl, no definitely not | 17:31.40 |
| chrisl | OKay | 17:31.47 |
| kens | Goodnight all. | 17:31.54 |
| ray_laptop | bye, kens | 17:31.58 |
| mvrhel | bye kens | 17:31.58 |
| chrisl | 'nite | 17:32.01 |
| henrys | it's 10:30 so let's call it | 17:32.35 |
| chrisl | alexcher: that multiple small images performance problem - I ran with -DNOGC and it seemed to be spending a lot of time in zlib (IIRC, the images are flate encoded). I did wonder if our use of zlib was suboptimal...... | 17:36.08 |
| henrys | alexcher:I saw the profile you didn't add the resolution or device. | 17:38.08 |
| alexcher | henrys: I was using the same command line as the user. | 17:38.49 |
| | chrisl: I'm profiling with -dNOGC | 17:39.46 |
| henrys | well we need the heavy stack from not just chunk_locate_ptr... | 17:40.28 |
| | s/stack/stack frame | 17:40.38 |
| chrisl | alexcher: it may be a red herring, I didn't have time to investigate further myself. | 17:40.56 |
| Robin_Watts | tor8: ping | 18:08.04 |
| tor8 | Robin_Watts: pong. sorry for the delay. | 18:34.01 |
| Robin_Watts | Sorry. | 18:35.51 |
| | I was pondering, just locking prime in pdf_cache_object. | 18:36.02 |
| | and similarly around other calls that actually hit the file. | 18:36.33 |
| | I think that might work out quite nicely. | 18:37.03 |
| mvrhel | bbiab | 18:38.45 |
| tor8 | Robin_Watts: it's worth attempting. what were the details about the mt banded rendering that didn't work out? | 18:47.38 |
| Robin_Watts | tor8: He has it working, except when 2 threads try and plot text at the same time. | 18:48.13 |
| tor8 | oh right, the font cache and freetype stuff. | 18:48.31 |
| Robin_Watts | Then it dies because it's conflicting on the ... yeah. | 18:48.37 |
| | That's what lead to me looking into this second lock. | 18:48.53 |
| | I've got the prime lock protecting freetype calls and glyph cache access. | 18:49.07 |
| | and I'm trying to get it to protect file access too. | 18:49.16 |
| tor8 | pre-parsing type 3 into display lists and locking around freetype should be the easiest way around it, but you're the one with your fingers in this mess :) | 18:49.18 |
| Robin_Watts | pre-parsing type3's into displaylists is going to be expensive on RAM. | 18:49.44 |
| tor8 | it shouldn't make much difference on typical image-based ones | 18:50.07 |
| | since they're already stored as fz_buffer content streams, and they usually have the image data inline | 18:50.35 |
| | at least the latex-generated files I've seen in the past do | 18:50.47 |
| | of course, making pdf_document accessible from multiple threads would be even better :) | 18:52.32 |
| | I worry though, from my limited experience with java where the synchronized keyword that adds a mutex around function calls caused massive slowdowns | 18:54.02 |
| | even in the single threaded use case | 18:54.13 |
| Robin_Watts | I've read conflicting articles on the web about that. | 18:54.49 |
| | There are ones saying that syncronized slows stuff down, and then there are others that say "Ignore the ones saying that it slows stuff down - it's people that don't understand the profiling". | 18:55.24 |
| | Certainly my experience hasn't been bad. | 18:55.37 |
| tor8 | my experience was almost 15 years ago on solaris machines, so probably irrelevant if even remembered correctly :) | 18:56.28 |
| Robin_Watts | I have a hacked version of mupdfdraw here, where I pass in locking/unlocking functions that do nothing but record locked or unlocked in variables. | 18:59.13 |
| | And assert that they are unlocked when you're locking and locked when you are unlocking. | 18:59.29 |
| | And then I assert that we're locked in seek_file and read_file. | 18:59.50 |
| | If I can get that to run cleanly through sane, then I think we can be confident it works. Can you see a flaw with that ? | 19:02.30 |
| | Actually, I am confused by pdf_load_stream | 19:04.56 |
| | That calls pdf_open_stream, which ends by seeking to the right offset in the file. | 19:05.42 |
| | Then we call pdf_load_object, which might cause another seek. | 19:06.03 |
| tor8 | there's the pdf_cache_object call before the seek that makes sure the object is pre-loaded | 19:08.53 |
| | a bit of a hack | 19:08.57 |
| | did you add any pdf_xref object eviction in the scavenging code? | 19:09.26 |
| Robin_Watts | No. | 19:09.34 |
| tor8 | good. then that assumption should still be safe. | 19:09.45 |
| Robin_Watts | If I'm reading the code correctly, pdf_open_stream positions the file pointer at the start of the stream offset. Which is the start of the dictionary, right ? | 19:10.10 |
| tor8 | no, the stm_ofs is at the end of the 'stream' keyword | 19:10.27 |
| | IIRC | 19:10.34 |
| Robin_Watts | oh, ok. So pdf_open_stream ends by positioning the file offset at the start of the stream itself. | 19:11.01 |
| tor8 | correct | 19:11.26 |
| Robin_Watts | But then... in pdf_load_stream, we call pdf_load_object. | 19:11.26 |
| | And we rely on that not moving the stream pointer, because it's already got the object cached. | 19:11.51 |
| tor8 | hm, in pdf_load_stream we should probably rearrange the order | 19:12.24 |
| Robin_Watts | Fab. I was about to ask if you'd mind if we did that :) | 19:12.41 |
| tor8 | because the dict_gets calls may very well resolve an indirect pointer | 19:12.42 |
| Robin_Watts | I think pdf_open_stream should lock prime as a side effect. | 19:14.20 |
| tor8 | we could add a fz_clone_stream call and rejig the file based streams. that would let us access the same file pointer without worrying too much about this sort of thing. | 19:15.56 |
| | but that may end up being more complex than just being careful like we are now | 19:16.13 |
| | is this 'prime' lock the same as for malloc or is this for the pdf_document lock? | 19:16.37 |
| Robin_Watts | We have 2 locks. | 19:18.39 |
| | The existing one (just for malloc) | 19:18.50 |
| | and the new 'prime' lock, that we use for higher level things. | 19:19.03 |
| | It's possible, we should have a separate 'file' lock. | 19:19.48 |
| mvrhel | Robin_Watts: sorry to bother you. any ideas why I can't build muPDF in windoze. lots of complaints from the VS solution about \thirdparty\jpeg-8d\ files | 19:23.48 |
| Robin_Watts | mvrhel: You need a new thirdparty.zip | 19:24.01 |
| | We've updated the jpeg lib and the freetype lib. See mupdf.com/download | 19:24.21 |
| mvrhel | ok it would be nice if that was part of the checkout | 19:24.25 |
| Robin_Watts | mvrhel: We don't have any of the thirdparty libs in our git repo. | 19:24.47 |
| | For reasons, I refer you to tor8 :) | 19:24.58 |
| mvrhel | in gs we do... | 19:24.58 |
| | but I wont get that discussion started | 19:25.15 |
| henrys | marcosw sorry I missed you at the meeting. Did you have anything? | 19:25.54 |
| mvrhel | ok. that fixed the 3rd party issues. still a build error in cmap_cns.h | 19:29.47 |
| | Robin_Watts: doesnt seem to like cmap_Adobe_CNS1_UCS2 | 19:31.03 |
| malc_ | Robin_Watts said, that i should say it here, so i do, if someone is on windows (s)he can try my stuff at http://boblycat.org/~malc/llpp.zip, if not one can watch a video at http://www.youtube.com/watch?v=9xtIqD_mHRw | 19:31.09 |
| Robin_Watts | mvrhel: Have you cleaned generated? | 19:31.29 |
| mvrhel | yup | 19:31.35 |
| | but I will clean and rebuild again | 19:31.47 |
| | same issue | 19:32.09 |
| | 2>c:\vrhel\artifex\mupdf\mupdf\generated\cmap_cns.h(1758) : warning C4047: 'initializing' : 'fz_store_free_fn (__cdecl *)' differs in levels of indirection from 'char [16]' | 19:32.51 |
| | 2>c:\vrhel\artifex\mupdf\mupdf\generated\cmap_cns.h(1759) : error C2078: too many initializers | 19:32.53 |
| Robin_Watts | How up to date is the source? | 19:33.25 |
| mvrhel | all updated | 19:33.34 |
| Robin_Watts | Let me stash, then pull then rebuild here. | 19:33.38 |
| mvrhel | I am at af7f17d4b497a1cb9920112079e7170195cca5e3 | 19:34.03 |
| Robin_Watts | OK, me too now. | 19:34.11 |
| | All builds fine for me :( | 19:34.49 |
| | Let me manually nuke generated. | 19:34.54 |
| mvrhel | well let me go in and blow away stuff too | 19:35.13 |
| tor8 | mvrhel: the windows build doesn't do proper dependencies for the generated files, try nuking them | 19:35.14 |
| Robin_Watts | mvrhel: Still working for me :( | 19:36.28 |
| mvrhel | hmm still fails here | 19:36.52 |
| Robin_Watts | Debug, Release or Memento build ? | 19:37.05 |
| mvrhel | Debug | 19:37.10 |
| Robin_Watts | Me too. | 19:37.16 |
| | Same error as before? | 19:37.27 |
| mvrhel | yes | 19:37.30 |
| Robin_Watts | Can you paste the relavent section of the file? | 19:37.38 |
| mvrhel | and release, which I had never built before also has the same issue | 19:37.40 |
| | 2>c:\vrhel\artifex\mupdf\mupdf\pdf\../generated/cmap_cns.h(1758) : warning C4047: 'initializing' : 'fz_store_free_fn (__cdecl *)' differs in levels of indirection from 'char [16]' | 19:38.09 |
| | 2>c:\vrhel\artifex\mupdf\mupdf\pdf\../generated/cmap_cns.h(1759) : error C2078: too many initializers | 19:38.11 |
| | 2>c:\vrhel\artifex\mupdf\mupdf\pdf\../generated/cmap_cns.h(1768) : fatal error C1903: unable to recover from previous error(s); stopping compilation | 19:38.13 |
| | static pdf_cmap cmap_Adobe_CNS1_UCS2 = { | 19:38.21 |
| | -1, "Adobe-CNS1-UCS2", "", 0, 0, 1, | 19:38.23 |
| | { {2,0,65535}, }, | 19:38.25 |
| | 2018, 2018, (pdf_range*) cmap_Adobe_CNS1_UCS2_ranges, | 19:38.26 |
| | 14886, 14886, (unsigned short*) cmap_Adobe_CNS1_UCS2_table, | 19:38.28 |
| | }; | 19:38.29 |
| | that is starting at line 1757 | 19:38.31 |
| Robin_Watts | Right. That's wrong. | 19:38.32 |
| | I get: | 19:38.38 |
| | static pdf_cmap cmap_Adobe_CNS1_UCS2 = { | 19:38.51 |
| | {-1, pdf_free_cmap_imp}, "Adobe-CNS1-UCS2", "", 0, 0, 1, | 19:38.53 |
| | so something is wrong with your cmapdump | 19:39.26 |
| | git diff ? | 19:39.33 |
| | git diff --cached ? | 19:39.50 |
| mvrhel | no diffs | 19:39.59 |
| Robin_Watts | Same here. | 19:40.14 |
| | git logg -20 definitely shows you at master ? | 19:40.33 |
| mvrhel | last change in this file was from tor yesterday | 19:41.07 |
| Robin_Watts | Look at line 125 of scripts/cmapdump.c | 19:41.10 |
| | fprintf(fo, "static pdf_cmap cmap_%s = {\n", name); | 19:41.30 |
| | fprintf(fo, "\t{-1, pdf_free_cmap_imp}, "); | 19:41.32 |
| | (That's lines 124 and 125) | 19:41.42 |
| mvrhel | yes | 19:41.46 |
| | same here | 19:41.59 |
| Robin_Watts | Right. So that's clearly not what's running on your machine. | 19:42.14 |
| | mvrhel: Did you blow away the build directory too ? | 19:44.01 |
| mvrhel | is muPDF VS solution set up like gs or is it a "real" solution | 19:44.08 |
| Robin_Watts | It's a *real* solution (with a hack for cmapdump and fontdump) | 19:44.27 |
| mvrhel | ok... | 19:44.39 |
| | which build directory | 19:44.54 |
| Robin_Watts | I think cmapdump.exe and fontdump.exe live in the top level build directory. | 19:45.04 |
| | called "build" | 19:45.11 |
| | If you don't have one, I may be wrong :) | 19:45.44 |
| mvrhel | hmm no build directory here | 19:45.45 |
| | there is a generated directory | 19:46.00 |
| tor8 | delete the generated directory and rebuild the solution, should be enough | 19:46.28 |
| mvrhel | now it is happy | 19:47.23 |
| Robin_Watts | What did you change ? | 19:47.37 |
| mvrhel | you should have it so that the clean deletes those in the VS | 19:47.38 |
| | deleted the generated files | 19:47.51 |
| Robin_Watts | I thought you tried that before ? | 19:48.03 |
| mvrhel | I deleted all the object files | 19:48.15 |
| | why would I have known that an upper level directory sat there with files that I needed to delete | 19:48.29 |
| Robin_Watts | <Robin_Watts>mvrhel: Have you cleaned generated? | 19:48.40 |
| | <mvrhel>yup | 19:48.42 |
| mvrhel | yes | 19:48.44 |
| | and it did not delete the generated files | 19:48.54 |
| | in the directory generated | 19:49.10 |
| | oh | 19:49.18 |
| | I see | 19:49.22 |
| | I thought you meant | 19:49.27 |
| Robin_Watts | Right. I meant "manually deleted generated". Sorry. | 19:49.27 |
| mvrhel | did I do a clean build | 19:49.31 |
| Robin_Watts | Sorry, my bad. | 19:49.42 |
| | Yes, this is a flaw in the windows solutions. | 19:49.52 |
| mvrhel | It would be nice to have the clean operation delete these | 19:50.03 |
| Robin_Watts | Yes. | 19:50.10 |
| | I had a different solution, but tor8 redid it. So I blame him :) | 19:50.31 |
| mvrhel | this is the sort of thing that would be good to have working. if someone is fooling with the solution and runs into an issue like this they are likely to be a bit frustrated | 19:51.10 |
| | sorry to chew up all your time on this | 19:51.40 |
| Robin_Watts | no worries. I think I can see how to do it. | 19:52.24 |
| | I think I have a fixed solution here. | 20:09.09 |
| | tor8: OK, pdf_reference17.pdf is whirring away now and not failing my locking tests. | 20:26.41 |
| | How would you feel about me pulling lock/unlock out of the alloc struct and putting them in a separate struct, as another arg to fz_new_context | 20:27.56 |
| | fz_locks_context | 20:28.07 |
| tor8 | I think that would be more logical. | 20:28.26 |
| Robin_Watts | Then people can still use NULL for fz_alloc_default. | 20:28.34 |
| tor8 | how about fz_register_alloc_context(ctx, alloc_ctx) instead of adding more and more arguments to fz_new_context? | 20:28.58 |
| Robin_Watts | and if you don't provide a locks struct, you can't do multithreading - which seems sensible. | 20:29.02 |
| tor8 | well, the alloc context should probably be part of the new_context argument since it's recursive | 20:29.17 |
| | fz_set_lock_context(ctx, lock_ctx) | 20:29.33 |
| | just tossing out ideas here | 20:29.37 |
| Robin_Watts | I'd worry that people would try to change the lock context after cloning other ones. | 20:29.59 |
| | locks, allocator and store size are absolutely basic. | 20:30.23 |
| tor8 | fair enough | 20:30.31 |
| Robin_Watts | For other ones in the future, then using such a function may indeed make sense. | 20:30.38 |
| | Balls. | 20:31.25 |
| | We are already requiring fz_lock to be recursive. | 20:31.38 |
| | When we evict a font from the store, we call fz_free_font_imp | 20:32.19 |
| | and that calls pdf_drop_cmap | 20:32.33 |
| | which calls fz_drop_storable, which locks. | 20:32.41 |
| | Windows locks are recursive anyway. | 20:34.02 |
| tor8 | what about posix locks? | 20:34.29 |
| Robin_Watts | It's an option for pthreads. Not all systems support it by default. | 20:35.09 |
| | But you can implement recursive ones out of non recursive ones with some trickery. | 20:35.39 |
| malc_ | haha, Oscar Toledo Gutierrez won (one of entries in)IOCCC again | 20:35.56 |
| Robin_Watts | Will ponder on it. | 20:37.04 |
| | I can probably solve it by not using the alloc lock to protect the store. | 20:37.22 |
| | mvrhel, tor8: New mupdf solution pushed. | 20:43.08 |
| | This should clean properly. | 20:43.18 |
| | I'm going to ponder the locking stuff overnight. | 20:43.33 |
| malc_ | tor8: btw. you are generating C/H files out of binary stuff (fonts cmaps) for portability? (cause it's much easier to just use objcopy to do things) | 20:44.08 |
| tor8 | malc_: for the fonts, for portability. for the cmaps, because we're dumping raw c struct representations. | 20:46.24 |
| malc_ | tor8: okay, understandable. just that droid fallback takes a while to be processed (on this machine at least) | 20:47.23 |
| tor8 | malc_: why don't you use a static build of freeglut for windows? | 20:47.27 |
| | malc_: I know, I hate it too | 20:47.36 |
| | malc_: a "workaround" is to remove it from the makefile dependencies | 20:47.56 |
| malc_ | tor8: i'd have to recompile lablGL to do that (and edit more files in the process).. and my migraine kicked in, so i couldn't stay in windows anymore | 20:49.07 |
| tor8 | understandable :) | 20:49.32 |
| malc_ | tor8: does it run at all? | 20:49.44 |
| tor8 | it locks up if I select text | 20:50.11 |
| | but otherwise seems to work well | 20:50.18 |
| malc_ | tor8: i only test windows once in a blue moon, needless to say X11 version doesn't lock up on text selection.. sorry about that | 20:50.49 |
| | btw. if you were to add complex script support to glfont, how'd you go about doing that (and i'm not saying i want to do it, just a curiosity thing) | 21:01.24 |
| tor8 | I probably wouldn't ... opentype is a complex beast | 21:11.02 |
| malc_ | okay let's rephrase.. pango, harfbuzz, icu which one is "da best"? | 21:11.51 |
| tor8 | if behdad hadn't rewritten harfbuzz in c++ I'd have used that | 21:12.11 |
| malc_ | i think it was always C++.. with all kinds of violations.. but heck mupdf itself violates C everywhere | 21:13.36 |
| tor8 | icu is a monster, and pango (IIRC) comes with the whole gnome baggage | 21:13.55 |
| malc_ | pango looks very glibish.. iow not my cup of tea | 21:14.22 |
| tor8 | no, harfbuzz started out as plain C, forked off the opentype table parsing in freetype if my memory serves | 21:14.25 |
| malc_ | and ICU is monster indeed | 21:14.26 |
| | oh | 21:14.34 |
| tor8 | now it's a c++ library with a c api | 21:14.37 |
| | of the three, it's the most palatable | 21:14.52 |
| malc_ | it's the smallest, palatable i can't argue for/against | 21:15.15 |
| tor8 | how does mupdf violate c? | 21:15.29 |
| malc_ | _[AZ] guards for instance | 21:15.43 |
| tor8 | I don't know what you mean by that term | 21:16.21 |
| malc_ | #ifndef _FITZ_H_ | 21:16.59 |
| tor8 | oh, right. | 21:17.09 |
| | those. | 21:17.11 |
| | yeah, except everybody does it so the standard can go bugger off ;) | 21:17.30 |
| | it should be FITZ_H though | 21:17.44 |
| malc_ | it should be | 21:17.51 |
| | and standards can bugger off right until the time you have to build your stuff on AIX where system headers "accidentally" leak "hz" define | 21:18.20 |
| | and where a = *(int*) 0, doesn't trigger segfault.. | 21:18.53 |
| tor8 | your own fault for using AIX ;) | 21:18.58 |
| malc_ | hey! | 21:19.05 |
| | it's cool to build stuff on pSeries boxen :) | 21:19.15 |
| | makes you feel like you are living on an edge of an airfield | 21:19.33 |
| tor8 | on my first job, I crashed an AIX machine by just logging on | 21:19.37 |
| malc_ | hah | 21:19.51 |
| | i wonder how soon IBM reps arrived | 21:20.04 |
| tor8 | never, it was common enough to just reboot the damned things every week anyway | 21:20.35 |
| malc_ | hmmm.. ours was rebooted when i had to port my device driver to PPC/linux, it was rebooted to insert the PCI card into it.. and porting amounted to.. if my memory serves me - nothing.. that's how bloody awesome i am | 21:21.37 |
| | tor8: http://boblycat.org/~malc/llpp.zip (no freeglut.dll) | 22:02.22 |
| | Forward 1 day (to 2012/02/08)>>> | |
IRC Logs