| | <<<Back 1 day (to 2012/06/18) | 2012/06/19 |
| kens | chrisl, amusingly the change you pointed out last night results in a PDF file with both /W and /W2 arrays, but all the entries in the /W2 array are 0. THis seems to work for everythign *except* the parentheses. | 07:06.49 |
| | It sort of makes sense when related back to the original file too. | 07:07.19 |
| chrisl | Hmm, I didn't get all 0's in the W2 array, and the parentheses were correctly placed, too | 07:08.20 |
| kens | Really ? Maybe I did something different. | 07:08.43 |
| | I did revert the changes in gdevpdtc.c first | 07:08.53 |
| chrisl | I'll check it again.... | 07:09.39 |
| kens | Everything *except* the parentheses are perfectly placed when compared to Acrobat for me though. | 07:10.04 |
| | Oh, note that I was using the reduced file for ease of readin gthe PDF, but I checked the big file too | 07:10.34 |
| chrisl | I didn't compare directly to Acrobat, but the parentheses *looked* okay | 07:10.45 |
| kens | THey are definitely out od place for me, too low. | 07:11.02 |
| chrisl | I need to rebuild, I just pulled fresh code..... | 07:12.07 |
| kens | OK no rush, I'm busy reading the Igor comments in gdevpdte.c | 07:12.30 |
| chrisl | Oh, that's weird: with that output, evince has the parentheses *very* nearly in the right place, but gs and Acrobat show them wrongly positioned....... | 07:17.37 |
| | kens: there's a problem with that approach, anyway: we can't rely (entirely) on the lack of vertical metrics in the font | 07:22.12 |
| kens | Why not ? | 07:26.50 |
| chrisl | What if the source file is a PDF? | 07:27.00 |
| kens | But then it will have vertical metrics, no ? | 07:27.12 |
| | Ah, actually, no it won't. | 07:27.23 |
| | Because the CDevProc will override them. | 07:27.39 |
| chrisl | Well, the CDevProc might not override them, it might be the source of them..... | 07:28.06 |
| kens | Yes, that's what I mean | 07:28.16 |
| | If its a PDF file the only verticla metrics might be the .W2 array | 07:28.30 |
| | I wonder if we could spot that.... | 07:28.58 |
| chrisl | Yeh, so we could revise the font_info method to check for a W2 entry in the font dict | 07:29.06 |
| kens | Is that where the W2 array is stored ? If so that would be useful. | 07:29.36 |
| chrisl | It can be either W2 or something like ".internalW2", but yes, the PDF matrics are added to the font dictionary | 07:30.16 |
| kens | I need to think, more coffeee required.... | 07:30.53 |
| chrisl | What we might miss is falling back to the default - I don't think is stored - but that would be easy enough to change..... | 07:31.16 |
| kens | Surely it must be stored, or how else can teh CDevProc use the defualt ? | 07:32.03 |
| kens | fetches coffee | 07:32.12 |
| chrisl | The default DW2 is stored, just as DW2 in the font dictionary - but there's some code in the CDevProc definition that suggests it isn't, so maybe that changed...... | 07:42.46 |
| kens | Well I can look for teh rpesence of DW2, I suppose I should check for both DW and W2 | 07:43.19 |
| | Just writing a stack overflow response. | 07:43.26 |
| chrisl | Trouble is, my hacked up code to do that didn't work right either :-( | 07:44.06 |
| kens | Umm, let me beat on it some more, I may come back to you | 07:44.31 |
| | chrisl that 'replaced_widths' thing is really interesting. | 08:54.12 |
| | Knocking that out makes some enormous improvements to a numebr of files. As well as breaking an almost equal number :-) | 08:54.37 |
| | Eg test 79 here: | 08:56.04 |
| | http://www.ghostscript.com/~regression/ken/compare4.html | 08:56.04 |
| chrisl | kens: huh, that is strange...... | 08:59.56 |
| | kens: and the very first test - 093-01.ps | 09:00.44 |
| kens | Yes indeed | 09:00.59 |
| | There are quite a few | 09:01.06 |
| | And many which are clearly broken :-) | 09:01.14 |
| | It looks like the ones which are now correct probably *don't* have vertical metrics, and the ones which are broken probably do (just looking at the files quickly) | 09:01.44 |
| | So it seems that this is an area that's always been broken | 09:01.59 |
| | Of course the flag 'replaced_v' is misleadingly named, it really means 'we have vertical metrics' | 09:02.28 |
| | For the PS file we get here, that flag gets set to true, and I think that it should not. | 09:02.51 |
| chrisl | Yeh, I think it's been broken since the CDevProc handling was added | 09:03.32 |
| kens | That could be it. If nothing else though I'm going to change the name of 'replaced_v' to 'have_vmetrics' | 09:04.02 |
| | I need to see why the flag is being set right now though. It may be because we get the widths from the 'copied' font, which is a TrueType, instead of the original font, which is a CIDFont (admittedly a TT CIDFont) | 09:04.53 |
| chrisl | What I was thinking was, if we get the metrics directly from the glyph info method, the "members" will correctly reflect that we don't have vertical metrics, but if we've called out to the CDevProc, the "members" value depends solely on the WMode | 09:05.59 |
| kens | No glyph_info is telling us it has vertical metrics | 09:06.22 |
| | gs_type42_glyph_info, gstype42.c, line 1023 | 09:06.49 |
| | loops checking for horizontal and then vertical metrics | 09:07.04 |
| | Hmm, actually no that says we didn't have vmetrics which is correct. Maybe it is the CDevProc | 09:07.43 |
| | Need to debug further | 09:07.50 |
| chrisl | It's *after* running the CDevProc..... | 09:08.11 |
| kens | Yes, before we run it its OK | 09:08.19 |
| | After its not | 09:08.23 |
| chrisl | gdevpdtt.c line 2663 | 09:08.50 |
| kens | chrisl yes | 09:09.05 |
| | But the first time into this routine, the glyph_info retrurns hte correct (empty) flag | 09:09.28 |
| | After we run CDevProc, it doesn't | 09:09.37 |
| chrisl | So, glyph_info gets us the correct information, and then we ignore it | 09:09.43 |
| kens | More or less | 09:09.51 |
| | Just a moment | 09:10.14 |
| | That's not strictly true | 09:10.46 |
| | We run glyph_info then we find we need to execute CDevProc, so we do. | 09:11.00 |
| | When we return we executte glyph_info again | 09:11.13 |
| | This time we get told we *do* have vertical metrics | 09:11.22 |
| | CDevProc seems to have added the metrics | 09:11.35 |
| | Doe shtat make sense ? | 09:11.40 |
| chrisl | No, but it might be because a request that includes the CDevProc returns immediately it finds a CDevProc, and doesn't test any further | 09:12.22 |
| kens | Gve me a minute, I'm confused now.... | 09:12.41 |
| | Not seeing what I expected | 09:12.49 |
| | Oh d'oh.... | 09:13.24 |
| | You're quite correct, but on my code its line 2653, not 2663 | 09:13.55 |
| | info.members = (GLYPH_INFO_WIDTH0 << wmode) | | 09:14.03 |
| | (wmode ? GLYPH_INFO_VVECTOR1 : 0); | 09:14.03 |
| chrisl | Yep | 09:14.14 |
| kens | THat seems mad | 09:14.27 |
| | THat *assumes* we got vertical and horizontal metrics returned. | 09:14.52 |
| kens | wonders what happens if I remove the arbitrary flag etting | 09:15.37 |
| chrisl | All kinds of weirdness, I found :-( | 09:15.54 |
| kens | Oh :-( | 09:16.00 |
| | I suppose it kind of makes sense. | 09:16.48 |
| | CDevProc does return all the values. | 09:16.58 |
| chrisl | But often (as I suspect in this case) the vertical ones are just random numbers to use up all 10 parameters | 09:17.35 |
| kens | Hehe, now my text is vertical when it should be horizontal, oops | 09:17.36 |
| | chrisl I'm sure you are correct. | 09:17.47 |
| ghostgum | One advantage of wasting half an hour chasing a command line bug is that you can write about it in the program documentation and warn others not to do it. | 10:28.36 |
| chrisl | at least it was *only* half an hour! | 10:30.32 |
| ghostgum | I was blaming changes in ghostscript for not working with RedMon, when I was giving ghostscript the wrong command line :-( | 10:31.14 |
| chrisl | With such an immense, bewildering array of options, it's all too easily done.... :-( | 10:32.15 |
| ghostgum | Instead of having the last command line option as "-", it was "-\n". Since it was generated by a cut and paste, and the \n wasn't visible... | 10:35.02 |
| | Ghostscript complained that it couldn't understand the command line option - | 10:35.28 |
| chrisl | Oh, that's a pretty nasty one, not surprised it took a while to track down! | 10:36.27 |
| kens | chrisl I can't seem to find a way to interrogate the font diciotnary to retrieve the /W2 or /DW2 entries, when in the glyph_info methods | 11:04.25 |
| | THey are all in 'base' and the font dicts and so on are in 'psi' | 11:04.43 |
| | The structure definitions and macros that is. | 11:04.53 |
| chrisl | Just a sec | 11:05.02 |
| | kens: I added it in: z1_glyph_info_generic() in psi/zchar1.c | 11:06.38 |
| kens | Hmm, but this is a type 42 font. | 11:06.55 |
| | Does it call that ? | 11:07.01 |
| chrisl | Yes, despite the name, that's the generic glyph_info call | 11:07.22 |
| kens | wibble.... | 11:07.29 |
| chrisl | z1_glyph_info_generic() can then call a font specific method supplied via its "proc" parameter | 11:08.02 |
| kens | Once I'm at that level I'm OK I think | 11:08.20 |
| | All I really want to do is see if there is an entry in the font dict | 11:08.34 |
| | And as long as I'm back at the tinerpreter level that's fine | 11:08.46 |
| chrisl | Sure, although I reckon it would be better suited to the font_info method..... | 11:09.06 |
| kens | I dfon't htink that will work, I trued that as well. This is a 'copied' font, so it just copies a pre-defined 'info' structure | 11:09.43 |
| | I'd have to add elements to that structure | 11:09.56 |
| | Potentially breaking stuff | 11:10.08 |
| chrisl | Oh okay, I thought this wasn't the copied font....... | 11:10.37 |
| kens | We do different things with the copied and original | 11:10.57 |
| | In pdf_glyph_widths, teh font we call with procs->glyph_info is the copied font | 11:11.32 |
| chrisl | Oh yes, so we do - cfont and ofont <sigh>...... | 11:11.34 |
| kens | Yes.... | 11:11.39 |
| | Just trying to complicat life.... | 11:11.50 |
| chrisl | kens: did you find the various names you need to check for in the dictionary? | 11:11.51 |
| kens | chrisl not yet, just trying to find DW2 and W2 right now | 11:12.07 |
| chrisl | I think you also need to check for "/.internalW2" | 11:12.32 |
| kens | and /.internalDW2 ? | 11:12.52 |
| chrisl | No, just .internalW2 | 11:13.05 |
| kens | OK thanks | 11:13.11 |
| chrisl | I think that's our "flattened" W2 array | 11:13.24 |
| kens | Ish | 11:13.31 |
| | We don't get into z1_gkyph_info for the copied font | 11:13.50 |
| | Because (aha) its a TrueType font | 11:13.59 |
| | We only get into it for the CIDFont | 11:14.08 |
| | Which is hte original font. | 11:14.13 |
| | So I think I need a totally new test *iff* we executed CDevProc. | 11:14.33 |
| | Which makes some sense | 11:14.41 |
| chrisl | That's not how I did it - but then, mine didn't work..... | 11:15.18 |
| kens | Oh :( | 11:15.25 |
| | What I'm looking at is.... | 11:15.34 |
| | If we don't execute CDevProc then the metrics information should be OK, so we use it | 11:15.49 |
| | If we do execute CDevProc, we cna't be sure if we got a valid vertical metric | 11:16.05 |
| | However, if the font says there are no valid verticla metrics *and* we a re a PostScript file, then we can ignore the vertical return from CDevProc (ie no verticla mertrics, treat as horizonrtal) | 11:16.45 |
| | We should only execute CDevProc, and pay attention to the vertical metrics if we are a PDF input. | 11:17.23 |
| chrisl | That's basically what I was trying to do, yes. With my code, the parenthesis were still out of whack - although this time, they were consistently out of whack | 11:17.36 |
| kens | So if we execute CDevProc *and* we have either /W2 or /DW2 then we set the vertical metrics true | 11:17.44 |
| | The parentheses are just plain weird, they are always different | 11:18.01 |
| | I would like to try and get this file to work without breaking PDF files, then go back and look t them | 11:18.21 |
| chrisl | Well, I believe this approach is the right one - I just couldn't face cutting down that file again, without a bit of a break first! | 11:19.18 |
| kens | Its a horrendous file certainly | 11:19.33 |
| chrisl | Another parallel builds problem solved....... | 11:21.28 |
| kens | Oh, that sounds good | 11:21.41 |
| chrisl | Yep, with this change, my own i7 has been running repeated 12 process parallel builds for the last hour, and none have failed | 11:22.22 |
| Robin_Watts | kewl. | 11:22.32 |
| kens | lunches | 11:22.52 |
| | oops, perhaps not | 11:23.06 |
| chrisl | Just can't tear yourself away :-) | 11:23.27 |
| kens | No bread :-( | 11:23.36 |
| ghostgum | Is this an excuse for a short walk to get away from the screen? | 11:24.39 |
| kens | Nope, my wife is off shopping and isn't back yet.... | 11:24.57 |
| chrisl | Given that for the first time in ages the weather is *actually* very pleasant, a bit of a walk might not be a bad idea...... | 11:25.32 |
| paulgardiner | Robin_Watts: What procedure do you suggest when I've fixed a forms bug? Is the testing facility at a stage where I can test for changed images? And do I have access to the golden repo? | 11:29.32 |
| Robin_Watts | paulgardiner: I haven't put the testing stuff into the cluster yet. | 11:36.54 |
| | Marcos is still in the Galapagos, and I don't really want to break everything with him away. | 11:37.11 |
| | You may well have access to the golden repo. | 11:37.30 |
| | We're trying a kinda peer review thing with mupdf at the moment; the idea is that we should all show our code to at least 1 other person before we commit. | 11:38.17 |
| paulgardiner | Sounds good.... are you busy? :-) | 11:38.42 |
| Robin_Watts | I try to run my changes (unless they are completely trivial) past tor, as he is the 'architect'. | 11:38.48 |
| | no, go for it. | 11:38.51 |
| paulgardiner | Last commit on paul/forms | 11:39.28 |
| | Robin_Watts: ACtually hold off | 11:40.28 |
| Robin_Watts | fetching it now. | 11:40.28 |
| | too slow. Tell me when to repull :) | 11:40.42 |
| paulgardiner | Robin_Watts: Just remembered I never tidied up a now unused function and a commented out line | 11:40.59 |
| Robin_Watts | dn = "N"; regardless of the if. | 11:42.23 |
| paulgardiner | Oh yes. And that. :-) | 11:42.49 |
| Robin_Watts | update_text_appearance seems to have jumped down the file a bit. I'm trying to spot what's changed... | 11:44.29 |
| paulgardiner | Ah yes. For the diffs, it would have been better if I'd have moved upwards the function it now needs to use | 11:45.23 |
| Robin_Watts | oh, it's moved to after load_or_create_form. | 11:45.24 |
| paulgardiner | yep | 11:45.33 |
| Robin_Watts | Well, it looks plausible to me. | 11:46.01 |
| paulgardiner | Just pushed an updated version - not fastforward | 11:47.17 |
| | the main changes to update_text_appearance were the use of load_or_create_form, and to use the text from the widget value rather than pass it in. | 11:48.36 |
| Robin_Watts | gives up using git itself and resorts to the webview in the browser. | 11:51.45 |
| | Looks perfectly reasonable to me. Pushed | 11:54.16 |
| paulgardiner | ta | 11:55.38 |
| | Robin_Watts: a bug you closed the other day includes some git commit messages. Was that cut and paste, or does bugzilla have a feature that makes it quicker? | 11:57.49 |
| Robin_Watts | cut and paste. | 11:58.14 |
| | git log -1 cut and paste to bugzilla. | 11:58.38 |
| paulgardiner | From git-show? | 11:58.38 |
| | Ah | 11:58.42 |
| Robin_Watts | ooh. I have the cursor turning to a caret over the form fields. | 12:03.44 |
| | paulgardiner: When compiling the forms stuff I get LOADS of warnings. | 12:05.08 |
| | All of the same type. | 12:05.15 |
| paulgardiner | assignments between const and non-const? | 12:05.38 |
| Robin_Watts | d:\cvs\artifex\mupdf.git\pdf\pdf_form.c(410) : warning C4090: 'function' : different 'const' qualifiers | 12:05.42 |
| | Yeah. It's a shame, because they are the only warnings we get from the non-thirdparty stuff. | 12:06.07 |
| paulgardiner | Yeah. I must fix that. I think I'll have to just not declare const, otherwise I'll need to make a huge number of changes | 12:06.16 |
| Robin_Watts | Isn't it a matter of making fz_buffer_printf take a const char *fmt ? | 12:06.52 |
| chrisl | kens: revised test file on its way to you....... | 12:07.41 |
| chrisl | lunches.... | 12:07.50 |
| paulgardiner | Robin_Watts: oh... actually yes | 12:08.13 |
| Robin_Watts | tor8 has a pathological hatred of const it seems. | 12:08.38 |
| | wherever possible he avoids it. | 12:08.55 |
| paulgardiner | I was ignoring it because I thought I'd looked at it before and it was due to many other things not using const. | 12:08.59 |
| Robin_Watts | actually... I think I have the fix here. | 12:09.19 |
| tor8 | Robin_Watts: const is an all or nothing thing, start using it and it infects everything | 12:09.35 |
| Robin_Watts | Aha. | 12:10.33 |
| tor8 | I'm not opposed to const in theory, but if we add it we should be consistently using it everywhere, and that's a lot of work | 12:10.39 |
| Robin_Watts | tor8: I've got the caret cursor thing working. | 12:10.55 |
| tor8 | and I really don't like casting away const... | 12:11.04 |
| Robin_Watts | It uses the same crap enumerator thing that I use for the mujs thing. I want to improve the enumerator a bit. | 12:11.26 |
| tor8 | hm, I think I should check out this forms branch :) | 12:11.48 |
| paulgardiner | tor8: On one project where we had warnings mapped to errors I had to cast const away using a union. | 12:11.54 |
| Robin_Watts | I figure on a routine that says "call this callback with this arg, and a pointer to a structure for each annotation in turn." | 12:12.17 |
| tor8 | if you need to cast const away, you're being either (a) wrong, or (b) lazy (by not spreading the infection) | 12:12.29 |
| Robin_Watts | At the moment the structure will just have the page rectangle in, but we can fix that. | 12:12.53 |
| | The thing that stops const being easy to do consistently is malloc. | 12:13.12 |
| | If I malloc a block, and fill it with data, I then might want to put that into a structure as a const thing. | 12:13.36 |
| | The problem comes when I then want to free it. | 12:13.46 |
| paulgardiner | tor8: almost always, but I seem to remember there was a (c). Can't remember what it was now. | 12:13.47 |
| Robin_Watts | i either need to cast away the const, or I need to have kept a separate (non const) pointer around purely so I can free it later. | 12:14.25 |
| tor8 | Robin_Watts: yeah. that just shows how forced const is in c, it just doesn't fit. | 12:15.00 |
| | as a workaround we could make fz_free take a const void *, and cast away the constness there. icky. | 12:15.48 |
| Robin_Watts | tor8: I've considered that before, yeah. That would, I think reduce the const breakage of most apps to a single place. (well, realloc too) | 12:16.28 |
| paulgardiner | I think the case I had was a whole load of iteration routines that applied functions to a complicated structure and I wanted to avoid creating the whole edifice twice one for the const case and one for the non-const | 12:16.30 |
| Robin_Watts | paulgardiner: You need polymorphic types! | 12:16.50 |
| paulgardiner | tor8: Yeah, I think it's correct to allow free to take a const pointer | 12:17.00 |
| Robin_Watts | ("Strong Typing is for Weak Minds", Richard Bird) | 12:17.16 |
| paulgardiner | Robin_Watts: Nice quote. Hadn't hear that one before. | 12:17.42 |
| | Taking something out of existence is sort different to changing it | 12:18.39 |
| Robin_Watts | yes, you shouldn't be accessing it after the free, therefore no compiler optimisations to avoid reading the same value back afterwards should apply. | 12:19.15 |
| | tor8: I'd like to get paulgardiner to publish the v8 source tree onto a repo on casper; it's in the form of a git repo now. You got any objections to that? | 12:20.16 |
| paulgardiner | And you should be avoiding freeing pointers to read-only memory by other considerations | 12:20.29 |
| Robin_Watts | yes. | 12:20.37 |
| tor8 | Robin_Watts: you can publish anything you want under your user namespace and I don't think anyone will object :) | 12:20.58 |
| | the big question is, do we mirror third party gits, or make gits from release snap shots? | 12:21.15 |
| Robin_Watts | tor8: back to the enum thing; I'd like to have to callback functions capable of stopping the enumeration by returning a non zero result. | 12:21.27 |
| | tor8: I think, in the v8 case, we should mirror it. | 12:21.42 |
| | And we can keep our own branch on there. And we can pull in changes from upstream, and merge into our own branch. | 12:22.14 |
| tor8 | Robin_Watts: where's the current enumeration code? | 12:22.38 |
| Robin_Watts | fz_bound_annots ? | 12:22.51 |
| paulgardiner | Robin_Watts: thinking about this since we talked, giving a reference to the commit within Google's main repo is equivalent. | 12:23.24 |
| | ... unless we have to make changes | 12:23.35 |
| Robin_Watts | (I think a general structure that we can extend is better than a specific function to just pass in rectangles) | 12:24.02 |
| | paulgardiner: But you *have* had to make changes, right? | 12:24.13 |
| paulgardiner | Hmmm. Let me look at my repo here | 12:24.32 |
| | I seem to remember telling you I'd made changes, but I now can't actually remember making them. :-) | 12:25.03 |
| tor8 | Robin_Watts: right. how do you feel about an iterator design rather than callbacks? | 12:26.03 |
| | might be less juggling around of state through structs for the callers | 12:26.30 |
| Robin_Watts | In general, I dislike iterators. | 12:26.41 |
| | They always seem far more work to me (both as a user and an implementer) | 12:26.59 |
| | I'd be happy with any system whereby a would-be porter can checkout (or clone) 1 thing, and then follow a set procedure to produce a lib. | 12:27.17 |
| | paulgardiner: ^ | 12:27.23 |
| tor8 | Robin_Watts: we could do something like fz_page (which is a type that doesn't exist, other than as a typed placeholder for void*) with annotations | 12:27.48 |
| | or just index them: fz_count_annots, fz_bound_annot | 12:28.15 |
| Robin_Watts | indexing them is bad if we hold them in a linked list. | 12:28.37 |
| | becomes n^2 to walk using iteration. | 12:28.47 |
| tor8 | fz_link is concrete and exposes the linked list nature... | 12:28.56 |
| paulgardiner | The number of fileds shouldn't be huge. We could ask the library for an array of them. | 12:28.58 |
| Robin_Watts | s/iteration/indexing/ | 12:28.58 |
| | tor8: Right. fz_link is bad, IMHO. | 12:29.06 |
| tor8 | and we use linked lists a lot, so that would be preferable somehow | 12:29.08 |
| | linked lists are easier to implement than dynamically grown arrays. sad really :( | 12:29.49 |
| | we could do a makeover on the linked lists and try to see which ones we can turn into arrays | 12:30.14 |
| Robin_Watts | I like the "iterate over this list calling the callback", so that does the walking over the linked list. The 'guts' of the structure can then be passed back to the caller as a pointer with no juggling at all. | 12:30.23 |
| tor8 | Robin_Watts: yeah, but without closures that sort of thing can easily turn into a big mess | 12:30.53 |
| | with extra auxiliary functions and structs | 12:31.14 |
| Robin_Watts | so fz_link would be the 'guts' of the structure, and fz_link_element would be { fz_link_element *next; fz_link guts; } | 12:31.25 |
| | and it means we can change it from a linked list to an array etc later if we want to. | 12:31.44 |
| tor8 | so basically you want 'map()' for all our list/array types? | 12:32.25 |
| paulgardiner | Why the two levels? Can't we have a function to get the first widget, and a function to get the next, given the previous? | 12:32.28 |
| Robin_Watts | tor8: map, (or app, depending) | 12:32.51 |
| tor8 | paulgardiner: an iterator :) | 12:32.59 |
| | well, both arrays and linked lists support an iterator based pattern of getting the 'next' element from the current one | 12:33.29 |
| paulgardiner | Yeah, I thought we were talking about iterators witht the link and link element stugg | 12:33.31 |
| | stuff | 12:33.34 |
| tor8 | oh wait no they don't, I need more coffeine | 12:33.42 |
| paulgardiner | I do like the callback approach too. | 12:34.10 |
| tor8 | well, an array can still have a linked list joining the elements in sequence :) | 12:34.24 |
| Robin_Watts | paulgardiner: Well, I'm not a huge fan of iterators, but I could live with it. Regardless, I'd like to figure out a reasonable style that we can all agree on, rather than commit my quick hack now. | 12:34.56 |
| | iterators are only better than callbacks if you have the ability to 'prev' as well as 'next'. | 12:35.41 |
| tor8 | let's ask sebras :) | 12:35.50 |
| Robin_Watts | And the nice thing about callbacks is that there is less possibility of dropping stuff during errors during the callback. | 12:36.22 |
| tor8 | Robin_Watts: iterators have the benefit of not changing our current code much | 12:36.33 |
| Robin_Watts | is called for lunch. | 12:36.36 |
| | iterators would be a larger change than callbacks, I feel. | 12:36.48 |
| tor8 | for (node = root; node; node = node->next) translates pretty easily into an iterator | 12:37.20 |
| Robin_Watts | callbacks are essentially taking the contents of while loops and putting them into small functions. | 12:37.25 |
| | iterators require another level of try catch to ensure the iterator is destroyed. | 12:37.53 |
| tor8 | I'm not vehemently opposed to callbacks, but I fear putting stuff into small functions will get annoying quickly. like with qsort... | 12:38.08 |
| paulgardiner | Is it: "iterators - more work for the implementor", " callbacks - more work for the caller" maybe? | 12:38.21 |
| tor8 | Robin_Watts: the iterator can be a static object that lives on the stack | 12:38.28 |
| Robin_Watts | tor8: as long as nothing it holds is refcounted, yes. | 12:38.51 |
| paulgardiner | With callbacks the caller usually has to invent a structure to pass in and have passed back. | 12:39.08 |
| tor8 | or we can make the iterator based on the current element | 12:39.16 |
| Robin_Watts | paulgardiner: Yes. | 12:39.17 |
| paulgardiner | But having said that, it often works out neatly. | 12:39.36 |
| Robin_Watts | If the iterator is just a fz_link * and the next step is just link = link->next, then yes, we're fine. | 12:39.47 |
| paulgardiner | I think I'm struggling to have an opinion, which probably means I should shut up. | 12:40.11 |
| Robin_Watts | I'd be happy either way I guess, though my natural inclination would be to callbacks. | 12:40.26 |
| tor8 | I'm not sure what I like best... | 12:40.42 |
| Robin_Watts | Let's summon the sebras; he can read the logs and chip in while I get lunch. bbs. | 12:41.04 |
| tor8 | we haven't done callbacks anywhere before, so I'm hesitant to start introducing them | 12:41.14 |
| sebras | hello! | 12:41.31 |
| paulgardiner | tor8: it is possible to get into an awful mess with callbacks, especially when locking is involved. | 12:42.06 |
| tor8 | sebras: we're discussing iterating over various lists of things using abstract interfaces | 12:42.31 |
| | iterators vs map-function-with-callback vs exposed linked list structs vs index based access | 12:43.01 |
| alex11 | hi all, can I bother you with a mudraw question? | 12:47.44 |
| sebras | tor8: I read the logs, but I'm not sure I understand where this discussion originated? is there a particular case where you might need them? | 12:48.57 |
| kens | ask away alex11 | 12:49.05 |
| alex11 | I'm using a -ttt rich-xml-text-output option and I want to confirm several things; first, the coords of bboxes are in DPI 72, right? | 12:49.37 |
| kens | 'user space' I believe | 12:49.52 |
| alex11 | I tried to alter it with -r, but it doesn't seem to work in that output mode | 12:50.06 |
| kens | That's because its resolution independent | 12:50.16 |
| | Its in 'user spce | 12:50.21 |
| tor8 | alex11: yes, it's hard coded to user space (72 dpi) | 12:50.27 |
| kens | tor8 what about UserUnit ? | 12:51.09 |
| alex11 | cool, thanks. one more thing: I've noticed that literals that are renedered bold on the PDF are appearing twice in all -t output modes, is this some kind of convention? | 12:51.18 |
| tor8 | kens: abstracted away :) | 12:51.20 |
| kens | alex11 more likely that's an artificial bold effect | 12:51.55 |
| tor8 | kens: same as Rotation and off-origin page MediaBoxes | 12:51.57 |
| sebras | tor8: map-functions-with-callback can be elegant, but I feel that often you want to carry around a closure to make them useful, which in C I guess translates to a void* to a struct that you need to cast back to whatever struct-type you wanted in your callback function. that's a lot of code... | 12:52.19 |
| kens | alex11 if you draw text twice in slughtly different positions, you get a 'bold' effect | 12:52.33 |
| tor8 | sebras: that's my main objection, you need to add both a function and a struct to hold the state you need. | 12:52.53 |
| | and iterators have scope visibility issues if you want to make them live on the stack, or try/catch cleanup problems if they're allocated on the heap | 12:54.01 |
| sebras | tor8: yeah, and if we're aiming to do this kind of iteration for some basic struct say for dicts, then I'm fearing that we'll have a whole slew of new struct-types that are basically only used in a single function (and it's callback). | 12:54.06 |
| alex11 | kens: that's strange, since the bbox coords are the same up to the 3rd decimal place... but it's iText generated PDF, so I guess strange things _can_ happen | 12:54.08 |
| sebras | tor8: scope vibility issues? | 12:54.28 |
| | visibility! | 12:54.35 |
| tor8 | alex11: is the text by any chance both stroked and filled? | 12:54.47 |
| | sebras: to live on the stack, it can't be opaque, so you need to expose the innards | 12:55.02 |
| | another iterator pattern is to just use the element as the iterator (like a linked list) | 12:55.23 |
| kens | alex11 as tor8 says could be stroked and filled, another way to do bold | 12:55.41 |
| tor8 | so we'd just hide the linked list next pointer behind a function | 12:55.47 |
| paulgardiner | tor8: with iterators, you could return an opaque pointer and then have method to get to the guts | 12:55.50 |
| alex11 | tor8, kens: dunno, will check. thanks for your help! | 12:56.18 |
| tor8 | paulgardiner: yeah. that works too, but leads to an explosion of types. ugh. | 12:56.19 |
| | paulgardiner: and for linked lists, the opaque pointer and the guts could be the same object | 12:56.52 |
| | but again, that's a pointer that needs to be cleaned up a try/catch | 12:57.19 |
| | I'm going to go sulk and pine for better programming languages than C for a bit, brb :) | 12:57.41 |
| paulgardiner | I was thinking more that you'd return an opaque widget and have several funtions for returning things like the bounding box, rather than have one get-guts fn | 12:58.00 |
| sebras | tor8: there's also the type of linked lists that the linux-kernel uses. | 12:58.26 |
| | tor8: those have their link-innards exposed, but common practice is to use a series of macros to iterate of them. | 12:59.21 |
| | tor8: one of them being some kind of foreach I believe. | 12:59.38 |
| tor8 | paulgardiner: right! I think I see what you mean. | 13:03.52 |
| paulgardiner | tor8: I think fz_widget is already in that style... not that implies it has to stay that way of course | 13:04.38 |
| sebras | where did this discussion originate? what type of objects are you envisioning this for? | 13:05.19 |
| | everything? or just something related to ui-stuff? | 13:05.28 |
| jen_ | We're getting UnicodeDecodeError - I need some advice: http://pastie.org/4114314 | 13:08.03 |
| | I guess, can GS handle unicode filenames? | 13:09.59 |
| paulgardiner | tor8: I guess I meant use accessors rather than direct access to fields, which you pointed out the disadvantages of before, but maybe it's worth it for the app/library interface | 13:10.00 |
| tor8 | sebras: mainly the fz_document stuff, which includes the forms and widgets and javascript things | 13:13.37 |
| jen_ | Any help for with my question? | 13:17.57 |
| Robin_Watts | sebras: Hi. | 13:18.04 |
| | The particular example that I had in mind was fz_annots (which are very similar to fz_links) | 13:18.25 |
| | At the moment, you can 'fz_load_links' and you get a fz_link * pointer. The implementation of that is completely exposed; callers can walk/fiddle with the list themselves, and get to all the structure. | 13:19.28 |
| sebras | Robin_Watts: right, and you want to prevent fiddling? | 13:20.06 |
| Robin_Watts | There is no concept of "public" and "private" parts of the structure, and we can never move to a different implementation in the future without breaking all the client code. | 13:20.08 |
| | So now we want to add 'annotation' support. | 13:20.20 |
| chrisl | jen_: that's not a ghostscript command line, nor is it a ghostscript error that I've ever seen | 13:20.38 |
| Robin_Watts | Now, annotations are probably going to have more private stuff in them (for the javascript/forms side) | 13:20.42 |
| | and we may well want to change away from a simple linked list thing in the future. | 13:21.04 |
| | So I'd like to define an interface for dealing with them, and work through that interface. | 13:21.21 |
| miha | mupdf on android is very slow for some pdf's... this one is one of the worst http://www.wolfey.si/tmp/telpoletje.pdf .. can anything be done? | 13:21.27 |
| Robin_Watts | essentially, future proofing it. | 13:21.34 |
| | miha:Without looking at the file, we can't comment. | 13:21.43 |
| miha | Robin_Watts: i gave you url | 13:21.53 |
| kens | He's given you a URL Robin_Watts | 13:21.56 |
| sebras | jen_: I believe this is a problem in a python script of yours? | 13:21.57 |
| Robin_Watts | miha: Well, I'll jump right to it immediately. Yes Sir! | 13:22.13 |
| kens | :-) | 13:22.18 |
| miha | thank You | 13:22.20 |
| jen_ | chrisl and seabras :Thanks for chiming in. Let me pastebin my python script. | 13:22.25 |
| Robin_Watts | </sarcasm> | 13:22.31 |
| sebras | jen_: unfortunately I have always had text encoding issues so I'm not skilled enough to help you. I'd suggest paying #python a visit... | 13:22.38 |
| miha | Robin_Watts: i dont expect miracles. just asking... | 13:22.44 |
| kens | does the file contain transparency ? Was it created by Caior ? | 13:23.01 |
| | Cairo* | 13:23.09 |
| Robin_Watts | miha: Right. Your best bet is to open a bug. That way it won't get forgotten. I'm certainly not going to have time to look today. | 13:23.16 |
| miha | Robin_Watts: but is it a bug? :) | 13:23.39 |
| | it renders fie | 13:23.43 |
| | fine | 13:23.46 |
| chrisl | miha: that file takes a few seconds for evince to display the first page, on the full workstation | 13:23.47 |
| miha | chrisl: yes | 13:23.52 |
| chrisl | So how long does it take on your Android device? | 13:24.09 |
| miha | on gallaxy S, some 10 seconds? | 13:24.19 |
| jen_ | chrisl and sebras : here is my python code: http://pastie.org/4114391 thanks so much for your help . | 13:24.23 |
| miha | chrisl: any ideas why is it so slow? images? | 13:24.49 |
| Robin_Watts | JPXDecode. | 13:25.03 |
| sebras | miha: and you file contains several large images. possibly an image scaling performance issue? | 13:25.16 |
| Robin_Watts | It's got Jpeg 2K images in it. I think they are slow to decode. | 13:25.27 |
| sebras | Robin_Watts: but they're tiny. 62x62! | 13:25.28 |
| miha | sebras: but tried some other with large images (Hustler, for example).. was much better than this one | 13:26.21 |
| Robin_Watts | Why do people never give us *those* files to look at? :) | 13:26.45 |
| miha | Robin_Watts: one moment Sir | 13:26.55 |
| chrisl | miha: as kens said, probably transparency blending - that's a time consuming process, and there's quite a lot of it in that file | 13:27.21 |
| Robin_Watts | sebras: 655x1511 ? 190x1078? | 13:27.27 |
| | miha: I was joking :) | 13:27.43 |
| sebras | Robin_Watts: once I stumbled over an autopsy report that failed to render (in sumatra I believe)... | 13:27.53 |
| | Robin_Watts: but aren't those images flated? | 13:28.11 |
| Robin_Watts | Nope. | 13:28.16 |
| sebras | Robin_Watts: or have I screwed up mupdfinfo again...!? :-/ | 13:28.25 |
| Robin_Watts | I searched in the file for JPXDecode. | 13:28.34 |
| sebras | Robin_Watts: ah! SMask! | 13:29.34 |
| kens | is concerned by sebra's choice of reading matter | 13:29.43 |
| chrisl | jen_: what happens if you always set the encoding to ascii? | 13:30.52 |
| Robin_Watts | sebras, paulgardiner, tor8: So... did we reach a consensus? Does anyone still care? | 13:32.16 |
| miha | chrisl: a follow up question, is it possible to 'optimize' pdf on server? | 13:33.18 |
| sebras | Robin_Watts: no. yes. | 13:33.30 |
| | Robin_Watts: I'm disliking the callback idea though. | 13:33.53 |
| miha | chrisl: so it'd look the same but open faster | 13:33.54 |
| sebras | Robin_Watts: mostly because of a high probability of lots of small functions and structs. | 13:34.25 |
| Robin_Watts | The downside to the callbacks are that you end up with lots of small functions and lots of small structs. | 13:34.40 |
| chrisl | miha: it *might* be possible to do something like that, but you'd almost certainly end up with a *much* larger file | 13:34.54 |
| jen_ | chrisl: same error | 13:34.57 |
| Robin_Watts | But I don't see that as a showstopper; the definitions for such things are small and local. | 13:35.15 |
| chrisl | jen_: okay, so instead of ascii, maybe use utf8? | 13:35.31 |
| Robin_Watts | They don't pollute the global namespaces. | 13:35.38 |
| miha | chrisl: like flatten all images to one big jpeg? | 13:35.39 |
| chrisl | miha: effectively, yes - if you want something that looks like it's got transparency, but doesn't, you basically end up with an image of the blended colors | 13:36.30 |
| sebras | Robin_Watts: you'd still have a fz_foreach() macro or similar. | 13:36.43 |
| | Robin_Watts: and I guess having a few macros/functions for iterating over lists makes sense. even the linux kernel has a list.h! :) | 13:37.11 |
| jen_ | chrisl: bingo!!!!!!!!! | 13:37.21 |
| chrisl | jen_: cool - btw, the error is coming from python, not from GS | 13:37.56 |
| jen_ | chrisl: My python programming skills are quite elementary and this project was dumped on me! Thanks for helping out. | 13:38.12 |
| chrisl | jen_: np - tbh, my python skills are rather atrophied, so it was mostly luck! | 13:38.59 |
| sebras | jen_: I recommend searching for python errors over at stackoverflow.com they usually have good answers... | 13:39.09 |
| miha | chrisl: is there a command line for that? :) | 13:39.16 |
| jen_ | thanks again. | 13:39.26 |
| tor8 | Robin_Watts: a linked list interface can always be exposed, regardless of how the list is actually allocated so we're kind of future proof there already. an array can still have (rather pointless) next pointers in it for the interface. the question is how to do the public/private and subclass (fz_annot vs pdf_annot) split. | 13:39.41 |
| chrisl | miha: you might be able to use Ghostscript and it's pdfwrite device, setting the PDF version to 1.3 - YMMV...... | 13:40.03 |
| | miha: note: that won't be quick, either :-( | 13:41.28 |
| miha | i tried generating thumbnails with gs, before moving to mupdf :) | 13:41.56 |
| chrisl | It's just an unpleasant file, on the whole :-( | 13:42.31 |
| sebras | tor8: you could state that all fz_annots are readonly (can they be?) and use pointer arithemtic to get at a linked list structure placed before the struct at hand. | 13:42.35 |
| tor8 | sebras: or require that the struct starts with a next pointer | 13:45.33 |
| Robin_Watts | tor8: The key thing, I feel is to define an interface and to use that, rather than to just expose everything. | 13:45.43 |
| tor8 | that might conflict with the storable struct too | 13:45.51 |
| sebras | tor8: how so? | 13:46.05 |
| tor8 | Robin_Watts: yes. accessors are probably best at this point, at least for these abstract types | 13:46.14 |
| | sebras: it requires that the ref count and free function are at the beginning of the struct | 13:46.30 |
| miha | chrisl: how to set pdf version? | 13:46.39 |
| kens | CompatibilityLevel | 13:46.57 |
| sebras | tor8: ah. | 13:46.59 |
| chrisl | miha: -dCompatibilityLevel=1.3 | 13:47.00 |
| Robin_Watts | So, I'll define a function to get the first one, a function to get the next one, and accessor functions for getting things within it. | 13:47.02 |
| chrisl | miha: but I fear it will be too slow to do at serve time | 13:47.39 |
| tor8 | Robin_Watts: fz_annot (opaque type) with fz_load_annots (or fz_first_annot), fz_next_annot, fz_bound_annot, fz_du_stuff_to_annot, etc? | 13:47.43 |
| Robin_Watts | tor8: yeah. | 13:48.00 |
| | If we like what I end up with, we should probably move fz_link over to the same style. | 13:48.39 |
| tor8 | Robin_Watts: so iterator without an explicit iterator, and accessors instead of exposing the innards. works for me, and I think it's similar to what paul has done (if I understood his earlier sentence correctly) | 13:48.44 |
| paulgardiner | tor8, Robin_Watts: yeah, that sounds good to me. | 13:49.20 |
| Robin_Watts | Yes. Not having an explicit iterator object removes my concerns about extra cleanup requirements, I think. | 13:49.21 |
| tor8 | Robin_Watts: agreed. all fz_document stuff should probably use the same pattern, except pages which are random access in nature so kind of deserve to be indexed | 13:49.27 |
| miha | chrisl: that took a minute, file is 50% longer and is slow as ever | 13:51.45 |
| | :( | 13:51.48 |
| chrisl | miha: there's no magic bullet, I'm afraid | 13:52.10 |
| Robin_Watts | tor8, paulgardiner: http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=f18dc92eb8014b335afdb720a8f5bc9c3bd7ceb7 | 13:52.12 |
| | That's the const fiddling to shut the warnings up. | 13:52.31 |
| tor8 | Robin_Watts: LGTM | 13:53.12 |
| paulgardiner | yep | 13:53.21 |
| Robin_Watts | Thanks. pushed. | 13:53.38 |
| sebras | Robin_Watts: I'm amazed that you can get away with having that few consts! :) | 13:53.53 |
| miha | for testing... these two load much faster ... http://www.wolfey.si/tmp/exotic.pdf http://www.wolfey.si/tmp/penthouse.pdf | 13:55.28 |
| | chrisl: thx for your support. at least i know i tried | 13:57.25 |
| chrisl | miha: neither of those have any transparency operations in them - so I think that identifies the culprit | 13:58.26 |
| miha | :D | 13:58.34 |
| paulgardiner | Robin_Watts: that "font missing from default appearance" bug is a bit worrying. Looks like it was overwriting of memory. No longer does it, but I haven't knowingly fixed it. | 14:09.40 |
| Robin_Watts | paulgardiner: AH. | 14:09.53 |
| | well, you could debug it using a memento build of the git revision that shows the problem ? | 14:10.18 |
| | I'll happily help you with Memento if need be. | 14:10.48 |
| | (there are tricks to using it within the debugger that can make things much easier) | 14:11.01 |
| paulgardiner | Robin_Watts: sounds like it would be worth it. | 14:11.10 |
| Robin_Watts | First off, I'd try and reproduce the problem using mujstest-v8 cos that will be completely reproducible. | 14:11.37 |
| | Was this the welsh file? | 14:11.48 |
| paulgardiner | yeah | 14:12.03 |
| Robin_Watts | http://ghostscript.com/~robin/dcacym.mjs | 14:12.34 |
| | win32/debug/mujstest-v8.exe -o out-%d.pdf -p ../ghostpcl/ ../ghostpcl/tests_private/pdf/forms/v1.4/dcacym.mjs | 14:13.14 |
| | You will need to edit the .mjs because your file is in a different place. | 14:16.04 |
| paulgardiner | yep | 14:16.11 |
| | What's the -p option? | 14:16.24 |
| | Ok. That seemed to work. | 14:19.25 |
| | png produced, and warnings appeared | 14:19.40 |
| Robin_Watts | The -p option gives a prefix to apply to the paths in the .mjs files. | 14:19.58 |
| | so all my mjs files are rooted at "tests_private", so I have to give ../ghostpcl/ to -p as I keep tests_private in ../ghostpcl | 14:20.33 |
| | So, in the solution, set mujstest-v8 as the startup project, select Memento configuration, and build. | 14:21.07 |
| | set the debug properties in the project to be your command line. | 14:21.22 |
| | breakset Memento_inited and Memento_breakpoint and run it. | 14:21.55 |
| paulgardiner | I may have got something wrong. It ran without hitting either of those breakpoints. When I added the breakpoints the files were greyed too. | 14:29.51 |
| | Ah. It forgot my config change | 14:30.29 |
| | Ok. It hit Momento_inited and now it's running. | 14:33.05 |
| Robin_Watts | ok. hopefully it'll run through and trip over a memory corruption. | 14:33.59 |
| | if it does, it should stop on Memento_breakpoint. | 14:34.12 |
| paulgardiner | It has stopped on that | 14:34.35 |
| | Output has overflowed the console | 14:35.09 |
| Robin_Watts | Ok, so look in the window for the running app (i.e. the windows window, not the MSVC one) | 14:35.10 |
| | You should be able to scroll it? | 14:35.28 |
| paulgardiner | Not far enough | 14:35.51 |
| Robin_Watts | fetches tea before meeting. | 14:46.00 |
| henrys | ahh meeting day | 14:56.42 |
| | paulgardiner:I wasn't expecting chrome to be missing so much. One has to wonder if forms are going to stay around with half assed implementations of an overblown spec. | 15:00.51 |
| | paulgardiner:sounds like your plan for the first release is right. | 15:01.51 |
| paulgardiner | henrys: Actually, I have yet to verify that Chrome doesn't support FDF responses. | 15:01.52 |
| Robin_Watts | I did a quick bit of googling to see if there were apache modules for FDF stuff. | 15:02.17 |
| paulgardiner | Robin_Watts: Oh yeah forgot to say. Adobe have an FDF toolkit for server side | 15:02.47 |
| Robin_Watts | http://pdfbox.apache.org/apidocs/org/apache/pdfbox/ImportFDF.html | 15:02.49 |
| paulgardiner | I'd imagined that handling FDF server side would be hellish, but the toolkit looks to make it fairly easy. | 15:03.36 |
| Robin_Watts | pdfbox wasn't what I was aiming for when I started searching :) | 15:03.57 |
| paulgardiner | That look interesting | 15:04.21 |
| Robin_Watts | pdfbox could maybe be considered a competitor. | 15:04.53 |
| paulgardiner | It would be good for us to support FDF responses, but I'm just hoping that we can avoid it for the initial release. | 15:05.38 |
| | The main thing that looks hard to me is having effectively two PDF docs open simultaneously and moving objects between the two. Is that something that falls naturally into MuPDF? | 15:06.59 |
| henrys | paulgardiner:well if you confirm chrome non support we are probably good. | 15:07.01 |
| Robin_Watts | paulgardiner: Currently, there is no support for that - but it's not outside the realm of possibility for that to be changed. | 15:07.56 |
| paulgardiner | henrys: Yeah. Not sure how to do that. I wondered about setting up the FDF toolkit on my own server. | 15:08.06 |
| Robin_Watts | paulgardiner: Either on your own server, or on a server running on casper ? | 15:08.42 |
| paulgardiner | Or we need to find a working site with forms using the feature and where filling out a form as a test wont upset anyone | 15:09.25 |
| Robin_Watts | Your own server is fine for you testing it, but when we come to try to test support it might help us to have a server we can all get to. | 15:09.28 |
| | I believe you have all the permissions required to build/run apache on casper on a non-standard port. | 15:10.05 |
| henrys | so does pdfbox use FDF or does it do everything itself? | 15:10.14 |
| Robin_Watts | pdfbox contains classes for handling FDF files. | 15:10.29 |
| | importFDF: This will takes the values from the fdf document and import them into the PDF document. | 15:11.12 |
| | ExportFDF: This will import an fdf document and write out another pdf. | 15:11.35 |
| henrys | and we would be okay license wise using that I believe. | 15:11.57 |
| Robin_Watts | It's in java. | 15:12.14 |
| | I suspect it's interesting to look at as an example of what someone else has found useful enough to write. | 15:12.44 |
| | and possibly the APIs they offer. | 15:12.50 |
| | dunno how much code can be carried over. | 15:13.00 |
| | Also XFDF import and export too. | 15:13.11 |
| henrys | well for now it's not an issue until paulgardiner establishes what chrome actually supports. | 15:14.00 |
| | tor8:do you have news for the meeting? | 15:14.33 |
| tor8 | henrys: no, I'm sitting back and enjoying watching robin and paul work :) | 15:15.38 |
| Robin_Watts | henrys: I've written mujstest (item 9 from last weeks list of tasks), and run all the v1.3 and v1.4 files in the test suite through them. | 15:16.32 |
| | A few files SEGVd, but those have all been sorted now. | 15:16.47 |
| henrys | Robin_Watts:I was just going to say quick nice work on the testing code. | 15:16.58 |
| Robin_Watts | I've committed all of paulgardiners 2500 test files into tests_private/forms | 15:17.06 |
| | henrys: It turned out far simpler than I'd dare hope. | 15:17.18 |
| | I haven't yet looked at all the png output for the 1.3 files, let alone the 1.{4,5,6,7} ones. | 15:17.48 |
| paulgardiner | Robin_Watts's test app also helps with item 8: it shows warnings from v8 where DOM objects are missing. | 15:18.04 |
| Robin_Watts | but already my pngs are out of date because pauls fixed some rendering issues. | 15:18.05 |
| | When marcosw returns we can talk to him about integrating mujstest into the cluster. | 15:18.32 |
| | paulgardiner: Did you see if you'd made any fixes to the v8 code? | 15:19.26 |
| paulgardiner | Robin_Watts: No. Hang on a min | 15:19.46 |
| henrys | paulgardiner:any other items on the list 1-9 to discuss? | 15:20.36 |
| paulgardiner | henrys: No. I don't think so. | 15:21.36 |
| Robin_Watts | henrys: We were discussing earlier whether to clone the v8 repo onto casper, and then keep our changes on a branch. | 15:22.17 |
| henrys | Robin_Watts, paulgardiner:you guys have to make tor8 do stuff, just kick him or something. | 15:22.44 |
| | Robin_Watts:ugh | 15:23.12 |
| Robin_Watts | That way we have a central repo for our changes, and can keep merging changes from upstream into it. | 15:23.30 |
| paulgardiner | Robin_Watts: I have made a change, but it's just two (int32_t) casts that were necessary because of having warnings mapped to errors | 15:23.44 |
| Robin_Watts | At the moment, the only copy we have is on paulgardiners machine, which isn't ideal :) | 15:23.49 |
| henrys | Since the mupdf developers are all okay with git maybe mupdf should consider using git externals? | 15:23.59 |
| tor8 | henrys: you mean git submodules? | 15:24.30 |
| henrys | or is v8 revisioned otherwise? | 15:24.34 |
| | tor8:yes | 15:24.46 |
| tor8 | v8 is in git, anything not in git we can put snapshots of releases into a git we host | 15:24.55 |
| | like for libpng and zlib | 15:25.00 |
| Robin_Watts | henrys: v8 is under git from google. | 15:25.03 |
| tor8 | the problem with mirroring third party gits directly is that they're often quite big | 15:25.21 |
| Robin_Watts | and I think we'd still want to do snapshots of v8 to coincide with our releases. | 15:25.24 |
| tor8 | but with huge disks and infinite bandwidth, maybe we shouldn't worry about that | 15:25.30 |
| henrys | I guess it should be consistent with all other 3rd party right? | 15:26.12 |
| Robin_Watts | The reason for mirroring v8 and keeping our stuff on a branch is to provide a buffer against upstream v8 changes suddenly breaking things. | 15:26.18 |
| henrys | why is v8 different? | 15:26.22 |
| tor8 | cloning the v8 git is probably ten times bigger than the mupdf git repo | 15:26.24 |
| paulgardiner | I think I may have given the impression that I'd needed to make significant changes to get it to build, but it seems that it was only a tiny change. | 15:26.42 |
| tor8 | Robin_Watts: we can do the same if we just base our code off of v8 release snapshots | 15:26.51 |
| | and even so, we can still have local fixes in a branch | 15:27.14 |
| | so I think it doesn't really matter if we mirror upstream or make snapshots in a git repo we maintain | 15:27.35 |
| Robin_Watts | tor8: ok. | 15:27.38 |
| jen_ | chrisl: I feel silly to ask but I need help building GS on Win. If you have time, I don't mind paying you to help guide. PM me. | 15:27.44 |
| chrisl | jen_: what are you going to use to build GS? | 15:28.13 |
| tor8 | Robin_Watts: so I'm undecided, mirroring has benefits too. | 15:28.41 |
| jen_ | chrisl: coming from a Mac world, I have no idea. I am running Win in VM. | 15:29.01 |
| henrys | so we are at the half hour now, request for next week that we return to the list format and have a progress report for each item that has progress. Does that seem reasonable to everyone? | 15:29.03 |
| Robin_Watts | Given that v8 is in git, and it's likely to get fixes between releases (security and otherwise), tracking it in git seemed natural to me - but I'm happy with any scheme that means we can easily rebuild source on other platforms without having to jump through hoops. | 15:29.10 |
| jen_ | chrisl: see, I do feel stupid. | 15:29.14 |
| tor8 | Robin_Watts: yeah. the problem is an upstream git is out of our hands, so if they move things around or break stuff we're in a more difficult spot. probably not worth worrying about though. | 15:29.53 |
| jen_ | chrisl: I need to create a frozen binary of GS since this will be packaged with an Adobe Air front end. | 15:30.03 |
| Robin_Watts | tor8: Right, but our branch would always be buildable. | 15:30.21 |
| chrisl | jen_: the only Windows dev environment we support is MS Visual Studio, so you 'll need to get hold of that | 15:30.28 |
| paulgardiner | henrys: Ah right. Sure. I was unsure whether to include the list and edit the items, but yes that makes sense: just include the items where progress has been made. | 15:30.40 |
| Robin_Watts | henrys: sure, except some things don't fit into the list format. | 15:30.52 |
| chrisl | jen_: you are aware of the license restrictions on Ghostscript? | 15:31.02 |
| jen_ | yes | 15:31.47 |
| henrys | Robin_Watts:well a separate paragraph for that stuff. I think we can have a better sense of progress with the list (well at least for me that's true) | 15:32.00 |
| Robin_Watts | jen_: Download "Microsoft Visual Studio 2005 Express" (The visual C/C++ edition if you have to choose) | 15:32.20 |
| | Then read win32/ReadMe.txt | 15:32.27 |
| | henrys: Sure. Listing progress within the items makes sense. | 15:32.55 |
| henrys | paulgardiner:are you good until next week? Anything else for this meeting? | 15:34.00 |
| paulgardiner | henrys: I would have added the list this week, but it seemed odd to put the unchanged items in. I didn't think of just listing the changed one, with the same numberings | 15:34.17 |
| | henrys: Yep. I'm good ta. I guess it's important to establish with more certainty what Chrome's FDF capabitlities are even if it means spending some time setting up a local server. | 15:35.25 |
| henrys | paulgardiner:I don't know your call I would be okay going with your assumption for a first release. | 15:36.36 |
| paulgardiner | henrys: Ok. Then I might look at it a bit, but back out if it looks like it will be more than a quick trivial test. | 15:37.44 |
| chrisl | jen_: Actually, now I think about it, why can't you just use the binary we build and ship? | 15:37.50 |
| paulgardiner | Did we decide what to do with v8? | 15:38.25 |
| | If we are going to mirror t then it doesn't have to be pushed from here since my changes are tiny. Easier to pull from casper | 15:40.26 |
| Robin_Watts | I still reckon that cloning the main v8 repo, keeping stuff on a branch and doing snapshots from that at release time is easiest all round, but if someone else wants to do it some other way, then so be it. | 15:41.03 |
| chrisl | What are the v8 changes? Just build related? | 15:41.37 |
| Robin_Watts | At the moment, it seems there are 2 casts :) | 15:41.57 |
| | and we could also check in the generated makefile to avoid people having to have the makefile generating tool installed. | 15:42.17 |
| ray_work | Robin_Watts: that _definitely_ sounds like a good idea | 15:42.45 |
| chrisl | Robin_Watts: probably not worth trying to push those up to v8 proper, then...... | 15:43.18 |
| paulgardiner | Robin_Watts: yeah and I could check in the sln files for Windows too | 15:44.06 |
| Robin_Watts | yeah, when I said makefiles I meant .sln etc too, sorry. | 15:44.47 |
| paulgardiner | chrisl: There was just a couple of casts it turned out. I misremembered it as more than that. | 15:44.55 |
| chrisl | paulgardiner: and do they serve a functional purpose, or just to silence a warning? | 15:45.32 |
| ray_work | are we using different cc flags (warnings as errors) to the "normal" v8 build, then ? | 15:46.01 |
| Robin_Watts | The makefile/solution sets warnings to be errors, so the lack of the casts makes it fail to build. | 15:46.24 |
| | ray_work: We may be using a different version of VS. | 15:46.43 |
| ray_work | how come the upstream doesn't see that error then ? | 15:46.45 |
| paulgardiner | ray_work: Yes that was the problem. It was easier to add the casts than find where to change the compiler options. | 15:46.46 |
| | All comments subject to my failing memory | 15:47.03 |
| | ray_work: I'd guess Windows isn't tested regularly | 15:47.32 |
| chrisl | Maintaining a repos and and branch just for a couple of casts (which ought to be fixed upstream anyway) seems a lot of effort, relatively speaking | 15:47.50 |
| ray_work | so they may be amenable to taking the changes upstream | 15:47.53 |
| Robin_Watts | chrisl: But a) it's not much work, b) presumably in future we're going to want to pull more changes in. | 15:48.40 |
| ray_work | chrisl: well, there is the issue of them keeping stuff working (they don't test with mupdf), so our users would have to be informed of what git hash we tested with (when we do a release) | 15:48.59 |
| paulgardiner | chrisl: Yes agreed, but doing so to easily include makefiles etc would be a big help to people. | 15:49.16 |
| chrisl | Yes, true enough. | 15:49.36 |
| | To all the above..... | 15:49.44 |
| Robin_Watts | At the point at which we want to make more changes, or we want to pull in fixes having it cloned from their git becomes a big advantage. | 15:50.01 |
| ray_work | is in favor of having our own v8 repo | 15:50.10 |
| | and if I am reading rightly, Robin_Watts is as well | 15:50.34 |
| Robin_Watts | As far as I can see the downside is "disc space". | 15:50.59 |
| | (and tor8 can just buy another SSD :) ) | 15:51.25 |
| chrisl | This does strike me as one case where we *might* have to support using pre-built binaries....... | 15:51.44 |
| henrys | everyones going to argue about it until we converge on something then tor8 will go off and do whatever he wants. | 15:51.52 |
| ray_work | would we be taking the entire v8, or just starting one fresh from a certain v8 snapshot | 15:52.01 |
| Robin_Watts | ray_work: That's the argument I think. | 15:52.16 |
| chrisl | henrys: you sound so jaded, suddenly :-) | 15:52.19 |
| Robin_Watts | I reckon to take the whole thing, tor8 (I think) is keener on working from v8 snapshots. | 15:52.36 |
| ray_work | chrisl: just experienced in tor's behavior | 15:52.52 |
| henrys | chrisl:I know all about these cat herds | 15:52.59 |
| chrisl | :-) | 15:53.09 |
| paulgardiner | I think, in addition, prebuilt libraries for selected builds would be good | 15:53.20 |
| ray_work | paulgardiner: for how many different platforms ? | 15:54.21 |
| paulgardiner | Well, just every time we build for one ourselves, include it. | 15:55.42 |
| ray_work | on the SSD topic, I saw the "hybrid" SSD+HD drives -- almost went for one on my new laptop | 15:55.46 |
| Robin_Watts | win32,x86 linux,amd64 linux (debug/release) | 15:56.19 |
| | maybe memento too if it can be done usefully. | 15:56.32 |
| | possibly win64 too, and maybe arm linux. | 15:56.50 |
| ray_work | Robin_Watts: don't forget mac ? | 15:57.05 |
| Robin_Watts | right. | 15:57.13 |
| | well, hopefully x86linux == mac for our purposes? | 15:57.44 |
| chrisl | Android, iOS? | 15:57.44 |
| mvrhel | good morning | 15:57.55 |
| Robin_Watts | arm linux == android? | 15:58.04 |
| kens | Hey a marcosw is here | 15:58.08 |
| Robin_Watts | Morning marcosw: Where are you? | 15:58.17 |
| ray_work | sounds like a nightmare to me, but maybe if we say "and if you have any problems, don't bother us, just build it yourself" | 15:58.46 |
| marcosw | hello everyone, I'm back home | 15:58.48 |
| henrys | Ahoy marcosw! | 15:58.51 |
| ray_work | morning, mvrhel | 15:59.00 |
| | hi, marcosw. We missed you (and have a lot stacked up for you) ;-) | 15:59.20 |
| | (not me, but I see "marcosw" mentioned a lot in the logs) | 15:59.59 |
| Robin_Watts | marcosw: Yeah, when you're recovered, I have a new test thing to add to the cluster. | 16:00.07 |
| henrys | so I guess we should start the meeting | 16:00.40 |
| ray_work | and it would be nice to get miles and kilometers kick started | 16:00.42 |
| marcosw | It was strange having no access to the internets or even email for nearly a week. The ship claimed they had satellite internets but it was so slow and unreliable to be worse then uselss. | 16:01.00 |
| henrys | I went to look at the macbook I wonder if X11 works with the high resolution. If so we could have close to 200 dpi with ghostscript on the screen. Anybody know if X does work properly with the retinal displays? | 16:01.16 |
| ray_work | marcosw: not to mention pricey (if it like the cruise I was on) | 16:01.24 |
| marcosw | it was $0.55/minute, which was better than the $5.00/minute they charged for voice calls. | 16:01.51 |
| ray_work | henrys: what is the dpi (200 ?) | 16:01.58 |
| tor8 | morning mvrhel. I dumped a transparency bug on you, sorry! | 16:02.21 |
| marcosw | henrys: I should be able to let you know in 2-3 weeks :-) | 16:02.27 |
| ray_work | for my eyes, my 150 dpi screen is plenty close enough the "retinal" | 16:03.02 |
| mvrhel | tor8: yes I saw that | 16:03.06 |
| tor8 | mvrhel, ray_work: the xps transparency bug looked like it was the same issue as the one you mentioned, ray. I don't think it is though: it only happens when the second soft mask is made up from a tiling pattern. | 16:03.11 |
| mvrhel | that does sound like a special case | 16:03.42 |
| tor8 | mvrhel: I also saw you added a gs_pop_transparency_state but no matching gs_push_transparency_state -- not sure if that was intended | 16:03.49 |
| mvrhel | not sure I have seen that in a pdf file actually | 16:03.49 |
| tor8 | mvrhel: if you unzip the xps file, you can run gxps on the _rels/.rels file to test without having to rezip it | 16:04.18 |
| mvrhel | ok | 16:04.29 |
| Robin_Watts | alexcher: Last I saw, you had commented on the bug that we'd need to use the whole clippath rectangle, or we'd need to prescan the content stream to work out a bbox. | 16:05.15 |
| henrys | marcosw:are you officially back if so I'll hand over a support item to you. | 16:05.28 |
| | ? | 16:05.30 |
| Robin_Watts | I commented that using the complete clippath rectangle sounded right to me. | 16:05.36 |
| mvrhel | tor8: there is a push and pop for the state | 16:05.50 |
| Robin_Watts | alexcher: Are you happy to proceed on that basis or do you want other peoples input? | 16:05.51 |
| ray_work | tor8: I was just mentioning it in case XPS was tripping over the same issue where setting a SMask doesn't really do anything until the group is popped | 16:05.52 |
| henrys | mvrhel:you got norbert's ping right? | 16:06.27 |
| mvrhel | unless you are talking about in the xps interpreter code | 16:06.30 |
| alexcher | Robin_Watts: Yes, I'll use the full clip box. | 16:06.37 |
| mvrhel | henrys: yes I just read it | 16:06.37 |
| Robin_Watts | alexcher: Fab. Thanks. | 16:06.48 |
| mvrhel | henrys: I will take a look at that in the next couple days to see what the issue is | 16:07.01 |
| tor8 | mvrhel: I am talking about the xps interpreter, indeed :) | 16:07.04 |
| alexcher | Robin_Watts: More sophisticated approach can be added later. | 16:07.06 |
| mvrhel | ok. that would be a problem probably | 16:07.15 |
| | tor8 ^ | 16:07.21 |
| Robin_Watts | tor8, paul, sebras: New patch with the iterator stuff on my casper repo. | 16:07.27 |
| ray_work | Robin_Watts: in fact I think we have to use the groups bbox -- running through all of the painting operations until the SMask is changed is way too onerous | 16:07.41 |
| Robin_Watts | ray_work: I am inclined to agree. | 16:08.17 |
| henrys | alexcher:you've had a little progress on the mupdf parser integration can that be published? | 16:08.29 |
| Robin_Watts | Could we safely restrict it to the bbox of the SMask itself? | 16:08.56 |
| alexcher | henrys: yes, I'll commit to the branch. | 16:09.16 |
| ray_work | mvrhel: are there any blend modes where painting outside the SMask area would have an effect ? | 16:09.29 |
| mvrhel | ray_work: I don't quite understand the question. Is this while drawing through the SMask and you are drawing outside its bounding box? Like the issue that we talked about before? | 16:10.33 |
| henrys | well support has been fairly quiet in marcosw's absence. | 16:10.41 |
| mvrhel | ray_work: and you have the background color? | 16:10.51 |
| ray_work | Robin_Watts: thinking about it, that sounds right since the alpha for any painting outside the SMask will be zero | 16:10.58 |
| henrys | kens:anything for the meeting? | 16:11.06 |
| kens | Nope, buried in weird problems with CIDFonts and wrigin modes | 16:11.19 |
| | writing* | 16:11.26 |
| ray_work | mvrhel: yes, the drawing operations can (and probably often do) go outside the SMask bbox | 16:11.28 |
| henrys | oh does anyone know - top of head - why the tiff output geometry would change for the same page size between 8.x and 9.x? | 16:11.55 |
| kens | henrys I looked at the output, and I don't really understand it | 16:12.19 |
| chrisl | henrys: did the geometry change? I thought it was just the file size | 16:12.32 |
| kens | It looked like pages 2 & 3 were rendered at a non-square reoslution, but not page 1 | 16:12.37 |
| henrys | chrisl:no if you look at his tiffs the y extant is about 30 pixels different. | 16:12.55 |
| Robin_Watts | henrys: When did we move to using libtiff ? | 16:13.10 |
| chrisl | We switched to libtiff for tiff output at 8.71 | 16:13.11 |
| ray_work | mvrhel: isn't the SMask bbox the same as the group's bbox ? | 16:13.12 |
| Robin_Watts | chrisl: Wow, fast answer :) | 16:13.22 |
| ray_work | Robin_Watts: over 1 year ago (lars did that, iirc) | 16:13.33 |
| mvrhel | ray_work: yes, but the mask can have an effect outside the bbox due to its BC value | 16:13.52 |
| Robin_Watts | ray_work: Right, so the libtiff change would have been broadly coincident with the change. | 16:14.02 |
| chrisl | henrys: it could be a bug in our driving or libtiff, or with libtiff - might be worth checking with the current code, since I updated libtiff | 16:14.12 |
| | s/driving or libtiff/driving of libtiff | 16:14.33 |
| Robin_Watts | ray_work, mvrhel: I suspect alexcher is right - make it work to start with, then worry about adding optimisations later. | 16:14.40 |
| ray_work | Robin_Watts: alexcher: I am totally in favor of that | 16:14.58 |
| henrys | I was going to wait until he sent me answers to my questions but I can do a quick check with his postscript file to see if I can reproduce it now. unless marcosw wants to this over ;-) | 16:15.06 |
| | s/wants to/wants to take/ | 16:16.19 |
| | is he still on the boat or has he found land? | 16:16.40 |
| kens | claims to be home.... | 16:16.52 |
| henrys | I guess I'll look at it. | 16:17.34 |
| ray_work | scott-san told a guy at the new york show to contact raph (and must have told him to use raph's old artifex email address) -- just a salesman trying to sell cloud service | 16:17.56 |
| marcosw | I'm here and reading the last ~10 days of email. | 16:17.58 |
| henrys | and one other thing - the temporary files. I think it is a real bug if we don't clean those up even for a crash. and as ray_work pointed out this could be the reason for miles and kilometers being down. Thoughts? | 16:18.38 |
| kens | Sounds like a job for the cluster script to me | 16:19.09 |
| Robin_Watts | henrys: There is no portable way that we can clean these up when we SEGV. | 16:19.34 |
| henrys | my fear with the cluster fix is we'll call it a fix and forget about it. | 16:19.40 |
| marcosw | kens: the cluster script does clean out /tmp | 16:19.41 |
| ray_work | henrys: if we do it using a gp_tempfile hook, then it is fine with me. That way systems that support tmpfile can do the right thing | 16:19.54 |
| kens | marcosw at the end of the run ? | 16:19.56 |
| henrys | marcosw:we've had overflows on several machines? | 16:20.01 |
| Robin_Watts | marcosw: While you were away, we had 50Gig of temp files left on some machines, I believe. | 16:20.26 |
| henrys | marcosw:I assigned a bug to you. | 16:20.30 |
| ray_work | marcosw: I don't think the cluster script is doing that (correctly). I run a cron job on peeves because my /tmp was filling up | 16:20.49 |
| marcosw | actually it doesn't. I run a cron job on the cluster nodes that does it. | 16:20.58 |
| alexcher | henrys: I had it on i7b. | 16:21.02 |
| henrys | henrsyx6 filled the disk with 250 G in /tmp - all gs_* | 16:21.04 |
| Robin_Watts | http://ghostscript.com/irclogs/2012/06/14.html see around 18:33 | 16:21.08 |
| marcosw | 0 0 * * * sudo /usr/bin/find /tmp -ctime +2 -name gs_\* -type f -exec /bin/rm -fv '{}' \; | 16:21.32 |
| ray_work | does anyone else object to adding a gp_tempfile to the gp hooks so we can use that | 16:22.08 |
| | marcosw: maybe you weren't a sudoer on all of the machines ? | 16:22.52 |
| marcosw | there was (is) also an issue with large numbers of debugging files being written during clusterpush.pl runs. | 16:22.53 |
| chrisl | ray_work: it will need to communicate to the "close" method to delete the file on systems like Windows | 16:23.03 |
| marcosw | ray_work: probably I only set it up on machines that I control (which doesn't include peeves, macpro, i7a, i7b, and henrysx6). | 16:23.20 |
| Robin_Watts | marcosw: That was my bad, I pushed a test with a debug define in, so you got lots of .RAW files. | 16:23.24 |
| henrys | ray_work:I'm fine with it. As Robin_Watts said it doesn't work on all platforms but I think most modern systems will be covered. | 16:23.27 |
| Robin_Watts | Urm... 1 potential problem. | 16:23.39 |
| ray_work | chrisl: of we use a gp function, we don't have to implement it on Windows | 16:23.43 |
| Robin_Watts | AIUI, we currently write temporary files to /tmp/gs_.... so it's easy to know the files that are ours. | 16:23.58 |
| marcosw | Just reboot your machine every night, that cleans out /tmp :-) | 16:24.09 |
| Robin_Watts | If we move to using a gp_ hook for that then we'll write to /tmp/AASDFSDFSDF (i.e. random stream of garbage) | 16:24.32 |
| | If we then SEGV, the tmpfiles may well not be cleared out. | 16:24.49 |
| ray_work | Robin_Watts: tmpfile can't provide a prefix ? | 16:25.01 |
| Robin_Watts | and we can't automatically delete them based on name any more. | 16:25.05 |
| henrys | we do expect server behaviour right - that means (to me) recovery and restart from a crash without creating a mess. | 16:25.16 |
| chrisl | Robin_Watts: the point of tmpfile on Unix is that you don't have to delete the files when you're done | 16:25.17 |
| Robin_Watts | tmpfile can't have a prefix, no. | 16:25.32 |
| | chrisl: For normal exit of programs, yes. | 16:25.42 |
| chrisl | Normally temp files are unlinked on creation | 16:26.04 |
| Robin_Watts | I'm not sure they get correctly cleaned up on SEGV. (I'll happily be told I'm wrong) | 16:26.12 |
| | Ah... | 16:26.16 |
| henrys | Robin_Watts:no most modern os's support deletion of temps after segv. | 16:26.18 |
| chrisl | Robin_Watts: actually, I'm not sure if that's how the tmpfile API works..... | 16:26.45 |
| | "The tmpfile() function opens a unique temporary file in binary read/write (w+b) mode. The file will be automatically deleted when it is closed or the program terminates." | 16:27.14 |
| ray_work | a gp_tempfile that unlinks on creation (where the OS can support that) seems like an OK approach. That way we can have a prefix | 16:27.30 |
| Robin_Watts | ray_work: Right, so we can roll our own for unix in a way that should work. | 16:29.00 |
| henrys | see comment #5 693121 | 16:29.13 |
| alexcher | ray_work: AFAIK unlinking of open files doesn't work on Winsows. | 16:29.48 |
| chrisl | alexcher: I don't think there is any way to achieve this on Windows :-( | 16:30.15 |
| ray_work | alexcher: that's why I suggested using the gp_ method -- we only support it where the OS does | 16:30.22 |
| Robin_Watts | henrys: Right, but our solution that works right will not be portable. What ray_work said. | 16:30.30 |
| | chrisl: That's OK. Windows reboots every hour for exactly this reason :) | 16:31.02 |
| chrisl | :-) | 16:31.14 |
| alexcher | Windows can open temp files with CreateFile(). | 16:31.18 |
| henrys | yes scratch the last paragraph | 16:31.20 |
| ray_work | but at least "real" OS's (linux/unix) will not have the problem (and that will solve our cluster issue, but also help users) | 16:31.36 |
| Robin_Watts | ray_work: Certainly, we'll be no worse off on any platform, and better on some. So that sounds like a win. | 16:32.03 |
| henrys | ray_work:so are you going to take the assignment? | 16:32.18 |
| chrisl | alexcher: it still needs explicitly deleted on Windows, doesn't it? | 16:32.35 |
| marcosw | Why don't we right out the process id as part of the tmp filename (i.e. gs_1288_AgXBOA) and then when Ghostscript starts it can remove the tempfiles from any previous Ghostscript jobs that are no longer around. | 16:32.38 |
| Robin_Watts | marcosw: ew. | 16:32.56 |
| marcosw | exactly~ | 16:33.02 |
| chrisl | No cross platform way of getting a PID? | 16:33.04 |
| alexcher | chrisl: Yes, there was an article in MSDN how to do it. | 16:33.09 |
| Robin_Watts | suppose we have 1 disc shared between multiple machines? | 16:33.19 |
| chrisl | alexcher: does that handle the case where the program exist unexpectedly? | 16:33.39 |
| marcosw | Robin_Watts: no problem, we also write out the hostname as part of the temp filename. | 16:33.51 |
| chrisl | And no cross platform way of getting the host name...... | 16:34.10 |
| marcosw | of course there could be two machines with the same hostname :-) | 16:34.14 |
| ray_work | marcosw: that seems UGLY. but doing a 'rm' on /tmp/gs_* doesn't interfere with running jobs (otherwise my 'cron' would have caused errors) | 16:34.14 |
| alexcher | chrisl: Of course, that's Windows. | 16:34.18 |
| marcosw | when you guys say 'cross-platform' I think UNIX/Linux/MacOS, I keep forgetting there are others. | 16:35.13 |
| ray_work | chrisl: alexcher: I think the "FILE_FLAG_DELETE_ON_CLOSE" is what is needed for Windows | 16:35.19 |
| henrys | marcosw:I must be missing something why does this cron work at all? Isn't there the chance you will delete files in use? | 16:35.40 |
| | oh ctime. | 16:35.57 |
| | got it. | 16:36.02 |
| ray_work | but I have no idea if the OS closes the file when the job dies (since I've sometimes had files that were "locked" until I rebooted) | 16:36.09 |
| marcosw | right, 2 day old files are likely no longer being used. | 16:36.12 |
| chrisl | ray_work: that's my concern. | 16:36.34 |
| ray_work | henrys: it doesn't delete the files that are in use | 16:36.35 |
| | chrisl: but that's only on windows, and we don't care if the OS flakes out | 16:37.02 |
| henrys | ray_work:I believe it would on linux - that's why marcosw used ctime. | 16:37.09 |
| chrisl | marcosw: If we're talking over-complicated solutions, we could run a watchdog process which monitors the main gs process, and if it crashes, clean up the files | 16:37.38 |
| ray_work | henrys: on linux rm works while gs is running -- I tested it on peeves before doing my cron job | 16:38.15 |
| marcosw | deleting an open file on Linux doesn't cause problems, the original program can still read/write to the file. | 16:38.19 |
| chrisl | Deleting an open file is just like unlinking immediately after opening | 16:38.48 |
| marcosw | off course it can't close and reopen it a file, and I wasn't sure if Ghostscript does that, so hence my use of ctime. | 16:39.07 |
| ray_work | deleting (or attempting to) delete all _te* files on Windows just deletes those no longer in use | 16:39.42 |
| marcosw | oops, I need to run to school. I have a 11:00 meeting. Be back online later today. | 16:39.55 |
| henrys | it is 10 minutes past "meeting over" | 16:40.12 |
| ray_work | and I've never seen the _te* files be "locked" so I think that using CreateFile with FILE_FLAG_DELETE_ON_CLOSE will work as a gp_tempfile implementation | 16:40.35 |
| chrisl | ray_work: it's worth a try, at least. | 16:41.01 |
| | henrys: I've got a quick PCL font question for you....... | 16:41.19 |
| henrys | okay | 16:41.27 |
| chrisl | Artificial boldening: are there restrictions on when it can be used? | 16:41.45 |
| ray_work | chrisl: et al.: so we can agree on a FILE *gp_tempfile(char *str) addition ?? | 16:41.52 |
| chrisl | ray_work: I'm happy with that | 16:42.02 |
| Robin_Watts | ray_work: Sounds good to me. | 16:42.04 |
| henrys | okay ray_work take the bug assignment please | 16:42.28 |
| ray_work | henrys: OK. | 16:42.34 |
| henrys | chrisl:the algorithm is done on an image so the font must be "rendered" first (Artifex Scaler) Monotype scaler boldens the vector and we don't yet support that. | 16:43.49 |
| chrisl | henrys: UFST says it only support boldening on intellifonts | 16:44.27 |
| henrys | so perhaps we should just always use our boldening code. | 16:45.11 |
| | get a bitmap back from the ufst and bolden that? | 16:45.27 |
| chrisl | I can look at doing that - it seems a shame, though | 16:45.59 |
| ray_work | ALL: one thought, should we do it FILE *gp_tempfile(char *str, int strlen) and write the actual file name (limiting to strlen) back so we can explicitly delete it | 16:47.19 |
| henrys | you mean not to support ufst intellifont boldening? Intellifonts are completely dead we have just what's needed to get us through the fts. | 16:47.28 |
| Robin_Watts | We don't embolden at the vector level ? | 16:47.29 |
| ray_work | for systems that can't support gp_tempfile | 16:47.29 |
| Robin_Watts | ray_work: We should have a gp_tempfile_close() that gets called when the file is closed. | 16:48.26 |
| henrys | Robin_Watts and it is something I've never seen in a real world file so I've never given it much thought. | 16:48.36 |
| ray_work | Robin_Watts: why ? | 16:48.51 |
| Robin_Watts | henrys: Good answer. | 16:48.54 |
| chrisl | henrys: I guess I mean (once again) what a heap of cr*p UFST must be to only do it for one relatively pointless format :-( | 16:49.19 |
| Robin_Watts | ray_work: So the details of whether the implementation has to delete on close can be handled within the implementation. | 16:49.26 |
| ray_work | Robin_Watts: if we always attempt to close our tempfiles, then why do we need the gp_tempfile_close ? We use gp_tempfile just to allow for the 'abort' (SEGV) case | 16:50.29 |
| chrisl | ray_work: you could add a flag to the file object attributes, so the usual "close" method would know whether to delete or not | 16:50.30 |
| henrys | chrisl:I think monotype has moved on, pcl emboldening is probably not a priority. | 16:50.42 |
| ray_work | what's the harm in always doing an explict close ourselves ? | 16:51.01 |
| chrisl | henrys: fair enough. It's just that there are implications for XPS, too :-( | 16:51.24 |
| Robin_Watts | ray_work: On some systems, we need the "close" to "delete" too, right? | 16:52.30 |
| ray_work | I am misstating -- I meant that we always DELETE | 16:52.34 |
| | sorry. | 16:52.36 |
| Robin_Watts | deleting a file that has already been deleted is a race condition. | 16:52.52 |
| ray_work | and if the file was already 'unlinked' as the unix does, it will just get a file not found and continue | 16:53.10 |
| | Robin_Watts: a race condition ? how so | 16:53.26 |
| Robin_Watts | ray_work: someone else might have created a file with the same name in the meantime. | 16:53.39 |
| ray_work | Robin_Watts: not if the file is created with tmpnam (as we do now) | 16:53.51 |
| Robin_Watts | Yes, if the file is created with tmpnam. | 16:54.05 |
| | gs starts up, creates a file with tmpnam (fred), it uses it, it closes the file which deletes it. Then we get a context switch. | 16:54.48 |
| ray_work | Robin_Watts: I didn't realize that | 16:54.56 |
| henrys | chrisl:the only reason to buy the ufst is for the fonts and proper names. That's of no use to XPS, I wonder if any XPS folks use the ufst scaler. | 16:55.04 |
| Robin_Watts | Another program calls tmpnam, and opens a new temporary file that happens to get the same name (fred). | 16:55.12 |
| | Then we context switch back to our gs, and that deletes fred. | 16:55.22 |
| | It's unlikely, but it's possible. | 16:55.41 |
| ray_work | Robin_Watts: if the gp_tempfile returns with str[0]="\0 | 16:55.42 |
| | then the caller knows not to delete it later | 16:55.55 |
| Robin_Watts | It sounds horrid. | 16:56.06 |
| | If we're going to offer a general call to allow us to make a temporary file, I'd personally prefer if it handled everything. | 16:56.37 |
| chrisl | henrys: I don't know, but if someone was building a PS/PDF/PCL/PXL/XPS work flow, I could reasonably see them wanting the option to use the same scaler for all..... | 16:56.37 |
| Robin_Watts | And the easiest way to do that is (IMAO) to offer bracketing open/close calls. | 16:56.58 |
| | but any way you do it that doesn't present a race condition is fine with me. | 16:57.17 |
| ray_work | Robin_Watts: OK, I give in | 16:57.27 |
| chrisl | It seems unfortunate to have to treat temp file objects differently from file objects outside of the file handling code :-( | 16:58.20 |
| Robin_Watts | chrisl: indeed, but I can't see an alternative. | 16:58.41 |
| | Well, actually, we could do a 'general' implementation that uses tmpfile (with no prefix) and that will work fine as long as we don't SEGV. | 16:59.30 |
| chrisl | Robin_Watts: Well, as i said, set a tmpfile attribute in the file object when the file needs deleted, the the "close" method handles that automatically | 16:59.30 |
| Robin_Watts | chrisl: That's not portable. | 16:59.52 |
| chrisl | Huh? Why not? | 17:00.09 |
| ray_work | chrisl: I actually like having the gp_tempfile_close idea better -- we always call that (unless we segv) and the implementation decides whether or not it is a NOOP | 17:00.23 |
| Robin_Watts | Doing it on unix (and even on windows) is perfectly possible with only the open call, AIUI. | 17:00.30 |
| chrisl | Robin_Watts: yes, so on Unix the attribute wouldn't be set | 17:00.50 |
| Robin_Watts | but I can't see how to portably implement it without a _close() call. | 17:01.16 |
| ray_work | chrisl: but on ThreadX or some lame-o OS, we need to explictly close | 17:01.18 |
| Robin_Watts | (and if we're relying on a _close then we'll leak in a SEGV case anyway, so we might as well have used tmpfile and be done with it) | 17:01.48 |
| ray_work | Robin_Watts: I agree to the gp_tempfile_open and gp_tempfile_close (updated the bug comment to say that) | 17:01.49 |
| chrisl | ray_work: we always need to close, we sometimes need to delete | 17:01.49 |
| ray_work | chrisl: tempfile_close _does_ delete | 17:02.12 |
| Robin_Watts | chrisl: you said "set a tmpfile attribute in the file object when the file needs deleted, the the "close" method handles that automatically" | 17:02.43 |
| | in what file object? the FILE * ? | 17:02.55 |
| chrisl | No, our object - don't we wrap temp file objects up in a stream or something? | 17:03.22 |
| Robin_Watts | No. | 17:03.28 |
| | (not AIUI) | 17:03.33 |
| | certainly not in a way we'd want to export to the gp_ functions. | 17:03.54 |
| chrisl | Hmm, that's not ideal...... | 17:04.00 |
| ray_work | I assume int gp_tempfile_close(FILE *, char *fname) | 17:04.11 |
| | chrisl: no, we use FILE * objects for tempfiles | 17:04.56 |
| chrisl | Of course we do - consistency, pah! | 17:05.24 |
| | ray_work: in that case, I agree, matching open and close methods for temp files | 17:07.00 |
| ray_work | would it gp_tempfile_remove be more clear | 17:07.19 |
| chrisl | Yeh, that would be more descriptive | 17:08.02 |
| ray_work | and do we need the FILE * or should we just go with the fname ? | 17:08.10 |
| Robin_Watts | I would have expected: | 17:08.35 |
| chrisl | You need the FILE * to close the file.... | 17:08.51 |
| ray_work | chrisl: but you can remove (unlink) an open file, right ? | 17:09.09 |
| Robin_Watts | FILE *gp_tempfile_open(void **token); then gp_tempfile_close(void *token); | 17:09.12 |
| | ray_work: Not on all systems. | 17:09.22 |
| chrisl | ray_work: if you unlink an open file on Unix like systems, the data remains on the disc until it is closed | 17:09.53 |
| Robin_Watts | That way the open call can return a token with all the information in it it needs to close. | 17:10.10 |
| | But honestly... I think we may be overcomplicating here. | 17:11.00 |
| | Why not just simply have: FILE *gp_tempfile_open(); | 17:11.25 |
| | on unix machines this can make a file, and unlink it immediately. | 17:11.38 |
| | on windows machines this can make a file, using the 'DELETE ON CLOSE' flag. | 17:11.53 |
| | on all other machines we just call tmpfile. | 17:12.01 |
| | The sole thing we lose there is that "on all other machines" (i.e. ones we don't care about) we don't get a gs_ prefix on the file. | 17:12.29 |
| ray_work | Robin_Watts: since we are using standard C calls (tmpnam, fopen, remove, ...) what's wrong with: gp_tempfile_close(FILE *, char *fname) -- that's all ANY implementation needs | 17:12.30 |
| Robin_Watts | ray_work: OK, so you want to get a filename back from the open? | 17:13.22 |
| | I don't see that the added complication of _close is justified. | 17:14.13 |
| ray_work | Robin_Watts: sorry -- gp_tempfile_delete(FILE *, char *fname) | 17:14.47 |
| Robin_Watts | The only case where we ever lose out by not offering a close is on an a non-unix/non-windows box where we get a SEGV and fail to delete the file. | 17:14.52 |
| | ray_work: _delete/_close - name makes no difference to my argument. | 17:15.11 |
| ray_work | Robin_Watts: on unix that would just close the FILE * (since it was unlinked at open) | 17:15.24 |
| Robin_Watts | still seems nastier than just getting a FILE * from gp_tempfile_open and then just fclosing it as we do now. | 17:16.29 |
| ray_work | oops. Guess what? there is already a gp_open_scratch_file | 17:16.30 |
| | so all we are missing is gp_delete_scratch_file to do the clean up in a way that is consistent with unlink on open | 17:17.45 |
| Robin_Watts | hello? Am I talking to myself? | 17:18.09 |
| ray_work | Robin_Watts: the problem (AIUI) is that if we fclose then we can't unlink it, right ? | 17:19.24 |
| Robin_Watts | ray_work: On unix, we can create a file, and unlink it immediately. | 17:19.51 |
| | On windows, we can create a file with the 'DELETE ON CLOSE' flag. | 17:20.10 |
| ray_work | Robin_Watts: right, and then the client code can fclose it, but it doesn't know whether or not is ALSO can/must unlink it | 17:20.36 |
| Robin_Watts | so the only case we're arguing about is on non-windows, non-unix platforms. | 17:20.39 |
| | right? | 17:20.42 |
| ray_work | Robin_Watts: you were the one that pointed out that unlinking a file by name that has already been unlinked may race | 17:21.15 |
| Robin_Watts | Right. | 17:21.27 |
| chrisl | Surely, on UNix and Windows, you only supply a gs_tempfile_open() and no gp_tempfile_close()? | 17:21.29 |
| ray_work | Robin_Watts: yes, we need to allow for the OS not supporting 'delete on close' | 17:21.41 |
| chrisl | s/gs/gp | 17:21.43 |
| Robin_Watts | I'm saying that we NEVER NEED TO UNLINK | 17:21.54 |
| | If we use tmpfile from the standard C lib, then we will get 'delete on close'. | 17:22.18 |
| | We might not get "delete on close in the case of SEGV", but that's a small price to pay. | 17:22.37 |
| | (and we don't get to specify a prefix for the tempname either) | 17:22.55 |
| ray_work | I thought someone (Robin ?) had an objection to the lack of prefix with tmpfile | 17:23.15 |
| Robin_Watts | not me. | 17:23.25 |
| | but actually... we have a problem... | 17:23.27 |
| | Looking at the windows implementation of gp_open_scratch_file, it specifically says that we can't use FILE_FLAG_DELETE_ON_CLOSE | 17:24.02 |
| | because clist_fclose closes files, then reopens them later. | 17:24.29 |
| | because it can have too many files to have open all at once. | 17:24.42 |
| chrisl | Oh, no don't do that..... eek! | 17:24.43 |
| Robin_Watts | oh, no, I'm misreading the comment. | 17:26.12 |
| | clist_fclose closes the file, then optionally deletes it. | 17:26.28 |
| ray_work | oops -- I have to go. | 17:26.50 |
| | bbiab... | 17:26.57 |
| chrisl | me too.... | 17:30.16 |
| | Actually, I have a few minutes.... | 17:31.38 |
| Robin_Watts | chrisl: My brain has melted. | 17:31.48 |
| chrisl | Robin_Watts: This question of whether or not we need to unlink on close - surely it's moot | 17:31.56 |
| Robin_Watts | It looks like temporary files can't delete on close automatically cos of some strange usage in clist. | 17:32.14 |
| chrisl | Then they're not temp files... <sigh> | 17:32.36 |
| | Robin_Watts: anyway, what I was going to say was that we're talking about platform specific code which, by it's very nature, has to be tailored to the capabilities of the platform..... | 17:33.28 |
| Robin_Watts | I suspect we need a 2 step solution to this. Step 1) Understand what is going on, 2) fix it! | 17:33.30 |
| | chrisl: We are talking about creating an interface behind which platform specific code can live. | 17:34.05 |
| chrisl | Robin_Watts: yes, so the closing and unlinking would happen in the platform specific code, as required | 17:34.45 |
| Robin_Watts | The interface should serve to keep gs clean from platform specifics, but to be (just) expressive enough so that any platform limitations we meet can be accomodated under that interface. | 17:35.11 |
| | chrisl: Yes. | 17:35.24 |
| chrisl | So, the question is moot | 17:35.47 |
| Robin_Watts | what question? | 17:36.01 |
| chrisl | Whether or not we need to unlink at close time | 17:36.20 |
| Robin_Watts | 'we' being? 'gs' or 'a given gp implementation' ? | 17:36.45 |
| chrisl | gs - that choice would be part of the platform specificness of the code | 17:36.57 |
| Robin_Watts | absolutely. | 17:37.02 |
| | however... it looks like these aren't 'temp' files in the way that you and I would like them to be. | 17:37.25 |
| chrisl | But given your discovery about the clist files, the entire discussions seems pointless :-( | 17:37.34 |
| | FWIW, for now, I'd say let's have the cluster clean up after itself..... | 17:38.02 |
| Robin_Watts | Me too. | 17:38.09 |
| | Frankly when something dies with a SEGV, I reckon all bets can reasonably be off about it leaving crap around. | 17:38.29 |
| chrisl | Yes, I'd agree..... | 17:39.32 |
| | Anyway, this time I really do have to do - ttfn! | 17:40.17 |
| mvrhel | bbiaw | 18:05.50 |
| ray_laptop | OK, it looks like we _might_ be able to change to using 'tmpfie' (where the name is opaque to us). I will need to add a 'fclone' to the clist_ioprocs, however so that the mt render can open new streams on the same file without needing the name. | 19:10.01 |
| | in the gxclfile.c routines this would return the fdopen(fileno(f)) | 19:10.53 |
| | then we wouldn't fclose and fopen in the thread setup/teardown anymore. No fclose will happen until the clist device is closed, then | 19:12.19 |
| Robin_Watts | the problem with using tmpfile for clist (AIUI the comment) is that you can't keep all the clist files you'd like open all the time. | 19:12.37 |
| | (windows can only cope with so many files open at once) | 19:12.55 |
| | If we use tmpfile, then when we close the files they are lost. | 19:13.18 |
| | so we can't close the files. | 19:13.27 |
| ray_laptop | if you mean the comment about FILE_DELETE_ON_CLOSE (or wahtever it is) I don't think that comment is relevant | 19:13.41 |
| Robin_Watts | Also note that | 19:14.15 |
| | * gdev_prn_render_pages requires multiple temporary files | 19:14.17 |
| | * to exist simultaneousely, so that keeping all them opened | 19:14.19 |
| | * may exceed available CRTL file handles. | 19:14.21 |
| ray_laptop | Robin_Watts: right, that's why I need to add a clist_fclone function that can give me 'r' access without first closing the file | 19:14.37 |
| Robin_Watts | how does that help? | 19:15.13 |
| | fclone isn't an ANSI C function. | 19:15.46 |
| ray_laptop | fclone is a function name I made up -- as one of the clist_io_procs | 19:16.58 |
| Robin_Watts | ok, so I understand that by doing some wrapping at the clist level, you can reference count the FILE * or something and so avoid closing it and have it disappear. | 19:18.26 |
| | but I don't understand how this helps with the "too many clist files around to keep handles to them all" problem. | 19:18.45 |
| ray_laptop | I see. so the clist_fopen needs to know whether or not the clist files are to be 'saved' for the 'saved_page' deferred printing. However, I think the 'io_procs->unlink' in gdev_prn_render_pages is wrong -- it should only be a 'io_procs->fclose' | 19:29.57 |
| | what if the same files need to be retained for deferred printing again later -- the 'unlink' should be left to the client, or if we want the clist to be opaque, add some an "gdev_prn_delete_saved_page" accessors for saved pages | 19:32.22 |
| | Robin_Watts: and the "too many handles" problem is bogus. We only open the two files as we render each saved page. Maybe it _used_ to be different | 19:33.17 |
| | since nobody uses the saved page stuff, it wouldn't surprise me if it got fixed in gdev_prn_render_pages and the gp_mswin never got fixed | 19:33.56 |
| sebras | m tor8 ping! | 22:00.05 |
| Ch3rryC0ke2 | Hey there-- has anyone had success compiling the mupdf viewer VS project on Win 8 using VS 2012 RC? | 22:18.12 |
| | When opening the solution, I get a bunch of "This project is incompatible with the current version of Visual Studio" errors. | 22:18.15 |
| | My goal is to get mupdf to compile as a Win8 metro app, and I thought the first place to start would be to look at the existing VS project on Win8. | 22:35.34 |
| ray_work | Ch3rryC0ke: doesn't VS 2012 ask you if you want to convert the project/solution (that | 22:49.46 |
| | that's what it does (at least with the GS solution) | 22:50.06 |
| | it would be just like megasoft to not be able to import older solutions :-/ | 22:51.04 |
| Ch3rryC0ke2 | Yeah I tried that and it still didn't work. I just downloaded VS 2010 and that converted the project and it's building, so now I will try opening the "converted by VS 2010" project in VS 2012 | 22:51.04 |
| ray_work | microsoft is such an oxymoron | 22:51.37 |
| Ch3rryC0ke2 | Haha yeah | 22:51.51 |
| | Do you think you can help me come up with a general approach for getting this stuff to work in a metro app? I've been looking at the iOS xcode project since I'm more familiar with iOS/ OS X development. | 22:52.28 |
| | OK, so the project partially loads in VS 2012 RC. I get a bunch of these errors: "Project 'mupdf' could not be loaded because this version of Visual Studio only supports Metro style app, native desktop library (dll), and static library (lib) projects. | 22:56.27 |
| | C:\Users\Suneet\Documents\mupdf-1.0-source\win32\mupdf.vcxproj : error : Exception from HRESULT: 0x80042003" | 22:56.28 |
| | Guessing its because its VS Express | 22:57.03 |
| | OK-- started again with VS 2012 professional, and the project converts successfully and builds. | 23:25.21 |
| | Forward 1 day (to 2012/06/20)>>> | |
IRC Logs