| | <<<Back 1 day (to 2015/02/09) | 20150210 |
| Robin_Watts | mvrhel_laptop: In the installer commit, you have TARGET = gsview_setup. | 00:39.56 |
| | Could we have TARGET = gsview_setup_$VERSION or something? | 00:40.19 |
| | For the next commit, I think we handle .tif too. | 00:41.04 |
| | otherwise all looks good. | 00:43.04 |
| Travisty | Hi, I have many pdfs with annotations in them, and Iâd like to embed the annotations so that they appear properly in all pdf viewers (right now Preview on MacOS seems to display the annotations differently and it makes them very hard to read) | 01:18.26 |
| | Is there a simple way to do this with ghostscript? | 01:18.33 |
| | I was thinking I could use ghostscript to conver them all to pdf/a format | 01:19.06 |
| Alibaba | hi everybody, i a newbie to development PDFReader in iOS, how to integrate MUPDF with exits iOS project, plz help me | 07:51.29 |
| kens | You are going to have to be more specific in your questions. 'How to integrate' is too broad. Start by looking at the existing iOS demonstration application. | 07:52.19 |
| Alibaba | i clone the newest source in the git, i was make generate susccessfully | 07:53.49 |
| | the existing IOS, i was run it, but i don't know use it in other project | 07:55.30 |
| kens | You use the MuPDF library, you call appropriate entry points in the library as required. The demonstration application does this, when you want to know how to do a specific operation, if the demo app does it, look to see how it does it. | 07:56.44 |
| Alibaba | tsk you. i will try it | 07:57.39 |
| agile_ | hello guys! | 09:39.38 |
| Alibaba | make: *** No rule to make target `generated/gen_font_droid.h', needed by `build/debug-ios-i386/pdf/pdf-fontfile.o'. Stop. make: *** Waiting for unfinished jobs.... Help me! Plz | 09:50.24 |
| agile_ | Robin_Watts: you there bud? | 09:51.06 |
| kens | Looks like you haven't created the 'generatged' files, which suggests you haven't followed all the build steps. | 09:51.13 |
| tor8 | Alibaba: there's a fix for that issue on my private branch (which hasn't been merged to master yet) | 09:51.31 |
| kens | ought to learn how to build MuPDF for Android and such. | 09:52.11 |
| Alibaba | nơw what do ido | 09:52.30 |
| tor8 | kens: in general what you said is true, but I realized I'd messed up and left some remnants of the old fonts when upgrading the URW fonts | 09:52.40 |
| agile_ | anybody know why saving the annotations using mupdf takes some time on android devices? | 09:52.43 |
| Alibaba | i do step by step | 09:52.45 |
| tor8 | Alibaba: edit Makefile and change the line FONT_GEN := to remove $(GEN)/gen_font_droid.h | 09:53.26 |
| agile_ | you know when it ask document changes. Save them? hit yes and it will take a while to save | 09:53.30 |
| Alibaba | ok tor8, i will try this | 09:53.41 |
| tor8 | kens: http://git.ghostscript.com/?p=user/tor/mupdf.git;a=commitdiff;h=e45282fd963026b6f1a93325466a696927ad09fc;hp=aba57a20134e06fe8c81836ccdf9b94ad1ce2d71 could nod to this commit? | 09:54.39 |
| kens | LOL tor8 you must be desperate. | 09:54.55 |
| | Looks fine to me, though I cannot claim to understand it exactly | 09:55.07 |
| tor8 | I feel bad about having made the commit but then promptly forgetting to get it pushed :/ | 09:55.18 |
| | it just removes the (latin) droid font from the builtin fonts | 09:55.31 |
| kens | Yeah, that's about all I can tell :-) | 09:55.40 |
| tor8 | now that the URW fonts have greek/cyrillic support, we don't need it anymore | 09:55.44 |
| kens | OK makes sense | 09:55.50 |
| Alibaba | ok, it build successfully | 09:55.55 |
| tor8 | kens: thanks. | 09:56.18 |
| kens | YW | 09:56.23 |
| tor8 | Alibaba: or you can pull the latest commit, which now has the fix | 09:56.29 |
| kens | back to gstates..... | 09:56.31 |
| tor8 | Alibaba: ah, great! | 09:56.42 |
| Alibaba | when i edit the pdf file, it don't save position and remove page of file | 09:57.02 |
| | i using simulator of xcode | 09:57.30 |
| | when i save change the document it process very low | 09:58.33 |
| | ah, last commit fixed it | 09:59.02 |
| | sory :D | 09:59.11 |
| agile_ | Alibaba: im having the same issue on android devices | 09:59.18 |
| Alibaba | agile you try update the last commit, it will fix it | 09:59.50 |
| agile_ | Alibaba: since when | 10:01.11 |
| | Alibaba: i done a build yesturday afternoon | 10:01.27 |
| | uk time | 10:01.34 |
| Alibaba | agile_ when i get last commit of git since few minute ago. Hanoi Time | 10:02.39 |
| agile_ | Alibaba: ok ill give that a go | 10:03.28 |
| Alibaba | good luck :d | 10:03.53 |
| chrisl | Bear in mind that "saving" a PDF file is quite a complex operation, so it may well take a moment or two - also dependent on the original PDF | 10:03.57 |
| agile_ | chrisl: but on a nexus 9 it is like a second to save | 10:04.37 |
| Alibaba | +chirsl in Android save change slow at fisrt time use, after that it very fast | 10:04.55 |
| | in iOS the first or second or thrid is the same | 10:05.29 |
| Robin_Watts | Alibaba: It may well depend on how much of the file it is writing. | 10:05.47 |
| | If a file is repaired on loading (which will happen silently if it's broken), then the save will have to write the whole file back. | 10:06.25 |
| | Subsequent saves can just save the changes. | 10:06.33 |
| tor8 | Robin_Watts: might make sense to look at profiling the saving operations | 10:07.11 |
| Robin_Watts | tor8: maybe. | 10:07.21 |
| tor8 | it could be that the buffering is inoptimal so we get hit with a lot of syscall overhead? | 10:07.26 |
| | I'm woefully ignorant of the pdf saving code though | 10:08.02 |
| Alibaba | Robin_Watts: the pdf file was change and it lost page which changed | 10:08.02 |
| kens | chrisl naively adding a finalise routine to gstates causes GS to seg fault, because of the way that gs_grestore_only works. It frees tghe content of the current gstate, copies the content of the saved (ie next one up in gstate stack) gstate to the current, then frees the saved gstate. | 10:11.45 |
| agile_ | Another thing please.... | 10:12.13 |
| kens | I'm going to have to think for a minute or two here..... | 10:12.32 |
| chrisl | kens: nulling out the entries in the outgoing gstate would probably work | 10:13.14 |
| kens | chrisl yeah, I considered that. | 10:13.26 |
| agile_ | when i save annotation, open the pdf again, the top bar appears right. when you swipe to the next page and come back again, I can seem to get hold of the top bar again | 10:13.42 |
| kens | Kind of icky, but.... | 10:13.42 |
| | I wonder if I can use the gsave level | 10:14.01 |
| | Hmm, no. | 10:14.25 |
| Robin_Watts | agile_: Tap the middle of the page. | 10:15.11 |
| kens | OK I@ll set device to NULL and use that to determine if I need to free teh contents | 10:15.17 |
| agile_ | Robin_Watts: yes when i tap, it doesn't appear | 10:16.01 |
| chrisl | kens: I wouldn't have thought there were that many entries in the gstate that would need fettled - most will be handled by gc | 10:16.09 |
| kens | 6, plus hte two colour spaces | 10:16.36 |
| | So 8 overall. | 10:16.43 |
| agile_ | Robin_Watts: its kind of intermittent | 10:17.24 |
| kens | OK well I'm getting further now at least :-) | 10:17.39 |
| chrisl | kens: really, 8? I only see the colour spaces..... | 10:18.15 |
| kens | I'm guided by what gs_free_contents does. | 10:18.45 |
| | It decrements the rf counts on pgs->device, pgs->clip_stack, frees pgs->client_data, and calls gstate_free_parts | 10:19.22 |
| | Plus hte 2 colour spaces | 10:19.30 |
| | gstate_free_parts frees up to 7 objects | 10:20.01 |
| chrisl | Hmm, okay, I see - I'd have thought most of that was gc'ed memory..... | 10:20.04 |
| kens | Well, it probably is, but so is a colour space | 10:20.22 |
| chrisl | Yeh, good point.... | 10:20.37 |
| kens | I'll do a run now and see how big the VM usage gets | 10:20.55 |
| | Well its better, peaks at 372 Mb instead of 800 | 10:22.12 |
| chrisl | kens: which file are you testing? | 10:22.32 |
| kens | Oh and seg faults on exit | 10:22.36 |
| agile_ | this is the latest version right -> git clone --recursive git://git.ghostscript.com/mupdf.git | 10:22.38 |
| | with the save document fix? | 10:22.54 |
| kens | AIX361DC_Save.pdf | 10:22.55 |
| Robin_Watts | agile_: That is the latest version. I was not aware of any save document fix. | 10:23.34 |
| agile_ | Robin_Watts: Alibaba said he committed the fix | 10:24.10 |
| tor8 | Robin_Watts: agile_ might be confused with the Makefile fix I pushed that Alibaba ran into | 10:24.14 |
| Robin_Watts | Alibaba has not committed anything. As tor8 says, I think there is a misunderstanding here. | 10:24.38 |
| agile_ | oh right | 10:24.48 |
| | so the saving of annotation taking time not fixed? | 10:25.00 |
| Robin_Watts | no. | 10:25.05 |
| agile_ | Robin_Watts: do you know when it will be fixed | 10:25.41 |
| Robin_Watts | agile_: Not a clue. I don't plan to look at it any time soon. | 10:26.48 |
| | I have other more pressing things to do. | 10:26.54 |
| chrisl | kens: I can't find that test job...... | 10:26.59 |
| kens | Err 1 second | 10:27.07 |
| Robin_Watts | I am not convinced here is actually a problem. | 10:27.08 |
| kens | chrisl tests_private/pdf/PDF_1.7_ATS/AIX361DC_Save.pdf | 10:27.28 |
| Robin_Watts | If you want it looked at, then go to bugs.ghostscript.com, open a bug, and attach a test file so I can see the problem happen. | 10:27.34 |
| chrisl | 1 second to rewrite a PDF doesn't, necessarily, sound bad, to me..... given what might be involved | 10:28.05 |
| Robin_Watts | chrisl: indeed. | 10:28.40 |
| | In the absence of a bug report, it will never get looked at. | 10:28.50 |
| agile_ | chrisl: the 1 second i tested is on the nexus 9 which is pretty high spec. every other device i tested samsung, nexus phone it take around 10 seconds | 10:29.35 |
| chrisl | agile_: again, it's not hard to conceive of a PDF that would take that long, or longer.... | 10:30.13 |
| agile_ | Robin_Watts: chrisl ok what about a spinning progress bar instead of the ok button hanging | 10:31.06 |
| Robin_Watts | agile_: Sure. Open a bug. Attach a patch to do that, and we'll consider it. | 10:31.42 |
| agile_ | Robin_Watts: cool. also the saving the annotation process time depends of the PDF? | 10:34.57 |
| | on* | 10:35.15 |
| Alibaba | good bye! | 10:35.53 |
| Robin_Watts | agile_: Obviously, yes :) | 10:38.21 |
| | A big complex PDF will take longer to save than a small simple one. | 10:38.38 |
| agile_ | cool | 10:39.03 |
| chrisl | And the "complexity" means the internal construction, not necessarily what gets displayed. | 10:39.57 |
| agile_ | sure | 10:40.09 |
| chrisl | kens: an older version of a certain other PDF interpreter we know takes ~520Mb to rip that file (with a 16Mb band buffer), just for reference | 10:40.56 |
| | That's @ 300 dpi | 10:41.06 |
| kens | Intriguing | 10:41.07 |
| | Ray was running this at 600 dpi, I was just using pdfwrite just now which isn't a fair test | 10:41.33 |
| | But did throw up another seg afult I need to address. Let me try the rendering test | 10:41.48 |
| | Rendering peaks at 500 Mb, but there is a problem in that it enters an endless loop | 10:43.53 |
| | Oh not endless, just very long | 10:44.08 |
| chrisl | It takes some time to render..... | 10:44.13 |
| kens | It still has quite a lof of profiles open > 250 | 10:44.31 |
| | WHIch is better, but not great | 10:44.36 |
| | Let me just try 8.71 | 10:44.44 |
| chrisl | Well, that other rip took 1m35s to render at 300dpi, 16Mb band buffer. | 10:45.30 |
| kens | 8.71 peaks at just over 30 Mb | 10:45.52 |
| chrisl | poppler tops out at about ~400Mb, but fluctuates a lot | 10:46.26 |
| kens | GS 8.71 completes at 600 dpi in less than 1 minute | 10:46.42 |
| | Oh but it doesn't render it properly | 10:46.57 |
| chrisl | Just going to suggest that | 10:47.07 |
| kens | It doesn't fit the content to the page | 10:47.09 |
| | Its too old to understand a bunch of the settings | 10:47.25 |
| chrisl | In any case, even if 8.71 doesn't it *visibly* wrong, it may well still be missing a bunch of required stuff to make transparency work properly | 10:48.33 |
| kens | Yes, but the minimal amount of memory its using makes me think that most of it is going on the ICC profiles | 10:49.00 |
| chrisl | It does still sound like an awful lot of profiles - are they maybe stacking up in the link cache? | 10:50.08 |
| kens | IDK I suspect there's still some gstates not being freed until the GC kicks in | 10:50.36 |
| | full page @600 dpi takes 1 min 6 secs and peaks at 35Mb of memory with 8.71 | 10:51.15 |
| chrisl | Well, if we're saving gstates in the Postscript world, we *can't* free them until gc happens | 10:51.38 |
| kens | That's what I've been saying all along :-) | 10:51.54 |
| Robin_Watts | chrisl: That's not strictly true. | 10:52.23 |
| kens | THe problem arises because the ICC profiles aren't GC-visible, so it doesn't realise it should run | 10:52.25 |
| | I cna improve the situation by forcing a vmreclaim after every form is run | 10:52.47 |
| chrisl | Robin_Watts: Oh, how do we free them, then? | 10:52.53 |
| Robin_Watts | If you have correctly working ref counts, then when a ref count goes to zero, you can free something. This can happen at times when gc hasn't been run. | 10:53.29 |
| kens | There may very well beother conditions that cause us to use .pdfruncontext, which is where the problem arises. Ort even call /qstate which is what creates the orphan gstates | 10:53.29 |
| | Robin_Watts : we don't ref count many PostScript objects | 10:53.55 |
| | We rely on the garbage collector | 10:54.09 |
| Robin_Watts | If there are still references being held by stuff that needs to be gc'd, then you're in the same boat as before of course. | 10:54.11 |
| chrisl | gstates aren't reference counted | 10:54.20 |
| Robin_Watts | fair enough. I'll shut up then. | 10:54.51 |
| kens | in the rip chrisl mentioned earlier, everything was reference counteed, in GS this isn't the case. | 10:55.31 |
| chrisl | gstates contain other stuff that *is* reference counted, but the gstates (and imager states) themselves aren't | 10:55.42 |
| kens | If the garbage collector was counting the allocated ICC profiles into its memory usage it would run more frequently (I believe). | 10:56.23 |
| chrisl | Making ICC profiles gc'ed sounds like a rather major undertaking..... | 10:57.44 |
| kens | Its not somethign I'm planning, no :-) | 10:57.55 |
| chrisl | As this is likely to arise more often, as we allocate more stuff outside of gc memory, perhaps we do need some method of triggering gc based on overall memory use | 10:59.48 |
| kens | Well I can probably work around this one by forcing a vmreclaim after a .pdfruncontext, but its a but brute force. | 11:01.00 |
| chrisl | We *could* check the heap allocator and trigger on that | 11:02.19 |
| kens | will bow out of the memory discussion at that point :-) | 11:02.35 |
| | OK with a finalise in place we peak at ~300 profiles allocated. WHich is around 150-200Mb. So there's still about 150+ Mb unaccounted for I htink. Also this shows that the finalise routine is unusifficient, we genuinely don't know that the graphics states are nmot referenced until we run the garbage collector. So we need to make GC happen more frequently which either means the ICC profiles need to be GC'ed, or I need to force th | 11:09.31 |
| | e GC to run more often in conditions where this can happen (ie forms and maybe other things) | 11:09.31 |
| Robin_Watts | mupdf has a scheme for a situation like this. | 11:10.56 |
| | Whenever we call a malloc, if the malloc would fail, we evict from our cache, and then retry. | 11:11.20 |
| kens | THe mallocs don't fail | 11:11.34 |
| | It just uses up 800+ Mb of memory | 11:11.41 |
| Robin_Watts | evicting from our cache could be seen as a gc. | 11:11.43 |
| | kens: Right, but the fundamental idea of doing 'something' on the mallocs still holds. | 11:12.20 |
| kens | Well I think tha'ts what chrisl was suggesting above | 11:12.34 |
| Robin_Watts | (when the memory passes the next 10Meg mark or something) | 11:12.37 |
| kens | Tyring to do what chris was suggesting is beyond me, so I'll see if I can 'fix' it in the PostScript world. | 11:14.08 |
| | Looks like we use .pdfruncontext for form and pattern PaintProcs and for type 3 BuildChar. Each time we do so we have the possibility of ending up with orphaned gstates. This is OK, unless the colour space a the time is an ICCBased space. | 11:23.38 |
| | I don't fancy triggering a vmreclaim on type 3 BuildChar, but forms are sensible | 11:24.13 |
| | Patterns might be the other place where this is likely to help, that file does also contain a lot of shadings | 11:24.31 |
| chrisl | kens: what's the command line you're using to test this? | 11:26.33 |
| kens | Its in bug #695813: | 11:27.02 |
| | -sDEVICE=ppmraw -o /dev/null -sBandListStorage=memory -Z: -q -r600 -g5100x6600 \ -dFitPage -dMaxPatternBitmap=8m | 11:27.02 |
| | I dropped the -Z: it wasn't helping me | 11:27.15 |
| chrisl | I hack up of what I suggested above keeps memory use to a little under 360Mb, and completes in 1m25s | 11:30.10 |
| kens | looks like all the offenders in the first file are due to .execgroup | 11:30.18 |
| | chrisl that's an improvement, I think I can get it lower by sitcking a .vmreclaim in, give me a second | 11:30.37 |
| | I'll want the timings for mine too | 11:30.45 |
| chrisl | using -sBandListStorage=file memory tops at 190Mb and completes in 46s | 11:32.06 |
| kens | Peaks for me at 242 Mb and takes 1m55 | 11:33.41 |
| | need timing without vmreclaim, one sec | 11:33.56 |
| agile_ | see you guys later. appreciate all your help | 11:35.52 |
| chrisl | My idea is less than ideal because things like the clist size will influence triggering of garbage collection...... | 11:36.29 |
| kens | Peaks at 980Mb in 1 minute 1 second | 11:36.44 |
| | chrisl your solution is potentially better than my hack though. More GC resutls in lower pre3formance. Yours seems like a better compromise | 11:37.19 |
| | There's no getting away from teh performance/GC tradeoff though | 11:38.20 |
| chrisl | Where are you calling vmreclaim? | 11:38.34 |
| kens | In .execgroup, after 'pdfopdict .pdfruncontext', pdf_draw.ps line 619 | 11:39.05 |
| | Err actually line 623 | 11:39.26 |
| | You need to call 2 .vmreclaim, regular vmreclaim is hacked to do nothing | 11:39.54 |
| chrisl | We don't do any save/restores around there..... | 11:41.04 |
| kens | No, that's not the problem, its the gstate that causes the problem, and that gets called form .pdfruncontext | 11:41.27 |
| chrisl | No, I was wondering if we might reduce the impact of the garbage collection by putting it inside a save/restore, but as there isn't a save/restore pair, we can't do that | 11:42.18 |
| kens | Or are you thinking that a save/restore would fix it / oh what you said | 11:42.24 |
| chrisl | My feeling is that my solution is going to unacceptably affect performance across the board | 11:44.01 |
| kens | Hmm, mine is harsher, but only affects forms with transparency | 11:44.17 |
| | But hits performance harder | 11:44.25 |
| | FOr pathological cases like this one | 11:44.32 |
| chrisl | Uh-oh, I think at least some of this might be work I did...... | 11:45.55 |
| kens | is surprised... | 11:46.08 |
| | But to be honest, why would you expect that a gsate would lead to this kind of problem ? | 11:46.34 |
| chrisl | Having the gstate correct at the time we actually run a softmask group was a fix I did | 11:47.11 |
| kens | Fair enough, but if it was me I wouldn't expect that to cause a memory explosion is all I'msaying | 11:47.40 |
| chrisl | No, I wouldn't expect that | 11:48.22 |
| | kens: so I guess this file must contain a lot of softmask groups? | 11:50.23 |
| kens | thousands | 11:50.32 |
| | 1400+ forms, all with transparency groups, dome of them nested | 11:50.50 |
| chrisl | But I'd expect the gstate problem to only occur with softmask groups, not just ordinary groups | 11:51.51 |
| kens | Possibly, I don;t know. I can say that we call qstate, which calls gstate, and that the gstate is the owner of the Profiles, and does not get freed until we do a GC | 11:52.54 |
| | I'm not certain what conditions exactly prompt the execgroup | 11:53.19 |
| | If I put a save/restore around .excgroup, and .vmreclaim in it, the performance improves a bit, up to 1m45 or so | 11:54.07 |
| chrisl | So the gssmask procedure isn't the only place we save the gstate...... | 11:54.17 |
| kens | No, there's 3 places where we call .pdfruncontext, and that calls qstate | 11:54.40 |
| | TYpe 3 BuildChar, Pattern PaintProc and execform | 11:55.02 |
| | sorry .execgroup that shold have been | 11:55.53 |
| chrisl | Well, it doesn't seem to be the root of the problem, but I really think the lazy evaluation of smask groups is far more trouble that it's worth...... | 11:56.04 |
| kens | I haven't looked at it at all. | 11:56.31 |
| | Its the .pdfruncontext that causes the explsoion though | 11:56.52 |
| | Mainly because there are thousands fo forms, which is just silly | 11:57.03 |
| | Note that DoForm does not call .pdfruncontext, so this only affects forms with transparency of some kind | 11:58.01 |
| chrisl | .execmaskgroup eventually gets to .pdfruncontext, but as I said, the stuff I added may contribute to, but doesn't seem to be the core of the problem | 11:58.34 |
| kens | ah, I think its execmaskgroup you are thinking of, .execgroup seems to be called for any form with a Group | 11:58.41 |
| chrisl | .execmaskgroup calls .execgroup | 11:59.11 |
| kens | Yes, but its not in itself the problem | 11:59.21 |
| | It may not help, but its not a major contributor | 11:59.36 |
| chrisl | Yeh, I just noticed it, and remembered I'd hacked around in there | 11:59.52 |
| kens | The fact that all the forms go through .execgroup is the main problem | 11:59.52 |
| | OK I'm going to try a cluster push with my changes and see what happens. Since its essentially a stupid case I don't mind if it goes slow, we have to trade off the performance to get the memory usage down. | 12:00.52 |
| | I don't think this will be a major issue with any sensible files.... | 12:01.37 |
| | Hmm, except that its an Illustrator CS3 file :-( | 12:02.26 |
| | FWIW hte 8.71 output of that file is definitely incorrect, teh coke can is the wrong colour and the background is visible through it. So its memory usage isn't a reliable guide anyway | 12:06.14 |
| chrisl | kens: my fix *kills* performance on the cluster...... | 12:08.32 |
| kens | Oh that's not good, let me see how mine performs | 12:08.45 |
| | Do we get a performance report back ? | 12:09.02 |
| chrisl | Yes | 12:09.12 |
| kens | OK then I'll make a point of looking at it. | 12:09.20 |
| | Its at 14% right now, should be done soon | 12:09.33 |
| chrisl | The timings aren't exactly reliable, though. But previous = 37:47:49 and current = 57:40:23, plus a huge pile of files timing out is pretty incontrovertible | 12:10.53 |
| kens | Yeah that's quite massive sadly | 12:11.05 |
| | I'm sure mine will have some impact, I hope it won't be quite *that* bad | 12:11.29 |
| | Wow, that produced a number of seg fault, that's surprising | 12:17.12 |
| | Timing went from 38:29:03 to 39:00:11 | 12:17.40 |
| chrisl | I would say that was below the noise level..... | 12:18.21 |
| kens | Yes, probably, but the seg faults are a worry. I'll look into that after lunch, if I can resolve that this may be the best we can come up with | 12:18.46 |
| | Well, putting a save/restore around the .execgroup does indeed cause a GP fault in transparency processing..... | 13:29.17 |
| | chrisl I think I give up..... Trying to wedge a save/restore pair into that form processing always ends me up with an 'invlidrestore' error. I'm not sure why and I've lost any further interest in trying to figure it out. If I just put the .vmreclaim into .execgroup then overall the cluster time goes from 38:37:36 to 39:42:48 | 14:07.49 |
| | I'm going to write this up in the bug report, I think I've gone as far as I can with this | 14:08.12 |
| chrisl | kens: I think that may be best - the save/restore thing I'd *guess* is that the transparency code is relying on some Postscript VM objects..... | 14:08.46 |
| kens | Yeah I think it must be something like that, something has changed and the restore can't deal with it. I've had enough of trying to chase it | 14:09.16 |
| chrisl | I tweaked the values in my fix idea, and got some interesting results..... | 14:10.29 |
| | The test job ran in a whisker under 400Mb and took 38.4s.... but the cluster test was slightly slower: 41:06:19 vs 38:35:18 | 14:11.32 |
| kens | Hmm, so your fix is more general than mine, but uses more memory and goes slower | 14:11.55 |
| | Mine peaks at slightly over 240Mb | 14:12.16 |
| chrisl | Yeh, mine is better for that problem file, but poorer in general | 14:12.55 |
| kens | Its quite a lot faster ofr that file though | 14:13.19 |
| | I'm at 1:53 for that file. | 14:13.29 |
| | I guess *because* it has so many forms its doing a vast amount of GC | 14:13.57 |
| chrisl | My inclination would be it's not worth penalising "normal" files to cater for an extreme case like that | 14:14.13 |
| kens | I'm inclined to agree, the file is mad | 14:14.29 |
| | If we have to deal with it, I think I'd prefer my solution as it is quicker ocerall (less penatly on sensible files) and does use less memory, at the expense of the stupid file being slow. | 14:15.34 |
| chrisl | Another solution would be have the ICC related memory come from a dedicated chunk allocator, so we can specifically track that, rather than *all* allocations, as my hack-up does now. More invasive than what I have, but less than making ICC memory gc'ed. | 14:18.07 |
| kens | Hmm, that makes some sense. Let me finish up writing the explanationin the bug threa, and what I;'ve tried, then you should add your own observations and theories. | 14:19.08 |
| agile_ | hey buddies im back | 14:23.17 |
| | quick question | 14:23.21 |
| | is there any ability to add notes to the pdf? | 14:23.55 |
| kens | You mean a note annotation ? | 14:24.10 |
| agile_ | yeah | 14:24.14 |
| kens | There's some annotation support but I don't know exactly what | 14:24.39 |
| | The developer that wrote it is out today | 14:24.40 |
| agile_ | there drawing, text highlight, underline but just wondering if there is notes | 14:25.05 |
| kens | No idea, sorry | 14:25.23 |
| agile_ | cool, no worries | 14:25.36 |
| kens | If you can't see it there, I'd guess it isn't implemented | 14:25.51 |
| agile_ | what other stuff you guys developed? | 14:26.06 |
| kens | We have currently 3 main products | 14:26.19 |
| | Ghostscript, MuPDF and Smart Office | 14:26.29 |
| agile_ | cool | 14:26.49 |
| kens | MuPDF also uses MuJS | 14:26.54 |
| | I think that might be described as a product :-) | 14:27.11 |
| tor8 | kens: then you might as well mention jbig2dec as well ;) | 14:27.36 |
| kens | Oops oh yes | 14:27.44 |
| | And even tone screens ? | 14:27.52 |
| tor8 | and well tempered ones too | 14:28.17 |
| kens | had never counted them up before | 14:28.34 |
| agile_ | can make alot of money with mupdf alone intregrated into app | 14:28.35 |
| tor8 | or did we scrap those? | 14:28.36 |
| kens | not sure tor8 | 14:28.42 |
| | I thought we discovered the patent had expired | 14:28.50 |
| | After that, I don't know | 14:28.59 |
| | chrisl I've put everything I found out in the bgu report, could you put your ideas in there too please ? THen we can decide at this afternoon's meeting what (if anything) to do about it. | 14:44.06 |
| chrisl_r500 | kens: I'll do it in a mo..... | 14:44.29 |
| kens | No rush, 45 minutes till the meeting :-) | 14:44.43 |
| rayjj | good morning | 15:00.44 |
| kens | morning ray | 15:01.13 |
| rayjj | well, the pattern-clist transparency code will need a change to the flow. I've been (with pain) through the numbers and mad sure there isn't just a typo or something (like an extra rounding step) | 15:03.15 |
| chrisl | kens: I just realised, I don't have your gstate finalize fix, so my memory numbers aren't really valid...... | 15:03.51 |
| kens | chrisl I don't have a gstate finalise fix | 15:04.11 |
| | Just a .vmreclaim | 15:04.18 |
| rayjj | I think the problem comes from trying to play back a pattern-clist that has transparency into a non-isolated group | 15:04.24 |
| chrisl | kens: then the colour space reference counts will still be wrong, won't they? | 15:05.32 |
| kens | No, they work out fine | 15:05.46 |
| | Because when we free the gstate, its no longer pointing at the colour spacfe, and the colour space gets freed, even though its refcount is 1 | 15:06.21 |
| rayjj | kens: chrisl: it occurred to me that we could have an 'adjust allocated +/-' that we used when allocating/freeing. When the ICC code allocates a profile, it could tell the 'base' allocator to adjust. This would be ignored (no-op) by other than the GC allocator | 15:06.36 |
| | this would be a new gs_memory_t method (proc) | 15:07.16 |
| chrisl | rayjj: I think the problem is that the ICC code bypasses the gc allocator altogether | 15:07.27 |
| kens | Indeed. | 15:07.34 |
| | The profile is tracked by the colour space, when the colour space is freed, its finalise decrements teh ref count of the profile | 15:08.19 |
| rayjj | chrisl: what I'm saying is we could inform the GC that there is more allocated than it thought (because there is a foreign allocation we want it to take into account) | 15:08.31 |
| kens | And when the gstate is freed, there is no longer anything pointing to the colour space, so it gets freed (even though its ref count is 1), and that way the profile is freed. SO the gabrabge collection works fine | 15:09.02 |
| | OK so you are saying 'this GC object is bigger than you think it is' | 15:09.42 |
| rayjj | kens: BTW, have you checked the icc link cache ? It has ICC_CACHE_MAXLINKS 50 which may be too high | 15:09.48 |
| | kens: yes, to the 'inform the GC' | 15:10.07 |
| kens | rayjj I don't know that its anything to do with teh icc_link_cache, and I don't think its relevant. The prfile is allocated, stored, counted and trqacked as part of a colour space. | 15:10.28 |
| chrisl | rayjj: we could allocate profiles in gc memory and explicitly register_root/unregister_root for each allocation...... | 15:10.30 |
| kens | Its probably best to write these up and put them in the bug report. | 15:10.53 |
| rayjj | chrisl: if the profiles are in GC memory, pointed to from a colorspace, why do we need to worry about the root? | 15:11.26 |
| kens | THe profiles are *NOT* in GC memory | 15:11.40 |
| rayjj | have to do a run to the school. | 15:11.54 |
| kens | The colour space, however, is | 15:11.58 |
| chrisl | rayjj: the problem is, if we want to add them to gc memory in the conventional way, we'd have to find *every* reference to them, and enum/reloc etc for all those cases. | 15:12.35 |
| rayjj | chrisl: agreed | 15:14.14 |
| henrys | chrisl: ray and michael designed the icc stuff not to use gc I thought. | 15:14.32 |
| chrisl | henrys: and that is exactly the root of this problem | 15:14.56 |
| kens | henrys yes, the ICC profiles do not use GC< andthis is (part of) the problem | 15:15.02 |
| | I have tried to explain the problem in detail in the bug report | 15:15.16 |
| henrys | can we explore color spaces not using gc, instead of the direction I think this is going? | 15:16.37 |
| chrisl | henrys: that'll make the situation *worse* | 15:17.01 |
| kens | I cna't see any way tha'ts going to work | 15:17.09 |
| | THat's like 'lets rewrite the PostScript interpreter to not use GC' | 15:17.24 |
| henrys | I think of color spaces being an object in the graphics library and we've discussed many times trying to get the library using gc less. | 15:18.23 |
| kens | Colour spaces are as PostScript object | 15:18.34 |
| | s/as/a/ | 15:18.44 |
| henrys | kens: not when I build pcl or any of the other languages. | 15:19.11 |
| kens | If the colour spaces are to be non-gc, then graphics states should be as well, since they contain colour spaces | 15:19.16 |
| | henrys in PostScript a colour space is a PostScript object and it can be retrieved with currentcolorspace | 15:19.51 |
| henrys | I know we'd and that postscript object could manage a separate object the graphics library color space. rayjj looked at making these kinds of changes before but found it was too slow. That was because we were malloc'ing everything as I recall. | 15:21.57 |
| kens | I don't really see how this would help | 15:22.31 |
| | THe ICC profile is allocated, and attached to a colour space, which is attached to a graphics state | 15:22.47 |
| | Its not hugely relevant whether any of these is GC'ed or not | 15:22.59 |
| | WHen we do a 'gstate' we copy the PostScript object which poitns to the graophics state. | 15:23.14 |
| | Once we do that, we cannot release any of the objects until we no longer need teh gstae, agreed ? | 15:23.34 |
| henrys | kens: yes and reference counting should work for that. | 15:24.20 |
| kens | henrys no it doesn't | 15:24.34 |
| chrisl | It's relevant from the point of view that the decision whether to gc is partly based on how much memory has been allocated by the gc allocator - we now have large amounts of memory effectively under the control of the garbage collector, but which the garbage collector doesn't know about | 15:24.48 |
| kens | Because hte *PostScript* object which represents the gstate is not reference counted | 15:24.54 |
| chrisl | No gstates are reference counted in any language..... | 15:25.21 |
| kens | We do not know that the PostScript gstate object is no longer used until we run a garbage collection pass, and discover that nothing is pointing at it any more | 15:25.30 |
| henrys | chrisl: sure they are. | 15:25.38 |
| kens | At which time we release the PostScript gstate object, and that triggers a release of all the objects it points at | 15:25.52 |
| | henrys the PostScript gstate object is not reference counted | 15:26.06 |
| | As long as that object exists the reference count of the gstate is, effectiuvely, at least one | 15:26.21 |
| henrys | oh I though you meant the the graphics library gstate. | 15:26.35 |
| kens | Because the PostScript gstate object needs the graphics library gstate# | 15:26.36 |
| | No, I'm referring to the PostScrip object | 15:27.05 |
| | But the PS object needs the library gstate | 15:27.16 |
| | Until we get rid of the PS gstaet object we can't free the graphcis library gstate, so we cna't free the colour space and therfore can't free the profile | 15:27.42 |
| henrys | all I'm saying is the PS object and GLIB object can be separate. One gc'd and the other reference counted. Why is that not possible? | 15:28.08 |
| kens | Its possible, it won'r solve the problem | 15:28.17 |
| | Remember, until we release the PostScript gstate, we cannot release the graphics library gstate | 15:28.54 |
| Robin_Watts | The PS object would have to hold a reference to the glib object. | 15:28.56 |
| henrys | well let's bring rayjj and mvrhel_laptop into this, it's their design ... | 15:29.03 |
| kens | In effect it *does* hold such a reference | 15:29.06 |
| Robin_Watts | Hence until a gc frees the PS object, the glib object could not be freed. | 15:29.20 |
| kens | exaclty Robin_Watts | 15:29.27 |
| Robin_Watts | kens; yes, I'm trying to agree with you. | 15:29.29 |
| mvrhel_laptop | morning | 15:30.20 |
| henrys | right but I don't understand why you can't have a finalize with the ps object that decrements the references of does a grestore or whatever... | 15:30.21 |
| kens | Morning Michael | 15:30.28 |
| Robin_Watts | henrys: We could have that, but how would it help? | 15:30.41 |
| | Nothing would get freed any sooner. | 15:31.03 |
| chrisl | henrys: I'm looking at the graphics library gstate structure, and I'm not seeing any reference count related entries..... | 15:31.04 |
| kens | henrys it doens't matter, we cannot release the gstate until the PostScript object is freed, and we *never* know that until we run a GC pass and discover there are no references ot hte *PstScript* object | 15:31.07 |
| fredross-perry | morning | 15:31.12 |
| kens | henrys PostScript objects are not reference counted. THey are only gc'ed | 15:31.31 |
| | THere are some halfway houses like the gstate which I thoguth was ref counted and garabage collected | 15:32.04 |
| Robin_Watts | (In general, I think it's a good idea to keep as much as possible out of gc'd memory, cos then the gc can run faster, but I don't think it will help in this case) | 15:32.06 |
| kens | THe problem here is that the GC doesn't see how much memory the grapchis library is using for the gstate. | 15:32.31 |
| | And so it runs too infrequently | 15:32.37 |
| henrys | Has fredross-perry been introduced to skype for SOT discussion? | 15:34.16 |
| fredross-perry | no sir | 15:34.41 |
| jogux | fredross-perry: do you have a skype id? (can you tell me it) | 15:34.57 |
| henrys | kens, chrisl : I think mvrhel_laptop, rayjj and you guys should take this on after the meeting. | 15:35.13 |
| fredross-perry | fross-perry.conceptua.math | 15:35.18 |
| kens | It might be better to hammer on this at the face-to-face meeting in March, but we can try and discuss it here | 15:35.49 |
| chrisl | henrys: I think there is a wider issue with this, outside immediate the color space | 15:35.57 |
| kens | To be honest, everything I know is in the bug thread | 15:35.57 |
| | And yes, there is potential for more cases liek this as chris says | 15:36.12 |
| jogux | fredross-perry: henrys: there we go, added now | 15:36.28 |
| | fredross-perry: sorry :-) | 15:36.31 |
| henrys | kens, chrisl: I think there is and I'm worried it will consume the 1/2 of the meeting. | 15:36.35 |
| kens | I suspect it could consume the whole meeting :-) | 15:36.48 |
| chrisl | henrys: I think half would be massively optimistic! | 15:37.00 |
| rayjj | henrys: I agree with kens, that we could have this be an apres' ski discussion (since mvrhel_laptop, kens and I will be there) | 15:37.17 |
| henrys | chrisl: I meant to say 1/2 hour. | 15:37.18 |
| chrisl | henrys: I still think that's extremely optimistic | 15:37.38 |
| rayjj | then we could justify the skiing as a business expense ;-) | 15:37.41 |
| kens | Hmm, not what I usually think of as apres ski | 15:37.44 |
| mvrhel_laptop | oh good point | 15:37.46 |
| kens | Maybe alcohol would help | 15:37.56 |
| rayjj | kens: definitetly ! | 15:38.07 |
| henrys | alrigh to the agenda but I'd really rather you guys got a head start on it but I'll roll with you. | 15:39.00 |
| | FYI it looks like there will be another trip much like mvrhel_laptop last trip if you want details I can discuss it on skype. | 15:39.43 |
| rayjj | henrys: but it isn't ultra urgent (yet) since we don't have customers complaining (although I'm surprised that it hasn't come up with 532 since they frequently run the ATS) | 15:39.57 |
| henrys | I'll be going to it. should be within a month. | 15:40.15 |
| Robin_Watts | henrys: The focus of the trip being gs or SOT or both? | 15:40.32 |
| rayjj | henrys: if we discuss it when we are together before the meeting, the discussion during the full staff meeting won't be as protracted | 15:41.09 |
| henrys | the focus of the trip is a printer rip wtthout a disk. | 15:41.27 |
| rayjj | Robin_Watts: and/or mupdf | 15:41.30 |
| Robin_Watts | henrys: right. | 15:42.04 |
| henrys | we are visiting SOT related folks but it's not the reason we are going. | 15:42.34 |
| rayjj | henrys: do we want to benchmark mupdf (banded mode) with the same files. If the printer has an ASIC for color conversion/halftoning, then mupdf->RGB might be workable | 15:42.38 |
| mvrhel_laptop | right | 15:42.58 |
| rayjj | is going to do the benchmarking anyway, out of curiousity | 15:43.14 |
| tor8 | mvrhel_laptop: fredross-perry: Robin_Watts: I have spent some time doing API cleanups in MuPDF which will affect the viewers. I've updated the Android viewer, but the iOS and gsview and Qt viewers will need work. | 15:43.50 |
| mvrhel_laptop | tor8: ok | 15:44.07 |
| henrys | rayjj, mvrhel_laptop: I think that would be worthwhile. More than that though I'd like to have all this documented. If we get a printer customer tomorrow we should just need to point them at a document. As it is now the responsiveness is too lw. | 15:44.13 |
| | s/lw/low | 15:44.19 |
| fredross-perry | tor8: OK. | 15:44.32 |
| Robin_Watts | Roughly, that statement from tor8 translates as: You see that carpet under your feet? TUG! | 15:44.34 |
| rayjj | henrys: the reason that most multi-function printers have ASIC for color conversion + halftoning is (1) it's a relatively easy thing in silicon nowadays and (2) it's needed for the 'copy' function (scan to print) | 15:44.37 |
| mvrhel_laptop | henrys: we have a few spread sheets that we have handed out in the past | 15:44.40 |
| fredross-perry | :-) | 15:44.44 |
| tor8 | in summary, I've made naming free/drop/close functions and passing the ctx argument more consistent | 15:44.44 |
| fredross-perry | OK, Iâll pull and see whatâs what | 15:45.11 |
| tor8 | fredross-perry: it's on the tor/drop branch | 15:45.26 |
| henrys | mvrhel_laptop: I'm thinking something in gs/doc and the pcl docs. I'm going to do pcl this week. | 15:45.35 |
| fredross-perry | ok | 15:45.35 |
| tor8 | essentially, it means adding ctx to most function calls (no more embedded hidden contexts in other objects) | 15:45.48 |
| | and rename all drop/free/close to just plain drop | 15:45.58 |
| rayjj | fredross-perry: your 'pull' might end up with a lot of stuff falling on you ;-) (like trying to get something from a closet shelf) | 15:46.09 |
| henrys | tor8: I asked you about jeoung XFA last week do you want to look at it? | 15:46.25 |
| tor8 | there are a few more fairly minor changes incoming, but it's going to be a lot of search-and-replace to get things to compile again | 15:46.30 |
| jogux | fredross-perry: any sign of mac installers btw? :) | 15:46.39 |
| tor8 | henrys: I must've missed that question. | 15:46.47 |
| fredross-perry | yes I put something on my public HTML a bit ago. | 15:47.01 |
| | is there a better place for it? | 15:47.19 |
| tor8 | hopefully with this big set, we shouldn't have to change things so much in the future (like renaming free to drop, and adding ctx arguments when they're suddenly needed) | 15:47.29 |
| jogux | fredross-perry: that sounds as good a place as any :-) | 15:47.32 |
| henrys | tor8: well we can get it for free - he owes us ;-) otoh I don't know if it would more work to integrate than do ourselves. | 15:47.43 |
| chrisl | fredross-perry: BTW, I tried the linux installer, and running the app, I got: gsview: 'symbol lookup error: gsview: undefined symbol: _ZN7QString13toUtf8_helperERKS_' | 15:47.59 |
| devdev | hi, i have built the mupdf library and i have imported the generated .a files for ios into my project | 15:48.24 |
| chrisl | I'm guessing I have a dependency problem, so something on requirements would be good | 15:48.26 |
| henrys | chrisl: with the trip obviously language switch does get bumped in priority, which in turn waits for your library changes. | 15:48.44 |
| devdev | however i get the following error duplicate symbol _jround_up in: /libjpeg.a(jutils.o) dupplicate | 15:48.54 |
| fredross-perry | ouch. OK. Iâll look at that. | 15:49.03 |
| tor8 | henrys: so what exactly is the deal? this is jeong from epapyrus? | 15:49.29 |
| henrys | mvrhel_laptop: looks like you have the CM customer under control? | 15:49.38 |
| chrisl | henrys: well, I have pcl and xps building on Windows, but gs is now failing - then I got involved in all the font stuff. I'm back to the build stuff today | 15:49.44 |
| mvrhel_laptop | henrys: other than that he is not using git | 15:49.51 |
| jogux | fredross-perry: I guess Artifex doesn't have a mac dev account? (can't be opened, unidentified developer) | 15:49.56 |
| | fredross-perry: anyway, installs and runs for me, great :-D | 15:50.24 |
| henrys | tor8: right we can have just about anything in epapyrus' works... XFA stuck out to me. | 15:50.25 |
| rayjj | henrys: about the l-s (PS/PDF + PCL) we didn't send them that ROM info. Partly because I wasn't sure if the 'new' approach from chrisl would be different. | 15:50.33 |
| jogux | fredross-perry: doesn't seem to response to trackpad zoom gestures though :( | 15:50.50 |
| rayjj | henrys: should I do the old l-s just in case they ask, or wait for chrisl ? | 15:51.00 |
| fredross-perry | If Artifex has a Mac dev account, someone please let me know. | 15:51.11 |
| chrisl | rayjj: the numbers should be similar - taking account of the fonts being "merged" at some point | 15:51.25 |
| fredross-perry | I have trackpad zoom turned off on my box. Iâll turn it on and test that. | 15:51.36 |
| henrys | rayjj: I think they were happy with your response. | 15:51.44 |
| tor8 | henrys: it can't hurt to take a look, but I have a nasty suspicion that paul should be the one to do it (given that the forms work is his baby) | 15:51.51 |
| rayjj | chrisl: for fonts, I sort of assumed that the 136 was a superset of the PS + PCL fonts. Is that correct ? | 15:52.00 |
| jogux | fredross-perry: I... er... could suggest some changes to some of the icons. | 15:52.11 |
| chrisl | rayjj: according to henrys it is..... | 15:52.25 |
| rayjj | henrys: I haven't seen any response | 15:52.26 |
| henrys | tor8: okay let me talk to paul. I sort of also feel there is too much going on to take on more now. | 15:52.27 |
| jogux | fredross-perry: I don't understand what the 'pencil' icon does. nothing seems different to me whether it's on or off. | 15:53.24 |
| rayjj | henrys: I'll collect the l-s build size, then prepare an update on the ROM size -- after internal review you can let me know if I should send it | 15:53.51 |
| henrys | chrisl: is there a way someone else could start working on language switching with what you have now. maybe rayjj could look at it? Maybe that doesn't make sense, I'd love to be out there with a demoable shared build. | 15:54.34 |
| jogux | fredross-perry: I don't seem to be able to resize the window. I've also managed to make it wider than my screen somehow :-S | 15:54.59 |
| chrisl | henrys: not right now, maybe in a week or two - distractions allowing | 15:55.08 |
| mvrhel_laptop | jogux: is this gsview you are playing with? | 15:55.20 |
| jogux | mvrhel_laptop: yeah, the mac build. | 15:55.27 |
| mvrhel_laptop | ah ok | 15:55.31 |
| fredross-perry | heâs playing with gsview/qt on mac | 15:55.38 |
| mvrhel_laptop | did anyone install the windows version? | 15:55.46 |
| | I stumbled upon a couple print related issues I need to fix | 15:56.01 |
| jogux | fredross-perry: I'm not sure a full screen button in the toolbar is necessary on mac (given it's in the window furniture) | 15:56.40 |
| chrisl | mvrhel_laptop: I installed it, but I only had a very quick mess with it. It still seems slow to start up, to me. | 15:56.40 |
| Robin_Watts | mvrhel_laptop: Sorry, no. | 15:56.49 |
| henrys | chrisl: if you go hide for two weeks and work on it. I'm fine with that. | 15:57.16 |
| fredross-perry | the pencil is for show/hide annotations. | 15:57.37 |
| tor8 | an eye might be a better icon for that | 15:58.05 |
| mvrhel_laptop | chrisl: ok | 15:58.07 |
| chrisl | henrys: I'm hoping to do just that - I'm assuming you won't be dropping any more font-bombs in the next little while! | 15:58.07 |
| fredross-perry | itâs only in recent OSX versions that the Maximize button does full screen. | 15:58.11 |
| jogux | fredross-perry: from finder, open with -> gsview ends up launching preview. :-S | 15:58.19 |
| fredross-perry | the toolbar icons are a lift form the WIndows version | 15:58.41 |
| henrys | chrisl: ignore everybody we need you to do a release in mid/late march. | 15:58.44 |
| tor8 | fredross-perry: isn't there a "hidden" feature where if you hold down option/alt and maximize it does fullscreen in older versions? | 15:58.52 |
| jogux | it did once launch gsview, but that was the one from the disk image. when I unmounted the disk image it went to preview instead. | 15:58.55 |
| tor8 | at least in the Window menu | 15:58.57 |
| henrys | chrisl: you're free until then. | 15:59.03 |
| Robin_Watts | chrisl: The new opentype fonts from urw... I'm assuming they don't make use of the gpos or gsub tables, right? | 15:59.05 |
| mvrhel_laptop | I don't have a pencil icon in the windows one | 15:59.07 |
| fredross-perry | sure. If everyone thinks the full-screen button is overkill we can take it out. | 15:59.32 |
| chrisl | Robin_Watts: Ghostscript certainly won't, no | 15:59.40 |
| jogux | fredross-perry: 'Show annotations' might be a better tooltip | 15:59.42 |
| Robin_Watts | chrisl: Indeed. I don't think freetype handles those yet either, so mupdf is safe too. | 16:00.19 |
| tor8 | Robin_Watts: they have GPOS and GSUB tables for kerning and ligatures | 16:00.30 |
| | but neither gs nor mupdf will make use of them | 16:00.38 |
| chrisl | Robin_Watts: tor8 used bare CFF for mupdf.... unless he's changed them again | 16:00.55 |
| jogux | fredross-perry: I may be dim but I can't figure out how to create annotations | 16:00.58 |
| henrys | 1/2 hour is up, anything else? | 16:01.01 |
| tor8 | chrisl: Robin_Watts: yeah. I pulled out the raw CFF table from the OTFs to embed in mupdf | 16:01.15 |
| fredross-perry | You donât create them in gsview. But you can in Preview, then hode/show them in gsview | 16:01.31 |
| tor8 | we always create our own encodings in mupdf based on the glyph names | 16:01.35 |
| mvrhel_laptop | jogux: we still have to get adding annotations into the project | 16:01.40 |
| | that might explain it | 16:01.47 |
| | ;) | 16:01.49 |
| tor8 | so none of the TTF tables are actually useful for us, and it does save about 20k per font | 16:01.51 |
| jogux | mvrhel_laptop that'd do it :-) | 16:01.59 |
| marcosw | there are some regressions that should be fixed before we release. Should I mark them as blockers? | 16:02.05 |
| henrys | marcosw: yup | 16:02.22 |
| chrisl | then we can unmark them again ;-) | 16:02.43 |
| mvrhel_laptop | hehe | 16:02.49 |
| marcosw | chrisl: actually I don't think any of them are yours... | 16:03.08 |
| kens | I'm having a network problem, so going offline, I'll brb | 16:03.13 |
| chrisl | marcosw: doesn't stop me changing the classification! | 16:03.40 |
| henrys | marcosw: I did have something else on my list, can the bug report have customer numbers for miles' benefit. "open customer issues" etc. | 16:03.43 |
| marcosw | henrys: I thought of that too while I was adding the sot bugs to the report. I don't think it will be difficult. | 16:04.28 |
| chrisl | mvrhel_laptop: so, opening gsview 6 takes about 8 seconds on my Win7 VM, compared with about 1.5 seconds for gsview 4.9 | 16:05.05 |
| henrys | if you want to know about the customer stuff drop by skype | 16:06.19 |
| jogux | fredross-perry: print dialogue is a bit odd. it's not picked my default printer (instead picked first one in list), and it's somehow defaulted to us letter paper. (all printers and doc are A4) | 16:07.00 |
| fredross-perry | thanks | 16:07.26 |
| rayjj | chrisl: 6 seconds ? That's a LONG time (not as bad as Adobe, but...) | 16:07.44 |
| chrisl | rayjj: *8* seconds | 16:07.57 |
| jogux | fredross-perry: print job appears with 'untitled' as name (might have expected document filename) | 16:08.18 |
| fredross-perry | keep it coming | 16:08.32 |
| rayjj | chrisl: On my real win 7 the gsview 6.0.0.0 I have from a while ago opens in a "blink" (much less than .5 seconds) | 16:09.48 |
| jogux | fredross-perry: the open dialogue seems to always default to 'desktop' for no obvious reason. (I think Mac open dialogues tend to remember the last place usually?). | 16:09.55 |
| | fredross-perry: I think that's it for now :-) other things I tried worked fine. I didn't test anything other than PDF. | 16:09.58 |
| rayjj | mvrhel_laptop: where should I get an updated one from ? | 16:09.59 |
| jogux | fredross-perry: printing seems to work fine | 16:10.07 |
| rayjj | fredross-perry: or should that be from you ? | 16:10.12 |
| jogux | fredross-perry: possibly slightly odd that the current page number in the toolbar doesn't update when you scroll the main view (using gestures if that matters) | 16:10.31 |
| fredross-perry | gestures - it might, thanks | 16:10.54 |
| chrisl | rayjj: well, my Win7 VM is running on top of a core2 quad, so even the "real" hardware isn't exactly rocking! But then, a lot of people use the old gsview because they are on older hardware, where Acrobat is horribly slow | 16:11.16 |
| fredross-perry | mvrhel_laptop is WIndows, Mac and Linux is me. | 16:11.24 |
| jogux | fredross-perry: oh. silly odd that the app quits when you close last window. (unexpected to me, but not unprecendented) | 16:11.55 |
| | s/silly/slightly/ | 16:12.03 |
| rayjj | chrisl: I have an old laptop that is a core 2 dual that I use for stuff like that (running win 7 natively) so I'll try it on that (it was too slow for my 9 year old) | 16:12.56 |
| chrisl | Reminds me, I must put the bite on Miles for a new desktop.... :-) | 16:13.41 |
| rayjj | chrisl: I doubt you need to ask Miles (if unsure about the cost, just run it by henrys), but unless you are going "outrageous" I doubt anyone will care. If you want to justify a "screamer" just add it to the cluster :-) | 16:15.56 |
| mvrhel_laptop | ray_laptop: let me make a new installer that fixes a couple minor things | 16:16.36 |
| chrisl | rayjj: it's not so much a screamer, rather the opposite - I want a near silent machine, and that pushes the cost a bit | 16:16.44 |
| mvrhel_laptop | I will let you know when I get it made and up there | 16:16.45 |
| Robin_Watts | mvrhel_laptop: Did you see my comments about your commits last night? | 16:16.51 |
| rayjj | now, if it includes a 4k curved screen 80 inch "monitor", that might be pushing it :-) | 16:16.59 |
| mvrhel_laptop | Robin_Watts: no I will look at the log | 16:17.06 |
| | s | 16:17.08 |
| fredross-perry | stepping awayâ¦. | 16:17.15 |
| Robin_Watts | chrisl: I idly started speccing a machine last week, and was over a grand before I stopped... | 16:17.48 |
| rayjj | chrisl: rillian had a water cooled machine that had the fan outside the window | 16:17.50 |
| mvrhel_laptop | Robin_Watts: ok thanks. | 16:18.05 |
| chrisl | Robin_Watts: I easily got to that much! | 16:18.17 |
| kens | oil cooling is the new balck I understand | 16:18.24 |
| rayjj | chrisl: that and SSD drives means that about the only thing that makes noise is the KB | 16:18.39 |
| chrisl | rayjj: the other thing is a fanless PSU.... | 16:19.10 |
| jogux | I really like the PC I specced from quietpc. It's not the fastest, but it has zero moving parts, cost around a grand I think. | 16:19.12 |
| rayjj | kens: I use the term "water" loosely -- it's usually *not* water (too corrosive) | 16:19.12 |
| jogux | It's also not the slowest, it's what we ran the SOT ATS on for a good few months :-) | 16:19.51 |
| chrisl | jogux: it was quietpc I was looking at | 16:20.36 |
| rayjj | chrisl: If you have a garage, just get a long HDMI cable and use a wireless mouse and KB ;-) | 16:27.04 |
| chrisl | rayjj: that thought has occurred to me! | 16:27.28 |
| jogux | :-) | 16:27.37 |
| tor8 | chrisl: I've put together several silent machines, if you need any tips | 16:31.17 |
| | no need to go for the silenced expensive cases as long as you stick with a fanless PSU and SSD drives only | 16:31.38 |
| chrisl | tor8: I'm getting lazy, I want someone to build it for me ;-) | 16:31.46 |
| tor8 | there are plenty of passively cooled decent graphics cards, and if you settle for an i5 you barely need CPU cooling | 16:32.26 |
| chrisl | tor8: as I don't game, my graphics needs are *well* below active cooling! | 16:33.01 |
| tor8 | my machines have 2 slow-running silent fans each, which can't be heard over the ambient noise of the neighborhood | 16:33.03 |
| | one for the case, one for the CPU. I'm sure I could ditch either one of them and still be okay. :) | 16:33.24 |
| kens | So do we need to discuss this bonkers ref counting/gc/memory issue ? | 16:33.39 |
| henrys | chrisl: it's fine probably overdue ... | 16:34.04 |
| chrisl | henrys: thanks | 16:34.40 |
| | kens, rayjj: How about we add a gs_alloc_non_gc() method that the gc allocator would simply update it's own tracking for gc and pass onto it's "parent"? (We'd need to do something "clever" for freeing, of course). | 16:35.11 |
| kens | I don't think I know enough about the GC to have an opinion. But it sounds reasonable | 16:35.45 |
| rayjj | chrisl: but not all builds include the GC (e.g., PCL) which is why I was suggesting a gs_memory proc (that's no-op for other allocators) | 16:36.26 |
| chrisl | rayjj: yeh, so in a non-gc allocator, gs_alloc_non_gc() would just go straight to it's normal gs_alloc_bytes() method | 16:37.21 |
| kens | If there's no GC then this problem can't arise | 16:37.28 |
| chrisl | rayjj: basically, I'm just suggesting making your idea happen automagically, rather than having to explicitly inform the gc allocator | 16:38.42 |
| rayjj | chrisl: gs_alloc_non_gc would go through a gs_memory->procs then, right | 16:38.46 |
| chrisl | rayjj: yes | 16:38.55 |
| | It would we like _immovable, for example | 16:39.25 |
| rayjj | chrisl: OK. that's sort of what I was suggesting. But we'd need a free_non_gc as well, right ? | 16:40.15 |
| chrisl | rayjj: yes | 16:40.27 |
| rayjj | chrisl: sounds fine to me. | 16:40.55 |
| chrisl | rayjj: maybe I misunderstood what you were saying earlier..... | 16:41.16 |
| rayjj | chrisl: no, your suggestion is slightly cleaner | 16:41.38 |
| chrisl | rayjj: with things like this, I like it to be automatic, if at all possible | 16:42.23 |
| kens | chrisl should I assign the bug to you ? | 16:43.38 |
| rayjj | chrisl: we wouldn't have to touch all of the places that use non_gc_memory -- only those that alloc large things (or lots of things) that a GC'ed element pointed to (privately) | 16:43.50 |
| henrys | but rayjj is going to work on that right? chrisl is booked ;-) | 16:43.59 |
| rayjj | touches nose | 16:44.01 |
| | oops. not fast enough ;-) | 16:44.13 |
| kens | doesn't mind who does it.... | 16:44.29 |
| rayjj | henrys: kens: yeah, that's fine. Assign to me, kens | 16:44.46 |
| chrisl | As long as we don't expect it to be addressed in the next 6 weeks or so, it can come to me. If it needs doing sooner, then no..... | 16:44.48 |
| kens | ok ray | 16:44.53 |
| | I don't believe this can be urgent, its been like thi for ages | 16:45.09 |
| kens | moves on to the next insane bug report | 16:46.09 |
| rayjj | kens: no, not really urgent (as I mentioned earlier) | 16:47.02 |
| chrisl | I've added a very brief summary of the above to the bug - hopefully enough to remind us what we discussed. | 16:49.41 |
| kens | sounds like a good idea.... | 16:49.55 |
| rayjj | chrisl: thanks | 17:06.11 |
| mvrhel_laptop | chrisl: I do agree that the start-up of the new gsview might be a little slower for the large documents compared to russell's . I could probably fix that. Once the doc is loaded though, the new viewer is quite a bit faster | 18:37.36 |
| | people (including myself) do hate to wait for something to load | 18:38.03 |
| | zooming in the old version is horribly slow | 18:38.37 |
| | not to mention there is no way to scroll through the document | 18:40.27 |
| | Robin_Watts: so I do have the output file from the installer including the version number on the commit that is already in golden | 18:53.29 |
| Robin_Watts | mvrhel_laptop: Right. | 18:53.46 |
| mvrhel_laptop | there is OutFile "${TARGET}_${VERSION}.exe" | 18:53.46 |
| Robin_Watts | Ah, ok. | 18:53.52 |
| | Ignore my blithering then. | 18:54.01 |
| mvrhel_laptop | no problem | 18:54.05 |
| | thanks for looking things over | 18:54.10 |
| | Going to push this stuff and then see about recovering from tor8's rug pull | 18:54.44 |
| rayjj | kens: (for the logs) in your write-up (thank you, BTW) you said "At some future point that gstate is evicted from the dictionary" -- could we detect when that happens and have a PS operator that does "<gstate> .freegstate" when that occurs ? | 18:55.16 |
| Robin_Watts | mvrhel_laptop: Tor hasn't pulled the rug yet. | 18:55.29 |
| henrys | first the anthem break in now I'm on the phone with verizon fraud team... | 18:55.29 |
| mvrhel_laptop | oh ok | 18:55.35 |
| Robin_Watts | He's just warning us all that he'll be pulling any day... | 18:55.52 |
| mvrhel_laptop | henrys: great. yes. I am waiting to get a letter from them | 18:56.00 |
| jogux | henrys: :( | 18:56.18 |
| rayjj | will sure be glad when the regression runs settle down (after the font change) | 18:58.45 |
| mvrhel_laptop | henrys: have you watched black mirror yet? | 18:58.55 |
| henrys | mvrhel_laptop: no never heard of it. | 19:00.15 |
| mvrhel_laptop | its on netflix. its sort of like twilight zone | 19:00.30 |
| | but very twisted | 19:00.46 |
| | I have only seen the first episode | 19:00.59 |
| henrys | mvrhel_laptop: I'll check it out, watching the americans now, sort of like it. | 19:01.02 |
| mvrhel_laptop | henrys: these are only 30 minutes. you will be wondering about the brits when you watch it | 19:01.39 |
| | I heard an interview on NPR with the creator of the series a couple weeks ago | 19:03.37 |
| Robin_Watts | Charlie Brooker. | 19:03.49 |
| mvrhel_laptop | black mirror refers to all our display devices of course | 19:04.06 |
| | I have been watching all the old twilight zones with the kids for a while. I was thinking oh this might be good to watch with them. good thing I watched the first one without them | 19:04.53 |
| | need to work on expense report. never did it for the london trip | 19:06.27 |
| rayjj | mvrhel_laptop: BTW, gs _can_ handle JPEG input using lib/viewjpeg.ps and it might be reasonably easy to do a 'viewpng.ps' (I haven't looked into it). I don't know if CBZ is reasonable, but probably not | 19:10.03 |
| mvrhel_laptop | rayjj: oh ok | 19:11.14 |
| henrys | yup somebody broke into my verizon web account and ordered themselves and iphone 6 ... I do worry this is related to the info leak at anthem and more is to come. | 19:13.12 |
| rayjj | henrys: Oh, great. I have Verizon, too. | 19:13.41 |
| | I better check my account (and pay my bill) | 19:13.53 |
| mvrhel_laptop | henrys: ick. yes. | 19:14.03 |
| Robin_Watts | henrys: Someone ordered themselves an iphone 6 on me recently. | 19:14.04 |
| henrys | Robin_Watts: oh well then maybe it isn't anthem related. | 19:14.31 |
| | Robin_Watts: you sure it wasn't Helen ? ;-) | 19:14.48 |
| Robin_Watts | Turns out they had found my address from the registry of companies, and picked a random bank account (not mine). | 19:14.49 |
| | henrys: helen wouldn't be seen dead with anything as old hat as an iphone :) | 19:15.09 |
| | I was amazed just how lax these companies checks are. | 19:15.48 |
| mvrhel_laptop | henrys: someone doing password recovery with address and ssn is a problem for us with the anthem break in. my one bank has a feature that will send you a text with a code if you sign in on a different computer | 19:16.52 |
| | that at least reduces the chances a bit | 19:18.22 |
| | with that account | 19:18.26 |
| rayjj | mvrhel_laptop: most of the places I go (banks, credit card companies, and even verizon) now require me to answer a Security question if I try to log in from a different computer BEFORE they ask for my password | 19:18.44 |
| | Now if my laptop gets highjacked I'm toast (once they break into it) | 19:19.25 |
| mvrhel_laptop | yes. that is true to. one of the reasons that mother's maiden names go for a few bucks on the market (when tied with a SSN) | 19:19.52 |
| rayjj | I have no idea whether "ntpassword" still works on Win 7 -- probably does knowing MS | 19:19.58 |
| | mvrhel_laptop: I never use anything that ancestry.com can give them as a security question | 19:20.37 |
| henrys | mvrhel_laptop: yes I asked them if they had any info about the break in but they did not. | 19:20.50 |
| rayjj | name of first pet, or street I grew up on (that's long enough ago) are probably safe | 19:21.10 |
| mvrhel_laptop | fluffy | 19:21.18 |
| | First street | 19:21.24 |
| rayjj | mvrhel_laptop: won't confirm or deny that | 19:21.37 |
| mvrhel_laptop | hehe | 19:21.43 |
| henrys | rayjj: I had all that in place and somebody ordered the phone, it got picked up by fraud alert only because it went to a different address, so they made it past the questions... | 19:22.43 |
| kens | rayjj I don't see any way to know when a PostScript object is no longer referenced, except by running the 'mark' part of the garbage collector. While I could track the gstate's existence in PostScript, there's no provision for saying 'this PS object is now unreferenced' anywhere in GS that I can tell. That's the way the PostScript interpreter works, it uses GC, not ref counting | 19:23.04 |
| Robin_Watts | henrys: Yeah, the fraud team were absolutely useless for me. | 19:23.24 |
| rayjj | henrys: WOW!! Did verizon say that they had sent out a 'forgot password' or anything ? | 19:23.27 |
| Robin_Watts | They wouldn't tell me what account was used etc. | 19:23.36 |
| | So I had absolutely no clue about what cards to cancel etc. | 19:23.49 |
| | They were utterly useless. | 19:24.00 |
| mvrhel_laptop | that is not good | 19:24.20 |
| rayjj | kens: I was thinking of our use in the PDF interpreter -- I agree that PS input isn't possible, but our PDF interp is under our control | 19:24.26 |
| kens | rayjj we could probably do it (for now) in the PDF itnerpreter, because we only store the gstate in one place. But as chrisl poitned out, the more we store stuff in non-gc memopry, the more this is going to bite us | 19:25.02 |
| rayjj | kens: yes, if the method that triggers the GC when needed solves the issue, that's OK, but it's better not running the GC any more than we have to if we can keep things cleaned up ourselves | 19:26.12 |
| henrys | rayjj: I just got an "order confirmation" email from verizon that said to confirm or report fraud today that's the first I heard of it. | 19:26.21 |
| rayjj | kens: if we trigger the GC when we've allocated 8M and there are 800M of profiles, that's 100 GC runs | 19:27.12 |
| kens | rayjj we could do it in the PDF itnerpreter by adding a new magic operator to free the gstate, but I htink that we need a better solution. THat's only going to work in the PDF itnerpreter, and for as long as everyone remembers that this is a problem when using the gstate operator and the current colour space is ICCBased | 19:27.23 |
| | I'm not saying it can't be done, clearly it can, but its a sticking plaster | 19:28.06 |
| rayjj | kens: I'm quite in favor of chrisl's approach, but also want to keep GC runs down if possible (that's a performance issue, not a RAM usage issue) | 19:28.41 |
| kens | Yes I know, that's why I did so much testing and tried *hard* to use save/restore to avoid the work, it just wouldn't work. | 19:29.07 |
| rayjj | kens: tourniquet + plaster :-) | 19:29.12 |
| kens | And the more we do 'weird stuff' in the PDF interpreter the harder it is to make the next one work :-( | 19:29.38 |
| rayjj | kens: speaking of that, how's muscript coming ? | 19:30.08 |
| kens | And nobody will remember in a couple of years why the operator exists, or know that they need to track the result of gstate and call the magic operator when they are done with the gstate | 19:30.19 |
| | rayjj : I've done nothing with mooscript, I have no spare time | 19:30.34 |
| rayjj | kens: understandable. It's just that a lot of the headaches due to the PS based PDF interp would go up in a puff of evil smelling smoke | 19:31.24 |
| kens | Yes it would, but there's a ton of work to do to get it to work with high level devices | 19:31.45 |
| | And also a load of stuff that we can do now (in PostScript) that would have to be replicated somewhow | 19:32.06 |
| henrys | tor8: oh we did get nva reader to stop charging it is free (price-wise). Getting them compliant with GPL was not worth legal fees IMHO... let me know if ya'll disagree. Anyway meant to bring that up at the meeting. | 19:54.54 |
| Robin_Watts | henrys: Weren't they selling their app to other people to use as the basis for their apps though? | 20:00.16 |
| | Or am I thinking of someone else? | 20:00.22 |
| henrys | Robin_Watts: they must be up to something, why would they leave it up for free. | 20:01.17 |
| Robin_Watts | I reckon it's worth us hassling them to make them GPL compliant, but I reckon it's not worth legal fees. | 20:02.03 |
| tor8 | henrys: they still use their own proprietary encrypted file formats (just a garbled pdf file) though, so they must be making money somehow | 21:30.46 |
| | Forward 1 day (to 2015/02/11)>>> | |
IRC Logs