| <<<Back 1 day (to 2014/01/09) | 2014/01/10 |
bencc | is it possible to convert PowerPoint coordinates to PDF coordinates? | 00:37.26 |
| I have a pptx exported to PDF and I know the coordinates in the pptx (after dividing by 12700) | 00:37.56 |
ray_laptop | bencc: PDF coordinates are arbitrary. The default units are 720/inch but there is a 'cm' operator that changes the scaling, rotation and origin | 02:29.55 |
| bencc: If you render the PDF to an image (such as PNG) at 720 dpi, then you can use an image display program to show you coordinates (I use XnView on Windows) | 02:31.05 |
| bencc: but as far as looking at a 'decoded' PDF for object coordinates is not usually possible. You can use "mutool clean -d in.pdf out.pdf" to decode the PDF to make the page Contents viewable | 02:32.59 |
| bencc: if you are trying to find out what PDF sequence draws a certain item, I use ghostscript with -dPDFSTEP to find stuff. It usually takes repeated runs using large counts until it appears, then restarting to home in to the object of interest | 02:35.54 |
| bencc: but that doesn't work on PDF's that use transparency. To see if a PDF uses transparency, use: gs -q -- toolbin/pdf_info.ps in.pdf | 02:36.57 |
paulgardiner | tor8: another iOS commit if you have a moment | 12:02.29 |
Robin_Watts | paulgardiner, tor8: 2 commits on robin/master | 12:29.17 |
| paulgardiner: Your review seems OK to me, though I can't claim to understand any of it :) | 12:32.36 |
paulgardiner | ta. Just going to look at yours when synergy lets gives me back the ability to double click | 12:33.14 |
| First three from the branch point LGTM | 12:57.40 |
Robin_Watts | Thanks. | 12:57.52 |
| paulgardiner, tor8: So I've hit a SEGV in mutool clean. | 13:21.27 |
| It looks like we're parsing an array that contains [134234234 0 R ] or something similarly silly. | 13:21.56 |
| We parse that and make an indirect object that points to 134234234, despite their not being that many objects in the file. | 13:22.27 |
| The clean code then dereferences that number without checking its validity. | 13:22.48 |
| How should I fix this? | 13:22.55 |
| Should we automatically extend the xref table when we make a large indirection? | 13:23.17 |
| Or should I validate all indirections before using them? | 13:23.29 |
| Or both? | 13:23.32 |
paulgardiner | If you automatically extend then we probably need to keep explicit xref segments (which we've considered before). | 13:38.56 |
| ... and might that avoid the aliasing problem discussed yesterday? | 13:39.28 |
tor8 | Robin_Watts: dereferencing it should check its validity, IMO | 13:39.39 |
| and just resolve to the null object | 13:39.57 |
| would that work? | 13:40.04 |
Robin_Watts | tor8: That's what I've done locally. seems to work. | 14:10.44 |
| paulgardiner: Can't entirely follow that... | 14:11.08 |
paulgardiner | Robin_Watts: Probably irrelevant now, but I was thinking that if you had wanted to extend the xref table to include this huge number it would use a load of memory unless we represented the individual segments of each xref segment (in which case no reallocing of the existing segments, just creation of a new segment for this one entry). | 14:15.25 |
| ... and then with that in place, perhaps repair could be done without realloc, avoiding the aliasing problem | 14:16.01 |
| I man "individual segments of each xref section" | 14:16.35 |
Robin_Watts | I understand. | 14:16.46 |
| At the moment, we hold a 0...n array for each xref. | 14:17.06 |
paulgardiner | I mean "mean" not "man" :-) must learn to type | 14:17.07 |
Robin_Watts | zeniko suggested yesterday that we might hold just a 0..n array, but each entry would have an int in which says which xref level it came from. | 14:17.39 |
paulgardiner | So not keeping history? | 14:18.13 |
Robin_Watts | That would mean that we couldn't represent the difference between object 34 in the first xref and object 34 in the second xref. | 14:18.16 |
| We would keep history, just not quite as much. | 14:18.31 |
| I don't know if it's a good idea or not, just thought I'd mention it. | 14:18.50 |
paulgardiner | Would we not realloc still when adding new objects past the original end. | 14:19.33 |
Robin_Watts | We would. Unless we moved to a scheme where we kept a list of blocks or something. | 14:21.10 |
paulgardiner | List of blocks is sort of what I meant by segments | 14:21.41 |
| ... possibly. | 14:21.51 |
| and not unnatural because already in the spec | 14:22.01 |
Robin_Watts | yeah, I understood. | 14:22.15 |
paulgardiner | but I'm not sure it would mean that realloc was always avoidable | 14:22.19 |
Robin_Watts | I was thinking that we could break the range of numbers into 256 sized chunks. | 14:22.50 |
| Then for any given number n, we'd find it as being: block = index[n>>8] object = block[n & 0xFF] | 14:24.00 |
| So we might need to realloc index, but the block's would always be the same size and so would never need reallocing. | 14:24.21 |
paulgardiner | Nice. | 14:24.33 |
| Probably more time efficient than segments and nearly as space efficient in most cases | 14:25.18 |
Robin_Watts | We could move to such a scheme now, without fundamentally changing the way we store stuff, of course. | 14:25.23 |
| (I mean, without needing to move to just holding 1 xref table with the extra ints in) | 14:26.13 |
paulgardiner | Yes quite | 14:26.24 |
| Sounds like it removes the aliasing danger | 14:27.10 |
Robin_Watts | tor8, paulgardiner: http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=bf2aa9521b016d849eb4e88be3281fa84b5ab317 | 14:32.28 |
paulgardiner | Robin_Watts: LGTM | 14:34.53 |
Robin_Watts | Ta. | 14:36.32 |
paulgardiner | tor8: three iOS commits on paul/master. Robin has looked over the first, but pointed out that for iOS you probably should check too. | 14:58.33 |
tor8 | Robin_Watts: paulgardiner: I rather like the idea of allocating the xref sections in page-like blocks | 15:06.04 |
| paulgardiner: is the tap-image just a smaller sized button? | 15:07.00 |
| paulgardiner: does it do any sort of highlighting? (and is it too small to hit accurately, or likely to run afoul of apple's review?) | 15:07.31 |
paulgardiner | tor8: yes, but building my own seemed to be the only way to change the size | 15:07.55 |
| It doesn't visibly respond to touch. Partly I stuck it in a separate class so that I could address that if need be. | 15:08.34 |
tor8 | paulgardiner: okay. does it do any highlighting when depressed? (I know you really can't see because your finger is in the way, but the standard buttons have some sort of glow effect) | 15:08.35 |
paulgardiner | tor8: I can hit them accurately, but I can't guess what apple might think | 15:08.53 |
tor8 | is it possible to toggle the bg-color when depressed? | 15:09.23 |
paulgardiner | tor8: I looked at using a tint affect but that works only on ios 7 | 15:09.29 |
tor8 | paulgardiner: I'd just hit the bgcolor, we have transparent icons don't we? | 15:09.57 |
paulgardiner | I think prior to 7, the only way it to use two images (which we could do) | 15:10.04 |
tor8 | ick. I'd rather not have two images :) | 15:10.26 |
| the other two patches LGTM | 15:10.38 |
| but I think we really ought to have some sort of feedback on a touch | 15:10.48 |
paulgardiner | background color. Yes, worth a try. I didn't see anything letting me do that, but I can take a look now. | 15:10.56 |
| I thought we might be close to ready for a release but I've just noticed the slider doesn't respond under 7. | 15:12.06 |
| tor8: Strangely, calc.pdf (which requires javascript) is working on my 6.1 device. | 15:12.36 |
| tor8: in fact, I am building for 7, and the build is working on 6.1. | 15:13.06 |
Robin_Watts | Are you testing the same version you're building? :) | 15:13.26 |
paulgardiner | tor8: those commits okay then, given that adding touch feedback to TapImage is probably best as a separate commit? | 15:14.22 |
| Robin_Watts: hmm, good question. A lot of things have been working first time lately, which is always a bad sign. | 15:15.11 |
tor8 | paulgardiner: sure. | 15:15.52 |
paulgardiner | ta | 15:15.56 |
kens | Morning Ray | 15:16.58 |
ray_laptop | hi, kens | 15:17.03 |
| I have my coffee. Ready to hit the grind | 15:17.27 |
kens | :-) | 15:17.32 |
ray_laptop | I need to re-do the underlying device for the pdf14-clist accumulator. It's close, but I realized that I need a 'generic' Gray, RGB, CMYK or (more difficult) DeviceN device so that what I get out is suitable for an image to the "real" device | 15:19.14 |
kens | Makes sense | 15:20.02 |
ray_laptop | but the plumbing is basically working and I get something out (testing with the display device) | 15:20.03 |
kens | Sounds like a big step forward | 15:20.22 |
ray_laptop | kens: well, progress at least. and the buffer contents look reasonable | 15:21.06 |
kens | w/me is just pleased not to have to attempt it | 15:21.42 |
ray_laptop | I do want to chat with mvrhel later about why we never disconnect the pdf14 compositor device from the pgs. I do it (because I have to) and it works fine. | 15:23.05 |
| it's probably wrapped up in other save levels having pointers to the device, so if we 'restore' to one of those, it has a stale pointer | 15:24.46 |
| chrisl: do you have any thoughts (or haven't you delved into the device stuff enough) ? | 15:25.57 |
chrisl | ray_laptop: I don't know what to think, tbh. I don't think we do that for other devices, do we? | 15:26.52 |
henrys | chrisl: now the allocation will be non gc'd stable immovable just call malloc for f*ck sake. | 15:27.15 |
chrisl | henrys: I was wondering about switch it to use a chunk allocator, and remove the jpeg stream code's own "memory tracking" stuff | 15:28.11 |
henrys | chrisl: every time I look at one of these allocator issues I I just wonder how on earth any of this makes sense. | 15:28.39 |
kens | IMO the non-stable memory should be avoided, its as simple as that..... | 15:29.07 |
henrys | chrisl: by being immovable it already uses its own chunk so a separate malloc. | 15:29.22 |
chrisl | henrys: we need more than immovable - that is also still subject to garbage collection.... | 15:30.11 |
| I can't see why we need more than two allocators *ever* - one gc'ed and one non-gc'ed | 15:30.20 |
henrys | chrisl: get rid of stable entirely? | 15:31.10 |
chrisl | henrys: well, I think that would be desirable, but probably not practical at this stage :-( | 15:31.42 |
kens | I wouldn't want to try and recode pdfwrite :-( | 15:32.08 |
chrisl | To have gc'ed/stable/immovable/non-gc'ed just seems overkill....... but like I say, probably to late to change with a *lot* of pain | 15:33.15 |
kens | Although I think its insane the amount of stuff that pdfwrite allocates as GC'ed, for no good reason | 15:33.15 |
ray_laptop | usually, if I want 'immovable' I just go right to non_gc_memory | 15:33.54 |
chrisl | ray_laptop: I suspect everyone does nowadays | 15:34.20 |
ray_laptop | kens: right. We had discussed (everal staff meetings ago) about changing to use non_gc_memory more often | 15:34.35 |
Robin_Watts | Well, we could experiment with it without changing much code, right? | 15:35.03 |
kens | Yes, I agreed (and still do) but I don't want to have to try and redo pdfwrite like that.... | 15:35.07 |
ray_laptop | it would cut down the workload on the gc trace also | 15:35.23 |
Robin_Watts | Just change so that when we set up gs_memory_t's, make some of the pointers point to the same things. | 15:35.31 |
henrys | Robin_Watts: write we could rewire the memory manager to push all stable and immovable stuff through non-gc | 15:35.53 |
| s/write/right | 15:36.02 |
chrisl | But then we'd probably leak like a sieve...... | 15:36.26 |
ray_laptop | since the graphics lib runs without a gc'ed allocator, and doesn't leak, then the only thing that needs the gc allocator is the PS interp (z functions) | 15:36.47 |
Robin_Watts | chrisl: a test with a memento build might be worthwhile. | 15:36.55 |
| MEMENTO_LEAKONLY probably for speed (or whatever the option is called) | 15:37.18 |
chrisl | I think it's something worth looking at - when we have the opportunity..... | 15:37.26 |
henrys | but performance will kill us we have to make the heap memory manager a "chunk allocator" or something like it before we can do anything like that. | 15:37.46 |
| you can't have a malloc for each allocation. | 15:38.07 |
Robin_Watts | henrys: You really believe the chunk allocator is that much faster than malloc ? | 15:38.24 |
henrys | Robin_Watts: 10% in pcl malloc vs. chunk | 15:38.47 |
Robin_Watts | Actually, I know that 10+ years ago, this was an issue, cos I wrote a heap manager for Picsel that is still there in the current code. | 15:39.19 |
ray_laptop | henrys: right. we don't want to be going to the gsmalloc heap allocator frequently. The locking would kill us and the chain of allocated blocks would become unwieldy | 15:39.26 |
chrisl | That's not something we'd have to worry about initially, though | 15:39.54 |
Robin_Watts | oh, it's the free all with the chunks that saves the time. | 15:39.57 |
paulgardiner | tor8: iOS is strangely devoid of effects that one can apply to images. Using two sets of pngs is looking more and more the only practical way. | 15:40.33 |
ray_laptop | Robin_Watts: that and the locking. That's a killer on Windows. I saw 25% worse performance | 15:40.37 |
henrys | we really should have a chunk allocator for the heap allocations. We are letting many small allocs go through to the heap already. | 15:45.01 |
tor8 | paulgardiner: I thought it'd be just a matter of setting the background color of th UIImageView superclass and let that shine through the transparent png... | 15:45.49 |
ray_laptop | henrys: that's what we have on PCL | 15:46.33 |
paulgardiner | tor8: can't immediately see anything about backgrounds. Possibly I could try building a view containing the image and an explicit background below. Actually that might be reasonable to do | 15:47.22 |
henrys | ray_laptop: every time I look at memory issues like this bug I'm amazed at the sheer volume of allocations in gs | 15:47.34 |
paulgardiner | tor8: ignore me. I was looking in the wrong subclass | 15:48.16 |
henrys | ray_laptop:well how complicated is it to put the chunk allocator before the heap allocator? | 15:55.03 |
| hi marcosw | 15:58.06 |
marcosw | morning henrys | 15:58.14 |
ray_laptop | henrys: 'wrapping' the heap allocator in a chunk allocator is easy. That's what we do for MT rendering threads and for PCL | 15:58.35 |
henrys | marcosw: did you say you had an update for guillaumes bug? | 15:59.19 |
ray_laptop | The 'gotcha' is that the chunk allocator is not thread safe, but that's no different to the GC allocator. It just means that we don't share a chunk allocator amongst threads | 15:59.50 |
marcosw | yes, but i forgot to update the bug. I'll do it now. | 15:59.56 |
ray_laptop | henrys: which bug, BTW | 16:00.29 |
henrys | ray_laptop:the memory stuff started from the jpeg bug you gave me. | 16:01.05 |
ray_laptop | henrys: Oh, yes. I saw some of the comments on that | 16:01.32 |
| henrys: I thought that was a JPX bug, however | 16:01.43 |
henrys | http://bugs.ghostscript.com/show_bug.cgi?id=694871 | 16:02.21 |
ray_laptop | henrys: Oh, I was confusing it with 873 | 16:02.37 |
henrys | ah 873 - looks like hin-tak got to it before shelly | 16:03.52 |
| maybe we need a better system. | 16:04.04 |
ray_laptop | henrys: so you are thinking that the 'non_gc_memory will actually be a chun wrapped heap allocator, right ? | 16:04.13 |
| tring to type too fast :-( | 16:04.35 |
Robin_Watts | "chun wrapped" sounds like a style of chinese food. | 16:05.30 |
henrys | ray_laptop:we want to get the immovables also which get their own chunk and allocation. Maybe we should have a chunk allocator for the chunk allocations - a chunk of chunks so to speak. | 16:05.41 |
ray_laptop | and the GC allocator will have that chunk allocator as it's base | 16:05.42 |
| henrys: I don't recall. Are 'immovables' GC'd ? | 16:06.29 |
henrys | ray_laptop: I assumed that meant they couldn't be relocated but gc was okay but god knows I didn't check | 16:07.06 |
ray_laptop | ISTR that they are. In that case immovables sort themselves out since it is part of the GC memory | 16:07.07 |
henrys | chrisl probably knows | 16:07.37 |
chrisl | I believe immovable is gc'ed but relocated | 16:08.01 |
| but not relocated (sorry!) | 16:08.23 |
ray_laptop | as far as most allocators (except the GC) alloc...immovable is the same as alloc... | 16:08.27 |
| since no allocator except the GC tries to do anything as daft as moving things around | 16:09.22 |
henrys | chrisl: that's what I thought but I don't have the stones to say anything about the memory manager with conviction. | 16:09.23 |
ray_laptop | henrys: so how does having a chunk wrapper on the heap help with JPEG ? | 16:10.11 |
henrys | ray_laptop:it should improve the performance the jpeg allocator is doing many very small immovable allocations - this is an aside from the bug | 16:11.28 |
ray_laptop | henrys: I see. Well, the chunk allocator doesn't add much overhead since most (excpeting JPEG) non_gc_memory allocations are from the GC and those will each get their own chunk | 16:13.01 |
henrys | chrisl: I did assign that back to you, but I'll make the change to non-gc if you want, might as well get the customer on their way. | 16:14.08 |
ray_laptop | henrys: it's easy enough to try. | 16:14.24 |
chrisl | henrys: no, let me look at it - I think there is more this than we first thought. | 16:14.35 |
ray_laptop | OK, Just mention me here if you want me (or SMS if I'm not responding). Back to pdf14 clist accumulation... | 16:15.41 |
ray_laptop | always has to be careful typing c l i s t | 16:16.03 |
| chrisl: with memory allocations, there is always more to it :-( | 16:16.56 |
henrys | chrisl: oh have at it! | 16:17.02 |
Robin_Watts | paulgardiner, tor8: http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=29ce845021239655a6fb052e8d2754db54f92820 | 16:28.07 |
| actually, ignore that for a mo. | 16:28.27 |
| Better: http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=7e73f436a6d3ac42186983dfc612ce5d7a7da287 | 16:29.31 |
tor8 | Robin_Watts: LGTM | 16:31.53 |
| actually, I'd prefer if (i < n) for clarity | 16:32.19 |
| but it's not a big deal | 16:32.29 |
Robin_Watts | ok. | 16:35.11 |
paulgardiner | tor8: commit on paul/master. We now have animated buttons. | 16:54.51 |
Robin_Watts | nice. | 17:07.43 |
bencc | ray_laptop: thanks for your response yesterday | 17:13.32 |
| I have a pptx and a pdf (converted with powerpoint) | 17:13.54 |
| I can extract video files from the pptx | 17:14.08 |
| but I need an automatic way to find the position of the video in the pdf | 17:14.41 |
| anyway to convert pptx coordinates to the exported pdf coordinates? | 17:16.01 |
| or find a specific object? | 17:16.06 |
mvrhel_laptop | brb | 18:04.41 |
ray_laptop | bencc: sorry -- wasn't paying attention | 18:06.23 |
| bencc: If you use muttool clean -d on the PDF then look at it with an editor, you can (probably) find the video | 18:06.58 |
| bencc: then you can just save that object stream, or write a script to 'find' the video objects and save them. | 18:07.51 |
| bencc: the -d doesn't do anything to binary objects like images or videos (or at least it's not supposed to) | 18:08.56 |
bencc | ray_laptop: I need to make it automatic | 18:09.56 |
| powerpoint doesn't embed the video in the pdf, that's why I want to add it manually as annotation above the video image | 18:10.21 |
ray_laptop | bencc: modifying a PDF to add a video annotation isn't all that trivial. | 18:27.06 |
| bencc: so to 'automate' it, you'd use a script on the decoded pdf to find the object you are looking for. | 18:28.01 |
Robin_Watts | ray_laptop: That's what he's asking. He wants a way to find the position of the video "object" within the pdf. | 18:28.38 |
ray_laptop | bencc: but unless PPT is very consistent about how it writes coordinates, having an annotation be placed aligned with some object's position may not be easy | 18:29.43 |
| Robin_Watts: the video isn't there, but I read that as 'where the video is supposed to go' | 18:30.15 |
| Robin_Watts: and that there is some other object in the PDF that can give that position. | 18:30.48 |
Robin_Watts | ray_laptop: The video isn't there, but (I think from what he said) there is an object where the video was. Probably an image of the first frame or something. | 18:30.51 |
ray_laptop | bencc: without seeing a PDF, it's really just guesswork. I don't use PPT, and only have an old version | 18:31.29 |
| Robin_Watts: right, that's what I was trying to say. If there is a frame or a filled rect or an image or something, the problem is that due to concatenation of 'cm' up to that point, trying to look at the position of an image and make converrt that to the coorids for an annotation may be difficult | 18:33.40 |
Robin_Watts | ray_laptop: Right. He could decode to an image and then look in the image, but even then to get back to the original PDF coords requires looking at the mediabox etc and doing some scaling. | 18:34.36 |
ray_laptop | now if PPT is reasonably sane about cooridnates, and makes juducious use of q...Q to avoid confusiong concatenation ... | 18:34.48 |
| but I don't count on PPT (written by some so-called programmers under the guidance of Microsoft) being sane | 18:35.48 |
marcosw1 | henrys: I'm having trouble with my plotter. It's seems to have dropped off the intranet. This happened before and it ended up being a bad hp jetdirect card. I'll let you know when it's back up and I can update the bug. | 18:37.26 |
henrys | okay marcosw1 | 18:40.04 |
| Robin_Watts: I thought there were only a few gs fuzzing problems? | 18:52.35 |
Robin_Watts | Only 3 in what they just sent. | 18:52.50 |
| and they are more technically pcl -> pdfwrite ones. | 18:53.06 |
| Presumably marcosw1 is retesting original files? | 18:53.21 |
henrys | Robin_Watts: haven't looked just thought I saw a ton of emails for gs bugs. | 18:54.36 |
Robin_Watts | oh. | 18:55.11 |
| he's testing gs with the mupdf fuzzing files :) | 18:55.22 |
henrys | bastard | 18:56.10 |
Robin_Watts | But I only have to fix the files once, right? :) | 18:56.33 |
bencc | ray_laptop: coordinates of each image in the pdf are absolute? | 19:24.55 |
| assuming there is a sane conversion, I'll might be able to find an object in position x, y... | 19:25.21 |
henrys | Robin_Watts, chrisl_away : do either of you know shelly's status with the openjpeg folks: simon has a bunch of openjpeg changes, do we fork? | 19:44.32 |
mvrhel_laptop | brb | 19:44.49 |
Robin_Watts | I thought that shelly had got all our stuff passed back. | 19:44.54 |
| and was in negotiation with them over the malloc/free stuff. | 19:45.05 |
| possibly his last mail said that they'd gone quiet over that? | 19:45.30 |
mvrhel | hi Robin_Watts. I am going to have a commit that has some minor fixes that I needed to do to get the winRT 8.1 app to pass certification | 19:59.03 |
Robin_Watts | mvrhel: OK. | 19:59.13 |
mvrhel | I was missing a memory clean up for the directX stuff during the app suspension and I had to add a predefine in the project. | 20:02.18 |
Robin_Watts | say when it's ready for me to look at. | 20:04.34 |
mvrhel | ok rerunning validation tool now if it all looks good I will push | 20:06.12 |
henrys | Robin_Watts: did you talk to zeniko about these overflow checks to satisfy msvc assertions? | 20:09.08 |
Robin_Watts | henrys: urm... no ? | 20:09.36 |
| What msvc assertions? | 20:09.44 |
henrys | http://git.ghostscript.com/?p=user/zeniko/ghostpdl.git;a=commitdiff;h=1611eb12ae94212f24d79986b4ad862e83c27eb0 | 20:09.59 |
Robin_Watts | nope, not me. | 20:10.36 |
mvrhel | Robin_Watts: ok pushed | 20:15.11 |
Robin_Watts | mvrhel: Looks good. | 20:16.38 |
mvrhel | ok thanks | 20:16.45 |
Robin_Watts | tor8, paulgardiner, mvrhel: 2 commits (ignore the top 2) on robin/master for review. | 20:23.56 |
Robin_Watts | goes for food. | 20:24.04 |
| Forward 1 day (to 2014/01/11)>>> | |