| | <<<Back 1 day (to 2012/05/09) | 2012/05/10 |
| ray_laptop | 25% of the cluster servers are down :-( | 05:10.40 |
| | miles, x6 and i7 | 05:10.57 |
| | I think miles and x6 are ones marcos has | 05:11.27 |
| ray_laptop | thinks we need a way to know which machine is who's responsibility to reboot (and secondary contact). | 05:12.38 |
| | henrys: (for the logs) the icc_link and icc_linkcache semaphore free in finalization appear to be OK. The cluster run is with the changes. I'll check for the 693001 leaks while it is running. | 05:14.22 |
| | the windows task manager shows the memory constant at 20,772K throughout the customers 150Mb (500 page) test file. Looks like a good fix for 693001 as well as 692372 | 05:47.15 |
| | darn. I had staged the changes as individual commits locally so when I pushed them they each resulted in a regression run. Good thing it is at a time when most of us are sleepiing. | 06:07.47 |
| | henrys: (for the log). These changes fix the leak and also the semaphore issue of bug 692372 | 06:08.44 |
| | bug 693001 closed as FIXED | 06:13.20 |
| | g'nite all | 06:13.40 |
| naveen_ | Hi Robin..Morning.. | 09:27.41 |
| miha | is there an option in mupdf to show two portrait pages in landscape orientation, not just one? | 09:32.51 |
| sebras | miha: mupdf the viewer, no. currently it can only show one page at a time. | 09:36.58 |
| Robin_Watts | naveen_: Hi | 09:37.16 |
| sebras | miha: mupdf the library allows you to write a viewer that supports this though. | 09:37.24 |
| miha | sebras: looking at the viewer.. trying to understand it.. :) | 09:38.54 |
| | sebras: i guess i could go render two pages. but how to handle zoom properly then :( | 09:39.21 |
| naveen_ | Robin, I have written methods that will read data from socket and serve data to muPDF renderer...It looks like initially muPDF seeks alot and the time taken to load the document is long...Is there any way possible to make loading of pdf fast? | 09:39.56 |
| sebras | miha: zooming would require you to know what part of the two pages are visible and rendering only those. | 09:40.21 |
| Robin_Watts | naveen_: Yes, mupdf will seek all over the place to start with. | 09:40.33 |
| | In order to read a PDF file, we first have to read the first few bytes (10 bytes or so), then read from the end of the file (20 bytes or so), then seek back a bit from there (about 1K typically) | 09:41.47 |
| | then read that last 1K, then seek and read page info. | 09:42.04 |
| | And the page info will require more seeks too. | 09:42.13 |
| | This seeking is pretty much unavoidable, unless you want us to just read through the whole file and cache it all in memory. | 09:42.50 |
| | It's fundamental to the way PDF works. | 09:43.04 |
| | (There is a variant of PDF, "linearized PDF", where we can read the first page without seeking, but not all files are of that form, and we don't currently special case that) | 09:43.47 |
| | naveen_: BUT... with care your stream code can efficiently handle these cases. | 09:44.15 |
| | Firstly, you can arrange to not decode entire 64K encrypted blocks at a time, but rather to decode within subblocks (so when we read 10 bytes from the start of a file, you don't decode 64K to return those 10 bytes) | 09:45.23 |
| naveen_ | ok...I understand it .... | 09:47.38 |
| Robin_Watts | Secondly, when we read from the 20-30 bytes at the end of the file, if you decode your way through the last block and just return the requested bytes, be sure that you are keeping an internal copy of the rest of the decoded data. | 09:48.17 |
| | because the next request will probably be for just a few bytes earlier. | 09:48.30 |
| naveen_ | yes...I'm saving th e last decrypted block.....currently I'm reading to a buffer then copying back to buf* ...I guess reading directly to buf* may increase the performance a bit...but currently If I read data directly to buf* the initial bytes read in the buf are getting corrupted... | 09:48.59 |
| Robin_Watts | OK, if you're saving the last decrypted block then you should be fairly efficient. | 09:49.42 |
| miha | looking at source mupdf PageView.java | 09:50.39 |
| | // ignored, so we recreate it. There may be another way around this | 09:50.40 |
| naveen_ | I'm not sure why the data is getting corrupted if I read directly into buf* | 09:50.45 |
| miha | When hardware accelerated, updates to the bitmap seem to be ignored, so we recreate it. There may be another way around this that we are yet to find. | 09:51.04 |
| | hardware accelerated means bitmap drawing is done by opengl texture?! :) | 09:51.22 |
| | so it's quite logical updates later dont show :$ | 09:51.52 |
| Robin_Watts | naveen_: Without digging through your code, I can't say. | 09:52.40 |
| | miha: Hold on... | 09:52.50 |
| | This code was written by paulgardiner, so bear with me... | 09:54.43 |
| | AIUI, we make a bitmap (mEntireBm) and we render to it. | 09:55.05 |
| naveen_ | Robin,you can take a look at the read_stream function here http://pastebin.com/Lk0tDqqa | 09:55.18 |
| Robin_Watts | Then we display that. | 09:55.45 |
| | When we move to a new page, the code you highlighted there throws away the bitmap, and recreates a new one, that it renders to and displays. | 09:56.14 |
| miha | Robin_Watts: sure. i just comment why "update to bitmap" wouldnt show | 09:56.37 |
| Robin_Watts | The code in question may have previously tried to reuse the existing bitmap. | 09:56.45 |
| miha | i know, cause i tried rendering very large image on hardware acceleration. and got opengl texture too big :D | 09:57.03 |
| Robin_Watts | So I don't think it's hurting us - especially as we'd have to cope with the bitmap changing size (as not all pages are the same shape) | 09:57.09 |
| miha | Robin_Watts: and i'd guess it makes texture only first time. | 09:57.21 |
| Robin_Watts | miha: right. | 09:57.22 |
| | If you know of a better way around this, please drop us a mail. | 09:57.39 |
| | I'd have hoped there was a 'refresh texture' call or something. | 09:57.55 |
| miha | Robin_Watts: it's weird, cause android docs say | 09:59.38 |
| | Don't modify bitmaps too often Every time you change the content of a bitmap, it is uploaded again as a GPU texture the next time you draw it. | 09:59.39 |
| | http://developer.android.com/guide/topics/graphics/hardware-accel.html | 09:59.47 |
| Robin_Watts | naveen_: So, if I'm following you correctly, on EVERY seek you are writing/reading from a socket. | 10:00.11 |
| | sorry, on every READ. | 10:00.20 |
| naveen_ | yes Robin...thats correct.. | 10:00.29 |
| Robin_Watts | Well, that's going to be slow. | 10:00.39 |
| miha | Robin_Watts: http://stackoverflow.com/questions/6279700/why-do-ndk-updated-bitmaps-show-black-when-hardwareaccelerated-is-turned-on-in-h | 10:01.02 |
| Robin_Watts | Caching the last decoded block within the server ain't going to help much; the caching needs to be within the seek code to avoid the socket transfer. | 10:01.29 |
| naveen_ | hmm..ok I'll try that... | 10:02.15 |
| | When will I get " error: zlib error: incorrect header check" error? | 10:02.32 |
| Robin_Watts | paulgardiner: Check the logs; miha has been asking/giving links about android hardware accelerated bitmap stuff, and I'm a bit out of my depth :) | 10:03.34 |
| paulgardiner | Robin_Watts: ok thanks | 10:03.50 |
| Robin_Watts | naveen_: In a correct PDF file you should never see that. | 10:04.21 |
| | In a corrupt one, you can get that while reading a flate compressed stream. | 10:04.41 |
| miha | paulgardiner: no real solution there | 10:05.07 |
| | sorry :( | 10:05.11 |
| naveen_ | with this approach, I'm not getting images loaded sometimes...Is this something with the data I'm filling in buf*? | 10:05.38 |
| Robin_Watts | naveen_: It sounds to me like you have a bug that's serving broken data somewhere. | 10:06.37 |
| | I'd write a test app. | 10:06.49 |
| | Have your server set up to serve from an encrypted file, testenc.pdf, and have the unencrypted version of the same file, test.pdf. | 10:07.39 |
| paulgardiner | miha: mysterious problem. There are lots of methods that sound like they might inform the system that it needs to update the hardware buffers. I tried them all. | 10:07.53 |
| Robin_Watts | Then I'd request random chunks from the server, and compare what you get with the same chunks from test.pdf | 10:08.11 |
| | And leave that running. | 10:08.17 |
| | If you see differences, you'll know your server is broken. | 10:08.30 |
| | do you follow ? | 10:08.32 |
| naveen_ | yes .. I'll do that.... | 10:08.42 |
| miha | paulgardiner: i'm very interested in having two portrait pages drawn at once when in landscape | 10:11.24 |
| | best i thought so far is to have two pages rendered in MuPDFCore and returning sum of widths | 10:11.53 |
| Robin_Watts | miha: I don't think that's the way to go. | 10:12.07 |
| paulgardiner | miha: anything like that should be done in ReaderView. It's that class which controls the positions of pages. There are already actually 3 pages in a line, but they are positioned so that only the middle one is visible | 10:13.07 |
| miha | paulgardiner: but how would the zoom work then? | 10:13.53 |
| Robin_Watts | The drawing of the bitmaps is not fundamentally different for 2 up page mode; it's just the positioning of those bitmaps (which as paulgardiner says is done in ReaderView) | 10:14.06 |
| | ReaderView does the zoom calculatons I suspect. | 10:15.00 |
| | You need to decide what you want zooming behaviour to be. | 10:15.12 |
| | (Do you drop back to 1up mode when you zoom in too much? or do you stay as 2up and zoom both pages together?) | 10:15.36 |
| paulgardiner | Yes, as you zoom out, it could pull back into 2 up and then 3 up, and that could work in either orientation. | 10:16.26 |
| miha | interesting | 10:17.36 |
| paulgardiner | The changes to ReaderView wouldn't be at all easy, but that is where to do it. | 10:18.08 |
| | If we were considering showing many more pages simultaneoulsy then we'd need to change PageView too because we'd need to allow it to use smaller bitmaps when zoomed far out, so that having many views wouldn't take to much memory | 10:19.37 |
| | But the changes should be isolated to those two classes. | 10:20.16 |
| miha | paulgardiner: you already use bitmaps as they fit screen | 10:21.42 |
| paulgardiner | There are two bitmaps for each view, both approximately screen sized. One contains an image of the entire page, and is scaled up as you zoom in. | 10:23.00 |
| | The other contains a high res patch of just the area currently shown. | 10:23.22 |
| miha | yes | 10:23.34 |
| paulgardiner | If we were zooming out so far that a page took up 1/4 screen then that approach becomes wasteful. | 10:24.01 |
| miha | it's even seen. first when you zoom, it isnt very detailed, then it shows perfectly :) | 10:24.07 |
| paulgardiner | miha: yeah exactly. | 10:24.19 |
| | That mechanism of two bitmaps is copied from the iOS app, originally written by Tor I think. | 10:25.04 |
| miha | i did try my own viewer like that. problem was i didnt use 'patches' and bitmap got huge. :) | 10:26.04 |
| | other than that, it worked just fine | 10:26.15 |
| | that, and handling touch events was ugly | 10:26.24 |
| | then i looked for alternative | 10:27.24 |
| | :$ | 10:27.26 |
| paulgardiner | miha: I found the touch event handling a bit scary too, although Android does have some nice gesture recognition stuff | 10:27.29 |
| | miha: as I said before, there are actually 3 views always present (other than when you are viewing the first or last page). They are kept apart by a value called GAP. When zoomed out in landscape mode, there is code to expand that distance to stop more than one view being visible. Altering that might be a start. | 10:32.35 |
| | ... that's if I could find the bit of code I'm refering to. :-) | 10:32.54 |
| | Ah subScreenSizeOffset. Making that return (0,0) would be interesting, but would only be a start. | 10:34.00 |
| miha | paulgardiner: yeah, just noticed GAP, i wanted to ask :) | 10:45.22 |
| | paulgardiner: so this makes all three together int gap = 0/*leftOffset.x + GAP +cvOffset.x */; | 10:53.06 |
| | for both next and previous | 10:53.12 |
| | :) | 10:53.12 |
| paulgardiner | Yeah, that sounds right. Might get some funny behaviour because of assumptions elsewhere in the code, but worth a try | 10:55.14 |
| miha | no it works just fine, even zooming | 10:56.54 |
| | just need more than 3 views.. 5 i guess | 10:56.59 |
| | and then "center" horizontal position depending on how many views there are | 10:57.25 |
| | oh well it renders a lot :p | 10:58.11 |
| paulgardiner | Yeah, and you probably want the 5 views only in the case of portrait pages on a landscape screen, otherwise memory usage will increase a lot. | 10:58.47 |
| Robin_Watts | The 5 should be a calculated thing, I guess. | 10:59.21 |
| miha | well most screens have about same aspect ratio? | 10:59.51 |
| | like 16:10 | 11:00.08 |
| | not 30:10 | 11:00.15 |
| | but yes | 11:00.37 |
| paulgardiner | Robin_Watts: yes, that would be better | 11:01.05 |
| miha | and this calculation needs to be changed // Move to next or previous if current is sufficiently off center | 11:02.10 |
| | :) | 11:02.13 |
| | i scrolled by two pages, not one :) | 11:02.34 |
| Robin_Watts | sebras: I've pushed your patch. | 11:05.38 |
| paulgardiner | You can afford to have three screen's worth of pages without making memory usage any worse than the current worse case. | 11:05.40 |
| Robin_Watts | tor8: Do you want to look at my patch from last night, or are you happy with sebras having looked? | 11:06.09 |
| tor8 | Robin_Watts: I can't say I'm thrilled about the extra arguments... | 11:08.39 |
| Robin_Watts | Me either. but they are only internal. | 11:08.57 |
| tor8 | but at least they're in -internal | 11:08.59 |
| Robin_Watts | and I couldn't see an obvious other way of doing it. | 11:09.10 |
| | (unless we read and decrypt everything to memory before doing the reorder - and that would bloat our memory use a lot) | 11:09.36 |
| tor8 | a long time ago we had code to cache the contents of a stream from the xref_entry | 11:09.52 |
| | (so we could create new streams and have them written back out) | 11:10.04 |
| Robin_Watts | Right, but then we'd need to load the contents of every stream into memory (uncompressed) before writing it out. | 11:10.32 |
| | and that would do horrid things to our memory use. | 11:10.39 |
| tor8 | which would allow the decrypt it all to memory during reorder. but admittedly, this way you've implemented is better. | 11:10.44 |
| miha | having more views is better for portrait too, it preloads pages | 11:13.02 |
| paulgardiner | miha: yeah the views will be bigger in that case, and hence the memory use will be huge | 11:17.46 |
| Robin_Watts | tor8: I could have left pdf_{load,open}_image_stream with the same params, by adding new pdf_{load,open}_renumbered_image_stream functions. | 11:18.53 |
| | The former would have called the latter. | 11:18.59 |
| | but given that it was an internal only function, I didn't bother. | 11:19.14 |
| tor8 | Robin_Watts: yeah. feel free to committ. | 11:19.47 |
| | commit. even. | 11:19.50 |
| Robin_Watts | ok, thanks. | 11:19.53 |
| | I'm going to look at reordering the objects into the right order for linearisation today. | 11:20.23 |
| sebras | Robin_Watts: cool thanks for looking it over. | 11:20.34 |
| tor8 | Robin_Watts: aha! :) | 11:20.38 |
| sebras | Robin_Watts: What do you plan to do for object streams? | 11:21.23 |
| tor8 | sebras: like we do now in pdfclean I'd reckon: unpack them :) | 11:21.46 |
| Robin_Watts | sebras: Hadn't considered it - like tor8 says, we'll just unpack 'em for now. | 11:22.12 |
| | We ought to support encryption while writing out too. | 11:22.37 |
| tor8 | Robin_Watts: renumber them, or just write them out in a different order? | 11:22.39 |
| Robin_Watts | tor8: For linearisation, we need to both renumber, and write them out in a different order. | 11:23.00 |
| paulgardiner | Robin_Watts, tor8: in the forms work, I create streams in a way that I thought was savable. From the sounds of things, they aren't then. | 11:23.11 |
| sebras | Robin_Watts tor8: and no splitting of the xref? | 11:23.35 |
| Robin_Watts | Page 1 (and all the objects it uses) must be numbered n to m-1. All the other stuff needs to be ordered 1..n-1 | 11:23.37 |
| | and it gets written in the order n to m-1 then 1 to n-1 | 11:23.50 |
| | sebras: For today, no, no splitting of the xref. | 11:24.06 |
| tor8 | Robin_Watts: right. so that's why page 1 usually has high object numbers. | 11:24.18 |
| Robin_Watts | I'm trying to get to linearised output a step at a time. | 11:24.22 |
| tor8 | Robin_Watts: sebras: I'm fed up with linking half a dozen plus command line tools. Any opposition to me removing all but 'mubusy' and making mubusy look at both argv[0] and argv[1] for which command to invoke? | 11:25.14 |
| | and I guess rename mubusy to mupdftool? | 11:25.28 |
| sebras | tor8: no, that's not a problem for me. | 11:25.50 |
| Robin_Watts | That's bad on windows. | 11:25.52 |
| sebras | Robin_Watts: no soft links!? ;) | 11:26.10 |
| tor8 | Robin_Watts: it'll mess up the sane scripts. | 11:26.12 |
| | sebras: no links. unless you want to get into hairy ntfs mechanics. | 11:26.24 |
| Robin_Watts | sebras: junction. | 11:26.27 |
| | but you can't rely on it. | 11:26.33 |
| sebras | Robin_Watts: how come? | 11:26.38 |
| tor8 | Robin_Watts: hence my suggestion for argv[1] so you can invoke: "mupdftool draw ..." etc | 11:26.51 |
| Robin_Watts | Suppose you don't have ntfs? | 11:26.53 |
| sebras | Robin_Watts: so copy the resulting .exe for windows but not for other platforms? | 11:27.02 |
| Robin_Watts | tor8: yeah, but that's a huge step backwards. | 11:27.04 |
| sebras | Robin_Watts: no I don't... | 11:27.07 |
| tor8 | Robin_Watts: .bat files ;) | 11:27.17 |
| | why is it a huge step backwards? | 11:27.50 |
| Robin_Watts | I prefer sebras idea of copying mupdftool.exe as mupudfclean.exe etc | 11:27.55 |
| tor8 | it breaks script compatibility; and looking at both argv[0] and [1] lets us do both the "mupdftool draw" and copy to multiple exes | 11:28.46 |
| Robin_Watts | What breaks script compatibility ? | 11:29.22 |
| | Suppose in future versions we want to offer mupdf and mupdfV8. | 11:29.43 |
| | (or mupdfjs or something) | 11:29.57 |
| tor8 | renaming the tools breaks compatibility | 11:30.04 |
| Robin_Watts | I really don't see the problem with linking multiple times. | 11:30.39 |
| | It's not like it ever takes more than 15 seconds to do an edit/rebuild/relink. | 11:31.02 |
| | (unless you change the header file) | 11:31.11 |
| | and the link time is always a small fraction. | 11:31.28 |
| tor8 | Robin_Watts: linking takes a long time, when our recompiles are on the order of seconds the linking takes a long time. at least on my machines. | 11:31.42 |
| Robin_Watts | "make build=debug mubusy" then :) | 11:32.04 |
| tor8 | then there's the issue of namespace pollution and zip-file bloat by shipping a lot of separate binaries that all include the fonts | 11:32.25 |
| sebras | tor8: especially now that all tools include all of mupdf... | 11:33.00 |
| tor8 | I had a stab at separating that out, by splitting functions across source files and having a pdf_open_document_no_run function that didn't link in the interpreter | 11:33.02 |
| Robin_Watts | but you were thwarted by type3's? or pdf_resolve_indirect ? | 11:33.41 |
| tor8 | and that worked, but still pulled in more than I liked, so I may have a go at shuffling functions around a bit more and staring a bit harder at the inter module dependencies | 11:33.46 |
| | Robin_Watts: pdf_load_font called from pdf_run_* pulled in the cmap and font data | 11:34.19 |
| Robin_Watts | namespace pollution is a non-issue, IMHO; all our stuff begins with mu. | 11:34.34 |
| tor8 | if I just leave out the pdf_run from the document callbacks the linker leaves out the static stuff | 11:34.40 |
| | it still pulls in all the image loading and openjpeg stuff though | 11:34.58 |
| Robin_Watts | and if we're copying exes, or using softlinks to get backwardly compatible behaviour, we're polluting exactly the same. | 11:35.15 |
| | zipfile size isn't an issue really, as far as I'm concerned (we're still tiny compared to the alternatives) | 11:35.45 |
| tor8 | Robin_Watts: I'd be happy with 2 or 3 binaries: mupdf/muview, mudraw, and mubusy the swiss army knife of pdf manipulation | 11:35.47 |
| Robin_Watts | tor8: For unix, I agree, I think, because softlinks are so easy. For windows, I'm not so sure. | 11:37.15 |
| tor8 | Robin_Watts: well, consider git and svn and a lot of other tools that take a first argument as the command to execute. that's what I think we could emulate. | 11:39.05 |
| Robin_Watts | I am not saying that we shouldn't offer a tool that works that way. | 11:39.43 |
| tor8 | mupdf/muview and mudraw handle all the PDLs we can parse, but the other tools are all pdf specific | 11:39.44 |
| Robin_Watts | just that I don't feel the need to disable the other tools. | 11:40.17 |
| tor8 | so I think it'd make sense to gather the pdf manipulation into one swiss army tool which it'd be easy to add commands to without having to add lots of manpages and build targets | 11:40.33 |
| Robin_Watts | Actually, I don't think I've ever used pdfinfo, pdfshow or pdfextract in my life, so I guess I shouldn't be arguing about them disappearing :) | 11:41.45 |
| tor8 | Robin_Watts: just relinking all the binaries takes 8s on my machine. which could be an argument to upgrade instead :) | 11:42.40 |
| Robin_Watts | I am 100% sure that your machines are all faster than mine :) | 11:43.26 |
| tor8 | Robin_Watts: well, I'll have a bash at it. I'll leave mudraw as a separate tool though, that one isn't pdf specific as the others. | 11:43.28 |
| sebras | tor8: recompling mupdf with thirdparty takes 4s on my machine, and I run it underclocked to get rid of fan noise... ;) | 11:43.31 |
| Robin_Watts | Your SSDs are spoiling you. | 11:43.36 |
| sebras | tor8: that's recompling after having done make nuke and rebooted so the cache is cold... ;) | 11:44.07 |
| tor8 | sebras: so it takes longer to log onto casper than recompile mupdf from scratch... | 11:44.50 |
| sebras | tor8: yes, but my reboot takes 30s as well... | 11:45.16 |
| sebras | wonders if we have motivated Robin to buy an SSD yet... | 11:46.00 |
| Robin_Watts | I have an SSD. It's my primary drive on my Windows box. | 11:46.13 |
| | But I have a 1.5TB spinning disc too. | 11:46.45 |
| tor8 | I have 520gb SSD in total. no spinning discs anywhere in sight :) | 11:47.25 |
| sebras | Robin_Watts: ah, 2TB here and that's where my mupdf repo is located, but the OS is on the SSD. | 11:47.29 |
| | tor8: yeah, yeah, I know I should have bought more SSDs in .tw... | 11:47.42 |
| Robin_Watts | OS + toolsets on SSD. git repos on HD | 11:47.55 |
| miha | thing i got it working | 12:20.03 |
| | think | 12:20.07 |
| tor8 | Robin_Watts: sebras: commit 55a22c3 (clamp page numbers in mupdfclean) doesn't compile... | 12:30.16 |
| sebras | tor8: is that my latest commit? | 12:30.36 |
| tor8 | sebras: yes. | 12:30.47 |
| sebras | tor8: I compiled this last night, how on earth!?... | 12:31.10 |
| tor8 | mupdfinfo.c is also broken in the same commit | 12:32.37 |
| sebras | tor8: in the same way most likely. | 12:33.52 |
| tor8 | sebras: I think you were sleepy. very very sleepy. lots of mistakes in that patch. | 12:34.19 |
| sebras | doc undeclared... ok!? I'm baffled, and a bit embarrassed... ;) | 12:34.23 |
| tor8 | sebras: and you set final = fz_count_pages(doc) where both doc and final are undeclared and unused | 12:34.47 |
| Robin_Watts | I have a fix here. | 12:34.48 |
| | Want me to commit --amend and push -f ? | 12:35.09 |
| miha | paulgardiner Robin_Watts http://www.wolfey.si/tmp/pdflandscape.png | 12:35.18 |
| sebras | Robin_Watts: ok with me! | 12:35.29 |
| tor8 | Robin_Watts: s/fz_page_count(doc)/pdf_page_count(xref)/ and s/final/pagecount/ in the second instance as well? | 12:35.37 |
| Robin_Watts | yes. | 12:35.44 |
| paulgardiner | miha: Oh nice one. That was quick | 12:35.49 |
| tor8 | Robin_Watts: okay, go ahead then | 12:35.50 |
| miha | paulgardiner: it even seems to scroll fine | 12:36.01 |
| sebras | hides... | 12:36.03 |
| tor8 | Robin_Watts: and I'll zap my local fixes | 12:36.11 |
| miha | paulgardiner: diff from mupdf-1.0rc1 http://www.wolfey.si/tmp/ReaderView-landscape.diff | 12:37.09 |
| | probably ugly by your standards :) | 12:38.38 |
| Robin_Watts | just test rebuilding :) | 12:39.31 |
| | sebras: We've all done it, don't sweat it. | 12:39.48 |
| sebras | Robin_Watts: np, I'm just not used to it. ;) | 12:40.17 |
| Robin_Watts | Pushed | 12:40.48 |
| sebras | Robin_Watts: tnx! | 12:42.15 |
| paulgardiner | miha: Nicely unobtrusive. Looks like you may constantly have 9 views. so might hurt memory wise on some devices, but that would be easy to alter. | 12:42.31 |
| miha | well, wanted to see next views at scrolling | 12:43.47 |
| paulgardiner | Yeah, I guess that's still only 3 screen's worth, which is what you need to avoid a scroll uncovering an empty space. | 12:45.17 |
| miha | yes | 12:45.38 |
| | but on other hand, having more views (or just images) is nicer to read. less busy icons, preload | 12:54.02 |
| | might be worth, if there's enough memory on device | 12:54.28 |
| | especially on pdf's with lots of big images | 12:55.01 |
| | i dont see much 'loading' on tablet, but on phone it's all the time | 12:55.41 |
| henrys | the way things look now we should start shopping for a hosting service other than google. | 13:54.46 |
| | damn | 13:55.58 |
| Robin_Watts | henrys: So, requirements? | 13:58.47 |
| | You want spam handling to be done by the domain rather than locally? | 13:59.18 |
| | (I download all my email to thunderbird and do the spam control in that - it seems to do a very good job) | 13:59.41 |
| | but you want to keep all your email archived on the domain, so you can search it from anywhere? | 14:00.26 |
| | That would imply that all the spam handling needs to be done by the domain. | 14:00.37 |
| chrisl | Most places would support IMAP now, I'd reckon | 14:01.13 |
| paulgardiner | Setting up postfix and courier-imap on a virtual server is fairly straight forward. | 14:01.54 |
| chrisl | I have to say, I'm surprised Google are getting away with foisting these terms on business mail hosting | 14:02.47 |
| henrys | well I'll wait for the final feedback from the lawyer but first off we need a service that provided confidentiality protections and non disclosure agreements. | 14:02.49 |
| | your artifex email is spam filtered by google. | 14:03.08 |
| Robin_Watts | yes, but the email I get from artifex is a fraction of the email I get :) | 14:03.33 |
| paulgardiner | How much control do you have over casper. Could set up postfix on that. | 14:03.45 |
| Robin_Watts | I've had the same email address for 15+ years now, so I'm on every spam list known to man. | 14:04.00 |
| | paulgardiner: We have total control, I believe. | 14:04.25 |
| paulgardiner | And no privacy issues? | 14:05.01 |
| Robin_Watts | chrisl: How would imap work with spam? | 14:05.12 |
| paulgardiner | Robin_Watts: I have postfix set up using spam assasin for spam recognition. Spam is taken out before it is stored to the imap folders | 14:06.00 |
| chrisl | Robin_Watts: I don't understand the question...... | 14:06.29 |
| Robin_Watts | If I (or my local copy of thunderbird (or whatever other imap client I am using)) mark something as spam locally, I'd hope for the message being moved to the spam directory. | 14:06.33 |
| | s/directory/folder/ | 14:06.46 |
| | paulgardiner: spam assassin is a long way less than perfect IME. | 14:07.05 |
| chrisl | Oh, right, generally changes you make locally are reflected on the server, and vice versa, so local filtering should work just fine | 14:07.18 |
| Robin_Watts | OK. I didn't know that IMAP supported 'move this message from one folder to another'. | 14:07.37 |
| paulgardiner | Robin_Watts: I don't know how that is done in general, but I have a special email address I send a command to, and then the server tells spamassasin to look at all the messages in my spam folder. | 14:07.46 |
| Robin_Watts | paulgardiner: For training? right. | 14:08.06 |
| paulgardiner | Robin_Watts: Yes training. spam assassin seems to be working very well for me. I now get 1 or 2 spams a day if that. | 14:08.31 |
| Robin_Watts | Presumably when you have spamassassin "take spam out" you move it into a spam folder, rather than delete it ? | 14:08.36 |
| henrys | I really like keeping my email online also so I can access it anywhere without fooling with ssh tunnelling or whatever. I guess I can live without that if necessary. | 14:09.09 |
| paulgardiner | Robin_Watts: I actually reject it at the smtp level before it's even accepted, but there are all sorts of options, putting it in a special folder being one. | 14:09.19 |
| Robin_Watts | henrys: IMAP allows exactly that. | 14:09.23 |
| | Internet Mail Access Protocol? | 14:09.43 |
| | Internet Message Access Protocol :) | 14:10.03 |
| paulgardiner | henrys: yes imap gives you that, and you can have web mail via imp/horde | 14:10.15 |
| chrisl | Robin_Watts: I have all my e-mail done via imap now, and separate them into local and remote directories - the remote directories work pretty seamlessly (some mail clients need you to "subscribe" to folders on the server for "full" integration) | 14:10.20 |
| henrys | well yes but we have to find a service that will hold the mail. I have 5 gigs at google 23% full | 14:10.43 |
| Gigs- | hi | 14:10.55 |
| ghostbot | salut | 14:10.55 |
| Robin_Watts | henrys: We can run it on casper. | 14:10.55 |
| | Or on another EC2 instance. | 14:11.09 |
| Gigs- | mvrhel_: did you get a chance to run the other bug I CCed you on, the dropped background components | 14:11.19 |
| | mvrhel_: also, do you want me to assemble a corpus of the type of prepress PDFs we use for your planar testing? | 14:11.40 |
| chrisl | If we're going to host it ourselves, I'd vote for a separate EC2 instance | 14:11.44 |
| Robin_Watts | or someone like ix will offer IMAP as part of a standard hosting package. | 14:12.02 |
| paulgardiner | Robin_Watts: Yeah, IX are good. | 14:12.15 |
| Robin_Watts | disc space these days shouldn't be an issue | 14:12.19 |
| paulgardiner | Robin_Watts: although not so sure about the privacy issue. IX have been hacked a number of times | 14:12.40 |
| Robin_Watts | paulgardiner: We used ix for downloads for ghostscript. They lasted about 3 months before asking us to move elsewhere due to the volume :) | 14:12.50 |
| paulgardiner | Robin_Watts: Wow. Impressive. :-) | 14:13.11 |
| Robin_Watts | (well, they asked us to upgrade hosting packages, and it was cheaper to move elsewhere - they then refunded us - they were very good about it) | 14:13.24 |
| chrisl | Hmm, GoDaddy only seem to give 1Gb mailboxes by default | 14:13.30 |
| henrys | I can't have my email on any server where others at the company have admin, same for Miles. At least I don't think it is a good idea. There is salary information and senstitive stuff like that. | 14:14.25 |
| Robin_Watts | ah, good point. | 14:14.46 |
| | Each of my mailboxes on ix have 2Gig. | 14:15.44 |
| | but I'm using it in pop mode, not imap. | 14:15.56 |
| | and if that was a problem I'd archive stuff off by year. | 14:16.19 |
| chrisl | It's nice to access to (almost) all your e-mail wherever you are | 14:16.40 |
| Robin_Watts | I'd archive it off to a new mailbox on line. | 14:16.58 |
| | so I'd still have access from anywhere. | 14:17.28 |
| | I believe with ix I can't get into any mailbox without the password. | 14:17.37 |
| | I can delete the mailbox and recreate it, but I can't read the mail therein. | 14:17.53 |
| | (so it would be secure). | 14:17.59 |
| | (and I'm not necessarily pushing ix here, just taking it as an example of what is presumably a standardish service) | 14:18.22 |
| paulgardiner | Is casper safe from non-artifex admin access? | 14:19.18 |
| Robin_Watts | paulgardiner: Yes, AIUI. | 14:19.27 |
| | Basically, we can install any OS on it we like (that understands the hardware). | 14:19.53 |
| | so Marcos picked Ubuntu. | 14:20.01 |
| | So any external user would have the same problem of getting into it as anyone would getting into any unix system. | 14:20.41 |
| paulgardiner | Might someone on site be able to take out one disc of a raid array say and read the disc? Unlikely I know, but... | 14:21.32 |
| Robin_Watts | (well, an external admin could clone the instance and then work on that - so they could read the disc if unencrypted, yes) | 14:21.56 |
| henrys | anyway we'll discuss it when I get the final word from the lawyer this was just a head's up I didn't mean to interrupt work. | 14:22.34 |
| Robin_Watts | paulgardiner: basically, if someone had physical access to your linux server, could they get any data off that without the password? | 14:22.53 |
| paulgardiner | Mine yes, but as you say encryption is a solution. | 14:23.21 |
| Robin_Watts | I would assume that as lots of people are using EC2, someone must have thought this through :) | 14:24.08 |
| henrys | Robin_Watts:the customer is satisfied with GL/2 (with the bad fill) as is so, I'll continue to fool with it but in the background. | 14:24.16 |
| Robin_Watts | henrys: cool. | 14:27.25 |
| tor8 | henrys, Robin_Watts: I'd be happier if we dropped google for artifex email. at least it means I can stop using IE for reading my company email :) | 14:36.45 |
| henrys | why on earth would you use IE to do anything? | 14:37.26 |
| tor8 | henrys: because you can't log on to two gmail accounts at once, so I have to run two browsers | 14:37.43 |
| | and I don't feel like installing firefox | 14:37.49 |
| henrys | what if you run 2 instances/processes of the same browser? | 14:38.32 |
| Robin_Watts | henrys: chrome already does that for you - one process per tab. | 14:38.55 |
| | (plus one master one, IIRC) | 14:39.17 |
| henrys | on a mac open -n will open a whole new app but what robin_watts said also. | 14:39.22 |
| Robin_Watts | Some of googles services now cope with 2 simultenous accounts, but I wouldn't trust it. | 14:39.46 |
| tor8 | henrys: chrome uses the same session cookies across all processes though | 14:39.57 |
| | unless you run it in porn mode, which gives you a new blank session each time | 14:40.35 |
| henrys | tor8:I have 2 also but just forward one to the other. | 14:41.12 |
| tor8 | henrys: I prefer to keep private and company separate. so the whole google apps gmail merger madness has really pissed me off. | 14:41.47 |
| miha | paulgardiner: just got a crash of mupdf... it tried to create Bitmap at PageView setPage when i closed app | 14:44.26 |
| | E/AndroidRuntime( 5380): java.lang.IllegalArgumentException: width and height must be > 0 | 14:45.05 |
| | E/AndroidRuntime( 5380): at android.graphics.Bitmap.nativeCreate(Native Method) | 14:45.09 |
| | E/AndroidRuntime( 5380): at android.graphics.Bitmap.createBitmap(Bitmap.java:468) | 14:45.12 |
| | E/AndroidRuntime( 5380): at com.artifex.mupdf.PageView.setPage(PageView.java:144) | 14:45.15 |
| paulgardiner | miha: thanks for info. any more to that backtrace? | 14:46.28 |
| miha | i think it wanted to render page, just when i hit back and closed Activity | 14:46.56 |
| paulgardiner | I thought all the bitmap creations were in the foreground. That's a bit odd | 14:48.03 |
| miha | well i didnt change that code :p | 14:49.08 |
| paulgardiner | Ah no. The asynctask in MuPDFPageAdapter calls setPage. Thanks. I'll need to think about that one. | 14:49.18 |
| miha | paulgardiner: what about just putting if (mSize.x == 0 || mSize.y==0) return; before it | 14:49.38 |
| | if size is 0, there's no point anyway | 14:49.46 |
| | probably bad habbit to burry bugs deeper :D | 14:50.14 |
| paulgardiner | Yeah, but I think possibly we should be waiting on cancellation of the asynctasks on shutdown. It's an issue I've had in mind, but seen no sign of problems from it until now. | 14:50.58 |
| miha | might be side effect of my multiple page thing :) | 14:52.04 |
| paulgardiner | miha: that may be changing the timing and hence show up the bug, but I don't think it would be the cause. | 14:52.55 |
| miha | it happens when i scroll all the way (3 new pages) and close | 14:52.58 |
| | before pages are loaded | 14:53.38 |
| paulgardiner | That test you suggest is worth a try, but I really should take a closer look sometime. | 14:54.44 |
| miha | ok | 14:55.05 |
| paulgardiner | There's a section of google's documentation on shutting down in the presence of AsyncTask that sort of reads "We don't know how to explain this, but look at this app of ours for an example of how to do it" :-) | 14:57.14 |
| miha | well multithreading is a little unpredictable. i remember one bug when i was learning Java. I had thread to animate rotating some "selected" shape. so it rotated Graphics (or Canvas) by angle, drew and then rotated back. then it was all weird to me cause sometimes whole screen was rotated. then i figured i set rotation off and angle to 0 (unselect) just in betweeen rendering code, so it didnt rotate back by -angle. took me few hours to understand... of c | 15:04.11 |
| | probably offtopic, but it was lesson for me :) | 15:05.05 |
| | if it had happened every time, i'd have understand it faster. so it happened only now and then. | 15:07.18 |
| | sometimes i just try {} catch (Exception e){ e.printStackTrace();} .. just in case | 15:08.58 |
| ray_laptop | Robin_Watts: I saw discussions with naveen about reading PDF from a stream. | 15:38.25 |
| Robin_Watts | ray_laptop: naveen has an encrypted PDF (not standard PDF encryption). | 15:39.32 |
| ray_laptop | the gxclmem.c implements an LRU cache of uncompressed blocks in order to avoid decompressing repeatedly. That same approach would probably work better with PDF's | 15:39.35 |
| mvrhel | good morning | 15:40.24 |
| | Gigs: I have not had a chance to look at the bug yet | 15:42.39 |
| | if you can do testing with your huge pile of files with the new tiffsep changes, that would be great! | 15:43.03 |
| Gigs- | mvrhel: there will be byte differences right? | 15:48.10 |
| | mvrhel: I mean I can't just do something simple like checksums for regressions? | 15:48.20 |
| mvrhel | Gigs_: that is correct | 15:51.23 |
| | I did a threshold compare | 15:51.29 |
| | gradients have a diff that will show up | 15:51.40 |
| | in particular | 15:51.53 |
| Gigs- | did you use a custom tool for that? | 15:52.37 |
| mvrhel | We have a tool that is in the repository | 15:52.56 |
| | bmpcmp | 15:53.00 |
| | I ran bmpcmp -t 5 | 15:53.11 |
| | let me find it for you. hold on | 15:53.22 |
| Robin_Watts | Gigs: There is a script in the repo: gs/toolbin/htmldiff.pl that may help. | 15:53.36 |
| mvrhel | oh even better | 15:53.48 |
| Robin_Watts | You feed it a list of filenames, and it runs them through 2 versions of gs/pcl/xps etc and bmpcmps the output. | 15:54.19 |
| | bmpcmp.c is gs/toolbin/bmpcmp.c | 15:54.29 |
| Gigs- | ok, I'll look at those, thanks | 15:54.48 |
| ray_laptop | Robin_Watts: and bmpcmp understands psdcmyk files now, right ? | 15:54.57 |
| Robin_Watts | htmldiff.pl is a tool I used to use, but we now have it all automated in our cluster, so it may have bitrotted a bit. | 15:54.59 |
| | ray_laptop: It does. | 15:55.05 |
| mvrhel | thanks Robin_Watts for jumping in | 15:55.16 |
| Robin_Watts | np. | 15:55.21 |
| ray_laptop | because Gigs is primarily a spot color user (tiffsep) | 15:55.24 |
| Gigs- | mvrhel: planer is checked in to head now right | 15:55.28 |
| | ar | 15:55.31 |
| mvrhel | Gigs_: yes it is | 15:55.43 |
| Gigs- | k | 15:55.47 |
| ray_laptop | mvrhel: I haven't done any performance comparisons for spot color + transparency -- have you ? | 15:56.31 |
| mvrhel | ray_laptop: no I have not | 15:56.48 |
| Gigs- | I'll instrument for time too | 15:56.58 |
| ray_laptop | Gigs: if you are going to run some comparisons against 9.05, then collecting performance would be of interest to us | 15:57.11 |
| Gigs- | k | 15:57.15 |
| ray_laptop | Gigs: thanks (I typed too slowly) | 15:57.23 |
| | Gigs: but if any files are slower, we don't want to know about it ;-) | 15:58.07 |
| Gigs- | time for a CUDA fork | 15:58.39 |
| | man that would be blazing | 15:58.44 |
| | most of what you do could be reduced to vector ops right? | 15:58.55 |
| Robin_Watts | Gigs: Not so much. | 15:59.31 |
| Gigs- | aw | 15:59.46 |
| Robin_Watts | For most files text and images pervade. | 15:59.47 |
| | text tends to end up as cached bitmaps. | 16:00.07 |
| ray_laptop | Gigs: for the clist rendering CUDA could be used for per-band rendering (as with NumRenderingThreads) | 16:00.08 |
| Robin_Watts | using texture stuff is possible, I guess, but it'd be hard (I fear) to get an exact match to our output now every time. | 16:01.06 |
| ray_laptop | the pdf14 'blend' action that ends a transparency group is suitable for SIMD | 16:01.15 |
| | since it operates on the entire buffer (withing the bbox) of a stacked transparency layer, it is fairly 'expensive' currently | 16:02.32 |
| Robin_Watts | ray_laptop: Yeah. | 16:04.31 |
| mvrhel | there is the one group that did this with ghostscript for both transparency and for color look ups | 16:05.33 |
| Robin_Watts | mvrhel: People have SIMD'd this before? | 16:06.02 |
| ray_laptop | although it may be better to have 'chunky' rather than planar data for best SIMD optimization during the blending -- we'd have to think about that | 16:06.05 |
| mvrhel | Robin_Watts: no used GPUs | 16:06.20 |
| ray_laptop | mvrhel: transparency ? who ? | 16:06.27 |
| | I know the HP group used GPU | 16:06.58 |
| mvrhel | yes | 16:07.04 |
| ray_laptop | but I thought they were mostly focused on simple pages of text and trying to get fast performance | 16:07.40 |
| | and were partly frustrated by the lack of bitmap id's post clist | 16:08.14 |
| | which my changing the way tiles are stored in the clist will fix | 16:08.47 |
| mvrhel | I spent the day there with them going over all the transparency stuff. then when they presented their paper and the results they mentioned that they would like to do this with a real RIP. which made me a bit unhappy to hear them say something like that. and I told him after the talk | 16:09.00 |
| | anyway, that is water over or under the bridge | 16:10.10 |
| ray_laptop | mvrhel: did you get a copy of the paper ? (I don't recall any mention of transparency in the one I saw) | 16:10.14 |
| mvrhel | ray_laptop: I will look around here. I should have it on a DVD | 16:11.01 |
| ray_laptop | mvrhel: of course, cust 532 is still interested in using their GPU ASIC to render transparency buffers and do (at least the simple) blending | 16:11.08 |
| mvrhel | they did not have the papers at the time of the conference | 16:11.15 |
| | DVD was not ready | 16:11.19 |
| kens | Time to go, night all | 16:11.37 |
| mvrhel | bye kens | 16:11.43 |
| ray_laptop | mvrhel: a typical situation for conferences -- last minute "publication" | 16:11.46 |
| | bye, kens | 16:11.50 |
| mvrhel | yes | 16:11.53 |
| ray_laptop | mvrhel: is any of their code available / useful to us, do you think ? | 16:12.54 |
| mvrhel | ray_laptop: I have no idea. I tried to keep in contact with them but have not heard anything in a while. Maybe I should ping them to see what is going on | 16:15.53 |
| | HP labs is a bit lost these days I believe | 16:16.36 |
| | compared to what they were 20 years ago | 16:16.46 |
| | there is not much left | 16:17.19 |
| | of course that is true at most of the old large imaging and printing companies | 16:17.44 |
| | research has been majorly cut | 16:18.06 |
| ray_laptop | I think my laptop crashing may be a temp issue. The "absolute max" for the Core i5m is 99 C and running "CPUID Hardware Monitor" I'm seeing 98 and even 99 as a max sometimes :-( | 16:28.25 |
| mvrhel | oh that could be | 16:28.37 |
| Robin_Watts | Interesting. | 16:28.43 |
| | http://www.almico.com/speedfan.php | 16:28.59 |
| | Get that to warn you when the temp goes over a certain value. | 16:29.22 |
| henrys | ray_laptop:I've never seen temps that high on a system. | 16:29.23 |
| ray_laptop | the 'airflow' temps are 45 C, so that should be enough if the fan and heatsink are OK. | 16:29.27 |
| | henrys: me either | 16:29.33 |
| Robin_Watts | then if it crashes without you being warned, you'll know it's a red herring. | 16:29.50 |
| ray_laptop | that stoopid 7-zip "Download" ad is sure annoying -- took a while to find a link to actually download speedfan | 16:34.28 |
| | is marcos still out of town ? several of the cluster nodes are still down. | 16:35.46 |
| Robin_Watts | i7 and x6 are on the same connection as some of the others at marcosw's house. | 16:38.35 |
| ray_laptop | Robin_Watts: yes, and I think he still has 'miles' because it was unstable for a while and he was working on it at home. | 16:39.42 |
| | mvrhel: RU there ? | 16:45.44 |
| | mvrhel: what text_knockout mode should I be concerned about (if any) w.r.t. painting at the 'page level' for transparency ? Only if alpha is < 1.0 ?? | 16:47.05 |
| mvrhel | ray_laptop: good question | 16:52.01 |
| | let me go read about text knockout | 16:52.16 |
| | yes only if alpha < 1.0 | 16:53.06 |
| | otherwise it would have no real difference | 16:53.15 |
| ray_laptop | mvrhel: great. Thanks | 16:54.57 |
| mvrhel | henrys: I should have this delayed ICC stuff wrapped up today or tomorrow at the latest | 18:13.48 |
| | then I will work on the halftone planar image clist image fix | 18:14.02 |
| | halftone/planar/image/clist fix | 18:14.23 |
| | we should think about getting someone going on PDF/VT | 18:15.07 |
| henrys | yeah it is on my queue to bring that up... Before you joined we considered and knocked down PDF/VT at least 3 times, I don't expect we'll really do it, but since we have a serious request we should discuss it. | 18:24.21 |
| Robin_Watts | PDF/VT = print the same thing multiple times, with variable data ? | 18:27.06 |
| | "Dear Mr Smith, here is your personalised catalogue" ? | 18:27.29 |
| henrys | yes | 18:27.35 |
| Robin_Watts | Does a single PDF/VT document contain both the static and the variable data? | 18:30.18 |
| | or do we get 2 (or more) PDFs that we have to merge ? | 18:31.09 |
| | Or does no one know cos no one has read the spec yet ? | 18:31.57 |
| | ah, a main document + XML | 18:32.54 |
| | Has anyone purchased the spec? | 18:33.35 |
| | 140 swiss franks from ISO it seems. | 18:33.43 |
| saper | isn't it for free somewhere? | 18:34.04 |
| Robin_Watts | saper: The PDF spec is. I am not aware that the PDF/VT spec is. | 18:34.45 |
| saper | yeah you're right | 18:36.19 |
| Robin_Watts | http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=46428 | 18:36.48 |
| henrys | harlequin supports it - we may want to hear what chrisl and kens have to say about that. | 18:37.05 |
| Robin_Watts | If anyone in the company has a copy of the spec, I wouldn't mind reading it. | 18:37.10 |
| henrys | tor8:have you studied pdf/vt for mupdf at all? | 18:38.03 |
| tor8 | henrys: I remember discussing it briefly years ago with Raph. can't say I'm up to date with it. | 18:40.09 |
| Robin_Watts | The abstract suggests that graphical content can be supplied externally; which either means that it's going to use external XObjects, or it's going to require layout capabilities for text, I fear. | 18:42.45 |
| mvrhel | the folks in Utah were also interested in this | 18:42.59 |
| henrys | I do see looking on the web it has gotten a lot more important since we last considered it. | 18:43.56 |
| mvrhel | There has been quite a bit of talk about it at conferences and even at the ICC | 18:44.13 |
| | at least for those in the print industry | 18:44.24 |
| | the company for one guy on the ICC prints lots of catalogs | 18:44.54 |
| | with a personal touch I believe | 18:45.06 |
| | I suspect we can come up with a clever approach to support this | 18:45.41 |
| Robin_Watts | When this was mentioned before at the meeting, it got rolled into the idea of savepage. | 18:46.25 |
| | somehow rendering part of the document once, and then reusing that to 'add' new data on top. | 18:46.38 |
| | From what little I've read now, I wonder whether that approach will work. | 18:46.53 |
| | I suspect in its full generality it allows for a lot more than that. | 18:47.10 |
| henrys | yes it is similar to savepage and pcl has it sort of built in with a macro construct but doing it be the spec I imagine is going to have other requirements. | 18:47.36 |
| | s/be/by | 18:48.08 |
| Robin_Watts | I think savepage and this may be separable problems. | 18:48.52 |
| | (i.e. savepage may offer an optimisation, but it's not necessary for a first implementation) | 18:49.26 |
| | savepage may not even be the hardest bit. | 18:49.48 |
| henrys | wiki says "built on pdf/x-4" I don't know how much of that we're missing to do the implementation. | 18:50.48 |
| mvrhel | I think at least getting the spec and having someone do a read would be good | 18:51.27 |
| Robin_Watts | PDF-X/4 is a subset of PDF, rather than extensions to PDF, which is a relief. | 18:52.14 |
| mvrhel | likely the clist would be handy for all of this as ray had mentioned | 18:52.25 |
| Robin_Watts | I think the clist *may* be one route to save rerendering sections of the file for every variant we need to produce. | 18:52.59 |
| henrys | alexcher:are you about, did you want to weigh in on this? | 18:53.19 |
| Robin_Watts | but just rerendering it every time wouldn't be a killer for a first version. | 18:53.42 |
| | I could forsee uses of this where savepage wouldn't save us anything. | 18:54.25 |
| chrisl | PDF/VT is just a PDF - re-rendering each page is what we'd do now with a PDF/VT file | 18:59.11 |
| Robin_Watts | It's not a PDF + an XML file ? | 18:59.37 |
| mvrhel | this is why we need to have the spce | 18:59.50 |
| | sped | 18:59.52 |
| | spec | 18:59.54 |
| | cntat type | 18:59.58 |
| Robin_Watts | mvrhel: Yes. | 19:00.14 |
| | chrisl: Do you have a copy of the spec? | 19:00.25 |
| chrisl | No, I don't have the spec :-( | 19:00.38 |
| | But generally, PDF/VT adds tags to XObjects to help consumers efficiently cache the results. | 19:01.32 |
| henrys | chrisl:do you think kens might have it. If not I'll buy it now. | 19:03.46 |
| chrisl | henrys: I don't think so - if kens has seen it, it would be a copy owned by Global | 19:04.15 |
| | So, for a trivial case, you might have a multipage letter with a vector logo in an XObject on each page, but the letter body differs. PDF/VT would add a tag to the logo XObject to say it was reused on so many pages, and so would be valuable to cache. | 19:04.19 |
| | I found this quite easy, as a basic intro: http://www.pdflib.com/fileadmin/pdflib/pdf/whitepaper/Whitepaper-Technical-Introduction-to-PDFVT.pdf | 19:06.07 |
| Robin_Watts | chrisl: What you've just described there has no VDP stuff in it. | 19:06.57 |
| | And indeed the first paragraph of the link you give suggests that there should be. | 19:07.17 |
| | I was imagining that PDFVT should cope with having "Dear $customer, we haven't heard from you since $date. Please come back to us. We've got a special offer on $appropriateOffer this week." | 19:08.28 |
| chrisl | Robin_Watts: That's not my understanding, no, because that wouldn't be a valid PDF | 19:09.04 |
| Robin_Watts | Of the link you gave, see the "Variable Document Printing" section. | 19:09.32 |
| | How would you achieve what you suggest with that? | 19:09.43 |
| | Or are you saying that the PDF is created by a special creator? | 19:10.15 |
| | sorry, that was craply phrased, let me try again. | 19:10.35 |
| henrys | why wouldn't the letter body be the static xobject, why is it different than the logo? | 19:10.45 |
| Robin_Watts | Or are you saying that the database merging smarts are all done in the PDF creator ? | 19:11.02 |
| chrisl | AIUI the database stuff is handled by the creator, yes | 19:11.21 |
| mvrhel | ok. delayed of the ICC profile handles at least on the icc manager is done | 19:11.51 |
| | I suppose I need to also do it for the device profile | 19:11.58 |
| | that will likely also be a can of worms | 19:12.11 |
| Robin_Watts | So my example would have "Dear, we haven't heard from you since . Please come back to us. We've got a special offer on this week." as the Xobject, and then each page would just call the Xobject and fill in the blanks. | 19:12.17 |
| chrisl | henrys: in a normal PDF, there is no information about how often an XObject is used, so it's impossible to make a sane judgement on whether, or how to cache it for | 19:12.35 |
| mvrhel | Robin_Watts: I dont think that is the way it is done | 19:12.36 |
| | I think there is a header | 19:12.41 |
| | and a body | 19:12.43 |
| Robin_Watts | mvrhel: Within the PDF? | 19:12.53 |
| mvrhel | and the header is common for example on all the output | 19:12.54 |
| | but I have not read the spec | 19:13.02 |
| henrys | well let's get the spec. | 19:13.09 |
| mvrhel | so I really am going only from discussions with people | 19:13.12 |
| henrys | then argue. | 19:13.13 |
| mvrhel | exactly | 19:13.23 |
| henrys | brb I'll get the spec after lunch. | 19:15.07 |
| chrisl | "At the core PDF/VT documents are plain PDF files plus certain additional features. However, the PDF/ VT-specific additions do not affect page rendering: the pages of a PDF/VT document can be viewed | 19:15.13 |
| | with a plain PDF viewer or rendered with a PDF capable RIP which may not directly support PDF/VT, e.g. Acrobat or a third-party PDF viewer or RIP." | 19:15.15 |
| Robin_Watts | > Despite its name, PDF/VT does not include any variables: all pages contain or reference the final | 19:15.31 |
| | content, without any headroom for dynamic content creation or document formatting. Similarly, the | 19:15.33 |
| | claim that Acrobat displays PDF/VT documents except for the variable data parts is wrong. | 19:15.35 |
| | chrisl: OK, I understand now. So it's purely a performance thing then. | 19:15.58 |
| | If we make use of the extra tags, we can probably finish the job faster. | 19:16.16 |
| chrisl | At its core yes. Also, the tags can tell you to cache XObject results between jobs, as well as between pages - which is probably where Ray's use of saved page clist would arise | 19:17.44 |
| | I had considered using something like the Type 1 pattern cache, maybe with the option of loading the cache tile from disk. | 19:18.32 |
| Robin_Watts | I was pondering about using the PDF14 device. | 19:19.00 |
| | if an Xobjects contents can be encapsulated into a group, we may be able to play that group back several times. | 19:19.43 |
| chrisl | Well, that's exactly what a Type 1 pattern is | 19:20.01 |
| Robin_Watts | Is it? OK then. I agree! :) | 19:20.20 |
| chrisl | ;-) It also has the advantage of already coping with both bitmap and clist tiles | 19:20.41 |
| Robin_Watts | For transparency is a bitmap enough? | 19:21.04 |
| | different blending modes etc? | 19:21.17 |
| | I guess it must be. | 19:21.42 |
| chrisl | Ah, well, there's also stuff about the interpreter needs to decide *whether* to cache, we can't rely on the tags alone | 19:21.51 |
| | I'm not convinced (although it's not my area!) that we can cache when transparency is involved - in the general case. | 19:22.58 |
| mvrhel | we cache tiles now with transparency | 19:24.24 |
| | both clist and non clist | 19:24.55 |
| chrisl | Okay, there you go. | 19:25.29 |
| | I reckon a lot of the purpose behind PDF/VT is down to how bad PDF Form XObjects are | 19:27.10 |
| mvrhel | lunch time | 19:27.38 |
| chrisl | Robin_Watts: when I first looked into it (briefly!) the focus was entirely VT1, which I think wouldn't be *too* hard to do if we can leverage the pattern cache | 19:29.29 |
| mvrhel | I wonder if I should close 692885 or wait for the customer to reply | 19:30.06 |
| chrisl | How long have you waited? | 19:30.23 |
| mvrhel | hehe about 10 minutes | 19:30.31 |
| Robin_Watts | It certainly sounds more feasible now you've explained it. And the fact we shouldn't require savepage is a relief. | 19:30.49 |
| | mvrhel: Be reasonable. Give 'em at least 15. | 19:31.03 |
| chrisl | mvrhel: good enough - I'd close it ;-) | 19:31.13 |
| mvrhel | :) | 19:31.13 |
| chrisl | Robin_Watts: like I say, when you start getting into external resources, and resources which straddle jobs, things probably get a lot more unpleasant..... | 19:32.58 |
| Robin_Watts | yes, but we don't need to support any of that to say we support PDF/VT, right ? | 19:33.32 |
| chrisl | Well, strictly speaking we "support" PDF/VT files now, but I don't think we need that extra complexity for PDF/VT1 support. | 19:34.41 |
| | Again AIUI, VT1 files are self contained | 19:35.09 |
| Robin_Watts | In order for us to claim that "we have new support for PDF/VT files", we'd presumably have to be able to do some caching in at least some cases :) | 19:35.36 |
| chrisl | Sure, yes. Like I say, VT1 looks sane, would let us advertised PDF/VT support and I suspect would cover most use cases - but don't quote me on that! | 19:37.08 |
| | Actually, one thing that occurs to me is whether the few inquiries about external file references we've had might relate to handling VT2 type jobs | 19:38.58 |
| | I consider it a bad sign that I've just spent nearly half an hour trying to work out why my code changes are working as I intended, because I got the test results the wrong way round :-( | 19:43.12 |
| | So I think it's time I gave up for the evening....... | 19:43.31 |
| rayjj__ | Ref XObjects (Forms with external PDF) shouldn't be hard for OS based files. For URI (http:// or other socket) it needs substantial development | 19:53.14 |
| | but you guys are right, that if we treat the 'base' image of the page as a pattern, then MaxPatternBitmap can be used to decide if it is a bitmap or clist, and the VT content on top should be easy. | 19:55.35 |
| | and we won't need 'saved pages' | 19:56.09 |
| Robin_Watts | Does the pattern cache live across pages ? | 19:56.44 |
| rayjj__ | Robin_Watts: it is a member of an imager state, and if we are rendering with a clist, then it isn't "retained" | 20:02.07 |
| | since the clist rendering has its own imager state (actually each rendering thread does) | 20:02.39 |
| | but if rendering to page mode, there the pattern cache doesn't appear to ever be freed. | 20:03.26 |
| mvrhel | bbiaw | 20:03.36 |
| rayjj__ | but having a pattern element or cache for VT that is part of the lib_ctx isn't muchly ugly | 20:05.43 |
| | a cache is probably better because it allows for multiple 'patterns' (forms) for multiple pages of a VT doc that has different underlay for each page in a set | 20:08.04 |
| | darn! I misplaced my invitation to George's party tonight ;-) | 20:13.57 |
| | I wonder if Miles is going ? | 20:14.27 |
| henrys | the spec is in my home directory on casper | 20:30.44 |
| | George? | 20:31.04 |
| rayjj__ | henrys: George Clooney -- having a party here for Pres. Obama | 20:32.21 |
| henrys | ah | 20:33.02 |
| | when he was here (Colorado University) I was out for a run and looked up and saw Marine 1 flying him to the university. | 20:34.37 |
| rayjj__ | henrys: which file (and what directory) ? | 20:35.08 |
| henrys | /home/henrys/ISO-IEC* | 20:37.30 |
| | oh wait that doesn't look like the right file | 20:39.18 |
| | I must have copied it wrong. | 20:39.28 |
| rayjj__ | henrys: thanks -- that looked like an example. | 20:40.28 |
| henrys | /home/henrys/ISO_16612-2* | 20:40.30 |
| rayjj__ | hates the wording and construction of ISO specs :-( | 21:13.59 |
| | at least this one is better than the 32000 spec. They took a perfectly good spec from Adobe and butchered it | 21:14.40 |
| | Forward 1 day (to 2012/05/11)>>> | |
IRC Logs
