| | <<<Back 1 day (to 2011/10/10) | 2011/10/11 |
| kammerer|2 | Hello, is anybody now is there any optimization for arm devices in mupdf? | 07:28.33 |
| kens | You're really a bit early today for that question. If you come back in about 2 hours and ask then Robin Watts may be able to answer better. | 07:29.11 |
| | I believe the answer is 'no' but I'm uncertain | 07:29.23 |
| kammerer|2 | thanks | 07:34.17 |
| chrisl_x100 | stoopid broadband is still not working..... :-( | 07:55.44 |
| kens | Wondered where you were.... | 07:56.02 |
| chrisl_x100 | I'm tethered to my phone, but I don't want to have it like that all day. | 07:56.31 |
| | "Luckily" I have a bug to work on...... | 07:57.21 |
| kens | Well the cluster is playing games, so don't get to enthusiastic... | 07:57.43 |
| chrisl_x100 | This is going to take me a while - rotated glyphs on a non-square resolution, but somehow different from the last time I had a problem with that. | 07:59.23 |
| kens | You would thnk we'd fixed all those by now... | 07:59.49 |
| chrisl_x100 | Non-square resolutions are relatively rare | 08:00.11 |
| kens | Yes, but htey're bvasically the same as weird CTMs and we've had plenty of those | 08:00.42 |
| chrisl_x100 | Unfortunately, they're not, in the way we end up handling it. | 08:01.35 |
| kens | Well, cluster looks well and truly borked... | 08:25.09 |
| sebras | kens: there are some ARM optimizations in draw/arch_arm.c that are included in the android port. | 08:43.23 |
| kens | I did say I didn't really know ;-) | 08:48.04 |
| sebras | kens: mmm, but now I'm getting increasingly anxious about that this code may not be included, but could be bitrotted... | 08:48.57 |
| kens | Well, you would know better than me ;-) | 08:50.40 |
| | hopefully kammerer will come back and ask again | 08:50.58 |
| | If the cluster barfs once more I'll try killing the job. | 08:51.26 |
| | And running a test of master. | 08:51.39 |
| sebras | kens: aha, back in july tor removed those optimizations. I guess no one has compiled for android since or they would get an error message at the very least. I guess I need to setup a proper compilation environment... | 08:56.39 |
| AlecTaylor | hi | 09:17.27 |
| | What do I use for parsing & analysing PDFs in C++? - GhostScript? | 09:17.46 |
| kens | Depends what your requirements are. Probably MuPDF, but it depends | 09:18.06 |
| Robin_Watts | AlecTaylor: Ghostscript uses a lot of postscript code to handle PDF. | 09:18.41 |
| | Which makes it not ideal for easy hacking. | 09:18.51 |
| | mupdf is all in C and is therefore much more accessible. | 09:19.01 |
| kens | Well, that depends on whether you speak PostScript ;-) | 09:19.07 |
| AlecTaylor | ;) | 09:19.20 |
| | Thanks, asking on the mailing-list now | 09:21.26 |
| | (feel more like a newsgroup question rather than an IRC one) | 09:21.39 |
| kens | We are teh same people, either way ;-) | 09:21.50 |
| Robin_Watts | sebras: Android build works just fine, thanks. | 09:21.50 |
| kens | Robin_Watts : did you see the question about ARM optimisations ? | 09:22.13 |
| Robin_Watts | I did. | 09:22.26 |
| | There were ARM opts, and then there weren't. | 09:22.35 |
| kens | OK, so if he comes back you can answer ? | 09:22.43 |
| Robin_Watts | Sure. | 09:23.06 |
| kens | Hmmm /unregistered in --show--, that's an odd error. | 09:23.15 |
| AlecTaylor | kk, posted in comp.lang.postscript | 09:23.44 |
| | Reply there :] | 09:23.47 |
| kens | Oh, tht's just me then. comp.text.pdf might be better for PDF questions. | 09:24.12 |
| AlecTaylor | hmm | 09:24.22 |
| | I'll double-post, brb | 09:24.25 |
| | arghh, damn, can't find what I sent... guess I gotta wait for it to appear on the initial list | 09:27.06 |
| kens | Can't see it there yet. | 09:27.16 |
| AlecTaylor | yeah, so just need to wait... does someone need to approve it first? | 09:28.34 |
| kens | Not on UseNet :-) | 09:28.47 |
| AlecTaylor | :P | 09:29.06 |
| AlecTaylor | connects through Google Groups | 09:29.13 |
| kens | Ick. | 09:29.25 |
| | Might be a long wait | 09:29.39 |
| AlecTaylor | pfft, rewrote it. Sent | 09:32.27 |
| | So it will appear momentarily on both lists | 09:34.27 |
| kens | 'momentarily' :-) | 09:34.51 |
| AlecTaylor | o.O | 09:35.51 |
| | Look for the post entitled: "Comparing geometric layout information across "pages"" | 09:36.22 |
| | Contents: http://pastebin.com/ejDSEe07 | 09:41.41 |
| kens | Seems like you can use either the Ghostscritp txtwrite devce or MuPDF 'pdfextract' as a starting point for what you want. (I think its pdfextract). | 09:42.41 |
| | Depends what you mean by geometrical layout. | 09:43.02 |
| | Are you only interested in text ? | 09:43.09 |
| AlecTaylor | Yes | 09:43.58 |
| | But I'm not interested in being able to understand the text | 09:44.08 |
| kens | Then either of those will return positional/sizing information. | 09:44.10 |
| | They both also return the text , but you can throw that away easily enough | 09:44.32 |
| | MuPDF is more mature in this area | 09:44.45 |
| AlecTaylor | has been experimenting with the PoDoFo and pdfHummus | 09:44.48 |
| AlecTaylor | has been experimenting with the PoDoFo and pdfHummus C++ libraries | 09:45.06 |
| kens | I've seen them mentioned, I've never used them | 09:45.10 |
| AlecTaylor | So is there some sample MuPDF stuff for grabbing this information? | 09:45.31 |
| kens | Yes, I think pdfextract does it. | 09:45.45 |
| AlecTaylor | Let say, for starters, I want to grab each page and put it into a Linked-List. How would I do this? | 09:45.48 |
| | (so each node in the list is a page) | 09:45.59 |
| kens | You get the text back from MuPDF for each page you 'render', then you put that in a linked list of your own devising. | 09:46.26 |
| | Note that the text may not be in any order you would expect, and some of what you think is text amy be images.... | 09:46.47 |
| | And vice versa.... | 09:47.00 |
| Robin_Watts | Unless pdfextract has changed recently, I thought it just extracted resources from a pdf (images/fonts etc) | 09:47.55 |
| kens | Yes, I just opened it, wrong app..... | 09:48.06 |
| | So tell me, which is the app that extracts text ? | 09:48.39 |
| Robin_Watts | Probably pdfdraw with -t or -x or something | 09:49.11 |
| AlecTaylor | notes it finally posted to the newsgroup | 09:49.11 |
| kens | Ah, pdfdraw is hte one you want I think. | 09:49.22 |
| AlecTaylor | "comp.text.pdf" | 09:49.26 |
| | pdfdraw? | 09:49.37 |
| kens | One of the MuPDF smaple applications | 09:50.06 |
| AlecTaylor | downloads MuPDF | 09:50.31 |
| | Should I grab the git? | 09:50.51 |
| kens | pdfdraw -t extracts text and writes to file, -tt writes in a pseudo-XML format | 09:50.55 |
| AlecTaylor | Maybe the XML will be better | 09:51.41 |
| AlecTaylor | awaits the completing of the git clone on his Windows 8 x64 machine | 09:52.23 |
| kens | Have a look at the output, and then maybe you can talk to someone like Tor who actually know something instead of an ignoramus like me ;-) | 09:52.24 |
| Robin_Watts | So you're lookng for some way to magically decide which bits of the page are headers and footers? Good luck with that. Let us know if you come up with a good algorithm. | 09:52.24 |
| AlecTaylor | Robin_Watts: I already have a good algorithm :P | 09:52.41 |
| AlecTaylor | will publish it, and give you the code for the MuPDF project | 09:52.57 |
| Robin_Watts | AlecTaylor: Ah, cool! | 09:53.03 |
| | Very cool :) | 09:53.06 |
| AlecTaylor | (but wait for me to polish it!) | 09:53.13 |
| kens | So, lets see if the cluster problem was the cluster, or me.... | 09:56.36 |
| AlecTaylor | Robin_Watts: Using this as reference: http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=620669 | 09:57.50 |
| AlecTaylor | has git clone 55% of the objects | 09:58.22 |
| kammerer | hello, is anybody know is there any optimizations for arm in mupdf? | 10:13.51 |
| kens | Robin_Watts: wakey wakey | 10:14.09 |
| | Also tor8 you forgotten to sign in to IRC ? If you are reading this in the logs again.... | 10:17.04 |
| aleray | hi, is it possible to run gs as a deamon so every time I change a file it rerenders it ? | 10:25.15 |
| kens | Not simply | 10:25.27 |
| ab5tract | :/ | 10:26.50 |
| AlecTaylor | kammerer: Yes, there are | 10:27.19 |
| | Check mailing-list | 10:27.27 |
| | (I think that's where I heard them discussed) | 10:27.35 |
| kens | Or cehck the logs of the earlier discussion: | 10:30.18 |
| | http://ghostscript.com/irclogs/current.html | 10:30.18 |
| AlecTaylor | Is that where I saw it?! | 10:31.15 |
| kens | OK so it was my code that broke teh cluster :-( | 10:35.45 |
| aleray | kens, thxs | 10:46.17 |
| kammerer | i found just ARCH_ARM flag but it seems it's doen't work at mupdf 0.9 | 10:58.26 |
| Robin_Watts | kammerer: back. sorry, was runnng. | 11:14.35 |
| | I did some ARM optimisations for mupdf a while ago, and then tor8 updated the renderer to change the APIs, so they are no use. | 11:15.09 |
| | I can redo them easily enough if it's important. | 11:15.22 |
| kammerer | it would be nice - i try to use mupdf to render pdf in nook classic - on pdf pages with text it works very fast, but if pdf page contain's scanned image - it render it very slow | 11:18.13 |
| Robin_Watts | kammerer: It would be interesting to know what bit of mupdf is slow. | 11:18.44 |
| | It's quite possible that that is the image interpolation that's slow. | 11:19.01 |
| | I fear the time may well be spent in fz_scale_pixmap. | 11:22.00 |
| | The cores in there could easily be optimised in ARM, but I'd like evidence that that was the place before I spent the time doing it. | 11:22.46 |
| kammerer | is it be enough to insert traces before and after some function call? | 11:23.36 |
| Robin_Watts | Traces within fz_scale_pixmap would be smartest - it's called from several places. | 11:24.06 |
| | Or get a 'frames per second' timing, and then get another one where fz_scale_pixmap scales each pixmap twice. That would enable you to calculate what proportion of time was spent in fz_scale_pixmap :) | 11:24.59 |
| kammerer | ok i will made investigation | 11:27.44 |
| | thank you | 11:27.48 |
| Robin_Watts | kammerer: Thank you - any pointers to 'hotspots' on devices would be appreciated. | 11:39.05 |
| | Gah. *Still* plank vs pamcmyk4 differences :( | 12:10.46 |
| AlecTaylor | Where does MuPDF want me to extract the "thirdparty" folder? | 12:33.20 |
| Robin_Watts | Into the toplevel directory. | 12:34.10 |
| | (i.e. at the same level as pdf, draw, build, generated, fitz etc) | 12:34.22 |
| AlecTaylor | Thanks, done | 12:35.07 |
| | FYI: Here's the conversion log (sln converted to VS2010): http://pastebin.com/FRJ01rku | 12:36.31 |
| AlecTaylor | is impressed. He got no build errors :d | 12:38.16 |
| AlecTaylor | is impressed. He got no build errors :D | 12:38.18 |
| Robin_Watts | AlecTaylor: We deliberately supply it as VS2005, cos it's compatible with everything after it. | 12:38.52 |
| | (i.e. maximally accessible) | 12:39.03 |
| AlecTaylor | has seen a few projects which include multiple .sln... and a few more which include CMake stuff | 12:42.08 |
| | Hmm, so MuPDF is C not C++ | 12:43.47 |
| | All well, 'bout time I learned C :P | 12:43.54 |
| Robin_Watts | C is like C++, but with the tumors removed :) | 12:44.56 |
| AlecTaylor | But I love my templates and multiple inheritance! | 12:45.55 |
| AlecTaylor | is a C++ fanboy | 12:47.59 |
| Robin_Watts | C is a huge improvement on all it's successors. | 12:48.51 |
| AlecTaylor | xD | 12:49.24 |
| | C99? | 12:49.26 |
| | I rather like the idea of public and private variables, and member operators &etc | 12:49.56 |
| | Time Elapsed 00:00:11.08 | 12:50.20 |
| | ========== Build: 16 succeeded, 0 failed, 0 up-to-date, 0 skipped ========== | 12:50.20 |
| | I'm going to be analysing multiple PDFs. I'm thinking to extend from the currently libraries, rather than running the pdfdraw.exe executable separately. | 12:54.38 |
| kens | Yes, I would say so | 12:55.28 |
| | these are mostly demos | 12:55.33 |
| Robin_Watts | mupdf is designed to be used as a lib. the exes are just small wrappers around it. | 12:55.43 |
| AlecTaylor | Woh | 12:56.39 |
| | ewww | 12:56.41 |
| | Who wrote pdfdraw.c? | 12:56.48 |
| kens | Tor I expect | 12:56.57 |
| AlecTaylor | Can I fork it + fix it then give it to someone for review->commit? | 12:57.19 |
| AlecTaylor | notices a real lack of functions | 12:58.22 |
| Robin_Watts | AlecTaylor: There is a git repo. | 12:58.52 |
| | Clone that, then send us patches, sure. | 12:59.04 |
| AlecTaylor | Great, should be done within an hour | 13:02.55 |
| | Stay in touch! | 13:03.01 |
| AlecTaylor|Codin | Hmm, we can have anonymous struct's? | 13:12.40 |
| Robin_Watts | I'm not aware of them being outlawed. | 13:13.00 |
| AlecTaylor|Codin | -_- | 13:14.06 |
| tor8 | I don't think anonymous structs work on all compilers that we use though | 14:03.14 |
| | oh, wait, maybe I'm thinking of something other than what you mean | 14:07.13 |
| AlecTaylor|Codin | RAGE | 14:29.49 |
| | 3>..\apps\pdfdraw.c(82): warning C4013: 'drawRange' undefined; assuming extern returning int | 14:29.50 |
| | 3>pdfdraw.obj : error LNK2001: unresolved external symbol _drawRange | 14:29.50 |
| | 3>Release\pdfdraw.exe : fatal error LNK1120: 1 unresolved externals | 14:29.50 |
| | damn case | 14:30.36 |
| Robin_Watts | AlecTaylor|Codin: Re your C99 comment above... We try and stick to C89 or below - basically our code should work on any K&R compiler. | 14:37.12 |
| AlecTaylor|Codin | Okay, completed my changes. Now upping to git | 14:37.30 |
| kens | Hmm, looks like the PostScript and PCL interpreters don't set up the 'second' colour. | 14:39.23 |
| | Perhaps. | 14:39.33 |
| Robin_Watts | kens: By default, no. | 14:39.38 |
| | Only the PDF interpreter will ever touch it. | 14:39.52 |
| kens | That probably explains my crashes then ;-) | 14:39.59 |
| Robin_Watts | because it calls swapcolor then sets it, then swapcolor back, I think. | 14:40.14 |
| kens | So how can I tell if teh second colour is ever valid ? | 14:40.35 |
| | Not that I think I need to but.... | 14:40.46 |
| Robin_Watts | It should be set to NULL etc I hope. | 14:41.10 |
| kens | Hmm, I'll try and check that. | 14:41.22 |
| Robin_Watts | In PS at least. | 14:41.45 |
| kens | Well actually that looks like a scarlet fish | 14:42.00 |
| | I should only set the second colour if we have text with a stroke Text rendering mode. | 14:42.15 |
| | In which case it *ought* to be set,. | 14:42.21 |
| | OK, back to the debugger | 14:42.34 |
| | Surprise, the crashes on the cluster don't crash for me :-( | 14:45.39 |
| sebras | Robin_Watts: how come mupdf/android builds ok when draw/arch_arm.c is included in LOCAL_SRC_FILES and fz_accelerate_arch() tries to set the global fz_path_w4i1o4 which I can no longer see declared anywhere (it used to be in fitz/fitz.h)? | 14:46.12 |
| | Robin_Watts: doesn't that spell compilation error? | 14:46.40 |
| chrisl | kens: are you testing a release build? | 14:47.23 |
| Robin_Watts | sebras: It works. Haven't got time to look how at the moment. | 14:49.45 |
| sebras | Robin_Watts: alright, np. | 14:50.15 |
| ab5tract | i really would not expect the arc in this example to draw its own line, only to clip: https://gist.github.com/1278290 | 14:52.47 |
| AlecTaylor|Codin | git.exe push --progress "origin" master:master | 14:53.04 |
| | fatal: The remote end hung up unexpectedly | 14:53.04 |
| | Where do I push my changes to? | 14:53.19 |
| Robin_Watts | AlecTaylor|Codin: You can't push back to our git repo. | 14:53.23 |
| AlecTaylor|Codin | Robin_Watts: Where do I push it to, is there a temp repo? | 14:53.34 |
| Robin_Watts | You commit to your repo, then use git format-patch | 14:53.37 |
| AlecTaylor|Codin | is a github fan | 14:53.40 |
| ab5tract | i have written other code that does not have the same problem, but i can't tell any difference between what i did in the previous code and what i've done in the gist | 14:53.46 |
| AlecTaylor|Codin | Robin_Watts: So I need to install a git server locally?! | 14:53.57 |
| Robin_Watts | AlecTaylor|Codin: No. | 14:54.06 |
| | If you have git running locally, that's fine. | 14:54.15 |
| | When you clone our repo, you've got all you need there. | 14:54.27 |
| | Just commit to that. | 14:54.30 |
| | Then git format-patch. | 14:54.34 |
| kens | chrisl no, a debug build | 14:54.35 |
| Robin_Watts | Then attach that patch to an enhancement bug on bugs.ghostscript.com and we'll have a look. | 14:54.52 |
| AlecTaylor|Codin | kk, have my patch file now | 14:55.32 |
| | upping it ther | 14:55.34 |
| | upping it there | 14:55.35 |
| chrisl | kens: might be worth trying a release build | 14:58.24 |
| kens | Might try in a minute, trying something else first ;-) | 14:59.07 |
| Robin_Watts | ab5tract: Try a 'newpath' ? | 14:59.36 |
| kens | ab5tract : I don't think (haven't checked) tht clip does an implicit newpath, so the 'arc' is still left as the current path before you begin the loop | 14:59.58 |
| | So your first stroke strokes the arc as well. | 15:00.12 |
| Robin_Watts | The clip does NOT do an implicit newpath. | 15:00.14 |
| ab5tract | Robin_Watts, indeed, thanks! | 15:00.19 |
| | yes was just reading that again in the plrm | 15:00.30 |
| AlecTaylor | So yeah, my code is now up & online! | 15:01.14 |
| | YAY | 15:01.15 |
| | :P | 15:01.16 |
| ab5tract | so in other words that means that the path is drawn by the later call to stroke | 15:01.18 |
| AlecTaylor | Accept Patch plz | 15:01.20 |
| Robin_Watts | AlecTaylor: What bug number ? | 15:01.32 |
| kens | ab5tract : yes that's what I said ;-) | 15:01.32 |
| ab5tract | kens, ;) | 15:02.04 |
| AlecTaylor | Robin_Watts: 692582 | 15:02.41 |
| Robin_Watts | Oh, right, sorry, I didn't follow what you were doing. | 15:04.26 |
| AlecTaylor | NW | 15:04.47 |
| AlecTaylor | will now take an anime break [2:04AM] | 15:04.59 |
| Robin_Watts | tor8 is the arbiter of such things, but I fear there is little chance of us accepting this at the moment. | 15:05.07 |
| AlecTaylor | darn | 15:05.13 |
| | When will tor be online? | 15:05.19 |
| kens | tor8 claims to be present | 15:05.29 |
| Robin_Watts | We're in the middle of doing a whole huge change in the way we handle errors/globals etc. | 15:05.44 |
| | in a parallel branch, 'context', (which may or may not be up to date in that repo) | 15:06.08 |
| tor8 | sebras: linker voodoo, probably | 15:06.17 |
| AlecTaylor | Robin_Watts: Well this is just one of the sample projects... I made it more modular + easier to read | 15:06.38 |
| Robin_Watts | Taking on something that radically changes the structure of the file is probably not going to happen until we finish that - otherwise it just makes merging harder. | 15:06.53 |
| tor8 | wow, I hadn't realised just how much apple has bastardised objective in the last couple of years. | 15:09.41 |
| | objective-c | 15:09.47 |
| AlecTaylor | tor8 is online! | 15:09.52 |
| | tor8: Accept my patch? | 15:10.00 |
| | 692582 | 15:10.48 |
| tor8 | I don't really have time at the moment. It also doesn't follow our coding standards -- for starters, we don't use CamelCase naming. | 15:11.46 |
| | and as Robin_Watts said, we're in the middle of big experimental changes so we're holding off on committing major changes to files pending how that turns out to make merging easier | 15:12.41 |
| Robin_Watts | AlecTaylor: If you have a new feature to add, then we're much more likely to look on that favorably than just changes to the structure of the code. | 15:12.49 |
| | But also (needs to be mentioned) you need to be prepared to give up rights to your code before we can accept it. | 15:13.30 |
| | We license mupdf commercially, so can't accept code submissions that stop our ability to do that. | 15:13.47 |
| AlecTaylor | Release my code under whatever license you want, I don't mind | 15:16.35 |
| kens | OK, that fixed almost all the seg faults. | 15:18.48 |
| | Looks like I have 4 to fix. | 15:19.04 |
| | Actually 3. | 15:19.12 |
| AlecTaylor | Robin_Watts / tor8 / kens: Do you have any sample code which grabs textual info + positional info from PDF pages and puts each page individually into a different node of a linked list? | 15:19.18 |
| Robin_Watts | AlecTaylor: The example code we have is the -t and -tt (I think? Whatever flag does xml output) options to pdfdraw. | 15:20.12 |
| kens | No, but the txtwrite code could be modified to do that, right now it uses a likned list to store teh text fragments, but flushes it on every page. WOuldn't be hard to add another level | 15:20.46 |
| | I'm sure tor's code must do something similar because wee both analyse the text. POssibly tor does it on the fly, I do it at page end | 15:21.38 |
| Robin_Watts | I believe tor8 outputs the xml in page sized lumps. | 15:22.00 |
| | (You can't really output anything until the end of the page because more might come in to be added to the previous stuff) | 15:22.31 |
| kens | That's what I meant, but you can merge fragments on the fly, I just don't | 15:24.04 |
| Robin_Watts | My original code did, but I don't know what tor8 has done with it since. | 15:25.39 |
| | AlecTaylor: You *could* write code to hook into pdfdraw that reads the same data structures as the xml output routines use. | 15:28.42 |
| | But that's a slightly risky way to work, as we wouldn't like to guarantee that those data structures won't change. | 15:29.04 |
| | For proof of concept code, it may be easier to write an entirely separate program that reads in the xml and works from that. | 15:29.39 |
| | My mum has just spent 2 days white water rafting in the himalayas, camping by the side of the river. It must be alzheimers. Someone should remind her she's 60ish. | 15:45.31 |
| AlecTaylor | <Robin_Watts> For proof of concept code, it may be easier to write an entirely separate program that reads in the xml and works from that. | 15:49.40 |
| | hmm, I think I'll do that | 15:49.45 |
| | thanks | 15:50.09 |
| Robin_Watts | AlecTaylor: That's not to say we won't be interested in the results :) | 15:51.04 |
| | Meeting time, but no meeting, right ? | 15:59.03 |
| kens | what henry said, yes | 15:59.12 |
| | unless you want one | 15:59.17 |
| Robin_Watts | has questions for mvrhel2. | 15:59.27 |
| kens | nothing new there then :-) | 16:02.22 |
| Robin_Watts | And speak of the devil. | 16:09.36 |
| mvrhel2 | good morning | 16:09.41 |
| Robin_Watts | Aloha! All back safely? | 16:09.44 |
| mvrhel2 | yes. back to 50 degrees and rain | 16:09.55 |
| | from 85 degress and sun | 16:10.02 |
| Robin_Watts | :) | 16:10.02 |
| | Did you do the sub trip? | 16:10.25 |
| mvrhel2 | no. we did a bunch of snorkeling and my son did snuba | 16:10.54 |
| Robin_Watts | Ah, well, next time :) | 16:11.23 |
| mvrhel2 | yes | 16:11.28 |
| Robin_Watts | snuba? | 16:11.35 |
| mvrhel2 | it is like scuba, but the tank is on a float on the water | 16:11.51 |
| | he went down to about 20 feet | 16:12.03 |
| Robin_Watts | Coo. | 16:12.23 |
| mvrhel2 | I was snorkeling overhead and he was with an instructor who was in regular scuba gear | 16:12.50 |
| Robin_Watts | mvrhel2: Anyway... back to the grindstone. | 16:14.40 |
| | Is it reasonable for xci to be negative ? | 16:14.48 |
| mvrhel2 | yes. I did quite a bit of work on my halftone creation code on the ride back but I did not dig into that bug. I was thinking of doing that today | 16:15.13 |
| Robin_Watts | (I've only seen a value of -1, so it may be a rounding thing). | 16:15.20 |
| mvrhel2 | I need to step through to see what is going on | 16:15.24 |
| Robin_Watts | If it's reasonable for xci to be negative, then my fix should be correct. | 16:15.44 |
| mvrhel2 | let me go and look at what xci is.... | 16:15.57 |
| Robin_Watts | But I didn't want to commit it in case I was patching around something bigger. | 16:16.04 |
| | xci = integer x coordinate. | 16:16.20 |
| mvrhel2 | oh. that is penum->xci | 16:22.08 |
| Robin_Watts | Yes. | 16:22.17 |
| mvrhel2 | ok. let me just look at this for a couple minutes | 16:22.56 |
| | and grab some caffeine | 16:23.07 |
| | oh this is a landscape case too | 16:28.11 |
| | I think in clist rendering with landscape xci does go slightly negative. digging a bit more | 16:28.43 |
| Robin_Watts | Really? | 16:29.28 |
| ray_laptop | in clist mode we write data above and below the band (in the device y direction), so if xci is in the image coordinate space, it may also be negative | 16:30.16 |
| Robin_Watts | I wonder if that's related to my "new shiny landscape interpolation code being broken in banded mode" bug. | 16:30.18 |
| ray_laptop | and with interpolation, we write even more data (interpolation 'support') | 16:31.02 |
| Robin_Watts | ray_laptop: Right. I was thinking that xci was in source space, not destination. | 16:31.13 |
| mvrhel2 | we dont do this type of halftoning there | 16:31.28 |
| Robin_Watts | but that seems wrong now you come to mention it. | 16:31.31 |
| mvrhel2 | oh sorry . didnt follow the discussion correctly | 16:31.51 |
| ray_laptop | I can't recall whether xci is in the image (source) coordinate system, or the device's | 16:31.51 |
| mvrhel2 | it is the device | 16:31.57 |
| Robin_Watts | In which case, I think it's just the classic "% is broken in C" problem. | 16:32.26 |
| mvrhel2 | Robin_Watts: I believe your fix is correct | 16:32.33 |
| | yes | 16:32.37 |
| Robin_Watts | mvrhel2: I'll commit that then. | 16:32.50 |
| mvrhel2 | I just ran into that issue in my screen creation code | 16:32.54 |
| Robin_Watts | I do wonder if we should move to using (& LANDBITS-1) | 16:33.21 |
| | sorry. | 16:33.27 |
| | & (LANDBITS-1) rather than % LANDBITS. | 16:33.38 |
| mvrhel2 | Robin_Watts: can you go ahead and fix the color case too | 16:33.43 |
| | or do you want me to do that | 16:33.48 |
| Robin_Watts | mvrhel2: Have done locally. | 16:33.51 |
| mvrhel2 | it would be good to do both with the same commit | 16:33.58 |
| Robin_Watts | % takes the same amount of time as a divide (20 cycles) on x86. | 16:34.11 |
| | & takes 1 :) | 16:34.17 |
| | And &15 does what % 16 really ought to. | 16:34.38 |
| | % would be even slower on ARM, too. | 16:34.57 |
| mvrhel2 | Robin_Watts: if you think it makes sense I am fine with it. Just but a comment on what the operation is doing (i.e. we want to do % 16) | 16:35.11 |
| | otherwise my small brain will forget | 16:35.18 |
| Robin_Watts | We'd need to restrict LANDBITS to be a power of 2 though. | 16:35.29 |
| mvrhel2 | I think that is fine | 16:35.35 |
| | I don't see that really being a major restriction | 16:35.51 |
| Robin_Watts | So do I. It used to be hardwired to 16, and I changed it, thinking it would give a speedup, and it didn't. | 16:35.59 |
| mvrhel2 | yes | 16:36.08 |
| | I remember that | 16:36.25 |
| | please feel free to make that change | 16:36.48 |
| kens | Heading off now, night all. | 16:46.15 |
| Robin_Watts | Night kens | 16:47.10 |
| mvrhel2 | cool. looks like I fixed quite a few issues in my screen creation stuff. Now I need to add in a few features like different horizontal and vertical resolution | 16:57.13 |
| AlecTaylor|Codin | Robin_Watts: Arghh @ inefficiencies | 16:58.25 |
| mvrhel2 | oh and multilevel screening too | 16:59.16 |
| ray_laptop | mvrhel2: oh oh. You're going to beat me to multi-level screening unless I get back to work on genpat ;-) | 17:03.08 |
| | mvrhel2: but minimum dot size is more important that multi-level, IMHO | 17:03.34 |
| | since gs doesn't support painting with multi-level thresholds yet anyway. | 17:04.00 |
| | (except in my kloodge 'bitgray2' device) | 17:04.23 |
| mvrhel2 | ray_laptop: yes. I need to think about the minimum dot a bit. That should be pretty straight forward to add into my stuff. I also need to support different dot shapes | 17:04.56 |
| ray_laptop | mvrhel2: yeah, dot shapes are something that some people care about. | 17:05.27 |
| mvrhel2 | My feeling is that minimum dot size *should* come out when a TRC is computed at least in the case for the ordered screen. With your cluster screen min dot is a bigger issue. I do need to add in the TRC stuff to. darn | 17:06.21 |
| | lots do to | 17:06.25 |
| | ok. so I will work on TRC, dot shape and Vertical/Horizontal resolution first | 17:07.03 |
| Robin_Watts | TRC ? | 17:07.08 |
| mvrhel2 | tone reproduction curve | 17:07.18 |
| | linearization | 17:07.25 |
| ray_laptop | mvrhel2: as long as you write the entire threshold array data, then 'linearize_threshold' can apply the TRC afterwards | 17:07.28 |
| Robin_Watts | right. | 17:07.29 |
| AlecTaylor | Well, speak tomorrow guys | 17:07.47 |
| Robin_Watts | thanks. | 17:07.47 |
| | AlecTaylor: Night. | 17:07.51 |
| mvrhel2 | ray_laptop: that is true | 17:07.53 |
| AlecTaylor | localtime==[4:07am] | 17:07.55 |
| Robin_Watts | AlecTaylor: Australia ? | 17:08.13 |
| ray_laptop | AlecTaylor: you are up early | 17:08.15 |
| AlecTaylor | aye | 17:08.21 |
| | Robin_Watts: Yup | 17:08.24 |
| | :P | 17:08.26 |
| mvrhel2 | and much easier ray_laptop. Write out 16 bit values and apply the TRC | 17:08.46 |
| Robin_Watts | mvrhel2: How do you do multi-level thresholding with threshold arrays ? | 17:09.27 |
| | multi-level halftones, I mean. | 17:09.49 |
| ray_laptop | mvrhel2: linearize_threshold reads integers (32-bit) since stochastic thresholds may be bigger than 256x256 | 17:09.56 |
| Robin_Watts | With 1bpp, it's output[x,y] = (input > threshold[x,y]) | 17:10.28 |
| ray_laptop | Robin_Watts: 2^n-1 values per dot location, where n is the number of bits | 17:10.38 |
| mvrhel2 | ray_laptop: ok. I will do that approach. I will make sure that I get an output out that linearize_threshold can read | 17:10.42 |
| | Robin_Watts: basically an array of thresholds | 17:11.03 |
| Robin_Watts | So... output[x,y] = (input < threshold0[x,y] ? 0: (input < threshold1[x,y] ? 1 : (input <threshold2[x,y] ? 2 : ....))) | 17:12.03 |
| ray_laptop | mvrhel2: or multiple values adjacent in the threshold array. I think that works better for cache hits | 17:12.10 |
| mvrhel2 | yes. a chunky version | 17:12.22 |
| Robin_Watts | ray_laptop: but worse for SSE2 maybe. | 17:12.41 |
| mvrhel2 | I will need to think about that a bit | 17:12.50 |
| | chunky will be better for this case I think | 17:13.00 |
| Robin_Watts | Are we guaranteed that the threshold values are sane? | 17:13.17 |
| mvrhel2 | well, yes if we created them | 17:13.32 |
| Robin_Watts | threshold0[x,y] <= threshold1[x,y] <= etc... | 17:13.37 |
| mvrhel2 | yes | 17:13.41 |
| ray_laptop | mvrhel2: Generally, we store the pattern clist in the clist, then load it during the page playback and tile_by_steps to fill the area during playback of the "main" page playback, right ? | 17:20.41 |
| mvrhel2 | bbiab | 17:50.36 |
| | oops. missed your comment ray_laptop | 17:50.53 |
| Robin_Watts | mvrhel2: I have some ideas on how to do multilevel thresholding with SSE2. | 17:51.37 |
| mvrhel2 | ray_laptop: yes that is my understanding | 17:51.41 |
| | Robin_Watts: ok cool. I have a few myself. I am going to work on that after I add in the additional functionaliy | 17:52.45 |
| | in the creation tool | 17:52.56 |
| | oh. did you seem my comment about lcms and Marti? | 17:53.11 |
| Robin_Watts | About him being excited about it? Yes. Cool. | 17:53.28 |
| mvrhel2 | it makes sense to push on with the move to lcms2 | 17:53.33 |
| | that profile is a bit of an outlier | 17:53.42 |
| | in that I doubt we will run into it in the wild | 17:53.53 |
| | it is also not clear who is right in the rendering with it | 17:54.13 |
| | Marti is going to work a bit on that | 17:54.29 |
| | lcms vs. Adobe vs. windows ICm | 17:54.39 |
| | so, we should go ahead and do a comparison between the two | 17:55.19 |
| | to make sure there are no major regressions | 17:55.27 |
| | heading home from coffee shop. back shortly... | 17:57.07 |
| Robin_Watts | marcosw_: you here? | 18:10.42 |
| mvrhel2 | Robin_Watts: so what is your idea with regard to multi-level thresholding? | 18:19.39 |
| Robin_Watts | I'm just scribbling it into a text buffer. | 18:25.08 |
| | Give me a few mins and I'll be more comprehensible, I hope. | 18:25.18 |
| | OK, I think I have it. | 18:26.45 |
| | I have 2 ideas. | 18:26.49 |
| | Both of which use the 'mm_unpackhi_epi8' SSE2 instruction. | 18:27.05 |
| | Our existing SSE2 code provides a fast way of getting us a mask full of bits such that c < t: (c = contone value, t = threshold) | 18:27.57 |
| mvrhel2 | yes | 18:28.10 |
| Robin_Watts | For 2 level rendering, we want a mask full of pairs of bits, st: | 18:28.34 |
| | c < t0 => 00 | 18:28.36 |
| | c < t1 => 01 | 18:28.38 |
| | c < t2 => 10 | 18:28.39 |
| | else => 11 | 18:28.41 |
| | i.e. b[0] = (c < t1 & !(c < t0)) | !(c < t2) | 18:28.43 |
| | b[1] = !(c < t1) | 18:28.44 |
| mvrhel2 | yes | 18:28.47 |
| Robin_Watts | My first idea was to try and use unpackhi so that rather than making us a mm register with: | 18:29.17 |
| | mm0 = c0 c1 c2 c3 c4 ... | 18:29.31 |
| | we'd make: | 18:29.35 |
| | mm0 = c0 c0 c1 c1 c2 c2 c3 c3... | 18:29.51 |
| mvrhel2 | where the c's are the contone values | 18:30.23 |
| Robin_Watts | Then we'd run our existing SSE2 stuff twice. Once with: | 18:30.23 |
| mvrhel2 | yes | 18:30.26 |
| | I understand | 18:30.32 |
| Robin_Watts | mm1 = t0 t1 t0 t1 t0 t1 ... | 18:30.38 |
| mvrhel2 | yes | 18:30.42 |
| Robin_Watts | and mm2 = t2 00 t2 00 t2 00 ... | 18:30.48 |
| | and then we could 'just' do some boolean bitmasking magic with the masks we got out of that to get what we want. | 18:31.25 |
| | Make sense? | 18:32.13 |
| | My second idea was to load mm0 = c0 c1 c2 c3... mm1 = t0 t0 t0 t0... mm2 = t1 t1 t1 t1... mm3 = t2 t2 t2 t2... then do the same SSE2 stuff as we do now, until just before the mask extraction. | 18:34.13 |
| mvrhel2 | yes. It makes sense. I think there are a probably a few different approaches that we can look into for this. First though we probably want to get a non-sse2 solution working and also have a decent set of tools in place to generate the screens. | 18:34.16 |
| | yes that is a pretty clean approach | 18:34.52 |
| Robin_Watts | Then we can use the 'unpack' instructions to insert 0's into those. | 18:35.00 |
| | That means doing more SSE2 work, but makes the recombination in x86 easier, I think. | 18:35.22 |
| mvrhel2 | yes | 18:35.29 |
| Robin_Watts | As you say, it makes sense to have a non-SSE2 solution working. | 18:35.53 |
| | But the nice thing is that neither of those approaches require us to have the threshold stuff in 'strange' orders. | 18:36.12 |
| mvrhel2 | right | 18:36.21 |
| | I am eager to get to the SSE implementation. I also think we should spend a bit of time looking at doing a fast contone pcl solution at some point to use as a benchmark | 18:37.52 |
| | like we talked about a bit ago | 18:38.11 |
| | with the chunky-> color convert to planar -> threshold | 18:38.26 |
| Robin_Watts | mvrhel2: Well, if you can generate some nice multilevel halftone tables, I'll update my mupdf code to use them :) | 18:39.14 |
| mvrhel2 | where are we now on the planar / color threshold stuff? | 18:39.31 |
| Robin_Watts | As thats exactly the contone -> threshold solution. | 18:39.34 |
| | mvrhel2: I'm playing whack-a-mole with the last few plank vs pamcmyk4 differences. | 18:39.59 |
| mvrhel2 | ok. then we can turn on and test the color hafltoning | 18:40.13 |
| Robin_Watts | Indeed. | 18:40.17 |
| mvrhel2 | sounds good. | 18:40.21 |
| Robin_Watts | I'm sorry this is taking so long. | 18:40.28 |
| mvrhel2 | I am going to grab a bit of lunch right now. No worries. There is no shortage of things to do | 18:40.41 |
| | of good things to do that is. | 18:40.51 |
| Robin_Watts | Cutting down PCL files is HARD. | 18:40.51 |
| | This one is an HPGL file really, so that's even harder. | 18:41.12 |
| mvrhel2 | having tools for generating screens will help us with one potential customer in particular | 18:41.19 |
| | I can't imagine where one would begin in reducing those files | 18:41.42 |
| | going to grab some lunch | 18:42.27 |
| | bbiab | 18:42.29 |
| ray_laptop | Robin_Watts: I saw your discussion on the SSE2 approaches and agree that having the different threshold levels planar instead of chunky makes more sense. The C code can be made reasonably good either way, so I think we should store the thresholds in a way that makes SSE2 more efficient | 19:05.26 |
| | Robin_Watts: whether it's planar or line interleave is probably a toss up. | 19:07.49 |
| | Robin_Watts: I have one multi-level 2-bit (three values per pixel) from customer 532, but 'genpat' doesn't generate multi-level yet either. | 19:08.41 |
| Robin_Watts | Well, the biggest hit in a register starved architecture like x86 is likely to be the need to keep the linespan around. | 19:09.16 |
| ray_laptop | Robin_Watts: line stride or plane stride, either way | 19:10.07 |
| Robin_Watts | From an SSE2 efficiency standpoint, it would probably be best to store each line of the threshold array as 16 t0's, then 16 t1s then 16 t2s, then back to t0s. | 19:10.24 |
| | But that's probably a step too far. | 19:10.38 |
| | but we unpack the thresholds into a buffer anyway, I think to cope with tiling, so it's probably not a big deal either way. | 19:11.47 |
| | I'm going to give up on this file and try another. My poor brain can't cope :( | 19:12.08 |
| ray_laptop | Robin_Watts: for small fills (which are common) that's better, I agree since one "read" will get multiple groups of the 3x16-bit blocks into the cache line | 19:12.56 |
| | too bad there's not an instruction for "return position in sorted byte array" | 19:19.40 |
| Robin_Watts | makes mental note not to ever let ray_laptop design an instruction set. | 19:35.11 |
| marcosw_ | Robin_Watts: I'm here now. | 19:36.30 |
| Robin_Watts | marcosw_: We want to move from lcms1 to lcms2. | 19:36.46 |
| | but before we do, it'd be good to test lcms1 vs lcms2 to see if there are any regressions. | 19:37.06 |
| | Could you run an lcms1 vs lcms2 test somehow ? | 19:37.20 |
| marcosw_ | seems easiest to make an lcms2 branch and then I can compare that branch to master. | 19:37.51 |
| Robin_Watts | marcosw_: -dWHICH_CMS="lcms2" ? | 19:39.08 |
| | -D even, sorry :) | 19:39.15 |
| marcosw_ | that's even easier :-) Didn't realize we already had lcms2 integrated. | 19:39.43 |
| | remind me if we need to test pcl and xps as well. | 19:40.16 |
| Robin_Watts | marcosw_: I suspect a small test to start with would be good. | 19:40.34 |
| | because it's possible we'll immediately run into problems. | 19:40.47 |
| marcosw_ | I was going to run all the files at 72 dpi, that's very quick. | 19:40.57 |
| Robin_Watts | but if that passes, then we'll probably want to run a larger one on all the languages. | 19:41.02 |
| | cool. | 19:41.06 |
| marcosw_ | I'll start it now, should have results by tonight (PDT). | 19:41.25 |
| Robin_Watts | Thanks. | 19:41.48 |
| marcosw_ | BTW, in case anyone is wondering, inches is down for a HD upgrade. Moving from 1 to 3 TB. It addition to a cluster node it's also now running regression tests. | 19:42.27 |
| ray_laptop | I wondered if the other shoe was going to drop with Raed. Sure enough, he didn't take my subtle hint that "this is a sample starting point" and is asking for more :-( | 23:45.43 |
| Robin_Watts | ray_laptop: He asked "can the postscript be updated to ...". | 23:46.46 |
| | You need to reply and say "Yes it can. Have fun." | 23:46.55 |
| ray_laptop | Robin_Watts: and I answered "of course" | 23:47.00 |
| | in somewhat more detail. I "read between the lines" and assumed he really meant 'can _you_ update the PS to..." | 23:47.53 |
| | Robin_Watts: I cc'ed support on the reply, so you will see it. | 23:48.31 |
| | At least Phil's (cust 531) request was pretty easy. Looks like it was something that was supposed to be in PDFFitPage, but never worked right | 23:49.54 |
| Robin_Watts | I understand all that - I was (jokingly) suggesting that you reply in a way that makes it clear that that isn't going to happen. | 23:50.19 |
| ray_laptop | Robin_Watts: sorry I didn't understand that you were joking. I always give top ten customers the alternative of contacting Miles for an NRE quote. That either makes us some money, or makes them go away | 23:51.48 |
| | at the rates Miles charges for us, usually the latter | 23:52.21 |
| | time to take the boys to their music lessons (violin and piano)... | 23:54.52 |
| | bbiaw... | 23:54.55 |
| Robin_Watts | night. | 23:55.02 |
| ray_laptop | g'nite, Robin | 23:55.15 |
| mvrhel2 | good night Robin_Watts | 23:55.24 |
| | Forward 1 day (to 2011/10/12)>>> | |
IRC Logs