| | <<<Back 1 day (to 2011/10/24) | 2011/10/25 |
| kens | tor8 do you have any notes on the 'XML' format that you output for text extraction ? | 09:50.40 |
| tor8 | kens: no. is it that time now? | 09:51.00 |
| kens | Well I juts updated mine to do UTF-8 as well as UCS2 and I thought while I was meddling with it.... | 09:51.22 |
| tor8 | right | 09:51.32 |
| | at the moment I output a line for each glyph in the low-level format | 09:51.53 |
| kens | Note the UTF8 and UCS2 ar ethe 'simple' versions that just output text, in an apprimation of the layout | 09:52.08 |
| tor8 | with the unicode character and bounding box | 09:52.13 |
| kens | Hmm, so you do each glyph individually. OK I do groups but I can break that up. | 09:52.28 |
| tor8 | and some wrapping tags to specify font style spans, lines, and blocks | 09:52.50 |
| kens | Hmm, I don't identify spans and blocks at that level. | 09:53.05 |
| | I basically do what I see from teh PDF file. | 09:53.17 |
| tor8 | kens: I have them individually because we want to be able to select text and place the marker at arbitrary positions | 09:53.31 |
| kens | Unicode text, font name, size, colour, renderuing mode, position. | 09:53.34 |
| tor8 | I don't think I do color and rendering mode :) | 09:53.45 |
| kens | Not a problem, I cna modify my code easily in there | 09:53.52 |
| tor8 | mine is also easy to modify | 09:54.03 |
| kens | :-) | 09:54.08 |
| | I was thinking that Tr 3 (invisible text) might be somethign people want to skip, or perhaps specially find. | 09:54.27 |
| | Similarly for text in a given colour for example | 09:54.37 |
| tor8 | kens: yeah. I should add the rendering mode, and text wouldn't be hard either. | 09:55.12 |
| kens | I guess I'm going to have to check out your branch to be able to experiment with your output ? | 09:55.16 |
| | To make mine look the same at least :-) | 09:55.45 |
| tor8 | kens: yeah. you can check out 'text' in mupdf.git | 09:55.52 |
| | not saying it can't use improvement, but you'll get the general idea of what I was looking for | 09:56.10 |
| kens | I have some time (more or less) so I'll make mine more like yours to start with. | 09:56.31 |
| tor8 | the code is all in fitz/dev_text.c | 09:56.31 |
| kens | I do have a MuPDF check out, but not the branch I think. | 09:56.46 |
| tor8 | kens: all checkouts should contain the branches | 09:56.58 |
| kens | OK, so I'll do a git status then. | 09:57.12 |
| tor8 | "git branch -a" and you should also see remote branches that it has stowed away | 09:57.13 |
| | just try git checkout origin/text | 09:57.25 |
| kens | I have master and text | 09:57.34 |
| | Looks OK, just doing an update | 09:58.05 |
| | Hmm, "You are in a 'detached HEAD' state..." | 09:59.54 |
| | What does that mean ? | 10:00.01 |
| | And if I do a git pull it says I'm not on a branch.... | 10:00.38 |
| tor8 | kens: that you shouldn't try committing, because it won't know on which branch to attach any commits to | 10:00.39 |
| | git checkout -b text will move the detached head into a new local branch | 10:01.03 |
| kens | Well I wasn't planning to do commits | 10:01.07 |
| tor8 | you're in "peeking around" mode with a detached head | 10:01.13 |
| kens | Right, that's what I need, will try that | 10:01.17 |
| Robin_Watts | tor8: The context branch... | 10:02.04 |
| tor8 | Robin_Watts: yes? | 10:02.25 |
| Robin_Watts | Are there questions etc you still want answered before we think about folding that back to HEAD? | 10:02.31 |
| kens | Hmm, still not getting very far.... Oh well as long as its up to date I cna build it and play. | 10:02.32 |
| Robin_Watts | i.e. we did it to try out whether it was the right approach | 10:02.48 |
| | I'm satisfied that it is - are you ? | 10:03.00 |
| tor8 | kens: I haven't changed it for 12 days. you can always "git checkout master" to get yourself back to where you were. | 10:03.14 |
| | Robin_Watts: I haven't had the time to decide that yet. too busy with the iphone stuff. | 10:03.47 |
| Robin_Watts | ok. | 10:03.56 |
| kens | tor8 I did that, I'm just wondering how to get an update to the current code in 'text'. | 10:04.11 |
| tor8 | it doesn't look "oh my god my eyes they buuuuurnss" at least :) | 10:04.16 |
| | kens: git fetch | 10:04.27 |
| | kens: that'll update the 'remote' branches | 10:04.38 |
| kens | Ah, thanks | 10:04.52 |
| tor8 | that is, it'll update the locally stowed away copies of the remote branches | 10:04.56 |
| kens | Done, seems OK, thanks | 10:05.09 |
| | Now to build it :-) | 10:05.27 |
| tor8 | good luck ;) | 10:05.35 |
| kens | A new and exciting challenge for me.... | 10:05.46 |
| tor8 | don't forget the thirdparty package, that's where moste people get stuck | 10:05.50 |
| kens | Yes, I'd already noticed it isn't there | 10:06.01 |
| tor8 | other than that it should be clear sailing | 10:06.02 |
| kens | Where can I grab third part from ? | 10:06.17 |
| | party | 10:06.22 |
| tor8 | http://code.google.com/p/mupdf/downloads/list | 10:06.29 |
| kens | Great, thnks | 10:06.36 |
| | tor8 building I get 7 succeeded one failed. | 10:17.03 |
| tor8 | which one failed? | 10:17.15 |
| kens | xpsdraw.c(130) | 10:17.16 |
| | syntax error : '{' | 10:17.29 |
| | Doesn't seem to like the structure initialiser | 10:17.41 |
| | project is xpsdraw obviously | 10:18.16 |
| | If I take the cast to (fz_rect) away its happy | 10:19.32 |
| tor8 | that's odd | 10:20.34 |
| kens | This is VS 2008 after converting the solution | 10:20.44 |
| tor8 | kens: I use VS 2008 Express | 10:21.35 |
| kens | Hmm this is the Team Edition. I take it it works for you ? | 10:21.53 |
| tor8 | and yeah, I get the same error as you | 10:22.02 |
| kens | The line is: | 10:22.06 |
| | fz_rect mediabox = (fz_rect){0, 0, page->width * 72 / 96, page->height * 72 / 96}; | 10:22.06 |
| tor8 | I think that syntax may be C99 actually | 10:22.20 |
| kens | Could be. Certainly if I take the (fz_rect) away its perfectly happy | 10:22.36 |
| | It doesn't affect me of course, but I thought you should know ;-) | 10:22.53 |
| tor8 | you're right, it should be fixed | 10:23.30 |
| kens | tor8 can I interrupt you a moment ? | 13:40.49 |
| tor8 | kens: go ahead | 13:41.09 |
| kens | looking at your text output, | 13:41.23 |
| | I assume a 'line' is any text on (more or less) the same Y co-ordintate ? | 13:41.40 |
| | THis is -ttt by the way | 13:42.37 |
| tor8 | yes. | 13:43.08 |
| kens | OK so what constitutes a 'span' and a 'block' ? | 13:43.24 |
| tor8 | I do an expensive text grouping pass to assemble characters into contiguous blocks | 13:43.34 |
| kens | Hmm I do that, but for the 'simple' output, not the 'develoeprs' output | 13:43.52 |
| tor8 | think: flood fill algorithm to colorize connected bits of text | 13:43.54 |
| | the results of this are my 'blocks' | 13:44.07 |
| | these are then sorted into lines based on text being on roughly the same baseline | 13:44.28 |
| | I only have one output :) | 13:44.39 |
| kens | Well, I have 2 :-) | 13:44.46 |
| | My 'XML' output is intended for developers who will do the work of re-assembly | 13:45.02 |
| tor8 | and the lines are split into spans based on the font, size (and I should add render mode and color) | 13:45.09 |
| kens | My 'simple' output is for people who want me to do it. | 13:45.13 |
| | So if size or font changes, then the span ends ? | 13:45.33 |
| tor8 | correct | 13:45.42 |
| kens | Otherwise all text on the smae line is a span ? | 13:45.44 |
| tor8 | the spans are basically an optimisation so I don't have to put all the attributes on every character | 13:46.00 |
| kens | OK that makes sense. | 13:46.12 |
| tor8 | if I were to do a 'simple' output, I'd just skip sorting the blocks and lines, and dump every text draw call as a span | 13:46.33 |
| kens | THat's more or less what I was thinking. | 13:46.44 |
| | I don't really want to do lots of expensive stuff in this format, but I would lik my output to match yours | 13:47.06 |
| tor8 | so we could keep roughly the same syntax for both outputs, and then have a fancier let's process this some more into HTML or plain text | 13:47.12 |
| kens | SOunds fine. | 13:47.36 |
| tor8 | kens: if you just skip the block (and/or line) tags for your simple output we could match formats | 13:47.51 |
| | but don't take my current format as gospel :) | 13:47.59 |
| kens | You beat me to it. | 13:48.09 |
| | Was about to suggest that. | 13:48.16 |
| | I don't mind altering the format later, but it would be good to be compatible. | 13:48.42 |
| tor8 | if you add color and render mode to the spans, I'll make sure my output matches yours | 13:48.47 |
| kens | I'm not certain about colour. | 13:48.58 |
| tor8 | yeah. | 13:48.58 |
| kens | Because of weid spaces | 13:49.09 |
| | weird | 13:49.12 |
| | Also, what colour is a 'clip' mode :-) | 13:49.27 |
| | I'm going to go ahead with my XML output and make it match yours, then I'll see if I can do a more complex version that does blocks and spans like yours does. | 13:50.08 |
| | 'MuPDF' format :-) | 13:50.19 |
| tor8 | fair enough (regarding colours) | 13:50.30 |
| kens | I think I'll leave colour and if someone wants it they can add it. | 13:50.52 |
| | I'm sure I can do the 'line' and 'span' stuff, not so sure about blocks :-) | 13:51.35 |
| | Thanks for the input. | 13:51.46 |
| tor8 | one block for the whole page will be fine I think :) | 13:51.52 |
| kens | Oh, well that makes sense I guess. | 13:52.08 |
| tor8 | the blocks usually end up being columns | 13:52.17 |
| | unless there are blank lines between paragraphs | 13:52.25 |
| kens | I would expect so | 13:52.28 |
| | TBH I kind of do that, just in a different way, for my 'simple' output | 13:52.58 |
| | I notice teh characters in the 'char' tag are all ASCII in my sample here, you don't emit Unicode, or are these UTF-8 ? | 13:53.47 |
| | Whoops, just seen one that is different | 13:54.05 |
| tor8 | I don't remember if I emit raw UTF-8 or xml &escaped; text | 13:54.20 |
| kens | XML escaped ;-) | 13:54.30 |
| | EG É | 13:54.40 |
| | make that "É" | 13:54.57 |
| tor8 | right :) | 13:55.20 |
| | I print UTF-8 in plain text output mode | 13:55.36 |
| kens | Yeah, me too, or UCS2 | 13:55.49 |
| tor8 | the runetochar() function converts to utf-8 | 13:55.50 |
| kens | Hmm, going to have to look at your code for escaping XML | 13:56.07 |
| tor8 | I don't do UCS2 -- byte order marks and byte orders in general, me no likey | 13:56.18 |
| kens | BOM is easy, I always emit the same. | 13:56.34 |
| | Big-endian | 13:56.44 |
| tor8 | and I've never seen the point of UCS2 text files. then again, I'm not a windows hacker at heart :) | 13:56.59 |
| kens | I can load them straight into Wordpad and friends | 13:57.18 |
| tor8 | you can't do that on linux and unix systems. or at least you didn't use to be able. | 13:57.42 |
| kens | Well the default mode is 'simple' UTF-8. You have to ask for anythign else ;-) | 13:58.02 |
| tor8 | the worst offenders are those who put a BOM in UTF-8 files... | 13:58.24 |
| | that just doesn't make any sense | 13:58.30 |
| kens | Well no, the BOM is illegal in UTF-8. Or at least the UCS2 one is | 13:58.49 |
| tor8 | so much software barfs on it, but a lot of Mac OS X software insists on writing UTF-8 text files with BOM at the beginning. | 13:59.12 |
| kens | And obviously there is no byte order in a byte-oriented stream | 13:59.15 |
| | Bizarre Mac voodoo | 13:59.30 |
| tor8 | it's a three byte BOM, silly as anything | 13:59.31 |
| kens | boggles | 13:59.38 |
| tor8 | it's 0xFF 0xFE (or the other way round) encoded in UTF-8 | 13:59.55 |
| kens | Oh, the UCS2 BOM encoded into UTF-8 that's mental | 14:00.18 |
| | Hi henrys | 14:05.51 |
| henrys | hi kens, I guess I should make the debug message glyph index ... out of range conditional seems to bother everyone. | 14:07.54 |
| kens | henrys I thinkn I have a request to do exactly that. | 14:11.57 |
| | I haven't found the code yet though.... | 14:12.05 |
| | Got diverted by a customer bug | 14:12.20 |
| henrys | I'll do it. | 14:12.31 |
| kens | Thanks, I'll see if I can find the bug number ;-) | 14:12.46 |
| | Hmm, maybe there isn't one, I can't see it here. Perhaps I imagined it. | 14:13.18 |
| henrys | you closed it invalid | 14:15.02 |
| kens | D'oh.... | 14:15.10 |
| | Twice! | 14:15.42 |
| | 692601 and 692629 | 14:15.51 |
| henrys | colorado, balmy today 15 inches of snow tomorrow | 14:29.49 |
| kens | wow! | 14:30.51 |
| Robin_Watts | nice | 14:38.38 |
| | xps, pcl, ls, svg all go through pl_main_process_options, right ? | 15:13.11 |
| henrys | yes | 15:15.24 |
| | although as discussed last meeting I'd like to see all that go away and we use the same api as we do in gs | 15:16.55 |
| Robin_Watts | I've added: --debug (which lists all the debugging flags) | 15:17.19 |
| | and --debug=flag,flag,flag to enable them. | 15:17.30 |
| | and that works in gs and pcl. | 15:17.45 |
| | So presumably in xps etc too. | 15:17.56 |
| henrys | don't like -Z | 15:18.09 |
| Robin_Watts | And that means we can add new flags with long names only. | 15:18.12 |
| henrys | oh I see | 15:18.26 |
| Robin_Watts | all the code still does: if_debug6('c',...) etc. | 15:19.12 |
| | but for new flags we can do: if_debug6(gs_debug_flag_tiling, ...) | 15:19.28 |
| | so no wholesale changes to the source. | 15:19.41 |
| henrys | great thanks for taking the initiative on that one! | 15:21.37 |
| Robin_Watts | Couldn't face the clist rop stuff without talking to ray first. | 15:22.00 |
| henrys | thinks the newsletter typesetting is an abomination we need an agenda item next meeting about doing it differently. | 15:30.14 |
| ray_laptop | morning, all | 15:30.48 |
| ray_laptop | goes to check the logs ... | 15:31.00 |
| Robin_Watts | Morning ray_laptop | 15:33.59 |
| ray_laptop | Robin_Watts: I just noticed that my email to you didn't go out. | 15:44.14 |
| Robin_Watts | Ah :) | 15:44.27 |
| ray_laptop | Robin_Watts: basically my comments were that gxclrect.c looked OK. Bunches of stuff in gxclrast.c that I don't follow without explanation. | 15:45.48 |
| | Robin_Watts: and it looks like your gxclrast.c didn't have some recent patches. | 15:46.35 |
| Robin_Watts | ray_laptop: I've just pulled to bring myself up to date. | 15:46.49 |
| | yes, I was probably a few days behind. | 15:46.58 |
| | Are you sure it was that way around? | 15:47.40 |
| | gxclrast.c hasn't changed much at all. | 15:47.49 |
| | Would it be easier to work from a diff? | 15:50.31 |
| | I'd like to ask some specific questions if possible. | 15:50.39 |
| ray_laptop | Robin_Watts: a 'format-patch' that I can apply or look at for diffs would be the easiest to work from | 15:52.34 |
| Robin_Watts | OK. I can do a format-patch, certainly. | 15:52.47 |
| | http://ghostscript.com/~robin/0001.patch | 15:53.36 |
| Robin_Watts | makes tea for meeting. | 15:54.43 |
| henrys | a patch without whitespace changes would be better. | 15:56.41 |
| | looks like mvrhel2 is a bit late but lets start | 16:00.16 |
| | chrisl: so we are going to go ahead with investigating ft type 1 parsing? | 16:00.42 |
| chrisl | henrys: yes, I've made a small start today, but more needed | 16:01.14 |
| henrys | which should leave alexcher without a project - | 16:01.20 |
| | alexcher can we try to get openjpeg in the next release? | 16:01.35 |
| alexcher | henrys: that's fine. I have plenty of bugs. | 16:01.50 |
| | henrys: yes. | 16:02.01 |
| henrys | alexcher:your customer list should be much improved. I didn't look carefully. | 16:02.36 |
| | tor8:how's ios? | 16:02.53 |
| kens | Morning mvrhel2 | 16:03.06 |
| mvrhel2 | good morning. sorry I am a bit late | 16:03.16 |
| henrys | mvrhel2:we started without you but you didn't miss much | 16:03.20 |
| kens | One of Alex's bugs is fixed by using Luratech, I'm inclined to remove the customer number from the field ? | 16:04.25 |
| | #692619 | 16:04.32 |
| henrys | as long as the customer is notified to use the commercial download. | 16:04.57 |
| kens | I think I did that, but I'll check | 16:05.08 |
| | Yes, it was Raed. | 16:05.43 |
| henrys | does anybody have anything for this meeting I didn't have much else | 16:06.19 |
| | ? | 16:06.20 |
| ray_laptop | Robin_Watts: on the --debug flags I like this a lot, thanks, but. | 16:07.31 |
| chrisl | I was going to ask about switching to lcms2, but probably best wait until marcosw is back......... | 16:07.39 |
| kens | Nothing for the meeting here. Gone back to working on text briefly. ray_laptop the txtwrite device now outputs UTF-8 by default. | 16:08.04 |
| Robin_Watts | I have questions for ray_laptop, but that's best done after the meeting. | 16:08.06 |
| mvrhel2 | henrys: the current xps bug that I am working on has slowed me down a bit. I am still hopeful though to have most of these wrapped up by next week though | 16:08.15 |
| ray_laptop | Robin_Watts: the UNUSED on many of the uppercase flags I think is wrong in light of the code in the gs_debug_c function. | 16:08.48 |
| henrys | mvrhel2:great news! | 16:09.04 |
| Robin_Watts | ray_laptop: UNUSED is important because it means 'don't create an entry' in the table. | 16:09.16 |
| ray_laptop | so a point for all to consider: Do we want to get rid of the 'upper case implies lower case' aspect of the flags | 16:09.26 |
| Robin_Watts | (or create an empty entry). | 16:09.36 |
| | I considered adding an 'implies' table. | 16:09.51 |
| | so A implies a. | 16:10.01 |
| ray_laptop | The user change would mean that if I want what I get from '-ZL' I have to do '-ZlL' instead. | 16:10.21 |
| henrys | I'm fine without an implies table as as long as there is a way to get the output. | 16:10.48 |
| Robin_Watts | ray_laptop: If you remove the implication, then yes, you'd have a change. | 16:11.01 |
| | Adding an implies table is dead easy though. | 16:11.11 |
| | and would enable us to get exactly the results we do now, but would free up the extra table entries. | 16:11.40 |
| ray_laptop | Robin_Watts: I'm fine with that. | 16:12.04 |
| Robin_Watts | I'll look into that then. | 16:12.12 |
| henrys | meeting adjourned? | 16:12.14 |
| chrisl | Record time ;-) | 16:12.28 |
| ray_laptop | Robin_Watts: and thanks again for taking the lead on this ! | 16:12.32 |
| Robin_Watts | The docs suggest that 3 implies 2, etc, but that's currently not implemented. | 16:12.58 |
| | ray_laptop: No worries. It kept me away from clist rops for a day. | 16:13.24 |
| ray_laptop | kens: I have a patch for bug 692618 that gets rid of the crash, but I'm getting some unexpected differences -- pretty close | 16:13.25 |
| kens | Thanks ray, I saw the progress you were making. | 16:13.41 |
| Robin_Watts | My next 'spare day' project is making it so that we get warnings about unused parameters. | 16:13.49 |
| kens | Sorry about the new file. I attached the wron gone initially | 16:13.53 |
| ray_laptop | kens: np. | 16:14.01 |
| | kens: turns out it was a little different -- not a dangling pointer in the pdf14 usage, but it was not using 'stable_memory' in one place that it was needed | 16:14.52 |
| kens | Ah, well it certainly looked related.... | 16:15.06 |
| ray_laptop | kens: well, it was compressed_color related | 16:15.32 |
| kens | :-) | 16:15.39 |
| | We'll have to build them a new executable when you're done, these are a support contract only outfit. | 16:16.23 |
| ray_laptop | kens: and I tripped over an error that was causing us to use page mode when we shouldn't that led to unexpected VMerror | 16:16.35 |
| kens | Ah, interesting. Maybe that explains the VMerrors... | 16:16.54 |
| henrys | oh something else - ray_laptop looked into using the heap allocator more frequently in gslib and found it too slow. But didn't that predate an important peformance fix to the heap allocator by Robin_Watts, is it worth it to look again? | 16:17.02 |
| ray_laptop | kens: support contract doesn't entitle them to binary updates | 16:17.07 |
| | we should get them on board with building their own. | 16:17.24 |
| kens | Hmm, OK, I expect Marcos will be back by then and he can handl eit ;-) | 16:17.25 |
| ray_laptop | henrys: it will be too slow unless we use a chunk manager | 16:18.21 |
| Robin_Watts | henrys: possibly, yes. | 16:18.25 |
| | ray_laptop: based on... ? | 16:18.31 |
| ray_laptop | the heap allocator goes directly to malloc -- lots of systems have fairly slow mallocs | 16:19.32 |
| | we should try again after Robin_Watts' improvement to gsmalloc -- trying PCL without the chunk allocator isn't too hard I think. | 16:20.52 |
| mvrhel2 | bbiab | 16:21.35 |
| ray_laptop | also the heap allocator is thread safe, and does a mutex lock (monitor) on EVERY allocation and this is a KILLER on Wiindows | 16:21.40 |
| | that's what prompted me to do the chunk allocator (for cust 531) for the multi-threaded rendering | 16:22.41 |
| Robin_Watts | malloc is thread safe anyway right? And multithreaded mallocs don't kill most programs. | 16:24.00 |
| henrys | has always wondered about the wisdom of that. We have one lock for the memory bookkeeping and another lock from the os | 16:24.17 |
| | for the memory itself | 16:24.46 |
| ray_laptop | well we can't do bookkeeping without a lock | 16:25.03 |
| Robin_Watts | henrys: Yes. And I want to add exactly that second lock into mupdf at some point :) | 16:25.37 |
| | (to protect cache freeing etc). But that's a discussion for another day. | 16:26.02 |
| henrys | anyway sorry to drag on the meeting with that just a thought. | 16:27.04 |
| kens | OK heading off, goodnight all. | 16:29.09 |
| Robin_Watts | Night kens | 16:29.15 |
| | re the patch and whitespace - sorry, I don't think the whitespace changes are too bad in that. Certainly not in the files that ray needs to look at. | 16:29.46 |
| | I don't know how to generate a format-patch without the whitespace changes :( | 16:30.05 |
| ray_laptop | Robin_Watts: no, but I didn't know I was getting so many other changes :-( | 16:30.11 |
| | Robin_Watts: don't make so many whitespace changes in the first place ? ;-) | 16:30.34 |
| Robin_Watts | ray_laptop: This is all tied up with other stuff :( | 16:30.38 |
| ray_laptop | Robin_Watts: I realized that when I looked at the patch. no problem, however | 16:31.06 |
| Robin_Watts | We used to have a 'copy_plane' method, that was basically copy mono, but with a number sent for what plane to copy. | 16:31.53 |
| | Michael had cunningly put that through the clist by doing the same thing for both, but sending a plane number of 255 to mean copy_mono. | 16:32.26 |
| | Now we've changed to having a 'copy_planes' method, which is like copy_mono, but sends the planar_height. | 16:32.56 |
| | So I can still share with copy_mono - I just send 0 as the planar_height. That explains the 255 -> 0 changes you'll see in there. | 16:33.41 |
| henrys | mvrhel2 - do you have some tool to take basic xps graphic's operators and operands and build the zip file system bs. I'm writing the xps writer and don't want to get into the zip nonsense right away. | 16:34.23 |
| Robin_Watts | henrys: If you output into a directory structure, both gxps and muxps can read from that, I think. | 16:35.04 |
| | (i.e. you can ignore the zipfile and just produce the files on disc) | 16:35.21 |
| | ray_laptop: Most of the changes in clist_strip_copy_rop are because it used to go wrong if it had to split a transfer of a planar tile up into separate bits. That all works I think. | 16:36.43 |
| henrys | yeah maybe I'll do that. right now I have just have a stream of operators. IMHO a shit design for a pdl. | 16:37.00 |
| Robin_Watts | ray_laptop: So the key bit that I have questions about is in gxclrect.c in the chunk from line 742 onwards. | 16:38.22 |
| | If I'm sending multiple planes I send each subsequent plane using cmd_put_bits after the first one. And only the first one gets the command header. | 16:39.26 |
| | Is that doomed to failure? | 16:39.30 |
| ray_laptop | Robin_Watts: I don't think so, but you may want to 'break' in the loop for planes after the 0th when the code would require the splitting | 16:42.07 |
| | and you need to handle the planes when you split it up | 16:42.32 |
| Robin_Watts | if ((plane_height > 0) && (code >= 0)) | 16:42.40 |
| | I believe the splitting does the right thing with planar data there. | 16:43.26 |
| | OK, so that's the writing side. | 16:44.02 |
| | On the reading side, I need to make sure I can read it back. | 16:44.16 |
| | How does the reading side know to reassemble the bitmaps it reads into the same bitmap ? | 16:44.46 |
| Robin_Watts | has scared ray_laptop off :( | 17:02.57 |
| ray_laptop | Robin_Watts: sorry. phone call. | 17:03.12 |
| Robin_Watts | Ah, no worries. | 17:03.20 |
| | I have the implication thing working, I think. | 17:03.27 |
| ray_laptop | Robin_Watts: I don't think it does re-assemble. It just splits into multiplie copy_mono_plane actions | 17:03.43 |
| Robin_Watts | Right. | 17:04.07 |
| ray_laptop | with the smaller rectangles | 17:04.10 |
| Robin_Watts | Yes of course. | 17:04.18 |
| | Why did I think otherwise? | 17:04.23 |
| | I have the problem that I need to read multiple planes into the same bitmap. | 17:05.04 |
| | oh, my existing code had the code >=0 in the for loop, so it should be fine. | 17:10.34 |
| | oh, my existing code had the code >= 0 in the for loop, so it should be fine. (Stupid emoticons) | 17:10.54 |
| ray_laptop | Robin_Watts: how do you tell chatzilla to not do an emoticon ? | 17:11.32 |
| Robin_Watts | I added a space :) | 17:11.48 |
| ray_laptop | I see | 17:11.59 |
| Robin_Watts | But in Preferences => Global settings => you can turn 'Faces' off. | 17:12.39 |
| | but that only affects you locally. | 17:13.03 |
| ray_laptop | Robin_Watts: so since the planes are all written following the 0th that has the cmd, it should be, because it is in the outer do { } while ((re.y += re.height) < re.yend); loop | 17:14.17 |
| | it just may have been split into smaller 'copy_mono' ops, right ? | 17:14.50 |
| Robin_Watts | ray_laptop: I've lost the thread here... | 17:14.56 |
| | it should be what ? | 17:15.05 |
| ray_laptop | Robin_Watts: I was just re-stating what I think we already said. | 17:15.17 |
| Robin_Watts | it should be OK, gottit. | 17:15.28 |
| | sorry. | 17:15.29 |
| ray_laptop | Robin_Watts: when you read the cmd the height will be correct for that (sub-rectangle) and the planes will follow the initial one | 17:16.36 |
| Robin_Watts | ray_laptop: Right. I'm looking at the reading code now. | 17:17.09 |
| ray_laptop | Robin_Watts: OK. so any more questions ? | 17:17.14 |
| Robin_Watts | So, I'm looking at case cmd_op_copy_mono_planes>>4 at line 968 of gxclrast.c | 17:18.19 |
| | We read the plane_height there, and I know if that's zero, that it's a copy_mono. If not, the depth can come from the color_info.depth. | 17:18.54 |
| | Then we jump forward to copy. | 17:19.07 |
| | If op & 8 then we read into the current tile; presumably that will be setup already for us to have multiple planes (I hope so!) | 17:19.58 |
| | otherwise we go into the second half of the if, and we calculate the size required - I need to update that to calculate the planar version. | 17:20.44 |
| | Then I need to put the next bit (all the decompression stuff) into a loop, one per plane. | 17:21.16 |
| | OK. I'll give that a whirl. | 17:22.03 |
| | Thanks. | 17:22.23 |
| ray_laptop | don't you just _love_ all the jumping around. I used to not mind goto ___ so much. | 17:22.43 |
| Robin_Watts | There are circumstances where goto is required for performance etc. Then there are other cases... | 17:23.38 |
| ray_laptop | Robin_Watts: if you have anything for me, just make sure and mention my name. | 17:23.43 |
| Robin_Watts | will do, thanks. | 17:23.55 |
| ray_laptop | otherwise I'll be working on bug 692618 | 17:24.08 |
| Robin_Watts | ray_laptop: Do you have chatzilla set to automatically reconnect, btw ? | 17:24.13 |
| | (Preferences => {Global Settings or freenode} => general => Reconnect when disconnected unexpectedly ) | 17:25.07 |
| ray_laptop | Robin_Watts: yes, I have chatzilla set to reconnect | 17:35.47 |
| Robin_Watts | ray_laptop: I ask because you often drop off with a timeout and don't reconnect for a long time. | 17:36.27 |
| ray_laptop | Robin_Watts: probably because I am working and don't care if I'm connected. Some places I may go to work on things (I get bored in the office) don't have wi-fi. I can work better with some distractions around (been that way since college or before) | 17:44.16 |
| Robin_Watts | fair enough. | 17:44.36 |
| ray_laptop | I could never stand studying or working on papers in the library or dorm room -- I'd go to the student commons or just sit outside where there were a lot of students walking around. | 17:45.24 |
| | probably a psych condition that was never diagnosed ;-) | 17:46.15 |
| Robin_Watts | ray_laptop: Well, let me aid your concentration by further distracting you then :) | 17:49.01 |
| ray_laptop | Robin_Watts: OK. (thanks) | 17:49.18 |
| Robin_Watts | At around 1065 in gxclrast.c the code does: cmd_read(cbut.data, bytes, cbp); | 17:49.37 |
| | then source = cbuf.data | 17:49.44 |
| | If I'm following properly that's the 'use the data direct from in the clist buffer' case. | 17:50.15 |
| | That's going to be problematic for planar data, I fear. | 17:50.57 |
| ray_laptop | Robin_Watts: how so ? | 17:52.30 |
| | all of the planes are contiguous, right ? | 17:52.53 |
| | or doesn't 'bytes' include all the planes ? | 17:53.18 |
| Robin_Watts | Are we guaranteed that the clist will pack subsequent planes at a multiple of raster? | 17:53.30 |
| | I was intending bytes to be just bit enough for a single plane. | 17:53.56 |
| | But I can probably just disable this case, right? (i.e. force it through the previous branch of the if if planar_height > 0) ? | 17:55.27 |
| ray_laptop | Robin_Watts: sounds OK. As far as the packing, it should be OK since the op_size is 0. The problem you have, however is that you don't have an op to tell you if it is compressed or not, and the compression is tried on each plane individually | 18:02.18 |
| Robin_Watts | Ah, so I need to go back and send a byte before each new plane with the compression type ? | 18:03.16 |
| ray_laptop | you either need an op for each plane to tell you the compression state, or you need to force the 1-n planes to be the same as plane 0 | 18:03.47 |
| Robin_Watts | Doesn't need to be an 'op' per se, right? | 18:04.10 |
| ray_laptop | Robin_Watts: no, as long as your reader knows what it is | 18:05.10 |
| Robin_Watts | Ok, thanks. I'll try that. | 18:05.22 |
| ray_laptop | you could even have one byte with bits for each of up to 8 planes following if you wanted to get complicated | 18:05.57 |
| Robin_Watts | Is it just 'compressed or not' ? | 18:07.21 |
| | Ah, I see. I'm not going to get complicated :) | 18:08.15 |
| ray_laptop | but having the 'op' before the plane means that the planes won't be at 'raster * height' positions :-( | 18:11.44 |
| Robin_Watts | That's fine. I'm forcing the data to be read out into a buffer (i.e. avoiding the last case) | 18:12.26 |
| ray_laptop | hmm... I 'touch' gstrans.h and it rebuilds a lot of stuff, but it doesn't rebuild some things like 'gdevmem.obj' and 'gdevprn.obj' | 18:14.59 |
| | oops. the definition of gstrans_h= macro was too far down in lib.mak | 18:18.25 |
| Robin_Watts | nice catch. | 18:19.28 |
| ray_laptop | I kept rebuilding in VS and my 'fix' wasn't taking effect ;-( | 18:20.27 |
| | the 'ESTIMATED_PDF14_ROW_SPACE' was _really_ not conservative enough ! I found to allocating 2.5x the BufferSpace without even having a very deep transparency stack | 18:22.33 |
| | of course, that's partly because transparency needs color_info->num_components+1 buffers, which can be a really big number if separations are being computed | 18:24.51 |
| | hmm... yet another memory usage issue. We downgrade the band_height when the page has transparency, but then the extra buffer space is then unused during rendering | 18:58.21 |
| | something we should look into improving. | 18:58.39 |
| | I suppose it's not usually that big of a deal, but I'm using -dBufferSpace=500000000 and it's leaving ~450M unused | 19:02.41 |
| | then proceeding to allocate the transparency stack buffers elsewhere. | 19:05.40 |
| | so the 'peak' allocation is _still_ higher than one would think when limiting BufferSpace (850M). So we _would_ have had enough to keep the transparency stack buffers in the otherwise unused space | 19:08.16 |
| | I'd be a bit worried (on an embedded system without memory mgmt.) about freeing the space during rendering and re-allocating it afterwards because we might leave a sandbar somewhere, but... | 19:10.25 |
| | Robin_Watts: how common, in your experience, are embedded systems that don't have MMU's and OS's that we wouldn't have to be concerned. | 19:11.38 |
| | because the other approach is to have to pdf14 logic use that area when doing clist playback (uglier, IMHO) | 19:12.54 |
| | I'll open an enhancement bug for myself on this issue. | 19:13.22 |
| Robin_Watts | ray_laptop: Non-existent. | 19:13.28 |
| | These days *any* embedded system will be ARM or Atom. | 19:13.44 |
| ray_laptop | Robin_Watts: Oh, really ! that's good news ! | 19:13.45 |
| Robin_Watts | Almost certainly ARM. | 19:13.49 |
| | and all ARMs worth worrying about these days have an MMU. | 19:14.11 |
| ray_laptop | cust 532 uses a PPC, but it also has MMU | 19:14.22 |
| Robin_Watts | and atoms certainly do. | 19:14.27 |
| ray_laptop | Robin_Watts: thanks for your insight. I think I'll just do the free & alloc then | 19:15.41 |
| Robin_Watts | ray_laptop: np. | 19:16.02 |
| sebras | Robin_Watts: the last ARM I saw without an MMU was in nintendos GBA, possibly their NDS suffer the same fate. but for sure any embedded ebook reader would have one (probably because they want linux inside). | 19:17.21 |
| | Robin_Watts: any recently modern phone would too. | 19:17.47 |
| Robin_Watts | The NDS has 2 ARMs in. | 19:17.59 |
| | ARM7 and ARM9 if memory serves. | 19:18.16 |
| | The ARM7 has no MMU in that case. | 19:18.28 |
| | but the ARM7 was only used for sound, if memory serves. | 19:18.53 |
| Robin_Watts | played with TunaVIDS, a video player for the NDS. | 19:19.14 |
| ray_laptop | off to run an errand. My youngest turns 6 tomorrow -- need to go shopping for his present(s) | 19:39.30 |
| | Robin_Watts: if you need me, call me on the phone (or have henry or someone over here call) so I can get back online. | 19:40.30 |
| sebras | Robin_Watts: mm, the ARM7 was also used when running backwards compatible GBA games. I know because I played with my GBA. :) | 20:10.33 |
| Robin_Watts | Well, I'm getting SEGVs with some test files, but I've had enough for the night... will look tomorrow. | 20:15.49 |
| mvrhel2 | henrys: sorry I missed your earlier question. I think Robin answered it though about the xps output. You can render the fpage directly as long as you have all the resources there in the directory | 21:32.52 |
| ray_laptop | where the heck is the documentation on the cluster ? I need to run 'bmpcmp' with a 'window' tolerance to allow for things shifting (primarily in Y) | 22:10.48 |
| | BTW, where is the source for the regression dashboard ? | 22:12.07 |
| ray_laptop | thinks that might be a good place to have a link for the docs | 22:12.26 |
| Robin_Watts | bmpcmp -w 1 ? | 23:16.23 |
| | Forward 1 day (to 2011/10/26)>>> | |
IRC Logs