| <<<Back 1 day (to 2013/12/12) | 2013/12/13 |
mvrhel_laptop | Robin_Watts: for the logs. I pushed an updated commit that has a *much* better process of creating the thumbnails. Easy enough for even me to follow it after I wrote it | 04:17.50 |
| If you can have a look in the morning, that would be great | 04:18.08 |
Robin_Watts | mvrhel_laptop: Looking now. | 10:53.56 |
| if (ComputePageSize(spatial_info_local, k, &ras_size) == 0) do you mean S_ISOK ? | 10:54.29 |
| while (result->Status != Windows::Foundation::AsyncStatus::Completed) {} looks like a nasty busy wait. | 10:55.01 |
| mvrhel_laptop: See the logs. | 11:03.54 |
| Overall looks nicer, but busy waits are bad, IMHO. | 11:07.13 |
Robin_Watts | waves a red bull under the webcam in an attempt to attract tor. | 13:09.54 |
tor5 | hey robin! | 13:27.55 |
Robin_Watts | Morning. | 13:28.05 |
tor5 | was that the smell of sugar and caffeine I could smell over your webcam? | 13:28.19 |
Robin_Watts | It was. | 13:28.28 |
Robin_Watts | has been pondering some more on this locking thing. | 13:28.48 |
| Firstly, I have a commit up with the fz_stream context removal... | 13:29.14 |
| http://git.ghostscript.com/?p=user/robin/mupdf.git;a=commitdiff;h=95bbb782ce3b0ac61cd9e1acff6a65f7ba79235b | 13:29.35 |
| Hmm. The jbig2dec change in there is wrong. | 13:30.00 |
| But other than that it's ok. | 13:30.18 |
| It's a HUGE change. | 13:30.25 |
| But the more I think about this, the more I prefer the idea of having 'rebind' calls. | 13:30.47 |
| We'd discussed the idea of having fz_lock_document/fz_unlock_document. | 13:31.23 |
| and the same for the device. | 13:31.28 |
tor5 | yeah, that's a large diff. but straight forward. | 13:31.31 |
Robin_Watts | We can't do that without pulling thread lib dependencies into mupdf. | 13:31.51 |
| I think I'd rather just offer fz_rebind_document, and then people can build their own fz_lock_document out of that. | 13:32.24 |
| That keeps the threading mess out of mupdf. | 13:32.32 |
| Likewise fz_rebind_device, fz_rebind_stream, fz_rebind_output. | 13:33.03 |
| I strongly suspect that no one outside of stream implementers and us will see fz_rebind_stream. | 13:33.49 |
| as fz_rebind_document will just call fz_rebind_stream. | 13:34.03 |
| Certainly, all this will be hidden inside the JNI level for the java users. | 13:34.20 |
| Having to pass the extra context all the way through the the lexing code just feels wrong to me. | 13:35.04 |
tor5 | I don't know, it feels regular to me. we pass around the context almost everywhere else... | 13:40.11 |
| and I don't see how adding fz_lock_document would introduce a dependency on threading libraries? I thought the real locking mechanisms that are used are all proxied through the lock context | 13:43.01 |
Robin_Watts | tor5: Currently we have a limited number of locks. | 13:45.25 |
| 4 locks in fact, though 1 is unused. | 13:45.40 |
tor5 | oh, right. so the locking api would need changing. | 13:45.54 |
Robin_Watts | We would need 1 lock per document, plus 1 lock per device. | 13:46.00 |
tor5 | to add generic locks | 13:46.01 |
Robin_Watts | To add an aribtrary number of locks, yes. | 13:46.14 |
| We'd need to be able to create locks, destroy locks, take locks and drop locks. | 13:47.07 |
| We'd need to reference count locks internally, and we'd need to lock to protect the reference counts. | 13:47.53 |
| It's not impossible, it's just sounding nasty. | 13:48.25 |
| I still prefer the idea of a mupdf with no threading stuff built in. | 13:48.53 |
| i.e. we can build a single lib on a system that works with or without locks/threading etc. | 13:49.16 |
tor5 | right. so add bind/rebind calls for now and ponder the locking later? | 13:49.51 |
Robin_Watts | yeah, having rebind doesn't force us into anything. | 13:50.07 |
tor5 | and only the JNI needs worry about rebind for now | 13:50.08 |
Robin_Watts | indeed. | 13:50.13 |
tor5 | but I still think removing the ctx from the stream is the right thing to do... | 13:50.26 |
Robin_Watts | I would agree with that, were it not for the extra param wrangling in the parser. | 13:51.07 |
| actually... | 13:52.01 |
| maybe we can remove some of that wrangling. If we take the context out of the stream, but leave it in the lexbuf... | 13:52.20 |
| cos the lifespans of the lexbuf's are such that we don't need to rebase. | 13:52.50 |
tor5 | I take it you mean to insert the ctx into lexbuf->ctx at the top of the pdf_lex function to avoid passing it to lexstring etc? | 13:54.28 |
| reading through the pdf_lex.c diffs nothing looks really out of place with the ctx argument, so maybe I'm missing something atrocious you've spotted elsewhere... | 13:55.43 |
Robin_Watts | Maybe I'm being overly sensitive. | 13:57.52 |
| My background in embedded stuff makes me shy away from passing more params than I need. | 13:58.20 |
tor5 | 3 vs 2 params is only 50% more :) | 13:58.34 |
Robin_Watts | If we're going to have rebind stuff for document and device, why not have it for stream too? | 13:59.08 |
tor5 | but we've already made the decision that one register/argument is reserved for the context, can't have you back out of that *now* after arguing so hard for it :) | 13:59.13 |
| Robin_Watts: because it's stateful and icky. | 13:59.35 |
Robin_Watts | tor5: We've already made the decision that one register/argument is reserved for some form of context (whether it's an fz_context, or an fz_document, or an fz_device). | 14:00.04 |
| and I stand by that. | 14:00.31 |
| Passing both an fz_context and an fz_document for example feels wrong. | 14:00.58 |
| It is very rare for us to ever need to rebind the context within a stream. | 14:01.22 |
tor5 | in the context/document case, I agree. not so sure about devices. | 14:01.31 |
| a stream isn't a context though, in that case it was a shortcut to not passing the context | 14:01.56 |
Robin_Watts | The sole place we ever need to do that (I believe) is when we rebind a document, and that has a stream internally. | 14:01.59 |
tor5 | a device doesn't have a lot of context, and a lot of things that could have been in a device-type context (like the glyph cache) is held in the fz_context instead so that devices can be lightweight and short lived | 14:02.29 |
Robin_Watts | I agree. | 14:02.47 |
| And it's vanishingly rare that we'll ever want to create a device in one thread and use it in another. | 14:03.04 |
| So rebinding should almost never happen. | 14:03.12 |
| BUT... with the java APIs we can't be 100% sure that it won't happen. | 14:03.30 |
tor5 | if it does, I think that would be a user error | 14:03.38 |
| so maybe check that the TLS ctx matches the device in the JNI binding and throw an error otherwise? | 14:03.53 |
Robin_Watts | I guess we could add a function that checked the context and gave an error. | 14:03.56 |
| But if we're doing that, we might as well add rebind. | 14:04.10 |
| For instance, suppose we have someone trying to drive the device API programmatically. | 14:04.25 |
tor5 | someone doing that, would imply that they haven't got a clue about what their threads are doing, so giving them an error is probably better than letting them get away with broken code ;) | 14:04.45 |
Robin_Watts | For instance to use MuPDF as a canvas, or to do PDF generation or something. | 14:04.49 |
| They might be getting callbacks from some other process. | 14:05.10 |
| and hence they might be coming in on different worker threads. | 14:05.27 |
tor5 | I don't know of any graphics APIs that let you draw from multiple threads | 14:05.32 |
Robin_Watts | We'd only be allowing it to be drawn from a single thread at a time. | 14:05.51 |
tor5 | then they should pass the drawing operation to *one* worker thread that dispatches the draw calls | 14:05.53 |
| and in your scenario, what ends up on the page would be in completely random order. hardly what one would desire... | 14:06.21 |
| now, if we have ctx stowed into the device, then I still think we should add a rebind call as you're arguing for | 14:06.59 |
| to make it explicit that it's a context-proxy and can be moved from one context to another | 14:07.26 |
Robin_Watts | I could imagine (I think), a case where some output rputine would make callbacks that I would want to make device calls from. | 14:08.17 |
| While I would expect those callbacks to be nicely serialised (i.e. a well known order), I could imagine a case where each of those callbacks took place in a different worker thread. | 14:08.58 |
| For instance stuff coming in over RPC. | 14:09.04 |
tor5 | and there's some other (unknown to us) locking mechanism to ensure only one can execute at a time | 14:09.29 |
| sure, I can buy that | 14:09.34 |
Robin_Watts | so rebind would work there, throwing an error would not. | 14:10.33 |
| OK, I'll keep bashing. | 14:11.29 |
| Thanks. | 14:12.07 |
josip | hi | 14:49.47 |
| anyone around for a quick question about information comments on standard output ? | 14:50.42 |
kens | Don't know what you mean by 'information comments'. Is this Ghostscript or MuPDF ? | 14:51.22 |
josip | it's Ghostscript related question | 14:51.38 |
kens | ALso,don't ask to ask, just ask. The channel is logged and the developers read the logs | 14:51.39 |
josip | in short... when converting PDF to any image format, we get 'Processing pages 1 through 7.' and 'Page 1', Page 2'.... Page n... | 14:52.45 |
kens | Yes. | 14:52.51 |
| Depending on the PostScript you get input this is possible | 14:53.01 |
| OOps, PDF file, so yes, it does tell you that | 14:53.16 |
josip | is there any way to make Ghostscript behave a same when converting Multi page PostScript to PDF ? | 14:53.22 |
kens | No | 14:53.29 |
| A PDF file is a structureed format which includes (amongst other things) a definition of the number opf pages. PostScript is a streamable simple format. and does not contain that information | 14:54.04 |
| *Some* PostScript files contain 'DSC' comments which declare the number of pages in the document. However these can include the not terribly useful 'atend' value, which means that the number of pages is defined at the end of the program. By which time, of course, an interpreter knows how many pages there are, because its processed them all | 14:55.25 |
| However, these are *comments* they are not required to be present, and a conforming interpreter need not pay any attention to them. | 14:55.53 |
josip | got it | 14:56.05 |
kens | If you really want to you can play with Ghostscript's DSC parsing and see if it helps you | 14:56.17 |
| GSView uses this for jumping around DSC-complian PostScript documents | 14:56.40 |
josip | i did a same in Ghostscript.NET for .NET PS Viewer control | 14:57.01 |
| but.. this is more about converting PS to PDF by simply passing PS to ghostscript | 14:57.51 |
| and making 'Progress' event | 14:57.59 |
kens | A simple number_bytes_in_file / number_bytes_sent_to_intgerpreter is usually sufficient | 14:58.39 |
| If you want more than that, then you will indeed need to parse the DSC comments. Or, possibly, render to the null device and count the number of executions of EndPage | 14:59.33 |
josip | ok | 15:00.07 |
| I will find some solution... | 15:00.27 |
| thanks for your time and help | 15:00.34 |
kens | No problem | 15:00.40 |
Robin_Watts | henrys: For use during your ironman: http://www.thepoke.co.uk/2013/12/12/useful-dive-sign-of-the-day/ | 15:12.38 |
kens | ?me thinks 'bye bye' and swim away quickly would be better | 15:13.45 |
henrys | Robin_Watts: I'm surprised the maui patrol didn't use something like that on Sabrina and I. | 15:25.51 |
| Robin_Watts: I really wish we got a ticket for that. What a great souvenir - a ticket for swimming in a beach closed for sharks. | 15:28.51 |
Robin_Watts | henrys: ha! yeah, that'd be worth framing. | 15:29.30 |
henrys | can someone look at gs_matrix_multiply with me and confirm that we shouldn't pm2->tx and pm2->ty should be set to local doubles to be consistent? The matrix coefficients are floats. I assume we want to do everything in double and then cast it back to floats. | 15:57.01 |
Robin_Watts | henrys: looking. | 15:57.43 |
| I only see pm2->tx being used twice. | 15:58.42 |
| Firstly in: pmr->tx = tx1 * xx2 + pm2->tx; | 15:58.53 |
| tx1 and xx2 are both doubles already. | 15:59.02 |
| so pm2->tx will be promoted to a double before use there. | 15:59.11 |
| and then in: | 15:59.19 |
| pmr->tx = tx1 * xx2 + ty1 * yx2 + pm2->tx; | 15:59.31 |
| same thing there. | 15:59.44 |
| So I don't see why another cast would make any difference. | 15:59.56 |
henrys | Robin_Watts: yup you are right I forgot it would be promoted | 16:00.04 |
mvrhel_laptop | Robin_Watts: I understand about the wait. This does occur on a different thread, and I can't move on until this asnyc call completes | 16:00.25 |
Robin_Watts | mvrhel_laptop: Ideally you should wait on a semaphore or something, then have a .then thing that unblocks the semaphore. | 16:00.54 |
mvrhel_laptop | I am waiting on a status change that the write has completed | 16:02.06 |
| there is nothing for the thread to do until that finishes | 16:02.20 |
Robin_Watts | mvrhel_laptop: Right, but you're busy waiting on that status change. | 16:02.28 |
| You're burning CPU and battery running around a loop checking repeatedly. | 16:02.45 |
mvrhel_laptop | oh. the fact is that this is only writing about 100 bytes | 16:02.58 |
Robin_Watts | Better for the thread to sleep, and the other thread to wake it up. | 16:03.22 |
mvrhel_laptop | oh no I take that back | 16:03.33 |
Robin_Watts | mvrhel_laptop: Yes, but the number of bytes written is not the issue. | 16:03.39 |
mvrhel_laptop | I see what you are saying | 16:03.40 |
| let me see if I can fix this | 16:03.46 |
Robin_Watts | The scheduler may choose not to change threads until it's entire timeslice is used up. | 16:04.14 |
mvrhel_laptop | the issue is that the StoreAsync call is a WinRT call that I have no control over | 16:04.17 |
Robin_Watts | But there is a StoreAsync().then thing you can use, right? For exactly this reason. | 16:04.36 |
mvrhel_laptop | no :( | 16:05.13 |
Robin_Watts | http://msdn.microsoft.com/en-us/library/windows/apps/windows.storage.streams.datawriter.datawriter.aspx | 16:05.24 |
mvrhel_laptop | plus the way the windows works even if there was a .then | 16:05.26 |
Robin_Watts | sorry: http://msdn.microsoft.com/library/windows/apps/br208154 | 16:05.52 |
mvrhel_laptop | it would be tricky to keep the thumb creations in order | 16:05.58 |
| oh ok | 16:06.22 |
| that is going to complicate things a little since I may find things getting out of order | 16:07.04 |
| let me think about it. | 16:07.10 |
Robin_Watts | Why? | 16:07.11 |
mvrhel_laptop | well, the loop will move on to the next thumb | 16:07.28 |
Robin_Watts | No, the loop should wait on a semaphore. | 16:07.36 |
mvrhel_laptop | no it wont | 16:07.45 |
Robin_Watts | And the .then() should signal the semaphore. | 16:07.49 |
| That way the loop only moves on when the .then runs. | 16:08.08 |
| hence no busy wait. | 16:08.13 |
mvrhel_laptop | I see. | 16:08.18 |
| the trick to this is making sure the .then runs on the same thread as the for loop | 16:08.35 |
| let me make sure that occurs properly | 16:08.56 |
Robin_Watts | Why does the .then need to run on the same thread as the for loop? | 16:08.56 |
| I was imagining it would run on a different thread. Hence the use of a semaphore. | 16:09.33 |
mvrhel_laptop | ok. the for loop is not going to wait around for the task to complete. and it is not going to wait around for the then to complete | 16:09.37 |
| it is going to move on to the next index | 16:09.45 |
Robin_Watts | mvrhel_laptop: Not if it waits on a semaphore. | 16:09.52 |
mvrhel_laptop | what is "it" | 16:10.01 |
Robin_Watts | The main thread. | 16:10.07 |
| The main thread currently goes something like: while (more thumbnails to do) { Trigger the Async event, Wait for the Async event } | 16:10.42 |
| yes? | 16:10.45 |
| and my objection is that the wait is a busy wait. | 16:11.12 |
| So I'm suggesting we instead do: | 16:11.22 |
| while (more thumbnails to do) { Trigger the Async event.then(signal semaphore A). Wait for semaphore A. } | 16:12.10 |
| So each time round the loop the main thread will trigger the async event, and then sleep waiting for the semaphore to be signalled. | 16:12.45 |
| When the async event completes, the .then runs, and that signals the semaphore. That wakes the main thread up and allows it to continue. | 16:13.19 |
mvrhel_laptop | Robin_Watts: ok I will look at this further | 16:17.07 |
Robin_Watts | sorry. IME busy waits always come back to bite us. | 16:17.52 |
mvrhel_laptop | I understand. | 16:18.39 |
| Robin_Watts: ok I fixed the issue. it turns out that it is very simple to do this. | 17:38.32 |
| let me do one more update | 17:38.38 |
Robin_Watts | mvrhel_laptop: Ah, fab. I was hoping it wouldn't be that bad. | 17:40.32 |
mvrhel_laptop | Robin_Watts: no it turned out to be very easy | 17:40.46 |
Robin_Watts | I'm sure this must be a very common idiom - to turn an async thing into a sync one. | 17:40.52 |
mvrhel_laptop | yes. essentially task.wait() does what we want | 17:41.10 |
| you just can't do this on the UI thread | 17:41.18 |
| but since we are already on a different thread it is fine | 17:41.27 |
Robin_Watts | Ah, nice. | 17:41.59 |
mvrhel_laptop | semaphores and mutexs are not used in the winRT concurrency code. that all happens behind the scene | 17:42.31 |
| scenes | 17:42.36 |
| when I had tried to use wait prevsiously I was on the UI thread | 17:43.05 |
| and things exploded so I never looked at it again | 17:43.19 |
| hold on will push in a sec. Robin_Watts | 17:44.23 |
| Robin_Watts: ok pushed | 17:44.56 |
| I also added code to give some UI feedback as to the print progress | 17:45.10 |
| now I just need to add in the bandding | 17:45.17 |
| banding | 17:45.27 |
| then I hope I can move on to the desktop app | 17:45.45 |
Robin_Watts | if (ComputePageSize(spatial_info_local, k, &ras_size) == 0) | 17:46.20 |
| By 0, I assume you mean S_ISOK ? | 17:46.27 |
mvrhel_laptop | yes. let me add in a real name for that | 17:46.41 |
Robin_Watts | Can you explain the: if (k < THUMB_PREADD) thing ? | 17:47.36 |
mvrhel_laptop | hmm I did it correct every place else... | 17:47.37 |
| Robin_Watts: I wish I could :) | 17:47.43 |
| so essentially what happens is this | 17:47.51 |
| if I let the thumbs all get added this fast to the flip view | 17:48.07 |
| the UI becomes very unresponsive | 17:48.20 |
| I don't understand why | 17:48.27 |
Robin_Watts | This limits you to only ever updating the first THUMB_PREADD thumbnails ? | 17:48.35 |
mvrhel_laptop | while in the rendering of the thumbnails yes | 17:48.56 |
Robin_Watts | Could you do the wait trick here too ? | 17:49.01 |
mvrhel_laptop | for some reason, once they are all rendered I can add them all in one loop | 17:49.15 |
| hmm. let me try | 17:49.21 |
Robin_Watts | If the problem is that you end up with too many things queued on the UI thread, that might explain the lumpiness. | 17:49.43 |
mvrhel_laptop | I do have to be careful here | 17:50.02 |
| as this is the main thread | 17:50.07 |
Robin_Watts | Wheras if you ensure that you only queue one thing at a time (which the wait trick should do), you might be OK? | 17:50.16 |
mvrhel_laptop | right | 17:50.30 |
| let me try a couple things | 17:50.34 |
Robin_Watts | AIUI you'd be waiting in the same thread as you're already waiting for the DataWriter ? | 17:50.40 |
mvrhel_laptop | I am trying that now... | 17:54.11 |
| ok that failed, but I suspect I know what is going on here and it has to do with the communication about binding updates | 17:56.14 |
| I think I have a way around this | 17:56.26 |
| Robin_Watts: ok I was able to fix this | 18:10.09 |
Robin_Watts | cool. | 18:10.16 |
mvrhel_laptop | Robin_Watts: it is funny how you pointing these things out to me inspired me to figure them out. I fooled with this for a long time way back | 18:10.40 |
| and got tired of it and so put a hack in | 18:10.51 |
| perhaps I know a bit more now then I did then though | 18:11.06 |
| with the winRT stuff | 18:11.14 |
| let me clean this up then I will push again | 18:11.22 |
Robin_Watts | mvrhel_laptop: yeah, code review is really good for this stuff. | 18:11.35 |
mvrhel_laptop | actually I need to head out the door. Robin_Watts | 18:11.44 |
Robin_Watts | It's the 'cardboard programmer' thing, as much as anything else. | 18:11.47 |
| no worries. | 18:11.49 |
mvrhel_laptop | well thanks. I think things will be much cleaner and clearer now | 18:12.49 |
| bbiaw. | 18:12.59 |
Robin_Watts | tor8: 4 reviews on robin/master then (ignore the last 3 as usual) | 18:16.27 |
| Those 4 include one to do fz_rebind_stream. | 18:18.41 |
| We could leave that one out, and substitute in the stream de-contexting one instead. | 18:19.05 |
| But my preference would be for the rebinding one. | 18:19.21 |
| It's not much complexity, and it removes the need for lots of param passing. | 18:19.48 |
| tor8: 5 patches now. | 18:47.13 |
| kens, tor8, chrisl: Just looking at flights for the March meeting. | 19:06.00 |
| Assuming it's in SFO, I am tempted by UA931 out, and UA930 back. 777-200 with power and VOD etc. | 19:06.55 |
| cost = 667.85. | 19:08.20 |
| Cost for the equivalent virgin flight? 667.85. Price fixing on airlines? Surely not... | 19:08.42 |
kens | :-) | 19:08.49 |
Robin_Watts | PE upgrades look like $124 or so, and you can get exit rows ($144) | 19:13.46 |
kens | So exit rows cost more than PE ? | 19:20.01 |
| WHat's the in-flight entertainment like ? Did it work, is there a decent choice ? | 19:21.08 |
Robin_Watts | inflight entertainment was great. | 19:24.46 |
| at least, the screens looked really nice, and the choice was good. I didn't actually watch anything. | 19:25.02 |
| helen did though. | 19:25.06 |
kens | seat back power doesn't bother me, new laptop lasts ages | 19:25.31 |
Robin_Watts | For PE, you pick the seat you want, and they have different prices. | 19:25.52 |
| seat in the middle = $109. | 19:25.59 |
kens | ick | 19:26.03 |
Robin_Watts | seat on the aisle = $124 | 19:26.07 |
| window seat = $133. | 19:26.19 |
kens | lol would rather have aisle anyway | 19:26.27 |
Robin_Watts | exit row aisle seat = $144 | 19:26.31 |
kens | hmmm | 19:26.39 |
| ability to choose is nice, but that's quite a bit more than virgin for leg room | 19:26.54 |
| Or is that both directions ? | 19:27.00 |
Robin_Watts | No, that's one direction. | 19:27.08 |
| my laptop doesn't last the flight. | 19:27.32 |
| but it would if I watched a film or read a book for a bit. | 19:27.46 |
kens | bought 2 batteries | 19:27.59 |
Robin_Watts | Alas, you need to remove at least 8 screws to change my battery. | 19:28.28 |
kens | :( | 19:28.40 |
| Forward 1 day (to 2013/12/14)>>> | |