| | <<<Back 1 day (to 2011/09/01) | 2011/09/02 |
| henrys | my x6 will be offline for a bit. | 03:38.25 |
| mvrhel2 | sleep time | 06:08.30 |
| Robin_Watts | kens: Does your virgin atlantic booking show your frequent flier number? | 12:10.53 |
| | chrisl: I bet yours does, but kens doesn't. | 12:11.03 |
| kens | Not sure, wait one | 12:11.07 |
| | Robin_Watts : my booking confirmation has my number on it. | 12:12.42 |
| Robin_Watts | But when you look online? | 12:12.53 |
| kens | Doing that now | 12:12.59 |
| Robin_Watts | The form I get when I "retrieve booking" doesn't list a frequent flier number, nor does it let me enter one. | 12:13.35 |
| kens | Odd. | 12:13.40 |
| Robin_Watts | I guess that's because we spent miles to buy the upgrade. | 12:13.47 |
| kens | Oh yes, you don't get the air miles and tier points if yo udo that. | 12:14.14 |
| | I certainly can't see anything on the 'My Booking' page | 12:14.38 |
| Robin_Watts | OK, sounds like it's the same for the both of us, and I won't worry :) | 12:15.06 |
| kens | The slot is empty and I can't enter anything htere | 12:15.12 |
| Robin_Watts | same as me then. | 12:15.34 |
| kens | Fine, not a problem I don't think. | 12:15.42 |
| | :-) | 12:15.45 |
| Robin_Watts | Was just checking I'd entered all the advanced passenger details rubbish. | 12:16.07 |
| kens | Well, my number is on my booking refernece, just not on the form there. | 12:16.27 |
| | Its on the email that is. So they obviously know about it. | 12:16.38 |
| chrisl | Robin_Watts: (after the usual several minutes' panic about my user name is.....) yes, my frequent flier number is listed with the flught | 12:22.43 |
| | s/flught/flight | 12:22.55 |
| Robin_Watts | OK, so that would seem to confirm the theory then. | 12:23.03 |
| ray_droid | our power's out. is peeves down ? | 13:47.25 |
| Robin_Watts | tries | 13:47.54 |
| | I'm in. | 13:48.06 |
| | So, no, peeves would appear to be up. | 13:48.24 |
| ray_droid | interesting. so it's very localized since peeves is only 300 m from home. | 13:49.45 |
| Robin_Watts | different substation? | 13:50.02 |
| kens | Can happen we had one side of the street fgo out once | 13:50.03 |
| Robin_Watts | The next road over is on a different substation from us, for example. | 13:50.46 |
| ray_droid | the local power company web site doesn't provide a map -- and no estimate :-( | 13:51.19 |
| Robin_Watts | How long has power been out for ? Substations here attempt to reset themselves automatically after a few minutes. | 13:51.34 |
| ray_droid | it's only been down about 90 min. | 13:52.29 |
| Robin_Watts | oh, right, that's way longer than the auto reset time. | 13:52.57 |
| ray_droid | the web site says "equipment failure". | 13:53.27 |
| | it's been long enough that the UPS on peeves would be dead by now. | 14:03.35 |
| henrys | henrysx6 should be back in a few minutes sorry for the downtime | 14:43.42 |
| kens | No problem | 14:44.02 |
| henrys | kens:do I have all the message from adobe or did you delete some? | 14:54.12 |
| kens | No, that's all I ahve. I think I have a fix, I'm regressi0n testing ti now | 14:54.27 |
| henrys | oh good then we don't have to talk about it in Chicago ;-) | 14:54.49 |
| kens | I hope not | 14:54.54 |
| | Its a bit hacky, but if it works it'll do for now at least | 14:55.09 |
| | Great, no differences in the cluster test. Just need to run a couple more PCL search tests. | 15:04.11 |
| Robin_Watts | henrys: I did some more tests on the jpeg data changes. | 15:09.44 |
| | I hit problems in the pxl stuff, due to me setting buffersize incorrectly. | 15:10.08 |
| | but I think it showed up problems in the pxl jpeg codes handling of errors. | 15:10.26 |
| | so I have a fix as part of my patch. | 15:10.37 |
| | http://ghostscript.com/cgi-bin/clustermonitor.cgi?report=robin <- patch is at the end of that. | 15:10.54 |
| | The changes are in pxlimage.c | 15:11.25 |
| chrisl | Robin_Watts: I think there is a bug in libjpeg with this, because even if you use the non-suspending interface, you don't have information in your fill_input_buffer() call to handle a stream like the one in this case. | 15:13.52 |
| henrys | I really don't want these changes to mask a libjpeg problem best to fork jpeg we'll send them a patch. | 15:20.49 |
| | the mac migration assistant has changed all my timestamps - this is insane. | 15:28.02 |
| | Robin_Watts, chrisl:can we fix the jpeg library or change the api such that we don't need these changes? | 15:37.05 |
| chrisl | I don't know, I didn't really look at it enough - but I think we should fix the library, rather than hack our interface. | 15:38.23 |
| henrys | maybe chrisl can have a look as Robin_Watts is wrapped up in planar. | 16:00.59 |
| chrisl | Okay. I'll have a look next week | 16:02.08 |
| henrys | great we don't really have owners for third party libraries. | 16:02.46 |
| | maybe we can just report a bug but that seems unlikely. | 16:03.20 |
| chrisl | I'll try to sound out the libjpeg people about it | 16:04.15 |
| Robin_Watts | Sorry, was seeing parents. | 16:09.11 |
| | With the non-suspending interface, the lib will cope just fine, because we'll just pull more data in. | 16:09.33 |
| | It's only the suspending interface that's a problem. | 16:10.02 |
| chrisl | You still have to refill the input buffer | 16:10.19 |
| Robin_Watts | Yes. | 16:10.24 |
| | But that's fine - if you're not suspending, then you just suck more data in. | 16:10.40 |
| | refilling the input buffer is the normal case. | 16:10.54 |
| | It's only when you *can't* refill the input buffer that jpeg suspends. | 16:11.12 |
| chrisl | But you don't where you have to refill the buffer from | 16:11.17 |
| Robin_Watts | chrisl: Eh? The whole point of the jpeg data source stuff is that it can refill the buffer from a place of your choosing. | 16:11.46 |
| | It's because we can't (or we have chosen not to have ghostscript be able to) refill the buffer that we have to use the suspending interface, and hence have the problem. | 16:12.33 |
| chrisl | When the fill_input_buffer() is called, you get your input back, with the valid JPEG bytes still at the beginning, and you have to leave those in place. | 16:13.01 |
| Robin_Watts | chrisl: Why do we have to leave them in place? | 16:13.53 |
| | We'll have consumed them. | 16:13.59 |
| chrisl | So, why "unconsume" them during a suspend? | 16:14.29 |
| Robin_Watts | Because the jpeg lib can only be safely left at MCU line boundaries. | 16:15.25 |
| | When you call the jpeg lib, it will never return part way through decoding the data required for a scanline. | 16:16.04 |
| | It might call back to you to get you to refill the buffer so it can continue, but it will never *return*. | 16:16.26 |
| | If you fail to be able to refill the buffer, then it has no choice but to backtrack to the start of the last line, and then return. | 16:16.49 |
| chrisl | Okay, in that case, surely the call to fill_input_buffer() should have access to the number of bytes in the stream that have been decoded | 16:16.51 |
| Robin_Watts | (that's a suspension) | 16:16.56 |
| | Probably does, why ? | 16:17.25 |
| chrisl | Well, then we can pass that back to GS, and we'll know from where in the buffer to start filling with new data | 16:17.55 |
| kens | henrys, marcosw_ I believe the commit I just pushed will 'fix' the PCL type 3 font search problem in Acrobat. It seems OK with the two test files I have from teh customer and shows no regressions. | 16:17.58 |
| marcosw_ | kens: was the latest Adobe support message was informative? | 16:18.27 |
| Robin_Watts | chrisl: If gs wasn't getting its data from postscript, sure. | 16:18.30 |
| kens | marcosw_ Yes, they finally supplied the important information, its the FontBBox that causes Acrobat a problem. | 16:18.49 |
| Robin_Watts | The problem is that we construct a buffers worth of data using postscript, then call the jpeg lib, and that might call fill_input_buffer, and we can't at that stage run a load more postscript to get us another buffer full. | 16:19.17 |
| marcosw_ | will you let the customer know or should I email them the patch? | 16:19.20 |
| Robin_Watts | That's why we are forced into using the suspending interface. | 16:19.27 |
| kens | marcosw_ I was updating the bug, assuming you would contact them with teh fix as normal ? | 16:19.38 |
| marcosw_ | works for me, just making sure. | 16:19.54 |
| kens | No problem, just marked the bug resolved/fixed | 16:20.23 |
| chrisl | Robin_Watts: but in fill_input_buffer() we could record the current buffer state, and use it when we get to the GS stream layer | 16:20.25 |
| Robin_Watts | chrisl: having access to the gs data is not the problem. | 16:21.15 |
| chrisl | So what is? | 16:21.25 |
| henrys | marcosw_ did you use xcode 4 on your last mac update? | 16:21.33 |
| Robin_Watts | it's the fact that to run more postscript we need to exit the call to the jpeg code. | 16:21.56 |
| chrisl | So? | 16:22.06 |
| Robin_Watts | OK. Let's run through this. | 16:22.40 |
| | We run some postscript, and it builds a buffer full of data in the stream. | 16:22.53 |
| | We get a postscript operator (I forget which one) that causes us to call into the jpeg decoder code with that buffer full of data. | 16:23.24 |
| marcosw_ | henrys: yes, I updated Xcode when I moved to Lion. Other than the one issue that I found and fixed there weren't any problems. | 16:23.41 |
| Robin_Watts | The jpeg decoder starts to read through that buffer full of data decoding a scanline, and while it's doing so, it hits the end of the data before it finishes the scanline. | 16:24.09 |
| | It therefore calls the 'fill_input_buffer' method, which is in our code. | 16:24.30 |
| | The choices we have here are to somehow produce more data for the lib to use, or to return to the jpeg code with no more data - in which case it will suspend. | 16:25.42 |
| chrisl | Yes | 16:25.52 |
| Robin_Watts | We can't produce more data for the lib to use, because to do so we'd need to run more postscript operators. | 16:26.04 |
| chrisl | Again, yes | 16:26.22 |
| Robin_Watts | So, we have to take the second option, which causes the jpeg code to rewind back to the start of the data for the scanline, and exit back to us. | 16:27.10 |
| | [A]bort, [R]etry, [C]ancel ? | 16:28.32 |
| chrisl | So, in fill_input_buffer, has the buffer pointer been incremented to the last decoded byte in the stream? | 16:29.29 |
| Robin_Watts | I believe so. | 16:29.45 |
| chrisl | Okay, so we can record that in there, and then in s_DCTD_process() we can use that information to know from where to write new data into the buffer | 16:30.09 |
| Robin_Watts | chrisl: What new data can we write? | 16:30.21 |
| | The problem is that the buffer is a fixed size, and it's already full. | 16:30.30 |
| chrisl | But I thought the problem was a large chunk of the buffer was filled with padding bytes? | 16:30.58 |
| ray_laptop | Robin_Watts: I thought you were compacting out the FF's ? | 16:31.02 |
| Robin_Watts | Ah, right, yes. I *do* throw away padding bytes. | 16:31.19 |
| | I've written that code, and it works. | 16:31.26 |
| ray_laptop | so what is the buffer full with ? | 16:31.44 |
| | (that can't be consumed by libjpeg) | 16:32.07 |
| Robin_Watts | ray_laptop: for the sake of argument, let's say it's real valid data. | 16:32.17 |
| chrisl | Robin_Watts: So, instead of your code having to fathom what are padding bytes and what is valid data, can't we get that information from libjpeg at the fill_input_buffer call? | 16:32.32 |
| Robin_Watts | chrisl: We could, but honestly, the code to dump padding bytes is pretty trivial; basically you replace any stream of 0xFF bytes with a single one. | 16:33.11 |
| ray_laptop | so previously we've never had a case where one image row takes up more than 2048 bytes ? that's hard to imagine | 16:33.33 |
| marcosw_ | henrys: btw, why was henrysx6 down for a day? hardware trouble? | 16:33.54 |
| chrisl | Robin_Watts: So, the real solution is to allow the buffer the grow. | 16:34.12 |
| henrys | no I needed the cable for something else no failures. | 16:34.26 |
| Robin_Watts | ray_laptop: It's possible that the 0x800 figure is magically generated from somewhere, but I don't understand the stream init voodoo well enough. | 16:34.30 |
| henrys | doesn't printing partial scanlines also solve this? | 16:34.46 |
| Robin_Watts | henrys seems to object to my patch on the grounds that it's a bug in the lib. | 16:34.50 |
| | I claim it's NOT a bug in the lib. | 16:34.58 |
| | The rules for using the lib are, I believe, "if you call the suspending interface, you must be capable of providing a buffer large enough to fit your largest scanlines data in". | 16:35.37 |
| chrisl | Well, I'd agree with henrys: "restart from the beginning" is not the same as "suspend state" | 16:35.45 |
| Robin_Watts | It suspends to the start of the last MCU. | 16:36.04 |
| | MCU = Minimal Coded Unit. | 16:36.10 |
| | (or maybe 'line of MCUs', I forget the definition exactly). | 16:36.37 |
| chrisl | I know, but to me, suspend means save your current state, not save your state from a few cycles ago | 16:36.46 |
| Robin_Watts | I'm sure I could find examples from inside gs where we do the same kind of thing :) | 16:37.12 |
| chrisl | Unless there is no practical choice, then it would be a bug | 16:37.33 |
| Robin_Watts | At any rate, I don't believe it's a libjpeg bug, and I'm sure the maintainers will feel the same way. | 16:37.44 |
| | but if you want to raise it with them, feel free. | 16:37.53 |
| | The change I asked henrys to look at earlier was a change in pximage.c. | 16:38.11 |
| | It's not specifically related to this particular problem. | 16:38.24 |
| chrisl | Robin_Watts: So, can we make the buffer expand as required? | 16:38.35 |
| Robin_Watts | chrisl: That's what I asked ray_laptop yesterday. | 16:38.51 |
| ray_laptop | Robin_Watts: the s_DCTD_template (in base/sdctd.c) defines minimum buffer size for JPEG input processing as 2000 | 16:38.53 |
| Robin_Watts | He said "expanding the buffer is "not done"", but might be possible, he'd have to check the invariants. | 16:39.19 |
| henrys | Robin_Watts:I didn't completely understand the change so I couldn't review it. I would like our interface to be as simple as possible it seemed to diverge from that general goal. | 16:39.50 |
| Robin_Watts | The change in pximage.c is a generic one to cope with the fact that the code there can't cope with error codes being passed back. | 16:40.03 |
| | henrys on line 219ish, it does: code = (*s_DCTD_template.process)((stream_state *)ss, &r, &w, false); | 16:40.34 |
| chrisl | Robin_Watts: I'd figured we could spot when the situation arose and have an input buffer private to the DCTD stream | 16:41.06 |
| Robin_Watts | The return value of code will be: 0 (for no lines decoded, suspended cos needs more data), 1 (for decoded a line), -ve for an error. | 16:41.25 |
| henrys | the first part of the patch I read is a long comment about compacting the data buffer how is that a generic change to handle an error, am I looking at the wrong code? | 16:41.52 |
| Robin_Watts | henrys: You're supposed to be looking at pximage.c | 16:42.09 |
| henrys | oh I'm sorry I was confused. | 16:43.01 |
| | that part of the patch seems fine. | 16:43.29 |
| Robin_Watts | henrys: Right. | 16:43.41 |
| | The top part of the patch is an attempt to avoid us getting into an infinite loop. | 16:43.58 |
| | Need to backtrack a bit to explain it. | 16:44.22 |
| | Yesterday I committed some code, that spotted the jpeg decoder returning without producing any data (i.e. suspending). | 16:44.56 |
| | When it spotted that, it would compact the buffer, removing any padding bytes. | 16:45.16 |
| | That was enough to solve the bug. | 16:45.26 |
| | but it still left us in the situation where a scanline that genuinely took more than a bufferfull to fit would go into an infinite loop. | 16:46.02 |
| | So the top part of todays patch addresses that. | 16:46.23 |
| henrys | okay, but to really solve this we have to expand the buffer right?d We have to have enough memory for a scanline of data. We can still create job perhaps pathological that won't work | 16:47.05 |
| ray_laptop | well, returning an error if we can't compact and nothing was consumed before seems OK. | 16:47.24 |
| Robin_Watts | If we spot a suspension, and the buffer was as full as it was ever going to be, and we can't compact the buffer... then we give an error. | 16:47.29 |
| | i.e. we'll fail to decode, but we won't go into an infinite loop. | 16:47.49 |
| | Yes, a *real* solution to this requires us to expand the buffer. | 16:48.00 |
| ray_laptop | Robin_Watts: right -- just bubbles up an ioerror | 16:48.07 |
| Robin_Watts | right. | 16:48.12 |
| | and when I wrote that code I spotted a bug in the pxl stuff whereby it was swallowing errors, so I fixed that. | 16:48.43 |
| henrys | the scanline buffer could be allocated whenever the framebuffer changes right? So we'd have a max? | 16:49.58 |
| | but I guess it could be clipped. | 16:50.30 |
| Robin_Watts | The input buffer ? | 16:50.30 |
| chrisl | So, could we have a separate, variable sized libjpeg input buffer in the DCTD stream state which gets used iff we cannot get a scanline out with the normal stream buffer size? | 16:51.07 |
| Robin_Watts | As I understand it (and I could be wrong), the streams buffer is allocated when the stream is first opened. | 16:51.27 |
| ray_laptop | Robin_Watts: but the s_DCTD_process is called with pointers. It can update the 'ptr' of the cursor, but can't move the buffer (it's not supposed to touch the 'limit') | 16:51.32 |
| Robin_Watts | i.e. it's allocated before we've read enough of the stream to even know the size of the bitmap, so we can't estimate the required buffer size. | 16:52.12 |
| chrisl | Robin_Watts: I mean a completely separate buffer that we'd copy data into from the stream input buffer | 16:52.18 |
| Robin_Watts | ray_laptop: Where do I touch the limit ? | 16:52.33 |
| | chrisl: Yes, we could introduce a new buffer that we can copy into and realloc larger in a layer just above the jpeg code. | 16:53.20 |
| chrisl | Robin_Watts: which we would only use when we hit this type of case (so we would rarely have the data copying overhead) | 16:53.58 |
| Robin_Watts | chrisl: yes. | 16:54.05 |
| chrisl | That seems to me the best/safest solution | 16:54.23 |
| Robin_Watts | Right, so we'd need to detect *when* to start using that buffer. | 16:55.19 |
| henrys | how much would the input buffer need to be increased to get us through the problems we currently have? | 16:55.23 |
| | not synthesized problems. | 16:55.37 |
| Robin_Watts | and the logic for such a detection is basically what's in my patch now. | 16:55.41 |
| chrisl | Robin_Watts: yes, exactly what I was thinking | 16:55.55 |
| Robin_Watts | henrys: With the patch I committed yesterday, we have no known files that fail. | 16:56.01 |
| | but I could construct one that went into an infinite loop. | 16:56.15 |
| | With the patch I'm testing, that infinite loop would be averted. | 16:56.35 |
| ray_laptop | averted by returning ERRC, right ? | 16:56.55 |
| henrys | it should be right not just working... | 16:57.15 |
| Robin_Watts | If we then find a real file that triggers this, we can investigate chrisl's solution - but it would, I believe, require the logic in my new patch anyway. | 16:57.33 |
| | ray_laptop: Yes, just by returning ERRC. | 16:57.45 |
| chrisl | Robin_Watts: what sparked this (sorry!) was that it looked to me as though libjpeg was resetting the buffer pointers *before* calling fill_input_buffer - which I felt would be a bug. | 16:57.51 |
| Robin_Watts | (The trick is in knowing when to hit ERRC) | 16:58.00 |
| | s/hit/return/ | 16:58.12 |
| | chrisl: Right. I don't *think* it does that. | 16:58.40 |
| | henrys: "it should be right not just working..." can you explain that please ? | 16:59.23 |
| | Oh, the code should be correct, not just functional. Sorry. | 17:00.10 |
| henrys | yes | 17:00.36 |
| Robin_Watts | sorry, brain failure. | 17:00.43 |
| henrys | invariant gries all that stuff. | 17:01.01 |
| Robin_Watts | Well, yes, it would be nice if the code could cope with every case. | 17:01.19 |
| | but to do that, we either need to grow the stream buffer, or implement chrisl's suggestion. | 17:01.45 |
| henrys | I imagine 99% of clients grow the buffer... | 17:02.27 |
| Robin_Watts | Either way, we need all my current patch (and we'd change the 'return ERRC;' to be 'grow the buffer'). | 17:02.32 |
| | I'd imagine that 99% of clients don't call the suspending interface :) | 17:02.46 |
| ray_laptop | the separate buffer that would be in the DCTD state would avoid touching the stream logic (that is rather frail) | 17:02.50 |
| | expanding that buffer if needed would be perfectly legal. | 17:03.22 |
| henrys | so expanding the buffer would be the most expected thing but if it requires screwing with the stream code there will be horror. | 17:06.16 |
| ray_laptop | It's not that zapping the cursor pointers and the cbuf and bsize and cbsize isn't possible -- I'm just worried that there's code that won't expect the buffer to move and will have a pointer laying around that we can't get to that "knows" where the buffer was before it calls the 'process' to read more data | 17:06.25 |
| | but chrisl's suggestion of having the DCTD have its own buffer (that can be realloc'ed bigger) is clean (if less convenient) | 17:07.16 |
| | Note that the jasper JPX decode has that kind of thing already | 17:08.11 |
| | the jasper interface code is rather ugly because it doubles the size of the buffer, copies the data over to the new buffer, then frees the old one (or did last time I looked at it). | 17:09.39 |
| | not that jasper is fast anyway, but that method of buffering the entire JPX stream doesn't help. | 17:10.08 |
| Robin_Watts | Just doing some back of the envelope sums here... | 17:11.01 |
| chrisl | I wouldn't be very worried about the speed of the scheme, as we'd only use it in DCT for these (extremely rare!) pathological cases - it wouldn't be a normal mode of operation.# | 17:11.11 |
| Robin_Watts | I've got an 18MP camera, that takes 4Meg images. | 17:11.16 |
| | That's about 1.8 bits per pixel. | 17:12.12 |
| ray_laptop | I've got a letter size scanner that does 9600dpi ! 11" at 9600dpi would probably stress the 2000 byte limit | 17:12.31 |
| Robin_Watts | A scanline would therefore be 5000*1.8 bits per component and 16 scanlines encoded together suggests... 18K at a time. More than that if the components are interleaved... | 17:14.29 |
| | that can't be correct, as how could we be working? | 17:14.40 |
| chrisl | Eek, Pentax medium format camera is 40MP and can save to JPEG......... | 17:15.31 |
| ray_laptop | even 1200 dpi across a 36 inch wide printer is 40k pixels, so at .2 bytes per pixel would be 8,600 bytes | 17:16.11 |
| | Robin_Watts: your numbers (4 M bytes for a 18MP image) is .22 bytes per pixel | 17:17.01 |
| Robin_Watts | 4*8/18 = 1.77777 ? | 17:18.08 |
| ray_laptop | Robin_Watts: why the * 8 ? if you want bytes per pixel it is 4/18 | 17:19.25 |
| Robin_Watts | I was working in bits per pixel. | 17:19.35 |
| ray_laptop | 1.77 bits per pixel | 17:19.45 |
| Robin_Watts | Oh, right, sorry, we had the same results in different units. | 17:20.02 |
| ray_laptop | I see, so our buffer is 2000*8 bits | 17:20.14 |
| henrys | we probably need available memory to limit the buffer. resolutions are increasing all the time. | 17:20.29 |
| Robin_Watts | We could just decide not to worry until this actually presents itself as a problem. | 17:21.14 |
| ray_laptop | henrys: if the DCTD filter state has its own bufer it would just use alloc until it failed (then free it and return ioerror) | 17:21.35 |
| Robin_Watts | If we commit my new patch, we'd at least be 'unexploitable'. | 17:21.45 |
| | and should we find a file where it's a problem, we can worry about it then. | 17:22.10 |
| ray_laptop | Robin_Watts: right. I have no objection to returning the ERRC when we 'stall' | 17:22.12 |
| henrys | ray_laptop: that's what I thought the suggested solution entailed. | 17:22.30 |
| Robin_Watts | ray_laptop: The problem is in the technique I use to know that it's OK to return ERRC. | 17:22.33 |
| | In order to know it's OK to return ERRC, I have to know that: | 17:23.00 |
| chrisl | henrys: I'd want to allow for an explicit limit, for embedded applications. | 17:23.09 |
| Robin_Watts | 1) the jpeg lib exited without decoding any lines. | 17:23.16 |
| | 2) the buffer was as full as it can be | 17:23.29 |
| | 3) the compaction didn't save anything. | 17:23.42 |
| | 1 and 3 are easy. | 17:23.46 |
| | 2 is the tricky one. | 17:23.50 |
| ray_laptop | I thought that was Robin's patch (including the pxl change to handle error returns), but I thought chrisl's idea was for the separate buffer in the DCTD state | 17:24.15 |
| Robin_Watts | I've basically had to pass cbsize down by putting it into a new 'buffersize' variable. | 17:24.19 |
| | ray_laptop: Yes. | 17:24.36 |
| | The way of knowing what the buffersize value should be uses a bit of postscript interpreter magic, that I'd like blessed by someone with more knowledge than me. | 17:25.15 |
| ray_laptop | Robin_Watts: buffer full is known, right ? (you have ptr and limit and cbuf and cbsize) | 17:25.16 |
| Robin_Watts | ray_laptop: No. At the point at which I'm detecting this, I have ptr and limit. | 17:25.33 |
| | no cbuf and cbsize. | 17:25.43 |
| | (Look in sdctd.c at around line 274 where I call compact_jpeg_buffer in the current source) | 17:26.42 |
| | If you could find me cbsize there I'd be very pleased! :) | 17:27.07 |
| ray_laptop | Robin_Watts: the s_DCTD_process has the stream_state which includes the min_in_size (that isn't cbsize, but should suffice) | 17:28.39 |
| Robin_Watts | ray_laptop: In what way? | 17:28.52 |
| | min_in_size is a completely different value. | 17:29.08 |
| ray_laptop | you know that the bsize is at least as large as min_in_size (see s_init_filter -- it fails if the bsize < min_in_size) | 17:30.15 |
| henrys | bbiab | 17:30.30 |
| Robin_Watts | ray_laptop: So you're saying I should test for there being at least min_in_size bytes in the buffer? | 17:31.14 |
| ray_laptop | Robin_Watts: and s_add_filter allocates bsize = max(template->min_in_size, 256); | 17:31.28 |
| Robin_Watts | rather than 'as full as it will ever be'. | 17:31.34 |
| | min_in_size = 0x7d0, cbsize = 0x800. | 17:31.56 |
| | That would make my patch MUCH simpler and less invasive. I like that. | 17:32.10 |
| | OK, I'll test that out, thanks. | 17:32.40 |
| chrisl | Robin_Watts: are you okay time-wise to see this DCT stuff through? | 17:33.25 |
| Robin_Watts | Unless I hit massive issues, it should go in this evening. | 17:34.02 |
| | marcosw_: How did the last plank comparison run end up? | 17:34.26 |
| chrisl | Okay, good. I'm off, then - evening all! | 17:34.45 |
| Robin_Watts | I think (famous last words) the planar stuff should be almost done. | 17:34.53 |
| marcosw_ | I sent out an email this morning, there were still tens of differences | 17:34.54 |
| Robin_Watts | chrisl_away: night. | 17:34.58 |
| | marcosw_: tens of differences sounds perfect. | 17:35.06 |
| marcosw_ | okay then. | 17:35.15 |
| Robin_Watts | Can we run tests that include pcl etc please? | 17:35.29 |
| marcosw_ | The exact number of differences was 73. I can run the next test with the pal code, at 600 dpi only? | 17:37.31 |
| Robin_Watts | henrys: ? | 17:38.00 |
| | (henrys was the one asking for this - 600dpi only sounds good to me) | 17:38.20 |
| marcosw_ | Robin_Watts: btw, I'm currently not testing the pdfwrite/ps2write devices with the plank/palcmyk4 testing. Should I be? | 17:41.06 |
| Robin_Watts | I wouldn't have thought that should be a problem personally. | 17:41.47 |
| | Are there peculiarities of ps/pdfwritten files that we don't see elsewhere in the test suite? | 17:42.36 |
| marcosw_ | occasionally we run into a regression where we produce a file that we can't read, but usually there aren't bitmap differences that don't show up converting the file directly. | 17:44.52 |
| Robin_Watts | Then, no, I'd have thought we'd be fine without that. | 17:45.10 |
| henrys | yes 600 dpi only is fine. | 17:50.30 |
| marcosw_ | okay, will have results tomorrow morning my time (or late tonight, if I end up staying up late). | 17:53.10 |
| Robin_Watts | marcosw_: Thanks. | 17:53.29 |
| | I'm sure I had some really large format jpegs here... | 17:56.50 |
| ray_laptop | Robin_Watts: just do a tiger to ppm at 2400 dpi out to -sDEVICE=jpeg | 18:07.29 |
| | or higher res | 18:07.38 |
| Robin_Watts | tiger will compress way too well. | 18:08.04 |
| | Aha. 20000 pixels across. this should do. | 18:10.42 |
| ray_laptop | the biggest that I have laying around is 3780 by 2963 | 18:10.46 |
| mvrhel2 | bbiab | 18:20.51 |
| Robin_Watts | ooh gents, it appears I have been doing the jpeg lib a disservice. | 18:24.58 |
| ray_laptop | Robin_Watts: how so ? | 18:25.27 |
| Robin_Watts | It doesn't rewind to the beginning of the scanline, just to the beginning of the MCU. | 18:25.27 |
| | Hence the required size of the buffer is much lower. | 18:25.39 |
| ray_laptop | Oh, OK then that's MUCH better ! | 18:25.54 |
| Robin_Watts | (but I could still make a JPEG file with it larger than 0x800 bytes if I really tried I think!) | 18:25.59 |
| ray_laptop | what's the limit on MCU size (16x16 is all I've ever seen) ? | 18:28.10 |
| Robin_Watts | I believe I can generate larger than that. | 18:28.51 |
| | If I send Y U and V interleaved, and have U and V at 1/4 of the resolution of Y, then you'd get 32x32. | 18:29.37 |
| | I could do something wacky and send U as 1/3 and V as a 1/4 and it'd be 96*96 or something, maybe | 18:30.27 |
| | Hmm. | 18:40.54 |
| | I'm getting ss->template->min_in_size = 0 . | 18:41.03 |
| | ray_laptop: Ah. the pxl code doesn't use the ss->template stuff does it ? | 18:42.25 |
| ray_laptop | Robin_Watts: the template is required | 18:42.51 |
| Robin_Watts | ray_laptop: The pximage.c code doesn't set ss->template | 18:44.20 |
| | It *calls* things in the s_DCTD_template | 18:44.39 |
| | but it never sets ss->template | 18:44.46 |
| ray_laptop | Robin_Watts: I made a jpeg that was from the original I mentioned above that is 21000x2600 pixels that is a 44Mb jpeg file (just in case you want it) | 18:46.11 |
| Robin_Watts | ray_laptop: I have a 20000x2048 one here as a PDF that works. Only 6.4 Meg though. | 18:46.55 |
| | henrys: you here? | 18:47.02 |
| | Setting ss->template seems to solve it. | 18:51.24 |
| ray_laptop | Robin_Watts: it's defined in stream_state_common, so it _should_ be set to the template for the filter type. Probably worked so never got looked into. | 18:52.03 |
| Robin_Watts | Tests worked for all except pcl, so I'll retest that with the fix here. | 18:52.46 |
| | if it works, I'll commit it. | 18:52.52 |
| ray_laptop | pximage.c set it into the 'jddp' but didn't bother to set it for the stream_state | 18:53.07 |
| Robin_Watts | yeah. | 18:53.16 |
| ray_laptop | I'd recommend setting it before ANY of the calls (before the set_defaults) | 18:53.58 |
| | but that's up to you. | 18:54.43 |
| Robin_Watts | have done. | 18:54.53 |
| ray_laptop | why is there a 'xps' directory in the mupdf-0.8.165 source tarball ? | 19:14.55 |
| Robin_Watts | ray_laptop: muxps, innit. | 19:48.40 |
| kens | Robin_Watts : your last commit seems to cause a seg fault | 19:53.43 |
| Robin_Watts | kens: Damn. | 19:54.16 |
| kens | Its the output of ps2write, sent back to GS which then seg faults. | 19:54.35 |
| | I haven't looked at it yet. | 19:54.48 |
| Robin_Watts | That didn't happen when I clusterpushed earlier. | 19:55.02 |
| kens | Its not an indeterminism I've ever seen before | 19:55.15 |
| | tests_private/comparefiles/tpc2.ps.ps.pkmraw.300.0 macpro Seg_Fault_reading_Ghostscript_produced_PDF/PS_File | 19:55.44 |
| | Its on macpro so it oucld be specific to the Mac, maybe | 19:55.56 |
| | Which would explain why you didn#'t see it before. | 19:56.05 |
| Robin_Watts | SEGVs here too :( | 19:57.49 |
| kens | Not on a Mac presumably | 19:57.57 |
| | ? | 19:58.02 |
| Robin_Watts | on my 32bit windows box. | 19:58.05 |
| kens | Well, I guess I can look at it tomorrow | 19:58.15 |
| Robin_Watts | Crashing in freetype. | 20:01.20 |
| kens | THat's odd... | 20:01.27 |
| Robin_Watts | In a free operation. | 20:01.37 |
| | Maybe a memory overwrite, but I don't see how I could have caused that (famous last words) | 20:01.51 |
| | I'll memento it. | 20:01.58 |
| kens | Its hard to see how your change could have affected it. | 20:02.14 |
| Robin_Watts | ...tomorrow :) | 20:03.55 |
| mvrhel2 | alexcher: are you around? | 21:33.08 |
| | henrys: if it is ok, I am going to take over bug 692323 | 21:33.57 |
| | It basically involves doing the color managment before interpolation | 21:34.24 |
| | to reduce the number of calls to the color interpolation | 21:34.44 |
| alexcher | mvrhel2: yes | 22:03.51 |
| mvrhel2 | alexcher: is it fine if I assign 692323 to myself? | 22:05.56 |
| alexcher | mvrhel2: I'm looking into LZW bugs now and don't have much to say about 692323. | 22:06.30 |
| mvrhel2 | so is that a yes? | 22:06.43 |
| alexcher | yes | 22:06.48 |
| mvrhel2 | ok thanks | 22:06.55 |
| henrys | mvrhel2:yes that is fine | 22:23.29 |
| Robin_Watts | The SEGV looks genuinely like a freetype bug. | 23:48.27 |
| | It's allocating a data structure (of 64 bytes), and later in the code it's being cast to a 'superclass' structure (same first 64 bytes, but new stuff after it, and then it's getting confused when it tries to free the stuff that's off the end. | 23:49.48 |
| | I'll investigate more tomorrow. | 23:49.55 |
| | (Hopefully kens will read the irc logs) | 23:50.03 |
| | I don't think it's a ps2write problem. | 23:50.24 |
| | Forward 1 day (to 2011/09/03)>>> | |
IRC Logs