| | <<<Back 1 day (to 2013/11/06) | 2013/11/07 |
| Robin_Watts | ray_laptop: for the logs - result! | 00:14.01 |
| | mvrhel_laptop: printing with directX - nice. | 00:14.21 |
| mvrhel_laptop | thanks. it will work out much better as I can print page by page | 00:14.50 |
| | the xaml approach requires me to render all the pages | 00:15.08 |
| | which was dumb | 00:15.12 |
| Robin_Watts | mvrhel_laptop: I've got a small file here, that when rendered using fpng at 600dpi with NumRenderingThreads=4 SEGVs. | 00:15.53 |
| | I believe it's cos the buffer device is being closed to early. | 00:16.15 |
| | too early, sorry. | 00:16.21 |
| | and it seems to be being closed early because there is a pdf14 compositor device in the mix. | 00:16.40 |
| mvrhel_laptop | uh oh | 00:16.48 |
| orlok | Eurgh. Here i am again.. Its been 7 or 8 years. | 00:17.06 |
| Robin_Watts | I'll see if I can reproduce it with a non process_page device. | 00:17.20 |
| mvrhel_laptop | Robin_Watts: if you think it is pdf14 related you can dump it off on me | 00:18.21 |
| | although I am going to bed early tongiht | 00:18.29 |
| | I was up until 1am last night | 00:18.34 |
| | fooling with this direct X stuff | 00:18.44 |
| | at least I made some progress tonight | 00:19.09 |
| orlok | Anybody esle her eint he printing industry? | 00:20.03 |
| | Inneresting, pdf2ps on this PDF works on this new Ubuntu install , but not on my x64 CentOS6 install | 00:21.25 |
| | Can missing fonts make ghostscript shit itself? | 00:21.41 |
| Robin_Watts | orlok: It can make gs give errors. It shouldn't cause a crash. | 00:22.11 |
| orlok | unrecoverable error on CentOS, works on ubuntu | 00:23.32 |
| | I didnt create the PDF - i had just grabbed it for testing. Its the local electronics chain catalog | 00:24.29 |
| | SHould i submit it to bugzilla, or does soebody want to look at the stdout first? | 00:25.02 |
| | http://pastebin.com/t60JPw6L | 00:25.31 |
| Robin_Watts | syntaxerror, apparently. | 00:26.13 |
| | And that's gs 8.70, so only 4+ years old. | 00:26.24 |
| | Try 9.10 or the bug report will just be ignored. | 00:26.45 |
| orlok | Works on 9.10 | 00:26.58 |
| | i was just comparing versions when you said that | 00:27.04 |
| | Good enough for me, cheers! | 00:27.11 |
| | bringing back bad memories of running Distiller under Linux. Eurgh. | 00:27.44 |
| | Adobe's control scripts were taken directly from the SunOS/Solaris versions of Distiller, and were nasty ksh things | 00:28.24 |
| Robin_Watts | mvrhel_laptop: OK. I can confirm that it's not a problem caused my the fiddling with padding and alignment as it reproduces before this. | 00:53.35 |
| | but I can't (yet) make it go wrong other than in a process_fn device - but that may just be luck. | 00:54.06 |
| | and it's not related to planar mode, which is a relief. | 00:54.42 |
| | in clist_playback_band, we call close_device on target. target is a pdf14RGB device. | 01:00.05 |
| | That forwards the close to the underlying image24 device, which frees the buffer I'm still relying on. | 01:00.57 |
| | mvrhel_laptop: Do you have any memory of what's going on here? | 01:01.55 |
| ray_laptop | Robin_Watts: I get an error on my "release" build that I don't see on a debug build. The call to "pms_600x600to300x300_MTR_5C" (etc) use &buffer->params.data[0] which may not be aligned. I think this needs a method to get an aligned start address | 06:05.14 |
| | Robin_Watts: am I off base ? I'm going to bed now. | 06:05.40 |
| | Robin_Watts: BTW, I don't see a problem with a DEBUG=1 build, and it runs much faster | 06:06.28 |
| Robin_Watts | buffer->params.data[0] should always be aligned. | 10:14.10 |
| | tor7: Morning. | 10:14.16 |
| | Is there a function that does the reverse of pdf_to_utf8 ? | 10:14.34 |
| tor7 | morning robin | 10:14.36 |
| | I think the closest is pdf_from_ucs2 | 10:15.18 |
| Robin_Watts | Thanks. | 10:16.05 |
| tor7 | so feel free to add a from_utf8 | 10:16.58 |
| Robin_Watts | This is a user asking questions. | 10:17.14 |
| tor7 | I wonder how we handle authentication though, in the x11 and win apps | 10:17.15 |
| | Robin_Watts: ah, here we go: pdf_docenc_from_utf8 | 10:18.08 |
| | it's a static function in pdf-crypt.c | 10:18.22 |
| Robin_Watts | tor7: ah, thanks. | 10:19.18 |
| paulgardiner | sebras: you there? | 11:26.30 |
| sebras | paulgardiner: yo! | 11:28.46 |
| | paulgardiner: lunch here soon though. | 11:28.59 |
| | paulgardiner: ok.. catch me after lunch. | 11:29.31 |
| paulgardiner | Yes, I wondered about putting an endptr in the pdf_annot struct, but probably iterating through the list of annots is rarely going to be as time consuming as creating the appearance stream for the added annotation. | 11:30.04 |
| sebras | paulgardiner: ah. ok. I didn't have a good grasp of how many entries the list might have. | 12:35.53 |
| Robin_Watts | OK, so this clist device freeing thing; I can reproduce it with non process_page devices, but it doesn't actually cause a crash in those cases. | 13:38.55 |
| | I'll need to talk it through with mvrhel and ray later. | 13:39.07 |
| rayjj-phone | Robin_Watts: I came up with an idea. The memory device can keep a marked box per component. That way we don't have to process the entire band. | 14:37.23 |
| marcosw | Robin_Watts: a smallish box arrived for you at my house yesterday :-) | 14:54.12 |
| Robin_Watts | marcosw: Fab. Feel free to try it out if you want. | 15:00.39 |
| | rayjj-phone: A marked box is lots more work, I feel. | 15:00.56 |
| | rayjj-phone: When you get to a real computer we can talk about a problem I think I've found in the clist stuff and a solution I am testing for it. | 15:01.39 |
| mvrhel_laptop | Robin_Watts: did you figure out the pdf14 issue? | 15:02.36 |
| Robin_Watts | mvrhel_laptop: Yeah. | 15:02.59 |
| mvrhel_laptop | great | 15:03.26 |
| Robin_Watts | That's what I have to discuss with ray. | 15:03.39 |
| | It turns out that we close the device before we're done with it. | 15:03.53 |
| | It has the effect of throwing away the line_ptrs. | 15:04.07 |
| | but we get away with it currently, as the current code doesn't use the line_ptrs. | 15:04.25 |
| | It all feels a bit wrong, but... | 15:04.33 |
| | marcosw: If you do want to try it out, I will give you account details. | 15:05.00 |
| halabund | Is there a way to scale a PDF by a certain factor? For example, scaling it by 2 would make a 10 cm PDF into a 20 cm one. When displayed on screen at the same size of course they would look the same. | 15:09.11 |
| | My IRC client has died, so here it goes again: | 15:12.23 |
| | Is it possible to use ghostscript (or some other tools) to scale a PDF by a certain factor? | 15:12.42 |
| Robin_Watts | halabund: Why wouldn't you just render it at a higher resolution? | 15:14.01 |
| halabund | Robin_Watts: because I need to insert it into another document and the sizes need to match up. Scaling after inserting it is possible, but it's a tedious manual procedure. If ghostscript could do this, it could be automated and would save me a lot of trouble. | 15:15.23 |
| Robin_Watts | halabund: Ghostscript will not do what you want. | 15:16.37 |
| | It could possibly be fooled into doing so by recreating a new PDF with some horrible postscript in there to scale stuff, but it sounds like a nightmare to me. | 15:17.41 |
| kens | Robin_Watts : I htink it probably can | 15:17.48 |
| | Buit not trivially | 15:18.25 |
| halabund | if it's that complicated, then I don't want to go that way ... | 15:19.29 |
| | thanks for the answer | 15:19.33 |
| kens | Simplest way is to set the new page size, then run the PDF with -dFitPage and the pdfwrite device | 15:20.15 |
| | Of coure, as Robin says, it snot the *same* PDF, but if its scaled, it was never going to be anyway | 15:20.30 |
| Robin_Watts | kens: Not with gs. With other tools it would be quite simple to change the scale on a PDF without losing data. | 15:22.04 |
| kens | Robin_Watts : How ? | 15:22.15 |
| halabund | You know any other automatable tools? | 15:22.25 |
| kens | AFAICT you would need to rewrite all the co-ords | 15:22.47 |
| Robin_Watts | kens: Well, there is UserUnit for a start. | 15:23.03 |
| kens | Ah the horrible hack | 15:23.18 |
| Robin_Watts | Or you could fairly easily wrap all the content streams with a transform. | 15:23.35 |
| kens | Still not the same PDf anyway | 15:23.57 |
| Robin_Watts | A hell of a lot closer than gs gets. | 15:24.09 |
| kens | And you would have top be creful with patterns and forms | 15:24.13 |
| | And GS doesn't even pretend its the same PDF as you feed in | 15:26.25 |
| henrys | kens:what about redefining BeginPage to include a "scale" with -c⦠I haven't done that recently but I thought it used to work okay. | 15:26.36 |
| kens | henrys, you still have to redefine the media | 15:26.51 |
| halabund | what do you mean by not the same PDF? | 15:27.01 |
| kens | Or it falls off the page | 15:27.01 |
| halabund | it's a pdflatex-generated, tex-only pdf. if ghostscript doesn't mess up how it looks and doesn't rasterize anything, then it's okay. | 15:27.37 |
| | text-only, I mean | 15:27.49 |
| kens | halabund : pdfwrit4e does not 'manipulate' the PDF file, it creates a brand new one with none fof the original content | 15:27.53 |
| henrys | there is a tex newsgroup posting suggesting pdftops -expand -paper letter ⦠and then use ghostscript to distill back to pdf. | 15:39.06 |
| kens | Ick | 15:39.18 |
| | THat sounds like a worse idea | 15:39.28 |
| | PDF->PS->PDF :-( | 15:39.35 |
| | losing quality at every step | 15:39.41 |
| henrys | it does seem like a feature we should have though, and I wouldn't think it that complicated to do. | 15:41.14 |
| kens | I wold try what I suggested above | 15:41.24 |
| | set the required page size, run teh PDF and set -dPDFFitPage | 15:41.37 |
| henrys | kens:oh right that would work fine, why are we talking about this? ;-) | 15:43.07 |
| kens | I've not tried it, not certain it works | 15:43.32 |
| | But I believe it should | 15:44.13 |
| | Oh you must also set -dFIXEDMEDIA of course, so if PDF has differnt sized pages, it won't work right | 15:44.55 |
| Robin_Watts | Hi Ray. Are you free to talk now? | 15:46.49 |
| ray_laptop | Robin_Watts: Maybe. It depends on who takes my youngest. | 15:50.27 |
| Robin_Watts | ray_laptop: Well, just tell me when you are. | 15:51.02 |
| | no hurry. | 15:51.15 |
| ray_laptop | Robin_Watts: So what's the clist issue ? | 15:51.25 |
| | You mentioned that we close the device before we're done with it. The clist device or the buffer device ? | 15:52.43 |
| | or the pdf14 device ? | 15:52.50 |
| Robin_Watts | ray_laptop: It's probably easiest if I walk you through it in the debugger. | 15:53.18 |
| | In my root directory on casper you will find a very simple file, J11_3.pdf | 15:54.31 |
| | And if you do: gswin32c.exe -sDEVICE=fpng -o out -r72 -dMaxBitmap=10000 -dNumRenderingThreads=1 J11_3.pdf you should get a SEGV. | 15:54.59 |
| ray_laptop | henrys: BTW, the way to scale is not BeginPage, but Install -- that modifies the initial matrix | 15:55.13 |
| kens | But you still need to modify the page size | 15:55.29 |
| | Which you can do by hooking setpagedevice | 15:55.43 |
| ray_laptop | kens: it depends on whether you want to scale the graphics on the page, or change the page size | 15:56.02 |
| kens | scale the PDF> If you scale the content, and not the media, the content falls off | 15:56.20 |
| ray_laptop | Robin_Watts: I'm downloading the file... | 15:56.27 |
| Robin_Watts | ray_laptop: Either you have a very slow connection, or it took you longer to type that than to download the file :) | 15:56.51 |
| ray_laptop | Robin_Watts: I typed it before starting the download | 15:57.24 |
| | Robin_Watts: I have the failure in gx_get_bits_return_pointer | 15:59.52 |
| Robin_Watts | ray_laptop: Right. | 16:00.00 |
| | Now, to see what's happening, place a breakpoint in clist_render_thread on the setup_buf_device call | 16:00.23 |
| | line524 for me. | 16:00.32 |
| ray_laptop | so I see stored_base is 0 | 16:00.33 |
| Robin_Watts | yeah, we need to rerun and catch it earlier and step through to show what's going wrong. | 16:00.51 |
| ray_laptop | Robin_Watts: So I should step into the setup ? | 16:02.09 |
| Robin_Watts | If you rerun it so it stops on that call, you'll notice that we pass a 'base' address in there ('mdata') but no line_ptrs pointer. | 16:02.28 |
| | This means the mdev allocates it's own line_ptrs array. | 16:02.55 |
| | You can see that happening if you step through. | 16:03.17 |
| ray_laptop | right | 16:03.30 |
| Robin_Watts | When you're happy, step out of that function back into clist_render_thread | 16:03.30 |
| | Now put a data breakpoint on &((gx_device_memory *)bdev)->line_ptrs | 16:04.03 |
| | That should be triggered towards the end of the clist_render_rectangle call. | 16:04.57 |
| | clist_render_rectangle calls ... clist_playback_band. | 16:05.13 |
| | And at the end of that, we decide to close the device, because we have a pdf14RGB device in play. | 16:05.54 |
| | orig_target was the image24 device (the memory device), but this has been wrapped in a pdf14RGB device (target) to do the composition. | 16:06.29 |
| ray_laptop | I have orig_target == 0 | 16:06.50 |
| Robin_Watts | ray_laptop: You're triggered on the data breakpoint ? | 16:07.27 |
| ray_laptop | Robin_Watts: yes | 16:07.34 |
| | and the target is the image24 device | 16:07.53 |
| Robin_Watts | In mem_close, in gx_forward_close_device, in clist_playback_band ? | 16:08.09 |
| ray_laptop | Robin_Watts: yes to all 3 | 16:08.23 |
| Robin_Watts | well, that's odd that we differ, but... | 16:08.44 |
| | the main thing is you can see that this is closing the device. | 16:08.57 |
| ray_laptop | but it definitely closes the mem device | 16:09.04 |
| Robin_Watts | which frees the line_ptrs. | 16:09.14 |
| ray_laptop | which will zaps the line_ptrs | 16:09.24 |
| Robin_Watts | Now, my reading of the clist setup code is that space has been allocated for the line_ptrs within the clist data block. | 16:10.04 |
| ray_laptop | I'm going to re-run it and watch orig_target because that doesn't seem right | 16:10.10 |
| Robin_Watts | ray_laptop: You're using a debug build, right? | 16:10.31 |
| ray_laptop | Robin_Watts: yes, there is room in the buffer for "line_ptrs_adjacent" | 16:10.42 |
| | Robin_Watts: oops. I have it set for a profile build. Just a minute | 16:12.02 |
| Robin_Watts | So, I have a diff here that adds a new entry to the page_info structure, line_ptrs_offset, which I set to be the size of the bitmsap. | 16:12.42 |
| | (plus alignment etc, as calculated by gdev_mem_bits_size) | 16:13.05 |
| | and I therefore pass that into the setup_buf_device | 16:13.32 |
| | That means that the line_ptrs don't get zapped on device close, and the crash goes away. | 16:13.58 |
| | BUT... the fact that we're doing a device close feels very wrong to me. | 16:14.16 |
| ray_laptop | Robin_Watts: I agree, that doesn't seem right. From the comment, it looks like it wants to close the pdf14 device, not the mem device | 16:17.11 |
| Robin_Watts | Indeed. | 16:17.23 |
| | But the problem is that closing the pdf14 device is passed through to close the mem device. | 16:17.38 |
| ray_laptop | Robin_Watts: because the pdf14 device has its close hooked to forward the close to the target ? | 16:18.55 |
| Robin_Watts | yes. | 16:19.03 |
| ray_laptop | rather than to pdf14_close | 16:19.05 |
| Robin_Watts | the pdf14 devices close routine is gx_forward_close_device | 16:19.35 |
| | which calls mem_close. | 16:19.43 |
| ray_laptop | But there is a pdf14_close that frees up the pdf14 ctx | 16:20.19 |
| Robin_Watts | That would be an ecumenical question. | 16:20.48 |
| Robin_Watts | is willfully ignorant of the workings of pdf14. | 16:21.16 |
| | My feeling is that clist_playback_blah should return the 'target' device for callers to close once they are done with it. | 16:22.33 |
| | It would remove the nasty hackery in there where we have to 'guess' whether to close or not based upon reference counts etc. | 16:23.05 |
| | ray_laptop: At any rate, there is a commit for your consideration on robin/master | 16:24.05 |
| | This forces the line_ptrs into the block allocated for them by the clist, and hence sidesteps the problem. | 16:24.26 |
| | I'm not convinced that the clists device closing is correct, but regardless, this patch seems to be a step forwards. | 16:24.50 |
| ray_laptop | Robin_Watts: I see -- the pdf14_disable_device _does_ close the pdf14 device. That's why we aren't leaking buffers | 16:26.15 |
| | Robin_Watts: I'm fine with retaining the line pointer offset. Save an extra malloc / free. | 16:26.49 |
| Robin_Watts | Thanks. | 16:27.43 |
| ray_laptop | Robin_Watts: and we shouldn't do the close if the device is already closed | 16:27.44 |
| | which it is by the time we get to the check in clist_playback_band | 16:28.06 |
| Robin_Watts | But we shouldn't close the device and then assume we can still get_bits from it. | 16:28.07 |
| ray_laptop | Robin_Watts: right, by only closing if the device is_open we won't ever close the mem_device | 16:28.53 |
| | That nasty bit of rc hackery was something I had to add to cure leaks that I saw with cust 532's use. | 16:30.28 |
| Robin_Watts | ray_laptop: If closing a device closes all the underlying devices too, then closing any compositor device in clist_playback_band effectively closes the underlying memory device too. | 16:31.14 |
| | and to my mind, we should never try to get bits from a device that's been closed. | 16:31.32 |
| | which is what we are relying on here. | 16:31.37 |
| ray_laptop | Robin_Watts: the compositors are not supposed to forward to their target unless they are disabled, and in that case they should also have closed their self. I'll check the overprint... | 16:33.34 |
| Robin_Watts | So either we have to make closing the compositor NOT close the underlying memory device (which I don't think is possible, cos what if we have multiple compositors?) | 16:33.44 |
| ray_laptop | Robin_Watts: so with my change to only close the (compositor) target if it is_open, we don't close the mem_device | 16:34.13 |
| Robin_Watts | Not closing the mem_device sounds good to me. | 16:34.41 |
| | I don't claim to understand the logic behind that, but then I am hazy on the whole compositor thing, so I'll leave that to you. | 16:35.13 |
| | So, this sorts the J11 crashing. Phew. I was worried that there might be a massive thinko in the process_page stuff :) | 16:36.34 |
| ray_laptop | Robin_Watts: I pushed the change to my repo -- do you want to have a look ? | 16:42.21 |
| | In particular, tell me if the explanation in the log is adequate | 16:43.05 |
| | My band skipping logic works fine in a debug build, but hands during parsing (at least -Z: doesn't give me the Outputpage start message) | 16:47.40 |
| Robin_Watts | ray_laptop: Let me look now. | 17:06.42 |
| | The explanation doesn't really inspire me. | 17:08.38 |
| | Is the problem that the compositor has already been closed? | 17:08.59 |
| ray_laptop | Debugging a profile build, the call to gx_default_create_buf_device has funky parameters, different to what shows for gdev_prn_buf_planar :-( | 17:09.03 |
| Robin_Watts | profile builds have optimisation enabled. | 17:09.22 |
| ray_laptop | The problem is that the 'disable' closed the device and hooked all the procs to forward | 17:09.35 |
| Robin_Watts | Therefore while you can step through the code, you can't trust the variable values. | 17:09.39 |
| | ray_laptop: Ah. You should put that in the message. | 17:10.02 |
| | And actually, I'm not sure this is the right fix then. | 17:10.29 |
| ray_laptop | Robin_Watts: I'm seeing the hang that I get with the release build. | 17:10.50 |
| Robin_Watts | If the problem is that the device is screwily forwarding the close on, then probably the correct fix would be to NOT have the device forward it on. | 17:10.57 |
| | i.e. the if (target->is_open) test shouldn't be in the clist logic. | 17:11.30 |
| | It should be in the compositor. | 17:11.40 |
| | The compositor should ignore any closes when !is_open. | 17:12.05 |
| | And it shouldn't forward the close device when it's disabled. | 17:12.23 |
| | Do you understand my objection>? | 17:12.43 |
| ray_laptop | Robin_Watts: sorry -- on the phone | 17:13.00 |
| Robin_Watts | np. | 17:13.04 |
| | paulgardiner: I'm with sebras in that I'd prefer an endptr :( | 17:14.00 |
| | but it's OK. | 17:14.05 |
| | The ios one... You create arrayWithCapacity:3 and then put 3 or 4 things into it. | 17:14.54 |
| paulgardiner | Robin_Watts: oh okay. Actually I was very tempted to stick in the endptr. I'll do that then. | 17:19.06 |
| Robin_Watts | mvrhel_laptop: ping | 17:22.19 |
| ray_laptop | Robin_Watts: OK. Back now | 17:29.33 |
| Robin_Watts | Do my babblings above make any sense? | 17:30.06 |
| ray_laptop | So, you don't think a compositor should forward its close ? But that's how we get the close to the device when we need to, other than clist rendering | 17:33.19 |
| | i.e., the parser calls close | 17:33.41 |
| | The clist is a special case because it wants to free the compositors and it needs to make sure the device is closed before freeing it (to allow the device to free up any allocations) | 17:34.52 |
| Robin_Watts | Ah, so this is a special case. | 17:36.04 |
| | It feels nasty to have such special cases coded inline. | 17:36.35 |
| | I mean, we already have code in here that hackily spots whether we need to close at all, so this is small beer compared to that maybe, but... | 17:37.06 |
| ray_laptop | and closing a mem device specifically, by design doesn't free the buffer it used. The only problem is if the line ptrs are allocated by the mem device, it _does_ free them when it closes | 17:37.10 |
| Robin_Watts | ray_laptop: Well, with the patch I pushed earlier the line_ptrs are safe now | 17:37.47 |
| ray_laptop | Robin_Watts: right. And with my patch, the mem device doesn't get closed, which is the same as when there isn't a compositor | 17:38.36 |
| Robin_Watts | ok. | 17:39.29 |
| kens | Goodnight all | 17:40.11 |
| ray_laptop | chrisl: I noticed the thing you found with MT fonts (whatever those are) and UFST. Is this something we should pass on to cust 532 (or other customers we know are using UFST and have older code: Norbert ? others?) ? | 18:23.46 |
| Robin_Watts | mvrhel_laptop: Ping | 18:30.04 |
| ray_laptop | Robin_Watts: how are we back to having so many segfaults ? The remove is_native_planar seems to have bumped up to 6, and with the Add alignment and pad, it jumped up to 16. Do you know what's going on with those segfaults ? | 18:33.16 |
| Robin_Watts | ray_laptop: Yeah. That's why I was after mvrhel. | 18:33.33 |
| | Can you sanity check me please? gx_forward_strip_tile_rect_devn | 18:33.47 |
| | if tdev == 0 do the default else pass on to tdev. | 18:34.06 |
| | in the pass on call, shouldn't we be using tdev, not dev ? | 18:34.17 |
| chrisl | ray_laptop: in terms of their using our code? It won't affect them until they update, and then they get the fix, too. | 18:35.59 |
| ray_laptop | Robin_Watts: I see. Yes, that does seem like the case where tdev is not NULL should be dev_proc(tdev, strip_tile_rect_devn)(tdev, | 18:36.55 |
| Robin_Watts | I'm testing that now. | 18:37.11 |
| ray_laptop | otherwise it looks WRONG | 18:37.12 |
| | chrisl: I see, so this broke fairly recently, and is now fixed. | 18:37.44 |
| chrisl | ray_laptop: yes, there was an optimisation in gxfapi.c that was broken, and I fixed that as part of the performance improvement with PCL. That fix exposed the UFST problem that's now fixed, too | 18:39.14 |
| | With the broken optimisation in gxfapi.c we always recreated the scaler's font object | 18:39.51 |
| Robin_Watts | ray_laptop: I still see some problems even after that fix. Am investigating the SEGVs now. | 18:40.17 |
| chrisl | Off now - 'night all..... | 18:41.28 |
| ray_laptop | Robin_Watts: OK. Thanks. I was worried with my cluster test of the little is_open check and saw SEGV's, but then I looked at the recent runs and realized it wasn't me :-) | 18:47.47 |
| | Robin_Watts: I pushed the little fix. Now back to looking into this hang :-( | 18:49.33 |
| | I guess I have to add print stmts into the release build to test it :-( | 18:49.59 |
| mvrhel_laptop | Robin_Watts: I am back | 19:24.58 |
| Robin_Watts | mvrhel_laptop: I had 2 things to ask you about. | 19:25.19 |
| mvrhel_laptop | uhoh | 19:25.48 |
| Robin_Watts | One was for your blessing to me fixing gx_forward_strip_tile_rect_devn | 19:26.06 |
| | (The 'pass on to tdev' call should pass tdev, not dev, right?) | 19:26.26 |
| mvrhel_laptop | hold on let me look | 19:26.50 |
| | right | 19:28.18 |
| | weird that that was not a problem before | 19:28.28 |
| Robin_Watts | And the second one was... something I cannot for the life of me remember. | 19:28.44 |
| | so consider that a bullet dodged :) | 19:28.53 |
| mvrhel_laptop | yes! | 19:28.57 |
| Robin_Watts | eek! My fix to the clist line_ptrs was horribly broken. | 19:30.30 |
| henrys | Robin_Watts: have you looked at the ccitt decoder? | 19:34.56 |
| Robin_Watts | Not in any depth, no. | 19:35.21 |
| | Should I have done? | 19:35.26 |
| henrys | Robin_Watts: no I'm just baffled. It doesn't seem to check at all for scan line overflow but maybe I'm not "decoding" the decoder right. | 19:36.11 |
| Robin_Watts | henrys: Ah. Could it be one of Peters hateful 'sentinel' things? | 19:36.38 |
| | henrys: Where are you looking? | 19:37.14 |
| henrys | Robin_Watts: you know you're in trouble when the declarations are macros. Sigh.... | 19:37.15 |
| | scfd.c:579 - get_run - the byte count is never checked but it goes ahead and calls invert_data | 19:38.02 |
| | this is a fuzzing problem. | 19:38.39 |
| Robin_Watts | That's looking for a run of black bits, right? | 19:39.49 |
| henrys | get_run returns a run of black which overflows the line buffer in invert_data | 19:41.46 |
| Robin_Watts | Is get_run returning a run that goes beyond the end of the line then? | 19:42.47 |
| henrys | the fuzzer corrupted the ccitt, it wouldn't happen for a correct file. | 19:42.48 |
| Robin_Watts | Either get_run should know the position within the line and stop searching for runs when it hits the end, or the caller of get_run should clip the run to the end of the scanline. | 19:44.17 |
| henrys | yes the run length is 1472 and the raster is 66 bytes so it isn't just padding or something. | 19:44.25 |
| Robin_Watts | The former would be preferable I guess. | 19:44.26 |
| | oh, sorry! This is *DECODE*! | 19:45.32 |
| | I was thinking that get_run was searching for a run. | 19:45.43 |
| | Right, so the problem is definitely that the caller should be checking the length for overflow. | 19:46.06 |
| | PDF has "DamagedRowsBeforeError". Do we respect that? | 19:46.21 |
| henrys | Robin_Watts: we do | 19:46.46 |
| | Robin_Watts: I am using backward debugging in gdb for this and it did make things go much faster. Not undodb their prices have gone through the roof. | 19:54.17 |
| Robin_Watts | henrys: They've been licensed by some very big names now. | 19:55.31 |
| | henrys: I can't see prices offhand. | 19:56.49 |
| henrys | you have to get a quote | 19:58.12 |
| | gdb is really okay if you can get near the problem before you start recording | 19:58.47 |
| | 701.00 annual and 1913 perpetual | 20:00.05 |
| Robin_Watts | for a single seat? Jeez. | 20:00.47 |
| | dollars, I'd assume. | 20:01.05 |
| henrys | right | 20:01.17 |
| Robin_Watts | Unity costs $1500 for Windows/Mac/Linux, plus $750 each for ios, android etc. | 20:02.02 |
| | or $70 a month. | 20:02.20 |
| | I'm always staggered by such prices. I guess I shouldn't be... | 20:03.16 |
| henrys | if the market bares it power to them. | 20:04.47 |
| Robin_Watts | If you look at the prices for the IDEs they are embedded into, undo's prices probably still look cheap. | 20:05.35 |
| | ARMs development kits are $5K a seat or something stupid. | 20:06.17 |
| | so probably by selling their standalone stuff cheaper they make themselves look bad when negotiating with the big boys. | 20:06.48 |
| henrys | I wonder how they get away with not releasing source integrated with gdb as they are. | 20:07.26 |
| Robin_Watts | henrys: They implement a server that gdb calls, I believe. | 20:07.58 |
| | and the source for the gdb end of that *is* released. | 20:08.10 |
| henrys | ah interesting | 20:08.26 |
| Robin_Watts | (my information here may be 10 years or so out of date :) ) | 20:08.42 |
| henrys | Hmm I'd wonder about the "intimate clause" in GPL: such as by intimate data communication or control flow between those subprograms and other parts of the workâ¦. | 20:10.47 |
| Robin_Watts | I think the interface was general enough. Possibly it's the same interface that gdb's own reversible stuff works on? I really don't know details. | 20:12.24 |
| alesser | hi all | 21:08.13 |
| | i've been racking my brains for days to solve a GS problem | 21:08.29 |
| | and i'm hoping someone here with a smarter brain than me can help | 21:08.40 |
| | anyone game ? | 21:08.42 |
| | I need to convert PDFs to TIFFs, but not use ghostscript's dithering... i've tried lots of different options.. and the only thing that seems to help at all is -dDITHERPPI=300 for 300 DPI images.. but it doesnt really do what I need.. which is a threshold | 21:11.10 |
| | is there a way to do this ? | 21:11.14 |
| ray_laptop | alesser: do you want to use a threshold array to dither, or just threshold to all black or white based on the gray level ? | 21:13.03 |
| | alesser: GS can do both | 21:13.21 |
| | alesser: GS has a blue noise type of threshold array in the distribution that some people like. It gives you 256 shades, and the dots are dispersed so that it looks sort of like error diffusion | 21:17.47 |
| Robin_Watts | alesser: So you want to convert PDFs to greyscale tiffs? | 21:18.27 |
| ray_laptop | alesser: and to do high contrast, all black or all white you use a transfer function. | 21:18.32 |
| | for example: gs -sDEVICE=tiffg4 -o out.tif -c "{ .5 gt { 1 } { 0 } ifelse} settransfer" -f in.pdf | 21:20.25 |
| | the above thresholds at .5, but you can make it anything you want | 21:20.56 |
| | hmm... I tried that with tiger and for some reason, the "tongue" is still a shade of gray. Not sure how | 21:22.08 |
| alesser | huh | 21:26.17 |
| | really ? | 21:26.19 |
| | very interesting | 21:26.22 |
| | why isnt this available like a -threshold option ? | 21:26.32 |
| | i've been using something like this: | 21:26.44 |
| | gs -q -dSAFER -dNOPAUSE -dBATCH -dUseCropBox -sOutputFile=DITHER300%d.tif -r300 -sDEVICE=tiffg4 -dDITHERPPI=300 $FILE | 21:27.31 |
| | I want PDFs to B&W tiffs | 21:28.07 |
| | bitonal.. no grayscale | 21:28.14 |
| Robin_Watts | but with no dithering. Why does the dithering offend you? | 21:28.34 |
| alesser | it makes the documents unreadable unfortunately | 21:29.28 |
| | i'm working with items like checks and medical documents | 21:29.40 |
| ray_laptop | alesser: try the settransfer method then | 21:29.53 |
| alesser | i will try that post-haste | 21:30.02 |
| | does it work on images? | 21:30.04 |
| ray_laptop | and adjust the threshold to your liking | 21:30.13 |
| | alesser: it works on ALL objects | 21:30.26 |
| alesser | ah | 21:30.53 |
| ray_laptop | alesser: a higher threshold means that more levels will be mapped to black. | 21:31.45 |
| alesser | so, for example, gs -q -dSAFER -dNOPAUSE -dBATCH -dUseCropBox -sOutputFile=output.pdf -r300 -sDEVICE=tiffg4 -c "{ .5 gt { 1 } { 0 } ifelse} settransfer" input.pdf | 21:31.46 |
| | right | 21:31.49 |
| ray_laptop | alesser: after the -c "..." you need -f before the input file | 21:32.15 |
| | alesser: as with: -c "{ .5 gt { 1 } { 0 } ifelse} settransfer" -f input.pdf | 21:32.43 |
| sebras | paulgardiner: Robin_Watts: OTOH and endptr would cost 4bytes per page. OTTH the embedded systems targeted by mupdf are no longer memory constrained in any notable way. | 21:34.26 |
| alesser | ah | 21:36.09 |
| | i thought the last thing was assumed to be the input file | 21:36.37 |
| | huh | 21:43.47 |
| | i got an errror | 21:43.50 |
| | wonder what i did wrong | 21:43.53 |
| ray_laptop | alesser: read the Use.htm on the use of -c without the -f you will get an error like Error: /undefined in input.pdf | 21:46.28 |
| | alesser: what error did you get ? | 21:46.52 |
| alesser | yes | 21:47.02 |
| | i think thats what i did | 21:47.07 |
| | one sec | 21:47.08 |
| | let me try yours straight | 21:47.13 |
| | oh my god.. you have NO IDEA how long I have been trying to solve this problem.... DAYS of time.. I swear. | 21:50.30 |
| ray_laptop | glad that it worked for you. | 21:52.13 |
| alesser | i will play with the options | 21:52.50 |
| | but i think this will work | 21:52.53 |
| | where can i read more about the -c stuff ? | 21:53.05 |
| | i was looiking into it earlier | 21:53.10 |
| | but could not find it | 21:53.14 |
| ray_laptop | just to see the high quality dither (stochastic blue noise), try: gs -sDEVICE=tiffg4 -r300 -o out.pdf -Ilib stocht.ps input.pdf | 21:54.08 |
| | it'll be dithered, but the dots are dispersed, so it might be more legible without losing all of the shades | 21:55.09 |
| | you can darken the output using something like: gs -r300 -sDEVICE=tiffg4 -o out.tif -Ilib lib/stocht.ps -c "{ 4 exp } settransfer" -f input.pdf | 21:57.08 |
| | numbers smaller than 1 lighten the image, bigger than 1 darken it | 21:57.43 |
| alesser | is that instead of floyd-steinberg? | 21:57.50 |
| | i'm running on GS8.71 btw | 21:57.57 |
| | (stuck on it for now b/c it's in a production environment) | 21:58.10 |
| ray_laptop | alesser: yes, the stocht.ps is a threshold array | 21:58.12 |
| alesser | how does it work ? | 21:58.28 |
| ray_laptop | all this stuff has been around for ages | 21:58.31 |
| alesser | ah | 21:58.37 |
| | what, exactly is a threshold array ? | 21:58.44 |
| | says "dither below this threshold" ? | 21:58.57 |
| ray_laptop | there is a file in lib that contains the blue noise threshold arrays lib/ht_ccsto.ps the stocht.ps loads this as the default halftone | 21:59.33 |
| alesser | >>blue noise threshold arrays ? | 21:59.56 |
| | (is confused simpleton) | 22:00.02 |
| ray_laptop | when painting a shade of gray, if the value of the gray (range 0-255) is above the corresponding position in the threshold array, the dot is painted black. Otherwise it paints white | 22:00.51 |
| | alesser: look up threshold arrays of google | 22:01.22 |
| alesser | ah | 22:02.09 |
| | and the array is randomly distributed noise? | 22:02.47 |
| | what determines the values at each point in the array ? | 22:03.19 |
| | and I assume this is different from ETS on http://www.artifex.com/page/imaging-science.html ? | 22:04.50 |
| ray_laptop | alesser: yes, ETS is a error diffusion method. It is prettier than a BN array, but takes longer to run | 22:10.16 |
| alesser | is there a way to do something really silly.. like .. threshold + dithering ? | 22:10.27 |
| | so, everything above a certain threshold will be black, and everything below would be dithered? | 22:10.50 |
| ray_laptop | alesser: the array was generated by a program that I wrote. | 22:10.59 |
| alesser | haha | 22:11.07 |
| | rather than dithering everything ? | 22:12.03 |
| ray_laptop | alesser: yes, you can combine them. Try: gs -r300 -sDEVICE=tiffg4 -o out.tif -Ilib lib/stocht.ps -c "{ dup .5 lt { pop 0 } if } settransfer" -f input.pdf | 22:12.14 |
| alesser | which sets everything above 50% to black and dithers everything else? | 22:12.59 |
| ray_laptop | alesser: that maps values less than (darker than) the threshold (.5 in the example) but grays lighter than .5 will be unchanged | 22:13.12 |
| | in PostScript 0 is black, 1 is white | 22:13.37 |
| alesser | rigth | 22:13.40 |
| | sorry, forgot that | 22:13.43 |
| | it's been a loooong time | 22:13.54 |
| | believe it or not, I dont even write code | 22:14.02 |
| ray_laptop | is surprised you knew that ;-) | 22:14.03 |
| alesser | i did actually used to know that once a long time ago | 22:14.14 |
| | i know weird little tidbits of knowledge | 22:14.38 |
| ray_laptop | alesser: I do believe that you don't right code | 22:14.40 |
| alesser | just enough to get me by | 22:14.46 |
| ray_laptop | s/right/write/ | 22:14.51 |
| alesser | haha.. i used to | 22:14.51 |
| | sed? | 22:15.00 |
| | or regex | 22:15.06 |
| | looks like regex | 22:15.11 |
| | like i said.. just enought to be able to TALK to coders | 22:15.30 |
| ray_laptop | sed, ed, vi, ... all use that syntax, so we sometimes use it here to correct ourselves | 22:15.34 |
| alesser | hehe | 22:15.39 |
| | in any case.. yea, my role is very strange at my company... but I think of it as "jack of all trades" | 22:16.02 |
| | i'm actually trying to solve a complicated buisness problem that involves healthcare records | 22:16.22 |
| ray_laptop | a JOAT is a valuable thing to have | 22:16.29 |
| alesser | yea, I shield the developers from too much crazyness | 22:16.45 |
| | and I know just enough to speak "developer".. about 80% of the time | 22:17.03 |
| | anyways | 22:17.08 |
| | this is tremendously helpful | 22:17.13 |
| | i'll play with the parameters on my dev box | 22:17.21 |
| | and then recommend something to my devs. | 22:17.28 |
| | this definately made my day though | 22:17.52 |
| | i was sooo frusterated | 22:17.56 |
| ray_laptop | alesser: well, good luck. | 22:18.21 |
| alesser | by the way.. do you know where I can learn more about -C and the stuff that goes in there? | 22:20.20 |
| | it looks like postscript code | 22:20.25 |
| | from my limited experience | 22:20.32 |
| | I assume it's somewhere in the documentation tree (http://www.ghostscript.com/doc/9.10/Readme.htm) but not sure where | 22:21.09 |
| | it's certainly not in the man file | 22:23.18 |
| | ah | 22:25.49 |
| | found it | 22:25.50 |
| | http://ghostscript.com/doc/current/Use.htm | 22:25.53 |
| | so you can feed straight postscript to GS? | 22:26.07 |
| | how does it know in what context to execute that PS ? | 22:26.15 |
| | does it just apply the postscript to code to every output pixel ? | 22:27.19 |
| | Forward 1 day (to 2013/11/08)>>> | |
IRC Logs