IRC Logs

Log of #ghostscript at irc.freenode.net.

Search:
 <<<Back 1 day (to 2012/12/13)2012/12/14 
Robin_Watts marcosw: ping ?00:57.39 
  henrys: zeniko has volunteered his patches for openjpeg and jbig2dec. I've not looked at any of them yet, but I'm wondering what the best way to proceed is.00:58.49 
  Historically, his patches are very good at showing issues, but often the exact fix we want to take on is different.01:00.46 
  But I have no experience in openjpeg or jbig2dec, so I suspect I'd be less picky.01:01.20 
sebras_ Robin_Watts: are they in bugzilla?01:02.14 
sebras_ never sleeps either.01:02.23 
Robin_Watts If we could make the files that crash in openjpeg etc available to him we could invite him to see which ones were fixed with his patches, and to get us a patch for those that weren't.01:03.04 
  Or that might be something we want to pass to Shelly?01:03.15 
  sebras: See bug 69350301:03.31 
  The files are not available to the general public yet.01:03.42 
  henrys: I'll drop the guys at company 395 a mail to check whether it's OK for us to pass the files out to other people (I can't imagine it wouldn't be, but we ought to check).01:04.30 
henrys shelly is nda'd so you can git it to him now.01:06.31 
  Ideally they should go upstream, though01:07.07 
  Robin_Watts ^^^01:07.24 
Robin_Watts henrys: I was thinking that if zeniko is offering us his set of openjpeg patches (and I have no idea how large that set is), then that may be worthy of a bounty in itself.01:09.43 
sebras Robin_Watts: there appears to be a whole lot of fixes upstream that we don't have in thirdparty.01:10.01 
  Robin_Watts: well, changes at least.01:10.11 
Robin_Watts Yes, ideally we should push bug fixes upstream, but I rather suspect that the openjpeg folks are bashing on v2.0 at the moment.01:10.19 
  sebras: really? We were up to date not so long ago.01:10.34 
  henrys: if we pass the bugs to shelly, and he fixes a bug that zenikos patch has already fixed it's a bit of a kick in the teeth to zeniko.01:12.04 
sebras Robin_Watts: I might be confused by their repo though. looks like the moved files around a lot.01:12.32 
Robin_Watts If he fixes them by using zenikos patches then zeniko probably deserves some sort of recognition.01:12.39 
  sebras: They may be pulling stuff in from v2.0 to the 'stable' repo.01:13.00 
henrys well can we see what is fixed by zenikos patches ahead of creating bugs?01:13.24 
sebras Robin_Watts: oh... but they have released their 2.0 branch.01:13.25 
Robin_Watts henrys: I will look over his patch tomorrow and try to judge the scope of it.01:13.46 
  marcosw_: Hey. I looked in /home/support/bug694503 on casper, and it was empty...01:14.09 
  bug693503 too :)01:14.28 
  Should there be stuff there, or are you still sorting ?01:14.50 
marcosw_ Robin_Watts: oops, sorry about that. I was ftp-ing the file from my mom's house last night and it was taking too long so I was planning on uploading it from my home but forgot. I'll upload it now.01:15.11 
Robin_Watts ok. I'm off to bed now, so it'll be there for me in the morning. thanks.01:15.31 
sebras Robin_Watts: there are changes in openjpeg 1.5.1 that we likely want though.01:16.00 
Robin_Watts henrys: I'll apply zenikos patch tomorrow and see how many of them it solves.01:16.03 
  sebras: quite possibly.01:16.09 
  I'll try and look at that too, thanks.01:16.22 
  ok, bedtime for me.01:16.30 
sebras nn.01:16.33 
marcosw_ Robin_Watts: let me know what else I can do help. 01:16.34 
Robin_Watts marcosw_: The separation into categories will be a big help.01:16.53 
wart___ hi folks; maybe i'm misremembering, but i think the mupdf people hang out here, right? i have a simple question, and I must've misread the manpage on this, but how do i get smooth scrolling? i.e., i use up/down arrows to scroll up/down lines but when i reach the end of the page, i hit pg-down. but it jumps down to the end of the next page so i have to up arrow up to the top.01:21.08 
  pretty annoying.01:21.15 
  hope i'm no ti n the wrong channel, and i *hope* it isn't obviously in the manpage :-)01:21.23 
Robin_Watts wart___: What you describe is currently not implemented in the viewers.01:22.54 
  The core libraries support it, but the viewer (which is page based) does not.01:23.15 
  I think if you use space to move forwards rather than arrows, then when you go to the next page you appear at the top, not the bottom?01:24.07 
wart___ Robin_Watts: oh, ok. let me try that01:24.17 
Robin_Watts sebras or tor8 may know more. I'm off to bed now. best of luck.01:24.33 
wart___ space method works, but the increments are too big such that it "skips" the last few sentences on the former page and the first few sentences on the next page01:25.05 
  i assume the increment is adjustable pretty obviously in the code, so maybe i'll just hack that01:25.25 
  i like my increment level to be smaller anyway.01:25.37 
  oh i mis-interepreted the results01:26.51 
  actually space is just page down01:26.55 
  ok checking out pdfapp.c and I realize '.' will do what i want01:33.08 
  and actually PGDOWN does what '.' does, so RightArrow does something strange01:35.05 
  and so too Space01:35.09 
  so I'll avoid those.01:35.11 
  thanks01:35.12 
kens marcosw what exactly does the potential customer mean by 'thread safe' ? If they want to load GS once and have several threads call it with some kind of context then the answer is an absolute no.08:11.53 
  The only way to have GS process multiple files to PDF is to launch a separate process for each. And even then you may have to give them different TEMP folders.08:13.03 
Robin_Watts kens: So pdfwrite uses global variables?10:01.16 
kens2 Robin_Watts : no, but I'm fairly convinced its temp files will get confuised10:01.46 
  I *think* I removed all the globals some time back10:02.16 
Robin_Watts Are the temp files given particular names then?10:02.45 
kens2 Robin_Watts : I wouldn't like to be sure.10:02.55 
  THe way pdfwrite creates temporary files is 'odd'10:03.17 
Robin_Watts My simple tests a while ago suggests that pdfwrite works just fine in multithreaded circumstances.10:03.19 
kens2 Robin_Watts : well, if you want to fix it when someone complains :-)10:03.32 
Robin_Watts On unix: cd gs && make apitests && gs/bin/apitest -o out.pdf -sDEVICE=pdfwrite in.pdf10:03.58 
kens2 SOrry, that's gibberish to me10:04.23 
Robin_Watts I have added a new target to the makefile, so if you do "make apitest" it makes a test program called "apitest"10:05.09 
kens2 OK10:05.29 
  but what does it do ?10:05.41 
Robin_Watts When you run: gs/bin/apitest -o out.pdf -sDEVICE=pdfwrite in.pdf it runs 10 different threads. The first one does: -o out.pdf.0 -sDEVICE=pdfwrite in.pdf10:06.07 
  The second one does -o out.pdf.1 -sDEVICE=pdfwrite in.pdf10:06.19 
  etc.10:06.21 
kens2 Well, if you want to run the test suite through that, I'd be happy afterwards10:06.44 
Robin_Watts And in my tests, they all produce identical pdf files (modulo date/timestamp entries)10:06.45 
  I have run several hundred files through it.10:07.07 
kens2 The various ways pdfwrite can create files are twisted and complicated10:07.08 
  and depend often on the exact parameters used10:07.22 
Robin_Watts marcosw is doing more thorough tests now.10:07.22 
kens2 OK well if marcosw thinks iot works he can tell the cusotmer so10:07.37 
  I haven't tested it, (and don't want to) and it definitely *didn't* work in the past.10:08.01 
  And I'm unconvinced that there's a huge benefit over just running several processes10:08.24 
  Oh dear, the Indexed colour with CompatibilityLevel < 1.3 has been broken since Michael redid the colour code.10:10.57 
Robin_Watts tor8: ping11:21.53 
  Looks like openjpeg 2.0 has shipped.11:22.57 
  sebras: You understand aes,right?11:32.42 
chrisl tkamppeter: ping....11:37.59 
sebras Robin_Watts: no very well, why?13:04.50 
Robin_Watts sebras: I've got a file here with a keysize of 96, and that's an unsupported size, and hence we crash all over the place.13:05.17 
sebras oh.13:05.27 
Robin_Watts I have a fix here, which is to detect that and throw an error.13:05.29 
sebras that seems reasonable.13:05.37 
  can anyone else decrypt the file?13:05.46 
Robin_Watts I was wondering if there was a smarter fix ("Ah, you should round up to the next power of 2 and pad with 0's")13:05.56 
sebras Robin_Watts: not that I know of.13:06.21 
Robin_Watts sebras: No, I think it's a broken file.13:06.32 
sebras Robin_Watts: and with encrypted files the Info object's strings are encrypted too, right? so we can't deduce what creator was used.13:07.13 
  Robin_Watts: searching for AES and 96 bits only gives me this: http://tools.ietf.org/html/rfc356613:18.08 
  Robin_Watts: I guess if those files become common then we'll have to consider supporting 96 bits even if it is out of spec.13:19.34 
kens Does Acrobat open it ? I can try Acrobat X if required. If Acrobat doesn't open it, then I htink we are fine13:20.32 
Robin_Watts no acrobat doesn't open it.14:06.32 
  tor8: ping14:06.36 
sebras Robin_Watts: hm.. wouldn't it be better to keep fitz/crypt_aes.c unchanged and check the keylen in fz_open_aesd() before calling aes_setkey_dec()?14:52.52 
  or even before that if possible.14:53.01 
Robin_Watts sebras: It is wrong, IMHO that a call to a aes_setkey_dec should silently fail.14:53.31 
  If aes_setkey_dec can fail, then it should be capable of indicating that it can fail - after all, we'd like the knowledge of what key lengths it can cope with to be as local as possible.14:54.21 
  hence all I've done is make that function return 1 in the failure case and 0 otherwise.14:54.38 
  And I then made all the callers of that function check for that.14:55.02 
  Why should every caller be expected to know the conditions under which the callee can cope?14:55.26 
sebras Robin_Watts: yes, I see that from the patch. I'm just thinking that we don't want to change imported code unless we really have to.14:55.51 
  Robin_Watts: btw, why doesn't the AES 96-bit file trip up on the last fz_throw() in pdf_parse_crypt_filter() where we check if keylen != 256?14:58.00 
Robin_Watts cos it's not only called from pdf_parse_crypt_filter?15:01.48 
  sebras: It's a minimal change, and it's (IMHO) a vital one.15:02.11 
sebras Robin_Watts: I don't dispute that, I'm just thinking that maybe we ought to check the keylength at a different place, where we could catch all of them.15:14.00 
Robin_Watts Right. And is aes_setkey_dec not that place?15:14.22 
sebras Robin_Watts: I'm not sure about that. :)15:14.35 
  Robin_Watts: this is why I can't let it go.15:14.42 
  Robin_Watts: why isn't this caught in pdf_new_crypt()?15:14.54 
Robin_Watts because it's not only called from pdf_new_crypt?15:15.13 
sebras there are keylength checkts there too, but only for v == 2 coh v==4.15:15.17 
Robin_Watts putting key length checks 'further out' in the code might allow us to give better error messages, but it hurts us in that knowledge of what are permissible lengths gets spread throughout the code, rather than being localised to where it matters.15:16.52 
sebras so we have the checks in pdf_new_crypt() to generate the error messages and the ones in fitz/crypt_aes.c to not SEGV..?15:20.48 
  I haven't looked at arc4, but it may very well error out on incorrect lengths (or for other reasons).15:21.35 
Robin_Watts sebras: The ones in fitz/crypt_aes.c are the low level code stating what it can and can't cope with.15:21.43 
  The ones in pdf_new_crypt are (less important) ones stating what the PDF spec can and can't cope with.15:22.18 
  I suspect that if the pdf_new_crypt ones were removed, the inner ones would still catch the cases we don't cope with, just fine.15:22.47 
sebras yes, and if that is the case then we actually _need_ the low level ones.15:23.13 
  ok, so to answer your question -- no I obviously don't know the AES code... ;)15:23.51 
tor8 Robin_Watts: back.15:38.17 
Robin_Watts ok, I've started to look at the reports from customer 395.15:39.29 
  and I've come across an 'interesting' problem.15:39.57 
  In the draw device, when we 'push' a graphics state (either for a clip or a group), we need to be careful what happens in the event of a failure.15:40.46 
  One of their files sees us do the push (which makes a new graphics state, and copies the current shape/dest etc to it).15:41.24 
  Then we fail to decode the imagemask in use, which causes us to throw an error.15:41.46 
  The display list code catches this and continues, so we later get the pop.15:42.09 
  We therefore pop the 'dest' and destroy it, leaving us with a defunct dest on the stack which gets double freed later on.15:42.48 
  So, I'm trying to think of a nice way to fix this.15:43.26 
  There is a secondary problem of course, in that it could be the push itself that fails, and then we'd not successfully manage to make the new gstate, which would mean that all subsequent pops etc would be out of sync.15:44.12 
  And (somewhat simpler) 3 patches for your consideration on robin/master15:46.37 
  I'm trying to think of better ways to cope with this.15:47.00 
tor8 Robin_Watts: ugh. I had worried about that (gstate stack depth on throw/catch)15:47.03 
Robin_Watts In some ways the nicest way to work would be that if we throw during a push, then no other calls should be made until we pop.15:48.10 
  But that would require the display list device to be smarter.15:48.32 
  (and potentially the pdf interpreter to be smarter too, but then I suspect it might well work like that anyway).15:48.50 
tor8 Robin_Watts: I take it you've discussed the AES situation with sebras? the other two LGTM.15:49.32 
Robin_Watts I further suspect that to make this work properly, the display list would need to become a display tree.15:49.39 
  tor8: Yeah, we've discussed it here. Either he agreed with me in the end, or I bludgeoned him until he did :)15:50.15 
tor8 Robin_Watts: would it be feasible to record the depth around the try/catch cases?15:50.20 
Robin_Watts tor8: In what sense ?15:50.33 
tor8 and then make sure we pop the right number at good intervals15:50.36 
Robin_Watts The problem is not what happens within the draw_device, but rather what happens in the callers, I think.15:50.55 
tor8 this can happen in both the interpreter and display list, errors from device calls15:51.01 
Robin_Watts tor8: Yes.15:51.11 
tor8 or hang on, maybe we can do that in the dev_null.c wrappers?15:51.53 
Robin_Watts I'm inclined towards the idea that errors from a device call that would create a new group/clip should result in the caller not calling any more content calls, or the corresponding pop.15:52.22 
tor8 catch and flag a given "depth" of calls as errored out on the graphics side and ignore commands until the right number of pops have been given15:52.27 
  all device calls get routed through the dev_null.c wrapper functions IIRC15:52.51 
Robin_Watts tor8: I see, I think.15:53.00 
tor8 so we could make a guarantee that the fz_device calls never throw (by catching and handling in the fz_device wrapper functions)15:53.30 
  and then you can check the error flag at the end15:53.51 
  or (ugly) throw when closing the device15:54.16 
Robin_Watts OK, so we'd need to have some sort of error depth count in the device structure for the dev_null stuff to use.15:54.33 
  That seems to be a nice solution, if we can make it work.15:55.00 
  It means the implementer of a device always ensures that no pop actually happens if we throw an error during a pop.15:55.36 
  s/pop/push/15:55.53 
  and it means that callers can assume that pushes never give an error.15:56.08 
tor8 yeah. something like that would be nice.15:57.14 
Robin_Watts I will try for it.15:58.21 
  That's altogether nicer than anything I'd thought of so far.15:58.43 
  Jeez. Not content with making me reboot my machine yesterday, windows is downloading more updates now.15:59.43 
  tor8: Ah yes.16:05.20 
  Also, zeniko has offered to help us out with the openjpeg problems.16:05.39 
  specifically he's offered his set of patches.16:05.49 
  but openjpeg 2.0 is now out.16:06.06 
tor8 Robin_Watts: alright. I also saw mention that 2.0 is out.16:06.09 
Robin_Watts so I wonder if we should be updating to that, then seeing what problems still arise, patches are still applicable etc.16:06.31 
tor8 yeah.16:06.42 
Robin_Watts Hmm. The cookie (where the error count is stored) is only held by the caller, not by the device.16:12.15 
  hence if we swallow errors at the null device level, we can't increment the error count.16:12.35 
tor8 Robin_Watts: well, I guess it doesn't really matter as long as you can use the cookie for checking?16:14.04 
Robin_Watts tor8: That's my point.16:14.25 
  Normally the caller catches the error and increments the error count in the cookie.16:14.38 
  If the dev_null layer catches the error, then it doesn't have the cookie to update - and the caller can't update the error count cos it never sees the error.16:15.37 
  we could make the push swallow the error, and then rethrow it on the pop ?16:18.53 
  Or we could abandon the idea of the dev_null layer handling such things, and have the caller just skip until the next pop.16:20.47 
  That might be hard in the interpreter case though.16:21.30 
henrys Robin_Watts: has zeniko or anyone given patches to openjpeg? Are they responsive?16:35.48 
Robin_Watts henrys: I have not personally dealt with them, and I don't know about zeniko.16:36.12 
  The sumatra set of patches for openjpeg aren't huge. 300 lines or so.16:37.12 
tkamppeter chrisl, hi16:37.34 
Robin_Watts and we should look at openjpeg 2.0 before we look at sumatras changes, IMHO.16:37.44 
  as that may have more 'systemic' fixes in it.16:38.00 
chrisl tkamppeter: for https://bugs.launchpad.net/ubuntu/+source/cups-filters/+bug/978120 we need to add those Toshiba printers to the list of printers that need the compression filters disabled16:38.37 
Robin_Watts so I'd be tempted to update to v2.0, then see what issues remain before talking more to zeniko/looking more at zenikos patches16:38.53 
  henrys: Samsung 3d TV perhaps?16:45.17 
henrys Robin_Watts:are you getting one?16:56.14 
Robin_Watts I have one.16:56.24 
  I thought I recognised the glasses you/sabrina were wearing.16:56.42 
henrys ah haven't been to Facebook in a while16:57.18 
  actually it's sabrina's brother's tv.16:58.12 
mvrhel_laptop good morning17:06.24 
kens morning mvrhel_laptop17:06.39 
mvrhel_laptop hi kens I see your email17:06.39 
kens aha, excellent17:06.47 
mvrhel_laptop so you are correct. The cmm will not transform indexed nor separation spaces directly17:07.59 
kens ah, so I need some more steps17:08.15 
mvrhel_laptop yes17:08.19 
  let me see if there is a simple call you can do for this17:08.29 
kens please :-)17:08.37 
mvrhel_laptop kens: so are you wanting to get to the device profile color space from the index space?17:16.54 
  or are you wanting to get to the separation alternate tint transform color space?17:17.15 
kens mvrhel_laptop : I want to convert the indexed [whatver] into a device space17:17.25 
  Starting with a Indexed value, get a (eg) RGB output17:17.47 
mvrhel_laptop yes. well with concretize_color proc on the index color space that will map you to the the color space that is defined by the device profile17:19.16 
kens OK that sounds like what I want17:19.33 
mvrhel_laptop you will not have to worry about the separation space etc that is in the middle17:19.54 
kens That's fine I do have to look at the final space, as I newed to know if its valid for representationb. I only convert if it isn't. But I can do all tghat, What I don't (didn't) know how to do ws convert to a device cprocess colour when I do need to convert it.17:20.57 
  which sounds like the concretize proc17:21.21 
mvrhel_laptop yes. that will always take you to the process color space defined by the device profile17:21.48 
  for any source color space17:22.10 
kens yes, that's the thing I need, the device process color model is already correclty set up17:22.19 
mvrhel_laptop good. let me know if you have any other issues17:22.55 
  have a good weekend too17:23.02 
kens can you point me to somewhere where it is used ?17:24.20 
mvrhel_laptop oh yes17:24.31 
  actually you will see a call to it in gx_concretize_Indexed17:27.30 
  where the base space is called17:27.41 
kens OK I'll look there, thanks17:27.45 
mvrhel_laptop I can't find it used directly anywhere in the code. What you should see though in your case, the indexed proc to lead to the separation proc to lead to the ICC proc when you invoke the procedure17:29.46 
kens well I'll give it a try tomorrow17:30.33 
mvrhel_laptop It is also used in the conversion of the CIE PS color spaces17:31.19 
  kens: ok17:31.26 
kens That seems to be working OK as-is right now, but I have more testing to do17:31.46 
mvrhel_laptop marcosw_: you available for a question17:32.20 
marcosw_ mvrhel_laptop: in a minute17:32.56 
mvrhel_laptop ok17:33.01 
  marcosw_ just ping me when you become available17:45.11 
marcosw_ mvrhel_laptop: I have time now17:46.56 
mvrhel_laptop ok. so can you click on my recent log from my cluster regression run17:47.39 
  I am getting a log file to large issue. 17:47.52 
  Is the last file going to be the issue17:48.18 
  i.e. is 107200 temp/tests_private__pdf__sumatra__x_-_renders_slowly.pdf.pbmraw.300.0.log17:48.38 
  indicate the trouble file?17:48.46 
marcosw_ Let me look at the logs...17:49.01 
  yes. The 107200 is the size of the file, in blocks, so ~100 megs17:50.55 
mvrhel_laptop ok. thanks. 17:51.13 
marcosw_ we had problems with the cluster machines filling their disks if the log files became too large.17:51.21 
mvrhel_laptop I will grab that one and see what is going on. very odd to see this going on with my change17:51.35 
marcosw_ it might not be your problem. Looking around the cluster nodes I see some of the other log files are pretty big.17:54.10 
henrys marcosw_:you must have a log somewhere with all performance times somewhere? How do we get to that?17:55.12 
  s/somewhere//17:55.43 
marcosw_ it's on miles, in /home/marcos/performance/results/miles and .../kilometers.17:56.01 
  should be world readable and you should have an account on mlles.17:56.18 
henrys is this one of the no-ip hostnames?17:57.04 
mvrhel_laptop marcosw_: I was considering doing a clean push to see if I ran into the same issue17:57.07 
marcosw_ henrys: miles-artifex.homeip.net17:57.25 
mvrhel_laptop but perhaps I should run the file first and see what it is spitting out17:57.40 
marcosw_ mvrhel_laptop: I was just about to do that.17:58.04 
mvrhel_laptop which one?17:58.14 
chrisl henrys: can you remind me how to build PCL so the memory allocator goes straight to the "sytem" malloc, rather than our own?17:59.20 
marcosw_ tests_private__pdf__sumatra__x_-_renders_slowly.pdf.pbmraw.300.0.lo17:59.28 
mvrhel_laptop marcosw: I did run with my change 2 times and the same file came back as the issue. 18:00.03 
  so at least that is consistent18:00.09 
henrys #define HEAP_ALLOCATOR_ONLY18:00.19 
chrisl Thanks!18:00.29 
henrys you can put it in plalloc.h and the needed places will see it.18:00.49 
chrisl Oh, okay, I was going to add it to the CFLAGS....18:01.17 
henrys that's fine too18:01.30 
marcosw_ mvrhel_laptop: I get a bizillion (yes, it's a word, look it up) of "art_blend_pixel_8: blend mode 255 not implemented" messages.18:01.45 
mvrhel_laptop ok. that could be from me..... :)18:02.05 
  grabbing file now18:02.09 
  actually yes. that is definitely from me18:02.32 
marcosw_ http://www.urbandictionary.com/define.php?term=bizillion18:02.34 
mvrhel_laptop how about a googleplex18:02.47 
  I like the pointing pinkie to lower lip 18:03.10 
marcosw_ I think you mean googolplex18:03.20 
mvrhel_laptop oh yes. 18:03.28 
marcosw_ googleplex is a building18:03.39 
mvrhel_laptop I stand corrected18:03.50 
marcosw_ though it would take a lot of error messages to fill a building18:04.04 
Robin_Watts tor8: New patch pushed to robin/master for your consideration. Tests running now.18:06.13 
henrys marcosw:yikes miles' network is slow. Is that DSL?18:09.15 
marcosw_ henrys: Robin_Watts is running a cluster job, so the network is saturated when the log files are uploaded (which just occured).18:10.26 
  to answer your question, it is DSL.18:11.29 
mvrhel_laptop bbiaw18:13.01 
Robin_Watts Something (probably windows update) has screwed my machine. brb.18:43.22 
  Crumbs. Another mass shooting.18:54.56 
henrys young kids too, Jesus19:05.31 
Robin_Watts tor8: New review for you on robin/master implementing what we talked about earlier.19:20.06 
 Forward 1 day (to 2012/12/15)>>> 
ghostscript.com
Search: