IRC Logs

Log of #ghostscript at irc.freenode.net.

Search:
 <<<Back 1 day (to 2013/01/24)2013/01/25 
mvrhel_laptop henrys and ray_laptop: shipped off my answers to the customers color questions06:18.40 
  good night all06:20.42 
sebras tor8: did you see my comments about InputStream yesterday?09:37.24 
Robin_Watts sebras: So, if someone asks you to open a content:// URL where the data is passed as a stream, I don't think we can safely just save the data to a file and open that.10:43.27 
  Because that's insecure (people may not want their data exposed).10:43.48 
tor8 Robin_Watts: I think sebras meant to create a memory based fz_file from it11:09.49 
  Robin_Watts: sebras: ah, missed sebras last message from y'day. nvm.11:11.26 
Robin_Watts tor8: yeah, I'm going to push through changes to do it in memory.11:12.31 
  Scott turns 66 today.11:48.19 
paulgardiner I've found a place where fz_run_display_list applied to a bbox device is assumed not to throw. Is that okay, or should I stick try/catch around it?12:46.03 
Robin_Watts paulgardiner: ooh, probably should have a try/catch on principle.12:53.11 
paulgardiner Yeah. Might not need it now, but might do later on because of a change somewhere else.12:56.38 
tor8 Robin_Watts: didn't we make the device calls so they never throw?12:58.53 
  still, run_display_list may throw so we should guard just in case12:59.10 
Robin_Watts tor8: I think so, and indeed :)12:59.37 
  Or at least the device calls have to be careful about throwing because of the context in use.13:00.11 
  hmm. I can't immediately see anything that stops throws across the device calls in all cases.13:01.37 
paulgardiner The pdf device may throw because of reallocing the fz_buffer13:01.42 
tor8 paulgardiner: all device calls are wrapped by the "dev_null.c" functions, which catch and set a flag in the cookie if errors are encountered13:03.19 
  IIRC, robin did the work so I may have misunderstood the details13:03.49 
Robin_Watts tor8: I can't see that code in dev_null.c13:03.59 
tor8 Robin_Watts: wasn't that part of doing the push/pop state balancing on errors?13:04.20 
Robin_Watts There is code in there that keeps track of the depth of errors within pushes/pops.13:04.31 
tor8 or was that just limited to push/pop errors?13:04.38 
paulgardiner Ah. So should I leave as is?13:04.55 
Robin_Watts yes, it looks like we just track errors within push/pop.13:04.56 
  paulgardiner: All calls to run_list should be wrapped in a try/catch, I think.13:05.22 
tor8 right. it's not an important feature IMO. certainly not anything to rely on.13:05.30 
paulgardiner Right. Okay13:05.32 
tor8 so wrap in try/catch13:05.39 
paulgardiner Ta13:05.43 
tor8 Robin_Watts: some reviews up on tor/master15:07.44 
  Robin_Watts: fz_irect is still in progress, but I checked that the current fz_rect only code makes no diffs on sane15:08.02 
Robin_Watts tor8: I'm neck deep in jni at the mo, can I get to them in a bit?15:08.50 
tor8 Robin_Watts: no rush. I'm about to sign out for the weekend soon anyway.15:09.07 
  just wanted them out there before that15:09.15 
Robin_Watts ok. have a good one.15:09.16 
  So the fz_irect one is ready for commit now ?15:13.11 
  or should that one be rebased onto the top so I can commit all the others ?15:13.29 
  tor8^15:13.35 
  paulgardiner: Yeah. It was ****ing java not doing the write into the member.15:19.39 
  If I do it from the C, it's fine.15:19.52 
paulgardiner Nasty15:19.55 
tor8 the fz_rect one is ready, the fz_irect isn't pushed15:20.05 
  two of the others depend on the fz_rect so can't be easily rebased15:20.18 
Robin_Watts I dislike "fz_rect_covering_rect"15:21.37 
  That made sense when it was "fz_bbox_covering_rect", but now any rect covers itself.15:21.58 
tor8 Robin_Watts: me too. the use of it is confined to the draw_device and draw_affine though.15:22.23 
Robin_Watts fz_rect_round_outwards ?15:22.29 
  It's in the api though, right?15:22.45 
tor8 Robin_Watts: internal, IIRC. if not it should be internal.15:22.57 
  Robin_Watts: fz_round_rect -> fz_round_rect_fuzzy and fz_rect_covering_rect -> fz_round_rect was one thought15:23.24 
  any specific reason to have both though?15:24.06 
Robin_Watts I cannot think of one. We should check the commit that introduced fz_bbox_covering_rect though to be sure.15:24.59 
tor8 I think the reasoning was to let fz_round_rect potentially round down in a few instances15:25.42 
sebras Robin_Watts: tor8: I didn't anticipate the security perspective. though I doubt anyone cares on todays smartphones...15:27.07 
  Robin_Watts: tor8: how about accesing a memory buffer? that ought to be easy enough at least.15:27.29 
Robin_Watts sebras: I've just updated the code here so it works from a byteArray.15:28.05 
sebras Robin_Watts: tor8: more over I believe that any mail-service-providers would limit the attachment size and thereby effectively make it possible for us to use a memory area.15:28.14 
Robin_Watts My current test loads files as an InputStream, then pulls the data into a byteArray.15:28.38 
sebras Robin_Watts: we don't need to do that for _all_ content:// uris though.15:28.40 
Robin_Watts so I can test it without having a phone/tablet that passes in this way.15:29.10 
sebras then we wouldn't be able to access a gargantuan pdf file any more, right..?15:29.11 
Robin_Watts I'm about to change it back to use Files normally, and only to use byteArrays in the case where the content:// thing fails to open as a file.15:29.58 
sebras ah, great. :)15:30.19 
Robin_Watts When I have an apk built, could you get your friend to test it for me please?15:30.19 
sebras I can test it for you. I managed to reproduce it easily.15:30.34 
Robin_Watts Ah, cool.15:30.39 
  Thanks.15:30.49 
sebras just let me know. :)15:30.59 
  I tried to figure out the code last night, but I as simply not bright enough...15:31.17 
Robin_Watts It's taken me all day to make it work. Not aided by what I consider to be a bug in the java JNI system.15:36.52 
sebras Robin_Watts: why? which one?15:37.53 
  Robin_Watts: it looked relatively easy, but the sequence of logic tests with the uri threw me...15:38.26 
Robin_Watts Suppose I have a class A that has a constructor A(param)15:38.32 
  And that constructor does: member1 = param; member2 = nativeMethodOfA();15:39.21 
  where nativeMethodOfA is a native function (i.e. one called via JNI).15:39.46 
  You'd expect that within nativeMethodOfA, if you tried to read the value of member1 using the JNI functions you'd get 'param', right?15:40.18 
sebras Robin_Watts: I'm not sure what java/dalvik promises with regards to native functions.15:41.18 
  Robin_Watts: I guess the compiler is free to reorder the assignments if it can figure out that nativeMethodOfA() doesn't depend on member1 I guess.15:42.01 
Robin_Watts I have previously read members from inside JNI functions and it works fine.15:42.10 
sebras but sure, the naïve sive of me would agree with you.15:42.17 
  sive -> side.15:42.23 
  and you found that this doesn't happen I assume..?15:42.43 
Robin_Watts Right. The problem is that in this case, as far as I can see the java compiler (or optimiser or whatever) fails to realise that the native side might want to read the value of the array.15:43.00 
  s/array/member/15:43.07 
  That took me ages to debug.15:43.24 
  Change the code to pass param into nativeMethodOfA, and have that set member1 via the JNI functions, and it works fine.15:43.56 
  so, on what device are you reproducing this?15:44.51 
sebras my htc sensation.15:46.22 
  you can probably do it yourself if you want -- preview a pdf attached to a gmail-message from within the gmail app.15:47.01 
  crashes immediately.15:47.06 
  the reason being that the attachment is never saved to a file in the file system, but instead kept somewhere inside the gmail app...15:47.26 
Robin_Watts http://ghostscript.com/~robin/MuPDF.apk15:54.29 
  Hopefully that will fix it.15:54.40 
  Obviously any attempt to save the file back will fail.15:55.01 
  (silently, I fear)15:55.09 
  sebras: Oh, balls. 15:59.44 
  That won't work.15:59.46 
  Uploading a better version now.16:02.42 
sebras is installing the balled version...16:03.25 
Robin_Watts ok, fixed version online now.16:07.51 
sebras Robin_Watts: opening using com.artifex.mupdfdemo works well.16:13.32 
  Robin_Watts: how do I save a file?16:13.51 
Robin_Watts If you alter a file (by filling in form data) when you exit it gives you the option to save.16:14.44 
  Sorry, what do you mean by "opening using com.artifex.mupdfdemo works well" ?16:15.02 
sebras Robin_Watts: using the .apk you gave me, i.e. not the mupdf on google play.16:15.24 
  Robin_Watts: it would be named mupdfdemo, no..?16:15.40 
Robin_Watts Both mine and the one on google play are named com.artifex.mupdfdemo16:16.49 
sebras oh. then maybe I still have the old one here...16:17.24 
Robin_Watts The have the headline "MuPDF" name.16:17.36 
sebras but I did use yours when testing now.16:17.41 
Robin_Watts but the internal class that's called is "com.artifex.mupdfdemo"16:17.51 
sebras yes, and when you have two apps named the same thing then android will list the calss name when both contend for the same mime-/filetype when opening...16:18.53 
Robin_Watts right.16:19.19 
  oh, so you had an OLD version installed already? (i.e. one more than a week old)16:19.33 
sebras yes.16:19.49 
  Robin_Watts: I'm going to test this with a 21Mbyte file when I get home.16:22.33 
Robin_Watts all testing much appreciated. Thanks.16:23.05 
sebras I'll let you know.16:30.33 
Robin_Watts tor8: The fz_rect 'pass by value' thing. Yay for consistency, Boo for doing the wrong thing, IMAO.16:37.38 
  When we pass rects by value, that's 4 values on the stack/in registers.16:38.15 
  When we pass rects by reference, it's 1.16:38.33 
  When they get passed from function to function, that's 4 copies rather than 1.16:39.02 
  We'd be better moving to use references everywhere, IMAO.16:39.30 
  For ARM stuff, it's even worse.16:39.49 
  The first 4 args go in registers, more than that spill onto the stack which is slower.16:40.15 
  and with FP it's even worse.16:40.25 
  Likewise the following review. It's just the wrong thing to do.16:41.24 
  And, sorry to be a whinger, but I'm really not sure about the removal of fz_bboxes either.16:45.07 
  fz_new_pixmap_with_bbox makes perfect sense.16:45.18 
  fz_new_pixmap_with_rect, doesn't.16:45.27 
  fz_run_display_list makes sense with a bbox too.16:46.22 
  fz_clear_pixmap_rect_with_value - again, only makes sense with ints.16:46.53 
  We want to be using less floating point, not more :(16:47.47 
sebras Robin_Watts: how do you merge your desire for consistency with these opinions?16:47.50 
Robin_Watts sebras: Using fz_bboxes where we want integer things, and fz_rects where we don't.16:48.24 
  And we should consistently pass by reference, not by value.16:48.34 
sebras Robin_Watts: so no changes?16:48.37 
Robin_Watts For bbox->rect, broadly no changes. It's possible there are places where we could tweak stuff though, but it seems more or less right.16:51.39 
  Sorry, I hae to run to the vets. ill dog.16:51.46 
mvrhel_laptop good morning16:58.28 
  Robin_Watts: so it looks like the customer wants to detect source R=G=B and C=M=Y=0 like I had suspected. 16:59.16 
henrys I was very relieved to see they seem to understand the issues - I've seen projects like this given to newbies and it is years before we'd see revenue if it didn't' fail.17:00.49 
mvrhel_laptop henrys: yes. they seem to know what they want and have poked at gs enough to know what to ask17:01.51 
ray_laptop mvrhel_laptop: yep, you were correct. Unfortunately, that's not the easiest17:06.59 
  particularly for ICCBased source colors, it is going to be difficult17:08.05 
mvrhel_laptop ray_laptop: as I understand it, for source ICCbased colors the same rule will apply17:08.33 
ray_laptop although we can maybe just look at the colorspace of the tint transform and then do the appropriate check on the components17:08.57 
mvrhel_laptop that is if R=G=B for the source ICC RGB color, we consider it neutral17:09.01 
  like wise C=M=Y=0 we consider neutral for ICC CMYK based color17:09.18 
  so putting a hook in the CMM for this may make sense17:09.39 
  where we monitor the incoming source colors17:09.55 
  once one bad (non neutral color is found) we stop checking17:10.17 
  if we never find a non neutral color, then the source CMYK buffers at the end are converted to K17:10.44 
ray_laptop mvrhel_laptop: right, that's IFF the tint transform is mapping RGB or CMYK components, but that's not always the case17:10.48 
mvrhel_laptop ?17:10.57 
ray_laptop ICCBased doesn't have to be RGB or CMYK like17:11.24 
mvrhel_laptop I think one can safely say that for most ICC based color spaces, R=G=B implies neutral. and C=M=Y=0 implies neutral17:11.38 
  I have never seen one for which this rule did not apply17:12.02 
  I could make one that breaks this easily17:12.14 
  I guess I should be careful with my words17:12.23 
  I have made many profiles that break this17:12.33 
  but only for weird demos17:12.41 
  the bummer is that we would have to check the buffers that are coming in from images17:14.53 
  I could write an SSE2 operation for that17:15.14 
ray_laptop mvrhel_laptop: and for Lab we just check the a and b values as we discussed yesterday ?17:15.15 
mvrhel_laptop but not sure if that would help them17:15.20 
  ray_laptop: yes with respect to the Lab, but that would require a conversion and a check of values17:15.41 
ray_laptop mvrhel_laptop: I don't know what processor they are using17:15.44 
  I'll go back through all of the emails from last April and May to see if they mention CPU type.17:16.27 
mvrhel_laptop ok. sounds good17:16.42 
ray_laptop mvrhel_laptop: like you mentioned above, skipping the check once we see color might help17:17.41 
mvrhel_laptop right.17:17.55 
ray_laptop but the clist implementation isn't worth enhancing since that is (usually) AFTER colors are converted to CMYK and may not work17:19.09 
  especially if some of the object type device profiles 'fudge' the colors to make a "better" black/gray17:20.08 
mvrhel_laptop ray_laptop: yes. exactly17:24.10 
ray_laptop henrys: I see what's going on with bug 693586. It looks like it is outputting a page BEFORE it gets to the end of an image. After outputpage, we do clist_reset (which clears the cdev->image_enum_id)17:38.20 
tor8 Robin_Watts: if we want to pass by reference, we have to use const and make local copies everywhere when modifying the ctm.17:38.21 
  and on 64-bit architectures, a fz_rect is two words ... barely noticeable ;)17:39.07 
  Robin_Watts: and with the names of functions, I sort of agree. it would be better as _with_bounds or _with_bbox not _with_rect.17:41.18 
henrys ray_laptop:very strange not so clear how that could happen.17:44.31 
ray_laptop henrys: then when PCL continues to close_job, in close_raster it finds prast->src_height > prast->rows_rendered 17:44.32 
  henrys: I haven't looked at the PCL, but if it ends the file without enough data for the image ?17:45.27 
henrys that's fine it should fill in the remaining scan lines with black17:45.51 
  but I'll have a look again at it in a little bit. Probably easier to understand the PCL side first.17:47.02 
ray_laptop henrys: it does "process_zero_rows", but that's on the next page -- that's how we get into the clist logic and the pie->id (that was started on the previous page) doesn't match what's in the clist (since it was "reset" to gs_no_id)17:47.30 
henrys there certainly shouldn't be an output page after starting an image and before process zero rows.17:48.24 
  pcl does auto feed but it should never split a single graphic17:49.41 
ray_laptop henrys: I don't know why it's deciding to output the page, but doesn't PCL do that automatically when we image past the end of the page ?17:49.54 
henrys see my last17:50.31 
ray_laptop henrys: and as far as writing on the next page, since prast->pen wasn't reset, we don't get a clist_begin_image call on the next page17:50.53 
  so it looks like PCL is deciding it still wants to finish the image (with zero_row fill) -- there are 0x7a == 122 raster rows to go17:52.15 
henrys but why doesn't something break full frame?17:52.23 
ray_laptop I don't know -- let me turn look with page mode ...17:52.59 
  henrys: BTW, the page is being output by pcl_impl_process_eof (because the page is marked), so it must be a premature EOF17:55.16 
henrys ah yes let me take over this one - I know what is going on. I added end_graphics to the end_job thing but didn't do eof, maybe that is it.17:56.45 
  do you have the latest code?17:57.14 
  if so I'm certain that is it.17:57.35 
  bbiab17:58.20 
ray_laptop henrys: (for the logs) the page mode case simply goes ahead and calls 'fill_rectangle' on the new page (it uses the default image handling)18:02.28 
mvrhel_laptop bbiaw18:03.57 
Robin_Watts mvrhel_laptop: (For the logs). Yes, you were correct. I'm glad we have a knowledgeable customer that agrees with you!18:09.28 
  tor8: Yes, we'd use const, and we'd make local copies - but that would still result in less work overall.18:11.01 
  on 32bit ARM (our largest target arch), an fz_rect is 4 words, and more importantly, a lump of copying on every single function call that passes or returns an fz_rect.18:11.43 
ray_laptop henrys: (for the logs). I put the explanation in the bug and assigned it to you.18:12.42 
henrys ray_laptop:right thanks18:13.01 
ray_laptop henrys: oops didn't notice you were back.18:14.46 
sebras Robin_Watts: it fails byg SIGSEGV...19:46.29 
Robin_Watts sebras: With the 21 Meg file?19:46.53 
sebras y19:47.28 
  I/ActivityManager( 247): Process com.artifex.mupdfdemo (pid 7099) has died.19:47.38 
  appears to happen inside libmupdf.so though.19:48.07 
Robin_Watts sebras: Do you not get a traceback for the java?20:00.00 
  I'll have to try to reproduce it locally, but that won't be tonight :(20:00.05 
sebras mailed you a log.20:01.18 
Robin_Watts I have a crash with my old test version, but it's an OutOfMemory exception, not a SEGV.20:01.50 
  Thanks.20:02.00 
sebras is your java-patch available?20:02.21 
  it's not obviously at robin/master...20:02.32 
  if you push it then (even as WIP) I'll take a look.20:02.58 
  Robin_Watts: ping?20:09.28 
Robin_Watts sebras: sorry, will push now.20:34.33 
sebras Robin_Watts: thanks! hopefully I have solved it when you get back tomorrow/monday. :)20:49.01 
ray_laptop so is this too late to put in my change to pdf14 clist optimization ? 22 files changed and only 6 files show differences that seem to not be related to the change21:04.44 
  I'll email tech to see what people think.21:05.14 
  all of the actual diffs (after bmpcmp) seem to be halftoning related (plank or pkmraw) so I don't think they are due to my changes. Only one of them even triggers the optimization (others are either have no transparency or don't skip pdf14 for any bands)21:11.49 
mvrhel_laptop ray_laptop: is your optimization for pdf14 the thing that avoids the pdf14 push in bands that have no transparency?21:13.40 
ray_laptop mvrhel_laptop: yes21:14.58 
mvrhel_laptop ray_laptop; so you assigned 693583 to me. Do we want to look at doing the approach that I talked about. Adding in the monitor into the CMM21:14.59 
  ray_laptop: I am of the opinion that your optimization should go in21:15.15 
  the one customer has been using it, correct?21:15.38 
ray_laptop mvrhel_laptop: I was going to talk to you about that. It seemed to assign it to you automatically when I changed the component to "Color"21:15.40 
mvrhel_laptop ah. ok. You can reassign it back to you if you want :)21:16.00 
  ray_laptop: but if you want me to work on it that is fine too, but I think it will involve more than one person for sure to get the best most efficient design21:16.35 
ray_laptop mvrhel_laptop: well, since it isn't something I can do in the clist, I was hoping your expertise with the color flow would give you an advantage ;-)21:16.58 
mvrhel_laptop that is fine. I will likely want some feedback from people about efficiency as well as how best to hook in to transform the band buffers21:17.43 
  ray_laptop: if that is the approach that we want to go with21:17.58 
ray_laptop transform the band buffers ? do you mean map CMYK to K after rendering ?? That's easy !21:19.04 
mvrhel_laptop ray_laptop: yes that is the part21:19.25 
ray_laptop mvrhel_laptop: it seems obvious that the clist will be written in CMYK mode (since we don't know if the page has color until the end)21:19.56 
mvrhel_laptop right21:20.16 
  ray_laptop: actually, a question about that21:20.39 
  when the writer finishes, we will know if we need to go to CMYK or gray21:20.56 
ray_laptop mvrhel_laptop: but the other thing I could do is a "special" buf_device (image32to8) that takes the 32-bit colors in and just paints the 8 bit buffers directly21:21.09 
  mvrhel_laptop: but the simplest approach is to just write the CMYK buffer and then smash it down to K only21:21.57 
mvrhel_laptop ok. maybe we do the simplest first21:22.16 
ray_laptop mvrhel_laptop: that may be more of a performance hit, however, since it has to touch every pixel in the CMYK buffer and write the K buffer21:22.55 
mvrhel_laptop ray_laptop: if we know when we are doing the clist reading what we want, is there any special device that can be inserted at that point in time to handle the remap?21:23.37 
ray_laptop mvrhel_laptop: I think the image32to8 will be fairly simple. We only need to "front end" the existing 'image8' functions with functions that collapse the color to K.21:24.18 
  copy_color is probably the only hard one21:24.50 
mvrhel_laptop ok. we don't need to do ICC for this, I think it could be done in an ugly way (ugly from a color management viewpoint)21:25.19 
ray_laptop mvrhel_laptop: right. No ICC profiles needed. Just a "quick color" smashing21:25.52 
mvrhel_laptop ray_laptop: sounds good. I will start looking at the monitoring in the CMM21:27.15 
ray_laptop three cases, really. CMY with K==0 (just use M), C=M=Y=0 and K (use K) and occassionally CMYK all non-zero (that's the trickier one, maybe)21:28.04 
mvrhel_laptop oh. you mean for the smashing from CMYK to K?21:28.49 
ray_laptop but for CMYK I'd go with adding M to K and clamping21:28.51 
mvrhel_laptop ray_laptop: yes. the use of M and K in some clever way would work21:29.24 
ray_laptop so I guess M+K clamping works for all21:29.33 
  mvrhel_laptop: clever as in adding then limiting to 25521:29.56 
mvrhel_laptop ray_laptop: if we find out that are using intel then we could look at SSE2 for this21:30.06 
ray_laptop mvrhel_laptop: true, but if we are doing it in an image32to8 device, it doesn't lend itself to SSE2 (that's more for transforming entire buffers)21:31.03 
mvrhel_laptop oh. ok. Yes, I was thinking of buffers. You are right that for the device approach this does not apply21:31.43 
  but maybe for high level images?21:31.56 
ray_laptop the pdf14 put_image would be a place to consider SSE2, however21:32.20 
mvrhel_laptop oh. there too21:32.39 
  I need to head out for bit. I will poke around a bit in the CMM code to see what I can do on that side with respect to monitoring21:33.16 
ray_laptop most images (unless interpolated) aren't at full device resolution. And interpolated uses copy_color (where SSE2 may be useful)21:33.20 
  mvrhel_laptop: OK. Thanks21:33.37 
mvrhel_laptop ray_laptop: right. ok. ttyl21:33.56 
sebras Robin_Watts: I now know what happens. in the middle of the parsing array is all of a sudden == NULL, i.e. fileBuffer == null.22:51.18 
  this appears to happen after the inner fz_try() in MuPDFCore_openBuffer() has finished. could it be that they array goes out of scope?22:56.55 
  the next seek in the buffer fails because the seek function assumes that the array is still present.22:57.29 
Robin_Watts sebras: hi23:58.03 
sebras Robin_Watts: ok, so I added another fileBuffer member to MuPDFCore and assigned it just as you outlined in the comment. then I dereference and print the pointer values of both when reading/seeking/closing.23:58.35 
  it turns out that the hack reference you added is non-NULL until it goes out of scope in ths fz_try() I mentioned. and the normal Java object member is NULL up until this point, and then it becomes non-NULL(!).23:59.33 
Robin_Watts The hack reference?23:59.50 
 Forward 1 day (to 2013/01/26)>>> 
ghostscript.com
Search: