| | <<<Back 1 day (to 2014/05/07) | 2014/05/08 |
| mvrhel_laptop | blah. a bit of work to make sure gsview is set up for 64bit and 32bit and to make sure things are handled gracefully if DLLs are missing or we have a mismatch | 06:11.04 |
| | not hard just a bit of abstraction | 06:11.24 |
| | done for the night though | 06:11.37 |
| kens2 | chrisl there is (I hope) a commit on my repository to handle RGB images | 08:57.36 |
| frederick_sim | Hi, I wanted to use muPDF on Xamarin platform | 08:59.24 |
| chrisl | kens2: yep, got it | 09:00.37 |
| kens2 | aha, thanks chrisl, good to know I didn't mess that up | 09:00.55 |
| | chrisl I looked at trying to get the JPEG decoder memory sorted out, and got thoroughly lost :-( | 09:01.18 |
| chrisl | kens2: I looked at that a while back, and got pulled off onto something else - I'll look at it next | 09:01.54 |
| kens2 | When you have time it would help, a lot of my test files keep falling into it. I'm going to look at Indexed colour now, I have a test case for that. | 09:02.29 |
| Robin_Watts | frederick_sim: OK... | 09:10.00 |
| chrisl | kens2: do you happen to have a simple test with only a jpeg image in it? Save me searching one out? | 09:10.53 |
| kens2 | Offhand no, sorry | 09:11.03 |
| | every time I come across one I reject it and find another test file | 09:11.29 |
| chrisl | OKay, I'm trying to find one without transparency or other nonsense in it | 09:11.53 |
| kens2 | just a second I'll look as well | 09:12.09 |
| frederick_sim | i downloaded the source it's in android NDK | 09:13.08 |
| | just wondering how to change it into xamarin android project | 09:13.37 |
| kens2 | chrisl 01_0001.pdf looks plausible | 09:13.59 |
| Robin_Watts | frederick_sim: I know nothing about xamarin, just looking now. | 09:14.40 |
| | So it's a C# based system? | 09:14.45 |
| chrisl | kens2: I don't seem to have that - is it in tests_private? | 09:15.04 |
| kens2 | that's what the web site syas ;-) | 09:15.04 |
| | It might be jst mine chrisl | 09:15.13 |
| | and it has an SMask, so froget that | 09:15.32 |
| jhabjan | hi, anyone here knows in which gs release "gsapi_set_arg_encoding" api is introduced? | 09:15.37 |
| Robin_Watts | frederick_sim: MuPDF is entirely written in C. | 09:15.38 |
| | jhabjan: 9.10 or later, IIRC. | 09:16.01 |
| chrisl | kens2: don't worry about it, I'll have a file I can use somewhere | 09:16.12 |
| kens2 | Maybe Bug688159.pdf, just a mo | 09:16.26 |
| Robin_Watts | frederick_sim: So unless you plan to rewrite it into C# (or unless Xamarin supports C, which seems unlikely) I think you're out of luck. | 09:16.34 |
| jhabjan | Robin_Watts: thanks | 09:17.04 |
| kens2 | chrisl, yes Bug688159.pdf is a simple JPEG image and nothing else | 09:17.21 |
| | D'oh that's a JPX file, not JPEG, oops | 09:17.57 |
| chrisl | kens2: it's okay, I think I've found one - it's got an ICC color space, but I don't suppose that matters for this | 09:18.18 |
| kens2 | No, that'll be fine | 09:18.27 |
| | I think it barfs long before we get anywhere near worrying about that | 09:18.44 |
| chrisl | I'll make a start this afternoon | 09:19.17 |
| kens2 | OK thanks (back to colour spaces...) | 09:19.30 |
| tor8 | sebras: because they're not part of adobe's set of files... | 09:57.35 |
| | for some reason they're missing, along with Adobe-Japan2-0 | 09:57.54 |
| | the Hojo files seem related to the Japan-2 CID set, which I think was released but then obsoleted so we might be able to get away with deleting the Hojo CMaps | 10:02.10 |
| sebras | tor8: why don't we use jxrlib for jpeg xr in mupdf? | 10:22.01 |
| | tor8: I thought there was a licensing problem preventing us from using it..? | 10:22.14 |
| chrisl | sebras: tor8 doesn't like jpegxr and would prefer it didn't exist..... ;-) | 10:22.57 |
| sebras | chrisl: right, but it seems like it would be useful to be able to decode them. | 10:23.25 |
| chrisl | gxps can..... | 10:23.51 |
| sebras | chrisl: true. | 10:24.47 |
| chrisl | sebras: 'course it's the just M$ code. I *think* the license is not compatible with the GPL, and not sufficiently open for Debian and the like | 10:26.04 |
| frederick_sim | @Robin_Watts thanks | 10:26.06 |
| sebras | chrisl: it is licensed under New BSD and under their patent promise. | 10:26.36 |
| | chrisl: there are links to this at the bottom om the wikipedia page. I haven't read the references though. | 10:26.56 |
| kens2 | That sounds like it has changed, I don't remember it being like that before | 10:27.24 |
| chrisl | It always was fairly permissive: ".....to copy, distribute, and make derivative works" | 10:28.51 |
| | But I think the problem was: only in products "claiming conformance to the JPEG XR standard" | 10:29.43 |
| tor8 | sebras: I took a brief look at jxrlib and it looks like it's the same code as the reference implementation | 10:34.40 |
| | which is absolutely dreadful garbage code | 10:34.51 |
| chrisl | But is does seem to be under a more permissive license than the original release | 10:35.32 |
| tor8 | with a new bsd license without the "only if you conform to JPEG XR and don't extend" clause it might be usable | 10:35.39 |
| | oh, it looks like they've cleaned up most of the warnings too | 10:37.19 |
| | but they still massively pollute the namespace with public functions that ought to be private | 10:38.02 |
| | my bad, none of the functions we use in the gxps implementation exist, so I guess it really must be a different code base | 10:40.55 |
| Robin_Watts | tor8: In what way don't we conform to JPEG XR ? | 10:42.24 |
| tor8 | Robin_Watts: that clause is incompatible with debians license guidelines | 10:42.39 |
| Robin_Watts | oh, right, well, debian. pfft. | 10:42.53 |
| tor8 | oh, I think what I might have confused me. the official WMP dev kit is what this code looks like, not the ITU reference implementation | 10:43.03 |
| sebras | Robin_Watts: hey! I use debian. :) | 10:43.19 |
| tor8 | Robin_Watts: might also make it incompatible with GPL | 10:43.21 |
| | back when JPEG-XR was still called windows media photo | 10:44.04 |
| | or maybe it was when it was called HD-Photo | 10:44.19 |
| | what is it with microsoft and never settling on a name for something? | 10:44.33 |
| chrisl | I think the BSD licenses are accepted by Debian | 10:44.52 |
| | As long as it's not "based on BSD with further restrictions"...... | 10:45.29 |
| tor8 | yeah. the problem with this code, if it's not improved since the original WMP SDK release ... it makes libpng look sane. | 10:46.07 |
| | the interface is just plain impossible, and you need THOUSANDS of lines of boilerplate just to decode the image | 10:46.32 |
| | well, thousands may be an exagerration, but the sample code is full of beautiful examples like this: | 10:47.43 |
| | Call( PKCreateFactory(&pFactory, PK_SDK_VERSION) ); | 10:47.50 |
| | Call( PKCreateCodecFactory(&pCodecFactory, WMP_SDK_VERSION) ); | 10:47.58 |
| chrisl | But JPEG XR is required by XPS, so if you're supporting XPS...... | 10:48.05 |
| tor8 | Call( pDecoder->GetPixelFormat(pDecoder, &pix_frmt) ); | 10:48.06 |
| | chrisl: yeah... | 10:48.27 |
| | there is a bug open for it :) | 10:48.34 |
| chrisl | tor8: you could always drop XPS support...... | 10:48.56 |
| kens2 | tor8 2 questions related to indexed colour and Fitz: | 10:51.51 |
| | 1) in fz_ghost_fill_image, if the indexed space is not a RGB base space, will the indexed->base point to the original space ? | 10:51.51 |
| | 2) When the image does have an RGB base space, is the lookup table RGBa rather than RGB | 10:51.51 |
| tor8 | the lookup table is RGB (or CMYK) or whatever number of components the base space has. no alpha channel. | 10:54.11 |
| kens2 | OK that helps thanks. I guess I must be doing sdomething else wrong then | 10:54.41 |
| tor8 | kens2: are you calling fz_new_pixmap_from_image? | 10:55.35 |
| kens2 | Umm, not sure. | 10:55.44 |
| | One moment | 10:55.47 |
| | ATM I@m calling fz_image_get_+pixmap() Unless image->tile is non-NULL | 10:56.21 |
| tor8 | I expect you do (or some variant of it) or you'll have compressed image data | 10:56.44 |
| kens2 | Which I believe means its already cached | 10:56.45 |
| | Nope, just calling get_pixmap | 10:57.12 |
| tor8 | kens2: that's what fz_new_pixmap_from_image calls | 10:57.38 |
| kens2 | OK, would I be better wiht the higher level call ? | 10:57.57 |
| tor8 | source/fitz/draw-device.c has a (more complicated than you need) example of how things fit together | 10:58.05 |
| kens2 | I see this one doing decompression | 10:58.07 |
| tor8 | kens2: yes, I think you'd be better off with the higher level call | 10:58.36 |
| | then we do all sorts of pixmap scaling and color conversion into the destination colorspace | 10:58.49 |
| kens2 | OK I@ll look at that. Not sure why I'm using this one | 10:58.52 |
| tor8 | which you can safely ignore | 10:58.56 |
| kens2 | Eeek, I don't want colour conversion or scaling | 10:59.04 |
| tor8 | yeah. so don't call fz_transform_pixmap or fz_convert_pixmap :) | 10:59.41 |
| kens2 | OK..... | 10:59.56 |
| tor8 | but new_pixmap_from_image does the decompression into a color+alpha pixmap | 11:00.02 |
| kens2 | I'm already getting that for RGB (and undoing it again) | 11:00.21 |
| tor8 | but I think your original question was about the colorspace | 11:00.22 |
| kens2 | Sort of, it was limited to Indexed spaces, as the others seem OK at the moment | 11:00.39 |
| tor8 | yeah. if you want you could run fz_convert_pixmap on the indexed colorspace to get it into the base colorspace | 11:00.59 |
| kens2 | I'd prefer just to manufacture an indexed space to send to GS | 11:01.26 |
| tor8 | Robin_Watts has the details of this more fresh in his head I think, he's messed quite a bit with the image/pixmap conversion steps | 11:01.32 |
| kens2 | I'm creating the space myself now, and it all 'works' but the output looks incorrect. | 11:01.55 |
| Robin_Watts | mmm, sorry? what ? | 11:02.21 |
| tor8 | the indexed colorspace in mupdf is basically just a base colorspace + packed unsigned char lookup table | 11:03.14 |
| Robin_Watts | Conceptually fz_images are just handles to an image. You should call fz_new_pixmap_from_image to get a pixmap. | 11:03.29 |
| kens2 | That's what I thought, and I'm just pulling the data into an equivalent GS colour space, so it *ought* to work..... | 11:03.42 |
| Robin_Watts | The specific method used is intended to be hidden as an implementation detail. | 11:03.56 |
| tor8 | indexed image data is of the form (index, alpha) per pixel | 11:04.27 |
| | unless Robin_Watts has changed that while I wasn't looking | 11:04.41 |
| kens2 | AH, that would be it then, I haven't stripped the alpha | 11:04.42 |
| | Where does alpha live with a CMYK image ? | 11:05.01 |
| tor8 | kens2: yeah. for simplicity in implementation all fz_pixmaps have an alpha channel | 11:05.02 |
| | CMYKA | 11:05.04 |
| Robin_Watts | tor8: Nope, not messed with that. | 11:05.06 |
| tor8 | alpha is always last | 11:05.19 |
| Robin_Watts | Alpha is always the last entry. | 11:05.21 |
| kens2 | OK so I need to strip alpha for all num_components, I'm only doing it for RGB at the moment | 11:05.31 |
| Robin_Watts | It used to be the first, but we changed that for some reason. | 11:05.35 |
| kens2 | Robin_Watts : SO I gathered, hence my email to tech this morning | 11:05.42 |
| tor8 | Robin_Watts: I believe so we would have fewer mismatches with other APIs that expect alpha last | 11:06.04 |
| | like PNG and GDI+ and OpenGL | 11:06.12 |
| Robin_Watts | kens2: I ignored that, cos it looked like a gs thing. | 11:06.41 |
| kens2 | Robin_Watts : it is.... | 11:06.47 |
| tor8 | kens2: you might want to check if any of the alpha values are != 255 ... some image formats have an embedded softmask channel | 11:07.07 |
| Robin_Watts | yeah, I didn't realise this was a mooscript thing. | 11:07.07 |
| tor8 | and I'm not sure if we have a flag for that | 11:07.12 |
| kens2 | I was trying to tell GS to use an RGBa image, and have it drop the alpha rather than me doing it | 11:07.13 |
| | tor8 that's waaaay beyond where I am at the moment :-) | 11:07.37 |
| tor8 | kens2: okay :) | 11:07.46 |
| kens2 | will be happy to get a few more basic images working today | 11:08.04 |
| tor8 | I think it only happens for JPEG2000 in PDF, and we apply the color key transparency on the alpha channel when decoding | 11:08.34 |
| kens2 | Well, given that JPEG decoding doesn't work yet, I'm not going to worry about JPX | 11:09.08 |
| chrisl | I don't think the graphics library currently handles images with an alpha channel - hence JPX images with alpha we have to decompress twice, once to setup a softmask from the alpha, and a second time to get the image samples | 11:09.25 |
| tor8 | is it just me or is the cluster behaving oddly today? | 11:09.48 |
| kens2 | That can be done at the time I guess. I can check for alpah while I strip it and abort and retry if its no 255 | 11:10.02 |
| | For now I plan to ignore it, still fumbling through at this point. | 11:10.25 |
| tor8 | you could restart and split out the alpha channel if it is not all solid | 11:10.29 |
| chrisl | tor8: I haven't used the cluster today - odd in what way? | 11:10.29 |
| tor8 | chrisl: I'm getting thousands of diffs where I expect none, in files that shouldn't be affected in any way by the patch | 11:10.54 |
| | I have added a few CMap resources, and am seeing LOTS of diffs | 11:11.22 |
| chrisl | In XPS files, too...... | 11:11.39 |
| tor8 | let's see if bmpcmp tells me anything | 11:11.43 |
| kens2 | I haven't done a cluster push recently | 11:11.45 |
| | Is it only at 300 dpi, or 72 as well ? | 11:12.14 |
| | Hmm, no both, very odd | 11:12.46 |
| chrisl | And these aren't the new tests marcosw added - or maybe they are new for mupdf?? | 11:13.23 |
| tor8 | I have no idea | 11:13.46 |
| | they don't look new. I've seen the Ghent files in previous clusterpush diffs | 11:14.01 |
| kens2 | Some of them can't be new, the bug693006.pdf for example | 11:14.13 |
| | CityMap is not new either | 11:14.28 |
| | in faxt none of them look new | 11:14.44 |
| | Want me to try a GS push and see what happens ? | 11:15.10 |
| chrisl | They may not be new to us, but I thought they might be new to mupdf - but no, there's a bunch of sumatra tests in there | 11:15.39 |
| | Could be indeterminisms | 11:16.00 |
| kens2 | I didn't think MuPDF had any | 11:16.23 |
| Robin_Watts | tor8: Have you got my fast_cmyk_to_rgb fix in? | 11:20.58 |
| | If you push a vanilla tree does it show 0 diffs? | 11:21.14 |
| kens2 | Hmm looks like I have a black/white mixup, white is coming out grey and black is coming out white | 11:22.54 |
| | Oh, and half my image is corrupted :-( | 11:23.11 |
| | Time to make a reduced test file I think | 11:23.37 |
| | After lunch | 11:23.42 |
| | tor shadings seem different at a first glance. | 11:27.32 |
| | Not wrong, just different | 11:27.39 |
| | Actually I take that back, its the reference images that look different | 11:28.10 |
| | change in a JPEG/JPX decoder maybe ? | 11:29.00 |
| DaDaDadeo | I found a discussion that jroo successfully cross compiled ghostpdl back on 2014/03/12. I am able to create teh pcl6 binary, but it only runs on the build machine. | 11:34.30 |
| Robin_Watts | DaDaDadeo: Then you aren't cross compiling it :) | 11:35.42 |
| DaDaDadeo | I believe the arch.h file is not set correctly. I am stuck on "manually generate arch.h" with my level of experience. I am trying to run pcl6 on an ARM9. | 11:36.36 |
| Robin_Watts | essentially you should ensure that CC is set to the compiler you want to use for building target binaries, and CCAUX is set to the compiler you want to use for building binaries on the host machine. | 11:36.37 |
| | ARM9 LE ? compiling on an x86 PC? | 11:37.22 |
| DaDaDadeo | my configuration completes with no errors with this: ./configure CC=arm-linux-gcc CCLD=arm-linux-gcc CCAUX=gcc --host=arm-linux --target=arm-linux --without-x | 11:38.25 |
| Robin_Watts | DaDaDadeo: You're building on a PC? then the --host looks wrong to me. | 11:40.00 |
| DaDaDadeo | Yes. I am using a PC for an ARM9 LE | 11:40.43 |
| Robin_Watts | 32 or 64bit OS on the PC ? | 11:41.11 |
| chrisl | DaDaDadeo: I don't think you can cross compile just from the configure script - I'm fairly sure we don't support that (yet). You'll need to edit Makefiles, too | 11:41.24 |
| Robin_Watts | If it's a 32bit PC, then you'll probably find that the arch.h is pretty much right. If it's a 64bit PC, probably not. | 11:42.01 |
| DaDaDadeo | chrisl may have provided what I feared most. I will have to dig in then. | 11:44.30 |
| chrisl | DaDaDadeo: there is a (slightly vague and incomplete) overview at the bottom of the page here: http://ghostscript.com/FAQ.html | 11:45.58 |
| DaDaDadeo | The FAQ was one of my first finds. I found the discussion you had with jroo (web search) that started on 2014/03/11 and jroo described the CC the next day. I | 11:49.47 |
| chrisl | DaDaDadeo: the ghostpcl/ghostxps builds are slightly harder than the ghostscript one to cross compile, but it shouldn't be hugely difficult | 11:51.32 |
| DaDaDadeo | chrisl mentioned that you can leave out libtiff, I try using --without-system-libtiff in the main configuration but the log indicates it does not recognize the command. | 11:53.49 |
| chrisl | DaDaDadeo: that doesn't leave out libtiff. I think you'll need to remove the tiff devices from the makefile(s). But I think if you use the libtiff we ship, it should be okay. | 11:55.06 |
| | Also, --without-x won't have an effect on the ghostpcl build either | 11:57.21 |
| DaDaDadeo | There are two locations that I can congigure. First is the main folder and then the gs folder. When I configure in the main folder, I get an error regarding libitff because of the --host. I can then configure on the gs folder with no libtiff error. Either way. i still get a working pcl6 binary (PC only) | 11:58.14 |
| chrisl | configure in the gs directory is for gs, not ghostpcl | 11:58.46 |
| tor8 | Robin_Watts: yes, your fast_cmyk_to_rgb fix is in | 12:01.30 |
| | the bmpcmp makes me think something's going on with jpeg decoding... | 12:01.42 |
| kens2 | tor8 it looks like JPEG decodingto me | 12:02.58 |
| Robin_Watts | tor8: Are you git submodule updated? | 12:03.12 |
| kens2 | Inthe large images the diffs are at what look like tile boundaries | 12:03.15 |
| Robin_Watts | Are you using clusterpush.pl or gitpush ? | 12:03.28 |
| tor8 | kens2: which is odd, since that hasn't changed... but maybe there are git submodule or stale build configuration stuff around | 12:03.31 |
| | clusterpush.pl | 12:03.33 |
| | I have never gotten around to setting up gitpush | 12:03.40 |
| Robin_Watts | clusterpush.pl will push your local jpeg libs too, I believe. | 12:04.03 |
| tor8 | yes. and now I see what's wrong! | 12:04.24 |
| | I've managed to mess up my submodules so it's picking up the system libjpeg | 12:04.36 |
| Robin_Watts | ah! | 12:05.04 |
| tor8 | maybe that gitpush thing would be worth setting up! | 12:05.27 |
| Robin_Watts | gitpush has other foibles :) | 12:05.53 |
| tor8 | Robin_Watts: so, the commits on tor/master, I could use some opinions here | 12:09.45 |
| Robin_Watts | tor8: which ones specifically? | 12:10.42 |
| tor8 | I have added the UTF8 and UTF32 CMap resources, but there is a handful of ones I'm reluctant to put in (they're on tor/cmap-japanese-utf8) because they are all so big | 12:10.43 |
| | and the removal of the adobe-japan-2 based ones | 12:11.01 |
| | gs doesn't ship those, but I vaguely recall having added them in due to some bug report or other | 12:11.23 |
| | but I can't be 100% certain | 12:11.27 |
| Robin_Watts | tor8: does a git history give any clues? bug number in the commit message? | 12:11.53 |
| tor8 | it just says "add new open source cmap resources" | 12:12.04 |
| | so I think I've added them with no real concern | 12:12.10 |
| Robin_Watts | We crunch the cmaps, right? They don't go into the exes in their raw forms. | 12:15.04 |
| tor8 | yeah. but adding in the extra JISX2004 and JISX0213 tack on another 700k | 12:15.31 |
| | might be balanced out by dropping the Hojo CMaps though | 12:15.50 |
| Robin_Watts | will there not be files out there that use these old cmaps ? | 12:16.26 |
| tor8 | that's what I don't know | 12:16.43 |
| Robin_Watts | We don't have mechanisms for loading external CMAPs do we ? | 12:17.26 |
| | MUPDF_CMAP_DIR or something. | 12:17.49 |
| tor8 | Robin_Watts: we do, but because that entails runtime configuration of a directory we don't | 12:17.56 |
| | well, we have functions to load them, but not to actually locate the files | 12:18.19 |
| Robin_Watts | If we added an environment var, then we could feel happier about dropping cmaps from the exes. | 12:18.33 |
| | keep the exes small etc. | 12:18.50 |
| tor8 | the UTF-8 cmaps have other issues (they use input ranges >16 bit) | 12:19.32 |
| | yeah, but then we lose the single self contained binary benefits | 12:19.58 |
| | so I'm thinking we probably need to extend the pdf_cmap datastructure with 32-bit ranges as well | 12:20.19 |
| | that would be zero for most cmaps, but having a separate 32-bit range array would save us the bloat of making all ranges 32-bit | 12:20.47 |
| Robin_Watts | tor8: All the commits in tor/master look fine to me (except the CMAP ones we are discussing) | 12:20.56 |
| | tor8: I concur. | 12:21.09 |
| tor8 | thanks. I'll rebase and push the non-cmap ones | 12:21.23 |
| Robin_Watts | Supporting the larger cmaps would be nice. | 12:21.43 |
| tor8 | yeah. it's been annoying me that we don't. | 12:22.04 |
| Robin_Watts | Part of me says "put all the cmaps in the exe and be done with it. We're not transferring stuff on floppies anymore." | 12:22.10 |
| tor8 | but I've been reluctant to double the size of the cmap data structures just to get a few more bits in | 12:22.36 |
| | then I thought of having two separately sized arrays yesterday | 12:22.48 |
| Robin_Watts | Part of me says "Have the smallest sane set in the exe and then a complete set optionally on disc". | 12:22.54 |
| tor8 | Robin_Watts: you're probably right. | 12:22.57 |
| chrisl | You already have a build option to exclude CMaps, don't you? | 12:23.22 |
| Robin_Watts | I wonder if we can be more cunning, and share sections from CMAPs. | 12:23.45 |
| tor8 | chrisl: yeah, we do. -DNOCJK | 12:23.45 |
| | Robin_Watts: many of them already do with "usecmap" | 12:24.04 |
| Robin_Watts | i.e. if lots of the cmaps have overlapping sections, can we combine them? | 12:24.10 |
| chrisl | tor8: so -DNOOBSOLETECMAPS or something | 12:24.19 |
| Robin_Watts | tor8: our compressed versions will not respect that though. | 12:24.20 |
| tor8 | but maybe we could massage them and find ones that are pure subsets of others, and just use the superset | 12:24.24 |
| | the compressed versions also have the chained usecmap lookups | 12:24.50 |
| Robin_Watts | OK, so it doesn't cost us much then. | 12:25.01 |
| | I wonder, are there more common areas we could identify? | 12:25.22 |
| tor8 | the problem is it looks like these "newer" UTF-8 and UTF-32 cmaps are all autogenerated with no concern for size | 12:25.25 |
| | I suspect the various JIS flavours could be combined without too many adverse effects | 12:25.46 |
| | the Hojo CMap files are not available for download anymore, so I suspect we could drop them with no-one being the wiser | 12:26.47 |
| chrisl | tor8: how many Hojo CMaps were there? | 12:28.02 |
| tor8 | Hojo-[HV], Hojo-EUC-[HV] and UniHojo-(UCS2|UTF16)-[HV] | 12:28.57 |
| | all of the -H ones are pretty big, the -V are just usecmap with a dozen glyph overrides | 12:29.33 |
| chrisl | In gs, we only have UniHojo-UCS2-H | 12:29.44 |
| | IIRC, we dropped a few obsolete cmaps a two or three years ago | 12:30.09 |
| | kens2: this libjpeg problem is going to be a ball-ache :-( | 12:39.46 |
| kens2 | chrisl I suspected as much after my quick inspection | 13:08.48 |
| | Robin_Watts : tor8 it looks like fz_image_get_pixmap is not returning the indexed data, it seems to be expanding it to RGB (possibly RGBa) | 13:10.38 |
| tor8 | kens2: what does the colorspace field of the pixmap point to? | 13:11.41 |
| kens2 | Yes, in fz_decomp_image_from_stream if indexed then it calls fz_decode_indexed_tile and fz_expand_indexed_pixmap | 13:12.08 |
| | tor8 its an Indexzed space with an RGB base | 13:12.16 |
| tor8 | the image->colorspace might be different from the pixmap->colorspace | 13:12.38 |
| | if it's been expanded, the pixmap->colorspace would be rgb | 13:13.00 |
| kens2 | Hmm, OK but how (can) I get the original data, but uncompressed ? | 13:13.05 |
| | just a moment, I'll have to debug to that point | 13:13.14 |
| | Yes the tile colour space is RGB | 13:13.42 |
| Robin_Watts | kens2: Yeah, we can't plot indexed colors in fitz. | 13:14.51 |
| kens2 | I guess I cna work with that instead of the image colour space, I@m not sure what effect it'll have on pdfwrite | 13:14.53 |
| Robin_Watts | so everything goes to rgb. | 13:14.59 |
| kens2 | Robin_Watts : I really don't want everything in RGB, if its Indexed CMYK I need it as CMYK | 13:15.16 |
| Robin_Watts | sorry, everything goes to rgb or cmyk or greyscale. | 13:15.30 |
| kens2 | In the long run I'll need Separation and DeviceN too | 13:15.31 |
| Robin_Watts | bah. Let me rephrase. everything goes to a non-indexed space. | 13:15.53 |
| tor8 | all indexed images will get converted into its base colorspace | 13:15.56 |
| Robin_Watts | what tor8 said :) | 13:16.03 |
| kens2 | non-indexed I cna live with | 13:16.04 |
| tor8 | the conversion to the output colorspace happens later in fitz | 13:16.30 |
| Robin_Watts | I could make it so that you can optionally get the indexed data out. | 13:16.40 |
| kens2 | I don't remember what pdfwrite does with Indexed spaces, I can check that later. | 13:16.42 |
| | Robin_Watts : I don't think I'm that bothered, at least not yet | 13:16.54 |
| | Maybe in the future for efficientcy, but now I understand what's going on I have some hope of getting realistic images out :-) | 13:17.18 |
| chrisl | IIRC, indexed images are unpacked before passing into the graphics, so pdfwrite has heuristics to spot and recreate them | 13:18.11 |
| kens2 | goes off to refactor the code..... One day I'll have to rewrite this properly | 13:18.17 |
| chrisl | s/graphics/graphics lib | 13:18.20 |
| kens2 | chrisl I'm not certain it does convert back to /Indexed at all, but maybe..... | 13:18.39 |
| | I have a sneaky suspicion there's an open enhancement request that it should | 13:19.04 |
| chrisl | Oh, maybe it was distdv that did that..... | 13:19.15 |
| kens2 | distdv certainly did | 13:19.22 |
| | IIRC it could look at the original colour space (ort passed it down via a USER message, or something like that) | 13:19.56 |
| | If memory serves, pdfwrite just doesn't ever see the information though | 13:20.17 |
| chrisl | No, I think that's correct | 13:21.00 |
| kens2 | Anyway, quick hackery in progress :-) | 13:21.28 |
| | Well, its come out all red, but its recognisable..... | 13:28.12 |
| | Oh, and there are blocks missing, that's weird | 13:29.04 |
| tor8 | Robin_Watts: kens2: I can't remember, but do we use premultiplied alpha? | 13:33.51 |
| Robin_Watts | I fear we do. | 13:34.04 |
| kens2 | Urgh# | 13:34.10 |
| tor8 | shouldn't matter at the moment, but when you split the alpha channel out you might have to undivide | 13:34.19 |
| kens2 | Indeed :-( | 13:34.26 |
| | Well I was planning to store and insepct the alpha at the same time anyway, so, that's life | 13:34.44 |
| | At current rate of progress it's going to be quite a while before I need to worry about it | 13:35.15 |
| | I cna't quite figure out how I can have an image os nearly correc,t and yet still be wrong. I guess I need a simpler case. | 13:35.52 |
| | OK dumb question time. I've been using the image decode array from the source image, but obviously that;s not going to work for retrieving a pixmap. Can I assume the decode for the retrieved pixmap is always 0->255 ? Or should that more reasonably be 0->2^bits per component ? | 13:58.30 |
| tor8 | the fz_pixmap data is always 0..255, all decode stuff has happened already | 13:59.04 |
| kens2 | OK so 0..255 even for 12 or 16 bit input, and even for gray colour spaces ? | 13:59.32 |
| tor8 | it's a todo project to add 1bpp and 16bpp spaces | 13:59.42 |
| Robin_Watts | kens2: You shouldn't be using anything from inside the fz_image. Just get the pixmap from it and work with that. | 13:59.45 |
| kens2 | Robin_Watts : I need to know what the pixmap is in | 13:59.58 |
| Robin_Watts | what *colorspace* the pixmap is in? | 14:00.14 |
| | pixmap->colorspace ? | 14:00.18 |
| tor8 | the pixmap is always color+alpha, 0..255 | 14:00.31 |
| kens2 | Yes, but also what the samples are encoded as | 14:00.32 |
| | THanks tor8 | 14:00.41 |
| Robin_Watts | Everything you need to understand the content of the pixmap is in the pixmap. | 14:01.03 |
| | unless I'm missing something... | 14:01.12 |
| kens2 | No you are probably correct, but that's not what I was doing previously, and I still need ot know how the pixmap is encoded, which is what tor just told me | 14:01.45 |
| | Now I know how to present that data to GS | 14:02.07 |
| chrisl | It seems to me that this isn't going to scale well to Separation and DeviceN spaces....... | 14:03.04 |
| kens2 | I don't see any way they can work at present | 14:03.17 |
| Robin_Watts | chrisl: images in separation or devicen spaces decode to pixmaps in separation+alpha or devicen + alpha. | 14:05.22 |
| | where do you see the problem? | 14:05.38 |
| kens2 | OK well I haven't got that far yet, but is hould be easy to add them. Manufacturing a GS colour space will be more interesting | 14:05.49 |
| chrisl | Robin_Watts: I thought pixmaps were always in the device color space | 14:06.13 |
| kens2 | No they are in teh 'base' space | 14:06.23 |
| | So Indexed gets decoded, other ones don't (or sa I understand, I could bw wrong) | 14:06.44 |
| | I'm not sure what happens with ICCBased spaces yet either | 14:07.05 |
| tor8 | kens2: yeah, mainly because indexed images can't be interpolated for scaling | 14:07.17 |
| kens2 | Quite so. | 14:07.26 |
| tor8 | and the same for color keyed transparency, which is why we flatten that into an alpha channel | 14:07.51 |
| kens2 | Well, that comprehensively broke my code ;-) at least seg faults are easy to locate | 14:07.53 |
| | ah num_planes is incorrect | 14:08.14 |
| chrisl | So, at what point are DeviceN images, for example, converted to the "final" color space? | 14:08.21 |
| kens2 | During rendering I think | 14:08.30 |
| | same as other cases where the image space doesn't match the selected device space | 14:08.49 |
| tor8 | chrisl: after decoding, before scaling and rendering | 14:09.03 |
| chrisl | I suppose it is easier with PDF than with Postscript..... | 14:09.16 |
| | So, The only way I can see to make libjpeg work is to take jpeg decoding out of the hands of fitz and do it explicitly in the fz_ghost_fill_image() method :-( | 14:10.09 |
| kens2 | Oh..... | 14:10.27 |
| Robin_Watts | chrisl: What's the problem with libjpeg? | 14:10.36 |
| kens2 | That's going to be painful | 14:10.38 |
| | Robin_Watts : not libjpeg exactly | 14:10.45 |
| | Its thememory usage when both GS and MuPDF are present | 14:11.02 |
| chrisl | Robin_Watts: memory management - ghostscript fudges the memory management to it uses the gs memory manager | 14:11.21 |
| Robin_Watts | chrisl: And this is a problem with mooscript... why? | 14:11.40 |
| | presumably with mooscript you're overriding fz_malloc etc to use the gs memory manager too ? | 14:12.02 |
| chrisl | Robin_Watts: because the only interface we have between mupdf and gs graphics lib is via the fz_device interface | 14:12.25 |
| | Robin_Watts: I haven't hooked the mupdf memory management yet | 14:12.52 |
| | But that wouldn't matter | 14:12.59 |
| Robin_Watts | In order to create an fz_device, you must have an fz_context. | 14:13.14 |
| | and the memory management is setup in the fz_context creation. | 14:13.27 |
| chrisl | Robin_Watts: Yes, but all the libjpeg setup is done in the fitz calls - so the libjpeg "private" data is set to the fitz context, not the gs context the code expects | 14:15.24 |
| Robin_Watts | i'm a tad confused. | 14:18.25 |
| | as ever. | 14:18.28 |
| kens2 | I was confused shortly after following the code, but what chrisl says makes sense to me (now) | 14:18.47 |
| Robin_Watts | fitz sets up libjpeg only just before it calls libjpeg to do decoding. | 14:19.03 |
| kens2 | Its probably clearer if you try debugging the mooscript code and see the flow | 14:19.06 |
| Robin_Watts | hence surely fitz should setup libjpeg for its calls, and gs should setup libjpeg for its calls. | 14:19.34 |
| | or are fitz calls somehow ending up in gs or something? | 14:19.53 |
| kens2 | suspects its the GS hackery | 14:19.56 |
| ray_laptop | morning, all | 14:20.19 |
| kens2 | Morning ray | 14:20.26 |
| Robin_Watts | morning ray | 14:20.37 |
| ray_laptop | kens2: Sorry about that pdfwrite bug I tripped over. I hope it's not too nasty | 14:20.46 |
| kens2 | WOohoo indexed image now working :-) | 14:20.53 |
| | ray_laptop : I'm ignoring it for now :-) | 14:21.02 |
| chrisl | It *is* gs hackery - libjpeg doesn't allow the memory management calls to be "hooked", so when we're building with our (gs) libjpeg source, we leave out the libjpeg memory management source files, and build in our own versions. | 14:21.24 |
| ray_laptop | kens2: when was indexed images not working ? | 14:21.27 |
| chrisl | Robin_Watts: ^^ | 14:21.29 |
| kens2 | ray_laptop : Mooscript; never worked | 14:21.43 |
| ray_laptop | kens2: oh, mooscript. Cool | 14:21.57 |
| kens2 | I believe Gray, RGB, CMYK and Indexed images now work | 14:22.16 |
| | more omcplex cases await further work | 14:22.25 |
| | Part of which is the JPEG stuff chrisl is talking about | 14:22.49 |
| ray_laptop | kens2: yeah, like separation color spaces ? | 14:22.53 |
| kens2 | Yep, and DeviceN, and chroma-key and, and, and..... | 14:23.15 |
| Robin_Watts | So libjpeg under mupdf is not going through the fz malloc stuff? | 14:23.21 |
| chrisl | Unless you've got similar hackery in there | 14:23.39 |
| ray_laptop | how about the JPXDecode nonsense where the data defines the colorspace differently to the PDF Image dict ? | 14:23.43 |
| kens2 | That's up to MuPDF :-) | 14:23.55 |
| | I'm only going by what it hands me for a colour space | 14:24.10 |
| ray_laptop | kens2: I see. So it's already "resolved" correctly by the time you get it | 14:24.20 |
| kens2 | Resolved, decoded, uncompressed..... | 14:24.32 |
| Robin_Watts | chrisl: OK, so how about we write a common bit of hackery. | 14:24.36 |
| | If MuPDF is having libjpeg call malloc/free then we need to fix that. | 14:25.49 |
| chrisl | Robin_Watts: I'm not sure, that might work. | 14:26.01 |
| | Robin_Watts: it is possible you have similar stuff in there already - mooscript is using the libs and their builds from the ghostscript world... | 14:26.53 |
| Robin_Watts | if we could write the bit of libjpeg that you'd really hope was there (where you pass in malloc/free pointers) then we can use the same piece of code for both gs and mupdf. | 14:27.09 |
| | I can't immediately see it :) | 14:27.15 |
| chrisl | I'd possible have more success if I was in the correct directory..... | 14:27.38 |
| Robin_Watts | Yeah, we're calling jmemnobs.c which calls malloc. How shite is that? | 14:27.41 |
| tor8 | chrisl: how do you handle the case with a system libjpeg? | 14:28.01 |
| kens2 | Badly ? :-) | 14:28.09 |
| chrisl | tor8: the memory management calls aren't built when SHARE_JPEG==1 | 14:28.34 |
| kens2 | Possibly it just uses malloc in that case I guess | 14:28.35 |
| chrisl | Yes, so it goes off to the "normal" libjpeg implementation | 14:28.54 |
| Robin_Watts | chrisl: This would seem a nicer fix than having to do major surgery? | 14:29.12 |
| kens2 | The major surgery chrisl was descirbing woudl be 'painful' at best | 14:29.35 |
| | I don't think we want to get into decompressing JPEG data behind the back of MuPDF | 14:30.04 |
| chrisl | Actually, I think what I was talking about would be much easier - but a hacky, short term solution at best | 14:30.14 |
| | Robin_Watts: But I do think a shared implementation would be good. | 14:30.42 |
| | If, of course, tor8 is amenable..... if not, I won't even bother! | 14:31.28 |
| tor8 | chrisl: amenable to what? a shared fix for custom memory allocators in libjpeg? I'm all for that! | 14:33.24 |
| chrisl | tor8: okay, I'll look at that tomorrow, then.... it's just it gets a little hacky accounting for using system libs..... | 14:34.24 |
| tor8 | I'd just stick a macro in our customized libjpeg headers | 14:35.03 |
| | and ifdef based on the existence of that | 14:35.10 |
| | unless we're somehow building with our copy of the headers and linking against the system, but that's worth fixing if so | 14:35.32 |
| chrisl | I've trying to get away from patching the third party libs source..... | 14:36.03 |
| | s/I've/I've been | 14:36.03 |
| tor8 | ah, right. that makes for more build system trickery. | 14:36.59 |
| chrisl | tor8: personally, I'd rather handle in the makefiles.... | 14:37.42 |
| tor8 | but yes, we could simply not build against jmemnobs in mupdf and have a common API for custom allocators in both mupdf and ghostpdl source trees | 14:38.04 |
| | and pick that in the makefile | 14:38.08 |
| chrisl | Yes, that's essentially how the gs stuff current works | 14:38.32 |
| kens2 | chrisl I pushed that commit to my repo, images shold be a lot better now (as long as htey aren't JPEG) | 14:39.58 |
| tor8 | chrisl: so basically base/sjpegc.c with callbacks? | 14:40.32 |
| chrisl | tor8: basically, yes | 14:41.07 |
| | tor8: I was thinking I'd put the common stuff in a separate source file. | 14:42.56 |
| | kens2: got it | 14:43.02 |
| Robin_Watts | IF we get it nice enough we could offer it upstream. | 14:43.07 |
| kens2 | Looks like I misunderstand the purpose of image->tile....... | 14:43.20 |
| tor8 | yeah, there's a lot of gs specific stuff in there. a common file jmemcb.c we could use in both would be perfect, and hopefully pass upstream | 14:43.23 |
| Robin_Watts | Why are you using image->tile ? | 14:43.46 |
| kens2 | Well I assumed if it was already set I didn't need ot use get_pixmap | 14:44.02 |
| | Clearly I do | 14:44.06 |
| kens2 | wonders where I copied this source from originally | 14:44.21 |
| chrisl | I did offer to implement memory management callbacks in libjpeg a while back, but got no response - maybe a "can't see the point" response.... not sure | 14:44.48 |
| Robin_Watts | kens2: fz_images were introduced to mupdf a while ago as part of a (fairly major) rework that allows us to keep images around compressed. | 14:45.24 |
| kens2 | aha, OK JPX images now working also | 14:45.26 |
| | Robin_Watts : so the simple answer from my POV is always just toe get a pixmap | 14:46.06 |
| Robin_Watts | so it's possible that mooscript wasn't properly updated to cope. | 14:46.12 |
| | kens2: Yes. | 14:46.15 |
| | Unless you'd rather get the compressed data. | 14:46.20 |
| tor8 | kens2: you could have an else at the end of the colorspace testing chain that forcibly converts to RGB | 14:46.31 |
| kens2 | Oh no, I'm very happy with the uncompressed data | 14:46.36 |
| Robin_Watts | Then yes, always get a pixmap :) | 14:46.44 |
| tor8 | and then you'll handle (if not ideally) any other odd cases you haven't got special case code for | 14:46.46 |
| kens2 | tor8 yes you're probably correct | 14:46.56 |
| | Robin_Watts : yeah the problem is chris and I are beating on this in 'spare time' and I haven't looked at it since the last staff meeting. One fo the firsat things I had to do this time was get it to build again. | 14:47.48 |
| | tor8 how could I cope with the cases not yet handled ? | 14:48.32 |
| tor8 | kens2: there's an example in mupdf/source/tools/pdfextract.c:writepixmap() | 14:48.34 |
| kens2 | Even better :-) | 14:48.41 |
| kens2 | goes off to look (I still need to commit the fix that stops looking at image->tile too) | 14:49.00 |
| tor8 | create a new pixmap to hold the converted data using the colorspace you want, then call fz_convert_pixmap(dest, source) | 14:49.03 |
| chrisl | I think we should take Robin_Watts's suggestion of adding a mooscript build to regression testing in the near future..... | 14:49.18 |
| kens2 | Interesting, Looks like a good way to get more stuff working immediatley | 14:49.23 |
| | chrisl one to discuss with Marcosw :-) | 14:49.40 |
| chrisl | Well, I figure after the next staff meeting would be soon enough, so we can discuss it in person | 14:50.10 |
| kens2 | tor8 don't look too closely at this code yet, its evelving rapidly | 14:50.19 |
| | or even evolving | 14:50.30 |
| | No elves involved at this point | 14:50.43 |
| chrisl | "changing" does not always imply "evolving"....... | 14:50.56 |
| kens2 | believes its already fitter than it was this morning :-) | 14:51.19 |
| tor8 | kens2: no worries, just curious, and I might be able to point you to things you don't know exist :) | 14:51.24 |
| kens2 | Given I know nothing, that's certainly true | 14:51.39 |
| Robin_Watts | kens2: I am aware of the problems with "spare time" projects :) | 14:52.22 |
| | It's probably not helped by the fact that we rip the floorboards up on mupdf fairly often. | 14:52.47 |
| kens2 | That's going to become more true with time | 14:53.15 |
| | At the moment we aren't using much (its all very primitive yet), but as we add more funcitonality that will likely become a problem | 14:53.44 |
| Robin_Watts | It's happening less and less now. | 14:54.27 |
| | but having a regression test would prevent it being such a problem. | 14:54.44 |
| kens2 | Probably good from chris and my POV :-) | 14:54.45 |
| Robin_Watts | it's probably trivial for us to fix mooscript at the time we move a floorboard - but it gets harder the longer it's left. | 14:55.16 |
| chrisl | Robin_Watts: I'm just waiting for tor8 to disappear for a couple of weeks and reappear with mupdf rewritten in Go....... | 14:55.46 |
| kens2 | Yeah, I am going to try and make a point of doing 'something' between every staff meeting. If nothign else it should be a soothing distraction for a trans-Atlantic flight | 14:56.11 |
| | Hmm anyone got a PDF file with an image which is not JPEG compressed and uses Separation or DeviceN ? :-) | 15:01.23 |
| Robin_Watts | One of the QL turkey files I'm sure. | 15:02.34 |
| kens2 | I cna always make one, its probably quicker than searching the QL test files..... | 15:03.20 |
| | coffee first I think | 15:04.25 |
| ray_laptop | kens2: I think that there are ones in the PDF FTS set. I'll run a scan... | 15:06.09 |
| chrisl | Okay, I'm off to play squash - I may be back later, depending on how I feel afterwards........ | 15:06.20 |
| kens2 | ray_laptop : don't worry I'll build one myself, its not hard. | 15:12.16 |
| | LOL the URL quoted by the person complaining that our web site doesn't disable password autocompletion returns a 'content not found' error. | 15:14.22 |
| ray_laptop | kens2: try tests_private/pdf/PDF_1.7_FTS/fts_04_0403.pdf | 15:14.29 |
| kens2 | thanks ray | 15:14.36 |
| ray_laptop | kens2: it definitely has a complicated colorspace, but it might be Indexed where the base space is Separation | 15:15.15 |
| | I'll look at a few others | 15:15.22 |
| kens2 | ray_laptop : really don't worry, I cna make one form the turkey in a few seconds, and that way I know the PDF file doesn't have any other content. | 15:15.45 |
| | Other content might break mooscript at present | 15:15.53 |
| | OK the FTS one does have a image that now gets converted to RGB, so it works :-) | 15:16.52 |
| | The file is not 100% correctly rendered for the non-image ocntent, but that's a different problem | 15:18.05 |
| ray_laptop | kens2: 1600 is also an Indexed colorspace with a DeviceN base space with Separation components, but the "Alternate" is CalRGB, so mupdf might just be punting and using the Alternate | 15:27.05 |
| kens2 | one's enough ray | 15:27.54 |
| ray_laptop | kens2: 1702 has images with a sampling of colorspaces including DeviceN, Lab along with "normal" device spaces | 15:31.29 |
| | kens2: and 2429 has DeviceN where a transfer function is added for one of the pair | 15:33.24 |
| kens2 | ray one is enought right now, honestly | 15:33.38 |
| ray_laptop | kens2: I'm done. Just figured I'd go ahead and grab a bit more for you to chew through later :-) | 15:34.45 |
| mvrhel_laptop | good morning | 15:57.12 |
| Robin_Watts | Morning mvrhel_laptop. | 15:59.05 |
| | marcosw: Are you here? | 15:59.07 |
| kens2 | wonders if we could pass transfer functions to GS and get them right that way. | 16:19.29 |
| | Night all | 16:50.07 |
| Robin_Watts | marcosw_: ping ? | 17:33.01 |
| marcosw_ | Robin_Watts: morning | 17:33.07 |
| Robin_Watts | morning. | 17:33.11 |
| cinap_lenrek | http://www.intel.com/content/dam/doc/datasheet/82574l-gbe-controller-datasheet.pdf | 17:33.21 |
| Robin_Watts | There was an ETS mail that came to support the other day. | 17:33.23 |
| cinap_lenrek | i get "File has an invalid xref entry: 8353." | 17:33.36 |
| | then it tries to rebuild | 17:33.40 |
| | and then crashes | 17:33.43 |
| | Error: /typecheck in --get-- | 17:33.50 |
| | Operand stack: | 17:33.50 |
| | PageCount --nostringval-- Count | 17:33.50 |
| Robin_Watts | Have you replied to it? | 17:34.01 |
| cinap_lenrek | i looked in the pdf | 17:34.04 |
| Robin_Watts | cinap_lenrek: gs or mupdf ? | 17:34.15 |
| | cinap_lenrek: and what version? | 17:34.23 |
| | oh, that must be gs. | 17:34.39 |
| cinap_lenrek | a pretty old version of ghostscript | 17:34.51 |
| | (on plan9) | 17:34.53 |
| Robin_Watts | cinap_lenrek: Well, I'd advise you to try 9.14, cos if that works, we aren't going to go looking for a fix for an old version. | 17:35.13 |
| cinap_lenrek | the missing reference is for the /Pages object | 17:35.16 |
| | i know | 17:35.33 |
| | but i cant | 17:35.35 |
| | ghostscript is bigger than the whole plan9 kernel | 17:35.47 |
| Robin_Watts | So? | 17:36.48 |
| cinap_lenrek | isnt new version even using c++? | 17:36.51 |
| Robin_Watts | no. | 17:36.57 |
| cinap_lenrek | ok | 17:37.14 |
| marcosw_ | Robin_Watts: I have not. I recall seeing it but now it's not showing up in my in basket. I wonder if I accidnetly marked it as spam or somthing else weird is going on. | 17:37.27 |
| | never mind, it was in the tech folder (I recalled it being sent to support). | 17:37.53 |
| Robin_Watts | It was sent to support. | 17:38.07 |
| | but the word 'tech' does appear in the CC email? :) | 17:38.24 |
| henrys | cinap_lenrek: you will certainly want the gcc toolchain not he host tools | 17:38.58 |
| cinap_lenrek | see? | 17:39.23 |
| | i dont want that | 17:39.25 |
| | anyway | 17:39.30 |
| | i want to understand whats wrong with that pdf | 17:39.49 |
| | or how its supposed to work in general | 17:39.58 |
| | /Length 7763 /Root 8353 0 R | 17:40.06 |
| | thats the first missing reference | 17:40.12 |
| | but that seems to recover | 17:40.24 |
| | i greped the pdf for 8353 | 17:40.31 |
| Robin_Watts | It's probably a PDF with compressed object streams. | 17:40.34 |
| cinap_lenrek | and theres a object definition for it | 17:40.37 |
| | oh | 17:40.41 |
| | is there some kind of pdf structure dumper? | 17:41.03 |
| | it resolves the /Root object reference, then that has /Pages 8294 0 R | 17:41.50 |
| | and that seems unresolvable | 17:41.57 |
| marcosw_ | Robin_Watts: I'm not familiar enought with ETS to answer his quesitons so will have to forward it to someone else. Are you the current ETS engineer? | 17:42.10 |
| Robin_Watts | marcosw_: I have my finger on my nose :) | 17:42.51 |
| marcosw_ | Robin_Watts: that just shows you are guilty :-( | 17:43.11 |
| Robin_Watts | It's either me or Michael, I think. | 17:43.20 |
| | cinap_lenrek: PDF 1.6 introduced a new type of object stream, the compressed object stream. | 17:43.55 |
| cinap_lenrek | Robin_Watts: ah | 17:44.09 |
| | can i quickly check if thats the case? | 17:44.20 |
| Robin_Watts | The idea is that rather than objects being explicitly listed one after another in the file, they could be compressed into streams. | 17:44.26 |
| | I suspect your very old version of gs does not understand this. Or if it does that there is another bug. | 17:44.42 |
| cinap_lenrek | i see <</Filter/FlateDecode ...>>stream in the pdf | 17:44.50 |
| | its 8.x something | 17:45.08 |
| Robin_Watts | You'll see that in most PDF files. | 17:45.21 |
| marcosw_ | since mvrhel_laptop isn't paying attention I'll send it to him. | 17:45.23 |
| mvrhel_laptop | hey | 17:45.29 |
| cinap_lenrek | ah | 17:45.43 |
| | %PDF-1.6 | 17:45.46 |
| mvrhel_laptop | I was simply hoping it would go away | 17:45.50 |
| Robin_Watts | I suspect that michael and I can work together to come up with a reply. | 17:45.58 |
| mvrhel_laptop | I can live with that | 17:46.06 |
| henrys | marcosw: I just briefly read the email and it look like RTFM | 17:46.19 |
| Robin_Watts | I think henrys should handle all customer technical questions from now on. :) | 17:47.06 |
| mvrhel_laptop | the ETS email says that ray told them we had SSE2 for ETS? | 17:47.33 |
| henrys | Robin_Watts: that would thin the customer ranks | 17:47.42 |
| Robin_Watts | There *was* an SSE version of ETS. | 17:47.52 |
| | I don't think it's current. | 17:47.57 |
| mvrhel_laptop | oh that is right | 17:48.02 |
| Robin_Watts | My instinct would be to tell them that we have taken Ray outside and beaten him up. There is no SSE implementation currently. | 17:48.28 |
| mvrhel_laptop | hehe | 17:48.41 |
| Robin_Watts | unless you want to look at reviving it? but I suspect that's not a quick job. | 17:48.55 |
| mvrhel_laptop | no that is not going to be a trivial task | 17:49.08 |
| henrys | and gs/toolbin/halftone/ETS/_eb_sse2 is a figment of his imagination | 17:49.16 |
| Robin_Watts | henrys: And that is why reading the docs is a bad idea. | 17:49.40 |
| mvrhel_laptop | these are not the droids you are looking for.... | 17:49.49 |
| cinap_lenrek | AH | 17:50.02 |
| | i compared with a working pdf | 17:50.09 |
| marcosw_ | henrys: so the answer is RTFM but not pay attention to it? | 17:50.16 |
| cinap_lenrek | this one has /Type/ObjStm | 17:50.26 |
| | Robin_Watts: is that what you where refering to? | 17:50.41 |
| Robin_Watts | could be. | 17:50.46 |
| cinap_lenrek | i'll look in the spec | 17:50.58 |
| Robin_Watts | henrys: That is NOT the current version of ETS. | 17:51.25 |
| mvrhel_laptop | well for one thing the sse2 stuff is all written in assembly instead of intrinsics | 17:51.27 |
| | yes. who checked this in? | 17:51.34 |
| Robin_Watts | Raph, presumably. | 17:51.47 |
| henrys | Robin_Watts, mvrhel_laptop: well that's really a bug then right, the README says "Compiling SSE2" | 17:51.52 |
| Robin_Watts | hehe, git blame on README.txt points the finger at someone other than me :) | 17:52.30 |
| mvrhel_laptop | oh crap | 17:52.54 |
| | ok. let me clean up this mess | 17:53.16 |
| | Robin_Watts: let me draft some sort of reply | 17:53.35 |
| | and then run it through you first | 17:53.43 |
| Robin_Watts | henrys: I was given ETS a while ago. I hacked on it for a bit to clean it up, and then it got passed to michael. | 17:53.56 |
| | At no point did either of us touch the SSE2 stuff. | 17:54.03 |
| | And given the state it was in when we got it, I wouldn't trust the SSE2 stuff to work at all. | 17:54.23 |
| henrys | well can we fix the README to reflect that? | 17:55.15 |
| mvrhel_laptop | if we did any SSE2 stuff, it really should be rewritten with intrinsics. henrys: yes. I will get this cleaned up | 17:55.27 |
| Robin_Watts | mvrhel_laptop: Is the version in gs up to date with the version in ets.git then? | 17:55.53 |
| mvrhel_laptop | I am going to check that carefully Robin_Watts | 17:56.06 |
| Robin_Watts | cool. Let me know if I can help. Any distraction from SO gratefully received. | 17:56.53 |
| mvrhel_laptop | be careful what you wish for ;) | 17:57.06 |
| | going to grab some lunch, then I will review all of this | 17:57.44 |
| | want to get an installer for gsview made and have people play around with it a bit. hoping to get that done this week | 17:58.19 |
| | but I prob. should do a little testing on 32 bit and windows 7 here first | 17:58.42 |
| Robin_Watts | Anyone who was using MetalScroll (or RockScroll) on VS2005/8, apparently it doesn't work on 2010. | 18:13.43 |
| | But MS have released this, which does the same thing: http://visualstudiogallery.msdn.microsoft.com/d0d33361-18e2-46c0-8ff2-4adea1e34fef?SRC=Home | 18:13.58 |
| henrys | mvrhel_laptop: btw sse2 optimization could be done by a consultant. It is fairly well contained, no gs knowledge really... | 18:14.44 |
| mvrhel_laptop | true | 18:14.53 |
| | Robin_Watts: I have it on 2010 and 2013 | 18:15.09 |
| | I think you can find on the VS website | 18:15.23 |
| henrys | mvrhel_laptop: I'll look around for someone. | 18:15.28 |
| Robin_Watts | metalscroll? Or the Power thing? | 18:15.34 |
| | ooh, that's so nice! | 18:16.02 |
| henrys | or if anyone on the staff wants to make a recommendation ... | 18:16.02 |
| mvrhel_laptop | the http://visualstudiogallery.msdn.microsoft.com/1b0d7360-40dd-447e-8bef-90e2cf52f683 | 18:16.18 |
| | Robin_Watts: ^^ | 18:17.08 |
| Robin_Watts | Is that different to the one I linked to? | 18:17.12 |
| henrys | marcosw_: did you see my IRC comment about banding vs. non banding and ray's recent fix? | 18:17.19 |
| Robin_Watts | Productivity Power Tools looks nicer than MetalScroll. | 18:17.28 |
| mvrhel_laptop | yes it does actually | 18:18.27 |
| marcosw_ | henrys: with C411.bin? i'll take a look. | 18:18.44 |
| henrys | marcosw_: ray has fixed that one but I thought we were doing banded vs non and that one was noticeable, it wasn't pixel level | 18:19.44 |
| marcosw_ | henrys: only new banded vs. non differences are reported. | 18:21.46 |
| henrys | marcosw_: oh of course | 18:25.04 |
| marcosw_ | I could run a banded vs. non comparison but I thought I did that some time ago. Though if I did I clearly missed C411.bin. | 18:25.52 |
| henrys | marcosw_: sounds like a torture that should be infrequent as possible | 18:26.41 |
| | marcosw_: are a lot of the problems filtered with fuzzy? | 18:27.06 |
| marcosw_ | henrys: yes. | 18:27.43 |
| henrys | marcosw_: well your call. | 18:30.26 |
| | bbiab | 18:30.48 |
| luciddavid | hello | 18:50.55 |
| ghostbot | what's up, luciddavid | 18:50.55 |
| luciddavid | how can ghostscript be configured to only output spot colors and no CMYK? | 18:51.41 |
| Robin_Watts | Using what output device ? | 18:52.30 |
| luciddavid | TIFF for instance | 18:52.49 |
| Robin_Watts | I believe you can do that using tiffsep. | 18:52.52 |
| luciddavid | any specific details? | 18:53.26 |
| Robin_Watts | http://ghostscript.com/doc/current/Devices.htm#TIFF | 18:53.42 |
| | I'm thinking that SeparationColorNames is what you want. | 18:54.53 |
| | You can specify which separations you want to come out. | 18:55.08 |
| luciddavid | So - given a PS file for example with some number of spot colors - 12 for instance - where each color is named as it is used... | 18:56.31 |
| Robin_Watts | gs -sDEVICE=tiffsep -o out.tif -c "<< /SeparationColorNames [ /Spot1 /Spot2 ] >> setpagedevice" -f input.ps | 18:56.38 |
| luciddavid | And I want to output a composite date - with NO CMYK channels | 18:56.53 |
| Robin_Watts | Oh, you want a *composite* with no CMYK? | 18:57.14 |
| luciddavid | yes | 18:57.40 |
| Robin_Watts | Not sure that's possible with a standard device. | 18:57.52 |
| | mvrhel_laptop, chrisl, or ray_laptop might know more. | 18:58.29 |
| luciddavid | ok thanks - let me play with the tiffsep idea | 18:58.35 |
| chrisl | No, composite *only* contains standard colorants | 18:59.48 |
| mvrhel_laptop | Robin_Watts: I think there is a way to do this but it will require an icc profile and the format is going to be PSDCMYK | 19:00.03 |
| | but there will not be CMYK | 19:00.16 |
| | i.e. photoshop output is the only way to do this | 19:00.34 |
| | luciddavid so the psdcmyk device | 19:00.53 |
| chrisl | It's hard to see how you'd make sense of the output | 19:01.00 |
| mvrhel_laptop | chrisl: photoshop has the mapping of each colorant to equivalent CMYK values | 19:01.18 |
| chrisl | mvrhel_laptop: how does it know what the mapping is? | 19:01.36 |
| mvrhel_laptop | that is in the psd image format | 19:02.24 |
| | chrisl; i.e. it has the equivalent CMYK colors for the spot colors | 19:02.58 |
| chrisl | So we extract the tint transform from the source, and embed it in the psd image? | 19:03.18 |
| mvrhel_laptop | chrisl: we determine the equivalent CMYK value for max colorant. tonal levels may be off. of course if you have a DeviceN ICC profile then you will be all set | 19:04.13 |
| luciddavid | We have an implementation in our TaskForce rip that generates our RLE format, we currently have color options - Grayscale, CMYK, CMYK+Spot - we now need to add a new option SPOT Only | 19:04.53 |
| mvrhel_laptop | but, I don't know if photoshop is set up to handle that case (i.e. a Device N icc profile) | 19:04.57 |
| | luciddavid: is that a single spot or multiples? | 19:05.32 |
| luciddavid | Leonid retired a while back and my other developer is not as current in the GS word | 19:05.46 |
| | Multiple spots (up to 16 of them) | 19:06.09 |
| mvrhel_laptop | luciddavid: let me see what the color manual says. hold on | 19:06.22 |
| | i forget these things | 19:06.32 |
| luciddavid | So I'm trying to wrap my mind around what might be involved in this | 19:06.46 |
| | Thanks for your ideas and help! | 19:07.19 |
| chrisl | I can't help but feel it would be easier to output to separate plates, and then combine them into your custom format - but I may be wrong..... | 19:07.30 |
| mvrhel_laptop | ok. I would look at page 6 in GS9_Color_Management.pdf | 19:08.13 |
| | luciddavid | 19:08.17 |
| luciddavid | This is for our trapping software - so we need bands with all channels | 19:08.19 |
| mvrhel_laptop | luciddavid. In order to get spot only, you are going to have to create some sort of an ICC profile for your output though. This may not really work for you | 19:09.57 |
| luciddavid | certainly not an "easy switch" 8) | 19:10.35 |
| mvrhel_laptop | I would look over the psdcmyk devices and tiffsep devices carefully though and you should be able to figure out how to get only spots out. | 19:10.56 |
| | the first 4 will always by CMYK unless you are doing some funny ICC stuff | 19:11.11 |
| luciddavid | ok thanks | 19:11.30 |
| | Just one more question | 19:15.21 |
| | How hard would it be to make an ICC profile that did nothing to spot colors but eliminated cmyk? | 19:15.50 |
| chrisl | luciddavid: really mvrhel_laptop is the only one to answer that, and he's just headed off to lunch...... | 19:20.13 |
| luciddavid | no problem - thanks | 19:20.39 |
| | I'll do more reading | 19:21.03 |
| chrisl | And I'm off for the evening, now...... | 19:22.14 |
| cinap_lenrek | hm | 19:47.04 |
| | how is one supposed to know what the objectstream is for a reference? | 19:47.19 |
| mvrhel_laptop | Robin_Watts: so it looks the the ets c code is all correct. henrys: I will updated the documents. and should I go ahead and remove the SSE2 code and files? | 19:47.30 |
| | hmm there is also altivec stuff in there | 19:49.29 |
| | which I doubt works any longer too | 19:49.48 |
| | the code is going to be in the git history if anyone needs to get to it | 19:51.08 |
| Robin_Watts | mvrhel_laptop: I reckon removing it is the smart thing to do. | 19:56.56 |
| mvrhel_laptop | yes. I don't know why I did not clean this up to begin with... | 19:57.20 |
| Robin_Watts | mvrhel_laptop: You were probably doing about 8 urgent jobs at once :) | 19:58.19 |
| mvrhel_laptop | perhaps. I honestly don't remember being the one who checked this in.. | 19:58.58 |
| | I do think I was the last one playing around with it when I added the PSD format | 19:59.27 |
| | so that max could do some testing | 19:59.39 |
| henrys | mvrhel_laptop: yes if it isn't tested and in working order let's get rid of it. | 20:12.31 |
| mvrhel_laptop | ok | 20:12.40 |
| henrys | mvrhel_laptop: did you check if it is in sync with Robin_Watts git branch? | 20:12.50 |
| mvrhel_laptop | henrys: yes it is correct | 20:13.05 |
| | OK. I fixed the README to reflect where we are | 20:13.46 |
| | removed all the vec and sse2 files | 20:13.55 |
| | made sure it built and ran | 20:14.05 |
| henrys | mvrhel_laptop: I'd say document his other stuff and send it back to marcosw_ , he's not a lot of fun to talk to if memory serves. | 20:19.55 |
| | and the sse stuff is an enhancement which we can't really quantify exactly, maybe 50% ? not sure. | 20:21.12 |
| mvrhel_laptop | who knows. | 20:22.03 |
| | Robin_Watts: just sent you a patch to look over. Fairly simple since it is just a bunch of file deletions and some changes in the ReadMe.txt | 20:23.20 |
| | henrys: I will finish the email to marcos later this afternoon. | 20:35.33 |
| | need to head out for bit right now | 20:35.39 |
| ray_laptop | seems like a deal. Dell core i7 2.66 GHz laptops -- 4Gb RAM, only 160Gb HD, DVD R/W, 14.1" WXGA, no OS $399 -- Add Windows 7 Pro for $15 -- rerfurbed, of course, but does have a 30-day warranty and longer warranty for extra $ | 20:52.28 |
| | this would have been near $1000 when new a couple of years ago | 20:54.54 |
| mvrhel_laptop | henrys: ok I went ahead and pushed the ETS changes and sent a detailed email to marcosw | 22:59.53 |
| | now I think I have another support question to answer.... | 23:00.31 |
| Robin_Watts | mvrhel_laptop: Not sure we should be removing the patent messages. | 23:20.58 |
| | And I'd have left ipviewer.html in there. | 23:21.36 |
| mvrhel_laptop | Robin_Watts: ok. I will add that all back | 23:21.50 |
| Robin_Watts | I guess we remove the COPYING.txt thing because it's covered by the COPYING.txt in the gs distro? | 23:22.25 |
| mvrhel_laptop | that would be my thought | 23:22.35 |
| | it is confusing otherwise | 23:22.40 |
| Robin_Watts | OK, otherwise it looks good. | 23:22.41 |
| | yeah. | 23:22.43 |
| mvrhel_laptop | Robin_Watts: quick question | 23:27.37 |
| | so I already pushed the patch. sorry about that. but I have a fixed version here. | 23:28.08 |
| | If I push this one, will it just write over the last one, or do I do a forced push? | 23:28.40 |
| | mine is on the top of the commits still at git.ghostscript.com | 23:29.03 |
| | hoping that Robin_Watts is still here...... | 23:29.46 |
| Robin_Watts | sorry, yes. | 23:29.53 |
| mvrhel_laptop | I don't want to screw up the repository... | 23:30.10 |
| Robin_Watts | urm... just push one adding the missing bits back in. | 23:30.19 |
| | I'll do it. It'll just take a mo. | 23:30.31 |
| mvrhel_laptop | oh ok. I was hoping to have it just be one commit | 23:30.53 |
| | but that is fine | 23:30.57 |
| | Robin_Watts: if you had anymore to add to my email to marcosw please chime in | 23:31.18 |
| Robin_Watts | nah, that would require a force push, and I don't think we can justify that here (it's not like the cluster will be upset) | 23:31.34 |
| mvrhel_laptop | ok | 23:31.46 |
| Robin_Watts | Your email looks good to me. | 23:33.19 |
| | Do I just mention that patents exist that cover this? | 23:36.02 |
| | Or do I tell people to contact Artifex for more details? | 23:36.16 |
| | actually... one thing in the email... | 23:38.20 |
| | If they identify the exact set of options they want to use, then the code can be simplified. | 23:38.43 |
| | That will remove various switches within the inner loop, and that will give a speedup. | 23:38.59 |
| | So the code as is can possibly be sped up without going to assembly immediately. | 23:41.15 |
| | Did you ever form an opinion about the issue raised at line 706? | 23:41.48 |
| mvrhel_laptop | Robin_Watts: good point. about code clean up once they settle on what they want | 23:42.32 |
| | Robin_Watts: no I have not looked any more at the grayshade g value stuff. But the fact that you have an argument and it matches with the code (not the paper) makes me not worried about it | 23:44.44 |
| Robin_Watts | No, I think the paper and the code agree. | 23:45.06 |
| | and I disagree :) | 23:45.09 |
| mvrhel_laptop | oh I see | 23:45.13 |
| | your last line seems to say that the code agrees with your logic | 23:45.36 |
| | which I would take to me what I said above | 23:45.47 |
| Robin_Watts | Let me reread. | 23:45.49 |
| mvrhel_laptop | s/me/mean/ | 23:46.01 |
| Robin_Watts | oh, I see. So the code agrees with what I was saying. | 23:47.16 |
| | sorry! | 23:47.18 |
| mvrhel_laptop | ok good | 23:47.24 |
| | papers have a way of getting out with errors, and unfortunately cant be fixed later... | 23:48.03 |
| | Forward 1 day (to 2014/05/09)>>> | |
IRC Logs