| | <<<Back 1 day (to 2017/04/11) | 20170412 |
| norbertj | Hi guys, I was wondering: in gsdevice.c is there a reason why the parameters for gx_device_set_resolution() and gx_device_set_media_size() are doubles? Shouldn't resolution and mediasize be floats? | 09:46.11 |
| chrisl | norbertj: Not sure, that long predates my involvement. Possibly because we try to use doubles when there are floating point calculations likely to happen. | 09:49.54 |
| | norbertj: it does seem odd that they then get stored in floats being passed to the device, though. | 09:52.18 |
| | norbertj: FWIW, the only person who might remember some of the history is probably Ray - if I remember, I'll point him at your question when he comes online | 09:55.44 |
| norbertj | chrisl: thanks | 09:57.57 |
| chrisl | norbertj: the other thing that springs to mind is that there are various devices that set their resolution explicitly "gx_device_set_resolution((gx_device *)pdev, 300.0, 300.0)". Most compilers promote those "300.0" to doubles rather than floats. | 10:04.24 |
| ray_laptop | I guesss norbertj has gone. w.r.t. doubles and floats, I sure don't recall why resolution would be double some places and float others. Surely float is adequate. | 13:20.41 |
| chrisl | ray_laptop: he knows about the logs, so will probably look there | 13:22.09 |
| frostym | Hey is there a flag I should be using to handle 9-slice scaling? I have some EPS files that I'm attempting to rasterize but the output image is pixelated. I assume it's related to 9-slice scaling because when I open the image in Adobe Illustrator (or Photoshop) it looks fine but gives a notice about 9-slice scaling support. | 13:25.38 |
| | this is my current command: ./gs -dSAFER -dQUIET -dBATCH -dSTRICT -dNOPAUSE -dSHORTERRORS -sDEVICE=jpeg -dJPEGQ=100 -r300 -dMaxBitmap=500000000 -dUseCIEColor -sOutputFile=output.jpg -dGraphicsAlphaBits=4 -dEPSFitPage -g3000x3000 -f ~/dev/ghostscript/test.eps | 13:26.53 |
| Robin_Watts | frostym: I'm not sure what 9slice scaling has to do with ghostscript. | 13:27.08 |
| | You'd need to make the file available for us to look at in order for us to comment more. | 13:27.56 |
| frostym | Sure thing I can upload it for you, do you have a preferred site? | 13:28.28 |
| chrisl | If it's small you can use pastebin, otherwise dropbox or similar | 13:29.36 |
| frostym | It may be entirely unrelated to 9slice scaling and just an issue with ghostscript, I just happened to receive that notice for every problematic EPS file that I try to rasterize with ghostscript | 13:29.37 |
| | https://www.dropbox.com/s/1o3hvh2l9othg0c/test.eps?dl=0 | 13:31.28 |
| chrisl | It looks like the eps contains a rather low res image | 13:33.30 |
| | Despite the size of the file | 13:33.42 |
| frostym | I've compiled the program in debug mode and tried running it with various flags to see if something stood out but didn't notice anything. I'm not too familiar with how the project works though so I may have missed something, if there's any sort of log I can generate for you that will help just let me know | 13:33.46 |
| | Shouldn't it be scalable? I can rasterize it in any size with illustrator. I have many other EPS images I could provide as examples too | 13:34.41 |
| chrisl | Hmm, strange..... | 13:35.27 |
| frostym | https://www.dropbox.com/s/tjnk0y2czzm27i7/32925804%20%282%29.eps?dl=0 | 13:35.35 |
| | That one only parts of the image are pixelated, specifically the bug on the right | 13:35.46 |
| chrisl | Probably those parts are sampled images rather than vectors | 13:36.57 |
| | sampled images don't scale well | 13:37.10 |
| frostym | All of these images scale perfectly in illustrator which is why I assumed it was related to 9slice (they all give the notice) and I'm just missing a flag in ghostscript to scale those parts properly | 13:38.19 |
| chrisl | I bet they don't. I bet they are being interpolated | 13:38.47 |
| Robin_Watts | Do you know what 9-slicing is? | 13:39.15 |
| | I don't believe PS (or eps or pdf) have that concept. | 13:39.46 |
| chrisl | No, they don't | 13:39.54 |
| frostym | I didn't before yesterday so I could be misunderstanding, it defines how the symbol should scale by splitting it into 9 partitions | 13:40.22 |
| Robin_Watts | so such a message from photoshop must be bogus. | 13:40.26 |
| frostym | Sorry I should clarify, photoshop opens the image fine at any size but gives no notice. Only illustrator gives the notice | 13:40.44 |
| chrisl | frostym: try adding -dINTERPOLATE to your gs command line | 13:41.22 |
| frostym | Contributors upload EPS files to our site only and Illustrator is somehow able to identify they use 9-slice scaling because it doesn't give the notice for non-problematic files | 13:41.58 |
| Robin_Watts | frostym: Right. 9-slicing is used in UIs and webdesign etc as a way to have resources that can 'resize' to any size and not look pixelated. It works by repeating sections of the source image. | 13:41.58 |
| | None of them use 9-slicing. | 13:42.11 |
| | Not one. | 13:42.15 |
| | Because PS has no concept of 9-slice scaling. | 13:42.56 |
| chrisl | But Flash does, and I think modern Illustrator versions can author flash stuff | 13:43.35 |
| | So the warning may be simply that, as it's an EPS, I can't do 9-slice scaling | 13:44.00 |
| Robin_Watts | yes. | 13:44.39 |
| frostym | -dINTERPOLATE didn't solve the issue | 13:45.02 |
| | This is the notice Illustrator gives: | 13:45.31 |
| | "Illustrator now honors 9-slice scaling of symbols. Any scaling done on symbol instances with 9-slice scaling option in legacy version will look different now. Do you want to retain the 9-slice scaling option for symbols? | 13:46.05 |
| | If ghostscript isn't able to handle it, do you know of a way to identify in the EPS file that it's been used on 1 or more symbols? Illustrator must be able to recognize it somehow because it only gives the notice for problematic gs files. I'm not too familiar with how EPS data is stored in the file | 13:47.37 |
| chrisl | No idea. Postscript doesn't have "symbols". Maybe ask Adobe? | 13:48.12 |
| | FWIW, taking the central owl in that first file: I don't see any way that can be made sharp | 13:51.09 |
| frostym | I just opened it in illustrator and chose "Save for web", selecting 3000x3000px JPEG. This is the result: | 13:52.11 |
| | https://www.dropbox.com/s/w8pt4k1zh59gd04/test-5-%5BConverted%5D.jpg?dl=0 | 13:52.14 |
| | I did notice that when I open it back up in Illustrator the symbols aren't defined but it still is somehow aware that 9slice was used | 13:53.01 |
| chrisl | I have a nasty feeling I know what's going on here...... | 13:53.41 |
| | frostym: I just converted the eps to PDF using Adobe Distiller, and here's the result: https://ghostscript.com/~chrisl/test.pdf | 13:54.17 |
| | frostym: What's happening, I'm fairly sure, is that what you have is an EPS file with the original source Illustrator application file embedded inside it. When you open it in Illustrator, scans the file looking for that embedded application data, and if it there, it opens that, rather than interpreting the Postscript | 13:56.54 |
| Robin_Watts | That would explain the cryptic message, and the quality difference. | 13:57.49 |
| frostym | Shoot, looks like I'll have to find a way to identify that embedded data and fallback to Illustrator when it exists then. Was really hoping to avoid that and use ghostscript for every file. | 14:00.30 |
| chrisl | There is something odd, though | 14:00.59 |
| frostym | Thank you so much for the help guys, probably never would've reached that conclusion without it | 14:01.00 |
| | ?? | 14:01.07 |
| chrisl | The file has a *load* of vector drawing operations, then a large chunk of image data. | 14:01.56 |
| frostym | Those operations exist outside of the Postscript? | 14:04.43 |
| chrisl | Ah, yes. The file has transparent marking operations | 14:05.08 |
| | My guess is that whoever created the EPS has the resolution for flattening transparency set to something like 72dpi (which I suspect is the default) | 14:06.06 |
| | I should say, the Illustrator file has transparent marking operations | 14:06.32 |
| frostym | I'm unfamiliar with that whole concept so I just looked up a brief description. You're saying that the vector drawing options and image data are both part of the embedded Illustrator data and unrelated to what's defined in Postscript, correct? | 14:10.12 |
| | Or am I misunderstanding what transparent marking operations are and it's actually something being handled in the Postscript? | 14:10.39 |
| chrisl | The embedded Illustrator file and the Postscript marking operations are totally separate. | 14:11.27 |
| | Postscript can contain both vector drawing operations, and sampled images. | 14:11.54 |
| | Okay so far? | 14:12.05 |
| frostym | Yep | 14:12.10 |
| chrisl | Right, then we have PDF, which originally used the same imaging model as Postscript, but in a different file format | 14:12.46 |
| frostym | Understood | 14:14.29 |
| chrisl | In that original imaging model, drawing operations were purely opaque. That is, if you draw one object "on top" of another, the upper object would obscure any parts of the lower object that it intersected | 14:14.38 |
| | Then the PDF imaging model had transparency added: where (optionally) drawing one object on top of another would allow some or all of the lower object to "shine through" the upper object, where they intersected | 14:15.54 |
| | Illustrator historically used the same imaging model as Poscript, but was updated to include transparency when that feature was added to PDF | 14:17.23 |
| | However, Postscript was never updated to include transparency | 14:19.07 |
| frostym | Ah, okay | 14:22.08 |
| chrisl | Sorry, distracted by another channel...... | 14:22.38 |
| frostym | No worries | 14:23.27 |
| chrisl | When you have two mismatching imaging models, the only realistic way to get something close to representative output is to render it to one or more sampled images | 14:24.01 |
| | So, when you have an illustrator file that makes use of those transparent marking capabilities, then export to a Postscript (or EPS) file, those sections end up being rendered. | 14:26.40 |
| | Looking at the way the EPS is made, it *looks* like the Illustrator does it is to emit opaque operations until it reaches something that is transparent, then render the entire page | 14:28.13 |
| | Presumably the idea being that if it never encounters a transparent operation, it never needs to render, and you get a nice, small, scalable EPS | 14:28.55 |
| | It will do the same thing if you print from Illustrator to a Postscript printer, but it will probably render at (or close to) the physical resolution of the printer, so it would look okay | 14:30.35 |
| frostym | Ah okay, that makes sense. If I'm understanding correctly, this means that the owl image encountered a transparent operation early so the majority of the image was rendered but the bug image didn't encounter one until much later which is why most of the image is still able to scale. | 14:32.33 |
| | Right? | 14:32.46 |
| chrisl | Yes, basically. | 14:33.00 |
| | The PDF transparency model is *extremely* complex, so the exact details of how the flattening to image happens could vary a *lot* depending on how the file is constructed | 14:33.49 |
| | *Please* don't ask me to explain the PDF transparency model..... | 14:35.07 |
| frostym | Haha I won't, thanks for the in-depth explanation as to what's going on. I don't necessarily need to understand how the transparency model works, but is there a way to easily identify these operations exist in an EPS file programmatically? We process upwards of 300k EPS files a month so if we could identify which files can be handled using ghostscript and which need to handed off to a Windows server | 14:37.49 |
| | running Illustrator it'd be really useful. | 14:37.55 |
| kens | Well the problem is that by the time you see the EPS it doesn't have transparency any more. | 14:38.30 |
| | And you can't readily tell what's in an EPS file without interpreting it | 14:38.49 |
| chrisl | It's fairly easy to check whether the Illustrator file is embedded, but I'm not sure that's sufficient | 14:39.05 |
| kens | And you can't tell the differentce between a file with cvecotr data and an image (eg a phot) and one where there is some vvector data an d an image which is the rendered transparent portion | 14:39.25 |
| | chrisl if you want to pass all Illustrator EPS off, eys, otherwise, no. | 14:39.44 |
| frostym | How were you able to identify the transparent marking operations in the test file I sent? Or did you open it up in a program? I just assumed you were looking through the file with a text editor. | 14:40.17 |
| kens | Cleverness :-) | 14:40.28 |
| chrisl | frostym: I did say I was guessing a bit...... | 14:40.56 |
| frostym | Fair enough | 14:41.00 |
| kens | Seriously, you can ernde the file and look at the output, then deduce what must have happened to get there | 14:41.00 |
| | render* the file.... | 14:41.11 |
| frostym | The problem is we have millions of EPS files at this point that need to be checked and re-rendered with Illustrator. I'm assuming a very large percentage of those users use Illustrator, do you know if Illustrator always embeds the project file inside the EPS? I can investigate further on my own if you don't, just figured I'd ask. The more accurately we can identify problem files, the better. | 14:43.33 |
| chrisl | IIRC, it is an option when you save to EPS (or PDF). | 14:44.10 |
| | So, not *all* Illustrator created files will have the application file embedded | 14:44.36 |
| | frostym: FWIW, I would start by grepping the file for "userdict /ai9_skip_data get exec" or "%AI9_PrivateDataBegin" | 14:45.40 |
| frostym | Cool, thank you for the help! | 14:46.51 |
| chrisl | NP. Sorry I couldn't give you a more favourable answer | 14:47.07 |
| frostym | No worries. We've been having this problem for ages and kept putting it off because we couldn't figure out the root cause or a workaround. At least we can move forward now | 14:49.37 |
| chrisl | It is fairly annoying that illustrator doesn't make it clearer what it's doing. But, tbh, Adobe aren't great at keeping users informed | 14:50.56 |
| frostym | Yeah, I'm not too surprised the issue ended up being with Illustrator and not how ghostscript is handling the file. Would've been nice to be able to resolve the issue instead of working around it but oh well | 14:55.11 |
| Robin_Watts | chrisl: Stupid question... how hard would it be to convert ai files into gs files? (Do we understand the format enough to get the marking operations out? And if we do, do we understand the transparency enough to be able to turn it into gs tranparency ops?) | 14:58.33 |
| kens | Pretty sure it isn't a documented standard, so we'd be guessing | 15:00.32 |
| chrisl | Robin_Watts: Old style illustrator files were based on PDF syntax. I'm fairly sure the new one is totally different. | 15:00.54 |
| kens | It used to be that Illustrator documents were PostScirp and included the ProcSet needed in the .ai file. I doubt if that's still true | 15:00.59 |
| chrisl | I was about to say.... | 15:01.13 |
| Robin_Watts | kens: of course. It just struck me that from what chrisl said it sounded like he could understand most of what was in the file. | 15:01.19 |
| kens | Well, the file is an EPS.... | 15:01.29 |
| Robin_Watts | How did they indicate 'transparent' in this case? | 15:01.31 |
| kens | And we know that the private section is the Illustrator file | 15:01.40 |
| | Robin_Watts : they didn't, they rendered it to an image | 15:01.53 |
| chrisl | Robin_Watts: No, I was working backwards from what was in EPS - as I said, educated guesses | 15:02.19 |
| Robin_Watts | chrisl said: "Ah, yes. The file has transparent marking operations". | 15:02.33 |
| kens | He did say he was guessing :-) | 15:02.50 |
| Robin_Watts | OK, so you meant "Ah, yes, the file has a big block of AI specific data"? | 15:02.59 |
| chrisl | Robin_Watts: No, I pulled the image out, and what was left meant I could see what made up the image, and what was still line art. The stuff in the image is pretty standard Illustrator transparency effects | 15:04.06 |
| | Like drop shadows, etc.... | 15:04.28 |
| Robin_Watts | Then I still don't understand, but never mind. | 15:04.43 |
| kens | bah freenode kicked me off :-( | 15:04.50 |
| chrisl | Robin_Watts: It's possible the illustration could have been done without transparency, and I could have been wrong. But the "normal" way to get those kinds of effects in recent illustrator versions is using transparency | 15:06.34 |
| | Robin_Watts: If you look at the owl's eyes: https://www.dropbox.com/s/w8pt4k1zh59gd04/test-5-%5BConverted%5D.jpg?dl=0 | 15:07.04 |
| half-a-pony | ping anyone ? I'm having trouble with inserting a JBIG2 in a PS : "(amb_1.jb2) (r) file /JBIG2Decode filter". I'm starting to wonder if it's GS or me (I'm using 9.20 on Fedora 24, didn't manage to recompile upstream yet). | 15:09.20 |
| | If I convert the jbig2 into a jpeg (via png using jbig2dec) & use /DCTDecode, it works. if I gdb into the gs process, and compare with jbig2dec process, it looks like segments are given wrong offsets while reading the jbig2 file. Has anyone tried the /JBIG2Decode filter ? | 15:09.43 |
| kens | Well, like I said PostScirp doesn't support JBIG2. So even if its technically possible with Ghostscript, its not really supported, you shouldnt be trying to do it | 15:09.59 |
| | Ah, and you can't simply supply an image like that in PostScript. You need to supply the image dictionary with all the keys filled in like /Widht and /Height etc | 15:10.53 |
| half-a-pony | kens: but I think ray told me that the /JBIG2Decode should work | 15:10.53 |
| ray_laptop | half-a-pony: there may be filter params that are needed for the JIBG2Decode filter (check the PDF spec, or look at our pdf interpreter PS)? | 15:11.01 |
| kens | half-a-pony : technically yes, but its not in the standard, and you can't do it with the one line there | 15:11.14 |
| | half-a-pony : we know the JBIG2Decode filter works, because that's how our PDF interpreter does JBIG2 images, and we know that's working because we have examples in our test suite | 15:12.22 |
| half-a-pony | kens, ray_laptop : ack, thanks, I ll keep trying then. | 15:13.20 |
| frostym | chrisl: Sorry to bug you again, after testing it appears Illustrator *always* includes private data. I'm unfamiliar with the Postscript syntax, is there something I can search for that will identify there are one or more non-vectors in the EPS file that won't scale properly or am I stuck processing every Illustrator-generated EPS file using Illustrator instead of Ghostscript? | 16:22.04 |
| chrisl | frostym: I suspect you're probably stuck :-( | 16:22.55 |
| frostym | Whether it was included due to transparency marking operations or not is probably irrelevant, I'm just trying to avoid having to purchase multiple Illustrator licenses to scale this out when the majority of files don't need it. | 16:23.08 |
| | Shoot, okay thanks. | 16:23.16 |
| kens | frostym : You can search for images, but how will you determine an image which is the result of an Illustrator flattening of transparency, and one which is an intentional image ? | 16:23.17 |
| frostym | kens: I don't think that distinction needs to be made when the alternative is process all of those files with Illustrator. Even if intentional ones get caught in the mix, the total # being sent to Illustrator would still be less than if I included every Illustrator EPS file. | 16:24.38 |
| kens | Determining how Illustrator includes images would probably be hard, and error-pronte too | 16:24.38 |
| frostym | I'm just interested in maximizing the % of images I can confidently send to Ghostscript and which have to fallback to Illustrator on a remote Windows server. | 16:25.17 |
| chrisl | There's a procedure called "snap_to_device", which seems like an odd thing to have in device independent files | 16:25.43 |
| kens | frostym : well, PostScript is a programming language, so you would need to determine which language fragments in the Illustrator prolog are used to draw images. THen you'd need to ignore anything in the prolog (because that will obviously include the definitions of that code) and make sure you skip every iteration of the prolgo. THen you could grep the remainder for occurences of that program fragment | 16:26.17 |
| | Or alternatively, you cuold, with some effort, get Ghostscript to tell you if a file uses any of the image operators. So you could presacn for Illustrator, then have GS check for images. If both conditions are met then pass off the file. | 16:27.15 |
| | But it would be slow, clunky and I suspect error-prone | 16:27.26 |
| frostym | I'm already downloading the EPS file from S3 before passing it to Ghostscript so if I could parse it at the same time as that and look for keywords that'd be ideal. If I can't figure out anything to look for I'll fallback to relying on GS to check for image operators and notify me, is that something already built in or would I need to add it? | 16:31.09 |
| kens | You would need to add it. | 16:31.24 |
| chrisl | Three of the four illustrator EPS I have immediately to hand that use snap_to_device look like they started with transparency in the original - the fourth is just a single line of text, so probably something cut down from a larger file | 16:31.37 |
| frostym | chrisl: Is "snap_to_device" what you think may be related to images or were you just noting that it seems out of place and could be used? | 16:31.38 |
| kens | Easiest way is probably to write code to redefine the standard operators 'image', 'colorimage' and 'imagemask'. Have then send output to sdtout when executed | 16:32.05 |
| chrisl | frostym: I'm wondering if snap_do_device is something Illustrator specifically uses for images that are flattened transparency | 16:32.29 |
| frostym | Okay cool, that's definitely helpful. I'll take a larger sample from our collection and script something up to test against that, hopefully it pans out. | 16:33.33 |
| chrisl | frostym: This is *total* guesswork, of course. | 16:35.22 |
| frostym | kens: Thanks. Are these operators not also defined in the Postscript commands in the EPS file and therefore able to be parsed at download time? Sorry I'm completely unfamiliar with Postscript syntax & how it relates to what I'm used to. I can kind of follow along with what's going on in my EPS files but it's not like anything I've worked with before. | 16:36.56 |
| kens | forsty The operators *will* be in the EPS file. But like I said its a *program* | 16:37.22 |
| chrisl | frostym: And even operators can be redefined! | 16:37.41 |
| kens | Sop the EPS might contain a definition of (lets say) /Illustrators_imgae_Proc, which uses the image operator. But the actual program may not call Illustartors_Image_Proc | 16:38.14 |
| frostym | Ah I see, I've been thinking the EPS file is similar to a script (ie. Python) and not something compiled | 16:38.32 |
| kens | SO just because the program has the operator 'image' that doesn't mean it gets used | 16:38.41 |
| | frostym : Its not compiled, its interpreted | 16:38.57 |
| frostym | Ohhhh | 16:39.08 |
| kens | But its still a Turing-complete programming language | 16:39.13 |
| frostym | So you're saying it may define it regardless of if it's being used or not, makes sense. | 16:39.20 |
| kens | Yes exactly | 16:39.28 |
| chrisl | frostym: So, if you are going to try checking for snap_to_device, you'll need to look for it being called, not just defined | 16:39.57 |
| kens | But if you redefine the operator *before* the program uses it, then you can have it do 'stuff' like saying 'Hi I am an imagemask' on stdout | 16:40.08 |
| | There is a caveat; its possible (but really bad practice, especially for an EPS file)for a program to call the 'systemdict' version of an operator, which can negate this kind of hackery.. | 16:41.03 |
| | Its getting late here in the UK, I need to go make some dinner. | 16:41.34 |
| | Goodnight folks | 16:41.39 |
| frostym | Is there a prefix or suffix to snap_to_device on the same line that is a clear indicator to if it's a definition or function call? | 16:41.43 |
| | Take care, thanks for the help kens. | 16:41.50 |
| chrisl | frostym: Yes, the definition have a name object, which is delineated with a leading "/", so "/snap_to_device" would be the definition | 16:42.42 |
| | Generally, just "snap_to_device" will be the procedure being called | 16:43.23 |
| frostym | Perfect. I'm going to pop over to my coworker's Windows machine and investigate further using Illustrator before scripting something out, thank you! | 16:45.37 |
| chrisl | frostym: just quickly...... | 16:45.57 |
| | The definition being prefixed with "/" and the call not is true in this case, but it is not *always* true - don't rely on it for other procedures | 16:46.35 |
| frostym | Understood. | 16:48.03 |
| chrisl | I'm off now, too - 'nite! | 17:27.46 |
| frostym | Take care, thanks again | 17:45.37 |
| | Forward 1 day (to 2017/04/13)>>> | |
Log of #ghostscript at irc.freenode.net.
