IRC Logs

Log of #ghostscript at irc.freenode.net.

Search:
 <<<Back 1 day (to 2012/07/05)2012/07/06 
Robin_Watts sebras: I'll review your patches and comments first thing tomorrow.00:40.06 
  I'm too knackered now to be sure of anything :)00:40.20 
  bedtime.00:40.56 
sebras Robin_Watts: no problem. nn.00:55.09 
  eh. np. nn.00:55.21 
  ty.00:55.29 
paulgardiner Robin_Watts: ping09:33.14 
sebras2 good morning.09:56.54 
  note to self: don't work too late, you'll forget to push your commits to your repo and then you'll delete your working space...10:15.24 
Robin_Watts Do we have a copy of the jpeg 2k spec in the company? shelly is asking.10:16.21 
  paulgardiner: pong10:16.41 
  sebras2: ouch.10:16.44 
sebras2 Robin_Watts: yes, and I worked until 4.30am... :-/10:18.26 
  Robin_Watts: luckily I remember most of the solution at least.10:18.51 
Robin_Watts git ref-log ?10:18.56 
sebras2 git commit -m "this was my last commit"; echo "hey, I'm done for tonight, let's delete everything instead of pushing"; rm -rf *10:19.44 
Robin_Watts ow ow ow.10:20.10 
paulgardiner Robin_Watts: the changes needed to determine text-field content type are on paul/forms, tacked onto the end of your merge10:24.37 
Robin_Watts I just pushed my changes to master, and the merge to forms.10:28.13 
  I think there is a problem with the cluster.10:28.23 
  In the case of pushing several commits at once, it's only testing the last one :(10:28.47 
paulgardiner Ah. My diffs came out right last time, but that was just one commit10:33.39 
Robin_Watts user commits are fine - this is for git ones.10:37.34 
  sebras: Looking at your "Defer dereferencing of pdf_crypt until it's needed" change.10:47.52 
  Can you explain why?10:47.57 
  I can see that it's a bit neater but wondered if there was a larger driving issue ?10:49.06 
  sebras: All your patches seem fine, so I've pushed them all.10:58.29 
  One minor niggle; your commit messages are quite terse. It would be nicer if there was a bit more explanation in there (why you are fixing this, what caused it to fail, example files to show the problem etc)10:59.42 
  For some of them, a 1 line message is fine, but I often find I have to read the diffs to know what the commit was about.11:01.26 
  Maybe that's just me being overly wordy as usual.11:01.34 
  for instance: "60da3be Make length check for crypt dict entries similar." Similar to what? Similar to each other? You can't trivially tell that from looking at the patch.11:06.48 
  "87cfdef Reorder parsing of encryption dict." What do we gain from this?11:07.10 
  I'm just going to repush the forms changes again to see if I can get the cluster to properly pick up on multiple changes.11:55.55 
  tor8: I had the germ of an idea last night about the type3 stuff.12:02.07 
  I was thinking, what if we added the ability to the device to define a 'subroutine' of graphics operators, that could then be repeated again and again.12:03.21 
  That would enable us to define type3 glyphs in a natural way.12:03.50 
  If the device wanted to cache the output in a bitmap it could.12:04.03 
  Then it dawned on me, don't we already have something in this mould already with the tiling code ?12:04.20 
tor8 Robin_Watts: yeah. the other thought I had was to just store the type3 font as a list of display lists, one for each glyph, instead of doing it with callbacks12:09.31 
Robin_Watts tor8: How does that differ from what I just said? (I'm probably missing a subtlety)12:10.43 
tor8 not much , just nailed down the implementation details :)12:10.58 
Robin_Watts right :)12:11.04 
tor8 I don't see an immediate need for it though, I figured out a solution to my problems yesterday12:11.15 
Robin_Watts ok.12:11.20 
tor8 now I just need to implement and test them out12:11.21 
Robin_Watts ISTR that locking may be interesting in this area.12:11.50 
  (for multi threaded access to the font cache etc - we need to drop various locks before recursing)12:12.43 
tor8 better make you triple check my patch then :)12:15.59 
sebras2 tor8: yo. have you some time to take a look at sebras/master?12:44.33 
tor8 sebras2: "Remove unnecessary check for unsupported crypt revision." ... doesn't that make it less clear that we take care of the v==3 case?12:48.00 
  also, I know we talked about the debug print functions. I know you discussed with Robin putting them in a separate file to reduce bloat, and I said I'd rather #ifdef them than expose details in header files, but I wonder if the added bloat of not ifdef'ing them out is worth worrying enough about to add ifdefs.12:49.10 
Robin_Watts sebras2, tor8: I pushed all sebras patches already.13:07.19 
tor8 Robin_Watts: okay.13:07.40 
Robin_Watts sebras: If you didn't see my comments above, you might want to check the logs.13:07.52 
sebras2 Robin_Watts: you are probably right about the commit messages. Normally I feel constrained about the 60 char first line limit, but never add a paragraph below it. I'll try to think about it next time. and complain if I still make omissions of course. :)13:11.28 
Robin_Watts sebras2: At Picsel we had a template to follow for every commit. I feel that was overly restrictive, but it did force commit messages to be more useful in the long run.13:12.26 
sebras2 Robin_Watts: I hope we have self discipline enough not to warrant such a template to be enforced as a commit hook.13:13.54 
Robin_Watts sebras2: It wasn't a commit hook; because we did all commits through a review system, the review system would prepopulate the commit message field with the template for you to edit.13:14.45 
sebras2 Robin_Watts: ah, right.13:15.21 
  Robin_Watts: after you had read all the patches I guess they made sense?13:15.47 
  Robin_Watts: or is there still something unclear?13:15.54 
Robin_Watts All the patches made sense, certainly, or I wouldn't have pushed them.13:16.18 
sebras2 tor8: and no we don't take handle the case of v==3 other than bailing. and that is done at the top of the function.13:16.39 
Robin_Watts There were a couple I couldn't see immediate justifications for other than "it's neater this way" (which is no bad thing!)13:16.44 
sebras2 tor8: and the ifndefs were trivial, so. :)13:16.56 
  Robin_Watts: the defering I did because it seemed silly to extract three fields from crypt and add them as arguments to the crypt-filter function. originally there were two but I needed to look at crypt->r in a later patch.13:18.05 
  Robin_Watts: so I opted to try to reduce the number of arguments and dereference crypt inside the function instead.13:18.26 
Robin_Watts sebras2: Right -that's what I meant by neater. I agree it's better your way.13:18.35 
sebras2 Robin_Watts: I was hoping that the sequence of patches would make it obvious, but I guess I need to rethink that strategy. :) at least each patch was fairly trivial by itself, so.13:19.44 
Robin_Watts sebras2: Tiny fix on my master branch for the compactxref stuff.13:20.02 
  I think that was yours, originally, right?13:20.09 
sebras2 Robin_Watts: yes I remember that -ggg didn't work, but I never got around to solving it.13:21.32 
  though that was prior to pdf_write...13:23.02 
Robin_Watts pdf_write is just a refactoring of pdfclean, so I suspect the same bug was carried across.13:26.20 
  certainly the fix makes sense (and solves the supplied files problem)13:26.36 
sebras2 Robin_Watts: ok, that's good. thanks for fixing my bad conscience.13:27.06 
Robin_Watts tor8: Bug 693141 - do you have any strong feelings on this?13:50.50 
  The changes are 1) some tiny tweaks to the #ifdefs (including || defined(__BORLANDC__) in various places)13:51.35 
  2) a change to a struct initialiser in draw_device.c (which it could be argued is better done explicitly).13:52.27 
  3) some changes in the cmap sorting stuff.13:52.42 
tor8 Robin_Watts: I can't really say I care about an obsolete compiler (or isn't it?)13:53.16 
Robin_Watts I think taking 1 and 2 on are fine. 3 I suspect should go back to the original reporter and tell him to stop patching the symptom.13:53.28 
  I can't say I'd ever want to use it.13:53.47 
tor8 if the changes make general sense, and cleans up things that may be ambiguous I'm okay with them. can't really tell unless I start diffing.13:53.51 
Robin_Watts http://ghostscript.com/~robin/diff.txt13:54.28 
tor8 the problem with taking on #ifdef's for obscure platforms is the same as the issue with the QNX port -- who will maintain them13:54.32 
Robin_Watts tor8: yeah.13:54.53 
  I think the fz_matric ctm = { ... } thing must be failing on that compiler because 4 of the 6 things are ints not floats.13:55.39 
tor8 we use struct initializers elsewhere don't we?13:56.04 
sebras2 tor8: to me supporting QNX (hello RIM!) seems like it might be worthwhile, but bcc?13:56.05 
Robin_Watts I'd be tempted to use the long form and explicitly cast for clarity.13:56.09 
tor8 fz_matrix ctm = (fz_matrix) { ...init... } you mean?13:56.35 
Robin_Watts fz_matrix ctm; ctm.a = (float)glyph->w; ctm.b = 0.0f; etc13:57.03 
tor8 the 3) change with cmap sorting ... shouldn't matter should it? or do we actually have cmaps with overlapping ranges?13:57.49 
Robin_Watts tor8: I suspect that there is a compiler bug that is causing the generated cmaps to be wrong on bcc.13:58.13 
  (he says the generated cmaps are different on bcc than on MSVC and they clearly shouldn't be).13:58.35 
  So the fix should be to fix the code so that the generated cmaps AREN'T different. Then his change would be unnecessary.13:58.57 
  That's what I meant by "patching the symptom"13:59.22 
  I'll put the patch on the bug, and comment there.14:00.58 
  So this file has a 256x256 image, which is scaled up to 68267x68267 with interpolation. And they say "it's slow".14:55.19 
srathbun Hi all, I've got a problem with a 30,000 page pdf that I'm converting. The thing spools fast for the first 10,000 pages, then slows down and eventually crashes. Is there any way to get ps2write to not buffer output in memory?15:08.49 
Robin_Watts srathbun: Our expert on this is away until the middle of next week.15:10.38 
  But I'll do my best to answer.15:10.57 
  The short answer is no, I believe.15:11.03 
  The question is *why* are you converting a 30000 page pdf to postscript?15:11.36 
  You could maybe look at doing the file in chunks of 1000 pages ?15:12.23 
chrisl srathbun: ps2write doesn't buffer output in memory, but it has to track "stuff" in current state (fonts, images, etc) so it knows how/when to reuse - the more pages you have with different "stuff" on them, the more ps2write needs to keep track of.15:18.38 
srathbun Well I'm testing doing it in gs instead of letting the printer do the conversion. 15:20.06 
  Since the printer spools really slowly too15:20.34 
chrisl If this *just* for printing, how about splitting into several 5k pages jobs, instead of one 30k page job?15:21.13 
srathbun I'm slightly surprised though, I wouldn't have thought that 30,000 pages would cause issues15:21.20 
chrisl It really depends what's on the pages, rather than the number of pages15:21.44 
srathbun chrisl: The company procedures are such that a single file is *much* easier to deal with. I can break it up, but that defeats the purpose15:23.10 
henrys chrisl:the bytecode retry is fine by me if it doen't introduce a regression.15:23.14 
Robin_Watts srathbun: You don't need to break the file up per se.15:23.38 
  Process the file once with -dFirstPage=1 -dLastPage=100015:23.59 
chrisl henrys: the cluster test was clear - I'll give Norbert a while to chime in before I commit it15:24.11 
Robin_Watts then process it again with -dFirstPage=1001 -dLastPage=200015:24.16 
  and you can safely concatenate all the output postscript files together to give you one file.15:24.39 
srathbun chrisl: All the pages are text runs and stroke paths15:24.44 
chrisl srathbun: then it's prbably fonts causing the slowdown15:25.11 
srathbun Robin_Watts: would that not duplicate font information?15:25.13 
Robin_Watts srathbun: You're using ghostscript in a company? Which company ?15:25.14 
  Yes, but only once every 1000 pages.15:25.29 
srathbun L & D Mail Masters15:25.31 
Robin_Watts Do you have a support contract with us?15:25.45 
  (We prioritise customer requests)15:25.57 
srathbun No, we don't15:26.12 
  Management is learly of gs, since they had issues with it in the past15:26.27 
  I've reintroduced it for some processes15:26.37 
  leery (sic)15:26.46 
  chrisl: I know that 2 of the used fonts are *special* fonts used by the process that builds the pdf. I'll try removing them and see how it goes15:28.27 
  Thanks everyone!15:28.42 
Robin_Watts srathbun: You could thank us by getting a support contract :) That way your issues with gs might get fixed.15:30.28 
chrisl srathbun: That may not help - basically, ps2write doesn't get told whether two instances of "setfont" apply to the same font. It has heuristics to work it out, and it has to store information about each font for those to be applied. But it may be that those heuristics are being defeated.15:31.24 
srathbun Robin_Watts: If an opportunity presents itself, I'll let management know15:44.44 
Robin_Watts srathbun: Thanks.15:44.56 
srathbun chrisl: I won't know until I test it, so I'll try testing both batching it and dropping the special fonts15:46.06 
chrisl Oh, wait for a reply, why don't you......?15:47.13 
Robin_Watts chrisl: I think they are a clear case of "should have a support contract". Any company using gs to process 30K page files for commercial mailouts should really be prepared to pay something.15:49.00 
chrisl Robin_Watts: anyone creating 30K page PDF files should be ashamed of themselves, anyway!15:49.47 
Robin_Watts chrisl: Nah, for mail merges it makes sense.15:50.09 
  Lots of stuff repeated per page can be factored out into common contents streams etc.15:50.37 
chrisl It's not what PDF was really designed for15:51.02 
Robin_Watts PDF was designed to do everything. just ask adobe :)15:51.30 
chrisl <sigh>15:51.39 
Robin_Watts Why, when we are writing to a tiffg4 device, is dev->color_info.max_gray = 255?15:52.11 
  oh, pdf14gray.15:52.39 
henrys Robin_Watts:I think you should tell Scott what happened and let him harass them. I can't feel any remorse for pestering a junk mail operation.16:03.26 
carlos bom dia. nao consigo fazer um pdf com pontos para serigrafia16:08.51 
  Good day. I can not point to a PDF with screen printing16:09.03 
  I have installed corel 6, with ghostscript and GSView, and I can not put the pdf to get to points in 40 lpi.16:10.29 
Robin_Watts carlos: hi. Sorry, I'm afraid I can't understand the issue from that.16:18.26 
  henrys et al: Got time to discuss an issue?16:30.17 
  Customer 700 have just reported 2 bugs to marcos, one of which is bug 693166.16:30.43 
  The issue at the core appears to be that they are upscaling images which then go to a halftone device.16:31.08 
henrys reading the bug16:31.27 
Robin_Watts If used with interpolation, this takes a long time, partly cos they are upscaling a lot, and partly because we end up having to process lots more 'pixels' than we did before.16:32.37 
  (without interpolation, we output 500x fewer distinct pixel values into the underlying device).16:33.35 
  Mupdf performs well, as it doesn't bother to interpolate upscaled things.16:33.54 
henrys can we tell what adobe is doing when going to a tiff?16:34.13 
Robin_Watts I don't see how.16:34.22 
chrisl Can't you see whether Adobe is interpolating?16:34.53 
Robin_Watts chrisl: When going out to a 1bpp halftone device?16:35.06 
  Not with the files they supply certainly.16:35.24 
chrisl Well, if you render once with interpolation, and once without, it should be possible to see the difference16:35.38 
Robin_Watts It should be possible to see if there is a difference, yes, I'll try that.16:36.09 
chrisl It might be that Adobe disables interpolation for halftone devices, or for stupidly large scales, or for stupidly large scales to halftone devices.....16:37.06 
Robin_Watts I don't appear to have an interpolation on/off switch in acrobat pro 9 - should I?16:37.07 
henrys hmm I thought there was.16:37.31 
chrisl No, you'll have to change the PDF16:37.31 
Robin_Watts ha. acrobat pro won't output it at all at 400dpi :)16:37.42 
chrisl Only for the screen view, not for tiff output16:37.46 
Robin_Watts I could make a test file where interpolation would show up, I guess.16:40.37 
henrys Robin_Watts:you type in the 400 dpi - don't use the menu.16:41.01 
chrisl Robin_Watts: is GS interpolating just because we're upscaling - i.e. not because the flag is in the image dictionary?16:41.14 
Robin_Watts I typed 400dpi, and it changed it to 2400.16:41.20 
  even with typing it, I can't set it to 400dpi :(16:41.39 
chrisl henrys: the page is too big, I think, Acrobat won't render to a 400dpi tif anyway16:41.54 
Robin_Watts Looks like there are lots of "/Interpolate true" in the file.16:42.38 
  So, I *could* make gs ignore upscaling interpolation requests when we are going to halftone devices.16:43.37 
chrisl So at least we are being asked to interpolate - that's something.....16:43.38 
Robin_Watts but, as I say there are cases, where we would notice the lack of interpolation.16:44.01 
henrys chrisl:so marcos comment 1 is incorrect?16:44.01 
chrisl henrys: no, I'd opened the wrong file, sorry16:44.23 
Robin_Watts henrys: Evidently. Certainly, I can't make it save out at 400dpi16:44.33 
henrys Robin_Watts:worked for me.16:46.32 
  monochrome right?16:47.00 
Robin_Watts yes.16:47.11 
  G4 Tiff16:47.18 
chrisl Huh, no, with the correct file, I get the "page too wide" error @ 400 dpi16:47.41 
henrys I get a 4800 x 3200 tiff16:49.06 
  it doesn't really matter - does you can render it at a lower resolution question is does adobe disable interpolation for large/ht images.16:50.48 
Robin_Watts OK. I made a test file. http://ghostscript.com/~robin/testinterp.pdf16:51.08 
  and that clearly exports interpolated.16:51.23 
  One idea I had was to reduce the interpolation if we are going to a halftoned device.16:52.11 
  If we have a 16x16 image that's being displayed at 1024x1024, and then put through an 8x8 halftone grid, it seems silly to scale it up by 64 in each direction.16:53.15 
  We could probably get away with just scaling it up by 8.16:53.26 
  Which would send 1/64th of the data through to the underlying device.16:53.56 
  but I don't know whether that's possible with the way the gs code is structured.16:54.14 
chrisl I'm of the opinion we should reduce the extent of our interpolation all the time - it brings so many performance issues as it is16:54.29 
Robin_Watts chrisl: Well, for contone output and 8 bpc input, if we restrict interpolation to less than 256 times bigger, we'll get a detectable drop in quality.16:56.08 
  (theoretically detectable)16:56.25 
chrisl Robin_Watts: Well, I feel if you want good quality interpolation, do it in the producing application, where time isn't as critical.....16:57.12 
Robin_Watts but you can't know the output resolution there.16:57.28 
chrisl But clearly scaling up to the extent we see (like this one), it's going to suffer, with or without interpolation - HQN managed for years with no interpolation....16:58.54 
henrys so when testinterp.pdf is rendered as a mono tiff in adobe with interpolate false there is a difference?16:59.06 
Robin_Watts henrys: I didn't try that. The underlying bitmap is a 16x16 black and white one.16:59.50 
  hence by seeing any shades of grey in the export, we know it's interpolating.17:00.19 
chrisl Robin_Watts: Anyway, I certainly agree that the level of interpolation we're doing now is way OTT for halftoned output17:02.53 
henrys I think we should turn off interpolation in the customers file and see if the Adobe rendering is different. - that was the royal we - I'm back to a PCL problem I promised finished today.17:12.17 
chrisl The customer's file does render differently in Acrobat with and without interpolation :-(17:13.13 
  Robin_Watts: if you want the customer file with interpolation remove, I can put it up on casper for you17:14.52 
Robin_Watts chrisl: Wouldn't hurt, thanks.17:15.10 
chrisl Robin_Watts: it's in my user area, PW-880b-nointerp.pdf17:16.04 
Robin_Watts Thanks.17:16.19 
chrisl np17:16.24 
  henrys: while I remember: Shelly is looking is this OpenJPEG issue Robin sent his way - he said he'd be interested in tackling the OpenJPEG 2.x integration when we need it (it looks like a lot more than a trivial update).17:17.08 
Robin_Watts he also asked if we had the openjpeg spec in the company.17:17.34 
  chrisl: OpenJPEG 2 integration would require them to actually get around to releasing OpenJPEG 2 of course.17:17.58 
  (not openjpeg spec, jpeg 2K spec, sorry)17:18.12 
chrisl Robin_Watts: it's reached alpha......17:18.19 
Robin_Watts I wouldn't like to bet how long it will take to reach beta though...17:18.51 
chrisl No, but at least it's evidence of progress17:19.20 
henrys I do think alexcher should handle any major integration openjpeg stuff.17:20.35 
Robin_Watts Gah. gs' code is really not conducive to supporting 'interpolate, but not all the way'.17:20.53 
henrys I don't know anything about the spec.17:22.52 
chrisl Robin_Watts: I'm pretty sure Ken has at least the final draft, possibly Ray has a copy? Anyway, we'll ask Ken next week17:24.55 
Robin_Watts oh. I appear to have it here, squirreled away in a corner of my hard drive.17:25.55 
chrisl Cool, that saves some time :-017:26.16 
  Anyway, gotta go - have a good weekend everyone!17:26.36 
henrys Robin_Watts:just a thought does it render okay without banding? - max bitmap large enough - maybe the customer is okay throwing memory at the problem.17:27.38 
  I know if it's big but it's 1 bit.17:27.55 
Robin_Watts but the scaling is being done in 8 bit, as there is a pdf14gray device in there.17:28.17 
henrys ugh17:28.59 
Robin_Watts (and independently of the current problem, the existing "is it a halftone device" tests in the scaler are being defeated by the presence of that.17:29.00 
  I used the more anglo saxon version of "ugh" when I spotted that.17:29.21 
  how are the fires, btw?17:29.43 
henrys contained for the most part. We have air quality warnings from smoke coming in from other states.17:32.18 
  I question this whole business of sending humans in to protect people's shit - every house that's burned has been in an identified "dangerous" area. I hate to see stuff like the tanker crash with people killed to protect property.17:34.59 
Robin_Watts "If you build your house in this area, we will not protect it in the event of a wildfire."17:36.04 
  The problem with that is you'll get people staying at home to fight it themselves rather than letting the professionals in (who presumably are much smarter about saying "Fuck it" and running away when it gets bad)17:36.51 
henrys we had some clown start a fire around his house to consume the fuel as the fire bared down on him. I think he's in jail or something.17:38.39 
Robin_Watts That's the kind of smart thinking that saw a woman in the UK horribly burn herself by transferring petrol from one unsuitable container to another in the kitchen with a gas ring on.17:40.31 
  I could write a wrapper image class for the interpolater18:08.42 
  which would pass through to the existing interpolater to do the work, but would then postprocess the output to scale it up.18:09.28 
henrys we must be doing something horrendous the acrobat rip time isn't even noticable.18:12.16 
Robin_Watts 90% of the CPU time is spent in our interpolation loop.18:12.57 
  sorry, no. 90% of the CPU time is spent in the interpolation FUNCTION. 60% is in the actual interpolation loop.18:13.53 
henrys is the performance as bad with a contone device?18:17.29 
Robin_Watts Almost, I believe. Only 12.5% is down to cmap_transfer_halftone18:17.58 
  Hey kens. How's NY?18:23.52 
kens Hot :-(18:24.30 
  Currently lurking in hotel to get out of the heat for a bit18:24.49 
henrys hi kens18:25.38 
  Robin_Watts:the pdf spec does give us some wiggle room with interpolation - maybe at some size we should turn it off, that would be a quick fix for now.18:26.25 
Robin_Watts henrys: possibly.18:26.35 
  At 100dpi, we take 25 seconds.18:26.43 
  at 200dpi we take 25018:26.58 
sebras2 bib18:44.37 
henrys Robin_Watts:maybe mvrhel can weigh in with an idea.18:45.14 
Robin_Watts yeah. I'll ponder it over the weekend and talk to him when he returns.18:45.37 
henrys bbiaw18:46.08 
alexcher I've found a serious problem. Sometimes --restore-- selects memory allocators that have gc signal address pointing to the wrong place and corrupting a local variable in interp().22:25.51 
  This causes SEGV in 34_all.PS and a few other places.22:26.54 
  Does anybody know when 34_all.PS has been broken?22:28.54 
 Forward 1 day (to 2012/07/07)>>> 
ghostscript.com
Search: