| | <<<Back 1 day (to 2012/04/09) | 2012/04/10 |
| ppd | chrisl: Hi, I read your bug comment. I'm sorry for the 1,1 mb vs. 5 mb difference. but the 1,1 mb was the size if converted with ps2pdf. 5 mb is when processed through cups | 07:11.59 |
| kens | Presumably because the image is not compressed. Because of the problems the pritner has with compressed data. | 07:12.27 |
| chrisl | ppd: yeh, I worked out where the confusion came shortly after I hit the button to post the comment | 07:12.30 |
| ppd | so as far as I can see the image that gets to the printer is just too big for its limited capabilities to process fluently | 07:16.01 |
| chrisl | ppd: so it seems, yes. Still, 5Mb isn't *that* large for an image, these days...... | 07:16.41 |
| kens | From what you say chrisl I can't really see any prospect for improvement. Using a newer version of Cairo might help, if its a Cairo PDF file. | 07:18.05 |
| ppd | chrisl: true. I wonder what the windows driver does with such "big" images. I heard it was not such a problem there, but I don't have any windows here... | 07:18.13 |
| chrisl | kens: the problem here is that the transparency is "real", it's just another stupid cairo file :-( | 07:18.59 |
| kens | chrisl yes that's what I meant. | 07:19.14 |
| | Removing transparency means creating an image, no choice there. | 07:19.30 |
| ppd | say. this compression you speak of. Can't I turn it on easily and check whether this specific printer likes it better then? | 07:19.32 |
| kens | I can't recall if compression is off for that printer or not. | 07:19.54 |
| chrisl | kens: I meant it's *not* just another stupid cairo file :-( | 07:19.59 |
| kens | Ogh, the transparency really does cover the whole page ? | 07:20.13 |
| | Not much we can do then. | 07:20.19 |
| chrisl | kens: the transparency is only in a fairly small SMasked image *but* the group which it is in is the page group | 07:20.52 |
| ppd | because the pdftops tool produced a much smaller output | 07:21.05 |
| kens | Hmm, well that atill smacks of a Cairo problem. | 07:21.15 |
| chrisl | ppd: I'm not convinced that the poppler implementation is a"fully compliant" with PDF transparency, so it may be making assumptions | 07:22.20 |
| | kens: one problem is, this seems to me like a fairly reasonable use of transparency, and I suspect other producers will create files like this - I'm not even sure this is is a Cairo file. | 07:23.43 |
| kens | chrisl without looking at the PDF file I can't be sure, but putting an SMask ina Group seems overkill | 07:24.16 |
| ppd | this pdf is a libreoffice export | 07:24.20 |
| chrisl | kens: the entire page contents are in the page group, as they must be, and the SMask image is part of that page group - it's not in a group of its own (it would probably be better if it were). | 07:25.24 |
| kens | That's kind of what I meant | 07:25.40 |
| chrisl | ppd: I'm not sure if libreoffice is using cairo behind the scenes. | 07:26.00 |
| kens | I'm not certain you need a /Group with a /SMask anyway but I would have to reread the spec to be sure. | 07:26.13 |
| chrisl | kens: eh? This is an SMasked image, not an SMasked group, not an SMask group | 07:27.04 |
| | s/not/not | 07:27.14 |
| | s/not/nor | 07:27.18 |
| kens | chrisl but hte SMask applies to the image, so I don't think you need a Group to contain it. | 07:27.23 |
| | But like I said, I'd need to go look at the spec again. | 07:27.37 |
| | I try to forget transparency whenever possible. | 07:27.45 |
| chrisl | You always need a group with transparency - the page group | 07:27.48 |
| ppd | chrisl: you're talking of transparency. Can I avoid this problem by only using non transparent pngs? | 07:36.20 |
| chrisl | ppd: I suggested that on your bug report, but it depends on how libreoffice handles these things. You could certainly try it - I think gimp can "flatten" pngs | 07:37.16 |
| ppd | nice. I'll try | 07:37.58 |
| chrisl | kens: it *seems* that, even with the SMask image in its own group, ps2write still flattens the entire page group - hmmmm, does that sound right? | 07:41.15 |
| ppd | chrisl: the postscript is again as big. 5 mb. I think that didn't help | 07:44.50 |
| chrisl | ppd: so, libreoffice *is* stupid in how it produces these files - it does sound like they're using cairo :-( | 07:45.36 |
| | ppd: if you like, I can create a test file with the compression still in place, for you to try | 07:46.03 |
| ppd | chrisl: I'd love to | 07:46.49 |
| chrisl | ppd: okay, give me a few minutes - I'll attach it to your GS bug report | 07:47.22 |
| kens | sorry, went for a coffee. | 07:48.12 |
| | chrisl regarding transparencym you are asking the wrong person, better to ask Michael ;-) | 07:48.29 |
| | I am pretty sure that we normally only render transparency as required though | 07:49.15 |
| | Which suggests we aren't getting any benefit from teh group boundingbox. Maybe Michael would know why. | 07:49.41 |
| chrisl | It's just that it should be fairly trivial to push and pop a group around an SMasked image. | 07:50.24 |
| kens | Indeed, but I really don't know much about the internals in GS here. | 07:50.42 |
| chrisl | We already do a similar thing for a few things - Type 2 patterns, IIRC | 07:51.18 |
| kens | Yes, possibly some other things too | 07:51.41 |
| | We build a clist and run it to get an image, where the image size is based on the object. We then do some hacky stuff to reposition the image on the output | 07:52.17 |
| | I thought we did the same with transparency, but like I said, its not really my field. Ray is looking at it though. | 07:52.52 |
| | I think the transparency support in pdfwrite is a bit primitive, the problem ray is looking at is that we *don't* create a clist, we try to create a canvas big enough for the transparency operation. | 07:53.35 |
| | For full pages, at high resolution, we get a VM error. | 07:53.50 |
| | But I did think that we didn't always run to a full page, otherwise this would come up more often. | 07:54.11 |
| chrisl | The problem *might* be if the XGStates in the group all contain CA and ca settings (of 1.0, of course), it may think the entire page group needs flattened. | 07:55.26 |
| kens | Possibly, like I said, not an expert here :-) | 07:55.46 |
| chrisl | As I say, this one seems to me worth looking into, because it does seem like a fair use of transparency, which we're possibly not handling optimally. | 07:57.24 |
| kens | I can't spend time on it right now, I'm in the middle of memory optimisation. | 07:57.50 |
| chrisl | You're right, at least initially, we'll need input from Michael, although I could do the PS hacking, if required. | 07:59.10 |
| kens | Right, finally got my Linux Git to pull update my tree. | 08:01.57 |
| chrisl | ppd: I've posted two tests for you to try: the first is with compression, and the second uses a ProcessColorModel of DeviceCMYK as well as compression. | 08:04.44 |
| ppd | chrisl: very nice. I'll feed them to this lousy kyocera | 08:08.01 |
| kens | chrisl was it just the CCITTFax filter that had the problem or was there some other compression filter bug as well ? | 08:09.29 |
| chrisl | kens: that was the Brother printer, wasn't it? It was others, too - at least, LZW was a problem, as well. | 08:11.22 |
| ppd | chrisl: the compressed one came out faster. I'll compare it now to the poppler one... | 08:12.48 |
| kens | chrisl can't keep track ;-) | 08:13.05 |
| chrisl | I know the feeling :-( | 08:13.24 |
| kens | Really, the compressed image should be *slower*, unless the printer is on a really slow connection.... | 08:13.33 |
| | The time take to decompress the image I would expect to exceed the transmission time. | 08:13.49 |
| chrisl | Hmm, or Kyocera is doing some *really* bad buffer management - which would not surprise me. | 08:14.17 |
| kens | Or has a really bad comms implementation :-) | 08:14.34 |
| ppd | wait. I'm confused now. I'm printing an measuring the time it takes | 08:14.57 |
| kens | If the compression problem was a different printer, why would we disable compression on the Kyocera ? | 08:15.16 |
| | ppd that sounds like the right thing to d | 08:15.23 |
| | do | 08:15.26 |
| | Presumably you start timing when you press 'return' on the command line | 08:15.53 |
| ppd | 44.6 seconds for sending the compressed ghostscript ps | 08:15.57 |
| | 4.1 seconds for the poppler ps | 08:16.06 |
| chrisl | kens: we also found a couple of HP printers that had problems with the compression settings, so Till and I thought disable it, rather than maintain a list of special cases. | 08:16.24 |
| ppd | I do it like so: time nc -w1 ..... < file.ps | 08:16.32 |
| kens | ppd what are the file sizes ? | 08:16.45 |
| ppd | 807 kb for the poppler one | 08:17.18 |
| kens | chrisl, well it makes sense, unless it causes other printer problems..... | 08:17.24 |
| chrisl | :-( | 08:17.34 |
| ppd | and 1.1 mb for the ghostscript one | 08:17.39 |
| kens | That sounds like the priner is doign more work with out PostScript then. | 08:17.59 |
| | Did you try the CMYK version ? I wonder if its just spending a lot of time converting RGB to CMYK | 08:18.18 |
| chrisl | kens: the uncompressed one is ~5Mb | 08:18.22 |
| kens | chrisl did you look at the poppler code ? | 08:18.52 |
| chrisl | not in any detail, no | 08:19.10 |
| kens | OK was wondering if it was a full page image too. | 08:19.25 |
| | Given the size of our prolog it sounds comparable. | 08:19.42 |
| chrisl | no, definitely not | 08:19.45 |
| kens | Hmm, interesting. | 08:19.52 |
| ppd | I'll try the cmyk now | 08:20.53 |
| chrisl | Actually, I'll double check the content of the poppler PS..... | 08:21.42 |
| | kens: actually, I take it back, it is a full page image in poppler PS - but the PS also contains a font.... | 08:23.45 |
| kens | OK well that is useful to know. We *ought* to be able to match the poppler speed then. | 08:24.10 |
| | Though I wonder if its the way we handle images, I don't recall how the prolog deals with XObjects | 08:24.38 |
| | Possibly we will always be slower, but we ought to be more comparable I'd have thought | 08:25.00 |
| | Is the Poppler image CMYK ? | 08:25.06 |
| chrisl | No, RGB, but it is lower resolution - looks like it might 300dpi | 08:25.42 |
| kens | Ah, well that would make a big difference. I#'m surprised their file is so large, but the presence of the font would probably accoutn for that. | 08:26.07 |
| ppd | 53.3 seconds for the cmyk one | 08:26.15 |
| kens | Longer, that's a surprise. | 08:26.23 |
| | Can we try with ps2write set ot 300 dpi ? | 08:26.53 |
| chrisl | Let me double check the resolution | 08:27.13 |
| kens | Anyone know what the resolution of the printer actually is ? | 08:27.20 |
| | chrisl is there a make target for PCL debug only ? | 08:29.19 |
| chrisl | So, it does indeed look like poppler is render @ 300 dpi | 08:29.35 |
| | kens: IIRC, make pcl-debug | 08:29.48 |
| kens | hypehn not underscore, thanks ;-) | 08:29.59 |
| | Yes, that seems to work thanks | 08:30.19 |
| chrisl | ppd: so, what resolution are these Kyo printers? (preferably "real" resolution, not marketing bumph, if possible) | 08:30.52 |
| kens | Do we know the model numer ? | 08:31.10 |
| chrisl | FS1320D | 08:31.35 |
| kens | In fairness, a 300 dpi contone image is suitable for printing at p to ~1200 dpi | 08:31.46 |
| | Claims its max resolution is 1200x1200 | 08:32.25 |
| | fast is 1800x600 (what ?) | 08:32.57 |
| | 600x600 and 300x300 | 08:33.04 |
| | So 300 dpi should be suitable for any mode on this printer | 08:33.19 |
| | Also, because its an integer multiple of the resolution, and 720 isn't it will spend less time scaling. | 08:33.40 |
| chrisl | I bet 1800x600 is actually 600x600 with uber accurate placement on the X axis | 08:33.41 |
| kens | I suspect its really 600x600 pixel size, but 1200x1200 pixel placement | 08:34.14 |
| | But I suspect running the resolution to 300 will show marked improvements no matter what mode this printer is in. | 08:34.47 |
| | Since ours is 720 it will have to rescale, again no matter what mode it is in. | 08:35.07 |
| | So a 300 dpi test sounds well worth pursuing. | 08:36.10 |
| chrisl | So, I reckon tkamppeter will need to include a suitable resolution argument for Ghostscript - so "normal" would 300dpi, and "high" would be 600 | 08:36.13 |
| kens | That would be sensible. But by extyension of the argument above... | 08:36.34 |
| | Many printers have 360/720/1440 dpi so a 360 or 720 resolution would be faster on those printers, because they won't have to do so much scaling work | 08:37.11 |
| | But we should do the test first, it may not make any difference ;-) | 08:37.41 |
| chrisl | The resolution should be available from the PPD, shouldn't it? | 08:37.47 |
| kens | chrisl yes it should | 08:37.56 |
| chrisl | So, do we do 300 dpi uncompressed, or compressed - I'd prefer uncompressed, as I still think it would be better to avoid long term maintaining of a special cases list | 08:38.34 |
| kens | Try uncomrpessed | 08:38.47 |
| | If we're right about the scaling then the compression will not make much difference | 08:39.01 |
| | Is this a monochrome printer ? | 08:39.11 |
| | It seems to be. | 08:39.15 |
| chrisl | Oh, I shouldn't have done CMYK then..... | 08:39.40 |
| kens | It would explain why it is slower ;-) | 08:39.53 |
| | Oh yes, right at the top it says 'B&W pritner' | 08:40.23 |
| chrisl | how dare they hide it like that! ;-) | 08:40.39 |
| kens | Could always set /DeviceGray :-) | 08:40.45 |
| | Its my fault, I speed read straight past it. | 08:41.11 |
| chrisl | Again, shouldn't that come from the PPD and/or the driver settings? | 08:41.14 |
| kens | The documentation is completely clear | 08:41.20 |
| | chrisl yes, I was just thinking in terms of us dong testing. | 08:41.33 |
| chrisl | Yeh, I had the page up as well, and missed it | 08:41.35 |
| kens | If we find these are making a difference, we could make some specific reccomendations for Till | 08:42.07 |
| | It depends what he has access to exactly, and how much effort he is willing to invest in this of course. | 08:42.27 |
| | I suspect the compression is mostly irrelevant, network speeds are high enough that the decompression time in the printer probably outweighs any gains on transmitting less data. | 08:43.18 |
| | Setting the resolution 'appropriately' is probably the biggest gain. | 08:43.43 |
| chrisl | Doh, works better if I set the -r option...... | 08:43.56 |
| kens | Choosing a colour model is 'nice' probably, not least because long term it will be possible to make use of the LCMS2 colour management | 08:44.22 |
| | But probably has little to do with performance. Though I am interested that it is nearly 25% slower using CMYK vs RGB | 08:44.59 |
| | chrisl, ppd if you can spare the time a test using DeviceGray might be of value. | 08:45.25 |
| | I see the engine is pretty quick, its rated at 35ppm for A4 | 08:46.14 |
| | CPU is a 360MHz PowerPC, doesn't give the memory that I can see | 08:46.59 |
| | Ah, it does do duplex as standard | 08:47.50 |
| | only has 32Mb of Ram as standard | 08:48.11 |
| ppd | count me in. I'll be eternally grateful for working printers in the office... | 08:48.46 |
| kens | wishes he'd installed a bigger virtual disk in his Linux VM | 08:49.51 |
| chrisl | ppd: two more tests uploaded - one as CUPS produces, but 300dpi, and the second is as CUPS produces but 300dpi and gray scale. | 08:50.23 |
| | kens: what virtualisation are you using? | 08:50.37 |
| kens | chrisl VMWare | 08:50.52 |
| | I may just make a new VM | 08:51.15 |
| chrisl | You *can* increase the disk size, but it's a bit convoluted | 08:51.18 |
| kens | I might install a copy of Ubuntu :-) | 08:51.31 |
| chrisl | Hmm, good luck.... | 08:51.54 |
| kens | in fact I think I might have to do something, I can't build PCL in debug mode, not enough disk space | 08:51.55 |
| | One last try | 08:52.42 |
| | No, still runs out of disk space | 08:53.06 |
| chrisl | Oh, according to VMware, recent versions can increase the disk size from the GUI | 08:53.43 |
| kens | Sadly I don't have a recent version | 08:53.53 |
| | And when I try to update it barfs | 08:54.00 |
| chrisl | Can you download, and run the installer, instead of letting it update itself?] | 08:55.00 |
| kens | Hmm, interesting idea, haven't tried that | 08:55.18 |
| chrisl | If you have a utility called "vmware-vdiskmanager" you can do it "manually" | 08:55.50 |
| kens | What, in my Linux tools ? | 08:56.16 |
| chrisl | No, in windows - it works on the virtual disk file | 08:56.35 |
| | http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004047 | 08:57.14 |
| kens | Hmm, I'll look | 08:57.25 |
| chrisl | You do need to boot something like an RIPLinux iso image in the virtual machine in order to resize the partition. | 08:58.15 |
| kens | Well I'm downloading the latest installer, had to login and its 500Mb but its on the way | 08:58.48 |
| | We'll see if ti will update for me :-) | 08:59.07 |
| chrisl | kens: another option is to just create a second virtual disk, but then you may have to faff about getting Linux to mount it. | 08:59.58 |
| kens | I have a second disk apparently | 09:00.10 |
| | But it looks like home is not on it | 09:00.20 |
| ppd | so test results: both files are in the 0.2-0.3 seconds area | 09:00.24 |
| kens | That seems unreasonably quick..... | 09:00.47 |
| ppd | that is only the netcat time | 09:01.10 |
| | it always takes 2-3 seconds afterwards to start printing | 09:01.20 |
| | but that is constant | 09:01.23 |
| chrisl | kens: that's probably just the time it takes to get the marks onto the drum, rather than to page out | 09:02.07 |
| | ppd: so, the resolution of the flattened transparency is where Ghostscript's output has the problem compared to poppler's - if we compare like-for-like, we're in good shape, yes? | 09:03.23 |
| ppd | chrisl: from 15 minutes down to 0.3 seconds seems roughly acceptable ;-) | 09:04.12 |
| chrisl | so we need to ping tkamppeter and see where we go from here, then...... | 09:05.35 |
| | ppd: and thanks again for running those tests. | 09:08.08 |
| ppd | chrisl: I thank you for your time. I almost felt bad for stealing your time at last after all those kyocera madness | 09:09.00 |
| | chrisl: so as far as I can see the resolution of the document is too high. But I can change the/a resolution from the printing settings. Is that the same we are speaking of? | 09:11.04 |
| kens | Sorry, USB crashed and took out my WiFi. | 09:13.43 |
| | Had to reboot | 09:13.50 |
| | So it seems the resolution is the key. | 09:14.04 |
| chrisl | ppd: well, sort of. Normally, ps2write (and pdfwrite) will try to keep things device independent (so resolution shouldn't matter). There are a few exceptions that come up when converting between differing imaging models (PDF does transparency, Postscript doesn't), in those cases we have to render the object, and that's where this comes from. By default we use 720dpi. | 09:14.07 |
| ppd | chrisl: ah I see. and this is "fixable" without too much effort? | 09:15.11 |
| kens | The resolution is fixable, tehre's already a parameter to set that. | 09:15.38 |
| chrisl | ppd: well, that depends on cups - from a Ghostscript point of view, it's just adding a -r300 or equivalent | 09:15.45 |
| | ppd: as kens mentioned, the ideal solution would be to always use a resolution which can be integer multiplied to get to the printer's physical resolution. | 09:16.53 |
| ppd | chrisl: doesn't sound too complex, but we'll see. I think it should be investigated because it really makes average users yell at their printers ;-) | 09:19.02 |
| chrisl | ppd: well, getting to the same sort of performance as poppler is pretty easy, but we do have the chance to get the same performance with potentially better output, which would be nice. | 09:20.44 |
| | It may be that tkamppeter will want to do the "quick fix" for 12.04, and look at the improvements in the future. | 09:21.47 |
| naveen_ | Can mupdf open pdf files via web(i.e http file url) in android? | 09:21.53 |
| Robin_Watts | naveen_: You can download a file from the browser, and then open it. | 09:24.02 |
| | But you can't start mupdf, and then enter a URL, no. | 09:24.18 |
| kens | is not having a good network day | 09:24.54 |
| naveen_ | Hi Robin, actually my file is encrypted so I want to decrypt the data and stream it to mupdf...is this possible? | 09:24.57 |
| Robin_Watts | naveen_: Not with the current android app, but you have the source, so you can apply any decryption you need to yourself. | 09:25.41 |
| ppd | chrisl: good to know. shall I open an ubuntu bug or do you contact him? | 09:26.52 |
| naveen_ | my file is AES encrypted....so is it already supported in mupdf? | 09:27.02 |
| Robin_Watts | yes. | 09:27.10 |
| naveen_ | so how do I pass the key to mupdf? | 09:27.39 |
| chrisl | ppd: I was hoping we'd catch him on here - probably a good idea to open a ubuntu bug, though. It is possible to add a CC in a ubuntu bug? | 09:28.15 |
| Robin_Watts | MuPDF supports the standard AES file encryption mechanism. If you attempt to open a file that is encrypted in this way, it will ask you for a password. | 09:28.29 |
| | If you're using a non-standard encryption, then clearly that won't work. | 09:28.47 |
| naveen_ | Can you point me to some code example that displays the encrypted pdf or to the file where I need to change the decryption logic in mupdf? | 09:31.49 |
| ppd | chrisl: I'll stick around here and when I have the time later I'll open the bug. | 09:33.30 |
| chrisl | ppd: Okay, if you can CC me on the ubuntu bug, please do - if you can't, please ping me here, or message me via the ubuntu bug tracker, with the bug ID. | 09:36.46 |
| Robin_Watts | naveen_: Urm... | 09:37.31 |
| | I don't really follow the issue here. Can you make an example PDF available please? | 09:37.55 |
| naveen_ | actually my encrypted pdf will have a custom header(512 bytes) followed by the actual pdf content encrypted using AES | 09:39.11 |
| Robin_Watts | Right, so it's a non-standard encryption scheme. | 09:39.39 |
| | so we have no code to cope with that. | 09:40.11 |
| | And it doesn't sound trivial either. | 09:40.25 |
| naveen_ | If I remove the first 512 bytes the remaining file is encrypted using standard AES algorithm... | 09:41.06 |
| Robin_Watts | In order to read PDF files, we need to be able to seek to arbitrary points in the document and read data. | 09:41.12 |
| | AIUI, you can't do that with AES. You need to decode from the start to get to a particular part. | 09:41.35 |
| | Which means that in order to feed the doc to MuPDF you need to decode the file (either in memory or on disc) in its entirety. | 09:42.03 |
| | So MUPDF will only see the unencrypted file. | 09:42.20 |
| | Does that make sense? | 09:42.22 |
| naveen_ | yes that makes perfect sense.. | 09:42.34 |
| | So,I'm lloking of a way to send the decoded data to mupdf | 09:43.04 |
| | from memory.. | 09:43.15 |
| Robin_Watts | OK. We have functions like: pdf_open_document (where you pass it a filename) | 09:45.23 |
| | And we have functions like pdf_open_document_with_stream (where you pass it a stream handle). | 09:45.44 |
| | The latter should do what you want. | 09:45.49 |
| | The problem is we have wrapped the former up through the 'document' interface (fz_open_document), but not the latter yet. | 09:46.11 |
| | It should be pretty easy for you do extend the document interface to offer fz_open_document_with_stream. | 09:46.32 |
| naveen_ | thanks for the help Robin...I'll try to do it and revert back if I face any issues. | 09:48.46 |
| Robin_Watts | naveen_: Cool. Keep us up to date with how you get on. | 09:49.05 |
| | Can I ask, is this for a commercial project? | 09:49.13 |
| naveen_ | sure.. | 09:49.14 |
| | yes... | 09:49.23 |
| Robin_Watts | You probably know this, but I'll say it anyway... MuPDF is released both under the GNU GPL and the Artifex commercial license. | 09:50.00 |
| | If you use MuPDF under the GNU GPL license then any code you link with it becomes GPL licensed too (i.e. it must be released freely). | 09:50.34 |
| | So for a commercial project you will need to sign up to the Artifex Commercial License. | 09:51.03 |
| naveen_ | yes..I know this...I'm trying to evaluate if muPDF serves our purpose...If it serves we'll go for Artifex commercial license | 09:51.18 |
| Robin_Watts | Fab. Just wanted to be sure it was clear :) | 09:51.30 |
| naveen_ | Do you have any pricing sheet for different licensing available? | 09:52.07 |
| Robin_Watts | naveen_: I'm just an engineer, so no. | 09:52.20 |
| naveen_ | :) | 09:52.28 |
| Robin_Watts | I can put you in contact with Scott though if you want. | 09:52.33 |
| naveen_ | ok..I'll lask you once I successfully integrate mupdf into my app..thanks anyways... | 09:53.25 |
| ppd | chrisl: I hope I subscribed the right chrisl to the bug report. | 10:15.38 |
| | chrisl: bug nr. is #977912 | 10:20.27 |
| chrisl | ppd: I think you got the wrong chrisl..... on launchpad I'm "cliddell" | 10:22.45 |
| | ppd: I've subscribed myself, and added a further comment - we'll see what Till has to say..... | 10:32.51 |
| kens | Well that was an annoying way to spend 90 minutes. Manually deleting my old VMWare installation so that the installer would put a new one on. | 11:25.35 |
| | I really hate the Windows access control list :-( | 11:25.47 |
| chrisl | I wonder why it couldn't update | 11:27.01 |
| kens | It wanted to uninstall the old installation, but I'd deleted the uninstaller | 11:27.19 |
| | Which was stored in appdata/Temp, which is why I'd deleted it..... | 11:27.33 |
| | Talk about a stupid place to store somethign important | 11:27.47 |
| chrisl | Hmm, that sounds very like a bug, to me! | 11:27.52 |
| kens | Well I finally managed to delete the Program Files/VMware directory, that was the har d bit. | 11:28.24 |
| Robin_Watts | "i'm still clueless" - you don't say... | 11:36.32 |
| kens | :-) I saw that too | 11:36.47 |
| | Oh boy, now I have to restart WIndows again, looks like the installation worked though | 11:37.12 |
| alexcher | Can somebody ping casper.ghostscript.com ? | 12:55.50 |
| Robin_Watts | pinging OK for me. | 12:56.08 |
| kens | Me too | 12:56.22 |
| Robin_Watts | Reply from 184.73.189.105: bytes=32 time=95ms TTL=44 | 12:56.28 |
| | I know a great UDP joke, but you might not get it. | 13:45.51 |
| kens | Ho ho.... | 13:53.22 |
| Robin_Watts | Morning mvrhel. | 14:47.52 |
| henrys | Robin_Watts:seeing code like gxpcmap.c:1326 makes me a little nervous, was that intentional? | 14:48.12 |
| Robin_Watts | Did I do that? | 14:49.19 |
| henrys | I don't know if git blame is screwy it has you for the // rc_dec and mvrhel for // gs_free ... | 14:50.03 |
| | ahh maybe white space? | 14:50.12 |
| Robin_Watts | http://git.ghostscript.com/?p=ghostpdl.git;a=commitdiff;h=128650aac51fcf723428b8a902c4e3af7d1d058f;hp=6780bf7996f1d5a0be5b0dc55e89ea48bf89980e | 14:51.14 |
| | Looks like I did change it. | 14:51.18 |
| | And it looks like that chunk is the only non whitespace change. | 14:52.41 |
| henrys | I don't see how adev would be freed. | 14:52.59 |
| Robin_Watts | In the 'uses_transparency' case, it won't be freed there, but in the non uses_transparency case, it will. | 14:53.44 |
| henrys | we'd need at least one of those uncommented to get int freed right? | 14:53.47 |
| Robin_Watts | Actually, no. | 14:53.54 |
| | adev is 'closed' earlier. | 14:54.04 |
| henrys | ho okay | 14:54.18 |
| Robin_Watts | It's closed in both branches of the if. (But I'll be the first to admit that I was fumbling here) | 14:54.45 |
| henrys | oh okay, can we delete the code? it's confusing reading stuff like that. | 14:54.57 |
| Robin_Watts | Yes, sure. | 14:55.11 |
| | It bothers me that we close adev, then access into it to get the bitmap_memory etc. | 14:55.30 |
| | and the transbuf. | 14:55.36 |
| | Can we safely move the close of adev to be after the if is_clist thing? | 14:56.23 |
| | If we can, that would seem safer. | 14:56.30 |
| | And then we can move it out of the if entirely (and lose the second branch). | 14:56.42 |
| henrys | I have not idea what we can safely do in this code and I don't think anybody else does either, but that seems reasonable. | 14:57.29 |
| | s/not/no | 14:58.07 |
| Robin_Watts | Do you want me to try that ? | 14:58.30 |
| | or shall I leave it to you? (if you're in this area anyway) | 14:58.51 |
| henrys | yes I'll fool with it. | 14:59.13 |
| Robin_Watts | And I'd move the close_device on the saved->device to be after the if (is_clist) too. | 15:00.48 |
| | Oh, unless... | 15:01.25 |
| | Damn. Maybe there is a reason for it being this way around. | 15:01.36 |
| henrys | now closing the device does not free it. Where is the device struct being freed? | 15:02.29 |
| Robin_Watts | I don't know. | 15:02.41 |
| | I'd need to step through in the debugger and watch the refcount. | 15:02.58 |
| henrys | I guess kens will find out ;-) | 15:03.12 |
| kens | Eventually.... | 15:03.25 |
| Robin_Watts | I'll have a quick look. | 15:03.35 |
| henrys | with goto fail: it will be freed. | 15:05.03 |
| Robin_Watts | Ok, so it's alloced with an rc of 0. That becomes an rc of 1 in the dev_proc(adev, open_device) call, when it retains itself. | 15:10.25 |
| | The gs_setdevice_no_init then bumps the rc to 2. | 15:11.42 |
| | Ah... | 15:12.34 |
| | When we do gs_setdevice_no_init, we're making adev the 'saved' device (and that bumps the rc) | 15:13.58 |
| | But by the time we get to the gx_device_retain(saved->device, false), saved->device is now the pdf14 device, and the adev's rc has not been decemented. | 15:14.48 |
| kens | I went through a lot of pain whth device refernce counts wiht PCl and pdfwrite, it should be much better than it was, but obviouslu not perfect. | 15:25.31 |
| henrys | kens:we are in the pdf transp part so I doubt you'd have changed this. | 15:29.03 |
| kens | No, not that bit.... | 15:29.25 |
| henrys | all this what looks like ad hoc code completely uncommented e.g. saved->device->is_open = true? why? | 15:30.36 |
| Robin_Watts | I may have a version that works. | 15:30.52 |
| henrys | I think it is depending on the gs_state_free(saved) to free adev as I read it. | 15:36.22 |
| ray_laptop | I went through some pain with the cust 532 code to deal with the freeing of the pattern-trans-clist parts -- avoiding premature free and making sure it wasn't leaked when the pattern was bumped from the pattern cache | 15:36.41 |
| Robin_Watts | yes. | 15:36.58 |
| | henrys^ | 15:37.02 |
| henrys | but that doesn't free adev in the tranp case. | 15:37.03 |
| | or it shouldn't | 15:37.12 |
| Robin_Watts | I know. I have a patch that makes it do so. bear with me for 10 mins. | 15:37.23 |
| henrys | okay | 15:37.31 |
| ray_laptop | is the problem with the transparent pattern in clist mode or in bitmap accum mode ? | 15:37.51 |
| henrys | ray_laptop:there are some line numbers in the logs above. | 15:38.24 |
| | we are looking at the // code in gxpcmap.c and its neighbors. | 15:39.04 |
| ray_laptop | henrys: thanks -- I found the line # in the logs. // comments are sort of a FIXME flag | 15:44.28 |
| | FWIW the cust 532 code has the rc_decrement, but not the gs_free_object | 15:47.09 |
| Robin_Watts | henrys, ray_laptop, kens: http://pastebin.com/2ahMCRpz | 15:49.10 |
| | That's what I've come up with so far. | 15:49.16 |
| | It seems to work in the transparency case. | 15:49.27 |
| kens | It looks very similar to changes I've made in the past for the same problem. | 15:50.57 |
| | Sadly, I'm not expert enough to know if this is 'correct' | 15:51.49 |
| | just going to grab a coffee quick before the meeting, brb | 15:53.09 |
| Mike_Laughton | Hey everyone. I have a couple MuPDF questions -- is this the right place? | 15:55.37 |
| mvrhel_laptop | good morning | 15:55.51 |
| kens | Mike_Laughton : its a good place.... | 15:56.08 |
| ray_laptop | Robin_Watts: that seems to make sense. So this is all just in the "first pass" where we call the PaintProc and discover that we need to 'call out' to the interpreter, right ? | 15:56.08 |
| henrys | Robin_Watts:I am trying to remember the business with zPaintProc "scheduling" a call - that's what this code is for right? paintproc < 0 case. | 15:56.27 |
| kens | Mike_Laughton : but possibly not a good teim, we're going to be conducting a meeting in 5 minutes, and the channel can be hard to follow while we do that | 15:56.37 |
| Mike_Laughton | Ken: No worries. When should I come back? | 15:56.57 |
| henrys | an hour or so. | 15:57.15 |
| mvrhel_laptop | who is working in the pattern-trans-clist stuff? I am drowning in that right now with the separation planar code. pretty much my last issue | 15:57.16 |
| | what a mess | 15:57.17 |
| kens | Mike_Laughton : almost any time, but if its a quick question, ask now, and keep your fingers crossed :-) | 15:57.18 |
| Mike_Laughton | Ken : They are never as quick as I assume they will be. :) I'll come back later. Thanks! | 15:57.55 |
| henrys | I have a customer memory leak with tile clipping but then discovered other leaks. | 15:57.59 |
| ray_laptop | henrys: yes, in the e_RemapColor case (which is for PS and PDF, not other PDL's that have C code to paint the pattern) | 15:58.01 |
| henrys | mvrhel_laptop^^^ | 15:58.07 |
| kens | Mike_Laughton : No problem | 15:58.13 |
| mvrhel_laptop | henrys: ok. yes tile_clipping is also involved in my bug | 15:58.28 |
| marcosw | mvrhel_laptop: can you do a clusterpush of your current code followed by a bmpcmp so I can test the new email that's get sent out with bmpcmp error messages? | 15:58.54 |
| ray_laptop | mvrhel_laptop: Robin_Watts was the one that did the pattern-trans-clist stuff in the trunk | 15:59.07 |
| mvrhel_laptop | oh I was just curious who was working in that area right now. what ever changes they have I want to make sure to get into my branch | 15:59.41 |
| henrys | it was originally done by Igor right? | 15:59.44 |
| kens | shudders | 15:59.59 |
| mvrhel_laptop | I added the transparency stuff which is pretty good ;) | 16:00.18 |
| | marcosw: I am at the coffee shop on my laptop right now. the branch is on my desktop so I will have to do it later (the clusterpush) | 16:00.56 |
| ray_laptop | the pattern trans clist stuff was started (and gotten to about the 80% point) by mvrhel, then I "finished' it up (for cust 532) when mvrhel went on vacation, then Robin_Watts "ported" it into HEAD | 16:01.02 |
| mvrhel_laptop | oh that is right . ray_laptop clean up all my screwups | 16:01.22 |
| Robin_Watts | yeah, I was just the last person to be holding the hammer. | 16:01.26 |
| marcosw | just ssh into your desktop and ⦠oh wait, you run windows, never mind... | 16:01.28 |
| ray_laptop | mvrhel_laptop: and ray_laptop take a step backwards ;-) | 16:01.48 |
| kens | meeting time.... | 16:02.01 |
| Robin_Watts | marcosw: http://mosh.mir.edu/ <- might interest you. | 16:02.06 |
| | marcosw: http://mosh.mit.edu/ <- might interest you. | 16:02.12 |
| henrys | I just happened to notice some oddities while looking at my memory leak - my problem is pcl so no tranpsparency (thank god) | 16:02.22 |
| ray_laptop | mvrhel_laptop: I use a product called 'radmin' which is a sort of clone of remote assistance, but allows me to change the port and other stuff to make it more secure | 16:02.56 |
| Robin_Watts | actually... my current patch ends up doing the same as the existing code, I think. | 16:03.13 |
| mvrhel_laptop | that sounds interesting | 16:03.16 |
| Robin_Watts | henrys: Did you actually spot any problems with this, or was it just "that looks odd" ? | 16:03.36 |
| ray_laptop | mvrhel_laptop: radmin has file transfer mode, remote console window mode as well as full view of the remote desktop. | 16:03.56 |
| henrys | Robin_Watts:looks odd | 16:03.57 |
| marcosw | Robin_Watts: mosh seems like a useful combination of ssh and screen. too bad there isn't a windows version for mvrhel_laptop :-) | 16:03.57 |
| Robin_Watts | marcosw: Give it time. Someone will port it to cygwin. | 16:04.17 |
| mvrhel_laptop | windows 7 has remote desktop but I have not used it | 16:04.43 |
| ray_laptop | mvrhel_laptop: the console mode is sort of like ssh, but of couse, you can't pop up a local display window | 16:04.57 |
| henrys | I didn't have much for the meeting - Robin_Watts: i did say you'd add a schedule to the gl/2 bug. | 16:04.58 |
| Robin_Watts | As in "how long it will take me to fix it"? | 16:05.19 |
| henrys | the fill business - gapto | 16:05.21 |
| ray_laptop | mvrhel_laptop: I would stay clear of the microsoft version -- it is a favorite target for hackers | 16:05.28 |
| mvrhel_laptop | yes | 16:05.33 |
| Robin_Watts | I started to look at it again today. The stroking code is killing me. | 16:05.34 |
| | henrys: In the discussion of the bugzilla thing last week, we said we'd talk about that at the meeting. | 16:06.26 |
| henrys | Robin_Watts:so say it is hard and it will likely be a long time. | 16:06.32 |
| Robin_Watts | Will do. | 16:06.39 |
| henrys | oh right bugzilla | 16:06.41 |
| Robin_Watts | http://bugs.ghostscript.com/enter_bug2.cgi?product=Ghostscript | 16:06.54 |
| | Peoples opinion of the "provenance" field. | 16:07.11 |
| henrys | I like Robin_Watts' form okay but I think "providence" is a bit snooty, but that's fairly minor | 16:07.15 |
| | sorry provenance | 16:07.26 |
| mvrhel_laptop | you need a don't know entry | 16:07.34 |
| Robin_Watts | Ok, give me a better word... | 16:07.35 |
| | mvrhel_laptop: If they don't know, we don't want the bug :) | 16:07.53 |
| mvrhel_laptop | or chinese bootleg | 16:07.59 |
| ray_laptop | Robin_Watts: "where you got Ghostscript" ?? not one word, but that's OK | 16:08.01 |
| henrys | origin | 16:08.03 |
| Robin_Watts | Sure. The actual value selected never makes it off this page. It's just used to pop up warnings. | 16:08.31 |
| | mvrhel_laptop: I think "Romford Market" is the UK equivalent to "Chinese Bootleg" | 16:09.26 |
| ray_laptop | Robin_Watts: so if they say "it was installed when I installed pdf995" we can say "sorry -- contact them, not us" (same for Oracle users, ...) | 16:11.30 |
| Robin_Watts | ray_laptop: so an option for "Installed as part of another application"? | 16:12.11 |
| ray_laptop | although realistically pdf995 users aren't usually aware enough of ghostscript to ever submit a bug | 16:12.32 |
| henrys | chrisl:are you good with this? I remember you had some thoughts about it. | 16:12.36 |
| chrisl | henrys: I figured we can try it, if it causes problems, we can change it | 16:13.00 |
| ray_laptop | Robin_Watts: maybe -- but many linux users may select this because it got pulled in when the installed cups, so it might confuse them | 16:13.57 |
| Robin_Watts | ray_laptop: Well, in that case, they probably installed from a package. | 16:14.23 |
| henrys | I guess tkamppeter will do a static build but I doubt other distro folks are going to do that for us. I'd like to stop the shared lib only problems. | 16:14.32 |
| Robin_Watts | So again, the bug report should go to the packager. | 16:14.39 |
| chrisl | henrys: we really only get reports from Till and Tim W, I'm aware of getting reports from other maintainers | 16:16.05 |
| marcosw | I still think we are solving a problem that is insignificant. How many times have we had issues that were caused by shared libraries? | 16:17.14 |
| ray_laptop | Robin_Watts: I hadn't looked at the enter_bug2 screen -- with the "-- Where did you get the software you are reporting the bug for ? --" I think Provenance: or Origin: or even Downloaded From: is OK | 16:17.32 |
| chrisl | marcosw: I kind of agree - it's mostly on here we see these come up, and changing the bug form won't help that...... | 16:18.15 |
| henrys | marcosw:probably right - I think this is more a "show of our objection to the policy" than an attempt to save us time. | 16:19.06 |
| ray_laptop | if it aggravates the linux maintainers (till and timw) enough to consider making gs without shared libraries, then that's good enough ;-) | 16:19.35 |
| kens | Till doesn't have that control | 16:19.52 |
| henrys | kens:I finally did update the pdf server bug - don't know how much it helps though. | 16:19.59 |
| kens | henrys I saw, like I said, there's an interesting Memento memory corruption to look at too. | 16:20.25 |
| | But first I have to solve this bug that only occurs on Linux. | 16:20.38 |
| | Spent most of the day wrestling with my VM. | 16:20.49 |
| henrys | I think apple did this right - apps are directories and can have their own shared libraries in the directories. Then you get compatibility and sharing. | 16:20.52 |
| kens | THat's the way Windows used to work too. | 16:21.03 |
| | Until MS decided that wasn't good enough | 16:21.11 |
| | With my latest cghanges, owl ggoes from leaking 1000 blocks to < 300. | 16:21.42 |
| | And a good chunk of those are FM pairs in a cache | 16:21.58 |
| henrys | that certainly is progress. | 16:22.00 |
| ray_laptop | kens: well, that's an improvement | 16:22.05 |
| Robin_Watts | henrys: RISC OS did it 20 years before Apple :) | 16:22.33 |
| kens | ray_laptop : yes if I can solve the crash I may stop looking at leaks and concentrate on the crash henrys found and memory corruption Memento found | 16:22.39 |
| | Robin_Watts : Memento doesn't find these double frees for me. | 16:22.55 |
| henrys | Robin_Watts:oh I didn't know that but I'm not surprised. | 16:22.56 |
| kens | Not that I'm complaining | 16:23.08 |
| ray_laptop | kens: is the font cache actually a leak in the server mode ? i.e., isn't it retained for the next "job/page" | 16:23.18 |
| kens | ray_laptop : I don't beleive its truly a leak. | 16:23.32 |
| Robin_Watts | kens: Can you give me a command that shows them up please? | 16:23.42 |
| kens | Its just that we don't free the cache on exit | 16:23.44 |
| henrys | font cache shouldn't be a leak. It has a fixed max size. | 16:23.45 |
| kens | Robin_Watts : Not easily, you would need my code. | 16:23.55 |
| Robin_Watts | fair enough. | 16:24.03 |
| marcosw | did windows ever solve the problem that shared libraries (dlls) can't be removed when you uninstall an app since it's not known if any apps that were installed later expect that shared library to exist (i.e. reference counting)? | 16:24.09 |
| kens | henrys ray_laptop exactly so. The cache gets entrioes freed, but is not totally cleared before exit | 16:24.15 |
| ray_laptop | kens: not freeing the cache is OK as long as it is still available and used once we re-open | 16:24.32 |
| kens | marcosw : It just wanrs you these days | 16:24.32 |
| | ray_laptop : yes, which is why I'm not worried about those ones ;-) | 16:24.44 |
| henrys | kens:I could add that to PCL upon shutdown but it seems rather pointless. | 16:24.59 |
| kens | henrys I'm not at all worried by it. Iy just takes time to sift through the Memento warnings | 16:25.21 |
| henrys | anybody else have something for the meeting? | 16:25.40 |
| kens | I'm not sure it is a PCL cache either | 16:25.41 |
| ray_laptop | henrys: I agree -- when we 'release' the memory allocator, all those blocks are freed anyway | 16:25.45 |
| Robin_Watts | henrys: If it was trivial to do, then it would be worth it - if only so we can see that memento exits cleanly. | 16:26.26 |
| kens | Robin_Watts : there are so many one-time allocations, I don't think another matters | 16:26.47 |
| Robin_Watts | (make it #ifdef MEMENTO if you're worried about normal builds taking a speed hit) | 16:26.53 |
| henrys | kens:what I saw with -Z? was pdf cache elements and you were trying to access those after being freed. Are we dumping that cache each page and expecting the characters to be there across page boundaries? | 16:27.13 |
| kens | henrys I haven't seen that problem, but I haven't looked at it yet. | 16:27.33 |
| Robin_Watts | kens: Oh, it's a single one ? I thought it was a set of leaked blocks (one per glyph in the cache or something) | 16:27.37 |
| kens | The pdfwrite cache should be reset on open. | 16:27.43 |
| | Robin_Watts : its a lot of allocatio0ns, all stored in the cache | 16:27.56 |
| Robin_Watts | Right, so freeing them in the MEMENTO case would save a lot of noise, right ? | 16:28.24 |
| kens | It should also persist across pages (but not open/close device) | 16:28.29 |
| Robin_Watts | It's not just "one more". | 16:28.32 |
| kens | Robin_Watts : yes it would get ird of the noise, but there is a lot of that already | 16:28.47 |
| Robin_Watts | kens: Well, we should reduce it (IMHO), otherwise we'll all go deaf. | 16:29.21 |
| kens | Also, these are elements which are *not* exercised in the normal PCL case, only when running with pdfwrite | 16:29.22 |
| | Wihtout pdfwite the sa,e job comes in at about 150 if I remember | 16:29.45 |
| | correclty | 16:29.51 |
| | back in a moment | 16:31.00 |
| henrys | so meeting adjourned for those that want to head out. | 16:31.19 |
| Robin_Watts | Updated version: http://bugs.ghostscript.com/enter_bug2.cgi?product=Ghostscript | 16:33.19 |
| marcosw | just a quick mention that I've made some changes to the email that the cluster sends out when running bmpcmp. It now reports errors generated by bmpcmp and a list of those file that didn't have any differences (based on the tolerance set by the bmpcmp command). Robin_Watts is going to modify the web page generation code so this information will also be available on the web page. | 16:33.23 |
| kens | Robin_Watts : I'm happy enough with that, or the prior version, both are fine by me | 16:34.33 |
| henrys | oh cool that does remind me I noticed the "delta" is broken. | 16:35.20 |
| Robin_Watts | oh yes... | 16:35.30 |
| | And I need to make deltas work with mupdf too. | 16:35.40 |
| ray_laptop | kens: for performance, shouldn't the cache be used for subsequent pages when we are doing the -o out-%d.pdf thing ? | 16:36.54 |
| kens | ray_laptop : Yes, but since we close the device, I'm not sure how that would work (this is the pdfwrite cache not the graphics library) | 16:37.30 |
| ray_laptop | kens: Oh, I thought you were talking earlier about the fm_pairs (font cache). I don't know what the pdfwrite cache is | 16:38.12 |
| | sorry | 16:38.15 |
| kens | ray_laptop : that's the one that 'leaks' with PCL but isn't a problem | 16:38.31 |
| henrys | Robin_Watts:I'm good with the bugzilla form. | 16:38.32 |
| kens | ray_laptop : the other one is related to henrys crash | 16:38.43 |
| ray_laptop | kens: just curious -- what is the "pdfwrite cache" for ? | 16:39.11 |
| kens | ray_laptop : off the top of my head, no idea. | 16:39.25 |
| | But I suspect it is holding the type 3 bitmaps for glyphs | 16:39.46 |
| ray_laptop | kens: OK, nevermind. If I get more curious, I'll look at the sources | 16:39.54 |
| kens | :-) | 16:40.00 |
| | I will be able to answer the question after I fix the crash :-) | 16:40.15 |
| henrys | kens:it seems we went through this before but it would seems pcl character would never be in the graphics library cache, you need a type42 outline, not a rendered bitmap. | 16:41.10 |
| kens | henrys that only works for TrueType fonts. | 16:41.33 |
| | Not for other types of PCL fonts | 16:41.39 |
| | The FM pairs cache is only sued for TT fonts. | 16:41.55 |
| marcosw | mvrhel_laptop: I just received another email from the customer re. http://bugs.ghostscript.com/show_bug.cgi?id=692618 and http://bugs.ghostscript.com/show_bug.cgi?id=692961 I know you are actively working on these, but any progress that I can report to the customer would be appreciated. | 16:42.23 |
| kens | PCL can have bitmap fonts, and the stick font which is also a type 3 | 16:42.25 |
| henrys | but almos everything in owl is TT so I assumed they were being cached you said 300 entries? | 16:43.08 |
| kens | henrys that's what's left. | 16:43.19 |
| | But half of those are probably PCL one-time allocations. | 16:43.40 |
| | THe FM pairs are not so many | 16:43.48 |
| ray_laptop | kens: are you talking about the pdf_font_cache_elem_s things ? Those look like they are for the fonts themselves, not the glyphs | 16:43.54 |
| kens | Those do result in FM pairs in the cache though | 16:43.59 |
| mvrhel_laptop | marcosw_: I am *very* close to being done with these | 16:44.08 |
| | I have basically 2 files on the regression testing that are wrong | 16:44.20 |
| kens | ray_laptop : I don't know which structure henrys means, because I haven't debugged it yet. | 16:44.25 |
| | But yes, those are fonts. | 16:44.37 |
| mvrhel_laptop | the issue is that they are files with issues in the pattern-trans filling area | 16:44.43 |
| | one is even a cairo file | 16:44.52 |
| ray_laptop | good old cairo. | 16:45.12 |
| marcosw | mvrhel_laptop: thanks. I'll let the customer know that you've made progress and are close. | 16:45.34 |
| mvrhel_laptop | It is going to be at least another week judging from the way this is looking. the branch does render the customer files fine | 16:45.34 |
| henrys | kens:I'll go ahead and add some code to empty the cache to make things easier. It's very easy. | 16:45.39 |
| kens | henrys, well if its no trouble then please do, but its not a big deal. | 16:46.03 |
| ray_laptop | kens: BTW, I saw the discussion you and chrisl had with ppd earlier about the kyocera printer performance. | 16:46.07 |
| henrys | Robin_Watts:is there some reason memento doesn't output the client name from the allocation? | 16:46.19 |
| mvrhel_laptop | marcosw: this is a pretty big change in the code. are you going to just give them the current trunk after I commit? | 16:46.21 |
| kens | Most of the leakage (600 blocks) is in patterns which is what I'm chasing now | 16:46.22 |
| Robin_Watts | henrys: client name ? | 16:46.34 |
| kens | ray_laptop : yes, it is because of transparency and the default resolution | 16:46.41 |
| Robin_Watts | Memento works at the malloc/free/realloc level. No client names there. | 16:46.55 |
| ray_laptop | mvrhel_laptop: for cust 393, IIRC, the usually want binaries from us | 16:46.58 |
| kens | 720 dpi is not a good resolution for a printer which does 300/600/1200 | 16:46.59 |
| mvrhel_laptop | so we will want to bump up the client color setting for them too then | 16:47.22 |
| | probably to 20 based upon the one file with 18 colors | 16:47.39 |
| chrisl | ray_laptop: I was rather hoping the results from that discussion would take some of the pressure off the performance bug on your list - but "nice gent" on there doesn't seem satisfied :-( | 16:47.55 |
| ray_laptop | kens: from my internal knowledge, their code and ASIC handle image up-scaling, but the downscaling is highly inefficient (they sort of assume that people won't do that) | 16:48.08 |
| kens | ray_laptop : That's probably what it is. | 16:48.22 |
| Robin_Watts | henrys: If there are areas where you DO have a client name available, then you can change the code to call Memento_label(malloc(x), name); rather than malloc(x) and the blocks will be labelled (and Memento_label goes away in release builds) | 16:48.29 |
| kens | I ssupect the printer was running in 300 or 600 dpi mode | 16:48.32 |
| | ray_laptop : so down scaling 720 dpi images seemed to be a big penalty | 16:48.53 |
| marcosw | mvrhel_laptop: I hadn't thought about it, but I think so. Generating a patch is likely to cause more problems than having them use trunk; I don't believe there are any regressions in the current code but maybe I'll send an email to tech to make sure no one has any objections. | 16:48.55 |
| ray_laptop | they should be able to just add << HWResolution [ 300 300 ] >> setpagedevice to the PPD | 16:48.57 |
| kens | ray_laptop : THe image is *created* at 720 dpi by ps2write | 16:49.14 |
| | ray_laptop : from a full-page transparent PDF file | 16:49.30 |
| ray_laptop | kens: yes, but ps2write respects HWResolution doesn't it ? | 16:49.41 |
| kens | ray_laptop : yes, and -r too :-) | 16:49.52 |
| ray_laptop | (the PS / PPD equivalent to -r300) | 16:49.55 |
| kens | chrisl created a file to test by doing exactly that | 16:50.18 |
| | (-r300) | 16:50.23 |
| | And itt ran very quickly | 16:50.33 |
| | Demonstrating that the resolution was the problem. | 16:50.44 |
| ray_laptop | kens: right, that's what I saw in the logs/ | 16:50.48 |
| | good work tracking that down/ | 16:51.00 |
| henrys | Robin_Watts:well every allocation routine in the system is required to have a client name for debugging it is unfortunate that isn't transmitted down to memento somehow. I haven't read the memento code carefully so I'm probably missing something. | 16:51.09 |
| kens | And that if you are going to cmpare ps2write with poppler, its important to test the *same* things, as far as possislbe | 16:51.09 |
| Robin_Watts | henrys: As I say, Memento works at the malloc/free/realloc level, not the gs_alloc_bytes level. | 16:51.35 |
| | hence it doesn't have the names available to it. | 16:51.50 |
| chrisl | kens, ray_laptop: FWIW, it looks like the new cairo code is doing a much better job with PDF to PS conversions, according to: Bug 692959 | 16:52.21 |
| kens | Quitting time, night all. | 16:54.22 |
| mvrhel_laptop | I need head out for bit. bbiaw | 16:56.48 |
| Robin_Watts | henrys: OK, I think I've got a fix that should get us a lot more block labelling. | 17:04.32 |
| ray_laptop | chrisl: I added a comment to bug 692959 about getting K only text and trying to explain more fully about why transparency needs a full page. | 17:06.57 |
| chrisl | ray_laptop: okay, thanks - I just lost patience, and didn't feel I could (or wanted to) answer politely at that stage. FWIW, I think we *could* do better with some of these cases. | 17:08.04 |
| ray_laptop | chrisl: I think it would be a ball of snakes. We'd have to collect the ps2write "high level" output AND render the area that used transparency, then it certain cases, overlay the rendered image rectangles. | 17:10.35 |
| chrisl | ray_laptop: true, *but* since pdfwrite might end up having to do something similar to handle PCL input with ROPs, it's worth baring it in mind, in case that work does get forced on us | 17:12.03 |
| ray_laptop | chrisl: and as I mentioned in the comment, if objects cross the boundary, the chance that our rendered image will "match up" with whatever the RIP in the printer does is impossible | 17:12.16 |
| chrisl | ray_laptop: that is true, but it is what other flatteners do - at least one Adobe product does that. | 17:13.02 |
| ray_laptop | chrisl: IMHO, for PCL with ROPs we should just do a full page image with "invisble" text overlay (for searchability) | 17:13.26 |
| Robin_Watts | Say it quick and it won't give kens nightmares :) | 17:13.53 |
| ray_laptop | chrisl: I can't imagine that it is anything that really works well -- you have to render the transparent rectangles at some pre-defined resolution, but the rest of the PDF gets rendered at some (possibly/probably) different resolution | 17:15.23 |
| | Robin_Watts: note that I waited until after kens left to mention it ;-) | 17:15.56 |
| chrisl | ray_laptop: yeh, it does end up being resolution dependent, but then the only reason to convert PDF to PS is to send to a printer, so....... | 17:16.27 |
| ray_laptop | chrisl: do you know if Adobe does the region flattening to image if they are exporting a file with transparency to a PDF 1.3 or PDF/A1 ? | 17:17.35 |
| | chrisl: because PDF's are in general intended to be resolution/device independent | 17:18.09 |
| chrisl | ray_laptop: I don't know. I don't even know if a *current* Adobe product does that - it was an issue I investigated in Jaws a few years ago. | 17:18.49 |
| ray_laptop | maybe what I should do is put up a FAQ "Why do I get a full page image when converting PDF to PS or PDF/A1 ?" | 17:20.20 |
| Mike_Laughton | Hi everyone, is this a good time for a few MuPDF questions? (it was just one question, but the list grew during your meeting :) | 17:21.10 |
| chrisl | ray_laptop: Acrobat gives an error trying to save Ausgabe.pdf to PDF/A | 17:21.15 |
| ray_laptop | chrisl: well, at least we are better than them :-) | 17:21.39 |
| | Mike_Laughton: Robin_Watts is probably the one that will have your answers (Tor is not around) | 17:22.10 |
| Robin_Watts | Mike_Laughton: Go for it. | 17:22.23 |
| Mike_Laughton | Cool, I'll fire away. My company is evaluating MuPDF, and I was asked to make a modification based on a suggestion made to my colleague by Robin Watts... | 17:22.50 |
| | But first, I wanted to mention that I have been looking over the MuPDF code this morning and I'm really impressed with how well organized everything is. Very good first impression. | 17:23.02 |
| | *starts with a very important first question* Is it pronounced MEW-Pee-Dee-Eff or Micro-Pee-Dee-Eff? | 17:23.14 |
| ray_laptop | MEW | 17:23.41 |
| Robin_Watts | The former. But with a swedish accent. | 17:23.42 |
| | (bork bork bork) | 17:23.57 |
| Mike_Laughton | Perfect, I had the accent right. (50% swedish anyway) :) | 17:24.07 |
| henrys | bbiab | 17:24.19 |
| Mike_Laughton | Next question, a little harder. I added fz_open_document_with_stream() to match the existing fz_open_document() function. Mine looks like this: | 17:24.28 |
| | fz_document *fz_open_document_with_stream(fz_stream *file, const char *ext); | 17:24.34 |
| | 1) Would it be better if I pass fz_context* as a first parameter? and... | 17:24.59 |
| | 2) Is there something better as the final parameter, like an existing enum type, to indicate pdf v. xps v. cbz? | 17:25.03 |
| chrisl | ray_laptop: and Acrobat's export to PS also flattens the entire group (in this case the page group) to an image. | 17:25.21 |
| ray_laptop | chrisl: thanks for checking. That makes more sense. | 17:25.51 |
| Robin_Watts | 1) The fz_stream has a context baked in, and that is the one that's used. So better to not pass an unused context, I feel. | 17:26.15 |
| chrisl | ray_laptop: I have a *feeling* the product that did the partial flattening was a by-product of when CPSI had to "pre-flattened" transparency | 17:26.43 |
| Robin_Watts | 2) There is currently no enum, so an extension is as good as anything at this stage. | 17:26.52 |
| | Post 1.0 we may well want to look into doing the same kind of thing. | 17:27.19 |
| Mike_Laughton | Okay great, thanks. Next one is the hard one... | 17:27.36 |
| ray_laptop | chrisl: for CPSI, they then pass it to their own rendering engine, so that would avoid the "other RIP/other resolution" issues. | 17:27.42 |
| Mike_Laughton | Does the fz_stream implementation in rc1 support memory streams? I see that read_buffer() has a single line "return 0;" which implies no, but maybe I'm missing something obvious. | 17:27.52 |
| Robin_Watts | And we'll want to cope with extension/mimetype and auto detection - but none of that is completely trivial, so rather than rush it, we've left it for now. | 17:27.55 |
| | Yes, memory based streams are supported. | 17:28.38 |
| chrisl | ray_laptop: sure, which is why I'm not sure how we ended with the the "interim" form reaching Jaws - it was even dafter, as it didn't have fonts embedded, so the glyph render was *never* going to match! | 17:29.05 |
| Mike_Laughton | Okay, so the 'return 0' is just because there's no work required to populate the buffer | 17:29.23 |
| Robin_Watts | fz_open_memory is the one you want | 17:29.25 |
| | indeed. | 17:29.27 |
| ray_laptop | chrisl: I agree that sounds like you were asking for trouble reports | 17:29.47 |
| Mike_Laughton | Okay, perfect. I may need to modify the library slightly to support our specific use case, but we'd be purchasing a commercial license if we go down this path anyway so it would be legit license-wise | 17:30.34 |
| chrisl | ray_laptop: it was a very shouty and stubborn customer, too - reported it four times before we convinced them it wasn't our fault :-( | 17:31.00 |
| ray_laptop | sort of like some of our customers -- cust 130 or 580 come to mind | 17:32.07 |
| chrisl | They're everywhere! | 17:32.23 |
| Robin_Watts | Mike_Laughton: Fab. If you spot things we could do better, please let us know. | 17:32.24 |
| Mike_Laughton | So far it looks great. I maintain my own open source C library, so I understand what it takes to achieve the level you guys have here. (not that I have achieved it myself, I at least understand) :) | 17:33.18 |
| | Thanks for your help, Robin! | 17:33.23 |
| ray_laptop | Mike_Laughton: what open source project are you maintaining ? | 17:34.22 |
| | Mike_Laughton: just curious | 17:34.33 |
| Robin_Watts | no worries. Any more questions, just ask here. I'm UK based, but I'm always on here (and read the logs when I've been away). | 17:34.42 |
| Mike_Laughton | ray_laptop: It's a little barcode library called libdmtx | 17:34.56 |
| | ray_laptop: It reads and writes Data Matrix barcodes. I now have 2 kids under 3 years old though, so I have spent about as much time working on it this year as I have been in this IRC channel ;) | 17:36.17 |
| ray_laptop | Robin_Watts: I looked a fz_open_memory. We may want to consider a 'list of data blocks". The access methods aren't too bad/slow and the convenience for someone loading a PDF from a network/comm stream sometimes doesn't know the length to allocate as blocks are received | 17:37.30 |
| | Mike_Laughton: thanks | 17:37.50 |
| Robin_Watts | ray_laptop: Indeed. We are not limited to a single implementation; stream is a interface we can implement in many different ways. | 17:38.23 |
| ray_laptop | Robin_Watts: that's what the ghostscript customers I've talked to have wanted, and that's what cust 532 uses. | 17:38.24 |
| Robin_Watts | We could easily offer a stream of compressed blocks. | 17:39.00 |
| ray_laptop | Robin_Watts: well with PDF, (and XPS) compressed blocks usually don't save much since the data is usually already Flate or otherwise compressed | 17:41.49 |
| Robin_Watts | true. | 17:42.12 |
| ray_laptop | Robin_Watts: we'd definitely want to make compression optional | 17:42.12 |
| | now, for muPS we'd probably want compression ;-) | 17:43.12 |
| | don't tell Tor I even mentioned that front end parser !!!! | 17:43.53 |
| | marcos: (for the logs) I tried to gently push cust 393 to start doing their own builds | 17:54.43 |
| | chrisl: see previous comment | 17:55.18 |
| | at the very least we should make them wait until mvrhel's changes are in, otherwise they'll want another version | 17:56.12 |
| chrisl | ray_laptop: and the changes I need to make for the missing pages - should only be an extra day or two | 17:56.52 |
| ray_laptop | chrisl: OK -- it'll probably be you or I that does the build for them, so I'll bear that in mind. | 17:57.55 |
| | Maybe we need an extra charge for each outside release binary (priced slightly higher than the cost of Visual Studio Professional) to encourage folks more. | 17:58.52 |
| chrisl | ray_laptop: tbh, if they were up for learning to do their own builds, I'd be willing to drive up there and walk them through it - but they haven't shown any inclination. We could force the issue by just not doing it. | 17:59.43 |
| ray_laptop | Visual Studio Pro download from mychoicesoftware is only $479. Thus a binary should cost $500 ro $1000 | 18:02.51 |
| Robin_Watts | VS 2010 Pro costs £539 in the UK :( | 18:03.59 |
| tkamppeter | chrisl, kens, I have released a new cups-filters package (1.0.14) and uploaded it to Precise. Here the pdftops filter calls Ghostscript with the " -r..." option, supplying the printing resolution, determined from the PPD and the job options. This way images and transparency flattening should be done with a resolution to be faster processed by the printer. | 18:04.46 |
| ray_laptop | Robin_Watts: if it's a download, can't you buy it for the $479 price ? | 18:05.37 |
| Robin_Watts | ray_laptop: Probably. | 18:05.50 |
| | The academic edition is "just" 300 quid. | 18:06.01 |
| ray_laptop | Robin_Watts: The US price is $179 | 18:06.22 |
| chrisl | tkamppeter: that might not help if the Kyo printer's PPD says it's 1200dpi........ | 18:06.34 |
| ray_laptop | chrisl: tkamppeter: or 600x1800 | 18:07.11 |
| | the printer default mode is 600x1800 | 18:07.40 |
| Robin_Watts | Actually, £334 for a boxed copy. | 18:07.50 |
| tkamppeter | chrisl, I understood that the problem is the mismatch of 720 dpi and 1200 dpi not the high number of 720 dpi. | 18:07.50 |
| ray_laptop | tkamppeter: I didn't see that 1200 dpi was ever tested. | 18:08.27 |
| tkamppeter | chrisl, ray_laptop, for assymetric resolutions I use the smaller figure, so -r600 in this case. | 18:08.29 |
| chrisl | tkamppeter: I don't know, I just had ppd tested the 300dpi that matched the old poppler filter. | 18:08.39 |
| ray_laptop | the printer's lowest resolution is 600 dpi, so why is poppler generating 300 dpi ? | 18:09.07 |
| chrisl | ray_laptop: it didn't allow it to be changed | 18:09.27 |
| ray_laptop | chrisl: that sounds unfriendly | 18:09.58 |
| tkamppeter | ray_laptop, this is poppler's default if one does not supply a resolution, and supplying resolutions is even only possible with a very new Poppler. Under Oneiric it was not possible yet, but under Precise it is. | 18:10.13 |
| | ray_laptop, I have done upstream cups-filters in a way that if one uses Poppler-based pdftops, the Poppler version is appropriately auto-detected by ./configure. | 18:11.09 |
| chrisl | ray_laptop, tkamppeter: given that the complaint is the GS generated output is slower than the poppler generated output on the printer, I wanted to compare like for like - i.e. 300 dpi | 18:11.25 |
| ray_laptop | have to run lunch to my boy. bbiaw. | 18:24.56 |
| scott-san | Hi Ray, are you on? | 18:42.25 |
| henrys | kens:for the logs - I think the cache freeing code will be a problem... actually I took it out exactly to support pdfwrite - which accesses font cache data when the device is closed. The way things are set up it is difficult to get to the font cache that late (device close time). | 18:43.39 |
| Robin_Watts | Hey Scott. | 18:43.46 |
| | ray has just popped out to run lunch to his son. | 18:44.07 |
| henrys | scott-san:he just went to lunch | 18:44.09 |
| | scott-san:what Robin_Watts said. | 18:44.33 |
| scott-san | No problem, basically looking for Marcos | 18:44.50 |
| Robin_Watts | Is marcos still in Germany? | 18:45.41 |
| henrys | scott-san if you type "/names" it will list who is here. | 18:46.03 |
| | or there might be a list on your irc client | 18:46.24 |
| | Robin_Watts:so now kens will be able to see the client names for his font cache entries with that change? | 18:47.11 |
| Robin_Watts | I hope so. | 18:47.26 |
| henrys | I though with that he could just grep them away. | 18:47.28 |
| scott-san | I did that Henry. Just thought someone might know the whereabouts of Marcos | 18:47.40 |
| Robin_Watts | Unless he gets told they are all chunks. | 18:47.41 |
| | scott-san: He has been in Germany (at a conference maybe) | 18:47.57 |
| henrys | scott-san:he was here just a bit ago an hour or so. | 18:48.21 |
| scott-san | Alright, I'll send him another email. Thanks gents. | 18:48.49 |
| Robin_Watts | marcosw: scott was just on looking for you. | 19:23.53 |
| | Are you still in Germany? | 19:23.57 |
| marcosw | Robin_Watts: no, I'm back home. Did Scott say what he wanted? | 19:24.15 |
| Robin_Watts | He said he'd send you "another mail". | 19:24.37 |
| marcosw | thanks. i"ll check. | 19:24.48 |
| | Robin_Watts: the bmpcmp you are running is producing a lot of errors. Is that expected or did I break something in the cluster code? | 19:32.08 |
| Robin_Watts | Probably expected :( | 19:32.28 |
| | Yes, expected. | 19:32.46 |
| marcosw | It appears that the baseline vs. your clusterpush output differs in the number of pages in the output ppm file for many files. Unfortunately the output message is cryptic: Failed to load image 16 from '/dev/fd/62' | 19:39.59 |
| Robin_Watts | henrys: OK, the stroking stuff seems less broken now. | 19:52.08 |
| | will keep looking tomorrow. | 19:52.16 |
| | Forward 1 day (to 2012/04/11)>>> | |
IRC Logs