| | <<<Back 1 day (to 2015/07/06) | 20150707 |
| rayjj | Robin_Watts: my git state is confused. It doesn't help that git logg shows a whole raft of 'device_subclassing' commits. I have a branch that seems to be based (according to git logg to be based on: 7e8cd8f Commit f5444527 inadvertently introduced seg faults, fixed here. | 00:19.32 |
| | Robin_Watts: I'd like it to be based on origin/master and track that. | 00:20.05 |
| | is this possible or should I just grab the patches (format-patch) from the current base and apply them to a fresh branch | 00:21.08 |
| | usually this 'just works' so that the branch tracks my 'master' which tracks origin/master, so all I need to do is update my master and then the branch | 00:21.56 |
| henrys | rayjj: for the logs so you did 'git checkout master' and the branch does not change to master? What does 'git branch' say? Probably not understanding what you are asking. | 03:44.27 |
| felix___ | mupdf question here | 12:20.31 |
| | page->mediabox seems to take into consideration the cropbox as well | 12:20.55 |
| | however not the trimbox | 12:21.07 |
| | is there are reason behind this? | 12:21.36 |
| | (I might have an incomplete understanding of page boxes) | 12:22.54 |
| Robin_Watts | felix___: Let me check the source. | 12:23.42 |
| felix___ | cool, I am looking at pdf/pdf-page.c | 12:24.38 |
| Robin_Watts | felix___: Indeed, you are right. We look at mediabox and cropbox. | 12:25.07 |
| chrisl | I would suggest reading what the PDFRM has to say..... | 12:25.25 |
| Robin_Watts | felix___: There is no "one size fits all" solution here. | 12:27.58 |
| | For different needs, different sized boxes will be wante.d | 12:28.09 |
| | (i.e. for proofing, you want bleed and trim boxes left intact) | 12:28.32 |
| felix___ | yeah makes sense | 12:28.39 |
| Robin_Watts | Possibly there should be an option you can set to tell mupdf which boxes to respect. | 12:29.00 |
| felix___ | or if you want the pdf to be printed | 12:29.01 |
| chrisl | The meanings of the various *Boxes are clearly explained in the reference manual | 12:29.46 |
| Robin_Watts | chrisl: the different meanings are explained, yes. | 12:31.15 |
| felix___ | thanks, I understand now why the trim box is not considered the 'real' page box | 12:31.18 |
| chrisl | Robin_Watts: and when they are, in general, expected to be used | 12:31.45 |
| Robin_Watts | MuPDF always uses mediabox and cropbox. I can imagine that there are times when you'd want to use different combinations. | 12:31.55 |
| | So having a mechanism to set that would be a good thing. | 12:32.10 |
| chrisl | Depends - mupdf is correct according to the spec.... | 12:32.49 |
| | Usually, the other one most people are interested in is the ArtBox | 12:33.24 |
| felix___ | to me it seems that for on-screen viewing, the trimbox is the most important | 12:35.16 |
| chrisl | I suppose it also depends what's meant by "a production environment" | 12:35.18 |
| felix___ | it is the actually page that you would get if the pdf were printed | 12:35.42 |
| chrisl | No, not really - it's what you'd expect to end up with "after trimming" | 12:36.05 |
| Robin_Watts | chrisl: For screen use, you can argue that trimbox is what you want. | 12:36.31 |
| felix___ | and form what I understand this trimming is what happens during the print process | 12:36.32 |
| chrisl | I would expect trimming to be a physical process of cutting the media to a required (non-standard) size | 12:37.51 |
| | Having said all that, let's be honest, people abuse these settings just like all others, so allowing the user to choose which they want applied does make sense | 12:39.24 |
| Robin_Watts | chrisl: Yes. | 12:40.46 |
| chrisl | But my reading of the spec is that the TrimBox and ArtBox are not *intended* to be applied as a clip by the renderer | 12:41.26 |
| Robin_Watts | If you want the screen impression to be "what the actual end user will end up with in his hand" then you want to include trimming. If you want it to be "what will come out of the printer", then you don't. It depends on your intended use. | 12:42.10 |
| kens | and what does Acrobat do ? :-D | 12:42.36 |
| Robin_Watts | chrisl: This is not just for the renderer, it's for the page bounding calls too. | 12:42.42 |
| chrisl | Well, the "page" is the MediaBox | 12:43.12 |
| felix___ | I think most pdf renderers will show what is outside the TrimBox | 12:43.20 |
| kens | FWIW Ghostscript asllows you to render to any of the boxes for exactly ths reason, I think MuPDF should allow ths also | 12:43.57 |
| felix___ | but if you, for example, want to convert a pdf page to an image you would use the trimbox | 12:44.15 |
| chrisl | felix___: it depends what you expect the results to be - I'd probably opt for the ArtBox | 12:44.57 |
| felix___ | the artbox can apparently used to specify regions of special interest | 12:45.25 |
| kens | I'd suggest it depends what you intend to do with the image. If you plan to print it, you might want the full media | 12:45.32 |
| felix___ | so if a page would contain an advertisement, the artbox could be set to just that | 12:45.58 |
| chrisl | "art box (PDF 1.3) defines the extent of the pageâs meaningful content (including potential white space) as intended by the pageâs creator" | 12:46.11 |
| kens | Sounds like the old EPS BoundingBox | 12:46.40 |
| felix___ | I guess _meaningful_ is the ambiguous word here | 12:46.53 |
| | @kens correct | 12:47.08 |
| kens | In any event, since we can't tell for certain what someone wants, the obvious thing is to let them decide, while defaulting to what we think is the most likely | 12:48.06 |
| chrisl | My point is that, as far as I'm concerned, mupdf's current behaviour is correct according to the spec, and should remain the default, but adding the ability to choose is a very good thing | 12:49.11 |
| felix___ | yeah I agree | 12:49.40 |
| chrisl | Does mupdf's page object contain all the boxes, or just the currently applied dimensions? | 12:50.49 |
| felix___ | btw I found a bug related to this (in the poser tool) - I will submit a bug report | 12:50.55 |
| | poster tool | 12:51.16 |
| Robin_Watts | chrisl: Just the currently applied dimensions. | 12:51.28 |
| | felix___: The /poster/ tool? | 12:51.48 |
| chrisl | Robin_Watts: from an API perspective, it might be nice to have all of the boxes available | 12:51.55 |
| Robin_Watts | Wow. Didn't know anyone actually used that. | 12:51.59 |
| | chrisl: Yeah. | 12:52.04 |
| | Either I could add fz_meta calls to get the PDF specific boxes out, or I could add a config option to the context where you say "enable/disable the following boxes when calculating page bounds". | 12:53.05 |
| chrisl | If you did the former, would that then leave the application to calculate the actual media extents to apply? | 12:54.37 |
| felix___ | Robin_Watts: yeah we use it to normalize PDFs that have pages combined into spreads | 12:54.38 |
| Robin_Watts | chrisl: The former would give access to the raw values, but wouldn't affect the rendered output. | 12:55.33 |
| | The later would give width/height (but not the raw values), but would mean that new rendered stuff came out with the new values taken into account. | 12:56.06 |
| | s/later/latter/ | 12:56.13 |
| chrisl | Robin_Watts: So I was thinking you'd want to both have an option to apply the box(es), and have an API to allow an application to retrive the raw values | 12:56.29 |
| Robin_Watts | I suspect the latter is probably better. If you want the raw values you can always fetch them from the file yourself. | 12:56.41 |
| | Possibly we should have an fz_meta API to say "give me the PDF object for this page" to enable such fiddling. | 12:57.28 |
| chrisl | I figured a proofing app might want to display the entire media box, and show hairlines for the other boxes | 12:58.09 |
| felix___ | I guess the only odd thing at the moment is that mutool will often spit out a new pdf with pages converted to their cropbox | 12:59.50 |
| | so if you could make that and option (with the default being the mediabox) | 13:00.21 |
| Robin_Watts | chrisl: Yeah. | 13:00.40 |
| | felix___: The poster tool was something I wrote years ago on a whim. I've never known anyone actually use it :) | 13:01.09 |
| felix___ | Robin_Watts: :), well thanks for writing it! | 13:02.07 |
| | it is really fast compared to other tools that do the same | 13:02.28 |
| Robin_Watts | felix___: That's because it's REALLY dumb :) | 13:03.00 |
| Robin_Watts | foods. | 13:03.07 |
| henrys | fredross-perry: we had the SO meeing on skype before this one, have a look at the logs when you have a chance. | 14:30.29 |
| fredross-perry | ok will do | 14:30.40 |
| henrys | I now have jeong's XFA code and his entire pdf interpreter, I'll put that on casper shortly. | 14:31.45 |
| | well his entire product which is based on MuPDF. | 14:32.16 |
| | it does look like XFA is pretty well sectioned off. | 14:32.29 |
| rayjj | henrys: great. That might be worthwhile | 14:32.50 |
| henrys | so I think the next big push is geting chrisl work in the trunk. | 14:33.20 |
| rayjj | henrys: it'll probably be some work to move it to the current mupdf API (assuming that it's sort of old) | 14:33.28 |
| henrys | rayjj: yes | 14:34.01 |
| | rayjj: did you get git fixed? | 14:34.13 |
| Robin_Watts | rayjj: The next job is to see if it's actually of usable quality. | 14:34.17 |
| henrys | chrisl: it would be good to get this going before Robin_Watts leaves for vacation. I'd like to have everyone review like last time. | 14:35.04 |
| | with kens | 14:35.10 |
| Robin_Watts | is not leaving for vacation with kens. | 14:35.26 |
| rayjj | henrys: I think so | 14:35.31 |
| mvrhel_laptop | :) | 14:35.37 |
| chrisl | henrys: we agreed it would go in after kens's stuff | 14:35.41 |
| kens | I'm waiting to see if Marcos has anything else to add from the regression tests | 14:36.05 |
| | While working on his memory report | 14:36.13 |
| henrys | chrisl: oh I thought that was happening very soon. | 14:36.14 |
| rayjj | if anyone wants to take a look into bug 695622 it's really nothing that I have any particular background with (performance with shadings and DeviceN) | 14:36.33 |
| henrys | we do need to look at the release and how much "change" we want to risk also. | 14:37.04 |
| chrisl | I admit, that's worrying me a little..... | 14:37.34 |
| Robin_Watts | henrys: AIUI, chrisl's changes are basically moving files around? Not much code change? (other than makefiles etc) | 14:37.45 |
| | (or I am missing something?) | 14:38.06 |
| chrisl | No, that's right - a we *never* have build problems....... :-( | 14:38.27 |
| Robin_Watts | chrisl: I was not intending to suggest that it was therefore 'trivial'. | 14:39.05 |
| henrys | so if we are going to do it let's get it in as fast as we can and get to testing. | 14:39.20 |
| | marcosw: what is the status of kens stuff | 14:39.47 |
| chrisl | Well, the branch has been on my repo for ages, so people can review it at will | 14:39.58 |
| henrys | chrisl and Robin_Watts and the "merge" with kens stuff concerns me. | 14:40.07 |
| marcosw | kens and henrys: the regression tests should be done later today. Of the 12 tests configurations 10 finished with no problems. The only ones left are two that worked before Ken's latest changes. | 14:40.17 |
| chrisl | henrys: that's why we wanted kens's stuff on master first | 14:40.41 |
| kens | I di commit some changes to fix two devices (I htnk it was two) | 14:40.41 |
| henrys | mvrhel_laptop: is rayjj bug one you could look at? | 14:40.47 |
| mvrhel_laptop | henrys: just looking at it now. I can have a closer look this week | 14:41.05 |
| henrys | rayjj: reassign to mvrhel_laptop | 14:41.18 |
| marcosw | we still have the increase in memory consumption: http://bugs.ghostscript.com/show_bug.cgi?id=696075 | 14:41.21 |
| Robin_Watts | The point I was trying to make, I think, was that kens stuff changed lots of code in depth, hence a big review from everyone was justified. | 14:41.52 |
| kens | marcosw yes I know about that one. | 14:42.12 |
| chrisl | Personally, I don't think the memory increase is something that should keep the code from going onto master | 14:42.12 |
| henrys | marcosw: how well do you test for leaks? running jobs repeatedly ? | 14:42.18 |
| Robin_Watts | Chrisl's stuff might move files around the place, but the contents of the actual files haven't changed. | 14:42.27 |
| marcosw | henrys: not at all. I think valgrind would be the way to test for leaks. | 14:42.47 |
| chrisl | Robin_Watts: For the most part - there are a few changes in the sources, too | 14:42.53 |
| Robin_Watts | chrisl: but far fewer than with kens changes, right? | 14:43.21 |
| chrisl | Oh yes, and pretty trivial, IIRC | 14:43.31 |
| kens | THe memory consumption thing is not a leak | 14:43.35 |
| Robin_Watts | so I suspect it's not as big a review job as kens one was. | 14:43.45 |
| chrisl | I'd be stunned if it were | 14:44.01 |
| Robin_Watts | marcosw: To check for leaks, build and run with FORTIFY_LEAKONLY. | 14:44.02 |
| kens | I don't understand yet what it is, but its an increase in intermediate memory, the allocated size returns to normal at EOJ | 14:44.10 |
| Robin_Watts | *MUCH* faster than valgrind. | 14:44.16 |
| marcosw | Robin_Watts: sounds good. | 14:44.27 |
| kens | FORTIFY ? | 14:44.36 |
| Robin_Watts | MEMENTO_LEAKONLY. | 14:44.43 |
| | Gah. | 14:44.46 |
| | SO uses fortify, mupdf/gs use memento. | 14:45.04 |
| | context switches are degrading my neurons. | 14:45.15 |
| marcosw | I kind of knew what you mean. | 14:45.52 |
| rayjj | henrys: BTW, I really don't know how to tackle bug 696018. If anyone wants to try and duplicate that, please have at it (just 'take' it) | 14:46.14 |
| henrys | mvrhel_laptop: how's the viewer coming for Graph Expo? Robin is done with his piece right? | 14:46.23 |
| Robin_Watts | mvrhel_laptop: What is your current position w.r.t. gproof ? How buried in other stuff are you? | 14:46.30 |
| | henrys beat me to it :) | 14:46.37 |
| | henrys: I'm done with the core side of the changes (until I have something to test with and it all falls in a heap). | 14:47.04 |
| | There are app changes that need to be done too (but fred could be involved in doing them). | 14:47.19 |
| henrys | mvrhel_laptop: also miles was bugging me about look at David Lewis' issues. I guess he's called about his bugs. | 14:47.32 |
| | mvrhel_laptop: let me find those numbers | 14:47.52 |
| Robin_Watts | I *could* have a first run at the gs device, if that would take pressure off mvrhel. | 14:48.13 |
| mvrhel_laptop | I need to get to cracking on gsproof. I have been spending all my time on gsview pretty much. I will start dividing my time appropriately since I do want to get far on gsproof before Robin_Watts leaves | 14:48.22 |
| | Robin_Watts: if you want to that would be helpful | 14:48.36 |
| | very helpful | 14:48.41 |
| | David Lewis? | 14:48.51 |
| rayjj | cust 582 | 14:49.06 |
| henrys | rayjj beat me to it mvrhel_laptop | 14:49.21 |
| rayjj | iirc | 14:49.22 |
| mvrhel_laptop | I dont have any customer bugts | 14:49.27 |
| | bugs | 14:49.29 |
| Robin_Watts | mvrhel_laptop: Ok. I'll talk to you about that in the next couple of days. | 14:49.30 |
| mvrhel_laptop | Thanks Robin_Watts | 14:49.52 |
| henrys | mvrhel_laptop: oh let me fix that ;-) | 14:49.57 |
| | mvrhel_laptop: ah those are rayjj I assumed they would be yours because he is a high end color type... | 14:51.03 |
| | rayjj: have you had a go at either of those? | 14:51.43 |
| rayjj | mvrhel_laptop: henrys asked me to assign bug 695622 to you. That's a customer issue (cust 582) | 14:51.49 |
| mvrhel_laptop | yes you had not done it yet though... | 14:52.03 |
| rayjj | just did. | 14:52.10 |
| mvrhel_laptop | ah there it is | 14:52.11 |
| henrys | rayjj: and what about the other, is that going to be hellish | 14:52.50 |
| | ? | 14:52.51 |
| | error in clist? | 14:53.41 |
| rayjj | henrys: the bug 695766 is a bit hairy | 14:53.46 |
| | henrys: I'm wrapping up all of the fast HT changes for but 695929 then I can shift gears to that. | 14:54.48 |
| henrys | rayjj: okay great | 14:55.00 |
| rayjj | henrys: bug 695929 | 14:55.07 |
| henrys | anything else for the meeting? I'm good for now. | 14:55.23 |
| rayjj | mvrhel_laptop: I now have the TRANSFER_IN_THRESHOLD matching the non-fast HT code | 14:55.41 |
| mvrhel_laptop | rayjj: so on 695622, you think the issue is the number of trapazoids getting put in the clist? | 14:55.57 |
| rayjj | mvrhel_laptop: at least for my pared down test case | 14:56.02 |
| mvrhel_laptop | rayjj: great | 14:56.07 |
| rayjj | mvrhel_laptop: re 695622, yeah. The shading decomposition seems to be generating a LOT of stuff for DeviceN type devices. A 7.5Gb clist explains the timing | 14:57.09 |
| mvrhel_laptop | since ken is seeing a diff between 9.0 and the most recent release, I am going to focus on the why there | 14:57.28 |
| rayjj | mvrhel_laptop: sounds like a plan | 14:57.42 |
| mvrhel_laptop | blah. so Robin_Watts anything you are able to do to help with gsproof would be awesome... | 14:58.45 |
| Robin_Watts | mvrhel_laptop: I can probably do a simple device to grab the planar separations, and output the file. | 14:59.14 |
| | I'll then come back to you to fill in the "make a color correct rgb preview from the separations" bit :) | 14:59.39 |
| mvrhel_laptop | awesome | 14:59.45 |
| chrisl | Are we going to publish the spec for this gsproof image format? | 14:59.48 |
| Robin_Watts | chrisl: It's on the public twiki. | 14:59.58 |
| chrisl | It's just I'm sure people will poke at the device once it's in Ghostscript | 15:00.36 |
| rayjj | Robin_Watts: mvrhel_laptop: for the 'color correct RGB', we can only really get that if we have an ICC profile that has all of the colorants, right. Otherwise we are just blending based on the 'equivalent color' provided for the separation | 15:07.46 |
| | or am I missing something ? | 15:08.06 |
| | Robin_Watts: if we are using the 'equivalent color' information, then the blending done in tiffsep to produce the CMYK composite is an OK place to start | 15:08.55 |
| | Robin_Watts: then for CMYK->sRGB we do have profiles | 15:09.24 |
| mvrhel_laptop | rayjj: if there are non-standard spot colors, gs is certainly limited in what it can produce if someone does not supply source profiles for the DeviceN color spaces. If we have all standard CMYK source colors then we are fine | 15:09.43 |
| rayjj | mvrhel_laptop: OK. That's what I thought. The other thing we have to decide is how many CMYK variants to offer | 15:10.31 |
| | but that's just a matter of how much we want to bloat the app with resources | 15:11.07 |
| mvrhel_laptop | rayjj: We can allow the user to specify a profile that they have on their system | 15:11.15 |
| | too | 15:11.17 |
| rayjj | mvrhel_laptop: true | 15:11.36 |
| kens | chrisl were you able to reproduce teh problem in bug #696070 ? I couldn't but I don't have a 64-bit Linux to try | 15:12.42 |
| chrisl | kens: I was able to get the original report to crash, but only about 1 in 20 runs, and only with a release exe | 15:13.37 |
| kens | :-( | 15:13.48 |
| | I'd really like to know how that random binary is able to get pdfwrite to request that much memory | 15:14.10 |
| chrisl | Or even why the commit they keep babbling about would make it stop..... | 15:14.41 |
| rayjj | kens: you can log into peeved or peeves (both 64-bit binary) | 15:14.45 |
| kens | Iy might be some kind of uninitialised variable, and I'd like to fix that if it is | 15:14.52 |
| rayjj | s/binary/linux/ | 15:14.55 |
| kens | rayjj won't help if it won't fail on a debug build | 15:15.15 |
| chrisl | kens: I tried valgrind and it reported no issues at all | 15:15.16 |
| kens | Oh well, I guess I give up then. | 15:15.29 |
| rayjj | kens: true | 15:15.31 |
| | chrisl: what about a profile build ? | 15:15.43 |
| kens | My suspicion is that its meory-layout related, whch is why that commit makes it go away | 15:15.56 |
| chrisl | rayjj: no, a profile build didn't crash - at least, not in the number times I ran it | 15:16.19 |
| | As my comment on the bug: the extra check should solve the *security* issue they have, and seems like a reasonable safety measure anyway, so..... | 15:17.04 |
| kens | Indeed, I just dislike papering over the cracks | 15:17.30 |
| chrisl | The only backtrace info I could get had the crash happening inside glibc, so I ran out of ways forward | 15:18.16 |
| kens | Yeah its not worth wasting time on it | 15:18.27 |
| chrisl | My hope, as I said to the original reporter, is that someone will find the same problem in a more reproducible form. | 15:19.29 |
| scorneli | I have to leave in an hour, anything else you might need for bug 696070? | 15:34.55 |
| chrisl | Well, the gdb output is really no help at all :-( | 15:36.31 |
| | scorneli: I doubt it will be much more help, but could you add a full backtrace, too? | 15:37.54 |
| kens | Migh show me where its coming from in pdfwrite | 15:38.27 |
| scorneli | why? it shows the overflow. size is 4294967288, the amount it wants to allocate. but it actually allocates "added", and added is calculated via "size + sizeof(gs_malloc_block_t);". ultimately, that result is bigger than uint can handle, so it wraps around and becomes 40. and 40 is what it allocates | 15:38.38 |
| chrisl | scorneli: but it doesn't show *why* we're trying to allocate 4294967288 bytes | 15:39.08 |
| scorneli | ahh, now I understand | 15:39.54 |
| chrisl | scorneli: as we said above, we can put in a check to ensure we don't overflow, but there may be a deeper problem, and we'd just be papering over a crack | 15:40.46 |
| scorneli | yeah, that might be the case. I didn't really investigate after finding the overflow | 15:41.27 |
| chrisl | scorneli: thanks - based on that, I think the overflow check probably *is* the correct fix..... | 15:48.06 |
| scorneli | chrisl: well, I think the check should be in there, regardless of why size ends up being huge at that point. I can investigate where the huge size came from, but probably won't have time for that until tomorrow | 15:50.58 |
| kens | Umm, yeah that doesn't really look like its anythign to do with pdfwrite, its the scanner :-) | 15:51.12 |
| chrisl | scorneli: I think the problem is that the job asks for an allocation of that size | 15:51.39 |
| kens | Its treating the random binary as PostScript binary sequence, and trying to allocate an enormous array | 15:52.07 |
| chrisl | Indeed - why the hell did Adobe make the array size a four byte number?? | 15:52.38 |
| kens | So it could be *really* big :-) | 15:52.55 |
| | In truth, no idea | 15:53.00 |
| chrisl | When did binary tokens appear? | 15:53.16 |
| kens | level 1 I htnk | 15:53.23 |
| | For compactness | 15:53.30 |
| chrisl | So, at a time when there was a hard 64k limit on sizes, they allowed a four byte value for the array size? Klever! | 15:54.16 |
| kens | I'm happy to do the fix, (I always wa) I just feel a lot better understanding *why* theere's a problem, I hate fixing stuf wihtout understanding it | 15:54.16 |
| | chrisl , yeah madness..... | 15:54.29 |
| chrisl | kens: With the patch I put in the bug, it now gives a VMerror | 15:55.35 |
| kens | VMerror is sensible | 15:55.42 |
| | Any error is good, we don't want to craqsh. | 15:55.55 |
| chrisl | So I can cluster test and push this | 15:56.09 |
| kens | Sounds like a plan to me | 15:56.34 |
| chrisl | Cool... | 15:56.45 |
| | scorneli: thanks for the help! | 15:56.50 |
| scorneli | my pleasure! I'll disappear know, if you need anything else let me know in the bug | 15:57.36 |
| | s/know/now | 15:57.46 |
| kens | Night folks | 16:07.07 |
| chrisl | reboots | 16:52.21 |
| | Forward 1 day (to 2015/07/08)>>> | |
IRC Logs