| | <<<Back 1 day (to 2012/07/31) | 2012/08/01 |
| V_Tori | hey | 05:44.23 |
| ghostbot | que tal | 05:44.23 |
| V_Tori | what are the differences between fz_open_document and pdf_open_document ? | 05:45.24 |
| | my aim is to use mupdf to render the content of the pages in our graphic object | 05:46.25 |
| | basically, i see a lot of functions un mupdf.h that already exist in fitz.h, with a different namespace | 05:47.22 |
| | it seems to me that mupdf.h is actually useless | 06:04.23 |
| | ha no, dictionnary, for example | 06:05.52 |
| | is it valid to cast a pdf_document * to a fz_document * ? | 06:36.34 |
| | i think so as 'super' is the first member of pdf_document_s | 06:38.34 |
| | iarg ! i can't retrieve the version, now | 06:45.25 |
| | it's internal and not exposed by a function | 06:45.36 |
| | kens: http://pastebin.com/CHuLjrPV | 07:12.28 |
| kens | Cna't help you there, I don't know enough about MuPDF. Either to8 or Robin_Watts will be along in an hour or two and they usually read the logs. If you have to go you can read our IRC logs here: http://ghostscript.com/irclogs/2012/07/31.html | 07:13.36 |
| | One of them will respond I expect and you can always catch up | 07:13.51 |
| V_Tori | kens: ok, thanks | 07:42.09 |
| | i didn't know about the log | 07:42.17 |
| kens | You're welomce V_Tori, I'm sure Robin_Watts and tor8 will read the log when they come online | 07:44.41 |
| V_Tori | api had changed a lot between 0.9 and 1.0 | 07:45.19 |
| | and there is almost no doc | 07:45.32 |
| | only apps examples | 07:45.45 |
| kens | Yes, there was a lot of work done to make it ready for a v 1.0 release. I thought there was documentation, but as I said, I 'm not expert | 07:45.52 |
| V_Tori | in the headers, there are some | 07:46.06 |
| | but it's not sufficient, imho | 07:46.14 |
| | i read the apps code | 07:46.20 |
| kens | Reading the applicationj source is probably a decent way to go | 07:46.49 |
| V_Tori | not enough :) | 07:50.09 |
| | for example, the version of the pdf doc | 07:50.20 |
| | afaics, it's internal | 07:50.30 |
| kens | You'll have to take it up with someone other than me.... | 07:50.43 |
| V_Tori | but reading the source code of pdf/ and fitz/, it seems it's possible with fz_meta | 07:51.03 |
| | it works :) | 08:06.55 |
| Robin_Watts | V_Tori: Morning | 09:17.51 |
| V_Tori | Robin_Watts: hey | 09:18.32 |
| chrisl | bbiaw | 09:18.51 |
| Robin_Watts | Ideally you'd use fz_open_document to get an fz_document * and you'd then always use that. | 09:19.21 |
| | And that would isolate you from the specifics of different formats. | 09:19.35 |
| | but there are some things you can only do with a pdf_document currently. | 09:19.53 |
| V_Tori | dict for example | 09:20.32 |
| Robin_Watts | Right. If you want to go walking the structure of a PDF file, then you're obviously into pdf specific territory. | 09:21.12 |
| | In general, if you want a pdf_document * use pdf_open_document, where if you want an fz_document * use fz_open_document. | 09:22.39 |
| | but for now, you can cast from fz_document to pdf_document if you know it's a pdf_document. | 09:23.31 |
| kens | OK I'm off for a couple of hours, see you all later. | 09:26.01 |
| V_Tori | that's what i figured when i tried to get the version of the pdf from a pdf_document | 09:26.53 |
| paulgardiner | Robin_Watts: ping | 11:20.09 |
| Robin_Watts | pong | 11:26.01 |
| talib | Hello, to downgrade a version of ghostscript on archlinux, do I just delete the ghostscript directory and install a different version? I'm using emacs with AUCTeX 11.86 and Ghostscript 9.05 and whenever I try to preview a mathematical equation, I receive an error "Error: /invalidfileaccess in --file--" | 11:28.43 |
| Robin_Watts | talib: Hard to say. It depends on the packaging that they've done for that distro. | 11:29.42 |
| paulgardiner | Robin_Watts: I was just going to ask you if I was likely to run into problems merging master into forms, but I just tried and it seemed problem free. | 11:29.43 |
| Robin_Watts | We don't do packagin. | 11:29.46 |
| | paulgardiner: Ah, cool. | 11:29.57 |
| paulgardiner | Of course this means that merging forms into master would be fastforward until the next master commit. | 11:30.55 |
| Robin_Watts | paulgardiner: The big thing about the merge to master is (IMHO) that tor should do it, thus giving an overall code review. | 11:31.50 |
| | but he's off this week. | 11:31.59 |
| paulgardiner | Yeah and I absolutely agree, but just mentioning that the merge itself shouldn't be a problem. | 11:32.56 |
| | I need to do a cluster test to see if anything has broken. | 11:33.20 |
| Robin_Watts | Ok, bumping GS_CLIENT_COLOR_MAX_COMPONENTS to 64 works fine now. Costs 2 hours in the regression runs though. | 11:38.53 |
| | All the time difference will be in PS runs, as all the others know the number of components they need and only use that many. | 11:40.39 |
| | I wonder if there is some way to keep track of how many components we're actually using... | 11:41.13 |
| | So I can't install VS2012Express except on windows 8. | 12:02.51 |
| | so, 3.3 Gig of windows 8 download then... | 12:03.04 |
| kens | runs away screaming | 12:03.13 |
| Robin_Watts | Thank god for vmware player. | 12:03.45 |
| Robin_Watts | wonders if it'd be possible to introduce a 'num_active' field in the colorinfo struct in the graphics state. | 12:05.45 |
| | Probably that wouldn't work so well with the clist, as num_active would grow over time, then when we replayed the clist, it would increase again. | 12:07.12 |
| | Unless we put ops into the clist that go into every band saying "increase num_active" | 12:07.45 |
| kens | How is it done for PDF ? | 12:07.47 |
| Robin_Watts | PDF knows the number of spots in advance. | 12:07.56 |
| | so the device is set correctly once and for all. | 12:08.05 |
| kens | Only if you analyse each colour space on the page | 12:08.12 |
| Robin_Watts | with PS we don't know until we meet them, so it's set to a maximum number from the start. | 12:08.25 |
| kens | Do we do that ? | 12:08.26 |
| Robin_Watts | I believe we must. | 12:08.30 |
| kens | never knew that | 12:08.38 |
| Robin_Watts | cos mvrhel definitely said it was only PS that was affected. | 12:08.46 |
| paulgardiner | Robin_Watts: do cluster tests, other than mujstest use mupdfshow.c? I imagined so, but seeing some strangeness | 12:29.20 |
| Robin_Watts | paulgardiner: Nothing actually *uses* pdfshow. Are you seeing a compile failure for that? | 12:34.33 |
| paulgardiner | Yes, because of pdf_prinnt_xref not being defined in release builds. | 12:35.05 |
| Robin_Watts | Ok. Let me look at that. | 12:35.16 |
| paulgardiner | Ta. I can fix it here, but maybe it should be fixed in the cluster-test build command | 12:35.57 |
| V_Tori | how do I manage dict ? | 12:39.12 |
| | it seems more complicated than before | 12:39.24 |
| | I want to retrieve the title, author, etc... | 12:39.44 |
| | in the apps, you use internal stuff | 12:39.55 |
| Robin_Watts | V_Tori: Yes. We hope to add a generic metadata interface for fetching title/author etc from an fz_document, so you'll be isolated from the file format, but that's not there yet. | 12:49.19 |
| V_Tori | Robin_Watts: but, currently, we can retrieve them, right ? | 12:50.05 |
| Robin_Watts | currently you have to get them using the dict stuff. | 12:50.21 |
| V_Tori | and how do I use the dict stuff ? | 12:50.45 |
| paulgardiner | Something like: pdf_to_name(pdf_dict_getp(doc->trailer, "Info/Author"))) | 12:50.45 |
| V_Tori | pb: | 12:50.56 |
| | doc->trainler is internal | 12:51.10 |
| | trailer* | 12:51.14 |
| | and i guess it's pdf_dict_gets and not getp | 12:52.59 |
| Robin_Watts | no, it is getp. | 12:54.32 |
| V_Tori | it's not in mupdf.h | 12:54.45 |
| Robin_Watts | getp is a cunning addition by paulgardiner to let you get stuff by "path". | 12:55.02 |
| | It's in mupdf.h for me. | 12:55.09 |
| V_Tori | not in 1.0, then | 12:55.17 |
| Robin_Watts | line 75. | 12:55.20 |
| | No, not in 1.0 | 12:55.24 |
| V_Tori | ok | 12:55.33 |
| | so, I use mupdf 1.0 | 12:55.40 |
| paulgardiner | Ah. Multiple gets calls then. | 12:55.51 |
| Robin_Watts | You can do pdf_to_name(pdf_dict_gets(pdf_dict_gets(doc->trailer, "Info"), "Author"))) | 12:56.00 |
| | getp just saves typing. | 12:56.15 |
| V_Tori | and pdf_to_name is not in mupdh.h too | 12:56.20 |
| | pdf_to_utf8 exists | 12:56.32 |
| Robin_Watts | really? | 12:56.54 |
| V_Tori | ho | 12:57.23 |
| | sorry | 12:57.25 |
| | i've not search from the beginning of the file | 12:57.39 |
| Robin_Watts | ah, ok :) | 12:57.51 |
| V_Tori | so, anyway, i have to include mupdf-internal.h to get trailer from doc | 12:58.18 |
| Robin_Watts | V_Torri: Aha | 12:59.01 |
| | Look at win_main.c in the dloginfoptoc | 12:59.14 |
| | dloginfoproc | 12:59.18 |
| | That uses the fz_meta interface to get information. | 12:59.34 |
| | From an fz_document | 12:59.41 |
| | So, that stuff *did* make it into 1.0 | 13:00.01 |
| | Sorry. | 13:00.04 |
| Robin_Watts | goes for lunch. | 13:00.09 |
| paulgardiner | Robin_Watts: just checked origin/master and showxref seems to call pdf_print_xref. Any idea what I'm missing? | 13:00.21 |
| V_Tori | hmmm | 13:00.22 |
| alexcher | How to test the upcoming release? I build it and run a bunch of files. Perhaps, we can use a more formal test process? | 13:20.15 |
| chrisl | alexcher: I've been using smoke.ps (slightly tweaked) and using our "public" test files | 13:23.58 |
| alexcher | chrisl: that's fine, but the results are not matched against the checksums and the results of the test are not recorded. | 13:32.49 |
| kens | But presumably the hash where we branched is.... | 13:33.34 |
| chrisl | You can clusterpush from the release "tree" but that only tests what we already test....... | 13:34.09 |
| | Given that we expect there to be differences in output between releases, I'm not sure how useful testing each release against the previous one would be, in terms of the benefits vs time it would take | 13:37.59 |
| alexcher | chrisl: I want to compare the results on produced on other platforms and architectutes vs. the cluster. | 13:38.04 |
| chrisl | alexcher: I'm not aware of having a method of "harvesting" md5 data from the cluster.... | 13:39.14 |
| Robin_Watts | alexcher: Simplest method for testing using the cluster is just to clusterpush the release. | 13:40.38 |
| | That'll be tested against the current trunk. | 13:40.45 |
| chrisl | Robin_Watts: that only tests one platform, though | 13:41.01 |
| Robin_Watts | The md5 data can be had from the internal cluster records easily enough. | 13:41.03 |
| | True. | 13:41.09 |
| chrisl | And once we get the md5 data from the cluster, we then need a way to gather that same data on a.n.other system, and compare them...... | 13:43.02 |
| paulgardiner | Robin_Watts: in case you missed it when reading back - just checked origin/master and showxref seems to call pdf_print_xref. Any idea what I'm missing? | 13:43.45 |
| Robin_Watts | hold on. | 13:44.04 |
| paulgardiner | No hurry. | 13:44.19 |
| Robin_Watts | I see it changed in a4f38e03 | 13:45.44 |
| chrisl | alexcher: this is probably something we need to discuss with marcosw - how about getting henrys to put it on the agenda for the Sept staff meeting? | 13:45.52 |
| Robin_Watts | paulgardiner: OK. Now I see it. | 13:46.49 |
| alexcher | chrisl: fine but too late for this release | 13:48.03 |
| paulgardiner | Ah Tor viewed it as a typo | 13:48.17 |
| | I could remerge one back from there to avoid having to work out what this is about. | 13:48.51 |
| Robin_Watts | no, let me fix it. | 13:48.59 |
| chrisl | alexcher: true, but reminding me/us about it the day after I've created the release candidate isn't likely to get anything in place for the release in progress! | 13:49.31 |
| Robin_Watts | paulgardiner: Patch for review on my master branch | 13:50.37 |
| | sebras: ping. | 13:50.46 |
| paulgardiner | Robin_Watts: looks good to me | 13:55.09 |
| Robin_Watts | pushed | 14:01.37 |
| paulgardiner | ta | 14:02.13 |
| V_Tori | what is the cookie param of pdf_run_page ? | 14:03.41 |
| | it's not documented | 14:04.00 |
| Robin_Watts | The cookie param is documented. | 14:04.13 |
| V_Tori | not in 1.0 | 14:04.24 |
| Robin_Watts | It's a structure that allows for simple 2 directional communication between the calling app and the library. | 14:04.53 |
| | Hold on. | 14:05.21 |
| | http://git.ghostscript.com/?p=mupdf.git;a=blame;f=fitz/fitz.h;h=04215c710fa03193e7e51f8d853e9fd215b6c31e;hb=af5a6088a3c3e4d61f84d2c93a8968c04069864d | 14:06.23 |
| | Lines 1706 onwards. | 14:06.29 |
| | That's v1 | 14:06.38 |
| | You should always init the cookie structure to all zeros to allow for future expansion. | 14:07.32 |
| V_Tori | PDF_run_page | 14:07.40 |
| | http://git.ghostscript.com/?p=mupdf.git;a=blob;f=pdf/mupdf.h;h=b88f7423040a273d02f40da7729c9324bf560c73;hb=af5a6088a3c3e4d61f84d2c93a8968c04069864d#l202 | 14:07.53 |
| Robin_Watts | V_Tori: So, you'd like: "cookie: See the definition of fz_cookie" inserted? | 14:13.18 |
| V_Tori | if it's the same, then ok | 14:13.43 |
| | yes | 14:13.55 |
| | that would be sufficient | 14:14.11 |
| | not only, btw, but also definition of fz_run_page | 14:14.51 |
| | declaration of* | 14:15.01 |
| Robin_Watts | mvrhel: Is that really you? | 14:24.24 |
| V_Tori | fz_new_text_span disappeared ? | 14:36.22 |
| Robin_Watts | V_Tori: Urm... | 14:39.55 |
| V_Tori | it's now sheet instead of span ? | 14:40.25 |
| Robin_Watts | No, it's not that simple. | 14:40.39 |
| | IIRC sheet holds a list of styles. | 14:40.48 |
| V_Tori | i'll check that in an app | 14:41.49 |
| | while i'm seeing it: bug in doc (at least in 1.0) : | 14:42.05 |
| Robin_Watts | You probably want to create a page, and then we'll fill in blocks/lines/spans into that. | 14:42.07 |
| V_Tori | doc of fz_clear_pixmap | 14:42.21 |
| | the function name is fz_clear_pixmap_with_value | 14:42.36 |
| | copy/paste error :) | 14:43.07 |
| mvrhel | Robin_Watts I am here. Just give me 15 minutes to help get the kids out the door to swim lessons | 14:51.23 |
| Robin_Watts | no worries. | 14:53.16 |
| henrys | kens:yes compare against 1.4 but I think a 1.5 run would be useful also, it will help us set priorities. | 15:00.47 |
| kens | henrys yes, I thik I said I'd be interested in a 1.5 test | 15:01.05 |
| | But a comparison with 1.4 would be more immediately useful I think | 15:02.46 |
| mvrhel | Robin_Watts: I am here now | 15:22.02 |
| Robin_Watts | mvrhel: OK. The tests with max components of 64 seem to be running just fine now. | 15:22.46 |
| mvrhel | awesome | 15:22.54 |
| Robin_Watts | but it adds 2.25 hrs to the runtime. | 15:23.07 |
| | All the slowdown is in the 'gs' ones, (as opposed to xps, pcl, gs+pdfwrite etc) | 15:23.37 |
| | Which I guess makes sense, because it's only .PS files that are affected by this, right? | 15:23.54 |
| | I was wondering if it was possible to somehow limit the amount of unnecessary work we do when 90% of the time we don't need all the spots. | 15:25.01 |
| mvrhel | Robin_Watts: right because we max out the ps files always due to us not knowing like we do in pdf the number of colorants on each page | 15:25.09 |
| Robin_Watts | right. | 15:25.15 |
| | I was pondering the idea of having a "num_active" field in the color_info or something. | 15:25.32 |
| | So for CMYK+spot devices, num_active would start at 4, and would increase as we meet each new spot color. | 15:26.07 |
| mvrhel | Robin_Watts: why would we not be able to do that with num_components now? | 15:26.34 |
| Robin_Watts | Each drawing operation would know that it only had to deal with "num_active" components, rather than GS_CLIENT_COLOR_MAX_COMPONENTS | 15:26.45 |
| | mvrhel: Aren't various spacing calculations done with num_components ? | 15:27.07 |
| | We'd want to calculate things like banding etc based on the maximum number of possible spots. | 15:27.22 |
| mvrhel | well use max for those then | 15:27.41 |
| henrys` | urm you guys forgot to put people in the seats at the olympics - a minor oversight | 15:27.53 |
| Robin_Watts | OK. So you don't think this is a stupid idea? | 15:28.00 |
| mvrhel | Robin_Watts: I didn't say that ;) | 15:28.11 |
| Robin_Watts | henrys`: Yeah. They are talking about drafting in the army to help fill them :/ | 15:28.23 |
| mvrhel | Just that we already have the information you want I believe | 15:28.30 |
| | in color_info | 15:28.33 |
| | but this sounds like a nightmare a bit to me | 15:28.47 |
| Robin_Watts | mvrhel: I thought num_components was set to the maximum value? | 15:28.56 |
| mvrhel | for PS it is now. But not for PDF | 15:29.12 |
| | My question was why not have it changing for PS like we do for PDF | 15:29.30 |
| Robin_Watts | Right. It's specifically the PS case I'm talking about. | 15:29.36 |
| mvrhel | which is what I thought you wanted to do | 15:29.36 |
| Robin_Watts | I thought the PDF case it was calculated at the start of the page and set there. | 15:29.55 |
| | So it doesn't change during the running of page marking operations. | 15:30.06 |
| mvrhel | right | 15:30.09 |
| | I see | 15:30.13 |
| Robin_Watts | (important for things like the clist etc) | 15:30.16 |
| | Having a value that changes could give us grief with patterns, maybe? And we'd have to be careful so that when reading the clist, we knew how many colors we thought we were writing for when we read it back. | 15:31.10 |
| paulgardiner | henrys: It's probably the seats that the G4S security guards were supposed to be sitting in. | 15:31.14 |
| mvrhel | the color_info is used in so many places, I worry about the number of hiccups that may occur having it change 1/2 way through a page rendering | 15:31.24 |
| Robin_Watts | mvrhel: Right. | 15:31.32 |
| | mvrhel: Another idea, which may be simpler, is to spot the case where we are writing an empty bitmap to the clist. | 15:32.15 |
| | so copy_planes will be less painful. | 15:32.34 |
| mvrhel | that would be good yes | 15:32.35 |
| Robin_Watts | I've had a crack at that. | 15:32.44 |
| mvrhel | Robin_Watts: so how much did the move up to 32 hurt us time wise? | 15:35.24 |
| chrisl | Robin_Watts: in the short term, could we limit the number of active planes from the command line? Postscript would rarely have more than 8-12 plates in a job | 15:35.27 |
| Robin_Watts | chrisl: It's currently a compile time option. | 15:35.46 |
| mvrhel | That would make more sense | 15:35.49 |
| | to add in a command line option | 15:35.55 |
| Robin_Watts | Going to 32 cost us about 30 mins, I think. | 15:36.07 |
| mvrhel | generally someone is going to know ahead of time how many colorants they have | 15:36.18 |
| Robin_Watts | Really? You think Gemma would know, for example? | 15:37.04 |
| chrisl | Robin_Watts: they're all PDF | 15:37.15 |
| mvrhel | right | 15:37.19 |
| Robin_Watts | Well, compiling with MAX_COMPONENTS = 64 and then having a runtime value for ps that defaults to 32 seems a good approach. | 15:38.15 |
| V_Tori | can someone describe the meaning of the following links : | 15:38.19 |
| | FZ_LINK_LAUNCH, | 15:38.22 |
| Robin_Watts | Then people can push the 32 to 64 if they want. | 15:38.23 |
| V_Tori | FZ_LINK_NAMED, | 15:38.24 |
| | FZ_LINK_GOTOR | 15:38.25 |
| | ? | 15:38.27 |
| Robin_Watts | V_Tori: They exactly mirror the PDF spec. | 15:38.36 |
| V_Tori | (for the outline, for example) | 15:38.44 |
| | heh, unfortunately, i've no spec | 15:38.57 |
| Robin_Watts | LAUNCH = a string to launch. (so 'run this file') | 15:38.59 |
| | NAMED = a named target in another file. | 15:39.07 |
| | GOTOR = Goto a page in a remote file. | 15:39.16 |
| V_Tori | ok | 15:39.21 |
| | thanks | 15:39.23 |
| chrisl | Robin_Watts, mvrhel: If we limit the maximum plates to 12 for Postscript files, and leave PDF as it is, we'll still get good coverage of large numbers of spot color plate handling, but not have the extra cluster time. | 15:39.25 |
| mvrhel | right. I really wonder if we want pdf to be at 32 to be honest with yoy | 15:39.49 |
| | you | 15:39.51 |
| Robin_Watts | AIUI, having the CLIENT_MAX set to 64 wouldn't hurt us at all, as long as the ps max becomes a runtime thing that's lower. | 15:40.35 |
| mvrhel | ok | 15:40.48 |
| Robin_Watts | There is no intrinsic slowdown caused by having a CLIENT_MAX of 64. | 15:41.01 |
| chrisl | I've seen at least two PDFs with 36 plates in them (four process + 32 spots) - I don't think that's common - I *hope* that's not common! | 15:41.16 |
| Robin_Watts | So having that set to 64 and having a runtime "max spots" thing for PS would seem an ideal solution - except that PS users need to know that they can supply a command line flag to up that number of spots. | 15:41.50 |
| mvrhel | right | 15:42.06 |
| Robin_Watts | Ok. I've installed windows 8, and they've taken my start button. | 15:42.41 |
| | How the hell can I get to my installed programs? | 15:42.49 |
| chrisl | Robin_Watts: the cluster already has slightly different command lines for PS and PDF - I'd say default to the maximum, and have a command line to limit the number of available plates. | 15:42.55 |
| Robin_Watts | chrisl: So we slow our customers down, just not us? :) | 15:43.16 |
| chrisl | Robin_Watts: then we document the option, of course...... | 15:43.39 |
| mvrhel | this all sounds good to me | 15:45.30 |
| | henrys: when is the ironman? | 15:46.00 |
| chrisl | Robin_Watts: the other option would be to have a "flat" clist, so we can calculate the bands *after* the clist is created, and take the hit of executing the whole page description once for each band..... | 15:46.42 |
| Robin_Watts | chrisl: We could have a record of the number of active colors. | 15:47.29 |
| | That would start at 4 for CMYK + spots files, and increase each time we meet a new spot. | 15:47.45 |
| henrys | mvrhel:sunday | 15:47.47 |
| Robin_Watts | we'd have to write a command "new spot" to the clist each time we met a new color. | 15:48.11 |
| | That way, the clist on playback could know how many colors it was dealing with at each point. | 15:48.29 |
| | but it's a lot of effort. | 15:48.35 |
| chrisl | Robin_Watts: we'll still have the wasted memory of the unused to plates, then | 15:48.48 |
| Robin_Watts | we'd still calculate band sizes as if we had all the planes, yes. | 15:49.12 |
| | but we'd not have to write every single plane into the clist. | 15:49.29 |
| | only the used ones. | 15:49.34 |
| chrisl | You'd need to write the "erasepage" fill to all the planes, wouldn't you? | 15:49.59 |
| Robin_Watts | All planes would be created erased. | 15:50.37 |
| chrisl | Huh? | 15:50.58 |
| henrys | can everybody please give the release candidate a go on whatever platforms you have immediately available? | 15:52.17 |
| Robin_Watts | OK, so the fillpage clist thing would hit all the planes. | 15:52.52 |
| chrisl | I suppose all spot plates could be created erased - it is possible to erase to something other than white....... | 15:52.52 |
| Robin_Watts | If we are erasing to something other than white, then the spots must be defined at that point, right? | 15:53.21 |
| | (Sorry, if we erasing to something where a given component is non zero, then that component must be defined (must be active) at that point) | 15:53.53 |
| chrisl | Yes, that just occurred to me as I was typing - it's extremely rare, and I doubt anyone would want to erase anything other than process plates to anything other than white...... | 15:54.18 |
| Robin_Watts | I cannot see how they ever *could* erase spots to non-empty, unless they somehow define the spot before the erase. | 15:54.57 |
| chrisl | Robin_Watts: normally, a device knows what inks it has access to, so the spot plates are, by definition, defined before the erase | 15:55.42 |
| Robin_Watts | It's hairy for things like patterns, because we might get a pattern defined when we only have 4 colors, then have it played back when we have 8. | 15:55.52 |
| | Can we do some funky interogation of the device to find out what spots there are from a device before we start ? | 15:56.47 |
| kens | Robin_Watts : not in a device-independent manner | 15:57.04 |
| Robin_Watts | My understanding was that with PS we didn't know about spots until we actually met one. | 15:57.05 |
| chrisl | Robin_Watts: the whole point of tiffsep and co is that they handle *any* spot color the input requests | 15:57.19 |
| Robin_Watts | As I say, this sounds like a much larger, hairier project than just having a runtime configurable max. | 15:57.52 |
| chrisl | I think being smarter about this is going to be quite a big project! | 15:58.49 |
| Robin_Watts | So do we want -dMaxSpots= or something? | 15:59.05 |
| | or -dMaxComponents= ? | 15:59.16 |
| chrisl | I'd prefer a maximum spot plates setting, rather than an overall maximum plates setting, but that's just MHO..... | 16:00.38 |
| mvrhel | I agree | 16:00.45 |
| | -dMaxSpots | 16:00.51 |
| Robin_Watts | actually... is num_components still set to 32 even on devices like ppmraw ? | 16:01.00 |
| mvrhel | ? | 16:01.13 |
| | ppmraw should be 3 | 16:01.39 |
| Robin_Watts | At the moment, for PS files and the tiffsep or psdcmyk device (devices that cope with spots), we have num_components set to a large number. | 16:01.45 |
| mvrhel | right | 16:02.00 |
| Robin_Watts | Is it only those devices that are thus affected? | 16:02.06 |
| mvrhel | yes | 16:02.10 |
| Robin_Watts | So the 2 hour slowdown we're seeing comes JUST from the psdcmyk device? | 16:02.27 |
| mvrhel | so should this be a device parameter | 16:02.30 |
| | Robin_Watts: should be | 16:02.36 |
| Robin_Watts | Wow. | 16:02.40 |
| | I'm surprised it didn't manifest as timeouts then. | 16:03.00 |
| mvrhel | Robin_Watts: maybe you should do a check with some other devices on the cluster. eg. plank | 16:03.34 |
| | with and without 64 | 16:03.44 |
| Robin_Watts | yeah. I'll run some tests. | 16:03.55 |
| | We can scrounge some time back from the psdcmyk device by doing the tempfile thing to avoid rerendering multiple times. | 16:04.39 |
| | I suspect that I need to look at the downscaler too; if the bandheight is not a multiple of the downscale factor, then it's going to rerender bands I think. | 16:05.39 |
| kens | OK I'm off for now, goodnight all | 16:10.43 |
| mvrhel | henrys: so what kind of training things do you do in this last week? | 16:12.19 |
| henrys | last weekend I did the race in separate pieces and now I'm resting this week | 16:14.18 |
| alexcher | rc1 doesn't build on ppc32 Linux, the bug is somewhere in lcms2. | 16:14.53 |
| chrisl | alexcher: it builds for me on ppc32 Linux...... | 16:15.33 |
| henrys | mvrhel:mainly worried about the heat | 16:16.22 |
| mvrhel | henrys: I imagine. | 16:16.35 |
| alexcher | chrisl: it looks like I need to debug this. | 16:16.53 |
| henrys | supposed to be in the 90's ... | 16:17.01 |
| chrisl | alexcher: what distribution/version are you running? | 16:17.13 |
| V_Tori | Robin_Watts: another question : | 16:17.17 |
| | in fz_link_dest_s, there is an union | 16:17.32 |
| | especially, gotor struct | 16:17.43 |
| | is that gotor struct also valid for a goto link ? | 16:18.00 |
| mvrhel | henrys: hopefully they will have plenty of watering stations | 16:18.25 |
| alexcher | chrisl: Debian stable, a few years old, gcc 4.3.2 | 16:18.29 |
| Robin_Watts | V_Tori: Yes, same struct, but with a NULL remote pointer. | 16:18.54 |
| V_Tori | perfect | 16:19.00 |
| | thanks | 16:19.02 |
| | the api is, imho, quite better | 16:19.14 |
| | but new to me | 16:19.19 |
| | so i'm a bit lost | 16:19.24 |
| Robin_Watts | Yes, we spent a lot of time revising the interface for 1.0. It is different, but better that than carrying around too many inconsistencies for ages. | 16:20.08 |
| henrys | mvrhel:the race is very well organized. Boulder triathlons are big events that bring in the serious athletes and organizers. | 16:20.14 |
| chrisl | alexcher: I might have testing or unstable on mine - let me check...... | 16:20.26 |
| V_Tori | hmm | 16:20.44 |
| | about outline | 16:20.49 |
| | before there was a child member | 16:21.01 |
| | now there are 2 links : next and down | 16:21.15 |
| | ha | 16:21.34 |
| | ok | 16:21.36 |
| | binary tree | 16:21.38 |
| chrisl | alexcher: actually, no, I'm running Debian stable - "squeeze" 6.0.4..... | 16:24.55 |
| | and gcc 4.4.5 | 16:25.50 |
| | alexcher: it looks like lcms2 is assuming that at least a couple of C99 macros are available - that could be bad :-( | 16:33.50 |
| alexcher | chrisl: lcms defines CMS_DONT_USE_INT64, and assigns one array to another. | 16:35.16 |
| chrisl | alexcher: it looks like lcms2 won't build without a C99 compiler :-( | 16:39.02 |
| alexcher | chrisl: It will, wait a minute. | 16:39.37 |
| Robin_Watts | chrisl: It builds on windows (VS2005), and that's not C99. | 16:41.11 |
| chrisl | Robin_Watts: no, but it's >C89, though. I'm trying to build with gcc using the "-std=c89" option, and it don't work - it might assume that gcc is c99, I suppose.... | 16:43.13 |
| Robin_Watts | Gawd. CUPS needs winsock2.h for it's http stuff. What the hell is that doing in gs ? | 16:43.39 |
| chrisl | Robin_Watts: someone felt it was needed so Windows users could debug cups problems..... | 16:44.20 |
| Robin_Watts | :) | 16:44.36 |
| chrisl | We really ought to remove it from the standard set of devices, though | 16:44.46 |
| alexcher | chrisl: builds now. should I open a bug and attach my patch? | 16:44.49 |
| chrisl | alexcher: sure, thanks. | 16:45.05 |
| Robin_Watts | I have no objections to cups being a standard windows device. I object to it having crap in that we never use. | 16:45.17 |
| | and I fail to see how we need anything http related. | 16:45.28 |
| chrisl | Robin_Watts: we don't, but cups does.... | 16:45.39 |
| | cups is *only* in the Windows build for our (and possibly other developers' use), it's completely pointless from a user's point of view, so I really don't think it should be in the Windows build by default. | 16:47.05 |
| Robin_Watts | Yeah, fair enough. | 16:49.05 |
| chrisl | I just forgot to remove it ...... | 16:49.38 |
| halfie_ | any mupdf developers around? mupdf can't handle encrypted PDF files generated by Acrobat Pro X | 17:03.55 |
| Robin_Watts | halfie_: hi. really? Can you give me an example file? | 17:04.27 |
| halfie_ | Robin_Watts, sure http://openwall.info/wiki/_media/john/pdf_samples.tar | 17:05.06 |
| | and look for test-X-AES-256-open-testpassword.pdf file | 17:06.08 |
| | Robin_Watts, password is "testpassword" | 17:07.09 |
| henrys | chrisl:well I've smoke tested on mac and linux boxes here - wasn't expecting to find anything but best to be safe. | 17:09.01 |
| chrisl | henrys: thanks. I've done something on Linux x64/PPC32, Windows (32 and 64 bit), Solaris/Sparc and Mac (x86) - but only a few files on each....... | 17:11.04 |
| Robin_Watts | halfie: OK, it's using V=5, R=6 - we don't support that currently. let me look. | 17:11.36 |
| halfie_ | Robin_Watts, in that file I see OE and UE strings which I haven't seen before | 17:13.07 |
| Robin_Watts | we seem to have code for dealing with OE and UE. | 17:16.03 |
| pipitas | V_Tori: you said you've got no PDF spec. The one for PDF-1.7 (ISO 32000) is here: http://www.adobe.com/content/dam/Adobe/en/devnet/pdf/pdfs/PDF32000_2008.pdf | 17:19.25 |
| V_Tori | thanks | 17:20.15 |
| Robin_Watts | The U and O strings are 127 bytes long. That seems odd to me. | 17:22.57 |
| halfie_ | Robin_Watts, I trust Adobe to do crazy things! | 17:25.15 |
| Robin_Watts | V=6 is specified in ISO32000-2, apparently. i.e. PDF 2.0 | 17:26.03 |
| | henrys: Do we own a copy of the PDF 2.0 spec? | 17:26.52 |
| | http://www.iso.org/iso/iso_catalogue/catalogue_ics/catalogue_detail_ics.htm?ics1=37&ics2=100&ics3=99&csnumber=53041 | 17:26.59 |
| henrys | Robin_Watts:not that I know about | 17:27.37 |
| chrisl | Yay! Gold for Wiggins :-) | 17:28.16 |
| henrys | that's great especially after the let down at the road race. | 17:29.36 |
| chrisl | Yeh, the road race course was not a good route for that team | 17:30.06 |
| Robin_Watts | Aha: http://esec-lab.sogeti.com/post/The-undocumented-password-validation-algorithm-of-Adobe-Reader-X | 17:31.35 |
| chrisl | alexcher: is that patch for PPC32 complete? It doesn't seem to address the missing ULONG_MAX and ULLONG_MAX issue? | 17:31.59 |
| | alexcher: ah, scratch that, I see what's going on...... | 17:33.32 |
| Robin_Watts | halfie_: I can't promise when we'll get to look at that. | 17:34.38 |
| | sebras: ping ? | 17:34.42 |
| chrisl | alexcher: I'm happy with the patch, if you want to commit it to master, I'll cherry pick onto the release branch | 17:35.01 |
| halfie_ | Robin_Watts, no problem, I have plenty of stuff to do first before I start dealing with Acrobat X. Thanks! | 17:35.23 |
| V_Tori | if i have a destination like outline->dest.kind == FZ_LINK_GOTO | 17:38.31 |
| | how can i get the page number ? | 17:38.39 |
| | pdf_lookup_page_number requires a pdf_obj as 2nd param | 17:39.05 |
| | and i don't know what to pass | 17:39.12 |
| Robin_Watts | outline->dest.gotor.page ? | 17:40.27 |
| V_Tori | this is an int | 17:40.58 |
| | not a pdf_obj * | 17:41.04 |
| | (in 1.0) | 17:41.07 |
| Robin_Watts | outline->dest.ld.gotor.page, sorry. That's just an int. | 17:41.08 |
| | Why isn't an int what you want? | 17:41.32 |
| V_Tori | pdf_lookup_page_number(pdf_document *doc, pdf_obj *pageobj); | 17:41.58 |
| Robin_Watts | Right. The int in the goto link_dest is in fact the result of calling pdf_lookup_page_number. | 17:42.19 |
| | You just want to go to that page. | 17:42.30 |
| V_Tori | ho | 17:43.16 |
| | i'm stupid | 17:43.21 |
| | i can just get the page... | 17:43.27 |
| | it's just that the api was different in 0.9 | 17:43.37 |
| mvrhel | bbiab | 17:51.06 |
| ray_laptop | Robin_Watts: I saw the comments about num_components for PS, and I'd to explore more. | 18:04.45 |
| | Robin_Watts: I understand that with PS we don't know the actual as we do with PDF, but in clist mode, we know when the page is complete what the count was. | 18:05.51 |
| | Robin_Watts: if we avoid creating the planes and doing the render loop for the planes, that would help, right ? | 18:06.45 |
| henrys | hmm a periodic squeaking about one hertz coming from the macbook | 18:09.34 |
| ray_laptop | mvrhel: is all of the slowdown just with the psdcmyk device that supports the planes ? | 18:09.56 |
| | henrys: hope it's not a failing fan bearing -- watch the cpu temp | 18:10.48 |
| | henrys: or does it go away if you plug in headphones ? | 18:11.11 |
| henrys | yep temp looks okay right now but I'll watch it. | 18:11.25 |
| ray_laptop | henrys: maybe you have a cricket virus ? | 18:11.28 |
| henrys | ;0) | 18:11.55 |
| ray_laptop | Robin_Watts: how about adding a 'type' to fz_document so the client knows if it is 'OK' to cast to pdf_document ? (sort of like gs has with it's confused imager_state/gstate) | 18:20.18 |
| Robin_Watts | ray_laptop: There should be no need to cast, ideally. | 18:27.34 |
| | fz_meta should avoid that. | 18:28.13 |
| | ray_laptop: The problem with speed, is largely, I suspect, in the clist writing phase. | 18:28.39 |
| | ray_laptop: I don't know about which devices are slowed down. I hope it's just psdcmyk. | 18:29.11 |
| | I'm going to test that. | 18:29.14 |
| ray_laptop | Robin_Watts: Since clients may want to access stuff from pdf dicts, I still think some way to know the document type would be nice -- doesn't have to be a 'type' element in the struct, but _something_ | 18:47.29 |
| | Robin_Watts: since DeviceN colors are written to the clist with a 'mask' so that components that are 0 can be skipped, and since we always write a 64-bit mask, then I don't know why clist writing would be much slower for 64 vs. 32 | 19:15.46 |
| | Robin_Watts: if you do performance checks, please use -Z: so we can see the clist writing vs. rendering time comparison between 32 and 64 | 19:16.53 |
| | Robin_Watts: if you decide not to do the performance comparison, let me know and I'll do it | 19:17.19 |
| Robin_Watts | ray_laptop: My plan was simply to run cluster tests with filter=ppmraw with and without the bump to 64. | 19:17.55 |
| | ray_laptop: It's stuff like copy_planes I expect to be slower. | 19:18.18 |
| | we'll be adding 32 extra planes, all 0's into the clist. | 19:18.31 |
| | I am testing a patch now that should make planes that are all the same value take a lot less space in the clist. | 19:19.14 |
| ray_laptop | Robin_Watts: I see -- well, we could add a mask as well for planes that are 0 | 19:19.40 |
| | Robin_Watts: there is a gx_dc_devn_get_nonzero_comps function | 19:20.14 |
| Robin_Watts | ray_laptop: How does that help with bitmaps? | 19:24.19 |
| ray_laptop | doesn't help with that, so your patch to detect planes that are 0 sounds like it is needed | 19:25.43 |
| | Robin_Watts: copy_planes is primarily used for patterns, right ? | 19:27.01 |
| Robin_Watts | Sounds plausible. | 19:27.38 |
| ray_laptop | but for copy_mono (text) and hl fill and stroke, the colors are written more efficiently already (with the mask) | 19:27.44 |
| Robin_Watts | ray_laptop: Patch seems to test out OK. http://git.ghostscript.com/?p=user/robin/ghostpdl.git/.git;a=commitdiff;h=6fb466727ab6cb2558acc69b890cf5b49c69df58 | 19:35.35 |
| | Being as it's clist, if you wouldn't mind casting your eyes over it, I'd appreciate the double check. | 19:36.02 |
| ray_laptop | Robin_Watts: sorry -- was on the phone. I'll had a look. Does it make any difference (performance) ? | 19:51.08 |
| | Robin_Watts: looking at your most recent regression, it doesn't look like it, but your code is way behind origin/master so it's hard to tell | 19:52.00 |
| Robin_Watts | My most recent regression was ppmraw only. | 19:52.20 |
| | I'm trying ppmraw with 64 now. | 19:52.34 |
| ray_laptop | Robin_Watts: I thought we expected the difference with psdcmyk (does ppmraw use planar mode by default ?) | 19:53.31 |
| ray_laptop | didn't think so | 19:53.46 |
| Robin_Watts | I'm testing several things at the moment. | 19:54.41 |
| | The ppmraw vs ppmraw+64 thing is to see if the max components stuff makes a difference to non psdcmyk devices. | 19:55.07 |
| ray_laptop | Robin_Watts: I see. I'll wait for the testing then. The patch looks OK | 19:55.15 |
| | Robin_Watts: I'd be surprised if it did, since we use the Alternate colorspace for DeviceN colors if the device doesn't support devn | 19:56.17 |
| Robin_Watts | I was vaguely hoping that we'd see a performance improvement on the whole clusterpush @ 32 due to the 'all zero' patch, but didn't see it. I'll try a complete push @ 64 in a bit. | 19:56.34 |
| ray_laptop | but it's worth checking | 19:56.37 |
| Robin_Watts | ray_laptop: Yes, I really hope we aren't seeing a slowdown in ppmraw, but that's the purpose of the test. | 19:56.58 |
| | OK. ppmraw @ 32 = 5:34:06, ppmraw @ 64 = 5:38:26. That's well within the experimental error. | 19:57.53 |
| ray_laptop | Robin_Watts: it's probably pretty rare to get constant bitmaps from patterns and text except for planar mode where a plane (component) is blank | 19:57.53 |
| | Robin_Watts: I think running filter=psdcmyk would be worthwhile with 32 and 64 would be worthwhile | 19:59.01 |
| mvrhel | I was wondering about pkmraw Robin_Watts | 19:59.11 |
| | that uses planar | 19:59.14 |
| ray_laptop | since that's where we expected the 'hit' | 19:59.15 |
| mvrhel | I mean plank | 20:00.19 |
| | not pkmraw | 20:00.23 |
| | but again that should be using the alternate tint transform | 20:00.43 |
| | but perhaps just doing what ray_laptop suggests | 20:01.03 |
| Robin_Watts | Let me try a full 64bit run with my patch. | 20:01.37 |
| ray_laptop | I expect the pscdmyk preformance to be similar to tiffsep which is what we have customers using | 20:02.00 |
| Robin_Watts | Dinner time. will check back in later. | 20:02.44 |
| ray_laptop | Robin_Watts: did you update your tree ? (if not, hopefully the other changes you were missing won't make/mask a difference) | 20:09.50 |
| | I haven't looked to see what CMS_DONT_USE_INT64 is on a peeves build | 20:11.49 |
| | hopefully it only affects ppc so missing that patch won't be relevant | 20:16.06 |
| | wow!! with the change to 64, it is 1 hour slower (out of 18.5) about 6% hit. Given that the change to 32 cost us 1/2 hour that takes us from < 18 hrs to 19.7 hours for gs | 20:36.46 |
| Robin_Watts | Urm... | 20:37.09 |
| ray_laptop | and it adds 15 new timeouts | 20:37.29 |
| Robin_Watts | With the change to 64, it's 1 hour slower out of 31 and a quarter. | 20:37.33 |
| ray_laptop | Robin_Watts: I'm looking at the 'gs' time (which is where the 1 hour comes from) | 20:37.58 |
| Robin_Watts | right. | 20:38.02 |
| ray_laptop | Robin_Watts: and I suspect that if you only ran a gs filter=psdcmyk the comparison would be even worse | 20:38.38 |
| Robin_Watts | I suspect if I did that the proportion that changed would be worse, yes. | 20:38.55 |
| ray_laptop | but if the slowdown is seen with psdcmyk, then only those using DeviceN planar devices will (probably) see it | 20:39.25 |
| | as long as other devices aren't affected, then it won't hit everyone. | 20:39.57 |
| Robin_Watts | I'm going to back out my patch and run again at 64 to try and see what the benefit or otherwise of that patch is. | 20:40.20 |
| ray_laptop | I think a run with 16 (or 14), 32 and 64 is psdcmyk is worth doing to make sure that the 30 minute hit we took when we went to 32 was just concentrated in that device. | 20:41.38 |
| | Robin_Watts: can we focus on gs psdcmyk ? | 20:42.06 |
| Robin_Watts | ray_laptop: Let my current test run through, to complete the dataset for now. | 20:42.33 |
| | Then we can run psdcmyk tests at different color maxes. | 20:42.52 |
| | That'll probably be tomorrow. | 20:42.59 |
| ray_laptop | I _really_ don't want customers coming back to us with 9.06 release saying "it's slower than 9.05" | 20:43.05 |
| | Robin_Watts: I'm willing to do the tests of 14, 32 and 64 (with or without your patch) while you are sleeping | 20:43.58 |
| Robin_Watts | Go for it. | 20:44.05 |
| tkamppeter | I am trying to package GS 9.06rc1 now. | 20:44.14 |
| | I have a build problem: I build with --disable-gtk and it tries to build the gsx executable anyway and this fails, probably due to a variable not populated because of --disable-gtk. | 20:45.29 |
| | See output at http://pastebin.ubuntu.com/1124135/ | 20:46.42 |
| ray_laptop | Robin_Watts: MAX_COMPONENTS_IN_DEVN seems to be #defined but never used -- any idea what it's for ? | 20:53.18 |
| | mvrhel: same question for you | 20:53.43 |
| | oops -- I found it in icremap.c (I only searched base/*) | 20:54.40 |
| V_Tori | hmm, should the rotation be managed by the user, now ? | 20:55.00 |
| | that is, I have a member rot, and i use a matrix to perform rotation | 20:55.40 |
| | (before, a pdf_page had a rotation member, iirc) | 20:55.54 |
| | or rotate member | 20:56.08 |
| ray_laptop | bbiaw | 20:58.24 |
| chrisl_away | tkamppeter: Fix for your build problem ---> http://git.ghostscript.com/?p=ghostpdl.git;a=commitdiff;h=d723d72b | 21:18.53 |
| tkamppeter | chrisl_away, thank you for the quick fix, I will try it now. | 21:21.14 |
| Ch3rryC0ke | Hey there-- I'm trying to figure out the best way to work with GitHub and the mupdf repo. Can anyone help? | 21:48.07 |
| sebras | Ch3rryC0ke: I know how to clone and work with the upstream mupdf repo, but I haven't really used github. | 21:48.47 |
| | Ch3rryC0ke: is there any specific problem you are having? | 21:48.59 |
| Ch3rryC0ke | sebras: basically I want to be able to clone the mupdf repo, keep my changes to it, store my changes on github, and still be able to pull in new upstream changes | 21:49.31 |
| rsc | Evening folks...I'm looking for ideas about how to tune ghostscript on ARMv5. At the moment it takes about 10 seconds to process a *.ps -> *.pdf which takes about 1-2 seconds on a regular Intel arch | 21:49.35 |
| Ch3rryC0ke | I tried adding mupdf as a submodule to my repo, which works, but then I'm not sure how to push my changes to the submodule to my repo | 21:50.07 |
| sebras | Ch3rryC0ke: what is the url of the submodule? is that the upstream url? | 21:50.37 |
| | Ch3rryC0ke: if so then you do not need to push any change to the submodule repo. | 21:50.56 |
| ray_laptop | rsc: does your ARM have FP h/w ? | 21:50.57 |
| Ch3rryC0ke | The url of the submodule is git://git.ghostscript.com/mupdf.git | 21:51.05 |
| sebras | Ch3rryC0ke: git submodule will pull all commits from the upstream repo instead. | 21:51.18 |
| Ch3rryC0ke | So what's the best way to do this then? I want to be able to pull commits from upstream as well as store my own commits, and have all that on github | 21:51.47 |
| ray_laptop | rsc: Also, pdfwrite uses 'temp' files, so moving those to a RAMDISK or faster disk can make a BIG difference | 21:51.49 |
| | rsc: if your tempfile drive is slow | 21:52.06 |
| sebras | Ch3rryC0ke: with git submodule you will not have a clone of the upstream repository on github. do you want to have that? | 21:52.31 |
| ray_laptop | rsc: but ARMV5 ? Seriously, aren't 8 and 9 cores common now ? | 21:52.54 |
| Ch3rryC0ke | sebras: I think that's what I want. If that lets me commit my own changes and pull new changes, then yes. | 21:53.00 |
| sebras | Ch3rryC0ke: also git submodule is more useful for one project that includes another probject (like mupdf including freetype e.g.). | 21:53.00 |
| Ch3rryC0ke | sebras: Right I'm building a project that includes mupdf | 21:53.14 |
| rsc | ray_laptop: it's softfloat, I think. And vmstat(1) says, I'm spending the time on CPU, not on I/O at disk | 21:53.16 |
| | ray_laptop: Raspberry Pi ;-) | 21:53.28 |
| Ch3rryC0ke | sebras: I'm building a win 8 metro app based on mupdf | 21:53.29 |
| sebras | Ch3rryC0ke: ok, my suggeestion then is that you have a branch on top of upstreams master branch that you rebase every time up decide to pull down new changes from upstream. | 21:53.55 |
| ray_laptop | rsc: well, pdfwrite uses MUCH more FP than gs does when rendering | 21:53.58 |
| sebras | Ch3rryC0ke: or you have a branch that you merge changes to from upstreams master. | 21:54.28 |
| | Ch3rryC0ke: either way will probably be fine in your case. | 21:54.40 |
| Ch3rryC0ke | sebras: Ok, so to set that up, I would clone the upstream master of mupdf.git, create a local branch, and then push the local branch to github ? How do I do the last step? | 21:55.42 |
| sebras | Ch3rryC0ke: yes that seems like a good approach. | 21:55.59 |
| | Ch3rryC0ke: I'm not sure yet.. *reading github tutorials* :) | 21:56.09 |
| Ch3rryC0ke | Me too ;) I posted a question on stackoverflow about this: http://stackoverflow.com/questions/11751152/how-to-manage-a-project-on-github-that-has-an-external-git-dependency-not-stored | 21:56.41 |
| sebras | Ch3rryC0ke: seems like something along the lines of: git remote add github git@github.com:mupdf-winrt.git; git push github master-branch-with-patches ought to work fine. | 21:57.14 |
| ray_laptop | rsc: you can try 'tuning' the "vmthreshold" to see if reducing the frequency of GC (garbage collection) helps. The default is 1000000 try: -c 16000000 setvmthreshold -f before your input file | 21:58.11 |
| sebras | Ch3rryC0ke: I pieced that together from: http://goo.gl/3lnIe | 21:58.18 |
| Ch3rryC0ke | sebras: thanks, will try that in a sec. So are you familiar with WinRT then? ;) | 21:58.47 |
| ray_laptop | rsc: other than that, I can't think of anything offhand. If you do a profile build (make pg) then you can figure out where the hot spots are and we might be able to suggest something | 22:01.16 |
| rsc | ray_laptop: Features: swp half thumb fastmult vfp edsp java tls - that's what the CPU says | 22:01.39 |
| ray_laptop | rsc: just curious -- what clock rate and memory capacity and speed? We have some ARM boards that we can test on to see if we see similar times, but running on a similar system is better | 22:03.10 |
| rsc | ray_laptop: 700 MHz CPU, 250 MHz GPU, 256 MB SDRAM 400 MHz, shares 16 MB to GPU. | 22:06.02 |
| ray_laptop | rsc: I think we have a similar clock rate. Not sure about SDRAM speed. Is this an available board/system ? | 22:08.27 |
| rsc | ray_laptop: Raspberry Pi, model B (the standard one) | 22:08.48 |
| ray_laptop | rsc: thanks. | 22:08.55 |
| | rsc: that is an ARM11 core | 22:12.25 |
| rsc | raythat one you have? | 22:13.42 |
| ray_laptop | rsc: this doesn't have a disk -- how are temp files handled ? | 22:13.59 |
| ray_laptop | hopes that you don't say 'USB disk' | 22:14.24 |
| rsc | SD card | 22:14.24 |
| | MMC, thus USB disk somehow. | 22:14.47 |
| ray_laptop | rsc: none of us have that one (AFAIK) -- I was just looking it up | 22:15.32 |
| rsc | however, it writes5 MB/s in scp(1) easily onto that SD card | 22:15.47 |
| ray_laptop | rsc: are you using "Raspbian" ? (or if not, what OS?) | 22:16.36 |
| rsc | nope, Fedora. | 22:16.42 |
| Ch3rryC0ke | sebras: that did the trick. A little different always being in the branch but it seems to work, I have a Github repo that has the mupdf code that I can pull in upstream changes and push my branch to github | 22:16.56 |
| | sebras: thanks! | 22:17.06 |
| ray_laptop | rsc: 5 Mb/sec is VERY slow -- most disk drives are 50Mb/sec or better | 22:17.07 |
| | rsc: and SD drives I have seen are at least 200 Mb/s | 22:17.39 |
| | rsc: (that's write time) Read rate is usually > 300 Mb/s | 22:18.02 |
| rsc | ray_laptop: well...it's scp additionally. I should likely do it without scp ;) | 22:18.02 |
| ray_laptop | rsc: 'scp' ? | 22:18.31 |
| rsc | scp tux@bar:foo . | 22:18.41 |
| ray_laptop | rsc: I see -- so that's how you are guessing the SD card timing (timing of an scp execution) ? That's not as good as just copying some existing large file from SD to SD | 22:21.15 |
| rsc | ray_laptop: it only handles one SD | 22:21.33 |
| ray_laptop | rsc: your network speed enters into it. | 22:21.37 |
| rsc | oh, from same to same SD? Can try. | 22:21.43 |
| ray_laptop | rsc: even reading and writing from the same SD should give a better number | 22:21.55 |
| | rsc: I usually just time making a copy of the OS image file (it's usually > 1Mb) | 22:22.32 |
| rsc | 104857600 bytes (105 MB) copied, 4.6091 s, 22.8 MB/s | 22:22.59 |
| ray_laptop | rsc: or you can concatenate 10 copies to a single file :-) | 22:23.02 |
| rsc | (dd from /dev/zero to file) | 22:23.15 |
| | repeating is between 22 and 25 MB/s | 22:23.29 |
| ray_laptop | rsc: OK. That's a decent write time | 22:23.36 |
| rsc | that's even more a Class 10 is made for by standard AFAIK | 22:23.58 |
| ray_laptop | rsc: that is consistent with your IO time mentioned above | 22:24.06 |
| | rsc: so I guess profiling to see where the time is going is what you have to do | 22:24.27 |
| rsc | ray_laptop: is there somewhere a pointer about how to do that? | 22:24.48 |
| ray_laptop | rsc: I haven't done it, but if your gcc supports -g then "make pg" will do a profile build. Then running it will produce a file that 'gprof' can decode | 22:26.41 |
| | rsc: Robin_Watts has done this on ARM, iirc -- but he is in the UK, so it is late for him | 22:27.16 |
| rsc | I'm in UTC+2 ;) | 22:27.30 |
| | ray_laptop: well, -g is supported in Fedora, I think. So will do the "make pg". gprof can be run then on a normal Intel hardware to read that result? | 22:28.17 |
| ray_laptop | well, he usually starts a bit late for London TZ, but check in tomorrow AM and he should be able to help | 22:28.20 |
| | rsc: I'm not sure about running gprof on a different arch to the one that generated it (I mostly use x86 with profiling) | 22:29.19 |
| rsc | oki, doesn't then also matter anymore ;) | 22:29.47 |
| ray_laptop | rsc: ?? I didn't parse that | 22:30.18 |
| rsc | ray_laptop: if gprof needs to be run on the same arch, it's also okay. I'm now used to slow ARM ;) | 22:30.44 |
| ray_laptop | rsc: thanks. Well, give it a try and see what comes up. | 22:31.20 |
| rsc | I'm now starting a build, will likely anyway take the half night | 22:32.08 |
| ray_laptop | rsc: I am in CA, US so I'll be here for a while yet. I'll watch the logs in case you have any info... | 22:32.33 |
| | I have to run an errand now. bbiaw | 22:32.45 |
| rsc | ray_laptop: thanks for your time so far :) | 22:32.57 |
| ray_laptop | rsc: you are welcome. You are the 'pioneer' with doing pdfwrite on a (slow) ARM system | 22:33.37 |
| | AFAIK | 22:33.54 |
| | rsc: BTW, out pdfwrite specialist, "kens" is also in London TZ and he starts earlier. But he doesn't have an ARM system, AFAIK | 22:35.13 |
| | rsc: he may have some ideas about what's slowing down pdfwrite | 22:35.53 |
| | (it sounds rather slow, even for a 700 MHz CPU) | 22:36.19 |
| Robin_Watts | Hey rsc. | 22:45.03 |
| | Just reading the logs now. | 22:45.08 |
| rsc | ah | 22:45.25 |
| Robin_Watts | So you've got a raspberry pi, and you're trying to use gs with pdfwrite? | 22:47.49 |
| rsc | right | 22:48.10 |
| Robin_Watts | I'd recommend using oprofile if you have it in your kernel. | 22:48.14 |
| | an ARM11 chip is ARMv6, but I fear has no hardware FP. | 22:50.01 |
| | and I fear it will be FP that's killing you. | 22:50.39 |
| rsc | "swp half thumb fastmult vfp edsp java tls" in /proc/cpuinfo | 22:51.32 |
| Robin_Watts | vfp = Vector Floating Point. | 22:52.57 |
| | So Professor google says there is hardware fp support in the raspberry pi's chip, but not all distros support it. | 22:54.52 |
| | it's not helped by the fact that ARM abandoned VFP for NEON, so I'd imagine toolchains will be concentrating on NEON now. | 22:55.14 |
| tkamppeter | chrisl_away, the fix works, thank you very much. | 22:55.31 |
| rsc | means I could compile ghostscript with vfp? | 22:55.32 |
| Robin_Watts | When you build gs, are you using -mfpu=vfp ? | 22:55.34 |
| rsc | no. | 22:55.39 |
| | or let me verify. | 22:55.42 |
| | it's a Fedora package ;) | 22:55.47 |
| Robin_Watts | That would be worth a try, I reckon. | 22:55.48 |
| | Ah, right. Raspbian has that set up as the default. | 22:56.18 |
| rsc | -march=armv5te -mfloat-abi=soft, -mfpu isn't set at all | 22:56.47 |
| Robin_Watts | Right. You're using a soft float lib. | 22:57.01 |
| | I'd expect that to be a factor of 10 slower than using hardware FP (in raw FP terms) | 22:57.19 |
| rsc | oki, so I should give a try with -mfpu=vfp while rebuilding? | 22:57.42 |
| Robin_Watts | I would. | 22:57.55 |
| rsc | I'll do so then | 22:58.22 |
| Robin_Watts | Can I ask why you went for fedora rather than rasbian? | 22:59.10 |
| | -march=armv6 | 22:59.26 |
| | -mfpu=vfp | 22:59.28 |
| | -mfloat-abi=hard | 22:59.29 |
| | Those are the recommended Raspbian settings. | 22:59.33 |
| | but you may find that if the libs on your system are expecting soft float, that you'll have problems. | 23:00.10 |
| rsc | which kind of problems do you think of? | 23:02.00 |
| Robin_Watts | Well, if you call a lib function with a floating point value, for example. | 23:02.49 |
| | If your program expects to use hard float, it'll put the value into a hardware fp register and make the call. | 23:03.11 |
| | The library routine will then arrive, and expect it to be in a software floating point register. So you'll be failing to pass proper values. | 23:03.56 |
| tkamppeter | GS 9.06 (+ some GIT, needed for build) uploaded to Ubuntu Quantal (built nicely fast on my laptop with SSD). | 23:05.54 |
| rsc | Robin_Watts: I see... | 23:08.25 |
| ray_laptop | I've completed the performance testing with 14, 32 and 64 GS_CLIENT_COLOR_MAX_COMPONENTS with 'gs filter=psdcmyk'. The results are NOT great. | 23:21.17 |
| henrys | well a comparative profile should show quickly where things are going bad, or do you already know what the problem is? | 23:22.56 |
| mvrhel | ray_laptop: even with just pdf? | 23:23.55 |
| ray_laptop | With 14, the time is 187m -- with 32 the time is 224m, with 64 (I am retesting this) I got 318m | 23:24.27 |
| mvrhel | I don't understand how that could occur with pdf | 23:24.56 |
| ray_laptop | mvrhel: I ran 'gs filter=psdcmyk' (which, AFAIK, is PS + PDF) | 23:25.05 |
| mvrhel | with ps and the fact that we always use the max, I can see | 23:25.07 |
| | ray_laptop: you can limit it to just PS or PDF | 23:25.18 |
| | too | 23:25.20 |
| ray_laptop | mvrhel: PDF supports DeviceN colors | 23:25.21 |
| mvrhel | yes | 23:25.26 |
| ray_laptop | PDF input | 23:25.28 |
| mvrhel | but there is a big diff in how they are handled | 23:25.35 |
| Robin_Watts | mvrhel, ray_laptop: Those results are pretty much bang in line with what we'd expect. | 23:25.39 |
| mvrhel | in the device | 23:25.39 |
| | Robin_Watts, ray_laptop I would suggest we do different testing with pdf vs ps also though | 23:26.28 |
| ray_laptop | Robin_Watts: these are WITHOUT your patch for clist. Note that I get 2 more timeouts with 32 compared to 14, and 18 more with 64 | 23:26.30 |
| Robin_Watts | ray_laptop: Right. | 23:26.39 |
| | It's possible that my patch may help a bit, but it's entirely possible that it might even hinder. | 23:27.02 |
| | (having to check the bitmap is extra work, and checking the bitmap vs just copying the bitmap may not save that much time). But it should make the clists smaller (if that matters at all). | 23:27.45 |
| ray_laptop | Robin_Watts: I'll try your patch with all three configurations next. I'll email results tomorrow (with re-test if results look funky, as 64 looks to me) | 23:27.57 |
| Robin_Watts | gs filter=psdcmyk,pdf should do what you want. | 23:28.20 |
| | NOT gs filter=psdcmyk filter=pdf | 23:28.32 |
| | The other thing to note is that by pushing COMPONENT_MAX up to 64, we force the use of smaller and smaller bands. | 23:29.08 |
| ray_laptop | Robin_Watts: what does filter=pdf do (we don't have a 'pdf' device) | 23:29.15 |
| | Robin_Watts: that's true ONLY with PS input (PDF input knows the num_components and uses it) | 23:29.56 |
| Robin_Watts | ray_laptop: The cluster makes a list of tests to do, in the form: tests_private__customerfiles_blah.ppmraw.300.0 right? | 23:29.57 |
| sebras | Ch3rryC0ke: sure, no problem. I hope this solves your problem properly. | 23:30.18 |
| mvrhel | the filter is on the test name concatenation ray_laptop | 23:30.27 |
| Robin_Watts | ray_laptop: Right, but we're talking here about slowdown that only happens with the PS. | 23:30.30 |
| | So the filter stuff just looks for matches in those names (as mvrhel says) | 23:30.54 |
| mvrhel | I am wondering though if we have any issues with pdf | 23:31.00 |
| | PS we know there is a problem with | 23:31.14 |
| Robin_Watts | mvrhel: It's a test worth doing, but I *really* hope that PDF will be fine. | 23:31.21 |
| mvrhel | me too | 23:31.28 |
| | but ray_laptop has me worked up with his result | 23:31.50 |
| ray_laptop | Robin_Watts: If I understand you, 'filter=pdf' will run ALL devices, but only PDF input OR pdfwrite output ??? | 23:32.35 |
| Robin_Watts | Yes. | 23:32.50 |
| | filter=.pdf is probably better. | 23:32.59 |
| ray_laptop | Robin_Watts: OK, that make sense | 23:33.21 |
| mvrhel | filter=psdcmyk,pdf would probably be the ideal case | 23:33.46 |
| Robin_Watts | or filter=".pdf.psdcmyk" would run only pdf files through psdcmyk | 23:33.46 |
| ray_laptop | Robin_Watts: I'll go ahead with that. | 23:34.12 |
| Robin_Watts | , means "and". Supplying multiple filters means "or". | 23:34.27 |
| | At least in theory :) | 23:34.31 |
| ray_laptop | once the current one is done. I've discovered that (probably to support "abort") I have to wait for one run to complete before submitting the next | 23:34.59 |
| Robin_Watts | There are hairy perl regexps involved, and . might be a magic character. | 23:35.04 |
| ray_laptop | Robin_Watts: I _was_ a bit concerned about that | 23:35.30 |
| Robin_Watts | ray_laptop: You can only have one job queued, as otherwise if the cluster has problems and has to restart, it can't hold 2 copies of the source. | 23:35.52 |
| ray_laptop | every regexp seems to have different quirks (grep, egrep, fgrep, ... mygrep) | 23:36.02 |
| mvrhel | bbiaw | 23:36.22 |
| ray_laptop | Robin_Watts: so it doesn't queue up the 'diff' / 'patch', but immediately pushes the patch, then (eventually) runs it, right ? | 23:37.12 |
| Robin_Watts | ray_laptop: The cluster is based on rsync. | 23:37.35 |
| | so basically a copy of the desired source is rsynced to /home/marcos/cluster/users/whoever/ghostpdl/ on casper, and an entry is put into the queue to test that. | 23:38.25 |
| | If you try and queue a second entry, you overwrite the source, and BadThingsHappen(TM) | 23:38.53 |
| ray_laptop | Robin_Watts: the 'gitpush.sh' doesn't use rsync -- it uses "git push -f cluster stash:refs/heads/$myuser" | 23:39.18 |
| Robin_Watts | ray_laptop: Right. It uses git to get it onto a unix box, and THEN uses rsync :) | 23:39.43 |
| | sebras: Are you Mr Encryption with mupdf ? | 23:40.47 |
| | ISTR that you had a hand in the aes stuff ? | 23:41.14 |
| | off to bed now. | 23:46.53 |
| | sebras: If you're interested in encryption stuff, see earlier in the logs for a new password algorithm. | 23:47.11 |
| | Night all | 23:47.13 |
| ray_laptop | g'nite, Robin_Watts | 23:49.20 |
| | Robin_Watts: (for the logs) toolbin/localcluster/gitpush.sh gs filter='.pdf.psdcmyk' seems to have done what we wanted | 23:54.46 |
| | Forward 1 day (to 2012/08/02)>>> | |
IRC Logs