| | <<<Back 1 day (to 2016/01/21) | 20160122 |
| mvrhel_laptop | ok. I *finally* have it drawing the proper text | 00:05.07 |
| | tor8: so my issue was that having the encoding set to Identity-H does not work | 00:05.35 |
| | that ends up with a two-byte gid being extracted from the Tj content and the attempt to get the cid from that fails | 00:06.46 |
| | when I made my own cmap encoding based upon FT_Get_Char_Index then things worked | 00:07.51 |
| | actually I worded that wrong. | 00:10.28 |
| | it extracts a two-byte cid | 00:10.45 |
| | and then when it goes to pdf_font_cid_to_gid that will fail | 00:11.29 |
| | if I make my own encoding based upon FT_Get_Char_Index then it works | 00:11.55 |
| | tor8: lets try to chat about this Friday morning (your Friday afternoon) | 00:14.42 |
| tor8 | mvrhel_laptop: hm, I'm assuming the 2-byte cid you write to the pdf file is actually the glyph index | 00:15.11 |
| mvrhel_laptop | oh I tor8 | 00:15.23 |
| tor8 | mvrhel_laptop: was just about to shut off the computer and saw your messages :) | 00:15.37 |
| | I've got a few minutes | 00:15.41 |
| mvrhel_laptop | tor8: so looking at pdf_show_string | 00:15.59 |
| tor8 | pdf_font_cid_to_gid for an identity-h encoded should just return the input parameter | 00:16.01 |
| mvrhel_laptop | tor8; but if I have an input that is "0123" | 00:16.25 |
| tor8 | there is no fontdesc->to_ttf_cmap (this is only set for CJK fonts to map from say Adobe-Japan-1 to unicode) | 00:16.31 |
| | and the cid_to_gid array is only set for fonts that have a /CIDToGID array in the font descriptor | 00:16.48 |
| mvrhel_laptop | then in pdf_show_string, my cpt is "0x3031" | 00:16.51 |
| | and so is my cid | 00:16.58 |
| | so the cid "0x3031" is passed to pdf_show_char | 00:17.27 |
| | in pdf_show_string | 00:17.42 |
| tor8 | assume the PDF content stream has: (0123) TJ | 00:18.13 |
| mvrhel_laptop | yes | 00:18.17 |
| tor8 | pdf_decode_cmap will read out "01" as one cpt, value 0x3031 | 00:19.25 |
| mvrhel_laptop | yes | 00:19.34 |
| tor8 | pdf_lookup_cmap will route that through an identity-h encoding and come out with cid 0x3031 | 00:20.02 |
| mvrhel_laptop | yes | 00:20.07 |
| | that is what I was seeing | 00:20.12 |
| tor8 | and for an identity-h font, that would be glyph number 12337 | 00:20.39 |
| | unless this is a CJK font, that's a number way too large :) | 00:20.59 |
| mvrhel_laptop | yes | 00:21.04 |
| tor8 | where does the input string "0123" come from? | 00:21.45 |
| mvrhel_laptop | oh its in the content of that I am using | 00:21.59 |
| tor8 | from the contents.txt stream you use in pdfcreate? | 00:22.41 |
| mvrhel_laptop_ | sorry about that. | 00:23.18 |
| tor8 | I don't expect you to hand write the input strings here, this is going to come from the pdf-write device based on glyph indexes in the fz_text array | 00:23.31 |
| mvrhel_laptop_ | yes in my content | 00:23.34 |
| | so I have the following | 00:23.47 |
| | (slash) F0 72 Tf | 00:24.08 |
| | BT | 00:24.14 |
| | 0 0 Td(01234)Tj | 00:24.19 |
| | ET | 00:24.20 |
| | and F0 is defined on the command line | 00:24.30 |
| tor8 | I'd expect something like <07040b0b0e> TJ to display 'hello' | 00:24.32 |
| mvrhel_laptop_ | oh | 00:24.44 |
| tor8 | the string in the contents.txt should just be raw glyph indices | 00:24.57 |
| mvrhel_laptop_ | ah | 00:25.07 |
| tor8 | emitting identity-h encoded fonts is probably not going to be too useful for 'mutool create' | 00:25.24 |
| mvrhel_laptop_ | so that has been my problem all along? | 00:25.25 |
| tor8 | but it will be necessary for the pdfwrite device, where you're creating the content stream from graphics | 00:25.43 |
| mvrhel_laptop_ | I understand now | 00:25.44 |
| | ok | 00:25.49 |
| tor8 | pdfwrite will take the glyph indices and write them as hex strings in the BT <bladhblah> TJ ET | 00:26.03 |
| mvrhel_laptop_ | ok. That makes sense | 00:26.22 |
| tor8 | the ToUnicode is just so that we can look at that hex string of raw glyph numbers, and get searchable text out of it | 00:26.40 |
| | mvrhel_laptop: in pdf_show_char there's a line | 00:27.09 |
| mvrhel_laptop | tor8: ok. I will fix this up then and use the glyph indices to make sure it works | 00:27.10 |
| tor8 | if (fontdesc->to_unicode) that looks up the glyph index and finds the corresponding unicode value | 00:27.23 |
| mvrhel_laptop | yes I see that | 00:27.30 |
| tor8 | to stuff into the fz_text | 00:27.31 |
| mvrhel_laptop | ok | 00:27.36 |
| tor8 | but the actual glyph index comes directly from the content stream for these fonts | 00:27.50 |
| mvrhel_laptop | tor8: sorry about my ignorance on this. It is much clearer now | 00:28.08 |
| tor8 | mvrhel_laptop: no worries. glad to be of assistance, and I hope you didn't waste too much time being confused :) | 00:28.31 |
| Robin_Watts | And the third run comes in as 1 minute slower too. | 00:29.37 |
| | I think we're happy to take that hit. | 00:29.47 |
| mvrhel_laptop_ | hopefully I can get this wrapped up tomorrow. Thanks for the help tor8. have a good night. | 00:29.51 |
| | my internet seems to be a bit flaky here | 00:30.09 |
| tor8 | mvrhel_laptop: good night! | 00:30.24 |
| rayjj | mvrhel_laptop: I have a question about usefastcolor related to cust 532. Please let me know (call?) when you are available | 04:56.10 |
| mvrhel_laptop | rayjj: tomorrow (Friday) works. winding down for the night now | 05:04.46 |
| rayjj | mvrhel_laptop: ok, great. thanks. What time tomorrow is good for you ? | 05:05.13 |
| rayjj | is winding down, too | 05:05.40 |
| mvrhel_laptop | anytime after 7:30am works | 05:06.00 |
| rayjj | OK. It'll probably be 8:30 or 9:00 if that's OK | 05:07.37 |
| mvrhel_laptop | that will be fine | 05:08.06 |
| rayjj | sleep well... | 05:08.20 |
| mvrhel_laptop | my phone is all charged... | 05:08.24 |
| rayjj | :-) | 05:08.46 |
| | mvrhel_laptop: it'll probably be better here anyway since I like to capture tech issues here and there isn't really any sensitive stuff | 05:09.37 |
| mvrhel_laptop | ok. sounds good | 05:09.49 |
| rayjj | mvrhel_laptop: if you aren't here (or on #artifex) I'll call to wake you up ;-) | 05:10.20 |
| mvrhel_laptop | :) | 05:10.28 |
| rayjj | g'nite | 05:11.04 |
| mvrhel_laptop | night | 05:11.38 |
| tor8 | Robin_Watts: a few commits on tor/master | 11:21.14 |
| | most importantly, support for @font-face which lets us use system fonts with the user css option | 11:21.53 |
| Robin_Watts | tor8: cool. let me look. | 11:27.47 |
| | All but the last one are fine. | 11:29.39 |
| | tor8: See #artifex for some #defines pulled out of SOT. | 11:32.33 |
| | strcasecmp rather than strcmp ? | 11:33.01 |
| | is_bold and is_italic could be 1 bit bitfields? | 11:33.28 |
| tor8 | xhtml is case sensitive | 11:33.36 |
| Robin_Watts | printf? | 11:33.45 |
| tor8 | for logging, we still have a bunch of printfs strewn about the epub code | 11:34.01 |
| Robin_Watts | strcmp for "font-family" seems fine. | 11:34.31 |
| | but strcasecmp for "black" or "bold" etc maybe? | 11:34.52 |
| tor8 | I've never seen css use anything but lowercase, but I could be mistaken | 11:34.54 |
| | bitfields? I don't see the point of that for a struct that will have at most a dozen instances. | 11:35.17 |
| Robin_Watts | do chrome/ie etc accept "Bold" ? | 11:35.21 |
| tor8 | I wouldn't be surprised if browsers accept even worse abominations like "b0Ld" :) | 11:35.55 |
| Robin_Watts | :) | 11:36.15 |
| | I can't see where system fonts are enabled. | 11:36.28 |
| tor8 | having a more fine grained set of weights than plain/bold is overkill for now, IMO | 11:36.41 |
| Robin_Watts | Unless I'm misunderstanding what system fonts are. | 11:36.54 |
| tor8 | Robin_Watts: with the -U option, you can add a user stylesheet that loads system fonts | 11:36.55 |
| Robin_Watts | To me that means "fonts in C:\windows\fonts" etc ? | 11:37.07 |
| tor8 | with a rule like @font-face { font-family: serif; src: url("/usr/share/fonts/truetype/blah.ttf"); } | 11:37.19 |
| Robin_Watts | Ah, ok. | 11:37.24 |
| | Fair enough then. | 11:37.28 |
| tor8 | so not automatic, no. I don't want to climb down that rabbit hole... | 11:37.38 |
| Robin_Watts | Well, other than the strcasecmp thing, which I think would be a good idea, it looks good to me. | 11:38.14 |
| tor8 | Robin_Watts: well, we'd have to strcasecmp every css property then. | 11:38.32 |
| Robin_Watts | You just know that in 6 months someone will complain that it wasn't finding "ArialBold". | 11:38.41 |
| tor8 | and strcmp'ing properties is currently a bit of a hotspot for layout | 11:38.49 |
| | Robin_Watts: I already have a bug complaining about just that :) | 11:39.16 |
| | but now at least we read the whole font-family chain and find at least "sans-serif" at the tail and use that | 11:39.32 |
| | rather than not recognizing "ArialBold" and using "serif" :P | 11:39.47 |
| Robin_Watts | tor8: We could do the same kind of trick we use for PDF names. | 11:39.59 |
| tor8 | (given that the user is smart enough to use font-family: "Arial Bold", sans-serif. | 11:40.06 |
| | Robin_Watts: yeah. we could add a list of common synonyms for serif,sans-serif,monospace for the builtin fonts | 11:40.46 |
| | not sure that we'd actually need it. you're supposed to make the list of fonts like that. | 11:41.03 |
| | with fallbacks in order of preference | 11:41.14 |
| Robin_Watts | tor8: Are css specifiers utf8 in mupdf? | 11:56.23 |
| tor8 | Robin_Watts: yes. everything is utf8. | 11:57.42 |
| Robin_Watts | So, random idea. Whenever we read a css specifier in from the file, we feed it to a sanitizer. | 11:58.07 |
| tor8 | though I believe there are some weird things with encodings in the css spec | 11:58.10 |
| Robin_Watts | "a: foo b:bar c :baz" would get sanitized to "a:foo b:bar c:baz" | 11:58.40 |
| tor8 | Robin_Watts: I don't follow that example | 11:59.29 |
| Robin_Watts | and as part of that process, we can replace known words with dictionary indexes. So it might become "<1>:<24> <2>:<15> <3>:<16>" | 11:59.46 |
| | tor8: I just mean we trim unneeded whitespace. | 11:59.57 |
| | but the cunning part is how we represent dictionary tags, like <1> | 12:00.39 |
| tor8 | we tokenize the css and parse the css grammar, we don't operate on the source text beyond tokenizing it | 12:00.48 |
| | so I'm not sure what would be gained by trimming the whitespace in either selectors or property declarations | 12:01.15 |
| Robin_Watts | oh, maybe I'm misunderstanding then. | 12:01.27 |
| tor8 | now I have considered making a whitelist of property names so we discard any properties we know that we don't handle | 12:01.46 |
| Robin_Watts | I thought that we ended up holding strings like "a:foo b:bar c:baz" for the duration of the file being open? | 12:02.00 |
| tor8 | and then have the properties numbered and just use integer lookups | 12:02.01 |
| | we compile the css selectors and declarations into c structs | 12:02.35 |
| | fz_css_selector and fz_css_property | 12:03.12 |
| Robin_Watts | So the strcmp performance hotspot is in loading, not in layout ? | 12:03.17 |
| tor8 | it's in layout, in a few different places. one to compare tag names with selectors when matching rules. | 12:05.28 |
| Robin_Watts | I am confused then. | 12:05.43 |
| tor8 | the other one to find the properties is the bag of properties when applying the styles after all the rules for a node have been matched | 12:06.08 |
| Robin_Watts | tor8: Right, so we're still doing string lookups when we're operating on nodes. | 12:06.45 |
| | So we're holding strings like "a:foo b:bar c:baz" for the duration of the file being open ? | 12:07.27 |
| tor8 | no, we discard all the xml and css nodes once we've generated the box tree | 12:08.06 |
| | the box tree only has the fz_css_style structs which don't use strings | 12:08.41 |
| Robin_Watts | So what are we string searching for properties then ? | 12:08.45 |
| tor8 | the xml and css nodes when creating the fz_css_style struct | 12:08.59 |
| | 1) we parse the xml into a tree with tag names and attributes as strings | 12:09.22 |
| | 2) we parse the css into a list of rules, with selectors selecting on tags and attributes with strings | 12:09.42 |
| | and with property declarations as a list of key/value pair strings | 12:09.56 |
| | 3) we walk the xml tree, and for each node in the xml we match all the css rules to it to gather a list of properties | 12:10.24 |
| | all the rules that match, their property declarations get copied into a temporary list (with care taken to get the css selector priorities (aka specificity) correct) | 12:11.06 |
| | then we take this list of properties for the tag node and compute a flattened fz_css_style struct and generate the boxes | 12:11.32 |
| | after all this is done, we keep the boxes and toss the xml and css structs | 12:11.44 |
| Robin_Watts | right. Those are all load time operations? | 12:11.51 |
| | i.e. that all happens before layout. | 12:12.02 |
| tor8 | and *then* we run through the boxes to lay out the actual pages | 12:12.04 |
| | the flattened fz_css_style struct has a fz_css_number for all the numbers | 12:12.21 |
| | or a fz_font for the fonts | 12:12.28 |
| Robin_Watts | Ok, so where does the strcmping during layout come into it. | 12:12.35 |
| tor8 | the fz_css_number has a float and a 'unit' (percent, em-size, absolute value, etc) | 12:12.46 |
| | we don't strcmp when doing the layout-layout | 12:13.00 |
| | sorry if I misunderstood your question | 12:13.10 |
| | it's in loading, not layout, yes | 12:13.23 |
| Robin_Watts | right. gotcha. | 12:13.28 |
| tor8 | Robin_Watts: I took a look at harfbuzz | 12:13.52 |
| Robin_Watts | That makes much more sense. The strings are not as long lived as a I thought then. | 12:13.54 |
| tor8 | the thing is *completely* undocumented :( | 12:13.57 |
| Robin_Watts | tor8: urgh. | 12:14.09 |
| | Is there an alternative? | 12:14.13 |
| tor8 | ICU ... but that's easily ten times the bloat | 12:14.32 |
| Robin_Watts | USE - but that's MS. | 12:15.23 |
| | Graphite ? | 12:15.28 |
| tor8 | graphite only works on SIL's own fonts -- they have invented their own alternative to OpenType | 12:15.46 |
| | which is a lot more technically sane, but opentype is here to stay I'm afraid | 12:15.57 |
| | at least that's the case when I first looked at it a few years ago | 12:16.14 |
| Robin_Watts | tor8: right. | 12:16.24 |
| | so, basically we're stuck with harfbuzz. | 12:16.31 |
| tor8 | I think that's our only real choice :( | 12:16.44 |
| | harfbuzz is, ick of all icky things, c++ | 12:16.52 |
| | but with a C interface, at least | 12:16.56 |
| Robin_Watts | OR... we use pango, and let that talk to harfbuzz. | 12:18.46 |
| tor8 | pango doesn't do font styling, it's really made for gui widget labels only | 12:19.16 |
| Robin_Watts | ok. | 12:19.30 |
| tor8 | we want more control than pango provides | 12:19.33 |
| | and pango drags in cairo of all things too, harfbuzz at least lets us do our own fonts | 12:19.51 |
| | Robin_Watts: if you want to play with harfbuzz I can try to get it building as a thirdparty submodule | 12:20.20 |
| Robin_Watts | tor8: C++, urgh. How big is it? :) | 12:21.23 |
| tor8 | 28kloc of c++ and header files | 12:21.56 |
| | but I expect a good few thousand of those won't be used--they're bindings to graphite and uniscribe | 12:22.33 |
| | and directwrite and coretext etc | 12:22.51 |
| Robin_Watts | and it calls calloc etc directly. | 12:23.16 |
| tor8 | yeah... | 12:23.45 |
| | it baffles me, the code looks like 99% c code | 12:24.11 |
| | so why the c++ dependency? | 12:24.16 |
| Robin_Watts | tor8: templates maybe? | 12:24.26 |
| | If you want to try to get it building as a submodule, I'd be up for trying to actually call it. | 12:24.57 |
| tor8 | Robin_Watts: egads, autoconf madness ... but it might be skippable | 12:37.00 |
| | Robin_Watts: and it uses ragel to compile state machines | 12:37.13 |
| Robin_Watts | tor8: Can we do a build on linux machines and then just take the compiled state machines? | 12:48.32 |
| tor8 | Robin_Watts: yeah, I'm thinking of dumping the generated files in an 'artifex' branch | 12:50.18 |
| | Robin_Watts: okay, I have it building | 13:02.55 |
| | but I think I better try a simple example as well | 13:03.05 |
| Robin_Watts | Does it need c++ once you've got it to that stage ? | 13:04.36 |
| tor8 | Robin_Watts: it does... | 13:04.45 |
| | but only to build the library | 13:04.53 |
| | the example should be possible with plain c | 13:05.08 |
| Robin_Watts | I've had some success converting c++ to C in the past. I wonder if it's feasible here. | 13:10.28 |
| tor8 | Robin_Watts: I've got it building, and the SDL harfbuzz example I found works with the mupdf built freetype+harfbuzz combo | 13:11.31 |
| | it took a bit of head scratching to get it to build, but in the end it's pretty simple | 13:11.59 |
| Robin_Watts | cool. Does it use the ucdn stuff in mupdf? | 13:12.15 |
| tor8 | it does | 13:13.39 |
| Robin_Watts | Stick it on a branch in your repo, and I'll have a look after lunch. Thanks. | 13:15.14 |
| tor8 | Robin_Watts: working on it | 13:15.34 |
| | Robin_Watts: on tor/harfbuzz, you should be able to replicate the build in visual studio from the Makethird rules | 13:23.16 |
| Robin_Watts | tor8: So, do we want this stuff living above or below the device interface? | 14:26.36 |
| | Given that we want to do text extraction from the fz_text, I think we want unicode in the fz_text structures. | 14:27.04 |
| | That means the shaping all lives in the device. | 14:27.19 |
| | oh, hmm, fz_text's contain both gid and ucs. | 14:28.16 |
| | Oh, so those will be pre-shaping gids. | 14:32.31 |
| | Well, actually, I guess that's the question. | 14:33.44 |
| | Do we want the fz_text to contain the shaped or the unshaped stuff? | 14:34.04 |
| tor8 | Robin_Watts: the fz_text stuff should contain post-shaping data | 14:34.25 |
| Robin_Watts | tor8: OK. | 14:34.38 |
| | That keeps the shaping code in the document handler. | 14:35.08 |
| tor8 | the cid/ucs pairs may not be a perfect fit, but I follow the XPS convention which also has the same pairs which it calls 'clusters' | 14:35.13 |
| Robin_Watts | tor8: it's seeming sane, the longer I look at it. | 14:35.28 |
| tor8 | it handles one-to-many and many-to-one mappings by using -1 for them | 14:35.35 |
| Robin_Watts | How does fallback work with this? | 14:36.09 |
| tor8 | good question. I don't expect anything below the actual shaping/layout to use the fallback chain | 14:36.36 |
| | once in the device interface, the fallback stuff should be ignored | 14:36.45 |
| Robin_Watts | So the fz_font's in the fz_text's will always point to a single fz_font, not a 'chain' of them ? | 14:37.06 |
| tor8 | it might even be the wrong place to keep it; we may want an array of script+language->fallback font mapping | 14:37.12 |
| Robin_Watts | (or if they do point to a chain of them, we don't follow the chain) | 14:37.25 |
| tor8 | yeah. the fz_font->fallback is only for layout use. | 14:37.27 |
| | exactly. | 14:37.31 |
| Robin_Watts | Ok. | 14:37.32 |
| tor8 | as it stands, only the fz_encode_character_with_fallback looks at the chain | 14:37.44 |
| | and I expect that will all change once you add harfbuzz | 14:37.51 |
| | so feel free to rip it out and do something different | 14:37.57 |
| Robin_Watts | Yeah, still feeling my way a bit, please excuse the dumbass questions. | 14:38.16 |
| tor8 | we might want a fz_font_set that has the fallback chain and use that in the layout and html code instead of a fz_font | 14:38.17 |
| | it all depends on what harfbuzz expects as input. | 14:38.38 |
| | if we lay out a run of arabic text, I would expect to use the arabic font for common punctuation if possible | 14:39.11 |
| | I think harfbuzz does a preprocess (or expects us to have done) classifying runs of text as a given language and script | 14:39.44 |
| | there's a 'guess' function that does it for us | 14:39.51 |
| Robin_Watts | tor8: For things like Word, they sometimes have markup for script and language. | 14:40.26 |
| tor8 | I have *no* idea how the shaping will interact with line breaking | 14:40.30 |
| Robin_Watts | I suspect harfbuzz is just covering the bases. | 14:40.36 |
| tor8 | yeah. eventually we might be able to get that from the HTML and pass it on, but for now just guess | 14:40.50 |
| Robin_Watts | tor8: It shouldn't, unless you are doing hyphenation or anything smart. | 14:41.07 |
| tor8 | right, so we feed it a word at a time? | 14:41.28 |
| | where 'word' is defined as chunk of text between potential line breaks | 14:41.54 |
| Robin_Watts | tor8: Yeah. Shaping never goes further than a word. | 14:41.58 |
| tor8 | alright. | 14:42.06 |
| Robin_Watts | (AFAIK, and I'm sticking to that bit of ignorance) | 14:42.11 |
| tor8 | (worries about hyphenation ... but that'll be another days problem) | 14:42.15 |
| | right, so we should be good if we just feed it the flow fragments? | 14:42.44 |
| Robin_Watts | I believe so. | 14:42.51 |
| tor8 | that would be a relief :) | 14:42.59 |
| pooryorick | Hi, I'm just now getting back to an issue that I asked about just before the New Year, but gave the wrong error message for. | 15:49.43 |
| | When I start ghostscript, I get | 15:49.58 |
| | Unable to open the initial device, quitting. | 15:50.00 |
| kens | You haven't replied ot the questiopns on the bug report, which is why it was closed | 15:50.16 |
| pooryorick | I noticed it was closed when I went to look at it today. | 15:50.32 |
| | I compiled this gs myself, with --prefix | 15:50.57 |
| Robin_Watts | pooryorick: I'm going to take a wild stab and guess that you are trying to use the bbox device, and the bbox device is not built into your copy of gs. | 15:51.05 |
| | What's the exact command you're running? | 15:51.24 |
| kens | I think its in the bug report, I'm just trying to find it | 15:51.37 |
| | 696461 | 15:51.58 |
| pooryorick | bbox was the first command I noticed the error for, yes, but just invoking "gs" also triggers the error. | 15:52.02 |
| kens | gs --h ? | 15:52.27 |
| Robin_Watts | pooryorick: Yes, bbox is the default device in many builds. | 15:52.27 |
| kens | Sorry 696483 | 15:52.39 |
| | RIght, try gs --h | 15:53.21 |
| | Then look to see if the bbox device is in the list | 15:53.32 |
| | Might be -h | 15:53.56 |
| Robin_Watts | gs --help or gs -h I think. | 15:54.11 |
| pooryorick | Yes bbox is in the list. | 15:54.16 |
| kens | doesn't have a Linux instance right now | 15:54.16 |
| | OK so how did you build GS ? | 15:54.30 |
| pooryorick | Apart from --prefix and --disable-gtk, nothing too special. | 15:54.57 |
| | I've got a log of the build. Anything useful I could report? | 15:55.10 |
| kens | AFAIK disable-gtk does nothign because we don't use gtk | 15:55.15 |
| | THe lgo would mean nothign to me, you'd need chrisl for that | 15:55.28 |
| Robin_Watts | pooryorick: What does "Default output device: " say in the gs -h output ? | 15:55.29 |
| chrisl | pooryorick: what happens if you specify an output device? | 15:55.49 |
| pooryorick | default output device: x11 | 15:56.10 |
| chrisl | Are you running X? | 15:56.21 |
| kens | Try gs -sDEVICE=pdfwrite -o out.pdf | 15:56.32 |
| Robin_Watts | pooryorick: Right. so when you run gs without specifying an output device, it's trying to open an X window, and that (presumably) is what's failing. | 15:56.44 |
| kens | Robin_Watts : it also (allegedly) fails with the explicit bbox device | 15:57.03 |
| pooryorick | Oh nuts. Bad $PATH. I'm reporting on a ghostscript other than the one a built. Just a moment. | 15:57.04 |
| kens | Uhuh | 15:57.13 |
| | Lets go back to the start and use the right one :-) | 15:57.25 |
| | Start by doing gs --help (on the correct executable) | 15:57.45 |
| | Then pastebin that | 15:57.51 |
| | The result I mean | 15:57.58 |
| pooryorick | OK, bbox is still listed. Default output device is x11alpha instead of x11 | 15:58.39 |
| kens | Can you pastebin the output so we can look at it please ? | 15:58.51 |
| | Don't paste it here, it'll flood | 15:58.57 |
| chrisl | And "gs -sDEVICE=bbox -dBATCH -dNOPAUSE -c showpage" doesn't work? | 15:59.53 |
| kens | Lets not overwhelm him, one step at a time :-) | 16:00.32 |
| pooryorick | https://pastebin.mozilla.org/8857360 | 16:05.12 |
| kens | OK and if you do : | 16:06.39 |
| | gs -sDEVICE=bbox | 16:06.39 |
| | what happens ? | 16:06.43 |
| | Again pastebin please | 16:07.19 |
| pooryorick | With or without -sDEVICE=bbox, same thing: | 16:07.42 |
| | **** Unable to open the initial device, quitting. | 16:07.44 |
| | Not sure it's worth a pastebin | 16:07.52 |
| kens | I'd prefer to see the whole transcript | 16:08.02 |
| Robin_Watts | including the command invocation. | 16:08.11 |
| kens | When you came on IRC alst time you didn't mention that line and its important | 16:08.20 |
| pooryorick | https://pastebin.mozilla.org/8857361 | 16:08.46 |
| | (sorry about that, previously) | 16:08.58 |
| Robin_Watts | You haven't included the invocation line. | 16:09.26 |
| kens | OK so try : | 16:09.34 |
| | gs -sDEVICE=pdfwrite -o ./out.pdf | 16:09.34 |
| pooryorick | The command invocation was gs -sDEVICE=bbox | 16:09.47 |
| kens | I know that. | 16:09.57 |
| | I want you to try something else | 16:10.04 |
| pooryorick | https://pastebin.mozilla.org/8857362 | 16:10.45 |
| kens | Well, your build is well and truly broken, to be honest and I have no idea why | 16:11.10 |
| | You are failing to install any device, which is, shall we say, unusual | 16:11.44 |
| | How are you at using gdb ? | 16:12.17 |
| chrisl | pooryorick: are you using *just* the 9.18 release? No patches or any changes at all? | 16:12.45 |
| pooryorick | I've used gdb before. Not an expert, but can get by. | 16:12.47 |
| | chrisl, I've made some "reasonable" patches to jpeg_mak and png_mak, which I believe have already been accepted upstream | 16:14.02 |
| | I also drop jpeg/ltmain.sh into ijs | 16:14.25 |
| chrisl | pooryorick: you believe wrong, I closed that bug because you never replied to my follow up question(s) | 16:14.35 |
| kens | Well, I'd suggest that you start by using our software precisely as we release it | 16:14.36 |
| pooryorick | It won't build in my environment as released. | 16:15.06 |
| kens | Why ? | 16:15.17 |
| | And in that case, what *is* your environment | 16:15.32 |
| pooryorick | the jpeg_mak and png_mak issues are problems with CPPFLAGS pollution | 16:15.37 |
| kens | As chrisl said, we can't uynderstand that *if* you are using our sources as supplied | 16:15.59 |
| pooryorick | The environment is a software collection that lives in an alternate prefix that attempts to make minimal changes to software distributions. | 16:16.33 |
| kens | If you don't use the source as we supply it, its pretty hard for us to match what you are doing, and that makes it well nigh impossible to figure out your problem | 16:16.47 |
| pooryorick | Let me find that other bug report that explains the CPPFLAGS issue | 16:17.00 |
| kens | I already found it, you didn't reply on that bug either so it was closed too | 16:17.15 |
| Robin_Watts | pooryorick: As supplied gs has all the libs it requires within itself. Are you attempting to use libraries from outside gs? | 16:17.39 |
| pooryorick | I build a lot of software, and sometimes lose tracks of bugs I've filed upstream. | 16:17.44 |
| Robin_Watts | (like using system pnglib, or zlib etc?) | 16:17.46 |
| kens | pooryorick : gs should build 'out of the box' using its own libraries. If you change anything at all then we won't guarantee it will work | 16:18.19 |
| pooryorick | I try to build gs with the included libraries, but the ghostscript build pulls in the wrong headers because it prioritizes CPPFLAGS directories over its own directories. | 16:18.28 |
| kens | SO what OS is ths ? What compiler ? What make ? etc | 16:18.34 |
| pooryorick | This collection actually runs on a variety of OS's, and is mostly OS-independent. It does still use the libc from the OS, at this point. | 16:19.12 |
| kens | I cna't comment on the Linux build system, but it works for me on several Linux distributions without problem | 16:19.22 |
| pooryorick | I find lots of bugs in lots of software due to the alternate prefix. | 16:19.40 |
| kens | has no idea what htat means | 16:19.51 |
| | Why don't you start by *not* meddling with the build ? | 16:20.04 |
| pooryorick | If I start by not meddling with the build, it doesn't build. | 16:20.19 |
| Robin_Watts | pooryorick: It seems entirely reasonable that gs should look at include dirs specified in CPPFLAGS before its own. | 16:20.25 |
| kens | We seem to be going in circles here | 16:20.28 |
| Robin_Watts | pooryorick: How about you build a vanilla version without changing CPPFLAGS ? | 16:20.50 |
| pooryorick | Robin_Watts, no, local build includes should take priority over CPPFLAGS. | 16:20.55 |
| kens | what Robin_Watts said | 16:21.04 |
| | Is what I mean | 16:21.08 |
| pooryorick | CPPFLAGS is for third-party includes. | 16:21.33 |
| Robin_Watts | pooryorick: It sounds to me like you're saying "If I build your software using some strange CPPFLAGS, then it doesn't build" | 16:21.45 |
| pooryorick | If it takes priority over local includes, that's a problem. | 16:21.47 |
| Robin_Watts | and the answer then is "Don't give it crap CPPFLAGS". | 16:21.56 |
| | Again, you're wrong. | 16:22.08 |
| pooryorick | The CPPFLAGS are necessary to point to needed third-party headers. | 16:22.16 |
| Robin_Watts | pooryorick: EXACTLY. | 16:22.24 |
| pooryorick | ? | 16:22.29 |
| Robin_Watts | If I *do* want to try to build gs using third-party libraries, rather than the ones that it comes bundled with, then I need it to look at those libraries before it looks at mine. Thus the CPPFLAGS must get looked at before the local includes. | 16:23.17 |
| pooryorick | Let's say a build includes myprivate.h, and also needs libpublc.h from a third party. What if a different libprivate.h also lives in the directory where libpublic.h is found? | 16:23.30 |
| Robin_Watts | But we're telling you NOT to do that for the first attempt. | 16:23.31 |
| pooryorick | I can build without CPPFLAGS, but then gs picks up bogus .h files from /usr/include. | 16:24.11 |
| | I'll try it right now, and report the results. | 16:24.33 |
| | For one thing, it'll pick up the wrong X11 headers. | 16:26.16 |
| | Building now... | 16:26.50 |
| | BWT, I've been building various versions of gs in this collection for years. | 16:28.10 |
| rayjj | mvrhel_laptop: are you available to discuss the link_profile use when -dUseFastColor is used ? | 16:47.52 |
| mvrhel_laptop | rayjj: sure | 16:48.02 |
| rayjj | mvrhel_laptop: So, when it comes into gsicc_get_link from gx_concretize_ICC it is getting an "index" from gsicc_get_default_type of value 12 (gs_color_space_ICC) | 16:55.03 |
| mvrhel_laptop | hold on let me look | 16:55.31 |
| rayjj | mvrhel_laptop: line 698 then tests for index < gs_color_space_index_DevicePixel and ignores dev_profile->usefastcolor (which == 1) | 16:56.44 |
| | mvrhel_laptop: now the ICC profile in question (in gs_input_profile) has data_cs == gsRGB | 16:57.51 |
| mvrhel_laptop | ok. so the "profile" in this case is not really an icc profile | 16:58.53 |
| rayjj | mvrhel_laptop: so we'd really like to use RGB and *not* take the time to form a link. On the Raspberry Pi it is 0.7 sec, and on their CPU it is 1.5 seconds | 16:59.06 |
| mvrhel_laptop | rayjj: oh. I suppose we could do the check for identiry first | 17:00.50 |
| rayjj | mvrhel_laptop: it comes from an ICC profile embedded in the file, with /Alternate /DeviceRGB | 17:00.54 |
| mvrhel_laptop | rayjj: do you have the file handy. I want to check something | 17:02.27 |
| rayjj | I have a "decoded" version -- how do you want it (email?) | 17:03.00 |
| | the original (non decoded) was in an email I forwarded to support | 17:03.35 |
| mvrhel_laptop | ok let me see | 17:03.52 |
| rayjj | on 1/20 Subject: Fwd: File giving performance problems | 17:04.27 |
| mvrhel_laptop | I found it | 17:04.52 |
| rayjj | mvrhel_laptop: let me know if you don't have it | 17:05.00 |
| | ok | 17:05.02 |
| | mvrhel_laptop: there are 2 calls to gsicc_get_link_profile for "fillpage" first that _do_ use the nocm methods, the third call is from concretize_ICC | 17:06.22 |
| mvrhel_laptop | rayjj: ok. I understand. | 17:07.41 |
| | building now. hold on a sec | 17:07.49 |
| | rayjj: ok. so it looks like there is an issue here | 17:12.47 |
| Robin_Watts | tor8: Am I going blind? Where is font set in draw_flow_box ? | 17:12.53 |
| mvrhel_laptop | rayjj: let me spend a little time with this | 17:13.40 |
| rayjj | mvrhel_laptop: the complications are that this is a /Separation /Black that ends up resolving the ICCBased (object 32) and a function that maps the RGB ICCBased to Black | 17:14.50 |
| mvrhel_laptop | I am wondering if the separation color space is the source of the issue.... | 17:14.52 |
| | yes | 17:14.54 |
| rayjj | (for the equivalent color) | 17:14.58 |
| mvrhel_laptop | I suspect I am missing something for the sep case (and likely deviceN then too) | 17:15.25 |
| rayjj | mvrhel_laptop: the problem is in the concretize_ICC -- I suspect that the spot equivalent color mapping that follows is *not* a problem | 17:15.47 |
| mvrhel_laptop | rayjj: eventually that is the problem | 17:16.01 |
| | please let me spend a little time with it | 17:16.09 |
| rayjj | mvrhel_laptop: np. Just mention me and it'll "ding" (I have the sound on) | 17:17.01 |
| mvrhel_laptop | oh. this is not a Device*** color but an ICC color | 17:19.45 |
| | rayjj: Its been too long. Reading my documentation, -dUseFastColor just avoids doing ICC on DeviceGray, DeviceRGB and DeviceCMYK | 17:20.56 |
| | if a source color was defined to be ICC then the real CM is used | 17:21.21 |
| | In this way we emulate what GS used to do prior to 9.0 | 17:22.05 |
| | now, we could add an option where we avoid doing all CM with ICC profiles | 17:22.34 |
| | of course if we had a CIELAB profile that would be an issue | 17:22.46 |
| | but we could detect that one | 17:22.58 |
| rayjj | mvrhel_laptop: if the /Alternate is /DeviceRGB can we use that ? | 17:23.42 |
| mvrhel_laptop | oh perhaps. let me make sure there really is an ICC profile in here first. This was my suspicion | 17:24.16 |
| | ok. yes there is an RGB profile in here | 17:24.30 |
| rayjj | mvrhel_laptop: it has /Length 3144, so there probably is actually a profile | 17:24.48 |
| mvrhel_laptop | an sRGB profile with that length | 17:24.58 |
| rayjj | I assume that if the profile sets gsRGB, we could trust it ? | 17:25.26 |
| | in the input_profile->data_cs | 17:25.42 |
| mvrhel_laptop | oh. we could do that | 17:25.54 |
| rayjj | (and similarly for gray and cmyk) | 17:26.15 |
| mvrhel_laptop | rayjj: let me check something to make sure that this is working the way I think it is | 17:26.29 |
| | hold on for a couple minutes | 17:26.34 |
| rayjj | mvrhel_laptop: OK. | 17:26.42 |
| mvrhel_laptop | I just want to test out some ICC test files I have, but using -dUseFastColor | 17:28.09 |
| | rayjj: ok. yes that is the way it works. With -dUseFastColor, source color spaces that are DeviceGray, DeviceRGB or DeviceCMYK will not use ICC color management. ICC color spaces will use it | 17:31.49 |
| | We could add another option to avoid using ICC CM even for ICC color spaces | 17:32.08 |
| rayjj | mvrhel_laptop BTW, in "pd14_ok_to_optimize" I used the gsRGB, gsGRAY and gsCMYK to detect "acceptable" colorspaces | 17:32.18 |
| mvrhel_laptop | there are going to be some files that will be radically different from Adobe in those cases | 17:32.21 |
| | I am not sure if the customer would want to do that | 17:33.03 |
| rayjj | mvrhel_laptop: note that pdf14 optimization (skipping bands that don't need the pdf14 compositor) has been in for a long time, and works | 17:33.28 |
| mvrhel_laptop | rayjj: I am confused. Is this related to pdf14? | 17:34.09 |
| rayjj | mvrhel_laptop: sorry for the confusion. I found that when looking for other uses of gsRGB (with grep) | 17:34.45 |
| | mvrhel_laptop: this file _does_ use transparency on page 1, but that's another issue | 17:35.21 |
| mvrhel_laptop | rayjj: so the solution would be that we could add another option to avoid using ICC profiles all together. Basically doing what Mupdf doese | 17:35.50 |
| | except for the CIELAB case | 17:36.07 |
| | again though, there will be files that will differ radically from Adobe rendering | 17:36.30 |
| rayjj | mvrhel_laptop: so use the gsicc_nocm_get_link ? | 17:38.53 |
| | mvrhel_laptop: the current line 701 | 17:39.36 |
| mvrhel_laptop | rayjj: that would be the end result. I think we would avoid doing the ICC colorspace install and replace it with one of the Device color spaces | 17:40.34 |
| rayjj | I have to run an errand. bbiab | 17:45.31 |
| mvrhel_laptop | ok | 17:48.10 |
| | bbiab | 18:04.10 |
| Robin_Watts | Ugh. | 18:43.42 |
| | Mirroring is buggering all this up at the moment. :( | 18:43.57 |
| | We end up splitting lots of R2L text into 1 char long fragments for each individual char if they are mirrored. | 18:44.49 |
| | I hope we can get harfbuzz to do the mirroring as part of the shaping, and thus avoid that. But that means changing the bidi code... | 18:45.22 |
| | actually... that wasn't too hard. | 18:59.24 |
| HenryStiles | Robin_Watts: what became of accurate curves? | 19:37.29 |
| mvrhel_laptop | off to lunch | 19:39.46 |
| HenryStiles | I have a PCL commit on my personal repo for review, getting late in Europe so maybe rayjj can take a look, not a complicated change | 19:40.01 |
| Robin_Watts | I'll look unless ray is there already. | 19:53.57 |
| | [a] = b | 19:55.45 |
| | a is the index. | 19:55.51 |
| | so I reckon your description is wrong for resolutions. | 19:56.06 |
| | Or I'm failing to grok it. | 19:56.43 |
| | resolutions[x] = round_up_to_next_legal_resolution(600/(x+1)) ? | 19:58.37 |
| | but the code looks fine. | 20:01.05 |
| pooryorick | building without CPPFLAGS at all resulted in a working ghostscript, so thank you for that! | 20:34.46 |
| Robin_Watts | pooryorick: Fab. | 20:36.35 |
| HenryStiles | Robin_Watts: yeah the explanation is off... | 20:38.36 |
| | thanks | 20:39.23 |
| pooryorick | I still think CPPFLAGS shouldn't take priority over project includes. I guess ghostscript uses CPPFLAGS as the "switch" to decide whether to use included libraries or system libraries. | 20:40.23 |
| | That seems brittle to me. | 20:40.48 |
| | I've run into the scenario of CPPFLAGS overriding project includes often enough that I'm pretty convinced it's a bug. | 20:41.47 |
| | It's often the case that some include directory must be added to a build in order to make some .h file available, but another .h file in that same directory conflicts with one of the build .h files. | 20:44.19 |
| | CPPFLAGS is the appropriate place to specify such an include, and the build should take care to protect itself against .h contamination. | 20:45.07 |
| HenryStiles | hmm not commit email coming back | 21:27.52 |
| | s/not/no | 21:27.57 |
| rayjj | mvrhel_laptop: Thanks for the help earlier. I patched the HEAD gsicc_cache file as we discussed, and it works fine and does use the nocm as expected. The customer did timings while I was doing this and their timings didn't show the expected delay, so I looked at their code/simulator | 22:19.06 |
| | mvrhel_laptop: their stuff was already hacked enough (as chrisl mentioned, but wasn't sure) that it was already not doing the expensive creation of the link using lcms, but was using nocm :-( | 22:20.04 |
| | mvrhel_laptop: but it wasn't a waste since I have now go the code on the Pi without the ICC overhead, and am able to compare better, and identify some stuff they are doing that is slower than expected | 22:21.22 |
| | I just need to find out what they are doing that is slowing them down (relative to the Pi) | 22:21.46 |
| | that and make sure it is not something we improved since 9.06 (building that on the Pi, now -- takes about 40 minutes) | 22:25.06 |
| | bbiaw | 22:25.20 |
| Robin_Watts | HenryStiles: Yes. We reckon mailing lists are broken. | 23:33.06 |
| HenryStiles | Robin_Watts: we'll probably accomplish more ;-) | 23:38.58 |
| | Forward 1 day (to 2016/01/23)>>> | |
IRC Logs