| | <<<Back 1 day (to 2018/03/05) | 20180306 |
| Robin_Watts | ray_laptop: I'll think about that tomorrow. | 00:11.16 |
| ray_laptop | mvrhel: RobinWatts: It's hard to catch a thread causing problems. Any ideas are welcome. I KNOW that it is due to ICC color transform, both because it only happens if I enable gscms_is_threadsafe. | 04:08.35 |
| | mvrhel: Robin_Watts: the problems are always 16-bit colors being (over)written into a memory block, polluting the headers for the chunk memory manager. My concern is that when we share a link, that link has a context, but the context has a "MemPool" that it allocates small blocks from. That is *NOT* threadsafe | 04:12.19 |
| | I look forward to discussing this tomorrow. Maybe after sleeping on it, I'll think of a way to "catch" it. Note, that it may be something totally different, but from the desk view, this seems wrong | 04:14.09 |
| | In the meantime, going to try helgrind again. | 04:14.32 |
| | although I suspect that helgrind (which is designed to spot race conditions) may not spot buffer overwrites from an area alloc'ed by one thread being modified by another thread since that is a primary thread communication method | 04:46.56 |
| kens | So, given the status of zlib and libtiff, do you want to review those before we do a release candidate chrisl ? | 10:20.26 |
| chrisl | I'm updating zlib now. I'll take another look at libtiff in a bit | 10:20.58 |
| kens | OK was just thinking ahead a bit about what to do | 10:21.25 |
| chrisl | zlib, as expected, was a no effort change - so far | 10:21.38 |
| kens | I'm not surprised, its always been pretty straight forward. | 10:21.55 |
| chrisl | It's clustering now, and I'm just waiting on Win10 to talk to me again, so I can do a test Windows build | 10:22.36 |
| kens | Given your reply about libtiff (and the nature of teh CVEs) I was wondering if its even worth bothering with it before the release. We could do it after and that would give us time to sort ou any nasties that crop up before the next release | 10:22.41 |
| | Before *our* next release I mean | 10:23.04 |
| chrisl | Yes, I am inclined to not do libtiff right now, but I thought I'd double check on it | 10:23.31 |
| kens | Makes sense, OK. We'll need some of Robin's time to run release tests, won't we ? | 10:23.59 |
| chrisl | Hopefully just minutes here and there | 10:24.19 |
| Robin_Watts | Either I need to run them, or other people can. | 10:24.23 |
| kens | Yeah I thought it wouldn't be much | 10:24.27 |
| Robin_Watts | It's not hard to set up. | 10:24.28 |
| | I am going on holiday next saturday for 4 nights. | 10:24.47 |
| kens | I've just recalled that Henry is on vacation for the next few days, so I guess there's no rush, we'll probably need him to review the PCL diffs | 10:24.56 |
| Robin_Watts | so it would probably be wise to have at least tried a test run before then. | 10:25.13 |
| kens | Robin_Watts : then I guess chrisl and I need a tutorial on what you do :-) | 10:25.16 |
| | Because I think you and Henry's vacations will overlap | 10:25.34 |
| Robin_Watts | kens: There is a twiki page, I believe. | 10:25.35 |
| kens | I think you're right | 10:25.41 |
| kens | goes to look | 10:25.45 |
| chrisl | I've done it before following the twiki - it's more if we want anything special/clever doing | 10:26.02 |
| kens | I think I'm happy with repeating what we did last time | 10:26.20 |
| Robin_Watts | https://twiki.ghostscript.com/do/view/Ghostscript/ClusterHowTo | 10:26.52 |
| kens | Hmm, can't immediately see it on the twiki, any clues ? (feel free to use #artifex) | 10:26.56 |
| | LOL thanks | 10:27.00 |
| Robin_Watts | chrisl: Sure. I have time for that. My work for phase 1 of the SSE stuff is done. | 10:27.21 |
| kens | Probably it would be good to do one run before Friday I think | 10:28.03 |
| chrisl | Actually, there's more in that twiki page than I remember, so I think we're good | 10:28.07 |
| kens | OK if you're happy then we're good | 10:28.41 |
| | I guess I'll go think about identifying paragraphs and columns | 10:29.05 |
| chrisl | I'm hopeing all we'll need to do is vary the bmpcmp parameters, and whether we're culling, and that's all covered | 10:29.49 |
| Robin_Watts | kens: been there, done that, had the nightmares. | 10:29.51 |
| kens | Yeah well its never going to be perfect | 10:30.06 |
| chrisl | <sigh> the windows bulid fails with the new zlib :-( | 10:42.19 |
| kens | Well that's not good | 10:42.28 |
| chrisl | But the failure is on undefined symbols that I cannot find anywhere..... | 10:46.23 |
| kens | That's, odd..... | 10:46.35 |
| chrisl | A grep throws up the .obj file containing the symbols, but nothing else - weird | 10:48.10 |
| kens | Would it help for someone else to look too ? | 10:48.27 |
| chrisl | If you have time - the commit is on master on my repo | 10:48.51 |
| kens | I'll go fetch it , migth take me a fwe minutes | 10:49.03 |
| chrisl | I'm going to get a coffee | 10:49.20 |
| kens | :-) | 10:49.24 |
| | Yep, unresolved symbols s_error and s_verbose | 10:51.25 |
| | In ftgzip.... | 10:51.34 |
| | Bah z_error, I can't read... | 10:53.09 |
| | And indeed, not defined or used in there.... | 10:53.29 |
| | OK in zutil.h in ghostpdl/freetype/src/gzip | 10:54.36 |
| | They do seem to be defined in zlib as well, in zutil.h | 10:55.00 |
| | OK So zlib defines them is ZLIB_DEBUG is true, freetyp if DEBUG is true. I wonder if that's a problem. | 10:56.22 |
| | Oh, FreeType's zutil.c doesn't declare them at all | 10:57.45 |
| | chrisl is htis the latest version of FreeType ? | 11:01.32 |
| chrisl | No | 11:01.43 |
| kens | Umm, that might be the problem | 11:01.51 |
| | FreeType's zutil.c should declare z_error and z_verbose to match zlib's I think | 11:02.15 |
| | If I do that it ciomipiles | 11:02.21 |
| | I think that actually these are not used in the FreeType code, so the compiler is being dumb | 11:02.59 |
| chrisl | Okay, I'm inclined to add that, and update Freetype after the release - I was planning that anyway | 11:03.09 |
| kens | Does this fail on a release build ? I think it should be OK | 11:03.25 |
| chrisl | I didn't try it - just used the default in VS | 11:04.00 |
| kens | I'll try it in a second | 11:04.13 |
| | Just want to see where this is used (if at all) | 11:04.20 |
| | Well, that's bizarre. NOw I can't get it to fail..... | 11:05.46 |
| kens | does clean and rebuild | 11:06.03 |
| chrisl | Still have your change in it? | 11:06.06 |
| kens | No, I removed it back to your commit to test release builds, but I ran a debug build first to check it failed. And it didn't..... | 11:06.33 |
| chrisl | release build works | 11:08.35 |
| kens | Yeah thought it would | 11:09.51 |
| | Drat, VS restored my change that's why it didn't fail | 11:10.21 |
| | I wonder if the clash is caused by having m,ultiple zutil.obj files | 11:11.06 |
| chrisl | It hasn't been a problem before | 11:11.22 |
| kens | True, but I wonder if zlib has changed | 11:11.35 |
| | z_error and z_verbose are now described with 'ZLIB_INTERNAL' | 11:12.07 |
| chrisl | Ah, so they are no longer part of the ABI | 11:12.24 |
| kens | The ones in FreeType aren't. | 11:12.25 |
| | Yeah that's what I'm thining | 11:12.33 |
| | thinking* | 11:12.38 |
| | I don't know if that means FT has been amended to deal with this | 11:13.33 |
| chrisl | Well, I'm confused..... | 11:15.10 |
| kens | I am somewhat also | 11:15.31 |
| chrisl | ZLIB_INTERNAL is only defined if HAVE_HIDDEN is defined - and I can't see where that's happening | 11:15.59 |
| kens | You're ahead of me there | 11:16.11 |
| chrisl | top of zlib/zutil.h | 11:16.27 |
| kens | Oh yes, I knew abotu the HIDDEN, no idea whare its defined | 11:17.06 |
| | Bonkers; the linker says z_verbose are undefined in ftgzip.c, function inflate_codes. There is no such function in that source file..... | 11:18.30 |
| | Its all macros, that's why | 11:20.54 |
| | Tracev() seems to use it | 11:21.03 |
| | And indeed that's back in zutil.h | 11:21.26 |
| | So we coudl 'fix' FreeType by knocking otu the diagnostic functions | 11:21.51 |
| | Just remove the #ifdef DEBUG...#else and teh #endif | 11:22.07 |
| chrisl | Well, I would be happier if I understood how ZLIB_INTERNAL was getting defined | 11:23.06 |
| kens | Yes that builds for me, what do you think? I feel a little happier about that than defining stuff, I'm not sure why really | 11:23.08 |
| | Yes... that's somewhat odd, I can't see it being defined immediately | 11:24.16 |
| | VS seems to think its not set | 11:26.30 |
| | But that's just the editor, so not reliable | 11:26.41 |
| chrisl | As I said, it's based on HAVE_HIDDEN being defined | 11:27.05 |
| kens | Yes sorry I mean VS thinks HAVE_HIDDEN is not set | 11:27.23 |
| | But its not totally reliable, I've saeen it fooled before | 11:27.39 |
| chrisl | So, let's pull out the freetype diagostic stuff which we never use anyway, and then I'll update freetype after the release goes | 11:29.09 |
| kens | I think that's easiest. | 11:29.19 |
| | I do wonder if its simply that we don't compile the zlib zutil.c because we already compiled the FreeType one. | 11:29.41 |
| | And the FreeType one doesn't contain the definitions of those suymbols. | 11:29.58 |
| chrisl | I can look into that | 11:30.28 |
| | But then I'd wonder how it works on Linux.... | 11:31.00 |
| kens | I wonder where they were previosuly defined, since they clearly aren't in zutil.c in the FT code | 11:31.01 |
| | Is the compilation order different on Linux ? | 11:31.27 |
| chrisl | makefiles are all the same for those things | 11:31.52 |
| kens | Must admit I can't see why it would be | 11:31.52 |
| chrisl | Um, we don't build the freetype zutil.c | 11:33.23 |
| kens | Then I'm utterly confused | 11:33.36 |
| | Because I defined the symbols in the FreeType zutil.c and the problem went away | 11:34.06 |
| chrisl | Well, it ain't in the build | 11:35.35 |
| kens | I think we oculd wast e too mcuh time on this | 11:35.52 |
| | Like you said, we never use teh FT diagnostics so lets just pull that from the build | 11:36.06 |
| | While we do a release | 11:36.12 |
| | Then we can talk to Werner or check out the current FT code and see what we need to do at leisure | 11:36.32 |
| chrisl | Actually, based on what you said, I suspect I *do* know the problem.... | 11:38.22 |
| kens | Oh, well you're ahead of me again then | 11:38.34 |
| | :) | 11:38.40 |
| chrisl | I guess the ordering of the includes search paths means we're picking up freetype's zutil.h rather than zlib's | 11:39.12 |
| kens | Ah! That would probably do it | 11:39.27 |
| chrisl | But then I'm also struggling to see where that's happening, so.... | 11:41.22 |
| kens | I don't see how FT's gzip can work. It doesn't seem to define thos symbols anywhere | 11:42.34 |
| | OK so rlease will be OK (or anywhere when DEBUG is not defined) but that's silly | 11:42.59 |
| chrisl | Well, that didn't help, so we'll go with the temporary hacking of freetype | 11:53.10 |
| kens | You tried ot upgrade FT ? | 11:54.06 |
| chrisl | No, I tried changing the include paths | 11:54.21 |
| kens | Oh OK | 11:54.26 |
| | I'm not sure I'd fancy upgrading FT just before a release even if it did work | 11:55.17 |
| chrisl | So, removing the DEBUG section from FT's zutil.h works - but I can't work out how | 11:57.53 |
| kens | Well, it redefines Assert and Trace to no-ops | 11:58.16 |
| | And those are what's used in the code | 11:58.25 |
| chrisl | But how are we getting to that file??? | 11:58.37 |
| kens | Ah, now that I cannot explain :-( | 11:58.48 |
| | I also can't explain how the code could ever work in zutil.h | 11:58.59 |
| chrisl | It's path isn't in any search path I can find for ftgzip.c | 11:59.18 |
| kens | Because if DEBUG is defined we use z_error and z_verbose, and as far as I can see, those not defined *anywhere* in the FT gzip code. | 11:59.30 |
| chrisl | Well, we aren't using the FT gzip code | 12:00.08 |
| kens | Then why are we compiling ftgzip ? | 12:00.20 |
| chrisl | ftzip is a glue layer for either calling out to libz or using the included zlib bits | 12:01.18 |
| kens | Hmm, OK then | 12:01.28 |
| chrisl | But we aren't, or shouldn't be, using any other files in that directory | 12:01.48 |
| kens | I think we build infcodes.c and friends, at the very least | 12:02.01 |
| chrisl | Nope | 12:02.19 |
| kens | But possibly not. | 12:02.19 |
| | Hmmm | 12:02.57 |
| | I thought I saw them throwing errors when I removed the diagnostic functions | 12:03.23 |
| | But I could be mis-remembering | 12:03.30 |
| | I'll try it again | 12:03.36 |
| | I'm struggling to see anywhere in ftgzip.c which uses them | 12:03.47 |
| | Might be FT_THROW I guess | 12:04.16 |
| | chrisl so if I remove teh diagnostic section altogether, thereby droppign the definitions of Assert and Trac ect, then I get a compilation error in ghostpdl/freetype/src/gzip/infcodes.c | 12:06.28 |
| | So we are compiling some of the FT gzip modules I think | 12:06.56 |
| | And the fact that we are compiling those modules means that we need the definition of Assert and Tracev which use z_error and z_verbose | 12:07.26 |
| chrisl | That just makes no sense :-( | 12:08.20 |
| kens | The only place I can immediately see those mind you, is in zlib.mak | 12:08.21 |
| chrisl | There isn't an infcodes.c in the VS build output for me | 12:09.37 |
| kens | Yeah I think I have some out of date something here | 12:10.06 |
| | I've just cleaned everything out, reset to master and reconverted the solution | 12:10.22 |
| | Starting again basically | 12:10.29 |
| | I don't think I believe what I'm seeing here | 12:10.57 |
| | There will be a delay while everything rebuilds (again) | 12:11.22 |
| | No, I'm still seeing it build ft's infcodes.c | 12:12.43 |
| chrisl | Well, not for me..... | 12:13.08 |
| kens | Urk, its building it while its supposed to be building ftgzip.c | 12:13.12 |
| | What VS are you using ? | 12:13.32 |
| chrisl | 2005 | 12:13.36 |
| kens | Hmm mine's 2008 | 12:13.49 |
| chrisl | but this is all makesfiles | 12:13.58 |
| kens | What happens if you delete teh diagnostic section from FT's zutil.h ? | 12:14.12 |
| chrisl | Let me try | 12:14.26 |
| kens | OK so if I put the section back, then it gets past infcodes.c, and indeed the VS otuptu does not mention that file. | 12:16.16 |
| | So I think VS is lying. | 12:16.21 |
| chrisl | Oh, what a pile of crap :-( | 12:16.49 |
| kens | I'm inclined to agree | 12:16.57 |
| chrisl | Oh, I wonder..... | 12:17.22 |
| kens | Whenyou get the unresolved external symbol _z_error was it referenced from function inflate_codes ? | 12:17.26 |
| | Becuase that is in infcodes.c | 12:17.58 |
| | Even though VS says its referenced from ftgzip.c | 12:18.10 |
| chrisl | #include "infcodes.c" | 12:18.19 |
| | in ftgzip.c | 12:18.27 |
| kens | Yeah that would do it <sigh> | 12:18.29 |
| | Explains why the compiler was compiling ftgzip.c but threw an error onn infcodes.c | 12:18.50 |
| chrisl | In which case, the solution might be very simple..... | 12:18.55 |
| kens | Just don't #include it ? | 12:19.04 |
| | Oh, define NO_INFLATE_MASK | 12:19.50 |
| | Ah no | 12:19.58 |
| chrisl | There's a CFLAG missing in msvc.mak | 12:20.01 |
| kens | smacks head | 12:20.11 |
| chrisl | Nope, still didn't work :-( | 12:25.47 |
| kens | Oh :-( | 12:25.53 |
| chrisl | Okay, going with the simple "fix" and we'll worry about this later | 12:26.05 |
| kens | Right | 12:26.09 |
| | I need some lunch bbiab | 12:26.47 |
| chrisl | Out for a bit... | 13:08.22 |
| | Forward 1 day (to 2018/03/07)>>> | |
Log of #ghostscript at irc.freenode.net.