Log of #mupdf at irc.freenode.net.

Search:
 <<<Back 1 day (to 2017/06/18)20170619 
Robin_Watts nbjoerg: Sorry, but I really don't see why that should be an issue. All it has to do is to gather the bytes into a block of memory. It's not brain surgery.08:04.59 
  It's exactly the same process as reading strings into string literals.08:05.20 
  We need to end up with the data in a solid block, not as as series of lines interspersed with 0s.08:05.44 
nbjoerg Robin_Watts: pretty much every modern compiler keeps track of the position of each element in a character array08:43.23 
  for a 8M element array, that's a lot of data08:43.35 
  it does *not* have to do that for every single character in a string literal08:43.58 
  note that a string literal can be split over lines WITHOUT adding any additional NULs08:44.22 
  since the language rules require merging of adjacent string literals08:44.39 
  i.e. char foo[] = "ABC" "DEF" "GHI" gives sizeof foo == 1008:45.04 
avih tor8: hey. 1. i noticed that the Makefile adds -fPIC for the .so , and i don't _think_ it's necessary (at least one compiler issues a warning that it's ignored because it defaults to -fPIC for .so).09:27.15 
tor8 avih: okay.09:27.36 
avih 2. I noticed that with -O2 mujs performs ~20-40% faster, but it issues some warnings, might be worth cleaning them up.09:27.45 
tor8 avih: I don't get any warnings with my versions of gcc or clang09:28.34 
avih (mujs benefits nicely from optimizations in general, contrary to some other "glue only" code)09:28.34 
  oh. sec, let me post it then.09:28.51 
  tor8: https://0x0.st/lOa.txt09:31.05 
  3. Number.toString(radix) - only radix==10 is supported. 4. parseInt and parseFloat are not implemented09:31.51 
  gcc here is 6.3 on Alpine linux, and i get similar warnings with gcc for windows09:32.53 
  or rather, mingw gcc 6.309:33.10 
tor8 avih: oops. monday morning user error. I forgot to compile with 'build=release'09:33.12 
  I see the warnings with gcc09:33.16 
avih k09:33.27 
  the longjmp clobbered message is possibly weird, or the contness is not declared correctly at the formal argument09:35.03 
  constness*09:35.11 
  supposedly if the pointer is const then there should be no such warning09:35.29 
tor8 avih: gcc has *lots* of trouble correctly dealing with longjmp and warnings09:35.51 
  but I'll double check the ones that turn up with -O209:36.06 
  with -O3 it throws up even more such warnings09:36.31 
  in my experience they're all bogus09:36.39 
avih i'm not sure it's wrong here though. with const char *x, you could do x = $whatever, which means the pointer is not const, which means it can be clobbered09:36.42 
  you probably want char * const x09:37.49 
  or const char * const x09:38.00 
tor8 avih: I've made a note to look into adding radix support to Number.prototype.toString09:38.47 
avih k, and parse{Int, Float} ?09:39.08 
  also, note that toString(radix) is not limited to integers09:39.37 
tor8 what do you mean with parseInt and parseFloat?09:39.45 
avih (for the number. radix is integer i think)09:39.51 
tor8 avih: yes, that's why it's not implemneted yet09:39.54 
  parseInt and parseFloat are implemented in jsbuiltin.c09:40.39 
avih tor8: https://www.ecma-international.org/ecma-262/5.1/#sec-15.1.2.2 and same for parseFloat . these are global functions afaik09:40.44 
  oh.. hmm.. sec09:41.04 
  tor8: yeah, sorry, parseInt and Float are implemented correctly. I think i tried them off Number, but i now see that it should only work in later js standards.09:46.15 
tor8 you mean there is a Number.parseFloat in later JS?09:46.51 
avih yes09:47.00 
  for the sake of modularizing global functions. identical implementation09:47.15 
tor8 agh, gcc so dumb. I think in the S_fromCharCode function it gets confused by the fact that js_throw is marked _Noreturn but somehow forgets that09:47.40 
  we've run into this a *lot* with the exception handling in mupdf09:48.37 
  gcc just trips all over itself during optimization09:48.47 
avih well.. not "so" dumb. merely misses a bit :p09:48.52 
tor8 and the strict-overflow warning seems related to an optimization it does09:50.30 
  turning X < 0 || X >= Y into a X09:50.58 
  X - Y > 009:51.02 
  or similar conversion09:51.06 
  which has nothing to do with what's in the source code, and everything to do with gcc's optimizations09:51.28 
  avih: also, I can't build with -shared without -fPIC on gcc09:52.06 
  /usr/bin/ld: /tmp/ccP0CzQW.o: relocation R_X86_64_32S against `.data.jsS_sentinel' can not be used when making a shared object; recompile with -fPIC09:52.13 
avih gotcha. then keep it. thanks for testing.09:52.35 
  re the signed overflow, yeah, i suspected as much as well, though not sure what solution could be used here.09:54.06 
tor8 turn off -Wextra :)09:54.32 
avih i'm assuming you also don't expect signed overflow there09:54.38 
  (signed overflow is UB anyway, isn't it?)09:54.53 
tor8 I don't do any arithmetic, so there's no way for the source as written to get an overflow09:55.01 
  the optimizer introduces arithmetic to optimize the comparisons09:55.42 
avih hmm.. so the optimization could create UB where the original is safe? that's unkind09:55.52 
tor8 that's optimizing compiler writers for you...09:56.02 
  constantly breaking code that's worked just fine for 20-30 years in the search for the latest hottest benchmark09:56.24 
avih well.. ~30% speedup is non negligible for code (js) execution...09:57.32 
tor8 I'd rather a 30% slowdown than risk breaking code that works...09:58.02 
avih i don't disagree09:58.13 
tor8 but the tradeoff we're talking about here is more along the lines of a 29% vs 30% speedup09:58.25 
  and breaking shit along the way :(09:58.38 
avih WRT these specific checks?09:58.42 
tor8 no, the last couple of years of optimizer work abusing UB in compilers09:59.05 
avih oh09:59.16 
  does clang do similar nasties?09:59.27 
  or gcc 7?09:59.36 
  let me try with clang, and latest apple's llvm09:59.59 
  wither i'm not invoking it correctly, or i get zero warnings with clang-3.8 (except one which is for ar: )10:03.23 
  ar cru build/release/libmujs.a build/release/libmujs.o10:03.25 
  ar: `u' modifier ignored since `D' is the default (see `U')10:03.25 
  either*10:03.48 
  zero warnings also with Apple LLVM version 8.1.0 (clang-802.0.38)10:05.42 
  so just gcc, at least 6.3, but iirc also 4.8 and 5.410:06.01 
tor8 clang doesn't get confused with setjmp/longjmp as much10:06.15 
  and properly handles detecting dead code10:06.37 
  but misses a few other cases that gcc catches10:06.46 
  shame about all the false positives with gcc though10:06.53 
avih yup10:07.03 
tor8 avih: non-standard binutils configuration? -U is supposed to be the default.10:08.35 
nbjoerg clang intentionally faults on the side of having a reasonable FP rate10:09.01 
avih possibly. this is on alpine linux, which is supposed to have "hardened"/"safe" defaults on gcc and binutils10:09.10 
tor8 avih: yeah, sounds likely. such an approach is commendable, but unfortunately is bound to cause problems with most existing code and makefiles out there...10:09.43 
avih well.. this one is probably not really a problem, and typically fixing it for alpine actually makes it better in general, but yes, some packages need some patching. not much though.10:10.47 
tor8 nbjoerg: character array literals are limited in length (and the length is implementation defined)10:10.59 
  awkward giant arrays are not limited as such10:11.09 
avih tor8: fyi, i find alpine a very crisp distro. extremely lean by default (for me it uses 28M in bash prompt from ssh), but you can get anything you like really.10:14.29 
  and it uses musl, and safe defaults everywhere.10:14.50 
  but x runs fine, and it has xfce and other desktops available. it's really very nice.10:15.26 
tor8 I just can't be bothered with system administration... I use debian stable with no stinking desktop. just an X server, openbox wm, xchat, firefox and a ton of xterms.10:21.28 
avih tor8: actually, the ar warning was on ubuntu 16.04 lts (regardless of clang/gcc)10:28.27 
nbjoerg tor8: so are array literals10:54.28 
  tor8: neither behavior is portable, but the string limit is much less likely to be a problem compared to the giant array10:55.33 
avih nbjoerg: are you discussing char *x = "str" vs char x[] = "str" ?10:58.21 
nbjoerg no10:58.35 
  char x[] = "str" vs char x[] = {'s', 't', 'r' }11:00.37 
tor8 nbjoerg: in MSVC, the maximum length of a string literal is 64k11:09.59 
  ANSI C89 only guarantees 509 characters in string literals11:11.46 
nbjoerg sure, but the 64k limit exists for array literals too11:13.13 
tor8 not that I know. if that was the case, not much code would be able to compile...11:15.48 
nbjoerg that's what the C standard says11:16.14 
nbjoerg is as usually impressed by what Microsoft decides to limit and what not11:16.32 
avih nbjoerg: huh, is there a diff (other than terminating null) between x[] = "str" and x[] = {'s', 't', 'r'} ?11:22.02 
  i mean, isn't tat just syntactic sugar, plus null?11:22.47 
  that*11:22.51 
  (bbl)11:23.31 
tor8 avih: "str" is one token, and that is where you run into the most trouble with arbitrary compiler limits11:25.11 
  nbjoerg: C99 spec limits string literals to 4095 characters.11:28.05 
nbjoerg tor8: yes and whole objects to 64K11:28.14 
tor8 nbjoerg: C99 spec limits objects to 64k, but only in "hosted" environments11:28.22 
nbjoerg which is what almost everyone is using11:28.41 
tor8 so mupdf wouldn't work there at all. that's a hard limit even on "char x[65536];"11:28.54 
nbjoerg (hosted as opposed to freestanding)11:28.58 
nbjoerg wonders wtf microsoft has a limit on string literals11:29.21 
tor8 nbjoerg: I'd love to be able to use 8Mb string literals, don't get me wrong11:29.51 
  if that worked we could ditch the whole .incbin mess11:29.57 
nbjoerg ah well, the platforms I care about are fixed with the patch I gave behave11:30.00 
tor8 which is the workaround for compilers trying to optimize the array initializer :(11:30.27 
nbjoerg it's not about optimizing11:30.39 
tor8 we build the generated big fat array source files without optimization11:30.42 
nbjoerg just the position tracking for the tokens...11:31.02 
tor8 it's slow enough without, but have you tried compiling them with -O3? that takes ten times as long again...11:31.03 
  nbjoerg: would you mind trying out the commits on git://git.ghostscript.com/user/tor/mupdf.git on the 'incbin' branch, see if they work for you?12:01.18 
nbjoerg might be slightly better to explicitly use unsigned char12:14.38 
nbjoerg glares at ARM12:14.42 
  but that's a general issue12:14.47 
tor8 nbjoerg: that's easy enough to do12:18.56 
  but would have effects trickling throughout the code, and dealing with signed/unsigned char madness just makes me cry :(12:19.48 
  nbjoerg: thanks for checking.12:20.07 
nbjoerg looks quite nicer12:20.33 
  (with adding the -s of course)12:20.53 
tor8 being able to avoid the .incbin hack is nice12:21.03 
  I've been bitten by string literal length limits before, so I was a bit reluctant to try12:21.37 
  but this turns out to work well enough on linux, and MSVC will just have to suffer the insane array literals12:21.56 
  but that's what you get for using Windows :)12:22.00 
nbjoerg I'd maybe use \xnn and not the octal sequences, but just because it creates more consistent output :)12:24.20 
tor8 nbjoerg: good point.12:37.00 
  Robin_Watts: a handful of commits on tor/master12:45.04 
  Robin_Watts: with robin/lcms I get a segfault in pdf_load_colorspace with the file https://ghostscript.com/~tor/icctest.pdf13:06.46 
  RobinWattsLenovo: &13:06.50 
RobinWattsLenovo tor8: I have a modified fix for that fn.13:07.29 
tor8 actually, I get a segfault with everything with robin/lcms...13:07.39 
RobinWattsLenovo Let me finish lunch and I'll test that file.13:07.43 
  I just ran a clean cluster test :)13:08.49 
  (with my fixed version)13:09.01 
  I've updated robin/lcms with the latest.13:10.39 
tor8 error: unknown colorspace: DeviceRGB13:12.20 
  RobinWattsLenovo: robin/lcms (1197d585a) doesn't work at all for me13:14.27 
  segfaults and asserts left and right on PDF and EPUB files-13:14.36 
RobinWattsLenovo tor8: I'm retesting here, but have you tried a clean rebuild?13:23.39 
  (Captain obvious)13:23.45 
  I'm seeing a segv here... ass.13:24.20 
  I see the cockup. My bad.13:32.06 
  Just testing new version (on robin/lcms) now.13:38.14 
  OK, fixed version just clustered.13:43.19 
  and it runs your test fine.13:46.39 
tor8 RobinWattsLenovo: the output of icctest.pdf doesn't match gs rendering13:47.32 
  it should be very reddish13:47.49 
Robin_Watts I'll test it with an older version to see if I broke it...13:56.43 
tor8 it's one of the files I expected to be noticeably different with the lcms branch13:57.36 
RobinWattsLenovo tor8: yeah, I see little if no difference :(14:16.23 
  We should point mvrhel at it.14:28.43 
  tor8: Other than that, robin/lcms is pretty much ready to go in, I reckon.14:30.57 
  We just need a review from you.14:31.06 
tor8 I'm melting away here...14:33.11 
kens knows the feeling14:33.29 
RobinWattsLenovo here too.14:33.30 
kens sticking to the desk....14:33.38 
RobinWattsLenovo has to get fetch Helen in 10 mins. She's on a cramped train with no air-con.14:33.53 
kens :-(14:34.41 
sebras dabu (for the logs): ah, yes you'd need to do cd libmupdf git checkout 31a87a1da and then go back to the top and build.15:35.23 
  dabu (for the logs): I think that ought to work.15:35.32 
  tor8: kens: so... how many C?15:36.48 
kens 29 now in our bedroom15:37.01 
  London predicted at 31, but I think that may be low15:37.15 
  Nothing to your location of course15:37.29 
sebras kens: right now it is night so about 27C outside, tomorrow there's a predicted 36C.15:37.57 
  oh and rain.15:38.07 
kens Well if you will live in the tropics15:38.12 
sebras it's been raining for a week and half.15:38.14 
  the day after tomorrow 38C.15:38.35 
kens makes note not to visit sebras15:38.56 
sebras :) btw.. how did you make me yellow?15:39.24 
kens Ummmm, what ?15:39.37 
sebras kens: in your slash k3ens makes note not... sebras is colored yellow.15:39.58 
  oh... wait. can you prefix a message to me?15:40.19 
kens Must be because I said sebras and you have highlighting for your nick I guess.15:40.19 
  So name highlighting plus emote highlighting maybe ?15:40.42 
sebras kens: oh, or I have upgrade irssi by accident and some setting changed.15:40.45 
kens THat's possible15:40.52 
sebras kens: that might be it.15:40.53 
kens SO if I say sebras what colour are you n?15:41.02 
kens thinks sebras to try that too15:41.24 
sebras kens: actually "sebras" is in the normal white on black but your nick turns yellow in normal message, but in the /me's your name is bold white while my name is yellow.15:41.45 
kens Must be some kind of highlighting15:42.04 
sebras thanks kens for trying this.15:42.04 
  kens: yup, seems like it. btw, when was tor8 here?15:42.26 
  I fail to make sense of the timezones in the logs.15:42.33 
kens About 1 hour ago15:42.44 
sebras kens: ok, so maybe I logged in too late today.15:43.15 
kens Possibly, he's certainly been quiet for the last hour15:43.31 
  OTOH I'm off very shortly too, its too hot to think15:43.43 
sebras kens: I know the feeling.15:44.40 
RobinWattsLenovo .-15:47.52 
  mvrhel_laptop: Morning15:49.10 
mvrhel_laptop Morning RobinWattsLenovo15:49.33 
  looking at the customer file now15:49.40 
RobinWattsLenovo The icctest.pdf one ?15:49.47 
mvrhel_laptop to see what is going wrong15:49.47 
RobinWattsLenovo cool.15:50.14 
mvrhel_laptop yes15:50.16 
RobinWattsLenovo There is a commit on robin/lcms to rejig load_icc_based. If you're happy with that, then that was all the issues I spotted in my pass through.15:51.06 
mvrhel_laptop RobinWattsLenovo: ok let me look15:55.09 
  RobinWattsLenovo: it will be a few minutes. looking at this file. I see that we are assigning our sRGB profile to the image so it is never getting converted. Digging into why we would be doing that now16:01.13 
RobinWattsLenovo No hurry at all.16:01.29 
  Helen is home so I can relocate to my office, and start the aircon :)16:01.48 
mvrhel_laptop Hmm pdf interpreter is saying color space for image is DeviceRGB....16:03.31 
Guest59660 hello! i'm trying to link mupdf 1.11 to a qt app but i get crash. debugger says it come from harfbuzz's hb_calloc. please: 1. i'm only about to view pdfs, do i still need harfbuzz or is it possible to disable it during make? 2. if not, is there anything i can do?16:05.39 
mvrhel_laptop oh its a DefaultRGB def16:05.40 
  and we are not picking that up.16:05.51 
  brb to figure this out16:05.58 
Robin_Watts Guest59660: You don't need harfbuzz for pdfs.16:06.20 
  edit include/mupdf/fitz/config.h16:06.45 
  add #define FZ_ENABLE_HTML 016:07.33 
  and #define FZ_ENABLE_EPUB 016:07.44 
  at the top of the file, then clean and rebuild.16:07.50 
Guest59660 thanks robin, will try!16:09.14 
mvrhel_laptop hmm pdf_load_default_colorspaces16:14.42 
  res = pdf_dict_get(ctx, PDF_NAME_Resources, page->obj); is not returning a resource16:14.48 
  so the default color spaces are never getting set16:15.28 
Guest59660 and it works! thank you once again, robin!16:16.55 
mvrhel_laptop res = pdf_page_resources(ctx, page); finds it just fine16:19.34 
  Robin_Watts: ok fix coming in a minute16:20.46 
  Robin_Watts: ok on my robin_lcms branch16:24.16 
Robin_Watts fetching.16:24.23 
mvrhel_laptop there is a simple one line fix16:24.23 
  customer file is now a nice red16:24.42 
Robin_Watts Cool. My lcms branch updated, and cluster test running.16:26.37 
  I'll squash that down to a single commit in lcms_squashed, and then we need to pin tor8 down until he reviews it.16:26.59 
mvrhel_laptop Robin_Watts: right. and nothing back from Marti?16:29.18 
Robin_Watts No, nothing I've seen.16:29.31 
  Give him a week from when we last heard, and gently prod?16:29.58 
mvrhel_laptop yes sounds like a good idea16:30.09 
  Robin_Watts: commits all lgtm16:31.07 
  Trying to figure out how best to stage my next bit of work since it does rely upon the lcms branch16:33.09 
  I guess I can branch off of that?16:33.56 
  or not even do a branch16:34.08 
  it should not be anywhere near as significant16:34.19 
Robin_Watts I'd work off the lcms branch, and it may be that we can just commit it all later. Keeps your options open16:35.41 
mvrhel_laptop good idea16:35.49 
  Robin_Watts: so question16:36.11 
Robin_Watts http://git.ghostscript.com/?p=user/robin/mupdf.git;a=shortlog;h=refs/heads/lcms_squashed16:36.23 
  Properly got it owned by you this time :)16:36.31 
  go for it.16:36.37 
mvrhel_laptop cool thanks Robin_Watts16:37.23 
  ok so DeviceN. For the psd device, we will want to prescan the page like we do for transparency, but looking for presence of spot colors. This would be done though only if the device can support devn. How would you suggest we handle this?16:38.33 
Robin_Watts mvrhel_laptop: The MuPDF model would probably be to have a new device method that gets called with spot setup information.16:39.24 
  So the PDF interpreter would always scan and call the device method, and most devices would ignore it.16:39.52 
  but spot capable ones would take note.16:40.02 
mvrhel_laptop ok. Right now the pixmap for the psd device is allocated in mudraw I believe. That pixmap really needs be dependent upon the number of spot colors on the page16:40.40 
Robin_Watts We'd probably have to tell the draw device whether it should be spot capable or not when we start it up, but that's easy enough.16:40.47 
  mvrhel_laptop: oooh, yes.16:40.59 
  Ah, spots can vary per page,right?16:41.05 
mvrhel_laptop yes16:41.08 
Robin_Watts So a page level function to interrogate spots maybe?16:41.19 
  We have something like that for gprf.16:41.27 
mvrhel_laptop yes that is what I was thinking16:41.28 
Robin_Watts fz_page_count_separations16:42.40 
  fz_page_get_separation16:42.48 
  fz_page_control_separations etc.16:42.57 
mvrhel_laptop oh great16:42.59 
Robin_Watts There, it's practically written for you >8*)16:43.06 
mvrhel_laptop :)16:43.12 
Robin_Watts fz_count_separations_on_page, fz_control_separation_on_page, fz_get_separation_on_page, fz_separation_disabled_on_page16:44.04 
  This interface was designed to work with gproof stuff, so it may not be an exact fit, but we can bend it slightly if required.16:45.06 
mvrhel_laptop ok16:45.16 
  I am probably going to have many questions16:45.25 
Robin_Watts Would blend transparency modes be an easier thing to do first?16:45.41 
mvrhel_laptop maybe16:45.46 
  and the color spaces in transparen16:45.58 
  yes. maybe that would make more sense to do now16:46.15 
  you could probably do the devn stuff easier than me16:46.28 
Robin_Watts I'm happy to have a crack at it under your supervision.16:46.42 
mvrhel_laptop That would be great16:46.50 
Robin_Watts ok, cool, will have a think.16:47.17 
mvrhel_laptop I will start working on GWG164_Transp_Basic_BM_ICCbasedCMYK_x4.pdf and GWG161_Transp_Basic_BM_ICCBasedRGB_x4.pdf16:52.24 
Robin_Watts Do you have a separation or a DevN example file to hand ?16:52.43 
mvrhel_laptop sure16:52.51 
  do you want images or graphics16:52.58 
Robin_Watts yes :)16:53.04 
  either/both as is convenient for you.16:53.25 
mvrhel_laptop ok. will send you some16:53.34 
Robin_Watts Thanks.16:53.38 
mvrhel_laptop ok sent you the Ghent file that has images and graphics16:55.02 
  or an image16:55.07 
  and graphics16:55.12 
Robin_Watts Ta.17:03.36 
  tor8: You here?17:57.50 
 Forward 1 day (to 2017/06/20)>>> 
ghostscript.com #ghostscript
Search: