| | <<<Back 1 day (to 2020/06/24) | Fwd 1 day (to 2020/06/26)>>> | 20200625 |
| avih | ator: i think one of the commits past 1.0.7 can cause mujs to enter an infinite loop in some cases. | 06:11.33 |
| | i don't have a reproducible test case yet. so far it only happens in mpv, and i don't know where yet. i only just realized it's probably mujs. | 06:12.16 |
| | looking at the commits, i'm guessing maybe the gc de-recurse | 06:12.53 |
| | hmm.. could be two issues. the gc de-recurse seems fine, but the following one "0261579 Support embedded 0 in strings by using modified UTF-8" causes some/all scripts to not work for me, but without 100% cpu. | 06:27.13 |
| | and the following commits (832e069 Support 4-byte UTF-8 sequences) makes some scripts work, and some not work and having 100% cpu. | 06:29.06 |
| | this is with my utf8 wrappers. i'll now try to remove my wrapper. | 06:29.19 |
| | ok, the issue happens also without my utf8 wrappers. | 06:31.47 |
| | both issues happen in those commits. | 06:32.34 |
| | (without my wrappers) | 06:32.50 |
| | ator: with the embedded 0 commit, this piece of code which is the first thing i run in mujs in mpv for all scripts, fails on (some?) of my scripts with "SyntaxError: @/norm_err.js:1: unexpected token: '{' (expected ';')": https://0x0.st/i47U.txt | 06:45.09 |
| | i think it happens with all scripts. | 06:46.23 |
| | could it be the script name which i set which starts with @ ? | 06:46.45 |
| | the funny thing is that the next commit (utf8) makes the issue intermittent. some scripts work, some hang with 100% cpu. | 06:47.49 |
| | if i change the \ at the end of each line to \n\ (to add actual newlines), it still fails with the same error on line 1 in all scripts | 06:51.58 |
| | it's executes like so: https://github.com/mpv-player/mpv/blob/master/player/javascript.c#L414 | 06:53.14 |
| | ator: reproducible case with commit 0261579: mujs <this file: https://0x0.st/i477.js > errors with "SyntaxError: [string]:1: unexpected token: (identifier) (expected ';')" | 07:01.17 |
| | and it doesn't happen with the earlier commit (331c5ec) | 07:03.22 |
| | (it's the first time i try mujs with these commits past 1.0.7 - i didn't update it back then when you pushed them) | 07:04.56 |
| | amusingly, the next commit (utf8) "fixes" this issue with this specific test script, and in mpv most scripts run, but in mpv some scripts hang with 100% cpu | 07:06.42 |
| | iirc my opinion in the past was that the embedded 0 commit should not make it into master. it just adds noise. i still think the same now. | 07:13.48 |
| | (because embedded 0 on its own was never an interesting thing as far as i can tell. the related interesting thing is to preserve arbitrary blobs, but mujs doesn't do that anyway, so i don't really see the point is adding support only for embedded 0) | 07:43.26 |
| | in* | 07:43.49 |
| | though maybe it does have a value of supporting arbitrary codepoints. | 07:46.14 |
| | (but not 100% sure i see the value in that) | 08:29.39 |
| | (especially considering the size of this commit) | 08:30.02 |
| ator | avih: you're right, there snuck in a bug in "Support embedded 0..." that was fixed in the followup "Support 4-byte" when I split them up from a bigger combined commit | 09:50.00 |
| | if you can find a script that fails with commit 832e0690493eaa6b9875e477c79ea3200c2c4310 that would be helpful | 10:01.45 |
| avih | ator: with HEAD (i assume same as 832e069 because HEAD itself is no-op)i can reproduce it inside mpv, sort of, because it's intermittent. it's hard to reduce a test case becauseit always loads a big script first (to add the mpv APIs etc). but i can test patches if you have some | 10:29.56 |
| | most scripts work fine, but some just hang with 100% cpu. i don't know where exactly, because it's hard to tell. i think from the mpv side it's just inside mujs | 10:30.43 |
| | ator: fwiw, i have a hunch it could be related to comments. mpv always load the "base" scripts (defaults.js) and then the user script, and it happens for me even with a user script which contains only comments | 10:32.09 |
| | i don't really know how to start reducing it.. | 10:33.02 |
| | my defaults.js definitely cannot be made to run outside of mpv. its the glue between the c code and user scripts, and highly mpv-specific | 10:33.42 |
| | ator: fwiw, defaults.js is this https://github.com/mpv-player/mpv/blob/master/player/javascript/defaults.js but i don't think it matters. when the user script is https://0x0.st/i4FQ.txt then the issue happens, and when it's an empty file - it doesn't happen | 10:50.30 |
| | i.e. there shouldn't be any difference, it's just comments, and yet when i go back and forth between empty and these comments - the issue go away and come back | 10:51.17 |
| ator | avih: can you attach a debugger and get a stack trace? | 10:51.20 |
| avih | no | 10:51.25 |
| ator | run it in the debugger, ^C to interrupt the process and see where it is | 10:51.37 |
| avih | wait, i should do one more thing. i didn't test if there are any syntax errors when the issue doesn't happen | 10:51.46 |
| | no, no syntax errors. | 10:52.35 |
| | ator: i don't think i have a working gdb setup for mingw binaries. i don't know if one even exists | 10:53.02 |
| ator | or 'sudo perf top' to see which function is eating up all the cpu | 10:53.24 |
| | oh, windows? | 10:53.29 |
| avih | and i didn't build mpv on linux for now | 10:53.35 |
| | yes | 10:53.37 |
| | well, the stand alone reproduce case was on linux. easy to build mujs there, but mpv is harder | 10:54.20 |
| ator | well, that issue was a trivial bug (forgot to change the type of Rune to signed when adding checks for EOF) | 10:54.47 |
| avih | i can add debug prints on places you suspect could hang | 10:54.53 |
| | ator: "that" being? the embedded 0? or something else/new? | 10:55.21 |
| ator | avih: take out the "Eliminate recursion in GC..." commit and see if the "Embebdded 0" and "4-byte UTF" commits on their own cause the hang | 10:55.47 |
| | yes. | 10:55.56 |
| | one hunk in that patch sequence ended up in the wrong commit. | 10:56.23 |
| malc_ | ator: ermm... signed Rune? glfont always had them as unsigned, no? and i cant remember what original Pike(s?) code looked like | 10:56.53 |
| avih | ator: sorry, i still don't follow. do you think there's an unpatched issue on HEAD now? | 10:57.04 |
| ator | avih: mujs origin/master should be fine. | 10:57.28 |
| avih | right, ok, we're past the embedded 0 split commit issue. i'm now testing only HEAD | 10:57.47 |
| ator | malc_: changed Rune to "int", not unsigned short | 10:58.04 |
| avih | and that's where i reported the "only comment makes it hang" issue | 10:58.06 |
| ator | malc_: to allow for -1 as an end-of-data signal | 10:58.19 |
| | avih: and I wonder if you can get a backtrace or at least function name of where it's hanging | 10:58.41 |
| malc_ | ator: there is a qualifier missing (i think) , IOW who changed? | 10:58.42 |
| | ator: gotcha | 10:58.48 |
| avih | ator: i'll try your suggestion though of removing the gc recurse commit. | 10:59.16 |
| ator | malc_: I changed it when adding support for modified UTF-8 "nul" bytes and code points outside the BMP | 10:59.22 |
| malc_ | ator: okay... naughty naughty | 10:59.48 |
| ator | avih: thanks. it's either in that commit, or a typo somewhere in the embedded 0 commit. I can't imagine the changes in the 4-byte commit could cause a hang. | 11:00.06 |
| avih | k | 11:00.21 |
| | ator: happens also without the gc commit after: git revert 331c5ec | 11:01.49 |
| malc_ | ator: btw i ended up modifying punctuation in font forge by hand... not that anyone cares (evidently) but still... now i have a perfect typographical setup here | 11:02.53 |
| ator | avih: hm. try without the 'support embedded 0' commit (that one should be possible to revert without affecting the 4-byte UTF-8 sequences) | 11:03.07 |
| avih | ok | 11:03.18 |
| ator | malc_: oh, and to answer your question from a few weeks ago -- no, I don't know of a simpler tool to remap the 3 to dzhe. maybe by editing the 'cmap' encoding subtables, but that will likely also involve fontforge scripting. | 11:04.07 |
| avih | ator: it does not revert easily. conflicts at least at the docs | 11:04.18 |
| ator | or dumping to sfb and editing the text file and the regenerating the ttf | 11:04.22 |
| avih | i ignored the docs diff. the code itself reverted ok | 11:05.01 |
| malc_ | ator: thing is dze/ezh thing i can automate with python, period,coma,semi "scaling" not so much :( | 11:05.09 |
| ator | malc_: be glad you're not in the dark ages when there were no open source font editing software available :) | 11:05.41 |
| | in the transition era between bitmap and truetype fonts | 11:05.52 |
| malc_ | nuc:~ | 11:06.13 |
| | - fc-list -f '%{family}\n' "$@" | sort | uniq | wc -l | 11:06.13 |
| | 16 | 11:06.13 |
| | nuc:~ | 11:06.13 |
| | - fc-list -f '%{file}\n' "$@" | sort | uniq | wc -l | 11:06.16 |
| | 47 | 11:06.21 |
| | | 11:06.24 |
| | how bloody cool is that? | 11:06.27 |
| | ator: yes! | 11:06.33 |
| | williams(?) really deserves a monument in his honor | 11:06.51 |
| avih | ator: issue seems fixed after reverting the embedded 0 with this commit https://0x0.st/i4F9.txt . i went back and forth with this commit at least twice, the issue goes away and comes back as "expected" | 11:07.46 |
| ator | avih: thanks... now to reproduce it in a debugger | 11:08.23 |
| avih | i've spent quite a lot of time on it, setting it up in a debugger is going to be a PITA. maybe you want to add some debug prints at some places? i could apply such patch and retest | 11:09.16 |
| | the conflict markers at this revert commit were left intentionally, because only the docs had conflicts. | 11:10.17 |
| | ator: remind me again why it's good to support embedded 0's ? | 11:10.55 |
| ator | avih: because JS code is allowed to use "hello\u0000world" strings | 11:11.52 |
| avih | right | 11:12.16 |
| | well, it's a pretty big commit to add this support. kind of annoying it has to be that big | 11:12.49 |
| ator | avih: put a printf in jsY_lex, before and after it cals jsY_lexx, see if that's where it gets stuck | 11:13.13 |
| avih | ok | 11:14.41 |
| | ator: indeed, after quite a few calls, it hang and does not return | 11:17.33 |
| | ator: using this diff print https://0x0.st/i4FW.txt | 11:17.56 |
| | specifically, 5076 successful calls, and one which does not return | 11:19.16 |
| ator | avih: print out J->lexline, maybe gives a hint to where in the input source | 11:20.03 |
| avih | before each call? | 11:20.23 |
| ator | yeah, before the "[LEX0" | 11:20.34 |
| avih | k | 11:20.38 |
| | J->lexline is null-terminated char* ? | 11:20.55 |
| ator | no, it's an int line number | 11:21.10 |
| avih | oh | 11:21.55 |
| ator | J->source has the null-terminated char* if you want to print that instead | 11:22.00 |
| avih | and the filename? | 11:22.02 |
| | (there are two scripts involved) | 11:22.21 |
| ator | J->filename | 11:22.24 |
| avih | k | 11:22.27 |
| | and J->source is the line or the whole file? | 11:22.50 |
| ator | J->source is the text of the source code, at the position where we start scanning for the token | 11:23.23 |
| | so it really is where the error occurs | 11:23.30 |
| | printf("source: (%.50s)\n", J->source) | 11:24.10 |
| | prints the next 50 characters in the string being parsed at that point | 11:24.48 |
| avih | ok. it points to the file name with the comments-only, but it's wrapped inside common-js wrapper code, so it points at a line outside the actual source file, though i expect it's also outside the wrapped file. it says line 59, but the file is only 12 lines of comments, and the wrapper prepends code without newline, and appends just one line | 11:25.38 |
| | i think it reads out of bounds | 11:25.43 |
| | i'll try the %.50s now | 11:26.22 |
| ator | avih: I suspect I've found the error | 11:28.47 |
| | a // comment at the end of the file | 11:29.14 |
| | would not detect the end of file | 11:29.21 |
| | I've put a commit on tor/master | 11:31.06 |
| | " Fix typo in lexlinecomment" | 11:31.14 |
| avih | hmm.. well, it points to the start comment actually. that's the user file where it fails https://0x0.st/i4FQ.txt and that's the first 100 chars of J->source: https://0x0.st/-k.txt (including single quotes by the print function) | 11:31.34 |
| | sorry, first 100 chars: https://0x0.st/i4Fx.txt | 11:32.17 |
| ator | avih: yeah. it scans and ignores the comments looking for a token, at the final comment it runs past the end of the file | 11:32.26 |
| avih | it's "eaten" the first / if the // at the first line, and J->source is the second one | 11:32.43 |
| ator | avih: yeah. that confirms my hunch. thanks! | 11:33.00 |
| avih | ator: you mean when the last line at the file is a comment | 11:33.23 |
| ator | avih: yes. and is missing a newline. | 11:33.35 |
| avih | yeah, i.e. last line, which is not empty and only has a comment | 11:33.51 |
| ator | the last line which has a comment and is not terminated with a \n | 11:34.10 |
| avih | right | 11:34.18 |
| | ator: so why is J->source starting at the first line and not the last? | 11:35.07 |
| ator | avih: we don't return comments as tokens | 11:35.43 |
| | so it keeps looking for the next token, so it parses all of the comments in one go | 11:35.59 |
| avih | so it doesn't "progress" inside the source file if all of it is comments? | 11:36.09 |
| | gotcha | 11:36.15 |
| ator | not at the jsY_lex level, no | 11:36.20 |
| | it eats up comments and whitespace looking for a token to parse | 11:36.35 |
| avih | right | 11:36.43 |
| ator | give the "Fix typo in lexlinecomment" commit a try | 11:37.11 |
| | that should be the only one, I combed through the file looking at all other while loops | 11:37.23 |
| avih | ator: will do. so the intermittent behavior was because out-of-bounds memory is arbitrary? | 11:39.04 |
| | hmm.. i don't see this commit yet. | 11:39.50 |
| ator | avih: yes | 11:42.04 |
| | avih: http://git.ghostscript.com/?p=user/tor/mujs.git;a=commit;h=9f3e141d805cddec7cce1fae38373a82c61e3300 | 11:42.41 |
| avih | i see it http://git.ghostscript.com/?p=user/tor/mujs.git;a=blobdiff;f=jslex.c;h=3da6b550e5f928d45112200e11404a3bb60efc50;hp=446d2c8034cd6d2453602bf9eb3e1dcc51f723b3;hb=9f3e141d805cddec7cce1fae38373a82c61e3300;hpb=ac25ac54fa79dad253cbc156c22c6685013f8499 | 11:42.51 |
| ator | git remote add tor http://git.ghostscript.com/user/tor/mujs.git | 11:42.53 |
| | git fetch tor | 11:42.55 |
| avih | yeah, i have it, just don't fetch it frequently | 11:43.05 |
| | i look at the web interface if you have new commits, but not recently | 11:43.18 |
| malc_ | ator: that's not how you taught me... for me you have used git remote update... | 11:43.28 |
| ator | avih: don't rely on using any commits on there, though. they often get rebased and reworked before they land on the main repo. | 11:43.40 |
| malc_ | damened TMTOWTDI | 11:43.40 |
| ator | malc_: or git remote update :) | 11:43.53 |
| malc_ | kewl :) | 11:44.08 |
| avih | ator: i'm not. i'm aware it's the wild west there :) | 11:44.20 |
| ator | malc_: git config remote.thisUserSucks.skipDefaultUpdate true | 11:44.41 |
| | malc_: git config remotes.foo 'origin malc tor sebras' and then git remote update foo | 11:45.18 |
| malc_ | ator: nuc:~ | 11:45.21 |
| | - grep remup .gitconfig | 11:45.21 |
| | remup = remote update | 11:45.21 |
| | | 11:45.22 |
| avih | ator: seems to fix it. | 11:45.31 |
| ator | so you don't have to fetch everything every time | 11:45.33 |
| malc_ | typing more than necessary is overrated (even on topre) | 11:45.41 |
| avih | let me now try with all my scripts back. 2 mins. | 11:45.45 |
| malc_ | ator: tack!! | 11:45.57 |
| ator | malc_: too long. up = remote update | 11:46.19 |
| | and also psuh = push | 11:46.26 |
| avih | ator: looks good. please push :) | 11:46.47 |
| malc_ | ator: https://tpaste.us/yZQn | 11:46.56 |
| avih | sorry, psuh | 11:46.58 |
| ator | one psuh coming up! | 11:47.11 |
| malc_ | :) | 11:47.15 |
| avih | :) thx | 11:47.17 |
| malc_ | i'm not dyslexic :) and don't type at 120WPM and by extension do not mistype psuh all that often | 11:47.55 |
| ator | malc_: I blame dvorak | 11:48.06 |
| | psuh is as common as the | 11:48.18 |
| malc_ | I AM ON dvorak! | 11:48.18 |
| ator | t e h | 11:48.21 |
| | damn autocorrecting hexchat! | 11:48.28 |
| | malc_: oh, I thought for some reason you were on colemak :) | 11:48.50 |
| malc_ | i ditched workman because of `picture` (IIRC) | 11:49.01 |
| | ator: never used colemak -- so pedestrian :) | 11:49.20 |
| avih | teh is my favorite. i mistype it so many times when i type quickly. | 11:49.30 |
| malc_ | aj newa mistajp zose finks | 11:50.04 |
| | and given how t h e are located in Dvorak i would say you have serious right/left hemisphere issues! go see a neurologist | 11:51.09 |
| ator | malc_: I hit t-h and e at the same time. it's all just random whether the e happens between or after the th pair | 11:51.42 |
| avih | same :) | 11:52.06 |
| malc_ | bloody hell | 11:52.19 |
| ator | much less often with dvorak than with qwerty though! | 11:52.30 |
| malc_ | you guys should join the same institution or something | 11:52.51 |
| ator | same issue with push. if I move from p-u too slowly, the u comes between the sh pair instead of before | 11:53.29 |
| malc_ | ator: did anything good came out of your remapping the keys so that you press proper letter when the opposite shift is pressed? did it help? | 11:54.04 |
| ator | the hazards of typing with dvorak -- misplaced vowels because they're all on the other hand | 11:54.07 |
| malc_ | vowels are overrated just ask Semites | 11:54.44 |
| ator | yh wht gd r thy nywy | 11:55.02 |
| | ts sy t rd wtht vwls | 11:55.28 |
| malc_ | some people consider Y to be a vowel :) | 11:56.09 |
| | myself included | 11:56.14 |
| ator | it's a semi-vowel, like w | 11:56.23 |
| | and english r | 11:56.27 |
| | in swedish y would be a vowel, in english it's not-quite-a-vowel | 11:56.57 |
| malc_ | if Norwegian is of any indication y is really ы and thus a vowel indeed | 11:57.55 |
| | and as for "english" r whatever that might be, native scotts speakers will vehemently disagree | 11:58.27 |
| ator | yeah, swedish is the same where y is just a variant spelling of i | 11:58.42 |
| | malc_: https://en.wikipedia.org/wiki/Rhoticity_in_English here spend another afternoon on wikipedia :) | 12:00.00 |
| malc_ | one of my norwegian friend's second name is Trygve, he remarked that another friend of ours (Canadian with Romanian roots) had significant difficulty with it :) | 12:00.31 |
| | THANKS! | 12:00.52 |
| | just what i need | 12:00.56 |
| avih | malc_: there are some vowels in semite languages, mostly for o and i, but not for a and e | 12:01.35 |
| ator | avih: does א count as a vowel? | 12:02.18 |
| malc_ | avih: there are TONS of vowels, it's just that (usually) the short ones are not indicated (maybe with diacritics, in childrens book Quaran and Torah) | 12:02.25 |
| | at least that's what aforementioned Wikipedia told me a while ago | 12:02.39 |
| avih | but also, the full writing does include "dots" which define the exact pronunciation, it's just not needed once you're familiar with the language | 12:02.43 |
| malc_ | ator: א is a glottal stop i think | 12:02.57 |
| | and Kantors faworite letter to name things with | 12:03.42 |
| avih | ator: it can be both, but usually a consonant | 12:04.06 |
| ator | I'm showing my ignorance :) Just assuming it'll be an 'a' because it's the origin of the word 'alphabet' | 12:04.29 |
| avih | ALEPH is usually pronounced like the U in umbrella | 12:04.33 |
| malc_ | avih: how is gemal pronounced? | 12:05.31 |
| | it's camel right? | 12:05.37 |
| avih | but it can be pronounced like the first i in insulate, or the first e in emotion | 12:05.44 |
| ator | avih: those are all short stressed vowels, really quite similar | 12:06.19 |
| malc_ | https://www.youtube.com/watch?v=3DxMSufIIqY | 12:06.43 |
| avih | the letter GIMAL or GIMEL is like a "hard" g, like in geometry, but not like in gibberish | 12:06.48 |
| malc_ | avih: gotcha thanks | 12:07.11 |
| ator | wait, how do you pronounce geometry? | 12:07.18 |
| malc_ | jeometry | 12:07.33 |
| ator | hard g as in graphics (let's not get into how to pronounce GIF) | 12:07.38 |
| avih | no, same hard g, but GEOMETRIA | 12:07.49 |
| | GEH-OH-METRI-AH | 12:08.02 |
| ator | /dʒiˈɑmətɹi/ | 12:08.52 |
| malc_ | ator: once again - https://www.youtube.com/watch?v=3DxMSufIIqY Bally manages to pronounce GIF proprely! | 12:08.53 |
| | fick mich... ipa | 12:09.09 |
| avih | wait, i was wrong. indeed geometry is like with a j. so i meant like in gallop | 12:09.17 |
| malc_ | avih: i beg you to watch the video too | 12:09.39 |
| | and read the Chaos poem | 12:09.48 |
| avih | but in hebrew it's a hard g for geometry | 12:09.53 |
| malc_ | there is no correct English pronunciation, full stop | 12:10.26 |
| | ditto in russian | 12:10.39 |
| | because... greeks | 12:10.50 |
| ator | avih: thanks! that's what had me confused :) | 12:11.03 |
| avih | these days i actually read and write more english than hebrew, though i still speak hebrew more :) | 12:12.02 |
| ator | malc_: nice ending touch to the video "libary" | 12:13.20 |
| | at 3:16 | 12:13.35 |
| malc_ | ator: :) | 12:15.01 |
| ator | Try asking americans how to pronounce cavalry. No, it's not the hill Christ was crucified on. | 12:15.05 |
| malc_ | Calvary is a nice movie though :) | 12:15.25 |
| | https://en.wikipedia.org/wiki/Calvary_(2014_film) | 12:15.40 |
| avih | ator: the fix looks good so far. thanks :) | 12:18.56 |
| | <<<Back 1 day (to 2020/06/24) | Forward 1 day (to 2020/06/26)>>> | |
Log of #mupdf at irc.freenode.net.