| | <<<Back 1 day (to 2015/12/28) | 20151229 |
| rayjj | I thought kens was on vacation, but apparently he is still working on bugs (bug 696487). I had already started it, thinking he was off, but hadn't taken it, | 15:01.48 |
| | are we having a meeting today (with the few of us that are not on holiday) ? | 15:05.41 |
| HenryStiles | I wasn't planning on a meeting but I'll be here | 15:09.39 |
| Robin_Watts | Hi rayjj, henrys. | 15:14.40 |
| | rayjj: I may have some questions for you in a bit. | 15:14.56 |
| rayjj | Robin_Watts: on the clist pseudo file caching or some other clist problem. | 15:16.35 |
| Robin_Watts | rayjj: On that, yes. | 15:16.43 |
| | rayjj, paulgardiner: So, I've been debugging bug 695822 | 15:17.12 |
| | And the problem manifests itself as clist reads returning uninitialised data. | 15:17.39 |
| | The root cause of that appears to be that the filesize held in the IFILE is NOT the same as the actual physical file's size. | 15:18.13 |
| | The difference stems from calls to clist_fopen. | 15:19.00 |
| | *fname != 0, so we don't go to the top part of the if. | 15:19.16 |
| | We go into the bottom of the if, and fake_path_to_file(fname) returns a non NULL clist_file_ptr ocf. | 15:19.47 |
| | We then call 'wrap_file' to wrap the duplicated file, and we assign the filesize to be the same as the filesize record from ocf. | 15:20.24 |
| | But the actual filesize is 0 there. | 15:20.36 |
| | not 64. | 15:20.41 |
| | Does that spark any inspiration? | 15:21.22 |
| rayjj | Robin_Watts: no, but it doesn't sound right. | 15:22.34 |
| | Robin_Watts: the "actual file size" comes from the usual fseek to the end, then ftell ? | 15:23.06 |
| Robin_Watts | yes. | 15:23.21 |
| rayjj | Robin_Watts: and it is supposed to be 0, but ftell returns 64 ? | 15:23.55 |
| | or the other way around ? | 15:24.20 |
| Robin_Watts | No. ftell gives 0. the ->filesize = 64. | 15:24.25 |
| rayjj | so the filesize record from the 'ocf' is wrong ? | 15:24.59 |
| Robin_Watts | yes. | 15:25.09 |
| rayjj | OK. I was the last to touch that -- do you want to dig into it, or hand it off to me? | 15:25.53 |
| Robin_Watts | rayjj: I'm happy to hand it off, but am concious of how busy you are. | 15:26.23 |
| rayjj | it is strange that it only happens sometimes, but at least you have a repeatable case | 15:26.25 |
| Robin_Watts | I got into this because Marcos was complaining that it was causing him pain. | 15:26.38 |
| HenryStiles | sebras: are you about? | 15:26.44 |
| rayjj | Robin_Watts: after so many days struggling with getting my laptop up, I can barely remember what I was busy with | 15:27.21 |
| | but I should dive back into the cust 532 issues. | 15:28.24 |
| Robin_Watts | I'll keep bashing for a while longer then. | 15:28.57 |
| rayjj | Hi, marcosw. If you want, you can send Jason the temporary workaround of using -c "<< /MaxDictStack 9000 >> setuserparams" -f infile.pdf | 15:29.50 |
| | marcosw: it won't cause any slowdown or impact on "normal" files, and allows his screwy file to complete | 15:30.32 |
| | marcosw: that was for bug 696487 | 15:31.19 |
| | Robin_Watts: :thanks, I'll ignore it, tempting as it is :-) | 15:32.13 |
| marcosw1 | rayjj: thx, will do | 15:36.18 |
| rayjj | Robin_Watts: I suggest that most tech discussions stay here (for general enlightenment) and we use #artifex for customer related stuff -- I tend to miss #artifex discussions since all it does is change the color of the tab when I am mentioned. | 16:05.15 |
| Robin_Watts | rayjj: Sure. | 16:05.27 |
| | Do I need to repeat what I said before? | 16:05.36 |
| rayjj | the tab that has focus makes a more noticeable "pop up" | 16:06.03 |
| | Robin_Watts: no, so writing to a file that is cloned is not expected since we clone it in order to read/render it | 16:06.51 |
| | Robin_Watts: is that what you are seeing ? | 16:08.00 |
| Robin_Watts | ok, it looks like we're cloning a file, and then writing to it (clist_fillpage => clist_close_writer_and_init_reader => clist_end_page => cmd_write_buffer => cmd_write_band => clist_fwrite_chars) | 16:08.18 |
| rayjj | assumes so from the comment "that assumption is definitely being broken." | 16:09.12 |
| Robin_Watts | pick a channel, any channel... :) | 16:09.20 |
| rayjj | yeah, sorry. | 16:09.29 |
| | who is doing a fillpage on a cloned file ? | 16:09.56 |
| Robin_Watts | You think *I* understand this code? :) | 16:10.34 |
| rayjj | Robin_Watts: whenever you want to bail is fine. | 16:11.16 |
| Robin_Watts | This is "saved pages test", so stuff may be happening at strange times (at least, that's what I'm telling myself) | 16:11.33 |
| rayjj | Robin_Watts: saved_pages_test mode may be a bit hinky, | 16:12.12 |
| | Robin_Watts: you can't reproduce it with command line equivalent params ? | 16:13.04 |
| Robin_Watts | rayjj: I would not have a clue what such params would be. | 16:13.20 |
| | It only goes wrong in jobserver mode, using a pipe, AIUI. | 16:13.34 |
| | so it is picky. | 16:13.51 |
| | So, the file is being 'cloned' in gs_main_init_with_args -> gx_saved_pages_param_process -> gx_saved_pages_list_print -> gx_saved_page_load -> clist_fopen | 16:21.35 |
| | and somehow the cloned file (NOT the original one, but the cloned one) is then being written to in: | 16:22.02 |
| | gs_main_init_with_args -> gs_erasepage -> gs_fillpage ->clist_fillpage -> clist_close_writer_and_init_reader -> clist_end_page -> cmd_write_buffer -> cmd_write_band -> clist_fwrite_chars | 16:22.59 |
| | Now, the file seems to be being cloned in a reasonable place (to my uneducated eye) | 16:23.51 |
| rayjj | Robin_Watts: OK, so the "print" happens after the file is processed, so it sounds like the files are getting mixed up | 16:23.57 |
| Robin_Watts | but how the hell does it get written to again - are we putting the file into cldev-> or something ? | 16:24.28 |
| rayjj | Robin_Watts: yeah, it should be cloned in order to print it | 16:24.30 |
| Robin_Watts | Ok, let's see if I can watch where cldev->page_{c,b}file are written. | 16:25.59 |
| rayjj | Robin_Watts: saved_page_load puts the FILE * in when it does the io_procs->fopen of the file | 16:28.10 |
| Robin_Watts | oh god, macro hell for no good reason :( | 16:28.11 |
| | rayjj: so, should we store/restore that? | 16:28.49 |
| | Is that a saved_pages_test specific thing ? | 16:28.57 |
| rayjj | Robin_Watts: no, AFAIK, saved-pages-test mode just does a call to gx_saved_pages_param_process(ppdev, (byte *)"begin", 5) from gs_main_init2 | 16:30.19 |
| Robin_Watts | rayjj: Right, but is saved-pages-test sharing a device in a way that non saved-pages-test stuff isn't ? | 16:30.52 |
| sebras | HenryStiles: I am. | 16:31.55 |
| | HenryStiles: sorry for the delay. | 16:32.04 |
| rayjj | then after each file on the list is processed (only one with the regression tests) it psi/imainarg.c calls: gx_saved_pages_param_process((gx_device_printer *)pdev, (byte *)"print normal flush", 18) | 16:32.25 |
| HenryStiles | sebras: no problem. | 16:32.30 |
| | for those around today we are working on bringing sebras to the staff part time. Please help him if he has questions. | 16:33.29 |
| rayjj | HenryStiles: great! | 16:33.47 |
| Robin_Watts | fab. | 16:33.55 |
| rayjj | HenryStiles: I assume mostly for mupdf issues ? | 16:33.59 |
| HenryStiles | rayjj: yes | 16:34.07 |
| rayjj | :-( | 16:34.11 |
| | ;-) | 16:34.17 |
| sebras | rayjj: :) I'd need to level up my gs knowledge a few times to do anything meaningful with gs I'm afraid. | 16:34.45 |
| rayjj | sebras: welcome to the crew -- I probably will be of no use to you | 16:34.50 |
| | sebras: no, if you are able to fill in on mupdf, that's plenty (it may free up Robin_Watts for messing around with gs) | 16:35.31 |
| HenryStiles | sebras: we were going to talk with tor8 he messaged me earlier so he should be ready to go. | 16:35.38 |
| | lucky Robin_Watts | 16:35.59 |
| sebras | HenryStiles: ok, I am too. | 16:36.52 |
| rayjj | HenryStiles: I assume that marcosw or you will take care of getting him needed permissions on the servers | 16:36.52 |
| | HenryStiles: I expect Robin_Watts prefers gs to SOT, but neither is quite as nice as mupdf | 16:37.32 |
| | but messing around with the "fringe" code (saved-pages) in gs may change his preferences :-) | 16:38.26 |
| tor8 | HenryStiles: right, so I expect we'll need to come up with parcel sized chunks of work for sebras to do? | 16:39.10 |
| HenryStiles | tor8: right | 16:39.28 |
| rayjj | Robin_Watts: both the usages in imain.c and imainarg.c use "gs_currentdevice" | 16:39.42 |
| HenryStiles | marcosw: can you add sebras to the mailing lists, I see an account on casper already | 16:39.59 |
| Robin_Watts | rayjj: So, we're in gx_saved_pages_param_process in the 'PRINT' case. | 16:40.06 |
| HenryStiles | marcosw: he also needs to be added to the various groups on casper. | 16:40.54 |
| Robin_Watts | The problem is essentially, that when we enter that case, we have a cldev all set up for rendering. When we leave that case, we've printed a saved page, but the clist gubbins still points to the saved file. | 16:41.07 |
| | rayjj: I reckon we need to ensure the clist info is restored. | 16:41.58 |
| rayjj | Robin_Watts: so that _should_ be the same as command line: ... -dJOBSERVER --saved-pages=begin -f infile --saved-pages="print normal flush" | 16:41.59 |
| Robin_Watts | rayjj: I am loathe to mess with the command line that I have working :/ | 16:43.56 |
| tor8 | Robin_Watts: sebras: the most urgent projects on the mupdf front are getting the features most new customers are looking for in tip-top shape | 16:44.05 |
| | we're seeing a lot of demand for annotation editing | 16:44.18 |
| | and for customizing the android viewers | 16:44.22 |
| Robin_Watts | I have some work on the android viewer underway. | 16:44.41 |
| rayjj | Robin_Watts: I agree that it sounds like we need to restore the clist file pointers after changing them for the "print" operation. That is probably in the out: | 16:45.05 |
| Robin_Watts | Specifically, I have the real MuPDF API exposed into java in a nice way. | 16:45.18 |
| tor8 | Robin_Watts: in my unrest I decided to get my fingers wet so I can grasp JNI code as well, so I can keep up | 16:45.20 |
| Robin_Watts | and I have a hacked about version of the app that calls that interface now. | 16:45.46 |
| tor8 | I whipped up a prototype of how something using raw 1-to-1 function mappings just passing not at all typesafe longs for all pointers | 16:45.47 |
| | Robin_Watts: ah, fab! | 16:46.00 |
| Robin_Watts | That works, but there are lots of things commented out (like form editing etc) | 16:46.21 |
| | Also, in theory mupdf can drive java interfaces, or java interfaces can drive mupdf devices now. | 16:46.43 |
| tor8 | I expect a lot of the form editing interfaces need a bit of a revamp on the C side | 16:46.44 |
| Robin_Watts | tor8: I'd have to look at how you've done that to understand, I suspect. | 16:47.10 |
| HenryStiles | tor8: right I spoke briefly with sebras but as we discussed at the meeting we need to compete, pspdfkit is way ahead on a lot of this stuff. | 16:47.14 |
| tor8 | HenryStiles: yeah. we need to be able to expose easy to use interfaces for editing annotations, forms and nice plug-n-play UI components for android and ios | 16:48.49 |
| rayjj | HenryStiles: maybe those comments should be on #artifex ;-) | 16:49.00 |
| tor8 | Robin_Watts: on tor/master | 16:51.04 |
| Robin_Watts | rayjj: I'm confused. | 16:52.02 |
| HenryStiles | tor8: I don't know if it is best for sebras to work directly on that stuff or work on bugs and other busines and free others up. | 16:52.17 |
| Robin_Watts | gx_saved_pages_list_print loops around, calling gx_output_saved_page. That I understand. | 16:52.52 |
| | But then in out: it calls gx_saved_page_load. What's that about ? | 16:53.18 |
| HenryStiles | rayjj: yeah maybe artifex for that one... | 16:53.30 |
| sebras | tor8: HenryStiles: I'd need to read up on JNI a bit to contribute in that area. | 16:54.21 |
| tor8 | sebras: it would be nice to have someone triage the bugs and knock down the growing list. | 16:54.24 |
| Robin_Watts | sebras: I think there are 2 layers of stuff needing doing in tor8's list. | 16:55.05 |
| tor8 | sebras: http://twiki.ghostscript.com/do/view/MuPDF/Tasks | 16:55.17 |
| Robin_Watts | "expose easy to use interfaces for editing annotations and forms" is 1 thing (in C). | 16:55.29 |
| HenryStiles | tor8: I do feel something like that is more in line with his hours until we can get up to 20 hours a week. | 16:55.37 |
| Robin_Watts | "nice plug and play UI components for android and ios" is another. | 16:55.41 |
| rayjj | Robin_Watts: when we print, the clist files are supposed to be "empty" ones (since it is after processing a file, and we just done a "fillpage" for the next page, so we grabbed that using gdev_prn_save_page into saved_page and go back to it after printing | 16:56.25 |
| Robin_Watts | There will also be "expose the C interfaces for annotations and forms into ObjC/Java", but that can come later - and that's where JNI will be required. | 16:56.32 |
| sebras | tor8: if a bug is assigned to someone does that mean that ehy are actively working on it? | 16:56.59 |
| | tor8: or can bugs be forgotten in that state? | 16:57.07 |
| rayjj | Robin_Watts: sorry -- I have to do something for a bit, and it sounds like you're busy with sebras. | 16:57.18 |
| | bbiab | 16:57.23 |
| Robin_Watts | rayjj: So gx_saved_page_load is NOT loading a saved page ? | 16:57.31 |
| tor8 | sebras: in mupdf, most likely not. but it's best to ask on irc before, just in case. | 16:57.48 |
| | sebras: but that does pose a problem if you work on weekends :) then again, if you fix it over the weekend, I don't see the problem. | 16:58.16 |
| sebras | tor8: I would be ok with me to have a look at those bugs to start with. | 16:58.25 |
| | tor8: right, or I can hassle you guys in the evenings. | 16:58.49 |
| Robin_Watts | tor8: If you see a bug that's assigned to us, just hassle us here. | 16:59.01 |
| sebras | Robin_Watts: will do. | 16:59.28 |
| tor8 | sebras: there's also a long and growing list of epub bugs, and there are a few missing features. that could also be a place to start, but I expect that having you read up on HTML and CSS layout your first week in is going to send you screaming into the hills... | 17:00.16 |
| sebras | tor8: not unlikely, no. | 17:00.40 |
| tor8 | I would suggest that you focus on the PDF and XPS bugs primarily | 17:01.08 |
| | the apps secondarily; I expect them to get big overhauls or rewrites this year so spending a lot of time on them is inadvisable | 17:01.40 |
| sebras | tor8: for epub related bugs I could at least try to confirm that there is an issue. | 17:02.04 |
| tor8 | sebras: I've gotten a whole bunch of font related epub bugs submitted just last week | 17:02.27 |
| | ignore those :) | 17:02.36 |
| HenryStiles | tor8: what do you have xps bug wise? Are they labelled separately in bugzilla? | 17:02.48 |
| tor8 | simple layout issues with missing or wrongly applied CSS attributes should be doable | 17:02.53 |
| | HenryStiles: only a handful really old ones; there's a couple of odd matrix transform issues I just don't understand. | 17:03.30 |
| HenryStiles | bug triage and have a shot at fixing easy ones sounds like the best course to start for sebras ... checking in with tor8 to see if a particula bug should be attempted. | 17:09.58 |
| | s/have/having | 17:10.08 |
| | sebras: off to Taiwan soon, that's exciting, how did you come upon doing that? | 17:13.01 |
| sebras | HenryStiles: yes, I'll probably be meeting up with him IRL to discuss the specifics. | 17:15.08 |
| | HenryStiles: I have been wanting to learn chinese in taiwan for some time, but have not had the chance to do it until 2016. | 17:20.51 |
| | ok, I need to head out, but I'll discuss the buglist with tor8. | 17:23.42 |
| rayjj | Robin_Watts: sorry. back now. at out: saved_page_load _is_ loading a saved page saved upon entry to list_print | 17:28.32 |
| Robin_Watts | so gx_saved_page_load is the reverse of gdev_prn_save_page ? | 17:29.28 |
| rayjj | the goal was to restore all of the stuff captured by getdeviceparams that were current before starting to print. | 17:30.55 |
| Robin_Watts | gdev_prn_restore_page might have been a nicer name then. | 17:31.02 |
| rayjj | Robin_Watts: intended to be | 17:31.07 |
| Robin_Watts | And we aren't really restoring the state properly. At best it opens a new file onto the old one. | 17:31.44 |
| rayjj | well, it makes more sense during printing to "load" the page we want to print | 17:31.46 |
| | but after printing the intent is to "restore" | 17:32.18 |
| Robin_Watts | I don't think we can fclose and then later fopen. | 17:33.33 |
| rayjj | so, maybe re-factoring a 'prn_device_state_save' and 'prn_device_state_restore' and using that part in save_page and load_page (which mess with the files) might be better | 17:34.02 |
| Robin_Watts | yeah. | 17:34.08 |
| | will fiddle with that. | 17:34.17 |
| | Thanks. | 17:34.19 |
| rayjj | it can still use the 'saved_page' structure -- just with the {c,b}file, {c,b}fname etc. from the prn_device so the 'restore' would just copy them back without fopen/fclose | 17:38.16 |
| | Robin_Watts: one thing I don't understand is how paulgardiner handles avoiding the automatic 'delete on close' when gdev_prn_save_page closes the file | 17:39.20 |
| | it must work, and I recall asking him at the time, but I never pursued that | 17:39.51 |
| Robin_Watts | rayjj: The first time we open a file, it's opened with it's real filename. | 17:40.58 |
| | Sorry, let me try that again. | 17:42.50 |
| | When clist_fopen is called, it's initially called with an empty filename. So we make a scratch file. | 17:43.16 |
| rayjj | Robin_Watts: yeah, but when we call fclose in gdev_prn_save_page, why doesn't that delete the clist file on disk ? | 17:43.17 |
| Robin_Watts | We then write a MAGIC filename back into the clist_file_ptr. | 17:44.00 |
| rayjj | Robin_Watts: we call it with "false" as the last param, so it is not supposed to delete it | 17:44.01 |
| Robin_Watts | Any subsequent opens of that MAGIC filename give a new pointer to the same thing back. | 17:44.41 |
| rayjj | OK, and the call to "close" the MAGIC pointer just flushes it if the io_procs->fclose doesn't pass "delete" == true | 17:46.01 |
| Robin_Watts | yes, I think so. | 17:46.44 |
| rayjj | hmm... clist_fclose doesn't flush if delete == false. It probably should (it just returns 0) | 17:47.56 |
| | on linux and windows, where we use the gp_fpwrite functions it "goes around" the FILE * buffers and writes directly to the OS handle (unbuffered IO) so it doesn't matter, but if the system doesn't support gp_can_share_fdesc we really ought to call fflush (it couldn't hurt) | 17:55.05 |
| | but what I don't know is if we are using unbuffered IO, will calling fflush on the FILE * mess things up (the fflush won't know what's been written "around" it and the position will be "stale") | 17:56.26 |
| Robin_Watts | rayjj: I've plausibly fixed that problem. | 18:12.27 |
| | but I'm now hitting an infinite loop. | 18:12.38 |
| | rayjj: So in gx_saved_pages_list_print, we call close_writer_and_init_reader | 18:17.33 |
| | The first thing that does is to write and end_page - so we can't have the file pointing at the page we are hoping to print at this point. | 18:18.43 |
| | The next thing that does is to try and read the color usage array out of the file, so we MUST have it pointing at the page we want to print. | 18:19.06 |
| | So, we're buggered either way, as far as I can tell. | 18:19.19 |
| mvrhel_laptop | so this bug is very odd. During the clist writing, we end up doing a soft mask push which changes the color space, we have the copy alpha which uses the color gray color value like it should and we then do the pop of the mask. During play back we do the push of the mask, followed by a pop of the mask and then do the copy alpha | 22:46.35 |
| | need to chat with ray about this | 23:00.10 |
| | Robin_Watts: are you still around? | 23:13.42 |
| | it may be a bit late | 23:14.03 |
| Robin_Watts | I'm here. | 23:47.17 |
| | Forward 1 day (to 2015/12/30)>>> | |
IRC Logs