| | <<<Back 1 day (to 2014/03/20) | 2014/03/21 |
| Robin_Watts | mvrhel_laptop: Will do in the morning. | 00:36.12 |
| kens | chrisl when you have some free time could you ping me please ? Warning, need some help with memory problems..... | 09:20.27 |
| Jogux | pedro_ : the --windows8 build got a lot further on the vm then failed due to an expired certificate | 10:35.42 |
| kens | I've got some window installers here this morning, so I may be AFK without warning, and there could be some network outages as well.... | 10:36.34 |
| Jogux | I'd make some droll linux vs windows joke, but not enough caffeine yet. | 10:40.16 |
| kens | Well its really more a case of them unplugging my router..... | 10:40.38 |
| pedro_ | Jogux> cool | 10:41.10 |
| chrisl | kens: sorry, I've working on the other computer - what do you need? | 10:49.33 |
| kens | I've got a weird memory problem, I need to talk it through with someone who knows the memory manager better than me ;-) | 10:50.31 |
| chrisl | Eek... :-( | 10:50.47 |
| kens | Yeah.... | 10:50.56 |
| | Let me know when yhou have some time | 10:51.02 |
| chrisl | I'm good now - shoot | 10:51.28 |
| kens | OK so this is bug #693561 | 10:51.42 |
| | Alex is correct about the inefficiency and I have a patch to fix that. | 10:51.56 |
| | The fact that I'm using param lists is because I was assured this was the way to go, I;'d actually prefer to use a spec_op now we have them, but before changing it I need to understand the bug. | 10:52.34 |
| | Now in zfunc4.c there's a call to gs_getdeviceparams which trickles all the way through to pdfwrite. By tedious debugging I find that in gdevpsdp.c I read some mono image dict paraqeters | 10:53.59 |
| | ANd that's what causes the problem. I have to run this iwth -Z? in order to do pointer checking | 10:54.19 |
| | Which causes an error "Reference to free object 0x240a610(60), in chunk" | 10:54.45 |
| | I turned on -ZA to follow the allocations, and I see that the param list call allocates some memory for a gc_param_list structure, and its this which the allocator is complaining about. | 10:55.35 |
| | I see the memory allocated, and I see it freed. | 10:55.46 |
| | Then (much l;ater) I see the chunk being closed and after that it complains about the pointer being free. | 10:56.10 |
| | So I'm puzzled, if the chunk is 'closed' isn't it free ? If not what does clsing it do..... | 10:56.35 |
| chrisl | No, the chunk being closed means more memory shouldn't be allocated from it, it doesn't mean it's all free | 10:57.15 |
| | So, IIRC, we can close the current chunk due to a Postscript save, for example | 10:57.55 |
| kens | OK. I'm as sure as I can be that *nothing* is referencing the memory that was allocated. I only use the param list to get one boolean variable (not the one causing the problem). I free the list immediately after I retrtieve the value, so it is correctly freed. I can't see what would be maintaining a reference to the structure :-( | 10:58.36 |
| | I'm presuming that the param list code correctly closes all references to the objects in the list of course. | 10:59.22 |
| chrisl | I'm just doing a fresh build, and I'll have a look - how complex is the test file? | 10:59.43 |
| kens | Huge | 10:59.49 |
| | 3Mb of PostScript | 10:59.55 |
| | And you need a simple change I've made as well, in order to replicate my findings. Hold on and I'll send yo ua modified zfunc4.c | 11:00.20 |
| | The test file is referenced in the bug report | 11:01.08 |
| | File is on its way | 11:01.36 |
| | With that file, at line 319 in zfunc4.c should be: | 11:02.08 |
| | gs_c_param_list_write(&list, i_ctx_p->pgs->device->memory); | 11:02.08 |
| | code = gs_getdeviceparams(i_ctx_p->pgs->device, (gs_param_list *)&list); | 11:02.08 |
| | Oops | 11:02.10 |
| | " /* Check if the device allows the use of repeat in functions */" | 11:02.31 |
| | " /* We can't handle 'repeat' with pdfwrite since it emits FunctionType 4 */" | 11:02.31 |
| | " gs_c_param_list_write(&list, i_ctx_p->pgs->device->memory);" | 11:02.32 |
| | " code = gs_getdeviceparams(i_ctx_p->pgs->device, (gs_param_list *)&list);" | 11:02.32 |
| | If you set a breakpoint on gs_c_param_list_write, with a hit count of 18, that's the allocation which matters. | 11:03.05 |
| | The command line I'm using is : | 11:03.28 |
| | -Z? -sDEVICE=pdfwrite -sOutputFile=\temp\out.pdf -dCompressPages=false -dCompressFonts=false | 11:03.28 |
| | Add -ZA for the allocation dump, but its 1`3Mb of output, so I wouldn't advise that immediatley. | 11:04.10 |
| | 13Mb* | 11:04.21 |
| | Hmm, I wonder if this is some kind of bizarre save/restore problem | 11:08.28 |
| chrisl | So, I'm a bit confused: in check_psc_function() we initialise plist..... | 11:09.08 |
| | Then from psdf_get_image_dict_param () we appear to initialise the same parameter list? | 11:10.07 |
| kens | Umm where's the initialisation ? | 11:10.40 |
| | It may not be the same list.... | 11:10.58 |
| chrisl | line 319 in zfunc4.c gs_c_param_list_write(&list, i_ctx_p->pgs->device->memory); | 11:11.08 |
| kens | I thougth param_llist_write just set the procs to the write procs rather than the read ones | 11:11.40 |
| chrisl | Well, it sets the mem pointer, which may be the critical thing here | 11:12.16 |
| kens | Hmm, yes possibly it is | 11:12.33 |
| | Looks like psdf_get_image_dict_param maybe shouldn't be meddling with that | 11:13.15 |
| | It might explain why its a save/restore problem | 11:13.28 |
| | I see, psdf_get_image_dict_params also calls param_begin_write_dict | 11:14.16 |
| | It does seem to be the same list as well. | 11:14.50 |
| | Where is it setting mem ? | 11:15.23 |
| chrisl | in gs_c_param_list_write() | 11:15.59 |
| kens | As far as I can see that just uses the existing value of mem | 11:16.21 |
| chrisl | hang on - phone | 11:16.30 |
| kens | cplist = plsit, and it sets the mem to cplist->memory | 11:16.40 |
| | Hmm..... | 11:20.10 |
| | Ah, its a different list | 11:21.18 |
| chrisl | Why the hell do we have to set the procs for reading and writing - that's insane..... | 11:21.30 |
| kens | Tell me about it..... | 11:21.39 |
| | But the memory is set for 'dlist' which is a new list, not the existing one | 11:21.54 |
| chrisl | So we have a param list, one entry of which is a param list? | 11:22.46 |
| kens | That does seem to be the case yes | 11:22.57 |
| | Actually it looks like its reading a dict into a new list, which it possibly then appends to the existing list | 11:24.20 |
| | 'dict' is set up as a param list (it starts as a gs_param_dict), we then call param_list_copy which copies from one param list to another, which suggests we are passing in a param list. I'm confused now...... | 11:25.45 |
| | Oh I see. We seem to be maintaining some param lists in the device structure | 11:26.41 |
| | Now that's a hideous notion | 11:26.54 |
| | Presumably we use the param list stuff to set/reset/retrieve these values | 11:27.34 |
| | So what we are doing here is copying the param list from the device structure into the param list we are returning | 11:28.00 |
| chrisl | I don't understand param_list_copy() | 11:29.41 |
| kens | Any particular bit ? | 11:30.06 |
| chrisl | The copy_persists bit - I don't see where that influences the decision to take a reference or make a full copy | 11:30.30 |
| kens | IIRC it compares the memory pointers | 11:30.46 |
| | Yes " bool copy_persists = plto->memory == plfrom->memory;" | 11:31.09 |
| | If the lists have the same memory pointer then it makes them persistent | 11:31.34 |
| chrisl | Yes, but then copy_persists only seems to be used to set a flag later on | 11:31.44 |
| kens | True | 11:32.28 |
| chrisl | My reading of the comment is that if they are using the same allocator, we can use a pointer reference, otherwise we need a full copy of the content.... but I don't see where that is applied | 11:32.44 |
| kens | Essentially yes I believe that's teh case. | 11:33.09 |
| | If its persistent, tehn we don't free it when we free the list (I believe) as it is a pointer to a different list, and will be freed when that one is. | 11:33.39 |
| | Of course this is dangerous, because I see no reaosn why we couldn't free the previous list first, leaving a dangling pointer | 11:34.03 |
| | It does seem rather mad though, that arrays and dicts don't get set that way | 11:34.43 |
| | whereas strings, names int/float/string arrays do | 11:35.05 |
| | Possibly those are always copied ? | 11:35.28 |
| chrisl | Possibly, I still not clear where strings/names are actually copied | 11:35.59 |
| kens | "param_read_typed(plfrom, string_key, &value)" I think | 11:36.26 |
| | line 142 | 11:36.38 |
| | It is possible I'm using this param list stuff incorrectly of course. | 11:37.25 |
| chrisl | Of all the over complicated, spaghetti code nonsense....... | 11:38.47 |
| kens | nods sadly | 11:39.00 |
| | Yeah that does seem to be where it is copied | 11:39.32 |
| chrisl | And now, suddenly, applying a hit count to my breakpoints is causing a massive slowdown :-( | 11:46.58 |
| kens | Ran out of hardware registers ? | 11:47.18 |
| chrisl | Possibly.... | 11:47.36 |
| Robin_Watts | OK, so the new picsel source tree git conversion is online in /home/git-private/sot.git | 12:13.04 |
| | Jogu, pedro, paulgardiner: ^ | 12:13.15 |
| | actually, it's still unpacking. Give me a couple of mins. | 12:13.36 |
| | Can anyone see anything wrong with it? :) | 12:13.54 |
| | There is also an sot.tgz in the same dir | 12:15.13 |
| Jogux | let us know once it's ready? | 12:16.08 |
| Robin_Watts | Should be good to go. | 12:19.17 |
| Jogux | clones locally | 12:20.49 |
| chrisl | kens: I'm going to lunch (my father's here) - I'm wondering if the problem is triggered by the recursive call to check_psc_function()...... | 12:21.26 |
| kens | chrisl I'll look at that, i@m going to get some lunch myself as well | 12:21.48 |
| Jogux | and pedro_ are about to pop out for lunch too. | 12:22.23 |
| pedro_ | thinks this morning went too fast | 12:24.31 |
| chrisl | 's father has just gone out again, so have a bit longer before lunch...... | 12:28.46 |
| kens | has to go see to windows people, then will eat. | 12:32.21 |
| | I don't think the recursion is a problem, but after lunch I'll alter it so it doesn't recurse and see what happens (or rather I'll move the param stuff out so it isn't in the recursion) | 12:33.04 |
| chrisl | No I agree - I think it's just coincidence I see the problem during the recursion | 12:33.33 |
| kens | Robin_Watts : I can guess at the problem here, but if you or tor8/paulgardiner could answer it,it would be better: | 12:53.49 |
| | http://stackoverflow.com/questions/22558597/mupdf-how-to-open-pdf-file-with-openbuffer-method | 12:53.49 |
| | chrisl I moved the param list code to the functions calling check_psc_function and it still causes a problem with a free object | 13:03.16 |
| paulgardiner | Thanks kens. I've guessed at an answer. | 13:10.18 |
| kens | thanks paulgardiner | 13:10.27 |
| | Your guess wil be better than mine | 13:10.33 |
| Robin_Watts | Missing fz_register_document_handlers() ? | 13:10.44 |
| kens | I'm assuming that, but I could be mistaken | 13:10.59 |
| paulgardiner | robin_watts that was my guess too :-) | 13:11.30 |
| | Where is it safe to create private repos within casper home dirs? Is it only "repos" that is shown on the source web page? | 13:12.58 |
| Robin_Watts | paulgardiner: Indeed. | 13:15.11 |
| | so anywhere other than repos. | 13:15.19 |
| paulgardiner | ta | 13:15.36 |
| | robin_watts your sot repo includes CVSROOT. Is that intentional? | 13:17.51 |
| Robin_Watts | paulgardiner: Oh, urm, probably not :( | 13:18.10 |
| paulgardiner | Easily fixed with filter-branch though | 13:18.32 |
| Robin_Watts | Do you want to have a crack at that ? | 13:19.01 |
| paulgardiner | Sure | 13:19.17 |
| | robin_watts any particular machine that can most easily take the load? | 13:23.19 |
| | ... on casper at the moment | 13:23.28 |
| Robin_Watts | The data is already on peeves. | 13:23.32 |
| | peeved, sorry. | 13:23.36 |
| | in ~robin/PicselCVS | 13:23.41 |
| | as sot.git | 13:24.23 |
| | and is backed up in sot2.tgz | 13:25.01 |
| Jogux | returns from lunch. | 13:35.48 |
| | Robin_Watts : hey, remember when I said "everyone always followed the process carefully and tagged resources when they tagged epage"... | 13:49.18 |
| pedro_ | grins | 13:49.43 |
| | I like optimism | 13:49.50 |
| Jogux | well, I was wrong; certainly for the latest smartoffice release tag. but we can work around that easily enough I'm sure. Probably. Maybe. | 13:50.18 |
| | basically resources isn't present on that tag. I'm struggling to get my head around how that actually works. | 13:51.14 |
| | otherwise, from a couple of diffs, it at least seems consistent with the previous repository. | 13:51.34 |
| henrys | chrisl: how's the directory reorg toward language switching coming? I know ray was looking for his next project and I was hoping you were near a point where he could get involved. | 13:53.46 |
| Jogux | marcosw_ : good morning morning - how are the ATS vms, do you think me/mace can start trying to get the system running? | 13:57.11 |
| chrisl | henrys: I've only looked at it briefly since the staff meeting | 13:58.21 |
| pedro_ | reboots to convince PC that the left mouse button really wasn't continually down | 14:00.02 |
| marcosw_ | Jogux: I haven't secured my home network yet, I'll work on that today. | 14:02.37 |
| Jogux | marcosw_ : great, thanks! | 14:03.00 |
| marcosw_ | I do have all the vms stored on a local usb drive. | 14:03.21 |
| | and i think networking is working. | 14:03.52 |
| paulgardiner | pedro_: my vs2010-built dispman-fileviewer is still failing. It seems to be looking for fonts in file://localhost/epage_dir/Fonts/ and not finding them | 14:05.43 |
| chrisl | henrys: I got to the point where I *think* all the object files are being built on Windows for the three products, but it fell down at the link stage - I was *hoping* to the get the release out the way before picking it up again | 14:05.55 |
| paulgardiner | For this build should that map to a memory file system? | 14:06.20 |
| pedro_ | paulgardiner: I didn't have to do any setup for fonts (and dont have an epagedir explicitly set up | 14:07.27 |
| | paulgardiner: do you have an epagedir explicitly set in your env perchance | 14:08.39 |
| henrys | chrisl: sounds good. Ray is a windows guy and he could look at what you have for windows now - test review that kind of thing - using your private repo for sharing ⦠but whatever you think best. | 14:09.04 |
| Jogux | epage_dir should be in romfss. | 14:09.21 |
| paulgardiner | pedro_: I do. I'll unset it | 14:09.40 |
| chrisl | henrys: Okay, that's cool. I'd *really* hoped to get each product linking before pulling Ray in for the romfs rejig, but..... | 14:11.11 |
| henrys | chrisl: I did want to get him in early because he works the most with printer customers more than all of us and he could see something as the system evolves that could be better or different. But as you know adding another person to a project is not always the right play. | 14:13.59 |
| ray_laptop | I coded up the SSE2 landscape case for 4-bit and it's working, but it only seems to buy about another 6% on J8_landscape.ps (a full page image) | 14:14.38 |
| | I guess I did too well in coding up the C variant :-) | 14:15.06 |
| | chrisl: pull away | 14:15.35 |
| | chrisl: if you have something you want me to pull in from your private repo, let me know | 14:16.06 |
| | chrisl: and a statement from you as to how you want the romfs to work (here or email) would be a good thing to start with (I hesitate to label it a 'spec') | 14:17.27 |
| chrisl | ray_laptop: I was just going to push my latest changes to my repo, and git is throwing a fit - it seems to be rebasing the wrong way round, so give me a little while to smack it into shape | 14:17.43 |
| ray_laptop | chrisl: I often just to push -f to my repo | 14:18.09 |
| | s/to/do/ | 14:18.19 |
| chrisl | ray_laptop: I which I had done that.... :-( | 14:18.27 |
| | s/which/wish | 14:18.34 |
| Jogux | chrisl : you probably still could if you abort rebase and reset back to where you were? | 14:19.13 |
| chrisl | Jogux: I just did that - I want to make sure I won't lose anything before I push -f though | 14:19.39 |
| ray_laptop | I sometimes do a fomat-patch just to make sure I don't lose local changes :-) | 14:20.18 |
| chrisl | ray_laptop: so, the romfs stuff. At the moment the content of the romfs is built as part of the graphics library "component", which makes it impossible to build a language agnostic graphics library | 14:22.45 |
| | ray_laptop: so I think it would be preferable to have the romfs data be built with the interpreter ("client"), rather than with the graphics library. That *may* mean providing a method for the client to "register" a romfs "root" during initialisation | 14:27.12 |
| ray_laptop | chrisl: so when each language starts, it adds it's romfs to what's available (multiple subdirectories at the root level ?) | 14:29.43 |
| | such /Resource for PS, /PCLFonts and possibly /iccprofiles for all ? | 14:30.53 |
| | such as /Resource for PS, /PCLFonts and possibly /iccprofiles for all ? | 14:31.02 |
| chrisl | ray_laptop: something like that, yes. *Or* the library can only have one romfs "registered" at a time, and the process of switching languages involves registering the new language romfs..... | 14:31.30 |
| marcosw | Jogux: how do I want to give mace a login on the machines in my garage? Is there a public key for him? | 14:32.07 |
| ray_laptop | chrisl: adding subdirectories seems better. Some stuff, like iccprofiles are language independent | 14:32.28 |
| chrisl | ray_laptop: true, it would be preferable. The only thing we have to worry about is PS, since programs can enumerate the file system contents...... | 14:33.22 |
| Jogux | marcosw : hm. Not sure I have that handy; if you give me sudo access I can add his public key later? | 14:33.39 |
| ray_laptop | chrisl: and why is that a worry ? | 14:33.47 |
| chrisl | ray_laptop: I'm not sure it is a worry, it was just one of the things I thought of that *might* influence things | 14:34.29 |
| marcosw | Jogux: that works | 14:35.08 |
| chrisl | ray_laptop: TBH, this might be something you'd be better working on on the master code, since you'd at least be able to cluster test the resulting executables - at the moment, the build_consolidation branch cannot be cluster tested (the pcl6 and gxps executables build on Linux, but don't have the correct romfs contents). | 14:36.39 |
| ray_laptop | if some whacko PS CET enumerates /* then it will get different results depending on what other parsers have added to the romfs, but that's no different to running without compile inits and having it enumerate the OS filesystem | 14:37.09 |
| henrys | marcosw: I sent a response to support about guillaume's problem. | 14:37.15 |
| | marcosw: If you're busy I'll follow up with him. | 14:37.38 |
| ray_laptop | chrisl: and AFAIK, the CET doesn't ever enumerate %rom% | 14:38.03 |
| chrisl | ray_laptop: that's true, but I guarantee we'll get a bug report at some point that doing filenameforall on gs gives different results to those on a (future) language_switch build! | 14:38.29 |
| ray_laptop | chrisl: I can type WONTFIX pretty quickly :-) | 14:39.06 |
| chrisl | :-) | 14:39.22 |
| ray_laptop | chrisl: but I like the idea of a parser _init function adding its romfs contents to the %rom% system | 14:40.21 |
| | sort of like a 'mount' unix action | 14:41.16 |
| chrisl | ray_laptop: how about having the graphics library have a romfs (for common files) and then the ability for each interpreter to register it's own? That would work nicely for the iccprofiles, and give us flexibility if we wanted to share fonts in the future (also for UFST, where we can have common FCOs for all languages). | 14:41.25 |
| ray_laptop | chrisl: exactly what I was trying to portray. You just said it more clearly | 14:41.57 |
| chrisl | It would be nice if the graphics library also registered it's own romfs root via the same API call..... | 14:42.38 |
| ray_laptop | sure, why not? | 14:42.59 |
| chrisl | ray_laptop: such consistency is not always the Ghostscript way...... | 14:43.25 |
| ray_laptop | so the %rom% IODevice just copes with a list of 'directory segments' | 14:43.54 |
| | and each directory segment has a collection of files (recall that the filesystem is actually 'flat', like Adobe's) | 14:45.45 |
| | chrisl: so each parser component will run its own 'mkromfs' to make its set of files, right ? | 14:47.03 |
| chrisl | ray_laptop: yes, exactly | 14:47.23 |
| | Obviously, mkromfs will still be built by the graphics lib makefiles, and the MKROMFS_XE value set there for each interpreter to use | 14:49.01 |
| ray_laptop | chrisl: sounds good | 14:49.25 |
| chrisl | ray_laptop: another possibility would be to have each "registration" happen as a "new device" - so we'd have '%rom%' for the shared one, '%pclrom%', '%psrom%' etc for the language specific ones - if that makes it easier to retain the "flat" structure..... | 14:51.57 |
| | ray_laptop: but that might make things hard for libfile searches...... | 14:53.14 |
| ray_laptop | chrisl: Then we'd have to dynamically add IODevices, and that's harder, IMHO | 14:56.37 |
| | chrisl: I like just having one %rom% with contents added by registration from the parser as they init | 14:57.24 |
| chrisl | ray_laptop: that's fine, it was just thought..... | 14:57.37 |
| tor8 | Robin_Watts: ping | 14:58.26 |
| sebras | tor8: EHOSTUNREACH: No route to host | 15:00.02 |
| norbertj | henrys: hello, I found the problem in the jpegdecoding, but have not found a fix yet (see 695100) | 15:01.55 |
| henrys | norbertj: reading it now | 15:03.40 |
| | norbertj: it seems like this would come up more often, we don't see much jpeg though. | 15:07.10 |
| norbertj | henrys: and you only see this if the image is made up of seperate bands positioned adjacent to each other. If this happens the remaining scans of such a band just creep up, so if its a large band only some scans at the end will be missing. | 15:10.16 |
| Robin_Watts | tor8: pong, sorry, didn't see you. | 15:10.31 |
| | oh, that reminds me, I need to look at mvrhel_laptop's stuff. | 15:11.16 |
| marcosw | Jogux: I've sent you email with login information on my home machines. | 15:12.41 |
| Robin_Watts | mvrhel_laptop: (For the logs) Looks good to me. | 15:12.45 |
| | marcosw: Picsel used to have a domain set up 'intranet.picsel.com' which had all sorts of resources on. | 15:13.40 |
| | The twiki, bugzilla, the review system etc. | 15:13.49 |
| | Jogux found the VM for this. | 15:14.05 |
| Jogux | (it's the intranet2 one) | 15:14.31 |
| | marcosw : awesome, thanks, trying now | 15:14.38 |
| Robin_Watts | It used to be that that was only visible on the intranet, unless you ssh tunneled in or had a VPN or something. | 15:14.40 |
| tor8 | Robin_Watts: I've got a commit on tor/master that fixes the MSVC project files | 15:14.56 |
| | there's no v8 version, I'm not sure if you want to keep that as an option, but it'll make the project files more complicated | 15:15.23 |
| Robin_Watts | If we can get that same VM up and running again on one of your machines, we could do the same trick. | 15:15.25 |
| Jogux | robin_watts : I'm sure we had https password protected access at one point, though I think it got stamped on by the "security" people at some point for being far too user friendly or something. | 15:15.35 |
| Robin_Watts | Jogux: We did. Paul and I used to use it. | 15:15.48 |
| | And then Ali wossname came over all BOFH and cut it off. | 15:16.04 |
| Jogux | yeah | 15:16.08 |
| | he tried to take my ssh access away. (he failed ;) ) | 15:16.14 |
| Robin_Watts | but then we carried on with ssh tunneling. | 15:16.24 |
| pedro_ | they didn't really provide any failsafe for the replacement VPN going fut or being inaccessible from customer sites etc | 15:16.59 |
| | nice'n'corporate approach | 15:17.08 |
| Robin_Watts | marcosw: So could we try to get that VM up and running too please? | 15:17.46 |
| | That VM ran bugzilla, and ats and the review system, right? | 15:18.21 |
| marcosw | robin_watts: the internet tube to my machines is small for that kind of thing, better we set it up on one of the machine in Miles' office (which have the additional advantate of static ip addresses). | 15:18.31 |
| Jogux | yes. and pip and something. | 15:18.38 |
| | I think we need to colocate it with ats possibly, at least temporarily. | 15:19.07 |
| Robin_Watts | marcosw: I don't think we need massive bandwidth in/out. | 15:19.10 |
| | It's more important to be (as Joseph says) colocated with ats, I think. | 15:19.35 |
| | marcosw: What would be involved in getting better bandwidth to your house? Is cable an option or are you as good as you can get already? | 15:20.10 |
| marcosw | okay, I can (try to) set that up. Or Jogux can do it, since he has access to (some of) my machines now. | 15:20.16 |
| Robin_Watts | certainly paul and I used to use this stuff over dial up modems :) | 15:20.29 |
| marcosw | robin_watts: I have cable now, there is a faster tier but I'm not sure how much faster it is. Give me a sec and I'll check. | 15:21.02 |
| Jogux | marcosw : it's the upstream that's important I would imagine | 15:21.16 |
| Robin_Watts | tor8: So we're ditching v8 and no-js builds and we always build with mujs ? | 15:22.32 |
| tor8 | Robin_Watts: that was my thought | 15:23.08 |
| chrisl | kens: I'm confused by this: we seem to dip into and out of c_param and ref_param atr various points........ | 15:23.17 |
| tor8 | the unix makefiles can still build with v8, jscore or no-js | 15:23.20 |
| Robin_Watts | And it's a standard thirdparty dependency now, like jpeg or freetype? Seems entirely reasonable to me. | 15:23.31 |
| kens | chrisl, sorry I started working on a cut-down file | 15:23.35 |
| Jogux | marcosw : I'm into fermis successfully. | 15:23.47 |
| henrys | norbertj: I think I see it, the read_bitmap return value is the number of scan lines processed but in pl_image_data we always call with 1 - so code should be checked to be >= 1 and then passéd as the 5th parameter to pl_image_data. Is that something you already looked at? | 15:23.49 |
| kens | I don't recall seeing anything with ref_param | 15:23.52 |
| tor8 | Robin_Watts: yup. I've placed it in next to jbig2dec on our git server. | 15:23.52 |
| Jogux | marcosw : it's a bit laggy just now :-) | 15:24.02 |
| Robin_Watts | tor8: Then consider the review passed. | 15:24.06 |
| paulgardiner | thinks it would be bad if we broke v8 or jscore builds and didn't realise it... although I guess we'd know when a commit was likely to do so. | 15:24.37 |
| henrys | norbert all that is in pxReadImage | 15:24.38 |
| | norbertj: ^^^ | 15:24.46 |
| chrisl | kens: yes, some the values we're reading back come from a ref_param, and are written to a c_param - I'm not sure if/how those interact | 15:24.46 |
| kens | Err, that doesn't sound ideal..... | 15:24.59 |
| tor8 | Robin_Watts: paulgardiner: which of you two knows the android makefiles best? | 15:25.05 |
| kens | FWIW its definitely somethign to do with save/restore | 15:25.11 |
| Robin_Watts | I have new versions of the 'clone' fonts with kerning that should match now. | 15:25.17 |
| marcosw | Jogux: that's because I'm rsycing atsmaster to fermis. | 15:25.24 |
| kens | The problem occurs on a restore, whether its i nthe job or the EOJ restore | 15:25.26 |
| paulgardiner | has finger on nose | 15:25.30 |
| Robin_Watts | tor8: It's temping to say "he does" | 15:25.32 |
| | tor8: but ask away. | 15:25.39 |
| kens | chrisl let me send you a much simpler file | 15:25.44 |
| norbertj | henrys: I found that. I'll think a bit about this. | 15:26.04 |
| chrisl | kens: that doesn't necessarily imply a save/restore problem, it might just be that the restore is first time we notice it | 15:26.09 |
| kens | chrisl yes, true, but its the onl;y time I can see the problem | 15:26.32 |
| marcosw | Jogux: the nodes in my garage are connected to the internets via a powerline ethernet thingy and the bandwidth isn't great. | 15:26.40 |
| tor8 | Robin_Watts: actually, if we're happy to drop v8 with the android build I should be able to dael with it myself | 15:26.56 |
| Robin_Watts | tor8: There is more of an argument for keeping v8 with the android build, cos it's potentially the android build where v8 might be present anyway... | 15:27.57 |
| | but that would require hacks to the makefile anyway... | 15:28.05 |
| | so for simplicity I say ditch it. | 15:28.13 |
| Jogux | marcosw : ahhhh | 15:28.14 |
| marcosw | robin_watts: I think I currently have 25 Mbps down and 4 Mbps up. I can upgrade to 50 Mbps down but they don't quote a up bandwidth for that tier, so I suspect it's only 4 Mbps. Running the various servers you mention also violates my TOS, which I'm not too worried about. | 15:28.16 |
| tor8 | Robin_Watts: fab. I'll do that then :) | 15:28.27 |
| Robin_Watts | marcosw: So the slowness is the powerline adapters? | 15:28.44 |
| norbertj | so then in the dumpbuffer, if gs_jpeg_read_scanlines() has generated the 9th scan, the 8th scan should be copied after the 7th scan and then 2 scans to be returned from read_bitmap(). | 15:28.49 |
| henrys | norbertj: there might have been a design assumption of 1 scan line at a time, and that worked for the other formats - then jpeg was added later. | 15:28.51 |
| marcosw | Jogux: the rsync is supposed to be down in 27 minutes. | 15:28.51 |
| kens | chrisl modified zfunc4.c and test file emailed | 15:28.59 |
| Jogux | marcosw : great, ta. | 15:29.06 |
| kens | chrisl can you point me at the ref_param ? | 15:29.14 |
| chrisl | I'm looking now - it's very confusion :-( | 15:29.29 |
| marcosw | robin_watts: the powerline adapter runs at about 20 Mbps, so for downstream it is the limiting factor but not up. | 15:29.32 |
| chrisl | confusing | 15:29.32 |
| henrys | norbertj: yes "code" should equal 2 after read_bitmap() | 15:29.37 |
| paulgardiner | pedro_: windows exe still not working :-( Now it fails to load null://model/fileviewer.model. Somehow it works its way to File_getLocalFilename and there decides "model" isn't a sensible authority. Any thoughts? | 15:29.45 |
| Robin_Watts | marcosw: Is that real world speed or quoted speed? | 15:30.02 |
| kens | chrisl which file ? | 15:30.04 |
| chrisl | psi/iparam.c | 15:30.19 |
| marcosw | robin_watts: real world, quoted is 100 or 200 (I forget which model I ended up getting). | 15:30.22 |
| henrys | norbertj: I'm concerned though there may be some other 1 scan line assumptions but first I'd try passing code as the 5th parameter and check for explosions ;-) | 15:30.33 |
| kens | grr breakpoint in the wrong place.... | 15:30.49 |
| Robin_Watts | OK. Then that's as fast as my ADSL is then, so as long as no one else uses it while I want to, we're golden :) | 15:30.55 |
| kens | chrisl I never seem to be in iparam.c | 15:31.25 |
| Jogux | paulgardiner : boggle. | 15:31.28 |
| paulgardiner | Jogux: more or less my thoughts | 15:31.49 |
| marcosw | robin_watts: I could run ethernet to my garage, but up until today those machines have only been cluster nodes and so bandwidth has not been an issue. | 15:32.02 |
| chrisl | kens: ah, it's possible I ended up there on a different call - like I say, this stuff is a real mess...... | 15:32.08 |
| kens | Its equally possible I've not detected me getting in there ;-) | 15:32.24 |
| | pdfwrite certainly does a lot with ref_param stuff | 15:32.47 |
| Jogux | so. this new sot git. I can't figure it out. 'resources' isn't on the smart-product-v2_2-release-baseline tag. but it wasn't removed by any of the commits. | 15:32.56 |
| henrys | marcosw: are we getting to the point where we should consider a proper facility for all this stuff? | 15:32.58 |
| Jogux | robin_watts : err, the sot git seems to be missing all the branches? :-( | 15:33.09 |
| Robin_Watts | Jogux: Eh? | 15:33.22 |
| Jogux | wonders if I did something wrong | 15:33.30 |
| marcosw | henrys: probably. | 15:33.50 |
| Jogux | hm. yes. I clearly did | 15:33.50 |
| tor8 | paulgardiner: Robin_Watts: are any of you set up for building the android app? | 15:33.55 |
| Robin_Watts | tor8: We both are. | 15:34.02 |
| Jogux | I cloned it from a clone, the second clone is missing all the branches. odd. | 15:34.05 |
| tor8 | when you got a mo, mind testing the android build fix on tor/master? not sure I'll have time to get my android build system up to date today. | 15:34.33 |
| pedro_ | paulgardiner: are you running the exe in vs2010 or standalone? | 15:34.34 |
| paulgardiner | Jogux: I think you have to create local branches in the first clone for that to work | 15:34.38 |
| Robin_Watts | Jogux: paulgardiner is currently doing a filter-branch to remove the CVSROOT. | 15:34.46 |
| Jogux | paulgardiner : sounds odd! | 15:34.57 |
| Robin_Watts | paulgardiner: Really? That would be against my understanding. a clone should be a complete clone! | 15:35.05 |
| tor8 | Robin_Watts: but if you're neck deep in SOT don't sweat it, I've got all weekend :) | 15:35.16 |
| marcosw | henrys: we don't need much rack space, a total of 6U for the nodes. Power is a bigger problem, we'll need two 15 amp circuits (I think we peak at 24 amps total when running cluster jobs). | 15:35.21 |
| Jogux | robin_watts : but the branches may don't exist in the second clone as such. until you check them out? | 15:35.27 |
| paulgardiner | The clones have only remote branches until you make local copies | 15:35.32 |
| Jogux | yeah, what paul said :-) | 15:35.38 |
| Robin_Watts | That makes more sense. | 15:35.47 |
| Jogux | Robin_Watts : good. but I'm not quite sure how to fix this issue with resources being missing on branches. | 15:36.01 |
| Robin_Watts | tor8: I was looking at the twiki, but if we're going to run a VM, I can stop that. | 15:36.07 |
| | Jogux: Is it a flaw in the original CVS repo ? | 15:36.22 |
| Jogux | robin_watts : I believe so. | 15:36.28 |
| | problem is I can't get my head around how to fix it. | 15:36.49 |
| Robin_Watts | So the problem is that people didn't tag across epage/resources? | 15:36.53 |
| Jogux | robin_watts : yes | 15:36.59 |
| tor8 | Robin_Watts: if it would be a public wiki, please continue setting it up. I've long thought a wiki would be valuable for all our open source projects for stuff that doesn't really belong in bugzilla. | 15:37.00 |
| paulgardiner | robin_watts oh. Might I be running filter-branch on a repo with one branch? | 15:37.02 |
| Jogux | [without fixing cvs then redoing the import] | 15:37.09 |
| tor8 | but if it's only to get the SOT pages back up, then nvm | 15:37.15 |
| paulgardiner | I'll take a look | 15:37.17 |
| Robin_Watts | Jogux: Redoing the import is not impossible. | 15:37.32 |
| | It'll take a day of unattended time. | 15:37.48 |
| Jogux | I think it there was to be a proper artifex wiki, there may be better choices than twiki (eg. foswiki) | 15:37.50 |
| | s/it/if/ | 15:37.55 |
| Robin_Watts | tor8: I'll carry on with twiki.ghostscript.com then. | 15:38.39 |
| marcosw | Jogux: the rsync to fermis finished earlier than estimated, so bandwidth to fermis and plancks should be better. | 15:38.41 |
| Jogux | marcosw : ahh, yes. nicer now :) | 15:38.59 |
| paulgardiner | robin_watts if we need to redo the import. We could this time try importing just the resources and stiching them together. Might be faster | 15:39.09 |
| Robin_Watts | paulgardiner: faster elapsed time maybe. faster in use of engineers time... unlikely :) | 15:39.43 |
| marcosw | and all of the machines are in /ats on fermis now. I'll leave it to you to setup ssh keys and move them to plancks. | 15:39.43 |
| paulgardiner | robin_watts true | 15:39.58 |
| | and good point | 15:40.11 |
| Robin_Watts | Jogux: So, how do we go about fixing the CVS repo ? | 15:40.18 |
| paulgardiner | robin_watts should I kill my filter-branch? | 15:40.27 |
| Jogux | robin_watts : that is also a good question. | 15:40.44 |
| Robin_Watts | paulgardiner: Dunno yet. Let it run until we're sure :) | 15:40.46 |
| paulgardiner | okay | 15:40.53 |
| Robin_Watts | can do questions. It's answers I have problems with. | 15:41.00 |
| Jogux | find all branches that missed resources, figure out date there were created, check out resources from that date, branch resources, repeat... | 15:41.16 |
| paulgardiner | If it has all the branches but with resources not stretching along all of them it might be fixable in other ways | 15:41.27 |
| Jogux | is tor8 our resident git expert? possibly he can tell us how to fix it in git and then we can do it as/when we find it's broken in git? | 15:41.53 |
| Robin_Watts | Jogux: To fix it in git (other than at the tip of all branches) would change all the SHAs that follow on. | 15:43.24 |
| | Actually... how can it not be right? | 15:43.49 |
| paulgardiner | I reckon filter-branch can be tricked into doing it | 15:43.55 |
| Robin_Watts | A given hash in git gives the WHOLE state of the repo, both in epage and resources. | 15:44.30 |
| Jogux | robin_watts: yeah; it would only affect the hashes for /that/ branch aiui though, which is fine. | 15:44.51 |
| | robin_watts : the "how can it not be right" thing is puzzling me. The commit that the branch is taken from has resources; the first commit on the branch does not; the first commit does not delete resources afaics. puzzle. | 15:45.44 |
| marcosw | Jogux: I'm afraid to ask, but why is intranet2 a .raw file and not a .img file like the ats virtual machines. | 15:46.00 |
| Jogux | marcosw: I don't know. I think it's uncompressed, for not obvious reasons. I think possibly a config difference between vhost1 and vhost2. | 15:46.32 |
| | marcosw : if you rsync, enable compression - I think a lot of it is 0s... | 15:46.51 |
| Robin_Watts | Possibly the intranet2 thing was ripped from a real server, where the others were rebuilt from scratch as images ? | 15:47.05 |
| Jogux | robin_watts : that sounds like a reasonable theory, yep. | 15:47.34 |
| Robin_Watts | Jogux: Which branch are you looking at? | 15:47.37 |
| marcosw | file suggests that the .raw file is a disk image | 15:47.38 |
| Jogux | robin_watts : smartoffice-2-1-28-release | 15:48.09 |
| | marcosw : yep. kvm should grok it just fine regardless. | 15:48.23 |
| | I believe / hope / pray. | 15:48.32 |
| marcosw | okay, rsyncing it now. | 15:48.46 |
| kens | chrisl I definitely don't seem to be ending up in iparam.c which function are you in ? | 15:49.28 |
| Jogux | robin_watts : uh, wait, that's not a branch. confuse. | 15:49.37 |
| Robin_Watts | Jogux: Does it matter? In git branch=tag | 15:50.01 |
| chrisl | kens: I'm not getting there from anywhere important for this problem - ignore that. I must have set a hit count incorrectly at some point | 15:50.40 |
| Jogux | robin_watts : not quite. And I don't know. I'm still confused. | 15:50.42 |
| kens | chrisl OK that's fine. DO you see the problem with the much reduced test file ? | 15:51.01 |
| chrisl | kens: yes, I get the same error | 15:51.19 |
| kens | I believe the problem must be that we are sharing something between this param list and a previous one | 15:51.42 |
| Robin_Watts | smartoffice-2-1-28-release is a tag. | 15:52.06 |
| kens | Whenwe free this param list it frees the object pointed to by the previous list, resulting in a free object. | 15:52.08 |
| paulgardiner | pedro_: sorry, only just noticed your reply. Running outside of 2010 works, thanks. Why did that matter? Is it using files other than in resfs/memfs (whatever it is called)? | 15:52.25 |
| Robin_Watts | Midway along the origin/smart-products-v2_1-release_branch branch | 15:52.25 |
| chrisl | kens: yes, something, somewhere is taking a reference to an array of integers parameter, instead of a copy, I think | 15:53.06 |
| kens | That's about as far as I've got with this :-( | 15:53.27 |
| Robin_Watts | and when I'm standing on that tag: git diff master -- resources tells me that the resources have been deleted. | 15:53.32 |
| pedro_ | paulgardiner: shouldn't make any difference at all (I'm runningboth within vs2010 and outside) | 15:54.18 |
| chrisl | kens: this stuff with persistent keys/values seems really flaky to me..... | 15:54.28 |
| paulgardiner | pedro_: strange | 15:54.43 |
| kens | Well I'm definitely concerned about the dangling pointer I mentioned earlier... | 15:54.49 |
| Jogux | robin_watts : right. but there's no commit I can see that deletes resources. | 15:55.45 |
| pedro_ | paulgardiner: you don't have any custom 'run/debug' settings in your project file(s) do you? only thing I can think of is that the execution environment is different (although I can't immediately see what would impact the fs resolution stuff) | 15:56.23 |
| paulgardiner | pedro_: Ah I see. In amongst all the messing about while tracking down the problem with epage_dir, I swapped to a debug version that was partially created outside the FBS. | 15:57.27 |
| pedro_ | ah, cool | 15:58.06 |
| paulgardiner | It all seems happy now. Presumably I should be able to add -debug to the build line to generate a working debug version? | 15:58.13 |
| Jogux | robin_watts : gah. some of the commits start 'Description:' :( | 15:58.18 |
| pedro_ | yes | 15:58.22 |
| paulgardiner | ta | 15:58.30 |
| pedro_ | np | 15:58.33 |
| chrisl | kens: I believe the problem is in c_param_write() | 16:00.09 |
| kens | chrisl any call to psdf_get_image_params seems to cause the problem | 16:00.15 |
| | Hmm c_param_write gets called all over the place I think | 16:00.37 |
| | Or possibly not | 16:01.02 |
| chrisl | Er, actually, I believe wrongly - what a f*cking mess :-9 | 16:01.43 |
| Jogux | robin_watts : Ah. I sort of see. that branch has someone ended up a child of both master and libpng_1_2_2; the latter is missing resources. I wonder if it can be 'deparented' from libpng_1_2_2 and if that would fix it. | 16:02.10 |
| kens | I find that if I comment out all calls to psdf_get_image_params then the file works. If I leave any of the three calls in place, it fails. | 16:02.24 |
| Jogux | robin_watts : possibly rebase might do that. | 16:02.48 |
| chrisl | kens: Yes, the psdf_get_image_params() builds loads of parameter lists that it seems are then merged in some fashion which bemuses me. | 16:03.34 |
| kens | THis does look like its the problem area, I admit I don't understand htis code.... | 16:03.54 |
| Jogux | robin_watts : hm, but I guess rebase wouldn't fix up the tags; they'd still point at the broken (missing resources) commits. | 16:06.35 |
| Robin_Watts | I wonder if there is a switch we can use on the original cvs import. | 16:07.20 |
| Jogux | robin_watts : gtafts might fix this. | 16:07.43 |
| Robin_Watts | <scrabble>That's not a real word</scrabble> | 16:08.16 |
| Jogux | grafts :) | 16:08.57 |
| | " | 16:09.23 |
| | Grafts enables two otherwise different lines of development to be joined together by recording fake ancestry information for commits. This way you can make git pretend the set of parents a commit has is different from what was recorded when the commit was created. Configured via the .git/info/grafts file. | 16:09.23 |
| | it sounds fun. I'm not sure if it's actually a solution, or just a way to make the problem more confusing. | 16:09.42 |
| Robin_Watts | hmm. | 16:09.44 |
| paulgardiner | Jogux: robin_watts if we decide we are reasonably sure it is correct to extend the most recent version of resources along the branch, I'm pretty certain filter-branch can fix it | 16:09.55 |
| Robin_Watts | So smartoffice-2-1-21-release~818 is where the resources disappear. | 16:10.02 |
| Jogux | robin_watts : Is it? I reckoned it was smart-products-v2_1-release-baseline | 16:11.10 |
| Robin_Watts | The two may be equivalent/ | 16:11.29 |
| Jogux | is the ~818 a typo? | 16:12.00 |
| Robin_Watts | but I binary searched backwards along the branch to find where git blame resources stopped saying "WTF is resources?" | 16:12.06 |
| | no, that's deliberate. | 16:12.12 |
| Jogux | ohh, you're talking to me in git syntax. | 16:12.26 |
| Robin_Watts | X~818 means "818 commits before X" | 16:12.27 |
| | Yes. | 16:12.39 |
| Jogux | Yes. We're talking about the same commit. | 16:13.03 |
| | that commit has two parents. removed 8d9e925 will 'fix' it I think. | 16:13.28 |
| | removing, even. | 16:13.34 |
| Robin_Watts | I'm not sure what 'removing' a parent will do. | 16:14.33 |
| Jogux | tbh, neither am I. but it is a great theory. | 16:14.57 |
| Robin_Watts | Possibly nothing, except to change the internal history tree. Git is not actually doing any merging itself here. | 16:15.01 |
| | It gets a repo state from the git fast-import blob that cvs2git produces. | 16:15.18 |
| Jogux | robin_watts> hrm. | 16:15.22 |
| Robin_Watts | paulgardiner: How would you tickle git format-branch into doing it ? | 16:15.51 |
| Jogux | I now release that I don't understand how the git filter-branch thing you did preserved the tags. | 16:16.45 |
| | s/release/realise/ | 16:16.52 |
| paulgardiner | I "believe" that the --commit-filter would allow you to spot resources being removed and then not do so. | 16:16.57 |
| | the --comnit-filter option, that is | 16:17.18 |
| Robin_Watts | paulgardiner: Right... but at certain points in history we presumably *have* deleted stuff from resources. | 16:17.26 |
| paulgardiner | I mean spot the entire thing being deleted, or is that not the correct thing to do? | 16:17.59 |
| Robin_Watts | Oh, right, yes. | 16:18.10 |
| | So what would the command be? | 16:18.19 |
| paulgardiner | Ah you confuse me for someone with a fully thought out plan :-) | 16:18.42 |
| | I was about to look into it deeper, but then I thought I'd wait to see whether you might come up with something else simpler | 16:19.45 |
| Robin_Watts | git filter-branch --commit-filter bob -- --all | 16:20.10 |
| paulgardiner | yes | 16:20.18 |
| | but bob is probably not trivial | 16:20.25 |
| Robin_Watts | where bob is a script we need to write | 16:20.36 |
| paulgardiner | it needs to analyse the tree blob it is passed | 16:20.45 |
| Jogux | the analysis should be easy; "if resources doesn't exist". | 16:21.41 |
| paulgardiner | yeah. The bit I'm not certain of is whether there is something in the tree linking you back to the previous state from which to recreate resources | 16:23.51 |
| | You have access to the parents I believe, but I haven't looked into exactly how | 16:24.22 |
| marcosw | tor8: I get an error when trying to build the latest mupdf: | 16:25.28 |
| | GEN generated/gen_js_util.h | 16:25.29 |
| | /bin/sh: build/debug/cquote: not found | 16:25.29 |
| | make: *** [generated/gen_js_util.h] Error 127 | 16:25.29 |
| paulgardiner | --tree-filter would be simpler, but I'm pretty sure you cannot recreate the resources with that method, unless you could copy them from the previous iteration or something (doesn't sounds a great idea) | 16:25.50 |
| marcosw | I tried a clean git clone and get different error: | 16:26.04 |
| | GEN generated/gen_cmap_cns.h | 16:26.04 |
| | /bin/sh: build/debug/cmapdump: not found | 16:26.04 |
| | make: *** [generated/gen_cmap_cns.h] Error 127 | 16:26.04 |
| Robin_Watts | paulgardiner: git checkout HEAD~1 -- resources? :) | 16:27.02 |
| paulgardiner | If the already sot.git still looks to be a good starting point then I need to wait out the current --filter-branch I'm running in any case | 16:27.04 |
| | robin_watts possibly something like that would work | 16:28.04 |
| Robin_Watts | paulgardiner: And as Jogu says, we need to refilter again to remove 'Description:" :) | 16:28.08 |
| paulgardiner | Yeah, sorry I should have predicted that. I found my old filter and it mentiioned Description. | 16:28.56 |
| | I was looking at a different aspect of it and didn't twig | 16:29.23 |
| Robin_Watts | I can't see a way to do this *except* with git filter-branch --commit-filter | 16:31.11 |
| | I'll have a quick play on a test repo to see what form this stuff is in. | 16:31.35 |
| paulgardiner | Yeah, pretty sure that's the case | 16:31.36 |
| Jogux | robin_watts , paulgardiner : will regardless of the fact that it would lose tags, rebase is a failure as you get failures because of the darn $Id$ tag expansions. | 16:44.22 |
| henrys | marcosw: see my message about Guillaume above? | 17:08.20 |
| Robin_Watts | git filter-branch is not behaving as I would expect. | 17:18.41 |
| pedro_ | reboots | 17:23.43 |
| tor8 | marcosw: hm, seems like I must've broken the makefile somehow :( | 17:26.36 |
| Jogux | Robin_Watts : git grafts are sort of doing what I expected. I have a feeling the solution may be what I'm trying to do /and/ what you're trying to do. | 17:27.28 |
| | with a git graft, I now have 910b806426baab69baffd4e7419c491d8c66702f (which is the commit we were talking about) showing as deleting resources & CVSROOT. | 17:27.54 |
| | so I suspect what you're doing can now fix that commit | 17:28.04 |
| | robin_watts: Josephs-MacBook-Pro:sot joseph$ cat .git/info/grafts | 17:28.24 |
| | 910b806426baab69baffd4e7419c491d8c66702f 3ea04bf4e8efc3027789f4715150b078ac781d0c | 17:28.24 |
| | try that perhaps. | 17:28.31 |
| Robin_Watts | I'm confused. | 17:28.46 |
| Jogux | more or less so than before I started talking? | 17:28.58 |
| Robin_Watts | % | 17:29.08 |
| Jogux | I don't know what % means in this context :-) | 17:29.42 |
| Robin_Watts | a) I don't understand what I am seeing here. | 17:29.44 |
| | b) I don't understand what git grafts do for us | 17:29.56 |
| Jogux | nods | 17:30.02 |
| Robin_Watts | c) I'm clearly so confused that I'm typing shell commands into an irc window. | 17:30.12 |
| Jogux | I believe one of the core issues we have is that this release branch is a child of both libpng_1_2_2 /and/ a child of master. | 17:30.35 |
| | git grafts allow us to make git believe it is not a child of libpng_1_2_2. I feel that is a step in the right direction. | 17:30.54 |
| ray_laptop | chrisl: So, should I pull in stuff from your repo and start looking at romfs, or are you going to keep working on it. | 17:31.29 |
| Robin_Watts | While it *may* be nicer to not have that dependency in the history, that dependency has no bearing on the contents of the tree. | 17:31.38 |
| tor8 | marcosw: should be fixed now. sorry. | 17:31.44 |
| Jogux | with that git graft, git believes that the first commit on the branch is deleting resources and CVSROOT. that sounds like something that is emminently fixable filter-branch. | 17:31.45 |
| chrisl | ray_laptop: I think it would be best if you do the romfs work on the master code, and I'll merge it with my stuff when you're ready | 17:32.32 |
| Robin_Watts | But the thing that's really confusing me... | 17:32.57 |
| Jogux | robin_watts : what you say is correct. My current position is that having a commit that deletes resources is an easier thing to fix than having resources vanishing because a commit has two parents. | 17:33.00 |
| Robin_Watts | I disagree with your characterisation. | 17:33.38 |
| | We have a commit that deletes resources. It happens to have 2 parents. | 17:33.58 |
| | To say that it's deleting resources because it has 2 parents is misleading, I believe. | 17:34.27 |
| Jogux | my characterisation may be skewed by how gitk is showing me the commit. | 17:34.39 |
| Robin_Watts | Certainly changing it only have one parent will not stop it deleting the resources. | 17:34.47 |
| Jogux | that I agree with. | 17:34.55 |
| | and that is also what I actually see happening when I make it only have one parent :-) | 17:35.13 |
| Robin_Watts | Therefore I am more interested in fixing the deletion than in removing the second parent. | 17:35.20 |
| | Is there any truth in this being a merge of 2 parents? Or is that completely spurious ? | 17:35.44 |
| Jogux | that is the completely spurious thing. | 17:36.12 |
| Robin_Watts | Ok, so fixing the parents is presumably better done by using a git format-patch --parent-filter ? | 17:36.57 |
| Jogux | if that command you mention fixes it without losing the tags: yes. | 17:37.30 |
| Robin_Watts | git filter-branch --parent-filter, sorry. | 17:37.53 |
| | But I think that's a separable task to the one of not deleting the resources. | 17:38.31 |
| Jogux | that may work, combined with "--tag-name-filter cat" to preserve the tags. if I understand correctly. | 17:38.58 |
| Robin_Watts | so... looking at git filter-branch --commit-filter "protect_resources.pl $@" test-tag | 17:39.12 |
| Jogux | and yes, I agree with seperable task, probably. | 17:39.28 |
| Robin_Watts | Or, more properly: | 17:39.49 |
| | git filter-branch -f --commit-filter '~/PicselCVS/protect_resources.pl "$@"' test-tag | 17:40.01 |
| | My perl script currently outputs all the args to a log file, and does "git commit-tree $arg" where $arg = $ARGV[0] | 17:40.58 |
| Jogux | you sound like you know what you're doing so I'll agree :-) | 17:41.24 |
| Robin_Watts | essentially it shouldn't change anything, and should let me see what args I'm being called with. | 17:41.27 |
| | But here is the thing that's hurting my head... my log file shows that the args I am getting are simply hashes. | 17:41.58 |
| | I was expecting the occasional hash -p hash -p hash for merges. | 17:42.14 |
| | and I never see that. | 17:42.18 |
| Jogux | and you test-tag is on the 2.1 branch? | 17:43.08 |
| Robin_Watts | yes. | 17:44.04 |
| Jogux | then I don't understand that either | 17:44.40 |
| Robin_Watts | This is git 1.7.9.5 | 17:45.18 |
| | ok, documentation for 1.7.6.1 shows the same. so it's not that it's a later added feature. | 17:46.09 |
| Jogux | is it possible you just didn't wait long enough? | 17:49.43 |
| Robin_Watts | Maybe. Trying git filter-branch -f --commit-filter '~/PicselCVS/protect_resources.pl "$@"' test-tag~100..test-tag | 17:50.04 |
| | now. | 17:50.08 |
| | Aha! I have -p | 17:50.29 |
| | Phew. So I have something vaguely sane. | 17:50.43 |
| Jogux | okay, you're doing better than me, I still don't :-) | 17:51.46 |
| ray_laptop | robin_watts: based on the SSE code (which was only 6% faster) I modified the C code slightly and it is now within 3% of the SSE code, so I'm not going to bother with SSE. I'll keep it in a local branch in casse we think it might help later | 17:52.25 |
| Robin_Watts | ray_laptop: Cool. | 17:52.37 |
| Jogux | ah, maybe I have the same 'not waiting long enough' issue. | 17:52.48 |
| Robin_Watts | (It was mvrhel_laptop's SSE code I think :) ) | 17:52.50 |
| Jogux | robin_watts : can you write down what you do in case we later find more similarly broken branches? :-) | 17:53.26 |
| Robin_Watts | Jogux: Now I need to understand how the hashes I'm getting relate to the commit hashes. | 17:53.31 |
| | Jogux: I am trying for a solution to fix ALL branches. | 17:53.45 |
| ray_laptop | robin_watts: The 4-bit SSE was mine, adapted from the 4-bit you did for cust 801 | 17:53.48 |
| Robin_Watts | ray_laptop: My point was that mvrhel did the thresholding SSE code I think. | 17:54.16 |
| ray_laptop | robin_watts: the 1-bit code, yes | 17:54.32 |
| Robin_Watts | I did write some SSE code, but I can't remember what for. | 17:54.51 |
| ray_laptop | did he do the 4-bit code as well ? (I didn't think he was involved in the cust 801 work) | 17:55.10 |
| Robin_Watts | was it this? Gawd, my memory sucks. | 17:55.13 |
| | oh, I did the 4bit C version? Did I? | 17:55.45 |
| ray_laptop | robin_watts: and the 4-bit code you got working was partly based on the cust 801 starting point | 17:55.52 |
| Robin_Watts | right. | 17:55.59 |
| Jogux | robin_watts : ahhhh, awesome. | 17:56.04 |
| Robin_Watts | oh, wait, I vaguely remember. | 17:56.18 |
| ray_laptop | robin_watts: np | 17:56.27 |
| Robin_Watts | ray_laptop: You know those scenes in movies where they have the geriatrics sitting around a room looking vacant and every now and then one of them comes up with a non-sequitor about the war? That's my house. | 17:57.14 |
| ray_laptop | actually, rather than keep the 4-bit SSE code on a private branch, I'll just put it in there #if'ed out | 17:57.22 |
| Robin_Watts | Jogux: The plan is to look for commits where 'resources' disappears. | 17:57.58 |
| Jogux | yes, sounds good. | 17:58.08 |
| Robin_Watts | and to rewrite that commit to not delete resources. | 17:58.11 |
| ray_laptop | robin_watts: hehe. Actually, my bringing up SSE code to you was rather a non-sequitur since you've mostly been wrapped up in SOT stuff | 17:58.23 |
| Jogux | actually I bet I can find a list of the problem commit hashes if that would help; I think they're all off the libpng branch. | 17:58.41 |
| Robin_Watts | Jogux: Actually... why are we using commit-filter? | 18:00.43 |
| | Can't we use tree-filter ? | 18:00.48 |
| Jogux | I don't know. can we? :) | 18:00.58 |
| Robin_Watts | tree-filter seems a better bet. commit-filter is for editing the commit message/parents | 18:01.18 |
| Jogux | nods | 18:01.35 |
| Robin_Watts | so I can simply check if resources exists, and if it doesn't, do git checkout HEAD~1 -- resources | 18:02.16 |
| Jogux | I don't understand what HEAD~1 means in the context of a commit with 2 parents. | 18:02.35 |
| Robin_Watts | HEAD~1 means first parent. | 18:02.49 |
| | ~ always follows the first parent back. | 18:03.10 |
| | There is another one for selecting which parent. ^ maybe? | 18:03.21 |
| Jogux | ah, right | 18:03.27 |
| | first parent appears to be the correct thing in the commits I just checked. | 18:03.38 |
| Robin_Watts | In general if you merge B into A, then A is the first parent. | 18:04.08 |
| Jogux | that makes sense to me. the tree we currently have with the fictituous merge commits doesn't :-) | 18:04.43 |
| Robin_Watts | :) | 18:05.00 |
| kens | goodnight all | 18:09.35 |
| Robin_Watts | I fear tree-filters may be slow :( | 18:10.29 |
| | oh hell, it doesn't update HEAD as it goes :( | 18:14.35 |
| | so I have no way of getting to the previous revision to get resources. | 18:15.00 |
| | aha! | 18:28.55 |
| | Jogux: This filter is slow. | 18:47.55 |
| | If you can figure out the set of branches that need fixing, that would probably reduce it to practical time. | 18:48.18 |
| tor8 | tree filters are ridiculously slow. what are you trying to accomplish? | 18:50.50 |
| | if you just want to prune out directories, I think there is a subdir filter of some flavor that will remove files | 18:51.07 |
| Robin_Watts | tor8: We want to avoid a directory being pruned. | 18:52.19 |
| Jogux | robin_watts : "longer than the weekend" slow? | 18:58.37 |
| Robin_Watts | yes. | 19:02.07 |
| Jogux | d'oh :( | 19:02.13 |
| | hrm, I'm sure I had a way to do it but can't figure it out again. | 19:05.27 |
| | as a random thought: it'll probably not break /this/ way if we make the cvs import ignore the libpng branch. Not sure having it broken in a different way helps ;) | 19:05.59 |
| | I reckon the children of a045cf9398bb3a19710c0f21295fc4249649d7cf may be the first commits on the broken branches. | 19:07.36 |
| Robin_Watts | We may have a better way of doing it. | 19:07.47 |
| Jogux | I also suspect every commit with the message 'This commit was manufactured by cvs2svn to create ...' is broken too. | 19:08.39 |
| | though not necessarily all in this way. | 19:08.45 |
| Robin_Watts | Jogux: It is possible, that by editing the stream produced by the converter that usually gets fed to git fast-import we can do all this in one hit. | 19:12.02 |
| Jogux | that sounds feasible | 19:12.24 |
| Jogux | is sure I saw a mention of a tool for editting the git fast-import input - repo<something> | 19:13.23 |
| mvrhel_laptop | robin_watts thanks for looking over the code | 19:28.08 |
| Robin_Watts | mvrhel_laptop: No probs. | 19:28.34 |
| | Forward 1 day (to 2014/03/22)>>> | |
IRC Logs