| <<<Back 1 day (to 2013/05/27) | 2013/05/28 |
alireza_ | How can i read pdf file as bytes , while using MuPdf Im using my own encryption/decryption and i need in first step decrypt pdf in ram | 04:25.50 |
Robin_Watts | paulgardiner: Morning | 08:33.15 |
| henrys spotted something yesterday in the android app. | 08:33.31 |
paulgardiner | hi. I saw the thing about select text in reflw | 08:33.35 |
| Was there anything else> | 08:33.50 |
| ? | 08:33.52 |
Robin_Watts | no, that was it. | 08:33.58 |
paulgardiner | I'll see I can sort that out. I had problems downloading the ndk last night, but hopefully I can get it now. | 08:34.45 |
| With r8e I can now build but clean fails | 09:01.16 |
Robin_Watts | paulgardiner: ndk-build clean fails? | 09:02.27 |
paulgardiner | yes | 09:02.33 |
Robin_Watts | yeah, they made changes. | 09:02.39 |
| I do ndk-build under a DOS window. Where are you running it ? | 09:02.50 |
paulgardiner | It still seems to cause a complete rebuild | 09:03.00 |
Robin_Watts | odd. It's r8e that I'm using here, and it only rebuilds what it needs to. | 09:04.43 |
| stupid question: have you still got the old one in your path ? | 09:05.02 |
paulgardiner | No sorry. I mean clean still seems to work although it reports failure | 09:05.28 |
| BTW, I found that to build without explicitly mentioning the location of the ndk, I just had to put it on the path. | 09:07.59 |
Robin_Watts | paulgardiner: Right. I have always had the NDK on the path. | 09:08.30 |
paulgardiner | Ah right, so had I, but had forgot that that's what's needed since last time I changed NDK. | 09:09.40 |
Robin_Watts | paulgardiner: Should we disable printing as well from reflow mode ? | 09:21.00 |
| If so, then we should probably disable the whole menu. | 09:21.46 |
paulgardiner | robin_watts: I think it still works, although it wouldn't be the reflowed version that was printed (well - the way cloud pring works, it might be) | 09:22.00 |
Robin_Watts | paulgardiner: ? | 09:22.35 |
paulgardiner | I think print works, but it will still just send the pdf to cloud print. | 09:23.12 |
Robin_Watts | right, so it won't be the reflowed version. | 09:23.23 |
| which makes me think we should disable printing in reflow mode. | 09:23.35 |
paulgardiner | Well it shouldn't be, but my attempts at using cloud print has yielded reflowed text | 09:23.57 |
| robin_watts: yeah ok. Makes sense | 09:24.11 |
Robin_Watts | hmm. I get print failed. | 09:24.48 |
| even without picking a printer. | 09:24.57 |
paulgardiner | Only in reflow mode? | 09:25.23 |
Robin_Watts | in normal mode too. | 09:25.48 |
| Why does the reflow button have a contentDescription of "search_document" ? | 09:27.02 |
paulgardiner | copy-and paste screw up I'd guess | 09:28.36 |
| Print works for me | 09:28.44 |
| ... although I still get a reflowed page - badly at that | 09:29.36 |
Robin_Watts | likewise outline view. | 09:29.43 |
| ok, so maybe it's my printer setup or something. | 09:29.56 |
| You get a reflowed page printed? | 09:30.05 |
paulgardiner | Yes, but not reflowed by the app of course | 09:31.09 |
Robin_Watts | paulgardiner: So do you get the same thing printed whether it's from reflowed mode, or not ? | 09:31.38 |
paulgardiner | Yes | 09:31.47 |
Robin_Watts | ok, so I think we should disable the print button (and the whole menu in fact) when in reflow mode. | 09:32.12 |
| At some point we should possibly consider offering PWG printing as an option. | 09:32.31 |
| rather than saving the PDF page out, we'd print to PWG raster and send that. | 09:33.18 |
paulgardiner | Yes. I'm just making the change now to disable the whole menu | 09:33.19 |
| We don't have "reflow mode" in our translated list of strings. Closest is "Entering reflow mode" | 09:38.11 |
| I guess if I add one, it will come out in english for all languages | 09:39.09 |
| robin_watts: ok. Patch is up | 09:49.44 |
Robin_Watts | Ta. | 09:55.57 |
| Pushed, thanks. | 10:02.44 |
paulgardiner | robin_watts, tor8: I've just pushed another one up to my repo. Changes all access the xref table to be via an interface. Perhaps best not to take on before generating the new android release | 10:12.14 |
Robin_Watts | paulgardiner: In pdf_repair.c ... | 10:19.15 |
| you get the len (1 function call). If it's not long enough, we call resize (another function call). | 10:19.36 |
| Then we call pdf_get_xref_entry (another function call) | 10:19.49 |
| Can we not call pdf_get_xref_entry and have that resize if required ? | 10:20.04 |
| In pdf_repair_obj_streams we call pdf_xref_len every time around the loop. | 10:21.24 |
paulgardiner | I wondered about that. Then I think we have to decide on the strategy used to increase the allocation and have the choice between wasting memory or reallocing every call | 10:21.41 |
Robin_Watts | twice. | 10:21.53 |
| pdf_get_xref_entry_resizing ? | 10:22.08 |
paulgardiner | yes | 10:22.17 |
Robin_Watts | I mean, we could have a separate entry point that got the pdf_xref and resized if required. | 10:22.54 |
paulgardiner | I was looking to make the changes as small and obvious as possible for now. | 10:23.07 |
Robin_Watts | In lots of places we call pdf_get_xref and then assume that it's non-NULL. | 10:23.13 |
| To date this has meant calling pdf_xref_len and checking that i < len | 10:23.50 |
| and that means the get can't fail. | 10:24.16 |
| but if our intention is to keep the xrefs as a sequence of tables going backwards in time, might there not be gaps between the tables? | 10:24.48 |
| i.e. the first xref might define 0-100, and the second might then define 200-300 | 10:25.08 |
| so if someone asks for object 150, 150 < 300, but there is no object to return ? | 10:25.29 |
| Also, I don't think it should be pdf_get_xref_entry - maybe lookup instead? tor dislikes get. | 10:26.23 |
paulgardiner | I have these things in mind, but this patch was just supposed to do most of the changes for the sake of adding the interface, while being minimal. | 10:28.04 |
Robin_Watts | I get that, but adding having an interface for the sake of having an interface doesn't feel like a step forward. | 10:28.48 |
paulgardiner | It was just to get the parts in that will conflict with other peoples changes. | 10:29.17 |
| To make a step forward, I will need the interface in place. | 10:30.19 |
| I was just trying to avoid building up the whole chunk of work while frequently rebasing changes all over the source. | 10:31.05 |
Robin_Watts | ok. | 10:31.44 |
paulgardiner | I could address some of the efficiency concerns if I can still keep the changes pretty minimal and obvious. | 10:32.52 |
Robin_Watts | I worry that to fix the interface you're going to need to change every line that you've just changed here. | 10:33.13 |
paulgardiner | I had thought before about making get_entry perform allocation, but then there's the worry of doing multiple reallocs, or using len doubling and wasting space. | 10:33.45 |
Robin_Watts | Well, if we use a separate entry point (lookup_with_resize), then it can perform exactly as it currently does, no? | 10:34.28 |
paulgardiner | In any case, I think I will eventually need two forms of get_entry, one for access the existing entry and one for creating a new one | 10:34.30 |
Robin_Watts | pdf_set_xref_trailer... why not have that take the reference it's passed ? | 10:37.16 |
tor8 | paulgardiner: I can't help but think that pdf_resize_xref should be private, and automatic | 10:37.24 |
paulgardiner | Oh! Right! In pdf_repair_obj_stm: I'd remembered that as doing the len check before the loop, but it does it repeatedly | 10:37.25 |
tor8 | pdf_*_xref_entry will allocate a slot in the xref table and return a pointer to it | 10:37.57 |
Robin_Watts | In lots of places you get the length in the for loop. | 10:38.13 |
paulgardiner | Oh ok. Somehow I hadn't picked that up. | 10:38.31 |
tor8 | however, maybe we don't need to expose the pdf_xref_entry struct at all if we make a call like pdf_update_xref_slot(xref, index, type, ofs, gen, stm_ofs) ? | 10:38.51 |
paulgardiner | I though I saw several places where we were checking before the loop. | 10:38.52 |
Robin_Watts | pdf_set_xref_trailer: Everywhere it's called you pass in a reference, the routine takes the reference, then the caller drops the reference. | 10:38.57 |
| tor8: I think that would be clumsy. | 10:39.59 |
| paulgardiner: Having just dumped all over it, I actually think it's a reasonable step on the way to where we want to be. | 10:40.19 |
paulgardiner | pdf_set_xref_trailer: I thought that was sort of the paradigm we use in mupdf. Lots of functions do that. | 10:40.21 |
tor8 | Robin_Watts: probably, yes. just a thought. | 10:40.27 |
Robin_Watts | and I must go pack, so bbiab, sorry. | 10:40.37 |
paulgardiner | Making the allocation private and automatic was originally my intention. Looking now, I'm not sure why I changed my mind. | 10:42.27 |
tor8 | paulgardiner: robin probably needs to pipe in, but pdf_xref_obj looks out of place to me (we already have pdf_load_object which does the same, and a little bit more) | 10:44.55 |
| maybe it's not safe to load the object when renumbering the xref | 10:45.11 |
paulgardiner | Line 406 of pdf_repair.c allocates to max before a loop... but I could use the "trick" of calling get_entry before the loop for the max entry. Is that nasty? | 10:45.15 |
| tor8: Good point. Yes: see what Robin says. | 10:45.51 |
tor8 | paulgardiner: how about pdf_new_xref(maxnum + 1) ? | 10:46.13 |
| I'm thinking we want a whole new api for just creating xrefs in two different ways | 10:46.36 |
| either by repairing, or by reading xref sections | 10:46.46 |
| so maybe separate the initial xref setup from the xref table creation a bit more | 10:46.58 |
| this looks like a good step towards that, but I think we can go further | 10:47.10 |
paulgardiner | Three: when making document changes. | 10:47.14 |
tor8 | and three: modifying an existing xref | 10:47.31 |
paulgardiner | yes | 10:47.41 |
tor8 | and four: cleaning it up before saving it | 10:47.44 |
| and five: saving it | 10:47.46 |
| and five b: saving an incremental update to it | 10:48.04 |
| and of course the implicit, "get stuff out of it" :) | 10:48.26 |
paulgardiner | Yes, but - as I said before - I'm looking now just to make some initial changes to avoid rebasing while I do the main body of work | 10:48.32 |
tor8 | yeah, I'm not averse to incremental updates getting there and breaking stuff along the way | 10:48.55 |
paulgardiner | So the interface doesn't need to be exactly correct, just reasonably close. | 10:49.15 |
tor8 | we have functions to: create (allocate), delete and update objects in the xref | 10:51.46 |
| now we need functions to do the same on slots in the xref, at a lower level | 10:52.04 |
| something to match pdf_create_object, pdf_delete_object, pdf_update_object but per slot, right? | 10:52.29 |
paulgardiner | I'm not sure. | 10:52.51 |
tor8 | pdf_(set_)xref_trailer is good to go, including the reference counting robin griped about anyway | 10:52.55 |
paulgardiner | Most of the time we will need exactly what we already have. | 10:53.13 |
tor8 | not too happy about pdf_xref_len, _obj, and pdf_replace_xref | 10:53.17 |
| maybe we should disconnect the xref table from the pdf_document struct? | 10:54.01 |
| so we can manipulate multiple xrefs without having it tied to an actual document | 10:54.20 |
| that would help renumberobjs and the repair code I think | 10:54.44 |
sebras | always gets nervous when renumberobjs is mentioned. | 10:55.09 |
paulgardiner | Yeah, in the plan, but I just wanted to get an interface in place for now. | 10:55.17 |
tor8 | pdf_get_xref_entry serves dual purpose here, both to read and to get a pointer to update | 10:55.21 |
Robin_Watts | tor8: That might be nice, if it means we can clone an xref, then do the pdf clean stuff on that cloned xref, and save it out without being destructive to the original xref. | 10:55.29 |
tor8 | paulgardiner: certainly. one step at a time. | 10:55.37 |
paulgardiner | I did have a separate opaque pdf_xref type, but I took it out because I am so far not changing behaviour (hopefully) | 10:55.59 |
tor8 | but I think separating the pdf_xref_table from the pdf_document would make xref table manipulations and iterations clearer in the interface. | 10:56.01 |
paulgardiner | robin_watts: yes I wondered about that. | 10:56.20 |
tor8 | and having that one be a linked list internally for the sections would fall out naturally | 10:56.28 |
| and as robin said, we can edit a cloned xref table without clobbering the original | 10:57.09 |
paulgardiner | tor8: yes, next step would be to change the existing table to something that can hold the sections | 10:57.13 |
| tor8: yes, we need to be able to add a new section for the sake of edits in any case | 10:57.55 |
tor8 | paulgardiner: can you split the xref_trailer changes out into a separate patch then we can commit that at once | 10:57.57 |
| or maybe we should rename it pdf_document_trailer if we're splitting the pdf_xref_table out? | 10:58.30 |
paulgardiner | I could, but I'd rather get the rest to a state everyone is happy with. | 10:58.40 |
tor8 | to separate the pdf_xref_* functions from pdf_document_* based on their arguments | 10:58.42 |
paulgardiner | We need separate trailers per xref section | 10:59.32 |
tor8 | paulgardiner: I'm always in favor of splitting commits into small independent parts (sebras always blames me for not doing it enough) | 10:59.34 |
paulgardiner | Okaty. I'll do that and be back. | 10:59.53 |
tor8 | paulgardiner: yeah, but most users of a trailer only needs the latest one, so that should be accessed through pdf_document | 10:59.57 |
| so s/pdf_xref_trailer/pdf_document_trailer/ and make that one commit without the other stuff | 11:00.18 |
paulgardiner | I can also do the privatisation of xref-resize if that helps | 11:00.19 |
tor8 | you know how to use git gui to stage individual hunks right? | 11:00.27 |
paulgardiner | tor8: yeah sure | 11:00.43 |
tor8 | just the trailer in one commit, please | 11:00.55 |
| (since it touches so many other files) | 11:01.19 |
paulgardiner | Is it really pdf_document_trailer? I imagine I am later going to need a version that asks for the trailer of a specific level of xref section. | 11:01.41 |
tor8 | paulgardiner: we'll need that later too in some places | 11:01.56 |
| but things like the pdf_outline and nametree only need the final trailer | 11:02.18 |
| question: pdf_trailer or pdf_document_trailer? | 11:02.44 |
paulgardiner | What if we want to provide the feature of displaying a previous version of the doucment | 11:02.57 |
tor8 | then you'd rewind the xref on the document, I would imagine | 11:03.14 |
| and all pdf_document_trailer and pdf_load_object etc would fall out naturally from that | 11:03.26 |
paulgardiner | Ah good. I was thinking, we might want to have a sort of set level method | 11:03.36 |
tor8 | yeah! | 11:03.44 |
| I think pdf_trailer is probably a better name | 11:04.14 |
sebras | sounds like there will be a sequence of patches that I want to review when I get home tonight. | 11:04.19 |
tor8 | the pdf_document_ stuff is more for the "page" level fz_document abstraction | 11:04.33 |
paulgardiner | I was thinking of introducing a pdf_xref type, but I don't think we'll ever want to deal with the xref separately from the document. | 11:04.35 |
tor8 | level | 11:04.35 |
| paulgardiner: I think we will, for the repair and saving and renumbering/garbage collection/linearisation code | 11:05.00 |
| and the xref loading code, obviously :) | 11:05.09 |
| enough places where I think it would help the abstraction to keep the xref table separate from the document | 11:05.25 |
paulgardiner | That would be pdf_xref_section, I think, not pdf_xref | 11:05.41 |
tor8 | paulgardiner: I was thinking they could be the same :) | 11:06.03 |
| pdf_xref has a trailer and a linked list "next" pointer | 11:06.12 |
| and a table | 11:06.17 |
| and pdf_document would point at the last such pdf_xref (_section) for its xref | 11:06.43 |
| or were you thinking a pdf_xref_section as such, and a separate wrapper pdf_xref that held those with some extra data and as a high level interface for allocating new sections and setting the level? | 11:07.16 |
paulgardiner | In a previous version of this, I introduced pdf_xref, and made all the methods take an object of that type, but then I found I almost always needed the doc also (if for nothing else the fz_context) and it was always the docs xref I was passing | 11:08.12 |
| So in this version I changed to just passing the doc, thinking we could also pass a level number if we wanted to distinguish sections. | 11:08.53 |
tor8 | paulgardiner: then you missed the gyrations we go through because the xref table being tied to the document in the object renumbering code :) | 11:13.18 |
paulgardiner | yeah, but that could probably be done by access two levels at once, or perhaps even having a renumbering method | 11:14.21 |
| spawn a new level; repeatedly { grab from one level down; put in top level} | 11:15.23 |
tor8 | sounds hacky to me. I prefer less coupling of data structures. | 11:15.55 |
paulgardiner | I was thinking it wasn't unlike the process of updating a document, where many object are still obtained from lower sections, but new additions go in the top section | 11:17.08 |
| ... but maybe that isn't a good way to think about it. I have to admit I don't completely understand the renumbering code. | 11:17.35 |
tor8 | I think when accessing objects at a high level, you're dealing with a document and the current way of having an implicit one-and-only xref makes sense | 11:18.49 |
| but doing these table manipulations, when loading and repairing, and in the garbage collection and renumbering and when saving out the document, having explicit xref table structs would help | 11:19.45 |
| an xref is still tied to a given document, but you can have multiple independent xrefs (that still point to the same file and offsets of objects in the file) in different states of completion | 11:20.41 |
| maybe it's a bad idea, I don't know, but I don't think it can hurt | 11:20.58 |
| the renumbering still needs to take care to renumber all the indirect references | 11:21.24 |
| pdf_load_object would probably be refactored to call pdf_xref_load_object on the "current" xref, if we're thinking of the final plan | 11:22.12 |
| (can't help but think face-to-face time with pen and paper would help in these sorts of discussions) | 11:22.35 |
paulgardiner | Hmm, so the xref type sort of encapsulates the document plus the level of xref section you are dealing with. | 11:22.39 |
Robin_Watts | http://ghostscript.com/~robin/MuPDF-40.apk | 11:22.50 |
paulgardiner | Even across IRC it's been very useful. | 11:23.30 |
| robin_watts: looks good to me | 11:29.48 |
tor8 | paulgardiner: yeah, I think we might need these structs: pdf_document, pdf_xref, pdf_xref_section and pdf_xref_entry | 11:30.00 |
| where the pdf_xref is the "head" of the linked list and has the xref access functions and does the resizing and allocating new sections etc | 11:30.27 |
paulgardiner | I did introduce pdf_xref, but decided to take it out again for now while I was just adding part of the interface. | 11:31.13 |
tor8 | paulgardiner: yeah. it's not crucial until you start adding separate sections | 11:31.42 |
| or rather, add the api now without the explicit pdf_xref argument, and then add that in a second commit once we're happy with the names? | 11:32.23 |
paulgardiner | Oh yes: pdf_replace_xref was only ever intended to be a temporary addition - juist so that I'd covered all direct access to the table by interfaces. | 11:33.54 |
tor8 | paulgardiner: http://pastebin.com/hS2qjV8z something like that's what I mean (just to make sure we're on the same page on what I'm actually saying) | 11:34.00 |
paulgardiner | Yes, pretty much what I was thinking, except I was avoiding for now making pdf_xref explicit in the interface until the stage of adding code that needs it. | 11:41.49 |
tor8 | right. | 11:42.50 |
paulgardiner | And presumably there would be a call to get an entry without specifying a specific xref, like there is for the trailer | 11:42.58 |
tor8 | paulgardiner: possibly, if we need it | 11:43.13 |
Robin_Watts | Guys, can you test MuPDF-40 and 41.apk please? | 11:43.25 |
tor8 | I think a function like that would sit at an uncomfortable level between pdf_*_object and the xref load/edit/save levels | 11:43.38 |
paulgardiner | But isn't that the one we mostly need. All the existing code wants the latest trailer | 11:43.41 |
| I mean entry not trailer | 11:44.27 |
tor8 | all the pdf_*_object functions want the latest entry, yes | 11:44.49 |
paulgardiner | Oh. Hang on. Maybe most of the code that needs explicit access to the entry is code that is saving or renumbering | 11:45.11 |
Robin_Watts | I've not read back through the logs yet, but... | 11:45.13 |
tor8 | so it may make sense to just make those functions that take the latest entry, and then inject the explicit level functions later | 11:45.15 |
| the editing and loading would be better off using an explicit one | 11:45.36 |
Robin_Watts | it seems to me that in order to seamlessly step backwards and forwards through time, we'll want the ability to set the 'current' level on the document. | 11:45.57 |
paulgardiner | robin_watts: yes | 11:46.12 |
Robin_Watts | and that current level will affect what the pdf_*_object returns. | 11:46.18 |
paulgardiner | Although, we also need to be able to ask what has changed between levels | 11:46.30 |
Robin_Watts | so they aren't strictly the "latest" at all times. | 11:46.31 |
tor8 | Robin_Watts: yeah, we talked about that. having the xref be an explicit struct and the document have a 'current' xref section would do that. | 11:46.52 |
Robin_Watts | I'm uploading the apks to google play now. I won't publish them until this evening, but I could really do with sanity checks asap please. | 11:47.55 |
tor8 | difference between -40 and -41? | 11:48.15 |
| Robin_Watts: the apks feel sluggish. debug build or release? | 11:49.05 |
paulgardiner | We could handle incremental update without mentioning explicit xrefs. By having a get_entry_for_the_purpose_of_update method. That would spawn a new top section if that hadn't been done since loading the doc or last save, and return an entry from that top section | 11:49.10 |
Robin_Watts | tor8: -40 is armeabi | 11:49.25 |
| -41 is armeabi-v7a | 11:49.32 |
| tor8: All release builds. | 11:49.39 |
tor8 | and the difference between those is? (I vaguely recall one of them using fp emulation) | 11:50.13 |
Robin_Watts | tor8: v7a has faster fp | 11:50.32 |
| 40 is softfp, I believe | 11:50.45 |
| really, you think they are slower than previous releases? | 11:51.32 |
tor8 | it takes a good second to open up pdfref13.pdf ... which I think used to be almost instantaneous | 11:51.52 |
Robin_Watts | tor8: which one? | 11:52.15 |
tor8 | could be my phone being slow | 11:52.18 |
| both -40 and -41 are equally sluggish | 11:52.25 |
| lemme reboot it see if that helps | 11:52.33 |
| ouch. 4 second startup time of pdfref13 from a cold boot :( | 11:54.45 |
| 2 seconds for pdfref17 while warm | 11:55.07 |
| do you have an ancient apk sitting somewhere? one from before javascript | 11:55.30 |
Robin_Watts | tor8: There are various in the same place you got -40 and -41 from. | 11:56.35 |
| At the moment it's version 16/17 on google play. | 11:57.43 |
| 17 being armeabi-v7a | 11:57.55 |
| That takes just as long. | 11:59.21 |
| version 12 is just as long. | 12:02.05 |
tor8 | right. so probably just my phone being slow then, not a recent regression | 12:03.08 |
Robin_Watts | version 9 seems faster :( | 12:04.07 |
sebras | Robin_Watts: hello bisect. | 12:05.13 |
Robin_Watts | sebras: bugger that. | 12:05.24 |
| D:/android-ndk-r8e/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/local/armeabi-v7a/objs/mupdfcore/__/__/pdf/pdf_unicode.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -I | 12:06.56 |
| ../thirdparty/jbig2dec -I../thirdparty/openjpeg/src/lib/openjp2 -I../thirdparty/jpeg -I../thirdparty/zlib -I../thirdparty/freetype/include -I../draw -I../fitz -I../pdf -I../xps -I../cbz -I../image -I../ucdn -I../scripts -I.. -I../thirdparty/v8-3.9/include -ID:/android-ndk-r8e/sources/cxx-stl/stlport/stlport -ID:/android-ndk-r8e/sources/cxx-stl//gabi++/include -Ijni -DANDROID -DARCH_ARM... | 12:06.58 |
| ...-DARCH_THUMB -DARCH_ARM_CAN_LOAD_UNALIGNED -DAA_BITS=8 -Wa,--noexecstack -ID:/android-ndk-r8e/platforms/android-8/arch-arm/usr/include -c jni/../../pdf/pdf_unicode.c -o | 12:06.59 |
| ./obj/local/armeabi-v7a/objs/mupdfcore/__/__/pdf/pdf_unicode.o | 12:07.01 |
| So it builds with -Os and -DNDEBUG, but it does have -g. | 12:07.32 |
| but -g shouldn't affect the speed, right? | 12:07.53 |
sebras | Robin_Watts: AFAIK, no. | 12:08.47 |
chrisl | Robin_Watts: -g will make the code bigger, that could affect cacheing, etc | 12:16.01 |
Robin_Watts | chrisl: Will it? | 12:16.13 |
| I thought -g meant "keep debug symbols" | 12:16.27 |
chrisl | The are stored in the exe | 12:16.43 |
Robin_Watts | so the .o files are bigger, but the linked final version (assuming -g isn't there on the link) should be the same size. | 12:16.47 |
| I will check to see if -g is on the link. | 12:16.57 |
sebras | Robin_Watts: even if the linked final version is bigger then I think it is only debug-symbols in the file. | 12:18.08 |
| Robin_Watts: and I believe these are in segments that are not loaded by the program loader so I doubt that they will affect performance at all. | 12:18.47 |
Robin_Watts | sebras: Right, but there is still time taken to load that larger lump off the storage. | 12:18.48 |
| sebras: maybe. | 12:18.56 |
| you have more faith than I do :) | 12:19.08 |
| I had the debuggable flag set in the apk. | 12:21.42 |
| unsetting that saves 100K from the apk. | 12:22.03 |
sebras | Robin_Watts: yes, probably. ;) | 12:23.41 |
Robin_Watts | New MuPDF-41.apk online. Still doesn't seem any faster :( | 12:26.54 |
| tor8: Still, it's speed when it's up and running seems OK ? | 12:30.08 |
tor8 | it's = it is. ;) | 12:31.28 |
| it's no 60fps updates, but seems okay | 12:31.46 |
Robin_Watts | tor8: yeah, sorry. | 12:32.19 |
| shame, I was hoping the redraw tweaks would have got us to 60fps | 12:32.33 |
tor8 | trying on my nexus 10 now | 12:32.49 |
| (that was my htc desire x) | 12:32.55 |
Robin_Watts | hmm. I know the desire and the desire hd. don't know the x. | 12:33.21 |
tor8 | it's a lower powered, slimmer, better battery life one :) | 12:33.53 |
| and significantly cheaper! | 12:34.02 |
Robin_Watts | tor8: feels smoother on the transformer, but maybe that's just me. | 12:34.40 |
tor8 | hmm. the nexus 10 has some odd epilepsy inducing flicker when paging! | 12:35.29 |
| I think that's a show stopper we need to fix. | 12:35.39 |
henrys | Robin_Watts: by all means skip the meeting today if you need time to get ready. | 12:35.43 |
tor8 | oops. | 12:35.59 |
| it killed my tablet :( | 12:36.05 |
| I held the finger down, massive 60fps flickering, then power down. | 12:36.16 |
Robin_Watts | tor8: how do you get the flicker when paging? | 12:36.18 |
tor8 | maybe because I'm low battery | 12:36.38 |
| but it's plugged in and charging... | 12:36.43 |
Robin_Watts | henrys: thanks, I think I should be Ok. | 12:37.53 |
paulgardiner | robin_watts, tor8: New version pushed. Two parts split up. Renamed pdf_xref_trailer to pdf_trailer. Made pdf_resize_xref private and automatic | 12:50.34 |
paulgardiner | hides under desk | 12:50.52 |
tor8 | trailer commit LGTM | 13:03.08 |
| robin needs to weigh in about pdf_xref_obj vs pdf_load_object in the second commit | 13:03.45 |
| I'm starting to dislike our naming convention :( | 13:04.23 |
| pdf_xref_entry as a type collides with pdf_xref_entry as my preferred name for "pdf_get_xref_entry" | 13:04.41 |
sebras | tor8: pdf_xref_entry_f() to signify that it is a function? *being ironic* | 13:05.19 |
tor8 | (PdfXrefEntry camelcase types, or no typedef explicit s"struct pdf_xref_entry" would have been my preference) | 13:05.39 |
sebras | (as usually people solve this by appending _t to identify types...) | 13:05.41 |
tor8 | yeah. ick. | 13:05.53 |
| sebras: pdf_xref_entry__pdf_xref_entry__pdf_document__int() to make sure you know what all the arguments are and the return value! | 13:06.40 |
| or we can just write it all in ObjC ;) | 13:06.58 |
sebras | detects a serious suggestion. | 13:07.42 |
paulgardiner | I quite like the _t thing. Makes the types stand out - but not mixed use within the project obviously. :-) | 13:21.42 |
| Oh - pretty sure I don't like this so I imagine you wont, but: Type pdf_xref_entry, function pdf_xrefs_entry | 13:23.52 |
| I think I would like it if I could put the apostrophy in | 13:24.19 |
tor8 | paulgardiner: genitive s? it's a thought. but not now, we'd have to rename too much to match. | 13:24.56 |
Robin_Watts | tor8: so, with a recharged nexus 10, does it still flicker/crash? | 13:26.52 |
Robin_Watts | was at pains to ask for people to test stuff a few days ago, specifically to avoid this kind of last minute problem :( | 13:27.24 |
tor8 | nope, it must've been low power related | 13:29.02 |
Robin_Watts | tor8: phew! | 13:29.15 |
| Does it look like 60fps or not? | 13:29.25 |
tor8 | gah, I finally figured out why I sometimes "fail" the flipping gesture | 13:29.39 |
| if the finger leaves the screen off the bottom edge, it won't flip | 13:29.53 |
Robin_Watts | I was really hopeful that the overdraw fixes might have got us to 60fps. | 13:29.55 |
tor8 | it looks pretty smooth, but not sure if it hits 60fps consistently | 13:30.33 |
| it may just be dropping a few frames a second, and that'll make it look jerky | 13:30.49 |
Robin_Watts | I think previously we were hitting 30fps all the time. | 13:30.53 |
tor8 | it's noticeably laggy though... the page trails after the finger motion by about 1/3 of a second | 13:31.19 |
| or 1/5 or something like that | 13:31.43 |
paulgardiner | I'm not seeing any sluggishness here, but maybe it just isn't apparent on the S2... or my eyes aren't sensitive to it | 13:31.44 |
tor8 | it has some ease-in-ease-out animatin thing going on that delays the page tracking | 13:32.11 |
| probably makes it look cooler, but probably a main reason why android feels sluggish compared to ios | 13:32.30 |
paulgardiner | page almost feels stuck to my finger on S2 | 13:33.01 |
tor8 | on portrait mode I get these jerky stops when panning between pages | 13:33.12 |
| landscape mode, less noticeable | 13:33.20 |
Robin_Watts | There is a slight lag on the transformer. | 13:33.58 |
tor8 | portrait is not hitting 60fps consistently | 13:34.04 |
| we should all go and buy high speed cameras! | 13:34.25 |
paulgardiner | This has to be in the Java, right? And I'd have though that we haven't changed anything in the java that could affect this other than the overdraw patch, and that has to speed it up if anything | 13:34.40 |
Robin_Watts | The slowdown in document loading is probably in the native code, and the redraw 'lag' is probably down to the java. | 13:35.35 |
tor8 | the stop/jerk might be context switches when queuing up a new page render that causes it to miss a few frames | 13:35.42 |
paulgardiner | S2 has to be hitting 60. Looks very smooth and I think I'd notice dropped frames | 13:35.44 |
tor8 | it happens when the page is exactly halfway across | 13:35.53 |
Robin_Watts | I can't see that we can ever be worse w.r.t. redraw than we were before. | 13:36.18 |
tor8 | Robin_Watts: it's not worse that before! | 13:36.30 |
paulgardiner | This page flipping, not dragging while zoomed in then? | 13:36.35 |
tor8 | but possibly more noticable when frames are dropped :) | 13:36.39 |
| the jerking doesn't happen when zoomed in, only when flipping and only when the page crosses the 50% halfway line | 13:37.21 |
| it's silky smooth, then BAM ten dropped frames, then a big jump to catch up | 13:37.43 |
Robin_Watts | tor8: yeah, I get the slight 'tug' as we drag from page to page. | 13:38.06 |
paulgardiner | I would expect that. | 13:38.12 |
Robin_Watts | but I have to say, I quite like it. | 13:38.12 |
| It means I get to 'feel' when I've moved to the next page. | 13:38.21 |
tor8 | (robin_watts: sorry, didn't get any time to look at the apks over the weekend) | 13:38.23 |
| Robin_Watts: I hate it. | 13:38.32 |
Robin_Watts | gaaah! | 13:38.59 |
tor8 | could we try not queuing until the next page has settled / animation stopped? | 13:38.59 |
Robin_Watts | google play won't accept me uploading these apks. All it says is "Upload failed" | 13:39.26 |
tor8 | I don't want to see it jerk just because of what happens in the background. this is a multi-core device dammit! | 13:39.27 |
Robin_Watts | presumably because it has had apk's 40,41,42,43 before, even though I never published them, and deleted them. | 13:39.55 |
tor8 | I think delaying the queuing up of new jobs until after animations would make the page tab look smoother as well | 13:40.07 |
paulgardiner | robin_watts: could you have run out of space on their server because their not really deleting them? | 13:43.21 |
Robin_Watts | I cannot delete any apk that has ever been published. | 13:45.18 |
| therefore I don't believe space can be an issue. | 13:45.25 |
paulgardiner | Yeah. Space can't really be an issue with Google | 13:46.02 |
Robin_Watts | tor8: I don't think that's something we can try to do today. paulgardiner ? | 13:46.13 |
paulgardiner | robin_watts: I think it would be risky to attempt something like that just before a release | 13:47.43 |
tor8 | paulgardiner: I want to try it on a branch, and hold off on merging it until after this release. | 13:48.39 |
paulgardiner | Yeah sure. No problem with that | 13:49.06 |
tor8 | and preferably before we forget :) | 13:49.15 |
| in fact, doing it myself gives me more excuses to procrastinate from the loom specter of Gtk+ | 13:49.53 |
Robin_Watts | tor8: but you're still going to have that done by the next meeting, right? :) | 13:50.49 |
tor8 | breaks down and cries. | 13:51.03 |
paulgardiner | I probably can't help much. I'm pretty sure my S2 isn't showing the problem | 13:51.09 |
sebras | tor8: there, there. I'm showing up at your place tonight. we can be baffled over GTK+ together! :) | 13:55.27 |
tor8 | or tear our few remaining hairs out over android multithreading issues... | 13:56.17 |
sebras | tor8: if we have to resort to hair tearing I'm not sure I want to show up. | 13:58.17 |
monxalo | Hello there, paulgardiner can you help me with some insights/directions to implement a 2 page view on the android app? :) | 14:08.59 |
| Btw, thanks for the documentation on the classes, that helps a lot. :) | 14:09.36 |
paulgardiner | monxalo: Yes, sure. I think I my have a patch on an old branch. I'll just take a look | 14:10.51 |
vtorri | tor8, you'll use gtk+ for the GUI ? | 14:12.05 |
monxalo | Great, so the approach i'm going, is using a container (ViewGroup) to wrap around 2 MuPDFPageView, in an effort to reuse most of the classes. | 14:12.35 |
paulgardiner | monxalo: There's this patch: http://git.ghostscript.com/?p=user/paulg/mupdf.git;a=commitdiff;h=3eb2c1685f1e9774eb8c2c8f953bae8190c14618 | 14:12.44 |
monxalo | Thanks, that looks a lot simpler. :) | 14:13.34 |
paulgardiner | monxalo: I was imagining that you'd want to alter ReaderView. It's that class that controls the motion of the pages and could bring two onto the screen rather than one | 14:14.13 |
| Ah yes. This patch seems to change only ReaderView | 14:15.04 |
Robin_Watts | I'd imagine that if you were going to start having 2 pages on the screen at once, you'd want to make it so that we cache 4 pages rather than 3. | 14:15.22 |
paulgardiner | You may want also to add more cached pages in mupdf.c | 14:15.22 |
| Ha. I beat you to it that time... uusual. | 14:16.08 |
| unusual | 14:16.16 |
monxalo | Humm, and this also handles zooming, updateHq on each respective page region? | 14:16.40 |
paulgardiner | monxalo: I'd have thought so. It's been a while. At least it would be a good starting point. | 14:17.34 |
monxalo | Eheh, 1 year patch :). | 14:18.29 |
Robin_Watts | gah. I still can't upload to google :( | 14:19.40 |
| The nook is an android 2.1 platform. | 14:39.16 |
| I think we require 2.2 :( | 14:39.20 |
| We are all published on google play, btw. | 14:40.10 |
| 2.2 introduces ScaleGestureDetector | 14:41.34 |
| i.e. multitouch. | 14:41.44 |
| paulgardiner, tor8: The first part of paul's review has "pdf_set_xref_trailer" | 14:49.25 |
| I guess we have sets, not gets ? | 14:49.52 |
tor8 | Robin_Watts: yeah. and pdf_set_xref_trailer should eventually take an explicit pdf_xref | 14:50.18 |
| we have sets, but not gets, in general | 14:50.29 |
Robin_Watts | I still think pdf_set_xref_trailer should take on the reference that it is passed. | 14:50.42 |
| i.e. ownership of the reference passes in. | 14:50.59 |
mvrhel_laptop | henrys: are you here? | 14:51.04 |
Robin_Watts | It makes both the implementation and the use of the function easier. | 14:51.12 |
tor8 | Robin_Watts: no. it's irregular. when using ref counting everything HAS to be regular or you'll mess up two years later. | 14:51.14 |
| or even two months :) | 14:51.18 |
mvrhel_laptop | Robin_Watts: when do you head to Boston? | 14:51.29 |
henrys | mvrhel_laptop: yes I thought you were out today? | 14:51.35 |
Robin_Watts | mvrhel_laptop: Tomorrow morning. | 14:51.36 |
tor8 | if you create it with a function that has "new" or "load" in the name, you must "drop" it | 14:51.41 |
mvrhel_laptop | henrys: getting ready to leave now | 14:51.44 |
Robin_Watts | tor8: We already have functions that take on ownership. | 14:51.55 |
mvrhel_laptop | just wanted to make sure you knew I am going to miss the meetings | 14:51.56 |
tor8 | Robin_Watts: which ones? I know we added some, but I think we gave them a suffix to indicate said behaviour | 14:52.27 |
Robin_Watts | We document them as taking ownership. | 14:52.32 |
mvrhel_laptop | I am hoping to wrap up the windows viewer this week. just need to add keyboard zooming and save the state when it goes into suspension | 14:52.54 |
paulgardiner | We have _drop versions of some | 14:52.55 |
tor8 | ah, yes. that's the suffix. | 14:53.08 |
| so we'd need pdf_set_xref_trailer_and_drop | 14:53.20 |
paulgardiner | And they drop even in error cases. | 14:53.21 |
henrys | mvrhel_laptop: great good luck at your talk. | 14:53.35 |
Robin_Watts | fz_new_list_device takes ownership of the list device. | 14:53.36 |
| mvrhel_laptop: Yeah, have fun. | 14:53.43 |
tor8 | Robin_Watts: does not parse | 14:54.08 |
mvrhel_laptop | ttyl | 14:54.40 |
| thanks | 14:54.52 |
Robin_Watts | tor8: ahem of the list. | 14:54.54 |
paulgardiner | The nice thing about the convention of not taking ownership is that you can call functions that either inspect or keep an object without having to work out which it does. | 14:55.04 |
Robin_Watts | tor8: OK, so using a suffix (_and_drop) would work too. | 14:55.25 |
| but my point is it simplifies the calling code. | 14:55.41 |
| - xref->trailer = pdf_new_dict(ctx, 5); | 14:55.54 |
| + obj = pdf_new_dict(ctx, 5); | 14:55.56 |
| + pdf_set_xref_trailer(xref, obj); | 14:55.58 |
| + pdf_drop_obj(obj); | 14:56.00 |
| + obj = NULL; | 14:56.02 |
paulgardiner | lessening the number of lines but possibly making it require more thought. | 14:56.13 |
tor8 | Robin_Watts: or just (_drop) but I think we should be a bit restrictive with that behaviour and reserve it for the instances where there are dozens of calls to the function | 14:56.15 |
Robin_Watts | pdf_xref_set_trailer_drop(xref, pdf_new_dict(ctx, 5)); | 14:56.46 |
tor8 | pdf_array_push_drop and friends are common enough that I felt it was okay to make an exception for it | 14:57.30 |
| to go out on a wild tangent here... since we have a context everywhere, autodrop in the objc style would be possible to implement, and could simplify a lot of reference counting. | 14:58.54 |
Robin_Watts | tor8: Would it? | 14:59.12 |
paulgardiner | I guess having both flavours is never a bad thing. Typically the _drop version has to use exception handling so you can end up with two levels of try catch - but not in this case I think | 14:59.14 |
tor8 | Robin_Watts: I think I've discussed it before, but concluded that it probably wasn't worth it. | 14:59.28 |
henrys | 1 minute of the meeting | 14:59.57 |
Robin_Watts | tor8: wouldn't we need to have markers to say when we enter and leave a function? | 15:00.11 |
tor8 | Robin_Watts: that one example is contrived, the other instances actually use the new dict a bit before sticking it in the trailer. | 15:00.20 |
| Robin_Watts: an autodrop pool would need to have "clear the pool" at safe intervals (like in an event loop) | 15:01.09 |
Robin_Watts | tor8: sounds filthy. | 15:01.19 |
tor8 | and that would need nesting scope | 15:01.24 |
henrys | did we ever get the third party app list for MuPDF up - chrisl I think took that one? | 15:01.32 |
tor8 | Robin_Watts: it is. but it takes away 90% of reference counting headaches in ObjC | 15:01.37 |
paulgardiner | robin_watts: my thoughts exactly until I programmed under iOS and found that is a very powerful idea | 15:01.48 |
chrisl | henrys: no, I didn't do it yet - I figured it wasn't required until the next release? | 15:02.00 |
Robin_Watts | tor8: Until something stops working. | 15:02.01 |
tor8 | in the Cocoa frameworks you can create and use temporary objc objects willy nilly without a care about reference counting, and the autorelease pools will clean up after you're done | 15:02.19 |
Robin_Watts | tor8: right, but we can't rely on an event loop happening soon enough. | 15:03.04 |
henrys | chrisl:no miles just requested it - didn't say when. I suppose it could wait until then. | 15:03.07 |
tor8 | the main issue it solves is "alloc" returning a new temporary object but you don't want explicit ownership of it (or you're passing it to some other function, which does take ownership), so it goes in the autorelease pool to be released "sometime after the function has returned" | 15:03.20 |
chrisl | henrys: I'll do it tomorrow - it won't take long | 15:03.23 |
henrys | chrisl: okay | 15:03.33 |
paulgardiner | robin_watts: That said, I did spend several hours the other day staring at some ObjC wondering why something was never getting freed. :-) | 15:03.42 |
tor8 | Robin_Watts: so if you're making lots of garbage, you're supposed to make a new pool and stick it around loops that do the work | 15:03.46 |
Robin_Watts | suppose we start using that on, say, a search operation. | 15:03.52 |
| and then we search a 1300 page file. | 15:04.00 |
tor8 | so in the search case, for each page you'd make and clear the pool | 15:04.10 |
henrys | paulgardiner:there is the issue of strong vs weak pointer in ObjC is that the issue? | 15:04.22 |
Robin_Watts | right, so it still requires work to ensure you clear the pool every now and then. | 15:04.30 |
| and it bloats every object. | 15:04.51 |
paulgardiner | henrys: It could be. Is a weak ref one that doesn't up the count? | 15:04.56 |
tor8 | Robin_Watts: yes. but it spares you the tedium of passing ownership around everywhere. | 15:05.00 |
henrys | paulgardiner: yes | 15:05.17 |
tor8 | in mupdf we made the "exception" that fz_dict_gets etc don't do "keep" on the returned object and that's worked fine so far | 15:05.33 |
henrys | basically it means if nobody else is using this free it I don't care about it. | 15:05.39 |
Robin_Watts | tor8: We'd still have to have the ability to be rigorous, so ownership passing would still have to be defined at the API level. | 15:06.26 |
| but it would free people to make slip ups without it mattering so much. | 15:06.37 |
tor8 | Robin_Watts: the idiom in objc for constructors is to "return [self autorelease];" | 15:06.48 |
paulgardiner | henrys: right. It was a CCNode heirarchy in cocos2d. I wondered if the parent pointers were holding on to it, but those are weak refs, so I guess not. | 15:06.50 |
tkamppeter | New Ghostscript crasher: http://bugs.ghostscript.com/show_bug.cgi?id=694275 | 15:06.51 |
Robin_Watts | I kinda feel that it's giving people more rope to hang themselves with. | 15:07.24 |
tor8 | which for pdf_dict_gets would be: return fz_auto_drop(ctx, value, pdf_drop_obj) | 15:07.26 |
| because any passing of a reference counted object has to have keep/drop | 15:07.49 |
| sorry, make that: return fz_auto_drop(ctx, pdf_keep_obj(value), pdf_drop_obj)) | 15:08.09 |
| which is more annoying than what we have now | 15:08.27 |
henrys | thanks tkamppeter I'll bring it up at the gs meeting in an hour or so. | 15:08.28 |
| paulgardiner:so you are beginning the iOS port? | 15:08.50 |
paulgardiner | henrys: he he. No. This was a weekend activity - a game that I'm very very very slowly writing - maybe, possibly. | 15:09.34 |
henrys | I guess this is mainly for michael but I think any desktop app should support loading files into an already running process - a bug to that effect was raised against the mupdf x11 viewer this week. Is that going to be a big problem? | 15:10.42 |
Robin_Watts | henrys: Any *new* app? | 15:12.10 |
tor8 | henrys: the app is hard wired to do one document | 15:12.19 |
| and it's one process per window | 15:12.25 |
henrys | paulgardiner:ah so your practicing with iOS so you _can_ port the mupdf android changes to ios | 15:12.26 |
| ;-) | 15:12.44 |
paulgardiner | henrys: exactly. Tor's comments have made me realise how much fun it must be. | 15:13.04 |
henrys | Robin_Watts, tor8: I'm talking about the viewer app michael is working on for windows and tor8 on linux | 15:14.37 |
Robin_Watts | right. | 15:15.20 |
| It depends on what model you take for the app. | 15:15.41 |
| Acrobat loads as a 'empty' app into which you can load multiple files. | 15:16.03 |
tor8 | henrys: none of our apps are multiple document interfaces (you know the windows-within-a-window, or tabs) | 15:16.16 |
Robin_Watts | (Multiple files, single window, there is a UI term for it that escapes me). | 15:16.18 |
paulgardiner | Actually, I am tempted to have a look that iOS port. The main thing that scares me off is not comletely understanding the tricks Tor had to come up with to control motion of the page. The equivalent Android mechanism, although complex, is at least straightforward | 15:16.24 |
Robin_Watts | MDI = Multiple Document Interface, that's the one, thanks tor. | 15:16.30 |
| MDI is a STUPID paradigm. | 15:16.46 |
tor8 | so it makes no sense to do multiple documents, single process, except for macosx | 15:16.47 |
| which is a stupid system to begin with | 15:16.56 |
Robin_Watts | sorry, let me correct myself. MDI as implemented on windows is a stupid paradigm. | 15:18.12 |
henrys | I think if you click on a pdf file folks will expect not to start a new process if one is open on either windows or mac os x - certainly not the latter, windows I'm not positive about. | 15:18.55 |
Robin_Watts | henrys: Who cares what goes on behind the surface? It's the UI that matters. | 15:19.22 |
henrys | I think even evince does this now. | 15:19.23 |
Robin_Watts | When you open a new browser window on chrome, that starts a new process. | 15:19.36 |
| s/browser window/browser tab/ | 15:19.50 |
| The question is whether the UI does what you expect. | 15:20.06 |
| With Acrobat in windows, if you click another PDF file, you get another window. | 15:20.23 |
| Who cares whether that's a single process or multiple ones ? | 15:20.43 |
tor8 | I loathe this new paradigm of apps opening documents replacing the old one I was viewing. stupid crap. | 15:20.45 |
| what if you want two open side by side? no can do, sir. now shut up and bow down to almight Jobs. | 15:21.14 |
Robin_Watts | What the user who opened the bug was wanting was some way to start a new PDF file up and have it replace the old one (same window, same position, etc). | 15:21.15 |
| That's *not* what most people want. | 15:21.29 |
| and would be a huge UI fail. | 15:21.38 |
henrys | tor8: you use open -n for a new process. | 15:21.44 |
paulgardiner | Might it be added as a configurable option? | 15:22.23 |
tor8 | henrys: the new lion (or mountain lion) preview has some stupid default that replaces whatever pdf you were viewing with the now one you opened. took me ages to find the setting to turn that shit off. | 15:22.31 |
Robin_Watts | tor8: windows photo and fax viewer used to do the same. drove me mad. | 15:23.02 |
| but they fixed that for windows 7. | 15:23.16 |
paulgardiner | Doesn't the app just have to announce itself as a consumer of certain file types and then react to a certain windows message? | 15:23.45 |
Robin_Watts | paulgardiner: You're thinking of RISC OS :) | 15:24.47 |
henrys | okay I'll let that request rest until the next battle uhum meeting. | 15:25.11 |
paulgardiner | Actually, I may be. I knew I'd had to handle something like that before. | 15:25.18 |
Robin_Watts | having an option is probably not a bad idea, but it should not be default behaviour. | 15:25.39 |
paulgardiner | I'm still not sure what we decided about openssl. | 15:25.45 |
tor8 | me neither. | 15:25.59 |
henrys | that was next in my notes - signatures | 15:26.15 |
tor8 | on tor/openssl is one way to build it using configure && make if openssl is present in thirdparty | 15:26.29 |
| but I want to polish that some more to deal with system installed and complete absence as well | 15:26.47 |
| and figure out a story for win32 | 15:27.04 |
paulgardiner | Still not got my signature checking code commited, or added to android. Parked it until we decide on openssl | 15:27.16 |
tor8 | paulgardiner: is libcrypto available on android? | 15:27.41 |
paulgardiner | I don't think so. Google shows lots of discussion about building it | 15:28.27 |
| We could use the same mechanism as for v8 | 15:28.38 |
| Distribute our own prebuilt libs that is. | 15:28.57 |
tor8 | rats. because their build system uses perl scripts so I bet it's a bitch to cross compile. | 15:29.06 |
paulgardiner | robin_watts: last week you said something about building it fore andrpod - or am I misremembering | 15:29.32 |
tor8 | I have another way to build it using our existing make rules, but that might miss some crucial #define or other that the configure sets up | 15:29.37 |
Robin_Watts | paulgardiner: I've never touched openssl. | 15:30.16 |
paulgardiner | oh okay. Not sure where I picked that up from | 15:30.38 |
henrys | wow I'd think there would be something native on android. | 15:31.38 |
| paulgardiner: I liked the android class documentation - nice. | 15:32.37 |
| Robin_Watts: good luck at AnDev - we'll try to be available for you SMS if you need to. | 15:35.22 |
Robin_Watts | henrys: Thanks. | 15:35.57 |
| I'll be on irc, I imagine. | 15:36.09 |
paulgardiner | henrys: oh good. Can probably add more detail for some of the classes, but I thought I'd wait to see what parts people ask about | 15:36.09 |
henrys | anything else for the meeting - sounds like openssl is still a problem. | 15:36.42 |
| ? | 15:36.44 |
| pulling out parts of a crypto library is like a major sin in the security world up there with rolling your own. | 15:37.28 |
paulgardiner | tor8: the openssl build using perl should be fine if we are going for the same approach as v8 | 15:37.45 |
| I'd somehow gotten the idea that someone else was looking at building openssl for Android. Sounds like I should maybe have a go. | 15:38.38 |
Robin_Watts | android has a customised version of bouncy castle in it. | 15:39.19 |
paulgardiner | Is that java though? | 15:39.34 |
Robin_Watts | yes. | 15:39.53 |
| so it'd require some cunning callbacks from jni -> java to be usable I imagine. | 15:40.13 |
paulgardiner | Yes. Sounds hellish | 15:40.40 |
tor8 | ick. | 15:40.42 |
Robin_Watts | OK, I can get mupdf installed on the nook, by knocking out all the ScaleGestureDetector stuff. | 15:42.09 |
| and building for armeabi. | 15:42.22 |
henrys | http://stackoverflow.com/questions/9697825/how-to-digitally-sign-a-pdf-in-android-device | 15:43.28 |
Robin_Watts | iText is java isn't it ? | 15:44.11 |
henrys | of course itext is java | 15:44.15 |
Robin_Watts | hence it can be using bouncycastle. | 15:44.50 |
| and indeed, it does use that (so Mr Google says) | 15:45.18 |
henrys | huh there is this pdfblackbox | 15:45.50 |
| I'm going to prepare for the next meeting. | 15:47.01 |
Robin_Watts | "PDFBlackbox includes own PDF processor and does not use third-party libraries for cryptography operations or for loading or saving PDF documents." | 15:47.36 |
paulgardiner | So openssl. I'm still uncertain who's doing what. | 15:51.53 |
| tor8: I have win32 versions of the libs here. Was it linux versions you build? | 15:53.35 |
henrys | Have we dismissed bouncy castle? | 15:53.46 |
tor8 | paulgardiner: yeah, I have the Makethird build unix versions | 15:54.24 |
| I can also have it pick up system version from pkg-config | 15:54.37 |
marcosw_ | ray_laptop: do you have time to look at the Parrots.zip file that came in on Sunday to see why the transfer curve isn't being applied? | 15:54.42 |
tor8 | and macosx has it already pre-installed | 15:54.44 |
paulgardiner | henrys: would be very awkward to use via a java api | 15:54.45 |
Robin_Watts | paulgardiner, tor8: I think we should proceed with openssl under the same terms as v8. | 15:55.07 |
| We only ever need to successfully compile it once per platform :) | 15:55.18 |
tor8 | Robin_Watts: use if it's there, else ignore completely? | 15:55.23 |
Robin_Watts | tor8: right. | 15:55.29 |
tor8 | we could try to use the system versions (which would work fine on unixes) | 15:55.39 |
Robin_Watts | BUT... we should design the API so that there is a veneer layer. | 15:55.50 |
| akin to the pdf_js stuff. | 15:55.59 |
| so that anyone can reimplement the veneer to hook us up to a different openssl lib. | 15:56.16 |
tor8 | I would wrap the signature checking stuff into a veneer layer with both a null and an openssl based implementation | 15:56.18 |
Robin_Watts | tor8: right. | 15:56.27 |
tor8 | which is predicated on HAVE_OPENSSL | 15:56.28 |
| then the source is independent of makefile hackery to link in the openssl | 15:56.47 |
ray_laptop | marcosw: sorry, I was reading the logs. I'll have a look at Parrots | 15:57.07 |
tor8 | on windows, I'm happy to go the v8 route and provide an openssl libcrypto.a download | 15:57.13 |
Robin_Watts | exactly like pdf_js_{v8,none}.c, right? | 15:57.16 |
tor8 | Robin_Watts: yeah. | 15:57.21 |
Robin_Watts | paulgardiner: The first of your API reviews looks fine to me. | 15:57.58 |
tor8 | Robin_Watts: my worry on windows is it'll lead to an explosion of targets. with/without both v8 and openssl makes 4 possible combinations | 15:57.58 |
paulgardiner | tor8: I think last week you suggested I move my crypto routines to fz_crypto.c | 15:58.06 |
tor8 | crypto_pkcs7_(openssl|null).c would be my suggestion | 15:58.31 |
paulgardiner | I guess it could be that that has a nulled version | 15:58.32 |
Robin_Watts | tor8: We can insist on the lib on windows, probably. | 15:58.34 |
| or have different configs in the solution. | 15:58.47 |
tor8 | or just crypto_pkcs7.c for now with a #ifdef HAVE_OPENSSL | 15:58.48 |
| Robin_Watts: it would be good to be able to check it out and just build | 15:59.02 |
| on unix, it's fine because we can skip the openssl library if it isn't there via the makefiles | 15:59.15 |
Robin_Watts | tor8: Right, so we have release-ssl and release configs. | 15:59.26 |
paulgardiner | we choose not to use configs for v8 | 15:59.50 |
| Can't remember why now | 15:59.58 |
tor8 | Robin_Watts: configs could work for ssl, especially if we go the one file with big ifdef route | 16:00.13 |
paulgardiner | configs can also select between files | 16:00.36 |
henrys | okay the gs meeitng | 16:01.12 |
| alexcher:did you see tkamppeter's new bug? | 16:01.34 |
alexcher | henrys: yes, I did. | 16:02.01 |
chrisl | FWIW, I can reproduce the problem here | 16:02.23 |
henrys | marcosw, marcosw_: do you need help dispatching your bizillion bugs, it looks like a very large number of problems to sort through. | 16:02.30 |
| ? | 16:02.37 |
| chrisl:is it just pdfwrite? | 16:03.04 |
kens | chrisl can you tell me what goes wrong / A back trace ? I cannot reproduce it | 16:03.24 |
chrisl | henrys: I don't know yet - I *literally* just saw it crash as your question came up | 16:03.38 |
marcosw_ | henrys: yes, i was going to ask people to grab bugs that were in their area of responsibility, but if you have time to help assign that would be great. | 16:03.40 |
kens | Given that its in sprintf I would guess its writing the link annotation | 16:04.01 |
henrys | marcosw_:I'll start from the largest bug id and go backwards. | 16:04.15 |
chrisl | kens: looks like it's happening in write_xref_section() | 16:04.50 |
kens | Oh, tha'ts 'unexpected' | 16:05.03 |
| Ah, but I did fix something there I think | 16:05.17 |
| Its because we reserve an entry for an item (the N'th page) but don;t ever write that page,so the xref has a hole in it | 16:05.42 |
henrys | ray_laptop:do you need more help with stuff from the new customer? alexcher and chrisl could jump in. | 16:05.56 |
kens | If you look at Till's pdfmark it has a Link annotation with a Dest of Page 3, but the fikle only has 1 page | 16:06.04 |
henrys | ray_laptop:getting them up and going I think is the high priority thing right now. | 16:06.38 |
| marcosw_ and all - all jpeg2000 and jbig2 stuff go to alexcher | 16:07.30 |
alexcher | OK | 16:07.37 |
ray_laptop | henrys: are you talking about cust. 801 ? | 16:07.48 |
henrys | ray_laptop: yea | 16:08.13 |
ray_laptop | if so, their requests are a bit strange. | 16:08.15 |
marcosw_ | henrys: does that include the mudpf issues (or have they been fixed? Robin_Watts suggested that in a recent email)? | 16:08.59 |
ray_laptop | for instance, they asked to speed up the first job, but they've coded up a server mode, so why do they care about the first job? | 16:09.02 |
chrisl | kens: it reads a value for "pos" of 571230650368 from the temp file, and that overflows the 21 bytes in "str" | 16:09.48 |
kens | chrisl that owuld do it :-) | 16:09.59 |
ray_laptop | henrys: I'll check, but I think there is a request pending from cust 801 regarding auto color mode that mvrhel didn't respond to (I thought he was) | 16:10.00 |
kens | chrisl sounds like it is the 64-bit file stufcf, as I suspected | 16:10.13 |
| Is it crashing woth the current code ? | 16:10.22 |
chrisl | don't know, let me check | 16:10.34 |
henrys | marcosw_:I'd like to just assign them alex - hopefully any problem in mupdf is reproducible in gs. | 16:11.09 |
| marcosw_:if not alexcher will sort it and reassign. | 16:12.15 |
ray_laptop | henrys: so, if someone wants to take a look at the one titled "#15 : Auto color mode" that would be fine. I am trying to resolve their BGPrint mode issue | 16:12.30 |
marcosw_ | henrys: alexcher doesn't like to give up his bugs :-) | 16:12.41 |
henrys | marcosw_:maybe this will give him practice. | 16:13.17 |
| ray_laptop:I do think michael has to deal with that one. I thought there were more. | 16:13.53 |
Robin_Watts | marcosw_: I believe I have fixed all the mupdf problems except 1. | 16:14.13 |
chrisl | marcosw_: we already have an open bug about long paths on the command line | 16:14.14 |
Robin_Watts | (the SEGVs, not the valgrind issues) | 16:14.27 |
| I will try and close the bugs asap, but it might not happen before I fly. | 16:14.45 |
marcosw_ | chrisl: i thought we did, but couldn't find it immediately. | 16:14.48 |
chrisl | marcosw_: give me a sec | 16:15.04 |
| http://bugs.ghostscript.com/show_bug.cgi?id=693380 | 16:15.33 |
marcosw_ | I found one for SCO builds, but I figured that was a different prolbem | 16:15.34 |
henrys | marcosw_:where is the code that created these mutated pdf files? | 16:15.42 |
| does google have a home for it? | 16:16.06 |
chrisl | kens: I can't reproduce the crash on current master | 16:16.18 |
Robin_Watts | paulgardiner, tor8: get vs lookup.... my memory was that get would get you a reference, and lookup would not ? | 16:16.24 |
kens | chrisl that's probably good news I think. I did do some fixes in that area | 16:16.41 |
| Hopefully either Till or the original poster can try tghe latest code. | 16:17.27 |
marcosw_ | I don't know where the code to mutate the pdf files is. I can ask for it. Why do you want it? Don't we have enough bugs :-( | 16:17.32 |
chrisl | kens: it does seem to produce a broken PDF though | 16:18.10 |
kens | chrisl if you or Till wants to repor that (and I can reproduce it....) I'll have a go at it. The minimal file Till attached works OK for me on WIndows | 16:18.44 |
tor8 | Robin_Watts: find vs lookup | 16:19.25 |
henrys | ray_laptop:actually if you haven't gotten to 694059 694060 it would be good to get either of those to alexcher or chrisl for preliminary analysis. | 16:20.02 |
| marcosw_:I was hoping for a README that explained the methodology | 16:20.52 |
chrisl | henrys: 694059 doesn't need much analysis - the code specifically disallows certain compression scheme/bit depth combinations. It doesn't say *why* though | 16:21.28 |
| henrys, ray_laptop: I'll take 694059 694060, since the tiff devices are not unknown to me | 16:22.34 |
henrys | chrisl:okay well this is a top ten p1 thing but it has one comment dated 5/9 so I thought somebody ought to do something. | 16:23.12 |
chrisl | henrys: I suspect the compression one is because it doesn't make sense to apply byte-wise compression to 1 bit image data | 16:24.12 |
henrys | and 060 is just a documentation issue? | 16:25.20 |
| maybe marcosw_ has already followed up on these - I didn't check emails. | 16:25.45 |
chrisl | No, I think there's probably an issue - we shouldn't be producing a composite file for 1 bpp output. But it's probably fairly trivial to fix | 16:26.15 |
ray_laptop | marcosw: those Parrots files are strange. They seem to use binary object encoding (just a few places) and they use the Adobe ProcSet. I can't easily find where they do settransfer or setcolortransfer, so I'm adding debug | 16:26.28 |
| chrisl: I wondered about 'pack' with 1 bpp. It will give you a little compression for large runs of white or black, and that's about it | 16:27.26 |
henrys | usually that customer is pretty noisy I'm surprised we got away with letting those sit so long. | 16:28.04 |
ray_laptop | chrisl: I assumed that it is because it is easier to unpack than G4 (they may have SSE or H/W for it) | 16:28.05 |
chrisl | ray_laptop: exactly, pretty daft. But I'm aware of any technical reason for it. I just need to double check that libtiff is okay with it | 16:28.33 |
| I'm NOT aware.... | 16:28.48 |
henrys | okay 30 minutes - done! | 16:29.25 |
marcosw_ | ray_laptop: okay, don't spend too much effort. It's not our problem if the customer is generating bad PostScript (as I said in my email, Photoshop and Acrobat don't perform expected transfer function either). | 16:29.38 |
henrys | marcosw_:I wonder if the bug aging list should come out more frequently if there is a top ten customer in it. We shouldn't have let these two slip. | 16:31.31 |
alexcher | I have RedMon working on 64-bit Windows. What else is needed for the port monitor project? | 16:32.09 |
henrys | alexcher:you were writing a port monitor that would be our code - not Russells | 16:32.58 |
chrisl | kens: Using the full file from 694275 I get a valid PDF, and no crash with current master. So I think it's all good. | 16:33.40 |
henrys | maybe were is a bad tense to use - "you are" would seem more appropriate. | 16:33.57 |
kens | chrisl, that's something of a relief, will you put something in the buig thread ? | 16:33.58 |
chrisl | kens: just about to, yes. | 16:34.11 |
kens | chrisl thanks :-) | 16:34.19 |
alexcher | henrys: Fine, it is mostly a DDK sample anyway. What other functionality is needed. | 16:36.10 |
ray_laptop | marcosw: I set a breakpoint in the C code in BOTH zsettransfer and zsetcolortransfer and the only 'hit' was during our initialization of the default transfer function. | 16:37.31 |
henrys | I think that is all that is on your desk - the other pieces are sprint and xps improvement and we'll bundle it all together so we have a complete printing solution for windows like gsprint - did you have other ideas about the project? | 16:38.19 |
| s/sprint/our own gsprint/ | 16:38.43 |
| alexcher ^^^ | 16:39.49 |
alexcher | OK | 16:40.06 |
henrys | alexcher:you should check each jbig2 and jpeg2000 problem in both ghostscript and mupdf, Shelly I think can work on both but I think Simon is mupdf only. | 16:41.28 |
alexcher | Yes | 16:42.12 |
ray_laptop | marcosw: If I use the Install procedure to change the default graphics state transfer function, it is effective for either file. For example, to make it lighter: | 16:42.24 |
| gs ... -c "<< /Install { { .4 exp } settransfer } >> setpagedevice" -f Parrots.ps | 16:42.26 |
henrys | alexcher:I assume you saw their email to Robin_Watts and support you can follow up with that note and dispatch them how you see fit. | 16:42.31 |
ray_laptop | marcosw: can you take it from here ? | 16:42.50 |
alexcher | henrys: yes, I did. | 16:43.05 |
henrys | ray_laptop:I think marcosw fell off. | 16:44.27 |
ray_laptop | henrys: OK. I'll just reply to them. | 16:45.39 |
Robin_Watts | paulgardiner: Second API review; same comments as originally, but I don't see anything hugely offensive in it. | 16:45.52 |
paulgardiner | What were the comments again? You mentioned the placed where there were three function calls check len; resize; get entry; but I've dealt with that. | 16:48.18 |
Robin_Watts | for's with function calls. | 16:48.47 |
| read len into a variable, and then use that. | 16:49.01 |
paulgardiner | Ah yes. Call in the test. | 16:49.38 |
kens | OK I'm going to call it a night, bye all | 16:49.51 |
paulgardiner | I could change that | 16:49.53 |
henrys | chrisl:I'm going to assign the free type fuzzing issues to you without commenting in each that it is a problem in the free type code. | 16:50.22 |
chrisl | henrys: I assumed marcosw would do that, anyway | 16:50.45 |
paulgardiner | tor8: what was it that you wanted to check with robin about pdf_xref_obj? | 16:50.47 |
henrys | which reminds me I did hear from URW - resounding "no" to the buyout and they are going to fix the widths for a fee. | 16:51.53 |
| they consider the GPL fonts unsupported | 16:52.29 |
| chrisl ^^^ | 16:52.43 |
chrisl | Hmm, oh well..... | 16:52.48 |
| henrys: I could have fixed them myself, but I doubt they'd take patches | 16:53.16 |
| Robin_Watts: does mupdf write jpegs? | 16:53.57 |
Robin_Watts | chrisl: Not currently, why? | 16:54.06 |
henrys | chrisl: best to keep things in their handwriting | 16:54.20 |
chrisl | Robin_Watts: I'm just starting on thirdparty list | 16:54.39 |
henrys | I think there are many more jpeg2000 problems than previously identified - Robin_Watts | 16:55.29 |
Robin_Watts | chrisl: ah, right. | 16:55.40 |
| henrys: Not from the mupdf fuzzing files? | 16:55.55 |
henrys | oh right no, ghostscript | 16:56.18 |
Robin_Watts | right. The mupdf ones have all been fixed. | 16:56.41 |
henrys | marcosw:for the logs we probably should fuzz luratech as well. | 16:57.00 |
| marcosw:the luratech build that is. | 16:57.17 |
ray_laptop | henrys: did we also fuzz the UFST build, or do we need to do that as well ? | 17:00.12 |
| henrys: those two are our 'customer only' libraries that GPL users never see | 17:00.44 |
chrisl | ray_laptop: OMFG!! *PLEASE* don't go there! | 17:00.44 |
henrys | ray_laptop:good point we probably should. chrisl is getting the bulk of the fuzzing action already. | 17:00.53 |
ray_laptop | sorry, chrisl | 17:01.01 |
henrys | at least of the problem I'm assigning now. | 17:01.10 |
chrisl | I won't be fixing any fuzzing problems in UFST | 17:01.22 |
ray_laptop | I guess I will wear my kevlar to the next meeting | 17:01.27 |
| and not turn my back on chrisl ;-) | 17:01.56 |
henrys | I should fix some of the default assignments for component owners in bugzilla while I'm doing this. | 17:02.16 |
ray_laptop | henrys: you could take ownership of UFST -- I doubt that chrisl would mind ;-) | 17:02.41 |
chrisl | ray_laptop: like I say, I'm refusing point-blank to work on fuzzing problems in UFST. I'd consider forwarding them to Monotype, though | 17:02.56 |
henrys | that would dull my pointy hair | 17:03.06 |
ray_laptop | henrys: we wouldn't want that. You're already dull enough ;-) | 17:03.43 |
Robin_Watts | paulgardiner: Could not find method android.view.MotionEvent.getActionMasked, referenced from method com.artifex.mupdfdemo.MuPDFReaderView.onTouchEvent | 17:03.57 |
ray_laptop | if our meeting is over, I've got to take my car in. Anything else for me right away ? | 17:04.33 |
henrys | nope we're done | 17:04.46 |
ray_laptop | henrys: (marcosw for the logs). Reply sent to cust 801 about the settransfer (and cc the usual population) | 17:05.16 |
henrys | ray_laptop:great. If marcosw pops back I'll point him to this stuff. | 17:05.41 |
ray_laptop | bbiaw | 17:05.56 |
chrisl | henrys: I'm a bit surprised there are issues coming up in Freetype, given the way we parse the font into a gs_font and then rebuild it to pass into freetype | 17:06.29 |
paulgardiner | robin_watts: looks like you can replace that with getAction() & ACTION_MASK | 17:08.08 |
ray_laptop | chrisl: thanks for taking the tiffsep bugs. | 17:09.03 |
Robin_Watts | event.getAction() & event.ACTION_MASK... compiled... | 17:11.27 |
chrisl | ray_laptop: no problem - they don't look very onerous | 17:11.57 |
henrys | I've been changing the component from "fuzzing" to the actual component. Hope that is okay - fuzzing is not really a component - maybe that should be stated elsewhere. | 17:15.53 |
Robin_Watts | I have mupdf running on the nook. | 17:28.54 |
| No multi-touch it seems, so I can't zoom. | 17:29.02 |
| and I had to disable the setImageFilter calls on the buttons. | 17:29.24 |
| but we can at least see mupdf running on an e-ink device. | 17:29.36 |
marcosw_ | henrys: replying to your comment regarding the bug aging list, I think an alternative is to have bugzilla send out reminders. I'll check it's that's an option. | 17:31.00 |
henrys | marcosw_:thanks ray_laptop had some stuff for you also (in the logs) | 17:31.37 |
| Robin_Watts: I didn't even know the nook was android. | 17:32.09 |
Robin_Watts | henrys: It is. I bought one (30 quid!), and rooted it | 17:32.34 |
| and installed the amazon kindle app :) | 17:32.42 |
henrys | wasn't MS going to buy it at one point? | 17:32.48 |
Robin_Watts | That was a very recent rumour. It might still be happening. | 17:33.04 |
henrys | Robin_Watts: oh cool. | 17:33.06 |
marcosw_ | henrys: found the irc messages for me in the archive. | 17:36.25 |
henrys | marcosw_:okay | 17:36.44 |
| marcosw_:I wonder why you can't change the component on multiple bugs - that would sure make this go faster. | 17:38.35 |
marcosw_ | henrys: particularly odd since they have a pull down for Component. Preusmably a bug in bugzilla. | 17:41.02 |
| is this for the fuzzing issues? I haven't been changing the component when I reassign them. | 17:41.31 |
henrys | marcosw_:yeah I said something about that in the logs - but I don't care if you want to leave it. | 17:42.21 |
| marcosw_:I did a handful of them I'll come back later and do some more. | 17:47.33 |
tor8 | Robin_Watts: whether it's necessary and why not use pdf_load_object instead | 18:12.29 |
Robin_Watts | tor8? | 18:12.53 |
| right. sorry. | 18:13.17 |
| I've got to run. Parents here. | 18:15.28 |
| It looks like there might be some overlap, yes. | 18:15.35 |
tor8 | alright. ttytm. have fun! | 18:15.47 |
SpNg | is there an easy way to extract what color space is being used in an EPS file? | 20:29.50 |
| Forward 1 day (to 2013/05/29)>>> | |