| | <<<Back 1 day (to 2011/07/23) | 2011/07/24 |
| Robin_Watts | ghostgum: "my preferred solution would be to make the display device work with the previous 96 entry palette" - but that's precisely the bug we had reported... | 09:32.18 |
| ghostgum | Robin: I thought the bug report was that it was displaying things badly, because something else had been changed in ghostscript between 8.70 and 9.02, causing colours and greys to be done poorly. | 10:51.56 |
| | Have a look at the images I attached to the bug report. These show how it used to behave in 8.70, how it behaves in 9.04 pre-release, and how it behaves after your fix (more broken). | 10:54.04 |
| | GS 7.07 with that PDF file is correct. GS 8.70 is not. | 10:59.12 |
| | Yet colorcir.ps with -dDisplayFormat=16#30801 is correct with both 7.07 and 8.70. | 11:00.21 |
| | So the bad behaviour is due to more than one change. | 11:00.39 |
| Robin_Watts | ah, ok. | 11:36.02 |
| | Changes between 8.7x and 9.0x are most likely to be because of the ICC stuff. | 11:36.45 |
| | which means it could be really hard to isolate the particular problem. | 11:36.59 |
| ghostgum | Yes, could be quite hard. | 11:37.11 |
| Robin_Watts | When you say it's "more broken" with my fix, you mean due to gsview using a mismatching palette? | 11:37.20 |
| ghostgum | In 7.07, the display of the PDF attached to the bug is correct. The half toning looks bad, but that's a problem that shouldn't be fixed here. | 11:37.41 |
| Robin_Watts | (sorry, that sounded bad, wasn't blaming gsview) | 11:37.54 |
| | If we were to update gsview to use a matching palette, would that solve all the problems ? | 11:38.30 |
| ghostgum | By 8.70, the PDF displays incorrectly. The background blue sky is missing etc. Colorcir.ps is still correct. | 11:38.54 |
| | By 9.03, even colorcir.ps is bad. | 11:39.00 |
| | Your fix does not work even with GS alone. | 11:39.09 |
| Robin_Watts | oh, really? Can you give me an example command line please? | 11:39.25 |
| ghostgum | gswin32c -dDisplayFormat=16#30801 examples/colorcir.ps | 11:39.48 |
| Robin_Watts | Looks fine to me. | 11:40.29 |
| ghostgum | It's not colour! | 11:40.31 |
| Robin_Watts | What's wrong with it? | 11:40.36 |
| ghostgum | The colour bits are greyscale. | 11:40.46 |
| Robin_Watts | ?!?!? | 11:40.51 |
| | It's color for me | 11:41.03 |
| | That's the png I posted the other day. | 11:41.10 |
| ghostgum | So just to confirm I have the right code, I do a "git fetch", "git rebase origin/master", then recompile? | 11:41.46 |
| Robin_Watts | Yes. | 11:41.57 |
| | Try a clean rebuild? I'll do the same here. | 11:42.05 |
| ghostgum | I'll recompile, but I believe I did a clean first. | 11:42.23 |
| | My compile will be really slow. My disk is being trashed with video recording and transcoding. | 11:42.56 |
| Robin_Watts | git diff is showing me no differences here... | 11:43.09 |
| | Let me boot ubuntu and try a fetch/rebuild there in parallel with this one. | 11:43.49 |
| ghostgum | I'm using Windows 7, 32-bit, MS Visual Studio 2008. | 11:44.13 |
| Robin_Watts | I'm using Windows XP, 32bit, 2005# | 11:44.35 |
| ghostgum | nmake clean isn't removing all files from obj. | 11:45.43 |
| | Not a major problem, but I thought it used to do so. | 11:46.16 |
| Robin_Watts | What files is it leaving? Probably worth mentioning to chrisl. | 11:46.51 |
| ghostgum | Subdirectories aux_ and cups. Also some *.exe build tools. | 11:47.53 |
| | I've deleted them manually so I can't check for a while. | 11:48.16 |
| Robin_Watts | aux_ and cups I would expect to be left. | 11:48.26 |
| ghostgum | There's also a few compiler warnings, particularly from cups. | 11:48.39 |
| Robin_Watts | oh, yes. OOOOh yes. | 11:49.40 |
| ghostgum | Where abouts in the UK are you located? | 11:50.36 |
| Robin_Watts | North Oxfordshire. | 11:50.45 |
| ghostgum | Don't think I ever drove through there. Nearest would have been heading from Suffolk, via M25 then around to A3. | 11:51.37 |
| Robin_Watts | The M25? Well, you've seen the best of Britain then :) | 11:52.16 |
| ghostgum | We did visit Stratford. | 11:52.21 |
| | M25 isn't quite a circular car park. | 11:52.32 |
| Robin_Watts | Stratford-on-Avon? That's not far from here at all (30 mins or so) | 11:52.46 |
| ghostgum | Ok, so I got slightly close. We've got a Stratford town in Victoria, on the river Avon :-) | 11:53.31 |
| Robin_Watts | We also have a Stratford in London. | 11:53.46 |
| | It's where the Olympics are going to be. | 11:53.56 |
| ghostgum | I only knew of the Avon one. | 11:53.59 |
| Robin_Watts | It's *completely* unlike the Avon one :) | 11:54.10 |
| | They are bulldozing most of it to build the Olympics, and that alone did billions of pounds worth of improvements. | 11:54.40 |
| ghostgum | Did a rebuild. Still getting black and white cor colorcir.ps. | 11:54.44 |
| Robin_Watts | And after my rebuild I still get color :( | 11:56.04 |
| | Let's wait for Ubuntu as the tie breaker :) | 11:56.21 |
| ghostgum | So I didn't design the 8-bit native mode very well. Your 6x6x6 colour cube would have been much better, but I think it's too late to change that. | 11:57.02 |
| Robin_Watts | Ubuntu gives color too. | 11:57.42 |
| ghostgum | Any change to the palette means that anything that uses the 8-bit native mode will break. I don't know how many programs use it apart from GSview. On GSview, it only got used on really old hardware, unless it is explicity selected. | 11:58.44 |
| Robin_Watts | actually, I'm not convinced ubuntu is correctly following the 8 bit instruction. | 12:00.21 |
| ghostgum | Something screwy here. | 12:01.42 |
| | -dDisplayFormat is doing nothing. | 12:03.52 |
| Robin_Watts | -dDisplayFormat=16#30401 gives me something that's definitely different (on windows) | 12:08.42 |
| | but it's having no effect on Ubuntu | 12:09.06 |
| | but ubuntu is using the x11 driver, which is different, I think ? | 12:09.19 |
| ghostgum | The values from -dDisplayFormat are getting into the device (as shown by currentpagedevice), but not affecting the display. | 12:09.42 |
| | On Linux, use gsx (not gs), with -sDEVICE=display | 12:10.04 |
| Robin_Watts | I don't have gsx | 12:10.50 |
| ghostgum | It's almost like some other change has broken the display device colours. | 12:11.05 |
| Robin_Watts | I'm now getting dithering in the black and white parts of the colorwheel on windows. | 12:11.54 |
| ghostgum | To get gsx, you needed to build ghostscript as a shared object. This used to be a separate makefile (unix-dll.mak) with "make so", but with rillian's changes it might be built into the auto make. | 12:11.56 |
| | Mode 16#30401 is VGA 4-bit. | 12:12.23 |
| Robin_Watts | Yes, but I'm back to 16#30801 now. | 12:12.37 |
| ghostgum | Everything gives me black and white with code from git. | 12:13.33 |
| Robin_Watts | With 30401, I lose the black and white stuff half way around the color circle. | 12:13.38 |
| | With 30801, it does all the way around, but is still halftoned. | 12:14.09 |
| ghostgum | It's too late for me to start with the debugger to trace what is happening. | 12:14.09 |
| Robin_Watts | yeah. I might just backout the patch anyway because we have a release looming. | 12:14.28 |
| | And I hadn't appreciated when I started this, that it was just the display device that's affected. | 12:14.46 |
| | OOH. Do we have any spare bits in the displayFormat ? | 12:15.07 |
| ghostgum | Please do backout the patch. | 12:15.13 |
| | Yes, up high. But the 8-bit mode isn't that important. I'm not even sure why the bug reporter was using it. | 12:15.54 |
| Robin_Watts | me either. | 12:16.03 |
| ghostgum | Perhaps we should ask that question! | 12:16.16 |
| Robin_Watts | If it really *is* important that we fix it, then we could use another bit to mean "better 8 bit palette". | 12:16.35 |
| ghostgum | Yes, or you could suggest that they use the 24-bit RGB mode. | 12:17.10 |
| Robin_Watts | or "alternative palette" and only define what that means for 8 bits. | 12:17.20 |
| | I think he was maybe assuming that the 8 bit palette he got with the display device would be the same that he'd get with any other 8 bit device. | 12:17.51 |
| | But I'll back out the patch and put more comments on the bug. | 12:18.08 |
| ghostgum | Never assume that two palettes are the same. | 12:18.12 |
| | Thanks. | 12:18.18 |
| Robin_Watts | np. Thank you. | 12:18.25 |
| ghostgum | And mark the bug as open rather than resolved. | 12:18.37 |
| Robin_Watts | Indeed. | 12:18.43 |
| ghostgum | Any other changes that might impact on GSview at this time? | 12:19.04 |
| | Sounds like some memory GC changes that may cause problems. | 12:19.31 |
| | Are they in yet, or just being talked about? | 12:19.44 |
| Robin_Watts | they aren't in, AIUI. Not sure why they'd affect ghostview... | 12:27.37 |
| ghostgum | I got added as CC on a bug, and there had been a discussion about GSview calling GS with gsapi_new_instance each time it re-ran GS. | 12:29.34 |
| | Off to bed. | 12:31.58 |
| Robin_Watts | henrys: Do we support the binary postscript stuff? | 15:36.10 |
| alexcher | Robin_Watts: We do. | 15:54.27 |
| Robin_Watts | alexcher: A stupid question I'm sure, but in Appendix F the PLRM lists a load of words for the System Name Encodings. | 15:59.27 |
| | In the binary section, it contains encodings (145 and 146) that select names from that list. | 16:00.24 |
| | they can either be literal or executable. | 16:00.34 |
| | What's the difference ? | 16:00.55 |
| alexcher | between literal and executable names? | 16:01.43 |
| Robin_Watts | yes. | 16:02.11 |
| alexcher | When an executable name is executes, it is looked up in the dictionary stack. | 16:03.42 |
| Robin_Watts | We can save lots of memory in the files we put into %rom% if rather than having "currentcmykcolor" (say) we put byte 146 then byte 28 | 16:03.48 |
| alexcher | Execution of a literal name is a no-op. | 16:04.03 |
| | Of course. | 16:04.47 |
| Robin_Watts | So in my 'compaction' stage that I do in mkromfs (where I strip comments, compact whitespace etc) I'd like to replace such known tokens. | 16:05.11 |
| | how do I know whether to make them a literal name or an executable name ? | 16:05.33 |
| | I'm guessing they should always be executable? | 16:05.55 |
| alexcher | A literal name is precedded witg '/', e.g /foo | 16:06.13 |
| Robin_Watts | Ah! | 16:06.19 |
| alexcher | A literal name is precedded with '/', e.g /foo | 16:06.21 |
| Robin_Watts | That makes complete sense, of course. Thanks. | 16:06.33 |
| alexcher | /foo is an "immediately evaluated name" | 16:07.55 |
| | An immediately evaluated name looks like //foo | 16:08.21 |
| henrys | I don't know if your idea of binary encoding in rom is going to be popular... | 16:08.33 |
| Robin_Watts | henrys: Why not? | 16:08.42 |
| | compaction has saved us 640K out of the 7.8Meg currently. | 16:08.59 |
| | binary encoding will save us LOTS more. | 16:09.06 |
| | It's less to parse when we come to execute stuff, so should be faster too. | 16:09.35 |
| henrys | what has rayjj said about it? | 16:10.24 |
| Robin_Watts | this only occured to me last night, so I haven't had a chance to talk to ray about it yet. | 16:10.49 |
| henrys | my issue would be finding bugs in the making of romfs has become much more difficult. | 16:11.14 |
| | but his issues are more important than mine since he owns it. | 16:12.00 |
| Robin_Watts | If you think that compaction is causing problems, you can turn it off at the command line. | 16:12.15 |
| | (i.e it's enabled/disabled for files using -C and -B) | 16:13.08 |
| | And it may make sense to offer it as a separate tool too, for people that want to optimise the startup files and still run from disc. | 16:13.52 |
| henrys | okay I did get the sense ray wanted the focus to be on cmaps and not on mucking more with gs_init.ps but that may be something I misunderstood. | 16:14.35 |
| Robin_Watts | Right. | 16:14.52 |
| | The CMAP stuff is where there is a lot to be run. | 16:15.11 |
| henrys | my 2 cents, and now it's between you and him. | 16:15.21 |
| alexcher | Robin_Watts: I think, a magic comment that controls the conversion would work the best to bisect the file. | 16:16.11 |
| Robin_Watts | afaict, the adobe cmap files are in the form of postscript code that sets up data structures. | 16:16.30 |
| | they seem to be in the form: <code1> <code2> to signify that code1 maps to code2. | 16:17.01 |
| | repeated. | 16:17.05 |
| | where subsequent lines are frequently <code1+1> <code2+1> etc | 16:17.35 |
| | The mupdf stuff encodes things as ranges, and hence is smaller. | 16:17.46 |
| | Someone that understood postscript better than me could probably write something to take the mupdf versions and output them as postscript code to build an equivalent table to what we currently have, but in much less space. | 16:18.48 |
| | And then, the binary encoding would save even more. | 16:18.59 |
| alexcher | Robin_Watts: Current CMap files are highly inefficient. We have deveral versions of the files in the resource directory. We need to store differences instead. | 16:19.29 |
| Robin_Watts | but I will confess to not understanding the exact PS formatting requirements. | 16:19.42 |
| | alexcher: Ah, you mean have various 'master' encodings, and then 'modification' encodings from them ? | 16:20.21 |
| alexcher | Robin_Watts: yes | 16:20.38 |
| Robin_Watts | Not sure whether that idea feeds back trivially into mupdf, as mupdf has the data arranged in such a way that it can binary search within it without unpacking it. | 16:21.18 |
| | but gs may have more cmaps than mupdf ? | 16:21.53 |
| alexcher | Robin_Watts: Most likely, gs doesn't need some of CMap files that we have collected from every source. | 16:23.37 |
| | Robin_Watts: gs has an internal CMap format. It decodes CMap files into the internal format first. | 16:24.27 |
| henrys | on my mac adobe uses 181 CMap files and we seem to have 242 - I was just curious... | 16:26.59 |
| Robin_Watts | alexcher: So... why do we put the raw ones in Rom ? | 16:30.04 |
| alexcher | Robin_Watts: do I have to answer? | 16:32.19 |
| Robin_Watts | alexcher: If you know of a reason :) | 16:32.48 |
| | hey ray_laptop. I had some ideas about using binary encodings to shrink the rom files some more. See the logs. We're now discussing CMAPs. | 16:33.33 |
| alexcher | Robin_Watts: we have limited resources and a big domain, etc... | 16:34.10 |
| henrys | Robin_Watts:the exact same thing could be said for fonts. We haven't had a great deal of pressure to reduce rom size, so we haven't. | 16:34.20 |
| Robin_Watts | Ok, so the answer is "we could, but we've never needed to, so we haven't" ? | 16:34.43 |
| henrys | Further rom size tends to only give improvement if there is a large change and customer 2 meg to 1 meg matter 2 to 1.5 does not. | 16:35.27 |
| alexcher | Robin_Watts: In fact, the whole Ghostscript VM is an internal representation of the start-up code. | 16:36.59 |
| Robin_Watts | henrys: By that argument, we'd just keep bloating bit by bit for ever. | 16:39.16 |
| | alexcher: Unless you're advocating us shipping a coredump, I'm not sure how that helps; we aren't SMLNJ :) | 16:39.49 |
| alexcher | Adobe does ship a dump of VM core with their products. | 16:41.49 |
| henrys | Robin_Watts:no it just says there is pressure to keep rom below 2 megs and not 1.5 becuase the latter does not give you a cheaper stick. | 16:42.51 |
| Robin_Watts | henrys: Ah, right. | 16:43.03 |
| | Well, they presumably ship other stuff than gs. Any bytes we save are bytes they can use. | 16:43.56 |
| henrys | yes, now that said I think the romfs situation is bad - Monotype gives a much smaller footprint for fonts than we do and I don't think PCL customers would put with moving from Monotype to UFST - postscript/pdf are a bit less resource senstive so I am not sure. | 16:46.04 |
| | s/would put/would put up | 16:46.30 |
| | bah moving from Monotype to Artifex font scaler. | 16:46.52 |
| Robin_Watts | given that our pdf requires ps, we probably look bad for people that just want pdf too. | 16:47.15 |
| alexcher | Our PS fonts can be easily converted to CFF format without any loss of information. | 16:48.44 |
| henrys | alexcher:what is the size difference - T1 fonts are fairly light now? | 16:50.25 |
| | well at least compared with the pcl tt fonts they are. | 16:50.39 |
| | for those with small children: let me give you some advice if your kids want turtles don't do it. My kids wanted turtles around the ages of 5 & 7 I now have the turtles the kids have left the house ... I will be cleaning up after these turtles for the rest of my life - they can live to be 40 ... back to the turtle tank. | 16:54.25 |
| Robin_Watts | henrys: Do you not have any friends with 5 to 7 year old children who would also like turtles? :) | 16:56.41 |
| henrys | not that haven't heard me complain about taking care of them. scared them off. | 16:58.24 |
| alexcher | CFF fonts will save a few K per font. Most of the font data are charstrings, and they remain unchanged. | 16:58.46 |
| Robin_Watts | henrys: freecycle ? | 16:59.00 |
| alexcher | We ship 35 font files. | 16:59.28 |
| ray_laptop | Robin_Watts: I saw the bit about binary. The PS 'setobjectformat' > 0 is OK for operators that are 'known', but NOT good for strings, names (and I think numbers). | 17:35.57 |
| | you can test what things become what size simply enough with 'writeobject' | 17:36.54 |
| | for example, the 17 bytes of "<1234><5678>9999\n" take 40 :-0 bytes in binary object form | 17:43.01 |
| | clearly for CMap files we can do MUCH better. | 17:43.23 |
| | the business of making names "immediately evaluated" (bound) cannot (in general) be done automatically by mkromfs since some things rely on late binding and name lookup | 17:46.01 |
| | but the 'name' in VM is only stored in string form once, and is stored in procedures as a name_index | 17:46.51 |
| | so long names (operators or otherwise) don't take up lots of VM in procs. | 17:47.18 |
| | as long as the compression works with frequently used strings, then the %rom% space may not decrease much by using binary object format | 17:48.37 |
| | I do agree that changing the fonts to CFF will help a bit -- in my experience from several years ago, CFF fonts were more compressible than eexec .pfb format | 17:51.08 |
| | if we are squeezing, we may look at other compression for some slight gain (bzip2 and lzma). | 17:52.28 |
| Robin_Watts | ray_laptop: "17 to 40" Not by my reading of it. | 17:56.09 |
| | <1234> goes to 4 bytes. 142 4 0x12 0x34 | 17:56.59 |
| | Sorry... 142 2 0x12 0x34 | 17:57.33 |
| | <5678> goes to 142 2 0x56 0x78 | 17:57.49 |
| ray_laptop | Robin_Watts: Here's what I get: | 17:57.56 |
| | 00000000: 83 01 0E 00 05 00 02 00 08 00 00 00 12 34 83 01 | 17:57.58 |
| | 00000010: 0E 00 05 00 02 00 08 00 00 00 56 78 83 01 0C 00 | 17:58.00 |
| | 00000020: 01 00 00 00 E7 03 00 00 |........ | 17:58.02 |
| Robin_Watts | You're using a 'binary object sequence' | 17:58.37 |
| | which is a different beasty. | 17:58.43 |
| | That's aimed at speed of execution, not space. | 17:58.52 |
| ray_laptop | The PS I used was: | 17:59.02 |
| | 4 setobjectformat | 17:59.04 |
| | /O (xxx.bin) (w) file def | 17:59.05 |
| | O <1234> 0 writeobject | 17:59.07 |
| | O <5678> 0 writeobject | 17:59.09 |
| | O 999 0 writeobject | 17:59.10 |
| | O flushfile | 17:59.12 |
| Robin_Watts | right. But you can do MUCH better than that. | 17:59.25 |
| | I make it 10 bytes. | 17:59.58 |
| | because you lose the whitespace too. | 18:00.14 |
| | In PLRM v2 see section 3.12.1 | 18:00.31 |
| | you're using 3.12.2 "binary object sequences" | 18:00.49 |
| ray_laptop | Robin_Watts: but what I proposed for CMap would let us get to 8 bytes | 18:00.51 |
| Robin_Watts | For CMAPs, we can do better by moving to a different representation, yes. | 18:01.30 |
| | but in general for compacting postscript, we can do LOTS better than we currently are by using the binary tokens. | 18:01.55 |
| ray_laptop | because we would use a different operator before hand to load the binary onto the stack as strings | 18:01.56 |
| Robin_Watts | ray_laptop: Where are you suggestions for cmaps ? | 18:02.12 |
| | alexcher says that the cmap stuff is converted to a different internal format anyway. | 18:02.30 |
| | We ought to seek to understand that internal format before we devise how we store it, because that may have an effect on which choice we make. | 18:03.02 |
| | Storing the cmaps in a way that requires additional processing to decode them is bad, as speed costs us too. | 18:03.46 |
| | using the binary representations, it should be faster than the ascii forms. | 18:03.57 |
| ray_laptop | Robin_Watts: the method I proposed wouldn't have to depend on the internal storage -- the operator would load the stack for the 'end___' operator that is the same | 18:04.06 |
| Robin_Watts | but if there is an internal version of the data structure then targetting that could be even better. | 18:04.37 |
| | being called to help with food. | 18:04.50 |
| ray_laptop | CMaps are NOT frequently loaded from the Resource/CMap file. Once per instance | 18:04.55 |
| | then they are kept in the internal representation | 18:05.18 |
| | I have to go get coffee... l8r | 18:05.40 |
| Robin_Watts | Right. If we could find a way to efficiently store that internal representation, we'd get the best of all worlds, I think | 18:05.45 |
| | I have to help with dinner too. bbl. | 18:05.54 |
| | Is the internal representation documented anywhere ? | 18:12.20 |
| | hey kens. | 18:17.16 |
| kens | HI Robin_Watts | 18:17.21 |
| Robin_Watts | My postscript compaction stuff stops ps2write working :( | 18:17.41 |
| kens | I read it in the log | 18:17.48 |
| Robin_Watts | any thoughts ? | 18:17.55 |
| kens | I don't know why , but I can look into it tomorrow | 18:18.02 |
| | After I commit the textwrite device. | 18:18.11 |
| Robin_Watts | OK, we can talk about it then, thanks. | 18:18.16 |
| kens | No problem. | 18:18.21 |
| | Just digitising the video of Melanie's jumping lesson today. | 18:18.37 |
| | Miranda starts up automagically, | 18:18.48 |
| sags | Hi all. Have a few seconds for a quick question? | 18:25.33 |
| kens | You can ask, whether I can answer is another problem :-) | 18:25.57 |
| Robin_Watts | hi sags. | 18:25.59 |
| sags | The 9.04 release, will it be WINDOWS_NO_UNICODE by default, or it will be with the utf8 support in? (And how soon will this release be?) | 18:27.22 |
| kens | 9.04 is due for release *before* the Ubuntu release freeze. | 18:27.42 |
| | So before August 11th IIRC. | 18:27.53 |
| Robin_Watts | sags: It will have the utf8 support in. | 18:28.06 |
| | have you found more problems with it ? | 18:28.15 |
| sags | @Robin_watts: Yes, actually one that can be pretty big. | 18:29.00 |
| Robin_Watts | damn. tell me more... | 18:29.15 |
| kens | Best report it soon. CHris was going to brancht the release on Mondya | 18:29.29 |
| kens | wishes he could type... | 18:29.46 |
| sags | In short: the GSAPI interface needs to be updated. The subject of the *encoding* used by the various char * inthere was bypassed until now. With the utf8 in the interpreter, it cannot be anymore. | 18:30.17 |
| Robin_Watts | sags: effectively it has been updated. | 18:31.24 |
| | everything is assumed to be utf8 now. | 18:31.35 |
| sags | Yeah, but that breaks compatibility without that being needed. | 18:32.08 |
| | I think there must be "A"/"W" (ANSII/ UNCIODE utf16) versions for the entry points that handle char *. Maybe add "8" to those too, for utf9- encoded strings. | 18:32.17 |
| | Consider the "command-line" arguments. The xes hide the fact that the PS interpreter wants utf8. But gsapi_init_with_args() does not. | 18:33.12 |
| | s/xes/exes/ | 18:33.22 |
| Robin_Watts | (or rather... all postscript strings etc are assumed to be utf8 encoded. Any postscript streams with literal bytes in are treated as literal bytes (which is important for binary postscript). | 18:33.51 |
| | sags: I don't see that anything has actually changed there. | 18:34.08 |
| | On linux, the utf8 encoding/decoding has been done for us for years by the shells and we've never noticed. | 18:34.38 |
| | All that we've done here is to pull windows into line with that. | 18:34.52 |
| sags | Yes, literal strings are OK. For example gsapi_run_string_begin() won't have "A"/"W"/"8" variants. but gsapi_init_with_args() and a few others need these. | 18:35.01 |
| Robin_Watts | sags: I'm not sure they do; can you give me an example please ? | 18:35.47 |
| sags | s/string sre OK/byte sequences fed to the interpreter and which are passed as char[]/ | 18:35.52 |
| Robin_Watts | brb | 18:36.14 |
| sags | An example. If you call the exe from a batch file, you specify the input filename on hte command lien as ANSII as you did before. The GSAPI equivalent is gspai_init_with_args(), but there a file stores in a localized "Document and Settinbg" won't work anymore as ANSII. Currentl;y is has to be explicitely converted to utf8 by the caller. | 18:38.05 |
| | There's where a gspai_init_with_argsA() helps. And a gsapi_init_with_argsW() if the GSAPI client is a UNICODE application. | 18:39.40 |
| | s/gspai_init_with_argsA()/gsapi_init_with_argsA()/ Sorry, my typing is bad... | 18:40.19 |
| Robin_Watts | sags: Let me run through that. | 18:40.36 |
| | I have to go. I'll ponder on it, but I don't think it's a big problem actually. | 18:42.11 |
| | sorry. | 18:42.17 |
| sags | And something more: one the GSAPI cleared, the exes will nedd to chage. Because they have utf16 (wchar_t) already, they call the "W" entries directly. If they explicitely convert to utf8, the will call the "8" entries. | 18:42.19 |
| | Let the "no suffix" entries to be equivalents of the "A" ones, for backwards compatibility. | 18:42.54 |
| Robin_Watts | sags: (just popped back) You may be right, I can see your argument. I'll talk it over with the others tomorrow. It doesn't sound like a huge problem to fix. | 18:48.17 |
| sags | Fro when you'll come back to read the logs: The problem is that existing GSAPI clients won't work especially on localized systems, where standard directories have extended characters in the path. | 18:48.26 |
| kens | I believe that doesn't work currently anyway (perhaps I'm wrong) | 18:49.07 |
| sags | A second problem, the GSAPI clients may need to know if it's dealing with the WINDOWS_NO_UNICODE ot the utf8 DLL; somehow from the gspai_revision(). | 18:49.18 |
| Robin_Watts | I personally dislike the 'A' and 'W' convention. I'm more tempted just to add a 'U' variant (for unicode). | 18:49.20 |
| sags | @kens: Before the utf8, that worked because everything was using whatever the host used. | 18:50.51 |
| ray_laptop | sags: Are you concerned with the 'gsapi_run_string' string or something else ? | 18:51.29 |
| | because, AFAIK, run_string just takes bytes and never had a "wide char" mode. Or are you concerned with the 'argc' contents for the gsapi_init_with_args | 18:52.39 |
| sags | @ray_laptop: not the run_strng(), that must not change. The gsapi_*() that take "true strings", like "init" (for the argv[]),. | 18:52.43 |
| ray_laptop | I meant argv | 18:53.00 |
| sags | And there are a few others, less obvious. For example gsapi_set_display_callback() has a "true string" as a parameter for display_callback_t.display_separation() | 18:54.12 |
| ray_laptop | sags: display_callback just has a void * | 18:54.49 |
| | but run_file may be an issue | 18:55.07 |
| | sounds like a can of worms | 18:55.54 |
| sags | gsapi_set_display_callback() has a 2nd argument of type display_callback, which is astrunt that contains a function pointer that has a char * argument "const char *component_name" | 18:56.57 |
| | Yes, run_file() too. | 18:57.08 |
| | About the gsapi_dispaly_callback(): that means we also need different structures, dispaly_callbackA and dispaly_callbackW. | 18:57.59 |
| | Can of worms? Yep. But if released this way, it breaks existing GSAPI clients, then after fixing it properly will break these again. | 18:59.22 |
| alexcher | ray_laptop: The sample file from the bug 692370 renders incorrectly on the display device, but it renrers fine on a bunch of other devices I've tried. | 19:59.04 |
| Robin_Watts | alexcher: Where in the source can I see the 'internal structure' used for cmaps please ? | 20:09.22 |
| alexcher | Robin_Watts: gs_cmap.ps | 20:11.12 |
| Robin_Watts | thanks. | 20:11.31 |
| alexcher | Robin_Watts: I'd be glad to take over CMap handling. | 20:14.36 |
| Robin_Watts | alexcher: Go for it. | 20:14.52 |
| | That probably makes a lot more sense :) | 20:15.05 |
| alexcher | Robin_Watts: CMap can be easily loaded as /FOO /CMap findresource and the internal format can be dumped with === . | 20:15.43 |
| Robin_Watts | cool. I'll experiment a bit more with this binary compression thing, cos it sounds like that should help a lot. | 20:16.42 |
| ray_laptop | Robin_Watts: please go ahead and assign the bug to yourself, but I suspect that it might get into areas where you will need help from Alex, ken, chirs or I if you get into modifying the PS. Note that whatever we do, the PS code still has to be able to load CMap resources that are embedded in PS | 23:11.46 |
| Robin_Watts | ray_laptop: Which bug? | 23:12.11 |
| | Alex has said he'll look at dumping the internal format of the cmap stuff. | 23:12.28 |
| ray_laptop | Robin_Watts: the one for CMap compression (the rom size one) | 23:12.31 |
| | Robin_Watts: he mentioned that you can just use the === to dump the CMap format | 23:12.56 |
| Robin_Watts | I've played a bit with rejigging my ps compaction routines into a form where I can output binary postscript, and it seems to be going OK. | 23:13.39 |
| | Hopefully I'll have something working by the end of tomorrow, and we can all look to see if it's worth it then. | 23:14.29 |
| ray_laptop | Robin_Watts: the binary token PS ? | 23:14.58 |
| Robin_Watts | yeah. | 23:15.02 |
| ray_laptop | well, that'll get you part of the way there with CMap files, and maybe the compression will make the 142, 2 prefix compact to only a few bits | 23:16.04 |
| Robin_Watts | The binary compression will help on ALL postscript files, not just the cmap ones (or the internal dumped cmap ones) | 23:17.23 |
| ray_laptop | Robin_Watts: yes, I realize that, but the gain on the non-CMap gs_init.ps file probably won't be nearly as significant (just guessing) | 23:18.56 |
| Robin_Watts | ray_laptop: I would hope it would be much more significant. | 23:19.30 |
| | Almost all the whitespace drops out. | 23:19.45 |
| | Lots of the operators drop to 2 bytes. | 23:20.01 |
| ghostgum2 | good morning | 23:20.08 |
| Robin_Watts | morning. | 23:20.24 |
| | ghostgum2: I backed out that change, and put a note on the bug. | 23:20.42 |
| ghostgum2 | Thanks. I saw that. | 23:20.52 |
| Robin_Watts | numbers drop to be much smaller. hexstrings too. | 23:21.43 |
| ray_laptop | Robin_Watts: that is true for a lot of the 'encoded system names', and I guess there are a fair number of those -- but common ones like 'if' 'eq' 'roll' 'pop' and others are already short, and compression will "recognize" those sequences I expect | 23:22.28 |
| Robin_Watts | Anyhow, we'll hopefully see tomorrow :) | 23:22.29 |
| | ray_laptop: True. | 23:22.50 |
| | but on balance I'd still expect to win. | 23:23.32 |
| | And the binary ones should be faster to parse. | 23:23.42 |
| ray_laptop | Robin_Watts: yep. I look forward to it -- particularly the CMap results (where I have maintained all along the big reduction is possible). reducing the gs_init.ps from 150K compressed to much more still will be a drop in the bucket | 23:23.53 |
| Robin_Watts | yes, I realise the bulk of the space is in the cmaps etc. | 23:24.32 |
| ray_laptop | I haven't looked at the logs. Any input from kens on the ps2write issue ? | 23:24.38 |
| Robin_Watts | he said he saw my comments, and when he gets the textwrite stuff committed tomorrow, we're going to look at it. | 23:25.06 |
| ray_laptop | that's a real puzzler since opdfread.ps is independent of the %rom% file system | 23:25.17 |
| Robin_Watts | yeah. | 23:25.23 |
| | I can only think that maybe I'm hitting some code that's used in invoke opfread.ps or something ? | 23:25.51 |
| alexcher | We have 2 internal formats for CMap: CodeMap and .CodeMapData | 23:25.55 |
| | ray_laptop: Do we plan to allocate PostScripr CMap structures in foreign memory, aka rom ? | 23:29.21 |
| ray_laptop | alexcher: I don't see why we would. The VM space is not that bad and recoverable. The %orm% will be compressed, so it needs to be uncompressed when we load it -- no point in it being in memory all at once | 23:35.11 |
| | One thing that customer 711 may want to look at is putting (linking) the gsromfs.obj into flash so that it doesn't get uncompressed with the rest of the executable into RAM for running. | 23:36.47 |
| alexcher | Currently CMap is stored in %rom%. The loaded CMap is also stored in CodeMap and CodeMapData. So we've got 3 copies. | 23:38.06 |
| ray_laptop | depending on their hardware, some types of flash are too slow to execute from, so they "copy" the executable to RAM, but the romfs can be slow without impacting performance very much | 23:38.07 |
| | alexcher: oh no. that is quite bad -- I hadn't realized the font code was that ugly | 23:38.54 |
| | why CodeMap and CodeMapData ? (just curious) | 23:39.19 |
| henrys | ray_laptop:mvrhel2 said something about the small profiles being ready for the next release is that true? I'd like to change PCL to use them. | 23:40.13 |
| alexcher | Probably, CodeMap is just a step in data processing but it's now used by PDF interpreter. | 23:40.40 |
| Robin_Watts | beds. Night all | 23:40.47 |
| henrys | night Robin_Watts | 23:41.12 |
| ray_laptop | alexcher: and for the internal structures, how large are they (pick any really large CMap such as UniCNS-UTF32-H) | 23:41.21 |
| | alexcher: if it is a step in the processing, will the reference to it be 'lost' so it can be GC'ed ? | 23:42.40 |
| alexcher | ray_laptop: It's stores as /.CodeMapData attribute of CMap. | 23:44.19 |
| ray_laptop | alexcher: you said: "CodeMap is just a step in data processing" -- I was curious if the data generated for that step would get GC'ed | 23:46.08 |
| alexcher | ray_laptop: No, because /.CodeMapData is not undef'd | 23:47.18 |
| ray_laptop | alexcher: but if .CodeMapData is the same VM (just a different def) as CodeMap then we only have 2 copies, right ? | 23:48.37 |
| | the two copies being the CMap (as loaded by gs_cmap.ps) and the CodeMap == /.CodeMapData form (I am not counting the %rom% compressed form) | 23:50.14 |
| alexcher | ray_laptop: Probably, not. I see a lot of allocations and memcpy() functions in C. | 23:50.35 |
| | ray_laptop: Yes, 2 copies besides %rom% | 23:51.32 |
| ray_laptop | alexcher: I see. Well, I'm still curious as to the amount of VM (RAM) for an extreme CMap like UniCNS-UTF32-H | 23:51.50 |
| | and the %rom% raw and compressed size we will know when Robin_Watts finishes (from the mkromfs output) | 23:52.38 |
| | the internal VM sizes won't change with Robin_Watts' change to binary token | 23:53.17 |
| alexcher | ray_laptop: I'll get the data. | 23:53.31 |
| ray_laptop | alexcher: thanks. I have to run an errand. bbiaw | 23:53.49 |
| | Forward 1 day (to 2011/07/25)>>> | |
IRC Logs