Jump to content

CnCNet Forums

Search In
  • More options...
Find results that contain...
Find results in...

Introducing ShapeSet and WSASet.


Blade
 Share

Recommended Posts

As some of you may or may not know, support for the TD shp format used by TD, RA and SS generally sucks in all tools that implement it. Shp files support 2 different kinds of compression for each of their frames, LCW (more widely but incorrectly known as format80) and XORDelta (format40). Most tools take advantage of the fact that the game will quite happily use shp files that just use LCW to avoid implementing an XORDelta encoder leading to shp files that are much larger than they need to be. Worse still is that the LCW encoders are generally don't achieve the same compression as the WW algorithm did. XCC suffers from this and worse the OpenRA utilities don't even implement the full LCW compression scheme leading to the worst case "compression" on all frames (generally around 2x as large as the original pixel data).

 

To remedy this situation I've written ShapeSet, this implements LCW the same as the WW algorithm and implements XORDelta which is slightly better than the WW original. It is command line only and currently only operates on a sequence of PCX files as might be generated by XCC, but if used with the -k switch it will do full LCW+XORDelta compression resulting in shp files at least as small if not smaller than the original WW encoded ones. Basically this is the tool you would want to use to generate your final shp files you ship to users.

 

But wait, there is more. Since very few programs implement a working XORDelta encoder, there are few tools that can create wsa files. Since the format is similar to shp I've also thrown in WSASet, a program to create wsa files that are again slightly smaller than the WW encoded equivalents.

 

Test away and let me know of any issues that crop up. I'm taking about bugs that break the encoders or that stop the generated files working correctly, not the fact that it won't do your palette conversions for you or doesn't take your favorite image format.

shapeset.zip

Link to comment
Share on other sites

  • Replies 62
  • Created
  • Last Reply

Top Posters In This Topic

Oh, wow...

 

What about Dune II's WSAs? The existing encoder (of the Red horizons toolset) had trouble with the remapping colours in these files; probably a result of certain decoding techniques not going on to the remapping algorithm while others did.

 

[edit]

 

Test away and let me know of any issues that crop up. I'm taking about bugs that break the encoders or that stop the generated files working correctly, not the fact that it won't do your palette conversions for you or doesn't take your favorite image format.

I know you said that, but could you add png as input type, though? PCX is annoying since it needs an actual editor just to preview it in Windows.

 

(By the way, I edited your post for readability. Using double lines between paragraphs is a very easy fix for "wall of text" ;))

Link to comment
Share on other sites

I can't believe you altered the artistic integrity of my post! :P

 

Dune2 is pretty much the same format as far as I can tell except it doesn't have the X and Y pos to draw stored in the file so writing the files shouldn't be difficult. The OpenDUNE code suggests that the file just gets decoded to a screen buffer, pretty much exactly the same as how the RA code works, there is nothing regarding remapping in the file as read from the disk other than what is in the pixel data.

 

The RedHorizon code handles the Delta field in the header incorrectly though, its treated as something related to frame rate when in fact its the size of the largest XORDelta encoded frame plus 33 (37 in RA). Its also handled as a dword when in fact its only a word and the word that follows it is a flag to determine if the file contains a palette and if the offsets should all be calculated +0x300 to compensate. The problems with remapping could be related to this somehow?

 

Regarding PNG support. There is a small lib that I might look into for adding support in the future. These tools are side projects to a larger project with CCHyper as you may be aware so adding support for formats not immediately related to that project is low priority. In the mean time, I would suggest using XCC and an intermediate shp/png file system when preparing graphics and then exporting to pcx sequence and processing with this tool as the final step. I realise its not ideal and involves an intermediate tool and extra steps though.

Link to comment
Share on other sites

  • 3 weeks later...

True, the implementation of the RLE command will have greatly improved compression over just the copy from source command, but its still very incomplete and IMO shouldn't be recommended in guides over XCC's compression as it currently stands. Here are the rules that the WW LCW compressor appears to follow when deciding which compression function to use:

 

1. Runs of the same colour should only use 4 byte RLE command if longer than

    64 bytes.

2. Runs of less than 2 should just be stored as is with the one byte fill

    command.

3. Absolute runs greater than 10 or if the relative offset is greater than

    4095 use an absolute copy. Less than 64 bytes uses 3 byte command, else it

    uses the 5 byte command.

4. If Absolute rule isn't met then copy from a relative offset with 2 byte

    command.

Link to comment
Share on other sites

  • 3 weeks later...

Yeah, the LCW compression code is currently some cleaned up and slightly rewritten pseudo generated from the assembly compressor used to write the mappack sections in the game itself. I would like to do a full rewrite to clean it up further before releasing it.

Link to comment
Share on other sites

  • 3 months later...

Posting this here. Not sure it's relevant to your tool or it's my mod that's at fault so if it's in the wrong place, sorry! For Blade and Nyerguds' eyes primarily:

 

I managed to import the stretch palette, however it seems to not have worked 100%? I'm getting a lot of pink instead of green here:

mapglitch.png

 

I also get this crash the second time I open the map in the same execution of C&C:

maperror.png

 

EDIT: Just to throw a spanner in the works, just managed to get it to crash on the third time and not the second with references 004d8295 and 01018000 respectively.

 

It's quite possible that it's something I'm doing wrong. I can verify I have a custom palette in my pcx that doesn't seem to be recognised by the game, despite me having the -p switch on. Yes I have -l switch on as well.

 

Not sure what files you need, so attaching the pcx frame shown there, the wsa and the stretch pal.

hearth_a.zip

Link to comment
Share on other sites

Looks like the stretch table still isn't right to me.

 

BTW, just in case you didn't realise, the palette has to be the same across all the pcx files and it will only include the palette from the first file if -p is specified as a result.

 

Regarding the error, I think we might need Nyerguds to look up what function it happens in to be able to guess what went wrong.

Link to comment
Share on other sites

All I can tell you is that it's in a function that's called in a function that's called in something which I called "WSAAnimClass_GetFrame", though looking at the stuff inside it seems like it doesn't get the frame, but rather paints it on the graphics surface.

 

As for what goes wrong, no clue without debugging... Blegh. Debugging.

Link to comment
Share on other sites

I tried getting a screenshot with ddraw but it would not work. Must be an ingame only feature? Or the transition blocking keyboard presses.

 

RE: Nyer's palette generator. This is what I used but I am not sure if I hit ignores fades to 0. I can certainly try this tonight.

 

Thanks for looking into this everyone.

Link to comment
Share on other sites

I tried getting a screenshot with ddraw but it would not work. Must be an ingame only feature? Or the transition blocking keyboard presses.

No, I've made screenshots of WSAs with it. If you're on an OS higher than XP make sure your game runs in compatibility mode for Win95, and press ctrl+s ingame to make the screenshot. It doesn't copy to clipboard; it saves a png.

Link to comment
Share on other sites

Yeah I definitely did that... that's the method I use to take my normal screenshots and nothing seemed to appear in the directory. I'll double check that when I get home to make sure I'm not doing something stupid. My C&C machine is XP incidentally.

 

I did attach the source files on the previous post, although yeah I can see why having the ingame palette would be probably the most helpful thing.

Link to comment
Share on other sites

Notably, the original palette for that WSA does. My dark green is pink in the original file, index 41. That's the XCC Copy to clipboard of the original file.

 

palettewoes.png

 

I can also confirm I can't take a screenshot with Ctrl+S during the cinematic and I'm running ddraw. Just screenshotted some normal gameplay to be sure XD

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...