Blade

You can reverse engineer a higher level language representation from it

XCC uses some inbuilt file "fingerprints" where it examines various bytes in a file that should have some known value in certain file types and uses that to decide if it knows what the file it. Its not perfect though and you are probably running into that issue with the wave files. It also doesn't recognise some TD/RA type shp files that come with the beta patch or SS even though they are valid shp files. Fact of the matter is that it seems you'll just have to wait for someone to write a new encoder that will handle all the wav files you want to use and that will encode a stereo file.

If its a proper engine reimplementation along the lines of OpenTTD or OpenDUNE I'm all for it, and I can't see how it would be any more legally questionable than these. Regarding defunct clone attempts, freecnc was the first, freera forked from it and as far as I can tell openredalert kind of borrows from both of them.

Well, are any of the VQA files from RA or TD stereo? That would suggest the games at least know how to deal with a stereo stream if they encounter one. Quite how Vladan Bato deduced that the header flags determine if a file is stereo or not presumably without having a stereo file would be interesting to know.

Because its so simple everyone assumes someone else must be doing it already I'll get round to it eventually I guess.

Bit off topic, but if you go manage to get in touch with ugordan, could you ask if he would consider open sourcing the VQA encoder? I'd like to have a go at making at making the command line version cross platform and if he used existing LCW code rather than writing his own I could probably improve the compression slightly too for a given quality. I've tried e-mailing him myself with no reply, but maybe he just doesn't trust the intentions of random people on the internet. Programatically you probably could get all the audio blocks from a VQA and stitch them together into an AUD file, but if you were going to go to that trouble, just write a new encoder as the AUD format is quite well understood and doesn't seem particularly complicated.

I have to disagree, properly reimplementing the old engines by reversing engineering the original code would potentially have several benefits. For one it would make it easier to remove many of the limitations and to maintain community patches than assembly hacking the binary. It would also make it easier to adapt it to changing graphics frameworks so fake ddraw dll files and the like would be unnecessary and it would be possible to port it to other operating systems making wine unnecessary as well. Finally it would also provide some preservation and documentation of the techniques that were used when writing games like these on resource limited systems working with constrained memory and CPU cycles.

Yeah, the LCW compression code is currently some cleaned up and slightly rewritten pseudo generated from the assembly compressor used to write the mappack sections in the game itself. I would like to do a full rewrite to clean it up further before releasing it.

Because of muscle memory and the fact that similar utilities have the send button around that location so in fact a lot of people won't read it before clicking. Do you honestly read everything you click when its similar to something you did before? Some user problems are because of teh stoopids, but some are down to human psychology and the fact its hard to do proper usability testing with no budget and volunteer coding and only user reports highlighting a possible usability issue get them fixed.

True, the implementation of the RLE command will have greatly improved compression over just the copy from source command, but its still very incomplete and IMO shouldn't be recommended in guides over XCC's compression as it currently stands. Here are the rules that the WW LCW compressor appears to follow when deciding which compression function to use: 1. Runs of the same colour should only use 4 byte RLE command if longer than 64 bytes. 2. Runs of less than 2 should just be stored as is with the one byte fill command. 3. Absolute runs greater than 10 or if the relative offset is greater than 4095 use an absolute copy. Less than 64 bytes uses 3 byte command, else it uses the 5 byte command. 4. If Absolute rule isn't met then copy from a relative offset with 2 byte command.

I can't believe you altered the artistic integrity of my post! Dune2 is pretty much the same format as far as I can tell except it doesn't have the X and Y pos to draw stored in the file so writing the files shouldn't be difficult. The OpenDUNE code suggests that the file just gets decoded to a screen buffer, pretty much exactly the same as how the RA code works, there is nothing regarding remapping in the file as read from the disk other than what is in the pixel data. The RedHorizon code handles the Delta field in the header incorrectly though, its treated as something related to frame rate when in fact its the size of the largest XORDelta encoded frame plus 33 (37 in RA). Its also handled as a dword when in fact its only a word and the word that follows it is a flag to determine if the file contains a palette and if the offsets should all be calculated +0x300 to compensate. The problems with remapping could be related to this somehow? Regarding PNG support. There is a small lib that I might look into for adding support in the future. These tools are side projects to a larger project with CCHyper as you may be aware so adding support for formats not immediately related to that project is low priority. In the mean time, I would suggest using XCC and an intermediate shp/png file system when preparing graphics and then exporting to pcx sequence and processing with this tool as the final step. I realise its not ideal and involves an intermediate tool and extra steps though.

As some of you may or may not know, support for the TD shp format used by TD, RA and SS generally sucks in all tools that implement it. Shp files support 2 different kinds of compression for each of their frames, LCW (more widely but incorrectly known as format80) and XORDelta (format40). Most tools take advantage of the fact that the game will quite happily use shp files that just use LCW to avoid implementing an XORDelta encoder leading to shp files that are much larger than they need to be. Worse still is that the LCW encoders are generally don't achieve the same compression as the WW algorithm did. XCC suffers from this and worse the OpenRA utilities don't even implement the full LCW compression scheme leading to the worst case "compression" on all frames (generally around 2x as large as the original pixel data). To remedy this situation I've written ShapeSet, this implements LCW the same as the WW algorithm and implements XORDelta which is slightly better than the WW original. It is command line only and currently only operates on a sequence of PCX files as might be generated by XCC, but if used with the -k switch it will do full LCW+XORDelta compression resulting in shp files at least as small if not smaller than the original WW encoded ones. Basically this is the tool you would want to use to generate your final shp files you ship to users. But wait, there is more. Since very few programs implement a working XORDelta encoder, there are few tools that can create wsa files. Since the format is similar to shp I've also thrown in WSASet, a program to create wsa files that are again slightly smaller than the WW encoded equivalents. Test away and let me know of any issues that crop up. I'm taking about bugs that break the encoders or that stop the generated files working correctly, not the fact that it won't do your palette conversions for you or doesn't take your favorite image format. shapeset.zip

I noticed that the scientist shp files in both TD and RA have a missing shadow on frame 25 (chan, moebius and einstein) so I've created fixed versions with a shadow copied from delphi. I've got both low and high res versions for the RA infantry and I recolored the lowres einstein to match his highres counterpart (I can do the normal moebius color version fixed on request I guess if anyone wants it). fixed_inf.zip

They are indeed contained within setup.Z which is an install shield 3 archive if my memory serves correctly. I wrote a tool for myself specifically to extract this on linux so it anyone wants some information on the file format I can provide it, but on windows universal unarchiver can extract it which uses install shield DLLs I believe.

From the looks of RA, only scores and movies1/2 and the enclosing container mix files (redalert.mix and main.mix) aren't loaded as cached so I'll go out on a limb and say only themes and movies in theory work from ss*.mix files.

I've been looking at the code of RA with a lot of assistance from CCHyper and I believe I understand the difference of how these files are loaded and why it appears that the SS prefixed files prevent the loading of files within them (assuming the system is the same in C&C). For many of the image file types (font, shp and such), the mix file must be cached entirely to memory as the loading system just asks for a pointer offset into the cached mix file rather than trying to actually read it from disk and the loading system does not fall back to allocating memory itself for uncached files. If the mix file that contains the file is not a cached file, then no pointer is returned and the game continues on as though the file wasn't present, but the locating system will think it found the file and look no further. The difference then is that SS files are loaded but not cached and SC files are loaded and cached. SS files should work fine for file types that are not retrieved as pointer but instead are opened as files. Which files are those you ask? Err, I'll get back to you on that one

D'oh, of course its 16 and not 10, its in hexidecimal, I clearly wasn't awake this morning. As I suspected, the number at the start of each line is the address for the first byte on the line. 10 in hex =16 in normal numbers (decimal). So just count inwards and take the first byte as having the number at the start of the line.

Is each row perhaps 10 bytes long in the display? Perhaps hacking isn't for you if you can't work out how to get to an exact offset in a hex editor.

None of the clones has really got near to being a real reimplementation of the C&C game engines, the C++ ones have all ceased development and the OpenRA guys have created their own game engine that happens to load the C&C graphic formats, but otherwise can't utilised the C&C game content (map and missions have to be converted and implemented as mod content for the engine rather than the engine being written around loading and playing the original content IIRC).

I'd worry about reverse engineering the game to the point it plays exactly like the original first, then think about the enhancements you can add (if any) beyond higher res visuals. Unless you already reversed it and are holding out on us

There is some red around the rocks next to the river in the second image that could probably do with being replaced with something a bit darker. Also, it looks like the ore images could do with some touching up around their edges to better blend to the desert.

I'm fairly certain that if you enable the Phase transport it gets the iron curtain noise no matter what mission its built in. Then again it has been a while since I messed around with it.

I've uploaded a further updated gmd, has all but one SS file name and most if not all missing RA filenames, at least from the english versions.

I wouldn't count on it, that is a naive guess. Maximums are all loaded during the game init to build object heaps, I wouldn't expect them to be re-initialised on map load.

I've got most of the file names that SS uses now, it just looks like some of the french and german UI elements have different filenames to the english ones, anyone know what the file names are so I can add them for completeness and I'll update the db again. I've also go all the filenames from wolapi.mix in the RA 3.03 patch as well that I'll add to the next update.