Blade

You should make a list of the various C&C music tracks and which discs (game or sound track) have them in CD quality for ripping.

Aside from the very different networking architecture, you are correct, SS is pretty much just a mod of C&C and in terms of content there is very little new apart from a few unit icons, create powerup animations and a new sound track. Most of the mix files are 1 to 1 duplicates of the C&C dos versions.

I've updated the mix database again with the themes that sole survivor has in its mix files. This dat also contains previous additions such as the chrono vortex files in RA as well.

At a guess, the obelisk has some kind of coords that tell it where to draw the line from that other objects don't so the line doesn't have a second set of coords to draw to/from and so doesn't get drawn?

I almost decided I wasn't going to dignify any of your posts with a response, but since you have no idea how scummvm is developed I feel I must. How do you think they created a scumm scripting language interpreter that behaves accurately compared to the original? Also, you have no idea how scummvm is structured, it doesn't just recreate the scummvm interpreter, it recreates the engines of a hugh swath of 2D point and click game engines, most of them reverse engineered from orignal binaries to behave accurately. This already includes many of the none RTS Westwood games and Blade Runner is being reverse engineered now as well. So you see, it is EXACTLY like reverse engineering a game like C&C apart from perhaps how much code there is to reverse.

Well, I'd argue it doesn't recreate any of those engines, its a novel engine that is skinned and scripted to imitate them. Of course you couldn't just merge those games, especially Dune 2000 which doesn't share much code with the others. However you could share code that makes sense to share such as IO. Scummvm manages to pretty accurately recreate many game engines that the team has taken apart over the years and has answered questions of the legality in several forum threads. I don't really understand how you can think its ethically questionable?

See, this is the problem, you think OpenRA is to RA what OpenTTD is to TTD and behave as such, but it isn't. OpenTTD reverse engineered the TTD binary and created a compilable version that pretty much matched the original and only then did they start to build upon it with mostly optional changes. It can to my knowledge still use the original data files as they are, any new formats are optional. If you had reverse engineered the original binaries and then built upon them I'm sure many of the qualms people are saying they have would not be an issue. As I've tried to explain, personally I have no problems with your project, I'm not hostile to it, I just think it is misrepresenting itself as a recreation when its really an imitation that uses the original GFX to make it seem more familiar than it really is. Also, I think you are being disingenuous suggesting this is some kind of "religious crusade" based purely on some kind of ideology like you are the victims of persecution for your strange minority beliefs. The people I see being most against the way you PORTRAY openra (and not against its existence) are the ones most versed with how the original games are actually programmed and who do have some inkling of what it would take to reverse engineer them to create something that could truly be called a recreation and the fact that isn't what the open RA team actually did. It doesn't help when your website suggests that the originals have been "improved" by "more modern RTS features" either.

I'm not strictly affiliated with CnCNet, but if I had access to the source code I'd probably start trying to port it/them to linux and OS X and modern windows (and maybe android) to get a nice cross platform base before worrying about anything else.

Don't worry, I have issues with that site's list as well, virtually all of those games should be under inspired by IMO Its the difference between an official translation of a text to a new language and a fan fic written the new language by someone who knows enough to get the gist of the original but not the nuance.

You must be new to the internet, so I'll clue you in... if the internet were a country, the national passtimes would be nit picking (or pedantry to be pedantic) and fapping to porn (not specifically tanya related, but I'm sure that makes up a part of it). If you aren't spending a good amount of your time on the internet doing these things you are doing it wrong For the record, I don't think the OpenRA guys have done a bad job at an RTS and I don't have anything against them or the project and any concerns I may have as to the accuracy of some of the claims on the website are minor in the grand scheme of things. However the thread was opened to get opinions and I gave them.

To make it simple for the end user to just use the game data that they currently have, to support the swath of existing custom content for the originals? Two good reasons right there. And right below that it states "These are not intended to be perfect copies, but instead combine the classic gameplay of the originals with modern improvements such as unit veterancy and the fog of war.". Then it isn't a recreation, its a re-imagining or imitation. You imply a disparity between recreating a "game" and an "engine", but I debate that this exists. The engine is the rules of the game and if you change the rules, you change the game. If you haven't recreated the rules accurately, you can't recreate the game. You can imitate it to some degree and my suggested rewording of the blurb IMO more accurately reflects that this is what OpenRA does. OpenRA does a good job of imitating the look where it tries to (unit and map graphics) and a reasonable attempt to capture the feel (which is far more subjective), but to say its a recreation is an exageration.

Can't openttd still use the original data files? OpenTTD was also originally a reverse engineered engine like opendune and behaved almost exactly like the original. The original was also only single player so people didn't have expectations of how it would play in a competitive setting so for multiplayer, the openttd team didn't have to worry about splitting opinion on its behaviour when implementing new features. Heck, OpenTTD is even more modest in its claims, saying its modelled after the original rather than a reimplementation which it actually is. The FAQ for OpenRA claims its a clean room implementation of the original WW engine, but it isn't because no one looked at how the original engine actually worked and documented it for the coders writing the reimplementation (which is what clean room reverse engineering actually means). I don't understand why the OpenRA project is marketed and branded like this at all, what is to be gained from mis-representing the project in what is essentially its marketing material?

We did, and the OpenRA name was originally coined back in those days. We then later found that they were too limiting for what we wanted to do, and replaced them with a more-expressive rules format (modelled after C&C Generals) and general Lua scripting. It would not possible to effectively support drop-in rules/assets/scripting simultaneously for the four game-reimplementations that we currently support. Expecting us to rename our established project because our goals changed would be like me insisting that you guys rename CnCNet to WWNet because you now also support D2K. This is silly. Why replace them rather than just leave them in place and have them parse in as overrides to your preferred configs and data formats? As Matt says, there is now a group writing automated tools to convert the original stuff into openra formats and that can't be too different from parsing them into the data structures used within the game itself, so why not just skip the middle step and have openra be able to load them in the first place, perhaps backed by you standard config format to set defaults and variables that are hard coded in the original? Plenty of projects rename when their focus changes and the original name doesn't reflect the project anymore, and there are plenty of others that don't, its not just a case of the name, its how some people involved with the project discuss it as though it was a recreation of the original WW games when it isn't, even if that was the unspoken intent when the project began. The website even states "OpenRA is a Libre/Free Real Time Strategy project that recreates the classic Command & Conquer titles." when it blatantly doesn't. It should say something along the lines of "OpenRA is a Libre/Free Real Time Strategy project that uses the graphical style of the old Command & Conquer titles." if who ever wrote it was being honest. OpenDune is an engine recreation, as are the many engines of ScummVM and for those of us who would like to see a real open source C&C recreation, seeing the OpenRA project proclaiming it already is is bit like a slap in the face.

A group at our development team started to write small tools to automatize the legacy rules import as it would have been a lot of manual work for all the Tiberian Sun assets. You can give https://github.com/Phrohdoh/ini2ora a try. It is in early experimental stages, but already asks for testers and feedback. Legacy tileset and map importers have already been integrated into the development branch. Automating conversion is not the same as a drop in replacement for the binary where you just install the original assets and go ala scummvm. pchote summed it up, the team is hobbyist game developers who started out thinking they could write a C&C clone just be observing the game behaviour and knowing a few of the file formats. When it became clear they would have to do some proper reverse engineering of the game to get close to its behaviour, they decided that they would have more fun writing things however they wanted rather than learning to read assembly and decompiling the code. That is fair enough, but I still think personally that they should have implemented parsers and functions for the existing game scripts though if they insist on calling it openRA rather than having to convert stuff. As an aside, there is a comment in the openRA mix handling code complaining how ugly and unfathomable the code to recover the blowfish key is. I can explain what it does if any openRA dev wants to rewrite it.

Yeah, it's a string in the exe, I'm surprised they went undocumented for so long.

So I figured out what the unknown filenames are in local.mix, they are all called hole####.lut. I've attached an updated global mix database.dat that contains the names (also contains my previous update with sole survivor file names too). I'm guessing these are some kind of precalculated lookup files for translating the pixels in the chrono vortex or something given the name. global_mix_database.dat.zip

You can extract from the mix files on the CD using XCC without agreeing to the EULA and installing the game. Seems you would be in the clear from a EULA perspective if you don't install it since you could never have agreed to it and being bound by its terms.

There would also be the slight issue of there being no map editor that understood the new map format, so you couldn't make maps with your new tiles easily.

Hey, if people want to go to the trouble of repaking the mix files just to avoid blowfish.dll, that is their business I admit that to some degree it is academic, but at least its doable now where it wasn't before.

Well, if XCC or the game can't open them I've not got the encryption right There are two points to this, 1. it means that you could use any standard crypto library to read encrypted mix files, XCC code has some issues building as 64bit with gcc when optimization is used so it allows it to be more cross platform. 2. it means that in theory we can generate encrypted headers rather than having to specify one from an existing mix file (I need to write an implementation to do it to test it first).

Okay, I believe I have an understanding now of exactly how decryption of a mix header works. The 80byte block in the header is handled as two 40 byte little endian BigIntegers that are decrypted by a raw RSA transform using the WW public key from keys.ini (after a few transforms) strangely the keys are stored as encoded big endian BigIntegers and are swapped to little endian in the XCC code (which is probably what the game does as well). For those familiar with RSA or capable of reading up on it, the public key in keys.ini is actually the modulus. You also need the public exponent which is 65537 (or 0x10001 if you are looking for it in the XCC code) to compose the full public key. The private key is the private exponent and is only needed to encode the blowfish key. After decryption you have two 40byte big ints that are concatenated together, but only 56 bytes are ever none zero and infact the XCC code just chops those off when it composes the final temp buffer to copy to the destination provided to the function. This is then set as the blowfish key and used to decrypt the header as described in the XCC documentation. Hopefully this will let me create a mix reader that doesn't rely on the XCC code and also create a mix creator that can generate its own encryption keys that the game will accept, rather than copying them from other mix files. The blowfish keys themselves are strange, they follow a pattern of two 28byte sequences composed of a short (3 - 11 bytes) random sequence padded to 28bytes with their last random byte.

Served!

I have another curiosity regarding C&C and RA, how do they render their graphics? I'm guessing in DOS they would pretty much have to blit to a frame buffer and then flip that to the VGA card and use their own primitives for clipping, line drawing and such. In windows do they do something similar and blit to a single direct draw surface using their own functions or do they make use of the direct draw primitives and functions instead? Also, do they redraw everything each frame or keep track of dirty regions that need to be redrawn?

That sounds about right, but the Sole Survivor bin files don't appear to record the 0xFF, 0x00 pairs in the file itself, preferring the record the coords of the none clear tiles only unlike the TD ones that record an entry for all cells and rely on the position in the file to dictate what coord the entry pertains to.

Looking at an actual file, Myg's explanation looks closest to what I see, short containing x*y pos, byte identifying tileset, byte identifying tile. struct MapPackStruct { short CellNumber; byte Template; byte TemplateTileIndex; }; This also explains why the files are different sizes, clear tiles aren't recorded.