Tiberian Dawn MRF Creation tool finished!

[Nyerguds]

I'll just repost this here, since PPMsite is a mess these days.

I have been making a tool to create .mrf files for C&C1 (RA seems to generate these tables in memory rather than keeping them as files). These files are basically palette remapping tables, used in the game to make colour effects like shadow, map shroud edges, stealth darkening (though not the actual effect), the targeting flash when giving attack / repair / dock commands, and a bunch more. Since they are relative to the theater's palette, they are a vital step to fully customizing terrain.

Download:
FilterCreator_v1.11.rar - The tool, plus all of the mrf creation projects. (requires .NET framework 3.5)
All icons used in the program (except the actual program icon) are based on the Fugue Icons 16x16 pack by Yusuke Kamiyamane (Copyright © 2010)

Test data:
FilterCreator_testdata.rar - All .mrf files in C&C, plus example screenshots to test on.

Source code:
FilterCreator_src_v1.11.rar (Visual Studio 2010 project)

Now, on to the tool itself:

The actual main tool's purpose is to recreate the existing filters in the game using brightness / contrast / colour balance / etc filters. However, since I already reconstructed all filters in the 11 .mrf files C&C1 needs for a theater, you can just go to [File] -> [Write multiple .MRF files...] and simply generate all the tables you need for a new theater.

All you need to do for that is selecting the folder with finished projects that comes with the tool, the palette of your new theater, and an output folder:

This sub-tool can also be started directly by starting FilterCreator from a shortcut with the command line parameter "-mrfui"

---

The current included projects are obviously just my own reconstructions. They were created to quickly allow people to make new theaters, but they're probably far from perfect.

So, if you want to try your hand at it, or if you simply want to view existing .mrf files, here's the overview of the UI functions of the main tool:

Right pane: the preview of the original .mrf file. It shows the filename of the loaded mrf file, and the amount of filters inside the file.

Left pane: the project. The filename of the loaded project is shown in the title bar of the program.

The controls get more specific as you go down on the left pane. Top is the project description... a completely optional field, really. Below that is the filters dropdown. Each filter is a "sub-project".

The three palette buttons allow you to manipulate the final palette in three ways. In all three cases, a yellow asterisk will indicate if anything is customized in the three options.

The first one, "Manual Remap", allows you to force certain colours to certain other colours before applying the filters. This can be used to make index tables as some mrf files have, or just to change certain colours before adding filters to them. Practically all existing filters seemed to have changed the animated water colours to white before applying their effect, so you probably need to use it for the same thing.

Notes:

• The specific indices set in this way are only preserved as that specific index if no effects are applied to it later.
• To set colours here that are not affected by the effects on the filter, you need to also set them in the third function, "Never remapped".

The second, "sub-palette", allows you to exclude colours from the used palette. This is usually used to exclude index #0 black from the used colours. Note that this can also be used in reverse, to select specifically the only colours to use for generating the palette. This is used for the unit shadow and some other filters like it, which use specific colours on the palette.

The third palette option is the "never remapped" list. This will allow you to select colours on the palette that will always remap to their own index (unless set to another index by option #1). These indices cannot be affected by the effects in the filter.

And, the last palette option, located at the top, next to the "enable filter" one, is the "Match back to palette" checkbox. If this is not enabled, you will see the filters as applied to the image, without taking into account the palette to remap back to. This also means it ignores the selected sub-palette, though force-remap and never-remapped entries are taken into account.

 

Under "Options", you can find an option to synchronize the filter between the left and right pane if possible, so that when you load the .mrf that corresponds to the project, it'll always show you the right filter from the file.

Then we have the effects. Available effects are Brightness, Contrast, Gamma correct, Colorize, Saturation change, Luminance change, and a combined Hue/Sat/Lum adjustment control. All effects you add to the list stack up to create the final filter. Since their order often makes a lot of difference, you can move effects up and down in the list.

Below the effects you see the actual effect sliders. The preview on the image is instant, though it might lag a little if you drag it around while "match back to palette" is enabled, because every small change will need to recalculate the remap table and apply it to the image.

That's it, I guess. The options in the menus should be pretty straightforward.

[Nyerguds]

Additional note: the best way to start a project is normally the "Create project from .mrf" option in the File menu. It'll allow you to select the .mrf to recreate in the project, and will automatically make all the filters, create the index table if it has one, and automate a bunch of the colour-excluding and subpalette-detection for you (if you select the options to do that).

[Nyerguds]

I finished all filters. They're included in the new v1.04. Making new theaters for C&C1 should be peanuts now

 

Mind you, I haven't actually tested the effects of my versions ingame. After all, they're only reconstructions of the original effects

[Nyerguds]

Filters in action: the edge shroud on the old TDX "pirates"  terrain experiment fixed.

(don't mind the trees and tiberium; they're obviously messed up because they were never adapted to the new palette )

 

Unfixed (using Temperate .mrf files)

 

Fixed, with .mrf files regenerated with my tool: (ignore the trees. The shroud edge is the thing that's important here)

[Nyerguds]

v1.5 released: Added the ability to use the program as command line tool to generate a .mrf file out of a .mrp project and a palette file.

 

Syntax:

FilterCreator.exe projfile.mrp -mrf palfile.pal mrffile.mrf

 

I also included a .bat file that allows bulk-producing all the .mrf files needed for a C&C1 theater from the included completed filters

[Nyerguds]

Right. Batch file was a dumb idea. v1.6 adds a UI for the bulk-create

This means that using the included finished projects, creating the necessary filters for a new theater becomes a very simple operation. In a few clicks, you got all the required .mrf files generated.

You can start MrfCreator in this export mode directly with the with the command line parameter "-mrfui".

[Blade]

For my own curiosity, have you documented the file format for .mrf somewhere?

[Nyerguds]

Not really. But, I can do it right here, if you want

The .mrf files are basically just remap tables; 256 bytes with each byte giving a replacement colour index for that original index, thus creating a filter's effect. It's the same system the game uses to change house colours on units and structures, only the house colour filters will map all colours back onto themselves, with the exception of the yellow range at C0-CF; the actual house colours.

Unlike my tool, which remaps the palette (because it's easier to show ), the game actually remaps the graphics, since 256-colour graphics only contain colour indices anyway. All it needs to do for the remapping process is going over these graphics pixel by pixel, and replacing the palette index value on that pixel by the new value at that index on the remap table.

The files containing multiple filters are a bit of a special case, though, in that they have a kind of index-to-filter mapping table at the start. Such a mapping table is, again, 256 bytes, but mostly filled with colour 255. Besides the bytes set to 255, though, it has specific indices set to 0, 1, 2 etc, to map these indices on the palette to the filters 0,1,2 etc contained further in the file.

For example, ?units.mrf maps colour 4 to the shadow effect (which is filter #0 in the file), that way. The shroud, in ?shadow.mrf, also uses this, to link its four filters (0, 1, 2, 3) to the indices 16, 15, 14 and 13 respectively. If you look at the shroud SHP file, you see it indeed uses these indices as its different shroud gradations in the fade to black.

But, that's it, really. That's the format. You can see the index-table thing pretty easily by using the tool to view one of the larger-than-256-byte .mrf files in the right-side pane, and clicking "View remap". The palette viewer I made shows mouseover labels showing which index each color is remapped to.

The format itself was never really an issue anyway; reproducing the existing filters on it was.

[Nyerguds]

Released v1.8:

-Fixed errors when using the filters' palette functions with no image loaded. This will now generate a dummy palette to show in the dialog.

-The "forced remap editor" now disables selection-related buttons if no colour indices are selected, to prevent getting a "paint selected" colour picker that won't actually do anything.

-The "clear selection" button is removed for the colour picker of the forced remap editor, since it only allow selecting a single colour anyway.

[Blade]

What kind of math does the engine use to get the final index to use for a pixel when using these tables? I've been doing a bit of reading on old school 8bit blitting engines and how they tried to avoid using ifs to decide if a pixel was transparent and instead used bitwise operations and lookup tables which is how I'm imagining these work.

[Nyerguds]

Ehh, as far as I know there's no math involved in remapping. It just relies on the fact that in 256-colour mode, each colour 'value' is really just an index on the palette, and thus a value from 0-255.

The table is simply a list with 256 entries, telling the game which colour to substitute, for each possible colour index. I imagine the game just goes over the pixels in the affected area, and for each pixel it takes the colour value (meaning, palette index), and uses that value as index in the remapping table to get the replacement colour value.

So basically, for the affected area, it just does

graphicsbuffer[pixelAddress] = remaptable[graphicsbuffer[pixelAddress]]

As for how it determines the "affected area", well, in the case of shadow and map shroud edges, that's what the index table before the filters in the .mrf file is for. It tells the game which of the .mrf file's filters to use for each colour index used on the .shp file.

[Messiah]

where can I get this pirate terrain?

[Nyerguds]

At the moment, you can't. This theater was originally meant to be part of the Tiberian Dawn Xtended project, but that never really went anywhere. It kind of died due to a combination of me not really continuing development on C&C95 after my last patch release, and the lack of method to create these .mrf files.

It'd be neat to get the Pirates theater finished and released, but, honestly, it's not up to me at this point, it's up to Reaperrr. It was his project. At the moment the terrain is still lacking a lot of stuff; none of the SHP files (tiberium, trees, civilian buildings) have been adapted, and even the actual terrain itself isn't 100% done; the river crossings and bridges simply don't exist, and have Temperate theater placeholders which look absolutely fugly in the new palette.

Reaperrr actually responded in the thread about this tool on PPM. Since the lack of method to generate these .mrf files was one of the factors that actually stopped the development of TDX, we figured we could probably at least finish this terrain now. I posted everything new I created for this test (including the .mrf files) in the old beta forum of TDX, so, we'll see what will come from that.

[Chimas]

*cof* not only finish it, but more important, including those that are finished in 1.06D *cof*cof* rsrsrsrs

[Nyerguds]

Nope. I will not add third party expansions to the official patch. This is exactly the reason I took the Spanish language out and made it a language pack. The Snow theater is a bit of an exception, on account of the fact it was originally made by Westwood, and just needed colour conversion.

Of course, if I ever release said 1.06d, it will undoubtedly have the theater and terrain object controls in ini files, making internal support completely irrelevant.

[Nyerguds]

v1.10 released:

• Added new option under Options: "Selecting effect focuses adjuster". This functionality has always been there, but this allows disabling it.
• Optimized ini reading of effects.
• Fixed the "Contrast" effect's "Exponential above zero" option.

[Nyerguds]

v1.11, with some small cosmetic changes:

• Fixed a bug where the checkbox statuses of the "remember images" and "synchronize" options in the menu were switched on startup. (I didn't notice this earlier because both are enabled by default anyway, heh)
• "Add Filter" list now contains filter names instead of internal class names.
• Internet links in "About" frame now only open on left mouse click.
• Icons added for all menu functions.