Dune 2000 map auto-download not working

[Shaokhan]

Hi,

 

when a host in Dune 2000 selects a map someone that joins doesn't have, in less than a minute it will disconnect the host from cncnet and cancel the game. I was told the code is written in PHP (which I'm getting more and more familiar with). The way I understand it the code very roughly looks or should look something like this:

 

===================================================

Client side code (cncnet):

$downloadfiles = missingmap.map (or missingmap.mis) + missingmap.ini

$MapNotPresent = check if selected map is present for user X (true/false);

if ($MapNotPresent = false) {

send query to server to request map download ($downloadfiles) from host to server;

once complete, create files for write access with file names ($downloadfiles) locally

receive data

close files

}

IF (confirmation info received = true) {

echo "Map missingmap was successfully downloaded";

show button to /restart

}

 

Server side code:

receive query to request map download from host ($downloadfiles);

create files for write access with file names ($downloadfiles)

download data from both files

upload files to player X

close files

send confirmation info to client side

===================================================

 

If I get provided with the right files, I think (and hope) I should be able to fix it.

[FunkyFr3sh]

No, the server recieves a .zip file with all 3 map files inside. You need to check each of these files and verify that it actually really is a map and not something else.

 

The .ini is plain text, should be easy to check.

 

The other 2 are binary, you actually need to understand the format

[Shaokhan]

Ok, so it seems the security is the main concern, then what about this solution:

do a check 1: is correct file extension?

if yes, do check 2: scan through virustotal

if ok, then download and check if the map file loads up correctly (i.e. loads the map preview)

if yes, we are all done, if not, tell user the files do not seem to be of correct type and ask user whether or not to delete the files.

 

Also if I understand correctly, then you can put nearly anything in a binary. So from what I read the PHP commands for checking file type and content do not seem to be very reliable security-like. Then perhaps the creator of the map editor is like the only person who would understand these dune map files in detail and could put in a check that would verify it is indeed a dune map file?

[FunkyFr3sh]

That's a really bad idea, it needs to be done properly

[Shaokhan]

Alright, mk is a Java developer, he can do it along with this website info about file format: http://old.d2kplus.com/wiki/index.php?title=Map_Files

But we need to know what is exactly missing / is wrong in the code. Do you just need the file check? As of now, we can go through map file (all tile indexes and special indexes) and validate them against valid index table. He did it in Java, but can easily rewrite in PHP.

[FunkyFr3sh]

about the .map file:

 

I would check if the filesize is correct. The first 2 bytes are width, next 2 bytes are height.

 

Verify filesize:

filesize = (width*height*4) + 4

 

Note: max map size is 128x128, it needs to filter anything bigger than that

 

 

Then I would make a loop to go through all cells, all tile values must be below 800 and special values below 1000

 

foreach (var cell in cells)

{

    if (cell.tile >= 800 || cell.special >= 1000)

        return false;

}

 

 

That should be enough

[Shaokhan]

This is what we came up with. Hopefully it will do it. But we could not really reliably tell what ranges the values are in .mis file, so the only check is the file size which should be constant for all .mis files. If you need more help, then we might need the php file on the server. Return value 1 passed check, value 0 didn't. Details about .mis file are here: http://old.d2kplus.com/wiki/index.php?title=Mission_Files

 

<?php

$name = "4play8";

$map = "$name.map";

$ini = "$name.ini";

$mis = "$name.mis";

$handle = fopen($map, "rb");

 

$height = unpack('s', fread($handle, 2))[1];

$width = unpack('s', fread($handle, 2))[1];

 

$returnVal = 1;

 

// Check if height is valid.

if ($height > 128)

{

$returnVal = 0;

}

 

// Check if width is valid.

if ($width > 128)

{

$returnVal = 0;

}

 

// Check if file size is valid.

if (($height * $width * 4) + 4 != filesize($map))

{

$returnVal = 0;

}

 

$cellCount = $height * $width;

 

if ($returnVal == 1)

{

// Check if all cells are valid.

for ($iter = 1; $iter<= $cellCount; $iter++)

{

$tile = unpack('s', fread($handle, 2))[1];

 

//Check tile index value of the cell.

if ($tile >= 800)

{

$returnVal = 0;

break;

}

 

$special = unpack('s', fread($handle, 2))[1];

 

//Check special index value of the cell.

if ($special >= 1000)

{

$returnVal = 0;

break;

}

}

}

 

// Check if the .ini file is of text type.

if (strcmp(mime_content_type('4play8.ini'),"text/plain") != 0) {

$returnVal = 0;

}

 

echo $returnVal;

 

// Check if .mis file size is 68066.

if (filesize($mis) != 68066 ) {

$returnVal = 0;

}

?>

[Shaokhan]

Anything else needs to be done?

[FunkyFr3sh]

I guess a check for the file size of the .ini would be cool too, should be smaller than 2kb.

 

With the .mis file I can't help, don't have a plan for it... Will be tricky to get it done, but just checking the filesize isn't enough lol

[Shaokhan]

(13) [23:49] <mk> About *.mis file we could go through specification and check whatever bits we can, I think we could do fairly good check.

(13) [23:49] <mk> But it would take time

(13) [23:49] <mk> It would be good to get some confirmation from administrator (funky), that after that he will do the rest to fix map sharing.

(13) [23:49] <mk> Or at least provide us with sources.

(13) [23:51] <mk> otherwise I wouldn't want to waste time on this (funky always responds unclear about project in general)

 

If it was up to me I would not even include maps that come with .mis files if it's such a big deal, and there would be less spam of maps. But other than that I understand mk's point of view. We spent quite some time on this and it doesn't seem you are very proactive about implementing it even though we are doing most of it. So, I don't know what the problem is, but I try to support the game as much as I can. And I think this is quite important for the game. If a game (or maps at least) cannot be updated, it dies (Westwood, Xwis, ...). Remember all the passion and stuff you did in the beginning for Dune on Cncnet? It was remarkable. Where has it gone?

 

It can't be difficult to find some sort of a solution. It's not natural for me to ask favors, I always prefer to do everything myself, but in this case, I don't have access to do it, so we are in this unfortunate situation. What I would like to see is under which conditions we can or cannot do this. Something like: "so if you guys do this, this and this, in this format and it will be good, I'll upload it and we are done" or "don't waste your time on it guys, it would require too much work from my side and I don't have enough time for it now" or just anything really that will tell us what the situation is so we know if we can do anything about it. Check for .ini file size is very simple.

[Iran]

Funky is probably just busy, he does A LOT also in the background. He's still patching Dune too in the private git repo.

[FunkyFr3sh]

I know it's a lot work, and that php stuff is only the half of it, I need to code the other part myself (but I know how to do it, it's not php)

 

I got a lot in my to do list, there are also 3 other games that need to keep running too. Next update will probably not have the map download working cause I have a lot things to finish first.

 

We can just keep the code like this for now and try it without .mis files and just add the rest later if everything works. At least we have something to use now, so it can be done (which wasn't even possible until now)

[Shaokhan]

Well guys, I can imagine someone like that is busy, that's why I wanted to help and requested server file so we can implement it and you have less work. So after couple hours we came up with 3 versions, which can be further polished if needed. Use whichever one you prefer.

MK's version - clean version - https://pastebin.com/S6iZQsZ5

According to MK this one is cleanest (least load on the server), for the file verification it uses data straight from the zip file. But when tested it gives errors in browser mode, but is ok in CLI mode. Mainly it doesn't like the zip file path. If it is tested and works, it should be best option.

 

My version - using temp files - https://pastebin.com/FKWZfB2t

This version is straightforward, it extracts the files from receiving zip into temporary folder, checks the files, then deletes temporary files and folder as well.

 

MK's version - using temp files - https://pastebin.com/h5H6SeHj

MK's version of the same.

 

I found a part of broken code that errors into "Map file checksum differs from Zip name, rejected." It is going to come out as this error every time, because it compares sha1 variable (which is the NAME of the zip file lowercased without extension) with an actual sha1 function that creates a HASH of all the 3 unpacked files. And hash will never equal the name of a file, so I in my version I removed it, MK commented it out in his.

Also the error "Zip file name not a valid hex value." shows up if the filename of the zip contains characters between g-z, looks strange, but maybe it's intended, so I don't know.

Btw, why no preview button on new forum? :-(

MK's version - clean version.php

MK's version - temp files.php

Shao's version - temp files.php

[Shaokhan]

The posts can't be edited now?

I have polished my version even more and tested all the checks. Removed the creation of tmp directory, there is now an exit function. Also added the 2kb check for ini files. Also the function will immediately delete all temp files and exit once any condition is false.

Shao's version 2 - temp files.php

[_mk]

Here are the latest versions of my source files. It has some improvements.
Also I don't know if mime_content_type function will work on game server. If not we will need to do that check differently.

https://www.dropbox.com/sh/woid3nfwt9ke4oz/AACT-8zNFt29hLGLynu2enZ8a?dl=0

[FunkyFr3sh]

Thanks to @Shaokhan and @_mk , we got the first test version now online.

 

Only .mis files are not supported right now. There are multiple problems with these files and it takes a lot of effort to get it done properly. 

We will probably have a workaround we can use until everything is done (only chosen players can upload .mis file maps)