It all started so innocently. I’ve been away from the scene for a while so I was catching up on some chat history on one of the modding/gaming servers on Discord that I’m on and I stumbled across the following.
Well we can’t have that now, can we? I’ll take that as a sign for me to get back into things. I mean, how hard can it be? Only one way to find out, let’s dig in!
“C3” in this case Refers to “Comanche 3” from Nova Logic, released in 1997. It turns out that there are some tools out there that can unpack the format to varying degrees, but no re-packing tools, or real documentation on the format that I’ve been able to find. My goal is and always has been to support archiving, and modding, so documenting and a repackaging tool is essential. It also appears that “Armored Fist 2” (1997) uses the same format. Though I wasn’t aware of either of these facts when I first dug into the format. It should also be noted that “F-22 Lightning II” (1996) Appears to use this format as well, but a slightly different version of it, and we’ll look more into that one in future post.
First look
As is tradition with these ventures, I’ll often pull up the file in a hex viewer before looking into the format any further.
Hex View 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 52 45 53 4F 55 52 43 45 31 0D 0A 1A DD 04 00 00 RESOURCE1.......
00000010 E9 9B A0 E3 83 93 A4 FF AD DE ED AC F0 4D 00 00 .............M..
00000020 E9 9B A0 E3 83 93 A3 F9 AD DE ED AC B1 62 00 00 .............b..
00000030 E9 9B A0 E3 83 8C A8 EF AD DE ED AC 68 82 00 00 ............h...
00000040 E7 91 B4 FC E2 8C B9 82 E9 8C BB AC 6A AC 00 00 ............j...
00000050 E9 9B A0 E3 83 8E A2 FF AD DE ED AC 3E AE 00 00 ............>...
00000060 E9 9B A0 E3 83 97 A3 EA AD DE ED AC 82 D1 00 00 ................
00000070 E9 9B A0 E3 83 94 A4 E1 AD DE ED AC 7E D3 00 00 ............~...
00000080 E9 9B A0 E3 83 91 BF E8 AD DE ED AC 31 D4 00 00 ............1...
00000090 EE ED C3 E9 F5 9B ED AC AD DE ED AC 7B DA 00 00 ............{...
000000A0 F9 96 B8 E2 E6 F0 A6 E8 EC DE ED AC 3C EF 0E 00 ............<...
000000B0 E4 90 BE E5 E9 9B C3 FB EC 88 ED AC 6F EF 0E 00 ............o...
000000C0 FF 91 B9 E3 FF F0 A6 E8 EC DE ED AC 5D 70 0F 00 ............]p..
000000D0 FA 9F B9 E9 FF 9B B5 FC 83 89 AC FA 81 9A 0F 00 ................
000000E0 EC 92 A8 FE F9 F0 BA ED FB DE ED AC AD E2 0F 00 ................
000000F0 E8 86 BD E0 E2 9A A8 82 FA 9F BB AC EF 32 10 00 .............2..
"RESOURCE1": Identifier string?
0D 0A: CR+LF
1A: SUB (DOS end of text stream marker)
DD 04 00 00: 0x000004DD (1245) Count?
XX XX XX XX: Offsets?A few things jumped out at me right away. Firstly was obviously the plain text “RESOURCE1” identity signature. The next thing I noticed was that the records appear to be 16 bytes, due to the alignment of the values in last 4 bytes in each row. Those values are also steadily increasing in value, so quite possibly offsets. We don’t see any filenames, but that’s not unusual, some of these archive formats don’t contain filename data, or possibly that is stored elsewhere in the file. Hopefully we’ll be able to figure that out as we look deeper into things. But at a first glance this looks to be relatively straight forward, and this should be pretty simple to parse. In fact we could probably re-purpose our CAT file code to read this, with a few minor modifications.
Before we get into any code though, lets see if we can make sense of this data. It appears that the Identity string is terminated with a carriage return and line feed (CRLF) and then A SUB character, which will cause DOS to stop dumping to the screen if you were to use the TYPE command on the file from the command line. What follows looks to be a 32 bit value which has a value of 1245 in decimal, it’s possible this is an offset or something, but is also a high, but reasonable, value to represent an asset count. Let’s see if we can answer that by looking at the next one. If we assume that the first record comes immediately after, and those last 4 bytes are indeed offsets, and that the data starts immediately after the records, we can calculate how many records fit in that space, assuming 16 bytes per record, starting at an offset of 16. The first offset is 0x00004DF0 which is 19952 when converted to decimal. Subtract 16 off of that to account for the header, we get:
19936 / 16 = 1246
That is remarkably close to our initial value of 1245, so I think we’re onto something here. Let’s jump down to that place in the file and see what’s going on and if that can confirm things any further.
Hex View 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00004DB0 FA 96 AF 9C 99 F0 BD EF F5 DE ED AC B3 B2 D4 00 ................
00004DC0 FF 9A BF F8 FA 8C C3 ED E4 DE ED AC DD BC D4 00 ................
00004DD0 EE 91 A0 F8 FA 8C DD 82 EC 97 ED AC 82 BF D4 00 ................
00004DE0 4B 59 4C 45 4B 59 4C 45 4B 59 4C 45 1C C2 D4 00 KYLEKYLEKYLE....
00004DF0 3B 76 43 33 0D 0A 3B 4D 69 73 73 69 6F 6E 3A 20 ;vC3..;Mission:
00004E00 43 32 4D 38 0D 0A 3B 4F 77 6E 65 72 20 20 3A 20 C2M8..;Owner :
00004E10 4B 65 69 74 68 20 42 75 74 6C 65 72 0D 0A 3B 46 Keith Butler..;F
00004E20 69 6C 65 20 67 65 6E 65 72 61 74 65 64 20 62 79 ile generated byWe can clearly see a shift in the data pattern at 0x4DF0, so I think we can pretty safely assume that those final 4 bytes in the records are indeed offsets. Further to that we can see the last record that fits the pattern we saw earlier is at 0x4DD0, which would be record 1245, matching our presumed count, so I think we can call that one confirmed as well. I should note that all the data from the start of the file to this point seems to hold to the pattern we’ve been seeing. What remains looks to be a dummy-record of sorts, perhaps to indicate end of file. If this is in the same format as the other records, that suggests that those first 12 bytes should be ASCII data, but that’s not what we’re seeing. The associated offset for this record of 0x00D4C21C or 13943324 in decimal matches the byte count of the size of the file, so if taken as an offset this would point to the first byte after the contents of the file, or otherwise EOF. So this is clearly an End Of File marker. Now the reference to the name “Kyle” is interesting.
Kyle Freeman
The reference to “Kyle” in the EOF record is clearly to Kyle Freeman, who worked at Nova Logic. Kyle is credited with creating the, at the time revolutionary, SpaceVoxel engine that was first used with the first title in the Comanche series, “Comanche: Maximum Overkill” (1992). SpaceVoxel went on to be used with several other Nova Logic titles, including “Armored Fist” and “Comanche3” that set us on this adventure, though with the later likely a more evolved version of the engine. It was also pointed out to me, by a friend who I was discussing this with, that Kyle is credited as the lead for programming and design on Comanche3. So it is no surprise that his name would appear in the data somewhere as a bit of an Easter egg.
The song remains the same
At this point we probably have enough information to actually extract the contents, we just have no indicators of what the contents actually are, short of doing hard analysis on the asset data itself. We need to figure out what is in those remaining 12 bytes of each record to hopefully give us a hint. There’s got to be some useful info there, or it wouldn’t be there… besides I hate unknowns. Let’s go back and take another look and see if we can spot anything else with the data. Thirst other bit of useful data we might expect to find would be the size of the contents of the resource. We can calculate by subtracting the offsets of adjacent entries. If we subtract the first from the second we get 0x14C1, which we don’t see. So it doesn’t look like we have size. Also I given the time-frame when this file is from, I would expect the size to be stored as a 32bit value. and we aren’t seeing any 00‘s beyond those in the offsets. Let’s take a closer look.
Hex View 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 52 45 53 4F 55 52 43 45 31 0D 0A 1A DD 04 00 00 RESOURCE1.......
00000010 E9 9B A0 E3 83 93 A4 FF AD DE ED AC F0 4D 00 00 .............M..
00000020 E9 9B A0 E3 83 93 A3 F9 AD DE ED AC B1 62 00 00 .............b..
00000030 E9 9B A0 E3 83 8C A8 EF AD DE ED AC 68 82 00 00 ............h...
00000040 E7 91 B4 FC E2 8C B9 82 E9 8C BB AC 6A AC 00 00 ............j...
00000050 E9 9B A0 E3 83 8E A2 FF AD DE ED AC 3E AE 00 00 ............>...
00000060 E9 9B A0 E3 83 97 A3 EA AD DE ED AC 82 D1 00 00 ................
00000070 E9 9B A0 E3 83 94 A4 E1 AD DE ED AC 7E D3 00 00 ............~...
00000080 E9 9B A0 E3 83 91 BF E8 AD DE ED AC 31 D4 00 00 ............1...
00000090 EE ED C3 E9 F5 9B ED AC AD DE ED AC 7B DA 00 00 ............{...
000000A0 F9 96 B8 E2 E6 F0 A6 E8 EC DE ED AC 3C EF 0E 00 ............<...
000000B0 E4 90 BE E5 E9 9B C3 FB EC 88 ED AC 6F EF 0E 00 ............o...
000000C0 FF 91 B9 E3 FF F0 A6 E8 EC DE ED AC 5D 70 0F 00 ............]p..
000000D0 FA 9F B9 E9 FF 9B B5 FC 83 89 AC FA 81 9A 0F 00 ................
000000E0 EC 92 A8 FE F9 F0 BA ED FB DE ED AC AD E2 0F 00 ................
000000F0 E8 86 BD E0 E2 9A A8 82 FA 9F BB AC EF 32 10 00 .............2..And that’s when I noticed the apparently static pattern of data at the end of the 12 byte sequence. This pattern caries on throughout the entire list of entries, with varying lengths of presence. There are a few like the one at 0x00D0, that look like they could be skewed, but that could be just coincidence. Once again my that itch in the back of my brain is tingling. Could this be a simple XOR based obfuscation? If we assume ASCII text here, as the EOF record suggests (and the header BTW), that could explain the otherwise nonsensical looking data, and this pattern. If in the first record, those last 4 bytes are 00‘s, then what we would have here are the key, for those last 4 bytes. Furthermore when we look at the record at 0x0090, we see what appears to be a repeated of the last two key bytes, so perhaps the key is only 32 bits and repeated over the 12 bytes? Only one way to find out!
E9 XOR AD = 44 => ASCII 'D'
9B XOR DE = 45 => ASCII 'E'
A0 XOR ED = 4D => ASCII 'M'
E3 XOR AD = 4E => ASCII 'O'Well that can’t be a simple coincidence. Add to that that this particular .RES file is from the Comanche3 demo, and I think we have a winner. We now have what looks to be a full understanding of the structure of the file, at least for this top portion. As we don’t have a size for each resource, we will need to calculate it using the offset of the next resource, and conveniently, we have that EOF marker to ensure we always have the next offset to calculate with, even for the last record. All in all no real surprises, except for the name obfuscation. The structure is very similar to moat other aggregate archive file formats we’ve looked at so far. This is surprisingly the first time we’ve come across an obviously active effort to obfuscate the contents of the file, since we started this journey with the PIC file format. With that said, now it’s time to write some code!
The code
As always we start off with our code by defining the structures needed to decode the file. In this case we have up to three structures, the header, the resource entries, and the EOF entry. We could reduce this down to one, as they are all in the same form of 12 bytes of ASCII data, followed by a 32 bit integer. But for this case I think we’ll define two, one for the header, that encompasses the one with the resource entries. We’ll keep the EOF the same as a resource entry to make things a bit more simple. I’ve also taken a bit of a liberty here, with a peek to the future, and am breaking the identity signature in the header into parts, to make things more adaptable when we move onto the variant used with “F-22 Lightning II”.
#define OBFUSCATE_KEY 0xACEDDEAD // 32 bit XOR key used to hide the filenames
typedef struct {
union {
uint32_t raw32[3];
char name[12];
};
uint32_t offset;
} nl_resource_entry_t;
typedef struct {
char sig[8]; // "RESOURCE"
char ver; // "1"
char term[3]; // CRLF[SUB]
uint32_t count; // number of entries in the file
nl_resource_entry_t entries[];
} nl_resource_file_t;
First thing to note is that I converted the four XOR key bytes into a single 32bit value to make the XOR code a bit simpler/more efficient. That is also the reason for the union between the 12 char value for the name, and array of 3 uint32_t values. Next up is to read in the header and the entries themselves. Note for simplicity in this case I’ve elected to read the header into a statically allocated var, and will use a separate dynamically allocated buffer for the entries, instead of reallocating/resizing a single buffer for the header and entries.
nl_resource_file_t hdr;
int nr = fread(&hdr, sizeof(nl_resource_file_t), 1, fi);
if(1 != nr) {
printf("Error reading file\n");
ERREXIT(-1);
}
// sanity checks
if(strncmp("RESOURCE", hdr.sig, 8)) {
printf("Error: Invalid Header ID\n");
ERREXIT(-1);
}
if('1' != hdr.ver) {
printf("Error: Unknown Version\n");
ERREXIT(-1);
}
printf("Resource catalog contains %d entries\n", hdr.count);
nl_resource_entry_t *dir = NULL;
// allocate for one more than the specified count to include the EOF entry
if(NULL == (dir = calloc(hdr.count + 1, sizeof(nl_resource_v1_entry_t)))) {
printf("Error: unable to allocate memory\n");
ERREXIT(-1);
}
// read in all the entries, including the EOF marker
nr = fread(dir, sizeof(nl_resource_v1_entry_t), hdr.count + 1, fi);
if((hdr.count + 1) != nr) {
printf("Error reading file index\n");
ERREXIT(-1);
}
At this point we should have all the index data. Next step would be to de-obfuscate all of the names. No special sauce really, all we need to do is iterate over all the entries, and then XOR each of the “raw” 32 bit values with the key, to leave us with the readable ASCII text in the name string. We just have to keep in mind that this string is not necessarily null terminated, so we have to mind that whenever we go to use that value.
for(int i = 0; i < hdr.count; i++) {
for(int j = 0; j < 3; j++) {
dir[i].raw32[j] ^= OBFUSCATE_KEY;
}
uint32_t ent_size = dir[i+1].offset - dir[i].offset;
printf("%4d: %12.12s %8d [@%08x]\n", i+1, dir[i].name, ent_size, dir[i].offset);
}
Note that in this loop, when we calculate the size, we can safely access index+1, because we have the dummy EOF entry as the last item in the buffer that has it’s offset set to the EOF point. Our loop will terminate at the last valid index, which is one before the EOF record.
That’s pretty much it. Reading the contents of a given resource is as simple as seeking to the specified location in the offset for that entry, and then reading the calculated size number of bytes. Of course the proof is in the pudding, let’s see what we get when we list the contents of the file.
Version 1 resource catalog found
Resource catalog contains 1245 entries
1: DEMO.MIS 5313 [@00004df0]
2: DEMO.MNU 8119 [@000062b1]
3: DEMO.REC 10754 [@00008268]
4: JOYPORT.DRV 468 [@0000ac6a]
5: DEMO.POS 9028 [@0000ae3e]
6: DEMO.INF 508 [@0000d182]
7: DEMO.JIM 179 [@0000d37e]
8: DEMO.ORD 1610 [@0000d431]
9: C3.EXE 922817 [@0000da7b]
10: THUNK.KDA 51 [@000eef3c]
⋮ ⋮ ⋮ ⋮
1235: TWB01.PCX 7041 [@00d3f3b9]
1236: DTWB02.PCX 4988 [@00d40f3a]
1237: DTWBL02.PCX 2305 [@00d422b6]
1238: DTWBL01.PCX 3384 [@00d42bb7]
1239: HUT08.PCX 3770 [@00d438ef]
1240: WHB01.PCX 5808 [@00d447a9]
1241: WHB02.PCX 5577 [@00d45e59]
1242: WHB03.PCX 16017 [@00d47422]
1243: WHB04.PCX 2602 [@00d4b2b3]
1244: RDRTWR.AI 677 [@00d4bcdd]
1245: COMTWR0.AI 666 [@00d4bf82]The listing looks pretty reasonable, no weird characters or anything, so I’d call it a success a this point. All we need to do now is dump some of the resources and see if we can access them. Assuming the PCX files are true PCX images, they should be the easiest to test. So let’s take a quick peek to see if the data we have what looks to be a proper PCX file. Let’s take a look at one of the first PCX file entries in the listing at 0x001AED43.
Hex View 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
001AED40 00 00 00 0A 05 01 08 00 00 00 00 7F 02 DF 01 80 ................
001AED50 02 E0 01 20 53 68 61 64 6F 77 20 54 6F 6F 6C 73 ... Shadow Tools
001AED60 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
001AED70 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
001AED80 20 20 20 00 01 80 02 01 00 0C 00 0A 00 00 00 00 .............
Highlighted area = Presumed PCX headerIt looks to start off as a valid PCX file. The “Shadow Tools” causes me some pause, however if this is inside of the 16 colour PCX palette this may be okay, as that would be unused for a 256 colour image, the 256 colour palette is appended after the image data in a PCX file. Luckily with ImHex we can apply a pattern evaluator to the data to help clear things up.
Well that does look promising, and indeed the “Shadow Tools” part looks to be inside the 48 byte / 16 colour palette area. Look like we should be okay to extract at this point. If there are any variances in the data formats, it’s likely not related to the .RES container, but rather an implementation specific thing with the game engine. Nothing left to do but to extract them and try to load them up. Most of the images are small textures, so not all that interesting on their own, but there were a couple of full-screen images from the demo.
Some great looking title screens there. Typical fantastic artwork of the period. All that is left now is to be able to make or modify .RES files. That’s a relatively simple process at this point, but I’ll leave that for a future post. I think we’ve done enough here for one day.
So while that wraps things up for this one, for now. In my next post in this format we’ll delve into version 2 of the format, that can be found with “F-22 Lightning II”. It throws us a few curve balls that have us scratching our head for a few minutes anyway. Till next time, thanks for reading.
Ouch my eye! 
Leave a comment