Ouch my eye!

In a previous post, we left off having validated that all the parts we created for the pipeline required to decode a PIC file worked. While it was helpful in testing/debugging each part to have it as a stand-alone program, and it will be helpful again when we go on to write an encoder, the time has come to put all the pieces together into a single unit, but thankfully no actual Assembly will be required.

For the most part each of the 3 components we’ve made so far will operate as they did. One major change will be that the destination will change from a file, to a memory buffer during decompression. The fewest changes will happen to the LZW code, with the only major change there being to call the RLE code, instead of writing the output to a file. The RLE code for it’s part will lose the iterator loop, and simply expand codes as they are fed in from the LZW code on subsequent calls. So essentially only the rle_expand() function will be needed. Just like the LZW code, the RLE code will be modified to call some pixel unpacking code, which is basically our write_pixel() code, without the palette lookup. Then external to the decompression loop, we can convert the indexed colour image into whatever format we choose… though for now we will stick with PPM.

In order to facilitate chaining of all the code we will create a new state context that holds both the LZW and RLE contexts, plus our packed indicator and the format identifier byte. Now we could have extracted the contents of both rle_state_t and lw_state_t into this structure, but i like how this keeps things compartmented.

typedef struct {
    uint8_t format_identifier;
    bool isPacked;    // flag indicating 2 pixels per byte packing
    rle_state_t rle;  // rle state machine context
    lzw_state_t lzw;  // lzw state machine context
} pic_state_t;

As we are switching from a file context to a buffer context having a structure to hold our buffer and related vars will make passing it around a bit easier.

typedef struct {
    size_t len;     // size of the buffer in 'data'
    size_t pos;     // current position in the buffer
    uint8_t *data;  // the buffer itself
} bitstream_buf_t;

Next we will need a new function to wrap all of our decompression, that will take care of allocating space for the context, and properly initializing everything.

int pic_decompress(bitstream_buf_t *dst, FILE *src, bool isPacked) {
    pic_state_t *ctx;

    if((NULL == dst) || (NULL == src) || (NULL == dst->data)) {
        return PIC_NULL_POINTER;
    }

    if(NULL == (ctx = (pic_state_t *)calloc(sizeof(pic_state_t), 1))) {
        return PIC_NOMEM;
    }

    // init the contexts here
    lzw_init(&ctx->lzw);
    rle_init(&ctx->rle);
    ctx->isPacked = isPacked;
    ctx->format_identifier = fgetc(src); // read in the format byte

    // decompress the rest of the file into the buffer
    int rval = lzw_decompress(dst, src, ctx);
    free(ctx);
    return rval;
}

As the first call here is our LZW code, lets look at the changes there. There are some minor changes around working with the new context passed in, and of course taking the buffer pointer instead of a file pointer, but I’m going to skip those as they are fairly trivial. The main change is in the decompression loop section shown below.

        // decompression loop
        // loop while the chain pointer is >= the symbol chain buffer
        // 'decode' chain always returns a pointer that is within 'symbol_stack'
        int rval = PIC_NOERROR;
        while((PIC_NOERROR == rval) && (sp >= ctx->symbol_stack)) {
            rval = rle_decode(dst, *sp--, pic);
        }

        // decompression ended with an error
        if(PIC_NOERROR != rval) {
            return rval;
        }

As you can see, now instead of simply writing the bytes to a file, we are instead repeatedly calling the RLE decoder, and monitoring the return value for any potential downstream errors. This of course brings us to our RLE code. rle_decode is essentially rle_expand() from our standalone version. (I preferred the decode name). The only real change here again is that instead of writing to the output file, we are calling our pixel unpacking code, and returning any error that the downstream code may emit.

int rle_decode(bitstream_buf_t *dst, uint8_t symbol, pic_state_t *pic) {
    rle_state_t *ctx = &pic->rle;

    // first handle if last symbol was the RLE_TOKEN
    // if so, this symbol is the length value
    if(ctx->isToken) {
        ctx->isToken = false;
 
        // check for zero-length - special case to encode the RLE_TOKEN value
        // itself into the output datastream
        if(0 == symbol) {
            ctx->last = RLE_TOKEN;
            return pic_unpack(dst, ctx->last, pic);
        }
 
        // we have a run-length, write it out to the output
        int rval = PIC_NOERROR;
        while((PIC_NOERROR == rval) && (--symbol)) {
            rval = pic_unpack(dst, ctx->last, pic);
        }
        return rval;
    }
 
    // see if this is the RLE_TOKEN, if so sset state nothing more to do to 
    // do until next symbol arrives.
    if(RLE_TOKEN == symbol) {
        ctx->isToken = true;
        return PIC_NOERROR;
    }
 
    // finally just output the symbol as-is if not handled by now
    ctx->last = symbol;
    return pic_unpack(dst, symbol, pic);
}

Next step in our path is pic_unpack() this function is essentially write_pixel() from our standalone code, except decomposed into 2 parts, the handling if packed vs unpacked is done here, the actual pixel writing is in write_pixel().

int pic_unpack(bitstream_buf_t *dst, uint8_t px, pic_state_t *pic) {
    if(pic->isPacked) {   // packed mode, write 2 pixels
        write_pixel(dst, px & 0x0f); // blind write, we'll catch the overflow on the next write
        px >>= 4; // shift in the next index
        px &= 0x0f;
    }
    return write_pixel(dst, px);
}

Finally we get to the last stage of the decompression path, and that is our write_pixel() routine. In this case as we want our memory buffer to hold the indexed image, we do not need to look up the RGB values here, only write the index colour value to the memory buffer, and update the state accordingly.

int write_pixel(bitstream_buf_t *dst, uint8_t px) {
    if(dst->pos >= dst->len) {
        return PIC_BUFFER_OVERFLOW;
    }
    dst->data[dst->pos++] = px;
    return PIC_NOERROR;
}

That just about covers everything for the decompression stack. All that is left is updating our PPM save code to take the indexed colour image as input, and stripping out anything to do with packing, as that is all done now.

int save_ppm(FILE *dst, bitstream_buf_t *src, uint width, uint height) {
    // write the header for a binary PPM image file
    fprintf(dst, "P6\t%d\t%d\t%d\n", width, height, PAL_MAX);

    src->pos = 0; // reset our position to the start

    for(uint y = 0; y < height; y++) {
        for(uint x=0; x < width; x++) {

            uint8_t px = 0;
            if(src->pos < src->len) {
                px = src->data[src->pos++];
            } 

            if(1 != fwrite(&pal[px], 3, 1, dst)) {
                printf("Error: Unable to write to output\n");
                return -1;
            }
        }
    }
    return 0;
}

That’s all the changes combining our 3 standalone parts into one decompression stack, and a good place to wrap up this post.

PIC to PPM image converter
Resolution: 320 x 200
Using Packed Pixel Arrangement
Opening PIC Image: 'LABS.PIC'	File Size: 2241
Decompressed: 64000 bytes
Creating PPM Image: 'LABS.PPM'
Writing PPM Image
Image export completed without errors

This Post is the eighth in a series of posts surrounding my reverse engineering efforts of the PIC file format that MicroProse used with their games. Specifically F-15 Strike Eagle II (Though I plan to trace the format through other titles to see if and how it changes). To read my other posts on this topic you can use this link to my archive page for the PIC File Format which will contain all my posts to date on the subject.