Compressed files

Compression is the process of encoding data in such a way that it consumes less storage capacity. This is done using special algorithms that identify and shrink down unnecessary and repeating data patterns using back-references. Some of Pokémon Ranger's files are compressed using the LZ10 algorithm, a derivative of LZSS that is used in many Game Boy Advance and Nintendo DS games.

In Pokémon Ranger, testing if a file is compressed with LZ10 is very simple. Whenever the game loads a file, it appends ".cat" to the file path and checks if that file exists. If it does, the file likely contains compressed data. Otherwise, it appends ".dat" (uncompressed data) to the path and tries to load that file instead. Additionally, the very first byte in the compressed data needs to be equal to 0x10. If these two conditions are met, the game will attempt to decompress the data.

Decompression

The following Java code shows the decompression process. getUInt8, getUInt24 and getSInt32 are functions to obtain integers from the specified position in the buffer.

public static byte[] decompress(byte[] in) {
	// Get decompressed size
	int outSize = getUInt24(in, 1);
	int posIn = 4;
	int posOut = 0;

	if (outSize == 0) {
		outSize = getSInt32(in, 4);
		posIn = 8;
	}

	byte[] out = new byte[outSize];

	// Decompress data
	while (posOut < outSize) {
		int flags = getUInt8(in, posIn++);
		
		for (int fi = 0 ; fi < 8 && posIn < in.length ; fi++) {
			if ((flags & (0x80 >> fi)) != 0) {
				int token = getUInt8(in, posIn++);
				int token2 = getUInt8(in, posIn++);
				int disp = ((token & 0xF) << 8) | token2;
				
				int lenCopy = (token >> 4) + 3;
				int offCopy = posOut - disp - 1;
				
				for (int i = 0 ; i < lenCopy ; i++)
					out[posOut++] = out[offCopy + i];
			}
			else
				out[posOut++] = in[posIn++];
		}
	}

	return out;
}

Compression

This section is work in progress. It may contain incomplete information and notes.