Endianness Fixes

I had some time this weekend where I wasn't at home able to work on ResidualVM. This provided an excellent opportunity to take a look at some of the issues that ResidualVM has on big-endian architectures like my PowerBook G4. I'm currently running Gentoo Linux on my Powerbook, and was able to play other OpenGL games without issue and Grim appears to work fairly well in ResidualVM. However, EMI is a mess!

The LucasArts intro has the wrong colors!

Even worse, the set wed has incorrect colors and alpha channel issues!

As can be seen above, the colors for many objects appear to be swapped from the intended colors. Let's investigate these issues, starting with the intro screen.

When debugging graphics issues like the intro screen, I often examine the colors between the working, little-endian version big-endian version to see if there the colors are easily identified as being swapped. Unfortunately, this wasn't immediately obvious as being the case. The colors were completely different:

No direct byte swap will make the right turn into the left...

However, I then checked and found that when loading bitmaps, two formats are present in ResidualVM, a packed 1555 format and a 32 bit format. With this information, it seems that the data was specifying an alpha channel, which was modifying the color components that were swapped. After some experimentation, I found that the native endianness is the preferred byte order for the Escape from Monkey Island binary format containing a bitmap image, a .til file. This worked well for 32-bit .til files, as found in the Windows version of EMI. However, the PS2 version uses 16-bit .til files. I found that it was easiest to convert the 16-bit input to a 32-bit output and doing the bit manipulation when the image was loaded.

With the .til files fixed and the intro displaying the correct colors, let's re-examine the set wed, to see if that was effected by the change:

That's a whole lot better!

While the output is now a whole lot closer, there's still a bunch of issues to be addressed. Let's focus on the background first. In the background, the enemy pirate ship is bouncing in the waves. This effect is actually achieved by an Actor object rotating and moving up and down on a timer. The Actor is drawn with a Sprite object, which are drawn using an image from a .tga file. After some inspection, I found that the desired behavior is to again read from the file in the native format. Another tweak was required to change the display format from BGR to RGB and this problem was solved. Here's the output now:

Getting closer!

Of note, this change also fixed the backgrounds for dialog boxes and other components that use .tga files. The next most obvious issue is that Guybrush and the brazier are both rendered as completely black with no color for the textures. I noticed that in the Act 1 title card, the actors were actually visible, although dim. This gave me the hunch that the lighting was to blame for the dark actors. To test this, I tried commenting out the lighting code when drawing the actor and found that the texture was drawn mostly correctly, albeit without lighting:

With lighting commented out

I then inspected the values used to dim Guybrush and found that the in the lighting object, the x, y, and z values were all NaN, causing the output to be black. As most endian issues appear when using data from the other endian systems, I looked at the loadBinary method for lighting and found that the floats used by the lighting were read in directly, rather than being interpreted by get_float() which performs the endian swap if needed. After adjusting both the Light and Shadow loadBinary methods, I got the correct output for the set wed as seen below:

Looks like it's all working correctly now!

The only outstanding issue right now on big-endian machines is that the 1555 ARGB color for the video frame isn't unpacked correctly. I suspect that an approach similar to how I fixed the 16 bit .tga files would work here as well, but I need to see if there's a better way to do this first. This work is in PR #974.