Unpacking FFXi Files For Static Analysis
Preface
During my years of hacking Final Fantasy XI, one of the things I do the most with it is offline static analysis. Live debugging does have its advantages to see how the data is being used live as well as debugging for various other information and aspects of the code, however offline analysis is also a huge part in what I do for finding new things, checking up on updated / changed bits of code and so on. However, because of this, we must first unprotect the game files.
Square Enix has protected their game files with a fairly simple protection method. All contents of the files .text section are “encrypted” and stored into a new section called POL1. This is more of a deterrent for newbies working on analyzing their game files I would assume, but this protection is so simple, it is not hard to undo. Given that there is no methods to hide what they are doing, everything is done in plain sight, it’s very easy to fix the file back to normal.
In this tutorial I will cover their protection, how it works, and how to unpack it. Included at the end of this tutorial is a link to a new project I have made called ‘xiunpack’ which demonstrates what I show in this post to automate the unpacking and fixing process.
Analyzing The File
To begin, we can open up the file in CFF Explorer and check out the sections of the file. Here is what the protection looks like:
Any file that is protected with this method will have a new section added to the section table called POL1. The POL1 section is actually the .text sections data, but “encrypted” / “compressed”. I use these terms loosely because in a minute you will see how unprotected it really is. To get a grip on what this protection does, look at the raw size of the .text section. It is 0. This means the POL1 section has fully taken over the .text sections data.
Along with this, the files OEP is changed to point to a stub that unpacks the file when it is first loaded. Afterward, the file jumps back to the original OEP and continues its normal process after the unpacking has completed.
Here we can see what the stub looks like:
I have broken the new entry point stub into a few parts, so we can cover each of them individually.
Analyzing The File - The New Entrypoint Stub
So in the image above I have broken the new entry point stub into 3 parts. Lets cover each part in the following order:
- Green
- Red
- Blue
Green - Preparing For Unpacking
The first section we will discuss is highlighted in green. This is the first part of the new stub that is executed. What this part does is prepares the unpacker section (in red) for unpacking the POL1 data into the .text section.
Here is a quick rundown of what the ASM is doing in this section.
10B3054D . 45 INC EBP
10B3054E . 58 POP EAX
10B3054F . 45 INC EBP
10B30550 . 45 INC EBP
10B30551 > 60 PUSHAD
10B30552 . 89E5 MOV EBP,ESP
10B30554 . 83EC 08 SUB ESP,0x8
10B30557 . BE 00000010 MOV ESI,FFXiMain.10000000 <-- Load the files base into ESI..
10B3055C . BF 00000000 MOV EDI,0x0
10B30561 . 01FE ADD ESI,EDI
10B30563 . 6A 00 PUSH 0x0
10B30565 . B8 2E4C2F00 MOV EAX,0x2F4C2E <-- Load the real (virtual) size of the .text section into EAX..
10B3056A . 8945 FC MOV DWORD PTR SS:[EBP-0x4],EAX
10B3056D . 8D45 FC LEA EAX,DWORD PTR SS:[EBP-0x4]
10B30570 . 50 PUSH EAX <-- Push the real size..
10B30571 . 8D86 00100000 LEA EAX,DWORD PTR DS:[ESI+0x1000]
10B30577 . 50 PUSH EAX <-- Push the address of the .text section..
10B30578 . 68 40851C00 PUSH 0x1C8540 <-- Push the size of the packed data..
10B3057D . 8D86 00809600 LEA EAX,DWORD PTR DS:[ESI+0x968000]
10B30583 . 50 PUSH EAX <-- Push the virtual address of the POL1 section..
10B30584 . E8 52000000 CALL FFXiMain.10B305DB <-- Call the unpacker routine..
From here we can see the following data:
- The starting address to the .text section.
- The size of the data once its uncompressed. (The .text section size.)
- The starting address of the POL1 section.
- The size of the data in the POL1 section to unpack. (This is not really important though.)
At the end of the POL1 section you will see a block of memory separated from the data we are decoding. This is the stub entry to unpack the POL1 data.
Red - The Unpacker Routine
Next, the unpacker routine is called. This is where the magic happens for unpacking the data from the POL1 section back into the .text section. To mimic this routine I just simply ripped it from the game file and created a simple .asm file (fixed up) to accomplish this task, like this:
;*******************************************************************************
;* XiUnpack.asm (c) EliteMMO Network, 2014
;*
;* This program is free software: you can redistribute it and/or modify
;* it under the terms of the GNU General Public License as published by
;* the Free Software Foundation, either version 3 of the License, or
;* (at your option) any later version.
;*
;* This program is distributed in the hope that it will be useful,
;* but WITHOUT ANY WARRANTY; without even the implied warranty of
;* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;* GNU General Public License for more details.
;*
;* You should have received a copy of the GNU General Public License
;* along with this program. If not, see http://www.gnu.org/licenses/
;*******************************************************************************
.586
.model flat, C
option casemap :none
.code
;*******************************************************************************
; @brief Unpacks the given POL1 section.
;
; @param packed The packed POL1 section to unpack.
; @param unpacked The unpacked buffer to write the data to.
;*******************************************************************************
XiUnpack PROC
PUSHAD
MOV EBP, ESP
MOV ESI, DWORD PTR SS:[EBP + 024h] ; packed section
MOV EDI, DWORD PTR SS:[EBP + 028h] ; storage
jmp_6:
MOV ECX, 8
MOV BL, BYTE PTR DS:[ESI]
INC ESI
jmp_5:
SHL BL, 1
JNB jmp_1
MOV AL, BYTE PTR DS:[ESI]
MOV BYTE PTR DS:[EDI], AL
INC ESI
INC EDI
JMP jmp_2
jmp_1:
XOR EAX, EAX
MOV AL, BYTE PTR DS:[ESI]
INC ESI
MOV EDX, EAX
MOV AL, BYTE PTR DS:[ESI]
MOV AH, DL
AND EAX, 0FFFh
JE jmp_3
INC ESI
NEG EAX
SHR EDX, 4
ADD EDX, 3
jmp_4:
MOV BH, BYTE PTR DS:[EDI + EAX]
MOV BYTE PTR DS:[EDI], BH
INC EDI
DEC EDX
JNZ jmp_4
jmp_2:
LOOP jmp_5
JMP jmp_6
jmp_3:
POPAD
RETN
XiUnpack ENDP
END
I am not going to waste time on explaining each aspect of this part of the code, I’m sure by looking at it some can understand what is going on. There is simple simple byte shifting and such going on to rebuild the original data.
Blue - Cleanup And Launch
This is where the tricky part can come in. As I mentioned above in the green section, there is the possibility that the .reloc table can have some adjustments made as well. However, I have not seen this in any of the files I’ve unpacked. If I do come across it, I can make another tutorial explaining that information as well. However it is another simple packing routine which is again in plain sight inside of the blue section.
Here is a quick overview of this section:
10B30589 . 83C4 14 ADD ESP,0x14
10B3058C . 89FA MOV EDX,EDI
10B3058E . 09D2 OR EDX,EDX
10B30590 . 74 40 JE SHORT FFXiMain.10B305D2 <-- Check if we have .reloc fixes..
10B30592 . 8DBE 0010B300 LEA EDI,DWORD PTR DS:[ESI+0xB31000] \
10B30598 > 8B47 04 MOV EAX,DWORD PTR DS:[EDI+0x4] | Process the .reloc table if there is extra fixing to do..
10B3059B . 09C0 OR EAX,EAX |
10B3059D . 74 33 JE SHORT FFXiMain.10B305D2 | At this time I do nothing with this in my unpacker as I have not
10B3059F . B8 2E5C2F00 MOV EAX,0x2F5C2E | found a game file that actually uses this block.
10B305A4 . 2B07 SUB EAX,DWORD PTR DS:[EDI] |
10B305A6 . 76 2A JBE SHORT FFXiMain.10B305D2 | So rather then waste time implementing it for now, I just skip it.
10B305A8 . BB 08000000 MOV EBX,0x8 |
10B305AD > 66:8B041F MOV AX,WORD PTR DS:[EDI+EBX] |
10B305B1 . 66:09C0 OR AX,AX |
10B305B4 . 74 0A JE SHORT FFXiMain.10B305C0 |
10B305B6 . 25 FF0F0000 AND EAX,0xFFF |
10B305BB . 0307 ADD EAX,DWORD PTR DS:[EDI] |
10B305BD . 011406 ADD DWORD PTR DS:[ESI+EAX],EDX |
10B305C0 > 83C3 02 ADD EBX,0x2 |
10B305C3 . 3B5F 04 CMP EBX,DWORD PTR DS:[EDI+0x4] |
10B305C6 .^75 E5 JNZ SHORT FFXiMain.10B305AD |
10B305C8 . 8B47 04 MOV EAX,DWORD PTR DS:[EDI+0x4] |
10B305CB . 01C7 ADD EDI,EAX |
10B305CD .^E9 C6FFFFFF JMP FFXiMain.10B30598 /
10B305D2 > 83C4 08 ADD ESP,0x8
10B305D5 . 61 POPAD
10B305D6 >-E9 50487BFF JMP FFXiMain.102E4E2B <--- Jump to the real OEP..
The last jump in this section is our jump to the files original / real OEP inside of the .text section.
Rebuilding The Unpacked File
Now that we have all the information needed in order to unpack the POL1 section back into the .text section, and get to the original OEP we can safely unpack the file and rebuild it.
To start we must understand the PE file format. I am not going to go into detail about this because that is not the point of this tutorial. So if you are not familiar with the PE file format, I would recommend researching it. Some decent articles from the makers of the file type:
- https://msdn.microsoft.com/en-us/magazine/cc301805.aspx
- https://msdn.microsoft.com/en-us/library/ms809762.aspx
To rebuild the file we must do the following:
- Rebuild the DOS header.
- Rebuild the DOS stub (if it exists).
- Rebuild the NT headers.
- Rebuild the section entries.
- Rebuild the section data.
So to cover what my unpacker does in the order of operations, we have:
- Load the target file to unpack into local memory.
- Validate the file is a PE file.
- Validate the file is a packed file with the POL1 section.
- Obtain the .text section and information.
- Obtain the POL1 section and information.
- Invoke our XiUnpack routine to unpack the POL1 data.
- Begin Rebuilding The Unpacked File
- Write the original DOS header to the new unpacked file.
- Write the original DOS stub (if it exists) to the new unpacked file.
- Write the original NT headers to the new file.
- Process Each Section Of The File
- If the current section we are processing is the .text section:
- Set the raw data size to the virtual size of the .text section.
- Realign the raw data size based on the file section alignment value. (Required to make the new unpacked file a valid Win32 file!)
- Write the new section header to the unpacked file.
- If not the .text section:
- Realign the sections raw data pointer based on the new .text section size.
- Write the section entry to the unpacked file.
- Write each sections raw data to the file at the appropriate file offset.
- If the current section we are processing is the .text section:
- Adjust the files SizeOfImage inside of the NT headers with the last sections new information.
- Adjust the NT headers based on the new unpacked file data.
- Adjust the OEP based on the new unpacked file data. (See below for more information on this.)
- Write the new NT headers to the file overwriting the original we wrote earlier.
- Close the new unpacked file from writing and done!
Once we are at this point, our new file should look something like this:
Keep in mind this will change based on the file you are unpacking, the version, etc. it will not match this image exactly because the size of the sections will differ between versions!
As you can see here, our .text section is now properly sized and aligned to the file. Along with that, all of the following sections are realigned into the file appropriately because the .text section raw data now exists. We set the raw data offset for each following section based on the previous sections raw size and raw address. So for example, the .rdata section’s raw address is based on the .text sections raw size and raw address (rawAddress + rawSize). And we do the same for each section afterward too.
Aligning The Sections
As stated on the MSDN from Microsoft, ‘SizeOfRawData’ and ‘PointerToRawData’ MUST be section aligned. This is required to be a multiple of the files ‘FileAlignment’ value found within the NT headers optional header. This is not optional. In order for the PE to be considered valid and loadable these must be aligned. Aligning is very simple though. We simply round up to the nearest file alignment increment based on our sections info. Like this:
(((in + align - 1) / align) * align)
So for example, our .text section would look like this:
PointerToRawData = (((0x400 + 0x200 - 1) / 0x200) * 0x200); // would equal: 0x400 SizeOfRawData = (((0x2F4C2E + 0x200 - 1) / 0x200) * 0x200); // would equal: 0x2F4E00
Obtaining The Entrypoint
The last step is to obtain the real entry point to write to the file header so that when the file starts, we are at the real OEP instead of the packer OEP. In order to this we must recalculate the jump offset we saw in the blue section above at the end to match the proper file offset inside of the .text section. In my unpacker you will see the code for the OEP reconstruction like this:
// Set the new entry point to the .text section..
auto baseAddressOffset = *(DWORD*)(((DWORD)packed + polSection.Misc.VirtualSize) - 0x51);
auto baseAddressOriginal = ntHeaders.OptionalHeader.AddressOfEntryPoint;
ntHeaders.OptionalHeader.AddressOfEntryPoint = (baseAddressOffset + baseAddressOriginal) + 0x9B;
This is reading the jump offset from the file on disk. So lets take a few things into consideration: (These values are based on my current FFXiMain.dll! They may differ for you!)
- baseAddressOffset : 0xFF7B4850 (Remember, jumps are calculated offsets not direct addresses!)
- baseAddressOriginal : 0x00B30540
Now we have to adjust for the size of the code between the baseAddressOriginal and the baseAddressOffset. So if you look at the ASM image I posted above with the red, green and blue sections you can get a better understanding of what I am talking about here. From the very start of the code (CMP BYTE PTR SS:[ESP+0x8],0x1) to the jump to our OEP (JMP FFXiMain.102E4E2B) this is exactly 0x9B bytes. We need this range to properly calculate the new offset to the OEP.
So after we know these values we can do:
- realOEP = (baseAddressOffset + baseAddressOriginal) + 0x9B;
Remember, OEP’s are RVA’s. (Relative-Virtual Address) This means its relative to the images base address when loaded into memory. So for example, our original base address is: 0x00B30540, this means when the DLL is loaded into memory, the base address will become:
- FFXiMain.dll Base Address + 0x00B30540
Since this unpacker stub is static between all the files that contain this POL1 section, we can safely hard-code the offset 0x9B into our packer. If not, we could easily obtain the value from the file through some simple signature scanning and calculations.
Conclusion
And that’s a wrap! Your file should be fully unpacked and able to be analyzed in a disassembler / debugger of your choice!
There is 1 last optional step that you can take from this point. You can fully remove the POL1 section from the file as it is no longer needed now. Just be sure to realign the sections that follow it, if any! Once that’s done it should help reduce the file size a bit more. The file should be runnable as-is, meaning you should be able to drop it in replacing the original with the unpacked version and use it with the game. However it may trigger the update required process in the client if you are playing on retail servers!
Hope this was helpful for some! :)
Source Code
xiunpack was open source for a short while until certain people started stealing my work and putting their name on it. Due to this, I have decided to no longer share anything open source that deals with the FFXi community. If you are a research company or similar and wish to obtain a copy of the source, you can contact me at: atom0s [at] live [dot] com
Discord Bot
As of recent (2019), I have build a Discord bot that sits inside of the Ashita Discord server whom will automate this process for anyone. He supports all files that contain a POL1 section that is made for FFXi. (He does currently have a whitelist for which files are supported to prevent abuse.)
You can join the Ashita Discord server here: https://discord.gg/Ashita
Update 08-25-2020
Added information regarding the Discord bot I wrote a for years back to automate this after taking the source code offline. {: .notice–primary}
Comments