Hex editing UPK files

This is a Work In Progress (WIP), and not yet complete.

Not all elements of the content have been posted as yet. While the available elements should be functional, other elements are needed to achieve the end-result as described.

This message will be removed when the content is complete.

Introduction

This tutorial will attempt to cover every aspect regarding hex editing UPK files for XCom:Enemy Unknown/Enemy Within. It is assumed you are already familiar with the DefaultGameCore.ini file options and that you are aware what changes can be achieved by just editing the ini file and what changes need of hex editing. If you know that, even if you don't know what hex editing is, you're in the right place.

Rev: Mar 2015

Content

Section 2 Programs & Tools offers a brief description of the tools used for hex editing UPK files, along with a download link and installation tips.
Section 3 Hex editing I: changing single values is a miscellaneous of several procedures to make a simple edit in a hex file organized in the form of a tutorial. It is aimed at beginners but it may contain useful tips for advanced modders.
Section 4 Hex editing II: Changing a small part of the script and those following are dedicated to advanced hex editing; as such they are rather focused on getting as detailed and precise a set of examples as possible. You will have to thoroughly understand those examples to apply them to your desired modding situation.

Hex Editing Overview

Hex editing is what we need to do when we want to change something in XCom that we can't achieve by merely editing the DefaultGameConfig.ini file (DGC.ini from now on). It means accessing the .UPK files (Unreal Package files) where the game stores compiled classes and functions (stored as UnrealScript bytecode - similar to machine language - not a table of values like in DGC.ini, although to look at it's not much different), and edit them by means of at least two programs. Unlike with DGC.ini, when editing the UPK files we do it via its hex representation, instead of editing readable text.

Hexadecimal format
Hexadecimal (hex for short) is a numbering system where a single digit can take up to 16 values, counting from 0 to 9 and then from A to F, consecutively, so with a single digit we can express a value ranging from 0 (0) to 15 (F). Like decimal counting, when we want to express a value greater than what we can express in a digit, we use more digits!, so to express the number '16' in hex we would write '10' instead, and so on. Expressing hex values always requires writing them as two digits, so single digit hex values in code requires the leading zero, i.e. "00", "02", or "0F". See Hexadecimal on wikipedia for more info.

Why hex editing
As XCom does not directly provide a means of incorporating mods, primarily what we can do is change bytes of existing code. As that code in the executable and UPK files is in hex format, hex editing is the most efficient means to make those changes.

Programs & Tools

See reference Modding Tools - XCOM:EU 2012 for links. Here are short descriptions of the necessary tools.

UE Explorer

An Unreal Engine decompiler. For more information on decompilers in general, see this Decompiler Wikipedia article. This program lets you see the code almost as their creators wrote it, providing key information you will need to change UPK files, such as getting its hex representation, etc.

UPK Decompressor

We need it to de-compress UPK files so we can edit them.

PatcherGUI/PatchUPK

UPKUtils is a collection of simple command line applications for analyzing and patching uncompressed UPK files. PatcherGUI is a separate download (from the same UPKUtils Nexus "Files" page) that provides a Windows "Graphical User Interface" (aka "GUI") front-end to the PatchUPK command line tool. The PatcherGUI download includes all of the essential UPKUtils command line files as well, so only one download is required by players. Mod developers want the additional files in the full UPKUtils package, as well as the UPK Format Document.pdf file at the bottom of the "Files" page. Their primary documentation is the PatchUPK_Readme.txt file.

XSHAPE

Used to update the SHA values used for hash checks in the XcomGame.exe to run with modified UPK files. However, these values need to be updated every time a UPK file is modified.

UPDATE: XSHAPE is no longer required to modify the SHA values. It has been found that modifying some string values in the XComGame.exe file will prevent the executable from checking SHA values at all. It’s better to remove hash checks completely for EU than to use XSHAPE, and EW doesn’t use hash checks any longer. PatcherGUI can handle this for you. See the Basic Guide - Disabling Hash checks article for more information.

HxD HEX Editor

A general purpose hex editor. This is the program used to actually change the UPK files.

Notepad++

Light-weight text editor with good search functionality and other cool stuff that is pretty useful when hex editing.

UPK Extractor

This program lets you extract the files from uncompressed UPK files.

WinMerge

WinMerge lets you compare text for differences and such.

JPEXS Flash Decompiler

JPEXS is a useful tool for decompiling actionscript files, once they have been extracted from the UPK.

Hex editing I: changing single values

When first attempting to hex edit a UPK file, it is strongly recommended to start by replacing a single value for another value of the same type (i.e. replacing one number with another). This chapter will try cover every step necessary to achieve this; more advanced changes to the UPK files will be discussed in later chapters.

A word of advice

Hex editing your UPK files is an easy way to screw up your game, so a few things you should know before jumping into it:

Making an incorrect change can cause the game to crash on start-up, and it won't show any error message that could tell what is wrong.
Making an incorrect change can also cause the game to crash when the changed code is executed. So...
Make back-up files before any change.
Document every change made so if one change is proven wrong it can be easily reverted.

Failing to do this, and continuing to blindly change hexes without direction will eventually lead the game to a point of no return - where the crash-causing error can no longer be located and the only way to fix it will be re-installing the game and thus loosing the changes made so far. There's no need to reach so far; just keep organized and save yourself a few headaches.

Preparing the ground

Disable hash check (EU only).
Backup XComGame.UPK, XComGame.upk.uncompressed_size, XComStrategyGame.UPK and XComStrategyGame.upk.uncompressed_size.
Unpack XComGame.UPK and XComStrategyGame.UPK with decompress.exe.
Remove XComGame.upk.uncompressed_size and XComStrategyGame.upk.uncompressed_size from the game’s folder.

If you have EW installed, be sure to check in which game folder (EU or EW) you are working. See the Basic_Guide_to_installing_mods#The_Loose_Files Basic Guide - Loose Files article for these folder paths.

Uncompress UPK files.
Most mods, if not all, edit these two UPK files: XComGame.UPK and XComStrategyGame.UPK. So the first recommendation is to decompress both of them using UPK Decompressor, then make a back-up of both uncompressed files and store them in a safe place (a backups folder is always useful).
Export UPK files.
This will allow you to search through all the UPK's content using Notepad++. It is an incredibly powerful tool so it is worth using. To export the UPK files: first open them in UE Explorer; then, for each file, go to the Tools menu >> Exporting >> Export Classes. (Note: there is a second "Tools" menu below the standard Windows Interface)
Exporting classes with UEE is optional, as UEE has “search in classes” functionality which allows to search through the decompiled code easily.

Browsing UPK files with UE Explorer

Browsing files
When attempting to make a change to the game we need to know which function we want to edit. More specifically, we need to know the package or file (XComGame.UPK or XComStrategyGame.UPK most likely), then we need to know the class which has the function we are after. Just knowing the file won't help us much unless we are given a unique hex string (see below). Once we find the data we want to change, open the file with UE Explorer, select the "Objects" tab on the left (see picture below, step 1), and we'll see an unsorted list of classes within that file. Write anything on the search field below to filter results (2), and erase after writing anything to make it sorts the list alphabetically. Once you locate the desired class, click on the "+" icon to expand the class and then to expand functions (3). Clicking on any of them will display it's code on the right screen (4).

[Image by Amineri, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

However, code can't be changed in UE Explorer - it is a decompiler, not a compiler. We need to get its hex representation and edit the UPK file with the hex editor. Not only do we need to know the hex representation for the numbers we want to change (that could be deduced without the need to look at the code) but we need to get them "in context". This way, when we are actually editing the UPK file we do it with absolute certainty that what we are changing is what we want to change and nothing else.

Searching for the code

Classes, variables and functions generally have sensible names, so you can often find what you want by summarizing it with one word and searching for that word. But don’t expect to find it in one pass: sometimes the chain of variables and function calls can be quite long.

Generally, everything that happens in the game strategy layer is defined inside XComStrategyGame.UPK and the tactical layer is defined inside XComGame.UPK. But there are a lot of widely used base classes and functions, specific to XCOM, which are defined inside XComGame.UPK but called from inside of both XComGame.UPK and XComStrategyGame.UPK classes.

Let’s try to find, for example, how soldier gender for a newly created soldier is determined.

Select your best guess as to which UPK file is most likely to deal with creating soldiers. (Hint: In which layer of the game do you create things?)
Open that unpacked UPK file and use “Find in Classes” functionality to search for the word “Gender”.

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

The logical place to look is XComStrategyGame.UPK, as new soldiers appear on the strategy layer, but if we search through it we’ll find only pre-determined demo and reward soldiers with gender assigned specifically and not randomly. But searching inside XComGame.UPK immediately gives an interesting result:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

With such a name XGCharacterGenerator class must be the one we need!
Expanding XGCharacterGenerator find results reveals a long list of references and one of them looks like the one we need.

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

So, as a result of our search we can say that the code for assigning a random gender looks like:


kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1);

And that this code is located inside the XGCharacterGenerator.CreateTSoldier function.

Now we need to find it inside XComGame.UPK file as a bytecode.

You can now enter the name of the function into the search field to see only this function’s code:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Select “View Tokens” option from “View” menu to see decompiled tokens along with their hex codes:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Now open the uncompressed XComGame.UPK file in your hex editor (i.e. HxD), copy all the hex code in between the brackets and try searching for it:

HxD editor 'Search' for hex sequence example

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

We found the code:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

And this code seems to be unique (which is very important!) as continuing the search (F3 hotkey) gives us this message:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Note: Always make sure the code you’re searching for is unique; as if it’s not, you might end up changing something completely different and break your game.

Understanding the numbers

Now let’s try to change the females to males gender ratio of new soldiers in our barracks as a specific example of this process.

The decompiled code says that "if Rand(2) is equal to zero, iGender variable is equal to 2; otherwise it’s equal to 1". So, the gender value can be equal to 1 or 2. That’s good to know, but what do these values stand for? Let’s get back to our UEE search results again:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Here we can see EGender enum, which is an enumerated data type with four possible values. By default, those values are enumerated starting from 0 with a step (increase) by 1, so 1 stands for eGender_male and 2 stands for eGender_Female.

Function Rand(N) generates uniformly distributed random numbers between 0 and N-1, so Rand(2) will generate 0 and 1 with equal probability. All this tells us that males and females have equal probability. This also tells us that is we change Rand(2) to Rand(4) for example, we’ll get fewer females, as "Rand(4) == 0" happens only 1 time in 4. So, by changing Rand(2) to Rand(4) we’ll get ¼=0.25 female’s probability out of 1, or 25%.

Now we know what we need to change in the code, but how to find Rand(2) in that long decompiled string we see?

We return to our XGCharacterGenerator.CreateTSoldier function and use another cool UEE feature: ”View Disassembled Tokens”.

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Use the “Find” field on the right to search for Rand(2) and skip to the line where Rand(2) is present alone.
And we find "A7 2C 02 16" is the bytecode for Rand(2). A7 is the Rand function "token", 2C is byte integer "token", 02 is the value and 16 is the end function parameters "token".

To replace 2 in Rand(2) with 4 we need to replace 02 with 04 (single digit hex values require the leading zero).

Applying the changes

First, let’s return to HxD, which you will recall we used to open the uncompressed XComGame.UPK file, and search for the "tokens view" hex string:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Now that we know what to look for we can easily locate the code we need to change. Point the mouse to 02 and click once, then type 04:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Save your changes.

Note: you might need to close the UPK file in UEE, or even close UEE itself, for HxD to be able to save the file.

Re-open your UPK file in UEE and check the XGCharacterGenerator.CreateTSoldier function. You should see this:


kSoldier.kAppearance.iGender = ((Rand(4) == 0) ? 2 : 1);

If so: Congratulations! You’ve made your first XCOM mod!

Keeping notes

While our mod wasn’t so big, if we were to repeat our changes we’d need to repeat all the searching steps again. To avoid this use Notepad++ to keep your notes as some sort of “mod files”. If you plan to publish your mod, organizing your notes into one of the formats used by PatcherGUI/PatchUPK (hereafter Patcher refers to either tool) makes this easier:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

You can now easily repeat the change by searching for the BEFORE hex and replacing it with AFTER hex while it is selected in HxD. Copy the AFTER hex and use “Paste write” (Ctrl+B) option:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

The final result should look like this:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Constructing BEFORE and AFTER hex

Make a copy of your code and delete all the comments to create a “mod file” with BEFORE and AFTER hex:


BEFORE:
if(eForceGender != 0)
kSoldier.kAppearance.iGender = eForceGender
goto J0x202
kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1)
07 C3 01 9B 38 3A 00 EC B9 00 00 38 3A 24 00 16 0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 38 3A 00 EC B9 00 00 06 02 02 0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9A A7 2C 02 16 25 16 02 00 2C 02 01 00 26
AFTER:
kSoldier.kAppearance.iGender = ((eForceGender != 0) ? eForceGender : ((Rand(100) < 30) ? 2 : 1))
+filler code
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9B 38 3A 00 EC B9 00 00 25 16 0B 00 38 3A 00 EC B9 00 00 10 00 45 96 A7 2C 64 16 2C 1E 16 02 00 2C 02 01 00 26 00 EC B9 00 00 00 EC B9 00 00 00 EC B9 00 00 00 EC B9 00 00 00 EC B9 00 00 0B 0B 0B 0B 0B 0B

Now you can use this code to apply modifications with HxD. Search on the BEFORE hex code, and replace it with the AFTER hex code. Leave out any of the "human readable" lines. To change a female’s probability replace 1E (30 dec) in A7 2C 64 16 2C 1E 16 above with the hex representation of the desired number between 0 (00) and 100 (64).

Using modding tools

As a next step you can convert your “mod notes file” to a PatcherGUI/PatchUPK mod file format:


UPK_FILE=XComGame.upk
OBJECT=XGCharacterGenerator.CreateTSoldier
[BEFORE_HEX]
07 C3 01 9B 38 3A 00 EC B9 00 00 38 3A 24 00 16
//if(eForceGender != 0)
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 38 3A 00 EC B9 00 00
//kSoldier.kAppearance.iGender = eForceGender
06 02 02
//goto J0x202
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9A A7 2C 02 16 25 16 02 00 2C 02 01 00 26
//kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1)
[AFTER_HEX]
//kSoldier.kAppearance.iGender = ((eForceGender != 0) ? eForceGender : ((Rand(100) < 30) ? 2 : 1))
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9B 38 3A 00 EC B9 00 00 25 16 0B 00 38 3A 00 EC B9 00 00 10 00 45 96 A7 2C 64 16 2C 1E 16 02 00 2C 02 01 00 26 
//filler code
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
0B 0B 0B 0B 0B 0B

This small script essentially does the same we just did manually: opens XComGame.UPK file, searches for XGCharacterGenerator.CreateTSoldier function, then searches for the specified hex code inside it (BEFORE hex) and replaces the BEFORE hex with the AFTER hex. Note that Patcher searches for the code only inside the XGCharacterGenerator.CreateTSoldier function, so we have to make our BEFORE hex unique only to that function and not to the entire UPK file.

Hex editing II: Changing a small part of the script

Understanding script’s file and memory sizes

Note: for better understanding of Function and UPK data formats in general, read this document.

A length of the script’s bytecode as it is seen in the UPK file is what we call the script’s file size. And the length of the script’s code when it is loaded into memory is what we call the script’s memory (or virtual) size. THE TWO ARE NOT NECESSARILY THE SAME SIZE!

Most of the tokens have file and memory sizes identical to each other. But there is a special token we call Object Reference, which has a memory size bigger than the file size. Since one script can contain dozens of such references, a script’s memory size is generally larger than its file size.

Important note: using incorrect file and memory sizes lead to an immediate "Crash To Desktop" (CTD) right after the game starts. So if you see this happening after you’ve made some modification to UPK files, you’ve probably messed up those values (both or just one of them).

Although it is possible to change both memory and file sizes, right now we stick to the case where we need to make our modifications fit the existing constraints of file and memory sizes. It can be useful in the case of a big function if we need to change just a small portion of it without affecting anything else.

Planning the replacement

Let’s return to our gender probability mod and try to rework it so it allows us to set the female probability value between 0 and 100%.

The relevant part of the script looks like this:


kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1);

If we could make it look like this, we would get any probability we want:


kSoldier.kAppearance.iGender = ((Rand(100) < FemaleProb) ? 2 : 1);

The FemaleProb variable here is set to the desired probability number, like 5 or 45.

At first it looks easy: that we only need to change two numbers and a token. But let’s look at how things are on the hex side.

Open XComGame.UPK with UEE and search for the XGCharacterGenerator.CreateTSoldier function; then go into "Tokens View". The line we need to edit looks like this:


(1C3/127) [0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9A A7 2C 02 16 25 16 02 00 2C 02 01 00 26]
   L(63/43) -> SM(47/27) -> SM(28/16) -> LV(9/5) -> C(15/15) -> NF(7/7) -> NF(4/4) -> ICB(2/2) -> EFP(1/1) -> IZ(1/1) -> EFP(1/1) -> ICB(2/2) -> IO(1/1)
   kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1)

Using "Disassembled Tokens View" you can split this line into a series of tokens to understand how it works:


0F // let token
35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 // kSoldier.kAppearance.iGender expression
45 // ternary condition token
9A // equal-equal token
A7 2C 02 16 // Rand(2) expression
25 // zero token
16 // end of equal-equal token
02 00 // skip size
2C 02 // number 2
01 00 // skip size
26 // one token

To make Rand(2) into Rand(100) we need to change 2C 02 in A7 2C 02 16 to 2C 64. This is a simple single value change and we can do it easily. But to change the "0" in the ((Rand(2) == 0) equal-equal expression to some other value, say: 30, we need to change the 25 (zero token) to 2C 1E (byte integer token + value). This means we need one more byte for our new code, which changes both the memory and file sizes.

Now let’s take a look at the surrounding code:


   if(eForceGender != 0)
   {
       kSoldier.kAppearance.iGender = eForceGender;
   }
   else
   {
       kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1);
   }

We have an opportunity here! If we change this code into a more compact version we’ll save some space:


kSoldier.kAppearance.iGender = ((eForceGender != 0) ? eForceGender : ((Rand(100) < 30) ? 2 : 1))

Let’s look inside the corresponding bytecode. Use "Tokens View" to obtain it:


(171/0F1) [07 C3 01 9B 38 3A 00 EC B9 00 00 38 3A 24 00 16]
    JIN(20/16) -> NF(17/13) -> CBTI(11/6) -> LV(9/5) -> CBTI(4/3) -> BC(2/2) -> EFP(1/1)
    if(eForceGender != 0)

(185/101) [0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 38 3A 00 EC B9 00 00]
    L(59/35) -> SM(47/27) -> SM(28/16) -> LV(9/5) -> CBTI(11/6) -> LV(9/5)
    kSoldier.kAppearance.iGender = eForceGender

(1C0/124) [06 02 02]
    J(3/3)
    goto J0x202

(1C3/127) [0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9A A7 2C 02 16 25 16 02 00 2C 02 01 00 26]
    L(63/43) -> SM(47/27) -> SM(28/16) -> LV(9/5) -> C(15/15) -> NF(7/7) -> NF(4/4) -> ICB(2/2) -> EFP(1/1) -> IZ(1/1) -> EFP(1/1) -> ICB(2/2) -> IO(1/1)
    kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1)

Copy this code into Notepad++ and edit it to more compact form:


07 C3 01 9B 38 3A 00 EC B9 00 00 38 3A 24 00 16
if(eForceGender != 0)
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 38 3A 00 EC B9 00 00
kSoldier.kAppearance.iGender = eForceGender
06 02 02
goto J0x202
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9A A7 2C 02 16 25 16 02 00 2C 02 01 00 26
kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1)

This code has 97 bytes of file and 145 bytes of memory size. You can calculate these values by looking at decompiled tokens in UEE:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

The two numbers in parentheses at the beginning of each line represent memory (the first one) and file (the second one) offset of the current line (i.e. of the first byte of the line). Note that those numbers are given in hex representation! So, to calculate the sizes, we need to subtract two pairs of numbers: The file offset of the next expression line from the file offset of the first expression line, i.e. 152-0F1=61 (hex)=97 (dec); and the memory offset of the next from the first, i.e. 202-171=91 (hex)=145 (dec).

Now we need to convert our new code into hex:


kSoldier.kAppearance.iGender = ((eForceGender != 0) ? eForceGender : ((Rand(100) < 30) ? 2 : 1))
0F // let token
35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 // kSoldier.kAppearance.iGender expression
45 // ternary condition token
9B // not equal token
38 3A // primitive cast
00 EC B9 00 00 // eForceGender
25 // zero token
16 // end of not equal token
0B 00 // skip
38 3A // primitive cast
00 EC B9 00 00 // eForceGender
10 00 // skip
45 // ternary condition token
96 // less than token
A7 2C 64 16 // Rand(100) expression
2C 1E // zero token
16 // end of equal-equal token
02 00 // skip size
2C 02 // number 2
01 00 // skip size
26 // one token

For explanations on how to manage Skip Tokens, see the corresponding section below.

Our new code is 66 bytes long (i.e. it has 66 bytes of file size), which is 31 bytes less than the original code. By using "Disassembled Tokens View" we can find the memory sizes of all the expressions:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Those numbers represent memory (the first one) and file (the second one) sizes. Note that unlike memory and size offsets at the beginning of the line, these are given in decimal representation in UEE.

Variables are represented by Object References in bytecode. As we know from above text, Object References have bigger memory size, so we need to account for this specifically. All the other tokens have memory size equal to file size.

So, the resulting memory size of our new code is 94. This means that we are 51 bytes behind the original code in terms of memory size. The difference between lacking memory and file sizes is 51-31=20.

With all this in mind we now need to construct a “filler code”: we call this a code which does nothing, but aligns the sizes of the new and the old code so they fit.

We can use the 0B (NULL/nothing) token to fill up the difference in file size and the local variable token 00 <ObjectReference> to fill up the difference in memory size. The local variable token has 9 bytes of memory and 5 bytes of file size, i.e. it has 4 bytes more of memory size. So our difference of 20 bytes between memory and file sizes can be filled up with 20/4=5 local variable tokens:


00 EC B9 00 00 // eForceGender
00 EC B9 00 00 // eForceGender
00 EC B9 00 00 // eForceGender
00 EC B9 00 00 // eForceGender
00 EC B9 00 00 // eForceGender

This gives us 5*5=25 bytes of file size, so we need 31-25=6 more bytes to fill up the remaining file size:


0B 0B 0B 0B 0B 0B

The resulting replacement code, which has the exact same file and memory size, looks like:


kSoldier.kAppearance.iGender = ((eForceGender != 0) ? eForceGender : ((Rand(100) < 30) ? 2 : 1))
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9B 38 3A 00 EC B9 00 00 25 16 0B 00 38 3A 00 EC B9 00 00 10 00 45 96 A7 2C 64 16 2C 1E 16 02 00 2C 02 01 00 26
filler code
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
0B 0B 0B 0B 0B 0B

Using modding tools II

Since Patcher (either PatcherGUI or PatchUPK) allows the use of comments, you don’t need to create solid hex blocks for BEFORE/AFTER data. Just comment everything which is not hex with "//" and Patcher will ignore the whole line.

Note that if you want to change the gender probability again to some new value, you need to undo previous changes first (use Patcher's uninstall file to do so), as the BEFORE hex becomes different after you apply the mod.

By using comments we can mark parts of the script to make certain things easier to find:


DESCRIPTION=Search for “editable area” to find the number which defines the probability of females and change this number to anything between 0 and 100 in hex representation.
UPK_FILE=XComGame.upk
OBJECT=XGCharacterGenerator.CreateTSoldier
[BEFORE_HEX]
07 C3 01 9B 38 3A 00 EC B9 00 00 38 3A 24 00 16
//if(eForceGender != 0)
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 38 3A 00 EC B9 00 00
//kSoldier.kAppearance.iGender = eForceGender
06 02 02
//goto J0x202
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9A A7 2C 02 16 25 16 02 00 2C 02 01 00 26
//kSoldier.kAppearance.iGender = ((Rand(2) == 0) ? 2 : 1)
[AFTER_HEX]
//kSoldier.kAppearance.iGender = ((eForceGender != 0) ? eForceGender : ((Rand(100) < 30) ? 2 : 1))
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9B 38 3A 00 EC B9 00 00 25 16 0B 00 38 3A 00 EC B9 00 00 10 00 45 96 A7 2C 64 16 2C
// editable area: this number (hex) defines the probability of females
1E
// end of editable area
16 02 00 2C 02 01 00 26
//filler code
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
0B 0B 0B 0B 0B 0B

You can use the "DESCRIPTION" key to add a description to your mod file. Descriptions (and all following lines down to the next <KEYWORD> line) are ignored by the Patcher and can be used as the mod’s ReadMe documentation.

We can make this file a bit smaller by using RELATIVE OFFSET instead of BEFORE hex. RELATIVE OFFSET of the code inside an object can be found by adding 48 (30 in hex) to the file offset in UEE (note: this only valid for cooked XCOM packages):
[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Here (from the above image) the original offset of F1 is made relative by adding 30, so F1+30=121 (hex). The resulting mod file (note: this relevant to vanilla EW!) is:


DESCRIPTION=Search for “editable area” to find the number which defines the probability of females and change this number to anything between 0 and 100 in hex representation.
UPK_FILE=XComGame.upk
OBJECT=XGCharacterGenerator.CreateTSoldier
REL_OFFSET=0x121
[MODDED_HEX]
//kSoldier.kAppearance.iGender = ((eForceGender != 0) ? eForceGender : ((Rand(100) < 30) ? 2 : 1))
0F 35 82 0F 00 00 84 0F 00 00 00 00 35 8E 0F 00 00 99 0F 00 00 00 01 00 E7 B9 00 00 45 9B 38 3A 00 EC B9 00 00 25 16 0B 00 38 3A 00 EC B9 00 00 10 00 45 96 A7 2C 64 16 2C
// editable area: this number (hex) defines the probability of females
1E
// end of editable area
16 02 00 2C 02 01 00 26
//filler code
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
00 EC B9 00 00
0B 0B 0B 0B 0B 0B

The 0x before the 121 in the REL_OFFSET line indicates that the number is represented in hex.
Note that the AFTER_HEX keyword was replaced with MODDED_HEX, as there is no BEFORE hex; and AFTER hex can’t be used without it.

As a finishing touch we can use Patcher’s pseudo-code to make the female’s probability easily adjustable:


UPK_FILE=XComGame.upk
OBJECT=XGCharacterGenerator.CreateTSoldier
// Female probability: change 30 to whatever value you like between 0 and 100
// Do not change anything except the number!
// You can use only integer values between 0 and 100, no floating-point values are allowed!
// Value 0 will result in all males, value 100 - in all females.
ALIAS=FemaleProb:<%b 30>
// modded code - do not change anything below this comment!
REL_OFFSET=0x121
[MODDED_CODE]
//kSoldier.kAppearance.iGender = ( (eForceGender != 0) ? eForceGender : ( (Rand(100) < 25) ? 2 : 1 ) );
0F 35 <XGTacticalGameCoreNativeBase.TAppearance.iGender> <XGTacticalGameCoreNativeBase.TAppearance> 00 00 35 <XGTacticalGameCoreNativeBase.TSoldier.kAppearance> <XGTacticalGameCoreNativeBase.TSoldier> 00 01 00 <.kSoldier>
45 9B 38 3A 00 <.eForceGender> 25 16
[@] // skip
( 38 3A 00 <.eForceGender> )
[@] // skip
(
45 96 A7 2C <%b100> 16 2C <!FemaleProb> 16
[@] // skip
( 2C 02 )
[@] // skip
( 26 )
)
// fill
00 <.eForceGender>
00 <.eForceGender>
00 <.eForceGender>
00 <.eForceGender>
00 <.eForceGender>
0B 0B 0B 0B 0B 0B

Hex editing III: Changing a big part of the script

This section is a Work In Progress (WIP), and not yet complete.

Not all elements of the content have been posted as yet. While the available elements should be functional, other elements are needed to achieve the end-result as described.

This message will be removed when the content is complete.

BEFORE/AFTER data format will not update jump offsets for you. And jump labels only work with REPLACEMENT_CODE.

If you need to update jump offsets, you can use the HexToPseudoCode decompiler on your mod's hex code to obtain ready-to-modify code with all the jumps replaced with labels. In this case if you inserted new code bytes inside some jump operator, Patcher will re-calculate and update all the jumps. See the Using PatcherGUI/PatchUPK pseudo-code to calculate skip size sub-section for more.

For more information on pseudo-code see the PatchUPK ReadMe file.

Making changed functions compatible with other mods

Anything that changes jump offsets makes the mod incompatible with the other mods which change the same function. If you want to make your mods compatible with those by others, plan your modifications so they fit the original memory size. Another trick is to use the GOTO operator to jump out of the current part of the code into another one, execute a big chunk of the new code, and then jump back. If you place your new code in between Return and EOS tokens, the rest of the function will look like it wasn't changed. We used this trick a lot with big functions and states with a lot of jumps. Example:


if (something)
{
    <do something>;
}
else
{
   <do something else>;
}

This can be changed into:


if (something)
{
   goto <new code label>;
   <filler code to fit existing memory and size>;
   <label> for "jump back here"
}
else
{
   <do something else>;
}
...
return <something>;
<new code label>
<new code with lots of things>;
goto <"jump back here" label>;
EOS

How to locate a script’s file and memory sizes

Script and memory sizes are part of object’s binary data, but not of the script. They are located in the UPK file right before the beginning of each script. We usually call this part of the object “header”, although it’s not technically correct, as Unreal Engine serialized data have no header (see the Function and UPK data formats document. Anyway, since XCOM’s UPKs are cooked and all the additional information is removed, all the functions have their file and memory sizes stored at exactly the same relative offset (i.e. file offset in bytes, relative to the beginning of the object inside the file): memory size is a 4 bytes long integer at 0x28 and file size is also a 4 bytes long integer which follows the memory size immediately.

Managing the skip tokens

The most important part of the skip token is the skip size. Skip size indicates how much of the script in terms of memory can be skipped under certain circumstances.

There are two types of skip sizes: with and without prepending token 18.

Sizes without prepending token 18 are used in: context tokens (12, 19), ternary condition token (45), array Find (46, 47), AddItem (55), RemoveItem (56), InsertItem (57) and Sort (59) tokens and the default parameter token (49). In this case skip size consists of just two bytes of memory size to skip.

Sizes with token 18 are used in && (82), ^^ (83) and || (84) comparison operator tokens. In this case skip size consist of token 18 followed by 2 bytes of memory size to skip.

Let’s take a closer look at all of the types of tokens.

Ternary condition token

Ternary condition token allows you to create inline comparison expressions and can be extremely useful if you’re trying to save some space. Example:


(Rand(2) == 0) ? 2 : 1

The expression before ? operator is a condition or boolean variable. When it's equal to “true”, the first part is executed (the one in between the ? and : operators) and the second part (after the : operator) is skipped. When it's equal to “false”, the first part is skipped and the second one is executed. In this example Rand(2) == 0 is the condition, which is “true” when Rand(2) is equal to zero and “false” otherwise: i.e. 2 is the result, corresponding to a “true” outcome; or 1 is the result, corresponding to a “false” outcome.

The corresponding bytecode looks like:


45 // ternary condition token
9A // ==
A7 2C 02 16 // Rand(2)
25 // zero
16 // end of ==
02 00 // skip size is equal to 2 as the “true” part has 2 bytes of memory size
2C 02 // 2 - the “true” part
01 00 // skip size is equal to 1 as the “false” part has 1 byte of memory size
26 // 1 - the “false” part

You can calculate skip sizes either manually or by using UEE disassembled tokens view:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

Here we can see the ternary condition (Rand(2) == 0) ? 2 : 1 disassembled into separate tokens. The ones we need are the last two. Values in parentheses indicate memory (the first) and file (the second) size of the expression in decimal representation.

TIP: If you’re having problems with calculating skip sizes for your code, temporarily set them all equal to FF FF, apply your mod and then use disassembled tokens view in UEE to obtain the correct values. Since UEE ignores those skips, your code will be disassembled correctly.

ERROR WARNING! UEE is a decompiler not an engine, so it simply ignores skip size values. This means that even if the code looks good in UEE, it might still fail in game.

Context tokens

There are two context tokens used in XCOM: class context (12) and context (19).

Class context is used to access static class members. Example:


class'Engine'.static.GetCurrentWorldInfo()

The corresponding bytecode looks like:


12 20 35 FE FF FF 0A 00 92 F9 FF FF 00 1C D5 FB FF FF 16

This can be translated into:


12 <Expression1> <Skip> <ReturnValue> 00 <Expression2>

The context token is used to access member variables and functions of an object:


WorldInfo.GetALocalPlayerController()

Corresponding bytecode:


19 01 A6 F9 FF FF 0A 00 AE F9 FF FF 00 1C 4B FC FF FF 16

Which translates into the same notation:


19 <Expression1> <Skip> <ReturnValue> 00 <Expression2>

For both class context and context tokens skip size equals the memory size of <Expression2>:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

(0A in hexadecimal representation is equal to 10 in decimal representation.)

Default Parameter token

Some of the functions have the “optional” keyword added to some of their parameters. Example:


function TSoldier CreateTSoldier(optional XGTacticalGameCoreNativeBase.EGender eForceGender, optional int iCountry, optional int iRace, optional XGTacticalGameCoreData.ESoldierClass eClass)

Optional parameters have their default values, which are used when the function is called with no parameters defined:


m_kCharGen.CreateTSoldier()

From the bytecode side of view: default values for optional parameters are declared at the beginning of the script. For XGCharacterGenerator.CreateTSoldier the corresponding bytecode looks like:


(000/000) [0B]
    N(1/1)

(001/001) [49 06 00 1D FF FF FF FF 15]
    DP(9/9) -> IC(5/5) -> EPV(1/1)
    iCountry = -1

(00A/00A) [49 06 00 1D FF FF FF FF 15]
    DP(9/9) -> IC(5/5) -> EPV(1/1)
    iRace = -1

(013/013) [0B]
    N(1/1)

Optional parameters are defined subsequently. If we don’t want to define any value for a certain parameter, we should use the 0B token to skip to the next one. So, in this code eForceGender and eClass are undefined and initialized by the game engine, if omitted, and iCountry and iRace are defined specifically.

The Default Parameter token can be translated into:


49 <Skip> <Expression> 15

The last token (15) is called the End Parameter Value token and used to mark the end of parameter definitions.

So, in this case the skip token value is equal to the memory size of the expression plus one (for the 15 token):

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

ERROR WARNING! Remember: all the tokens are given in hexadecimal representation, but the sizes of disassembled tokens are given in decimal representation!

Array Find, AddItem, RemoveItem, InsertItem and Sort tokens

All these tokens follow the same pattern:


<TokenType> <Expression(Array)> <Skip> <Expressions> 16

TokenType here stands for the particular token and the first expression represents an array object itself. Skip token is followed by several expressions (one or two, depending on the TokenType) and there is a 16 token at the end, which stands for End Function Parameters token.

In all these cases skip value is equal to the sum of memory sizes of the expression following the skip token itself plus one (for the 16 token).

AddItem example:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

InsertItem example:

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

ERROR WARNING! Remember: all the tokens are given in hexadecimal representation, but the sizes of disassembled tokens are given in decimal representation!

Comparison operator tokens

All the comparison operators follow the same pattern:


<TokenType> <Expression1> 18 <Skip> <Expression2> 16

TokenType here can be 82 (&&), 83 (^^) or 84 (||); 18 is the Skip token, <Skip> is the skip size, 16 is the End Function Parameters token, while <Expression1> and <Expression2> are the expressions to compare. Example:


if((iCountry == 1) || iCountry == 24)

In all the cases the skip size is equal to the memory size of <Expression2> plus one (for the 16 token). Example (look at the red markings):

[Image by wghost81, used with permission under Creative Commons Attribution-Share Alike (CC BY-SA) license terms.]

ERROR WARNING! Do not confuse SkipToken disassembled line with the actual expression! Remember: all the tokens are given in hexadecimal representation, but the sizes of disassembled tokens are given in decimal representation!

Another way to calculate the skip size is to find the SkipToken expression disassembled line (look at the green markings). The skip size is equal to SkipToken memory size minus 2.

Using PatcherGUI/PatchUPK pseudo-code to calculate skip size

All the *_CODE sections and keys of the Patcher allow using of pseudo-code to automatically calculate skip size. Example:


//Game().GetDifficulty()
19 1B <Game> 16 [@] <XGStrategy.GetDifficulty.ReturnValue> 00 ( 1B <GetDifficulty> 16 )

Code [@] here replaces two bytes of the skip size in context token and parentheses mark the portion of the code whose memory size needs to be re-calculated.

Warning! Since Patcher does not perform bytecode decompilation, you should use pseudo-code only in between the parentheses for Patcher to be able to determine memory size correctly.
CAUTION: UPK Utils do not support unicode characters (not in text, nor in paths)!

For additional material on pseudo-code, see the UPKUtils Pseudo-Code sub-topic.

To Be Continued...

Lessons Learned

I figured I would point out a couple of potential trouble-spots (which have caused me a lot of headaches in the past) that could cause issues in other situations.

Return Values in context statements

This error is really common because UE Explorer doesn't really label or display these.

for example, In XGFacility_Barracks.DismissSoldier say you constructed the line:


//kSoldier.GiveBottomTreePerks()
19 00 73 31 00 00 0B 00 D3 52 00 00 00 1B D1 13 00 00 00 00 00 00 27 16

which UE Explorer decompiles just fine. However there is a secret bug lurking (in this case it doesn't matter because you aren't using the return value -- in fact, GiveBottomTreePerks doesn't even have a return value!).

The trick is in the D3 52 00 00 references, whose resolved name is : "XGStrategySoldier.HasPsiGift.ReturnValue". If you were actually trying to assign and use the results of the return value this would be a crashing bug. And it's really sneaky / hard to find-or-fix. For this particular example technically you'd set those byte to 00 00 00 00 as that's the value used for a null return. But it runs just fine without it.

I'm sure the real Unreal Compiler automatically inserts the correct ReturnValue reference into the context statement, but since we're modding we have to do it manually. The easiest way to find the ReturnValue of a function is by using the UE Explorer "View Managed Properties" Dialogue for the function. The ReturnValue will be given as a Decimal integer value (which you have to convert to LITTLE_ENDIAN hex).

Update: Actually, it is safe to use null-object references according to Unreal Script Variables:

"Variables that refer to actors always either refer to a valid actor (any actor that actually exists in the level), or they contain the value None. None is equivalent to the C/C++ NULL pointer. However, in UnrealScript, it is safe to access variables and call functions with a None reference; the result is always zero."

Differences in types of "==" operators

In XGSoldierUI.DismissSoldierActionCallback, say you disabled a conditional:


//if((GENELABS()) != none)
07 9C 00 77 1B F2 10 00 00 00 00 00 00 16 2A 16

by changing the 77 token into a 9A token. However 77 is the "!=" operator token for class comparisons, while 9A is the "==" operator for integer comparisons. So this effectively works because the comparison of the GENELABS() class object (as an implicit cast to integer) doesn't equal none (it can't), which results in false, skipping the conditional.

The "==" operator for classes is actually 72, while the "!=" operator for integers is 9B. Floats, strings, vectors all have their own "==" and "!=" operators. Bytes don't have comparison operators, which is why you'll always see them cast to integers before doing a comparison (using 38 3A).

Another effective (if somewhat messy) way to skip conditional is to change the 07 into a 06, changing it from a conditional to an unconditional jump. I use this method all the time when debugging.

Removing error checking statement

This one isn't really a byte code error, but suppose in XGFacility_Barracks.DismissSoldier you removed the line:


//if(kSoldier == none)
07 11 00 72 00 73 31 00 00 2A 16

in order to free up space. While it appears to work, if begin the function by using:


//if (kSoldier.GetStatus() == 0)
07 66 00 9A 38 3A 19 00 73 31 00 00 0A 00 7D 53 00 00 00 1B 51 13 00 00 00 00 00 00 16 38 3A 24 00 16

This can be the source of maddening intermittent errors that users end up reporting. If the function is ever (for some reason) called with a null kSoldier, the game will crash.

Super Functions

Super functions are called to invoke the method of an identical name within a child class.

For example: working with the function PrefersLowCover, which is present in the base class XGAIBehavior as well as 3 child classes: XGAIBehavior_Sectoid, XGAIBehavior_ThinMan, and XGAIBehavior_SectoidCommander. In vanilla this function makes Thin Men and Sectoids prefer low cover on Easy and Normal difficulties. The goal is to modify this in general to make all units prefer low cover only if they have the Low Profile perk.

The trick is to not use the 1B virtual token identifier (combined with a simple namelist reference; see ExportObjectList table in the UPK Package Header in the wiki article UPK File Format - XCOM:EU 2012), but instead to use the 1C "final function" token combined with a direct objectlist reference to the specific function.

A virtual call to PrefersLowCover looks like:


// PrefersLowCover()
1B 1F 65 00 00 00 00 00 00 16

Where 0x651F is a namelist index to the string "PrefersLowCover" in the namelist.

To generate a specific call to super.PrefersLowCover() in XGAIBehavior_Sectoid.PrefersLowCover, the format is instead:


// return super(XGAIBehavior).PrefersLowCover()
04 1C E7 9B 00 00 16

Technically XGAIBehavior_Sectoid is a child of XGAIBehavior_Psi, which is in turn a child of XGAIBehavior, which is why UE Explorer puts in the explicit XGAIBehavior reference.

In this case the 1C token is followed by 0x9BE7, which is an objectlist reference to the specific function XGAIBehavior.PrefersLowCover.

For the case of XGAIBehavior_ThinMan, which is a direct child of XGAIBehavior, UE Explorer decompiles the same hex bytes as:


// return super.PrefersLowCover()
04 1C E7 9B 00 00 16

With this technique you can change game behavior to use your replacement function instead of a vanilla version.

Latent Functions

"Latent functions" are the functions which require some time to pass. One of the widely used examples is Sleep(fTime) function, which pauses the code for fTime seconds. A little more detailed description of latent functions can be found here.

The first thing to learn is that latent functions can't be used outside of states(!). Bytecode is "magic" and we can put a Sleep call into a regular function, but this will be terribly wrong and potentially buggy.

Even when used inside a state, latent functions need to be handled carefully. For example, calling Sleep in the middle of a ForEach loop will do nothing. In most of the cases Sleep is used to pause the state execution until something else finishes (like "aliens finish all their moves", etc.), and if such waiting happens inside a ForEach loop it will lead to a "runaway loop error".

Judging by the number of Google results on the subject ("unrealscript sleep runaway loop") latent functions are "magic" and mastering them, especially without compiler tracking code errors, can be somewhat hard. Anyway, if the game CTDs after you've put a Sleep call (or similar latent function) somewhere in the code, first thing to do is check your launch.log file and see if there is a "runaway loop" error message.

UPKUtils Modfile

Modfile is a simple human-readable text (.txt) file. PatchUPK/PatcherGUI (see the Programs & Tools section) is fully compatible with ToolBoks Custom Mods: it can read and implement all the changes found there.

Modfile consists of keys (NAME=VALUE), sections ([SECTION_NAME]) and comments. Comments are begin with a ({) and end with a (}). You may place comments anywhere; they also can be multi-line. Right now PatchUPK is case-sensitive and does not support Unicode characters, so be careful!

List of supported keys and their allowed values:

MOD_NAME, AUTHOR and DESCRIPTION are self-explanatory. :smile: Value is simple text. Can be multiline.
UPK_FILE - name of UPK file to patch.
OFFSET - dec or hex (0x) value.
FUNCTION - full function name to patch.
REL_OFFSET - relative offset, dec or hex (0x) value.

If absolute offset value is set through OFFSET or FUNCTION key, REL_OFFSET is added to that value, so you can patch parts of function code without a need to specify it's full offset.

FUNCTION_FILE - file (with path) to read data from. Can be used with OFFSET or FUNCTION or by itself, if file name has Full.Function.Name.Function format.
NAMELIST_NAME - change namelist name. Value is of OLDNAME:NEWNAME.
EXPAND_FUNCTION - move and expand function by its name. Value is Full.Function.Name:NEWSIZE.

List of supported sections:

[MODDED_HEX] - section which contains modded space-separated hex data (as in ToolBoks custom mod).
[BEFORE_HEX] and [AFTER_HEX] - hex data for search-and-replace style patching,
[/BEFORE_HEX] and [/AFTER_HEX] are permitted, by not used anyway, since parser uses different method to find data blocks.

Command line syntax: PatchUPK modfile.txt [PATH_TO_UPKS]

[PATH_TO_UPKS] is optional (omit the square brackets). If the path parameter is omitted, UPKs are expected to be in the current working directory.

Some examples

Note these are all related to implementing a single function (XGStrategyAI.GetAltWeapon:300) change.

Move and expand:


UPK_FILE=XComStrategyGame.UPK
EXPAND_FUNCTION=XGStrategyAI.GetAltWeapon:300

Find and replace:


UPK_FILE=XComGame.UPK

[BEFORE_HEX]
CC 04 00 00 18 03 00 00 0F 00 FF B2 00 00 25
07 C9 04 96 00 FF B2 00 00 2C 04 16 07 51 00
9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A
00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 10 16
04 0B 06 BB 04 07 BB 04 9A 38 3A 35 7A B2 00
00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6
B2 00 00 38 3A 24 00 16 0F 35 7A B2 00 00 7C
B2 00 00 00 01 1A 00 FF B2 00 00 01 A6 B2 00
00 24 10 07 BF 03 81 2D 01 88 B2 00 00 16 0F
00 FE B2 00 00 25 07 BC 03 96 00 FE B2 00 00
36 35 42 00 00 00 43 00 00 00 00 00 01 95 B2
00 00 16 07 DA 01 9B 38 3A 35 3D 00 00 00 3F
00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00
00 43 00 00 00 00 00 01 95 B2 00 00 38 3A 24
00 16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A
00 FF B2 00 00 01 A6 B2 00 00 7D 00 1B 54 0F
00 00 00 00 00 00 35 3D 00 00 00 3F 00 00 00
00 00 10 00 FE B2 00 00 35 42 00 00 00 43 00
00 00 00 00 01 95 B2 00 00 35 3A 00 00 00 3F
00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00
00 43 00 00 00 00 00 01 95 B2 00 00 16 16 07
C4 02 9B 38 3A 35 3C 00 00 00 3F 00 00 00 00
00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00
00 00 00 01 95 B2 00 00 38 3A 24 00 16 55 35
7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00
00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00
00 00 35 3C 00 00 00 3F 00 00 00 00 00 10 00
FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00
01 95 B2 00 00 35 39 00 00 00 3F 00 00 00 00
00 10 00 FE B2 00 00 35 42 00 00 00 43 00 00
00 00 00 01 95 B2 00 00 16 16 07 AE 03 9B 38
3A 35 3B 00 00 00 3F 00 00 00 00 00 10 00 FE
B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01
95 B2 00 00 38 3A 24 00 16 55 35 7B B2 00 00
7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2
00 00 7D 00 1B 54 0F 00 00 00 00 00 00 35 3B
00 00 00 3F 00 00 00 00 00 10 00 FE B2 00 00
35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00
00 35 38 00 00 00 3F 00 00 00 00 00 10 00 FE
B2 00 00 35 42 00 00 00 43 00 00 00 00 00 01
95 B2 00 00 16 16 A5 00 FE B2 00 00 16 06 C5
00 06 B8 04 55 35 7B B2 00 00 7C B2 00 00 00
00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B
54 0F 00 00 00 00 00 00 38 3D 35 6A B2 00 00
71 B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55
35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2
00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00
00 00 00 38 3D 35 69 B2 00 00 71 B2 00 00 00
00 01 82 B2 00 00 4A 16 16 55 35 7B B2 00 00
7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2
00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38 3D
35 66 B2 00 00 71 B2 00 00 00 00 01 82 B2 00
00 4A 16 16 06 C9 04 A3 00 FF B2 00 00 16 06
0B 00 04
[/BEFORE_HEX]

[AFTER_HEX]
A8 04 00 00 18 03 00 00 0F 00 FF B2 00 00 25
07 89 04 96 00 FF B2 00 00 2C 04 16 07 51 00
9A 38 3A 35 7A B2 00 00 7C B2 00 00 00 00 1A
00 FF B2 00 00 01 A6 B2 00 00 38 3A 24 10 16
04 0B 06 7B 04 07 7B 04 9A 38 3A 35 7A B2 00
00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6
B2 00 00 38 3A 24 00 16 0F 35 7A B2 00 00 7C
B2 00 00 00 01 1A 00 FF B2 00 00 01 A6 B2 00
00 24 10 07 7F 03 81 2D 01 88 B2 00 00 16 0F
00 FE B2 00 00 25 07 7C 03 96 00 FE B2 00 00
36 35 42 00 00 00 43 00 00 00 00 00 01 95 B2
00 00 16 0F 01 98 B2 00 00 38 3A 35 3D 00 00
00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42
00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07
84 01 97 01 98 B2 00 00 25 16 55 35 7B B2 00
00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6
B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38
3D 01 98 B2 00 00 25 16 16 0F 01 98 B2 00 00
38 3A 35 3C 00 00 00 3F 00 00 00 00 00 10 00
FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00
01 95 B2 00 00 07 79 02 97 01 98 B2 00 00 25
16 0F 00 57 B3 00 00 25 07 79 02 98 00 57 B3
00 00 35 39 00 00 00 3F 00 00 00 00 00 10 00
FE B2 00 00 35 42 00 00 00 43 00 00 00 00 00
01 95 B2 00 00 16 55 35 7B B2 00 00 7C B2 00
00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 7D
00 1B 54 0F 00 00 00 00 00 00 38 3D 01 98 B2
00 00 24 00 16 16 A5 00 57 B3 00 00 16 06 E3
01 0F 01 98 B2 00 00 38 3A 35 3B 00 00 00 3F
00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00
00 43 00 00 00 00 00 01 95 B2 00 00 07 6E 03
97 01 98 B2 00 00 25 16 0F 00 57 B3 00 00 25
07 6E 03 98 00 57 B3 00 00 35 38 00 00 00 3F
00 00 00 00 00 10 00 FE B2 00 00 35 42 00 00
00 43 00 00 00 00 00 01 95 B2 00 00 16 55 35
7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00
00 01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00
00 00 38 3D 01 98 B2 00 00 24 00 16 16 A5 00
57 B3 00 00 16 06 D8 02 A5 00 FE B2 00 00 16
06 C5 00 06 78 04 55 35 7B B2 00 00 7C B2 00
00 00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A
00 1B 54 0F 00 00 00 00 00 00 38 3D 35 6A B2
00 00 71 B2 00 00 00 00 01 82 B2 00 00 4A 16
16 55 35 7B B2 00 00 7C B2 00 00 00 00 1A 00
FF B2 00 00 01 A6 B2 00 00 2A 00 1B 54 0F 00
00 00 00 00 00 38 3D 35 69 B2 00 00 71 B2 00
00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B B2
00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01
A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00
38 3D 35 66 B2 00 00 71 B2 00 00 00 00 01 82
B2 00 00 4A 16 16 06 89 04 A3 00 FF B2 00 00
16 06 0B 00 04 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B
0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B
0B 0B 0B
[/AFTER_HEX]

Rename function:


UPK_FILE=XComStrategyGame.UPK
NAMELIST_NAME=GetAltWeapon:#GetPodProgs

Patch part of the function by relative offset:


UPK_FILE=XComGame.UPK
FUNCTION=XGBattleDesc.InitAlienLoadoutInfos
REL_OFFSET=0x28
[AFTER_HEX]
A8 04 00 00 18 03 00 00 0F 00 FF B2 00 00 25 07
89 04 96 00 FF B2 00 00 2C 04 16 07 51 00 9A 38
3A 35 7A B2 00 00 7C B2 00 00 00 00 1A 00 FF B2
00 00 01 A6 B2 00 00 38 3A 24 10 16 04 0B 06 7B
04 07 7B 04 9A 38 3A 35 7A B2 00 00 7C B2 00 00
00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 38 3A 24
00 16 0F 35 7A B2 00 00 7C B2 00 00 00 01 1A 00
FF B2 00 00 01 A6 B2 00 00 24 10 07 7F 03 81 2D
01 88 B2 00 00 16 0F 00 FE B2 00 00 25 07 7C 03
96 00 FE B2 00 00 36 35 42 00 00 00 43 00 00 00
00 00 01 95 B2 00 00 16 0F 01 98 B2 00 00 38 3A
35 3D 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00
00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00
00 07 84 01 97 01 98 B2 00 00 25 16 55 35 7B B2
00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6
B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D
01 98 B2 00 00 25 16 16 0F 01 98 B2 00 00 38 3A
35 3C 00 00 00 3F 00 00 00 00 00 10 00 FE B2 00
00 35 42 00 00 00 43 00 00 00 00 00 01 95 B2 00
00 07 79 02 97 01 98 B2 00 00 25 16 0F 00 57 B3
00 00 25 07 79 02 98 00 57 B3 00 00 35 39 00 00
00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42 00
00 00 43 00 00 00 00 00 01 95 B2 00 00 16 55 35
7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00
01 A6 B2 00 00 7D 00 1B 54 0F 00 00 00 00 00 00
38 3D 01 98 B2 00 00 24 00 16 16 A5 00 57 B3 00
00 16 06 E3 01 0F 01 98 B2 00 00 38 3A 35 3B 00
00 00 3F 00 00 00 00 00 10 00 FE B2 00 00 35 42
00 00 00 43 00 00 00 00 00 01 95 B2 00 00 07 6E
03 97 01 98 B2 00 00 25 16 0F 00 57 B3 00 00 25
07 6E 03 98 00 57 B3 00 00 35 38 00 00 00 3F 00
00 00 00 00 10 00 FE B2 00 00 35 42 00 00 00 43
00 00 00 00 00 01 95 B2 00 00 16 55 35 7B B2 00
00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01 A6 B2
00 00 7D 00 1B 54 0F 00 00 00 00 00 00 38 3D 01
98 B2 00 00 24 00 16 16 A5 00 57 B3 00 00 16 06
D8 02 A5 00 FE B2 00 00 16 06 C5 00 06 78 04 55
35 7B B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00
00 01 A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00
00 38 3D 35 6A B2 00 00 71 B2 00 00 00 00 01 82
B2 00 00 4A 16 16 55 35 7B B2 00 00 7C B2 00 00
00 00 1A 00 FF B2 00 00 01 A6 B2 00 00 2A 00 1B
54 0F 00 00 00 00 00 00 38 3D 35 69 B2 00 00 71
B2 00 00 00 00 01 82 B2 00 00 4A 16 16 55 35 7B
B2 00 00 7C B2 00 00 00 00 1A 00 FF B2 00 00 01
A6 B2 00 00 2A 00 1B 54 0F 00 00 00 00 00 00 38
3D 35 66 B2 00 00 71 B2 00 00 00 00 01 82 B2 00
00 4A 16 16 06 89 04 A3 00 FF B2 00 00 16 06 0B
00 04 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B
0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 0B 53
[/AFTER_HEX]

There is a fully functional mod example in the Nexus Mod file archive UPK Utils under the TestMod folder called LargerPodsTest.txt.

UPKUtils Pseudo-Code

BEFORE/AFTER data format will not update jump offsets for you. And jump labels only work with REPLACEMENT_CODE.

If you need to update jump offsets, you can use the HexToPseudoCode decompiler on your mod's hex code to obtain ready-to-modify code with all the jumps replaced with labels. In this case if you inserted new code bytes inside some jump operator, Patcher will re-calculate and update all the jumps. See the Using PatcherGUI/PatchUPK pseudo-code to calculate skip size sub-section for more.

For more information on pseudo-code see the PatchUPK ReadMe file.

Warning! Since Patcher does not perform bytecode decompilation, you should use pseudo-code only in between the parentheses for Patcher to be able to determine memory size correctly.
CAUTION: UPK Utils do not support unicode characters (not in text, nor in paths)!

The following are the results of attempts by someone other than wghost81 (the author of UPKUtils) to translate hex code into "pseudo-code" for implementation by UPKUtils/PatcherGUI. This is presented as a learning example only, but may not actually do anything in game. Based upon the thread "Modding Armor to add Grappler property" in the XCOM Mod Troubleshooting forum.

The goal is to modify armor types other than "Skeleton" and "Ghost" in "Enemy Unknown" to use the Grappler item.

My "Trial and Error" process to determine the correct pseudo-code once the correct function (in hex) to alter has been determined to be:


Function Token: 53 or 0x35
Name: StructMember
Expression: .
Hex data format example: 35 <ObjRef(Member)> <ObjRef(Struct)> <Byte> <Byte> <Expr>

where elements in <angle brackets> are parameters of the "StructMember" function.

UPKTools Pseudo-Code generic equivalent of the hex data format example:


35 <ClassName[.parentn[.etc]].parent0.child> <ClassName[.parentn[.etc]].parent0> 00 00 <Expr: NextMember or VarRef>

Notes:

For the 0x35 "StructMember" token: "<Expr>" is either the next "StructMember" or a "VarRef"; with the first "StructMember" being the least significant and the "VarRef" being the most significant.

The "VarRef" could be any form of a variable reference (local, global, etc).
Bear in mind the pseudo code "<XGBattleDesc.BuildSoldierContent>" would resolve to 4 bytes (file-size) of code; while a local "VarRef" would resolve to 8 bytes (file-size) of code - the last 4 bytes usually being hex "00 00 00 00". ... still attempting to get my head wrapped around the memory-size concepts / differences from file-size.

Virtual Functions might be used in more complex structures - where there is a ReturnValue-ref specified; else I'm not sure how they work.
For those that might not already know this: (in UEE) ClassNames are the first level objects of a UPK-file.
"Class." translates as UPK-filename: for XComGame.upk - "Class." works in place of "XComGame." (where "XComGame." will cause UPKTools to throw an error).

"Class." is usually only needed when referencing a ClassName (by itself: i.e. <Class.ClassName>); "ClassName" as a prefix (i.e. <ClassName.child>) tends to work just fine.

XComGame.XComGameClassNames may be used outside of XComGame.upk, etc? as UPKTools throws an error if you try to use "XComGame." inside of XComGame.upk, etc.

My "Trial and Error" process to determine the correct pseudo-code:

Devise possible pseudo code in a plaintext file.
Use PatcherGUI to mod file "XComGame.UPK".
Open modified "XComGame.UPK" in UEE.
Navigate to the modded 'test' function (in this case "XGBattleDesc.BuildSoldierContent").
'View disassembled tokens' in UEE to see what hex values I actually got in place of the 'possible pseudo code'.
... rinse and repeat ... (use PatcherGUI's uninstall file to uninstall patch and back to the first step, until I get what I am looking for.)

The rightmost part of my pseudo-code "<Expr>" usually was a function or reference as in:


kTransfer.kChar.kInventory.iArmor

has four "<Expr>" elements - "kTransfer", "kChar", "kInventory", and "iArmor"

A search in UEE on a given element (say "iArmor") will usually give some reference qualifier. For "iArmor" it's "XGTacticalGameCoreNativeBase.TInventory.iArmor", but it is not always the correct qualifier for the current context.

So we plug in "<XGTacticalGameCoreNativeBase.TInventory.iArmor>" where the pseudo-code needs to have "iArmor", then patch and see what hex code we get. Of course LOCAL Variables are either passed in as an argument to a function or declared at the top of a function - and the pseudo-code for the local variable reference "kTransfer" is "00 <.kTransfer>".

Usually I had to plug in two-to-four elements to complete the "<Expr>" syntax of the function to get the patch to succeed - such as (in the case of a "StructMember"):


35 <XGTacticalGameCoreNativeBase.TInventory.iArmor> <XGTacticalGameCoreNativeBase.TInventory> 00 00 00 <.kTransfer>

A nice thing about the "StructMember" token - the second "<Expr>" element is the qualifier(s) used for the first element (minus the specific - last piece of - the elements, in this case minus the ".iArmor").

If you get the same hex code as that search reference produced elsewhere in the function - you have the proper qualifiers for that context (though possibly only for in that function).

Otherwise you need to go back to UEE and do a manual search on "iArmor" - that is take a stroll through the UPK file and see where else you might find what you are looking for. There is a certain logic (if you can wrap your mind around it) to where one might look: like "Armor" is more-or-less a subset of "Inventory" which is more-or-less a subset of "Character"(s); and "class 'engine'" is likely to be found in the "Engine.UPK". (I found "<engine.engine>" under dependencies in the "XComGame.UPK" file.)

Structures are likely to be somewhere like "XGTacticalGameCoreNativeBase.TInventory.Structures" - but when used in pseudo code the ".Structures" and ".Functions" tend to get left out (as the pseudo code doesn't seem to know what to do with them).

Example Notes:

While my examples will produce the specified result, said result (may or) may NOT work in the game! ... yes, emphasis is where I want it.

NOTE: I did replace the original local variable ref with "TransferInfo" - but other than that the examples below are straight from my 0x35 token modding of "BuildSoldierContent".

Examples are context sensitive: that is ClassName, parent-refs and child-ref may vary between contexts.

Simple Example:


single Member and the VarRef
Given: TransferInfo.iArmor


UPKTools Pseudo-Code:
35 <XGTacticalGameCoreNativeBase.TInventory.iArmor> <XGTacticalGameCoreNativeBase.TInventory> 00 00 00 <.TransferInfo>

Complex Example:


three Members and the VarRef
Given: TransferInfo.kChar.kInventory.iArmor


UPKTools Pseudo-Code:
35 <XGTacticalGameCoreNativeBase.TInventory.iArmor> <XGTacticalGameCoreNativeBase.TInventory> 00 00
35 <XGTacticalGameCoreNativeBase.TCharacter.kInventory> <XGTacticalGameCoreNativeBase.TCharacter> 00 00
35 <XComContentManager.TTransferSoldier.kChar> <XComContentManager.TTransferSoldier> 00 00
00 <.TransferInfo>

Virtual Function Complex Example:


VirtualFunction(Arg0:three Members and the VarRef, Arg1:<%b 2>)
Given: ArmorHasProperty(TransferInfo.kChar.kInventory.iArmor, 2)


UPKTools Pseudo-Code:
1B <ArmorHasProperty>
35 <XGTacticalGameCoreNativeBase.TInventory.iArmor> <XGTacticalGameCoreNativeBase.TInventory> 00 00
35 <XGTacticalGameCoreNativeBase.TCharacter.kInventory> <XGTacticalGameCoreNativeBase.TCharacter> 00 00
35 <XComContentManager.TTransferSoldier.kChar> <XComContentManager.TTransferSoldier> 00 00
00 <.TransferInfo>
2C 02 16

Final Function Complex Example:


FinalFunction(Arg0:three Members and the VarRef, Arg1:2)
Given: ArmorHasProperty(TransferInfo.kChar.kInventory.iArmor, 2)


UPKTools Pseudo-Code:
1C <XGTacticalGameCore.ArmorHasProperty>
35 <XGTacticalGameCoreNativeBase.Tinventory.iArmor> <XGTacticalGameCoreNativeBase.TInventory> 00 00
35 <XGTacticalGameCoreNativeBase.TCharacter.kInventory> <XGTacticalGameCoreNativeBase.TCharacter> 00 00
35 <XComContentManager.TTransferSoldier.kChar> <XComContentManager.TTransferSoldier> 00 00
00 <.TransferInfo>
<%b 2> 16

The two function examples have a couple of easy to miss differences:

Where I placed the <%b 2> is used to demonstrate that <%b 2> is the same as the hex-code: 2C 02
And Token 1B doesn't want the "XGTacticalGameCore." that Token 1C has to have, for UPKTools Pseudo-Code purposes.

WARNING: If you are allowing UPKTools to calculate a script's MemSize using the [Before]-n-[After] method with a difference in sizes THEN you need to know that UPKTools doesn't do so good on pure Hex Code.

Caution: with different [Before]-n-[After] sizes - it is just as IMPORTANT to have what is being replaced in Pseudo-Code as it is to have your modded info in Pseudo-Code! Else calculate and set the Sizes yourself!

The Good News: UPKTools does wonderful on resizing calculations - when both the [Before]-n-[After] code are in Pseudo-Code.

Untouched hex-code (that contains only relative-offsets) may be sandwiched in with your Pseudo-Code, as long as you include it in both the [Before]-n-[After] sections.

Any absolute offsets, after your modded code and still within the modded script (such as the address included on a IF or IF-NOT token), will need to be modded TOO!

And remember, any control transfers into the area of your modded code, will also need your attention.

Control Structures

Jump / Goto

Goto statements are quite simple. They are always 3 bytes long. The jump addresses (the objects of those statements) follow 06, 07, 0A, 58, or 2F tokens. (See Hex values XCOM Modding for each token's purpose.)

Syntax consists of a token such as 0x06 (EX_Jump), followed by a two byte value representing the jump offset. The two byte value is stored "little endian" as are all multi-byte values.

For example, a Jump address of 0x4A2 would be represented in hex as the sequence "06 A2 04".

Jump offsets are computed as offsets relative to the beginning of the current function. Since jump offsets are limited to 2 bytes, a jump offset cannot exceed 65535 bytes, or 64KB.

Jump offsets are measure in VIRTUAL bytes (bytes the code occupies when loaded into memory), not FILE bytes (bytes the code occupies in the file). The most straightforward way of determining jump offsets is to use the "View Tokens" view in UE Explorer.

Bertilsson's Jump Repair Tool is an online tool which will validate and repair jump offsets based on token data provided from UE Explorer 1.2.4.

Instructions how to use:

Get UE Explorer 1.2.4 beta or later (older versions are not compatible).
Copy modified Hex code and View Tokens data from UE Explorer into the tool.
Press Validate and Repair code button to begin validation and repair process.
The tool will now provide you with drop down selectors based on object code and high level numbering for any valid or invalid jump reference which will be part of the repair process.
When all invalid jump offsets have been corrected you will receive a new set of modified Hex code with recalculated memory jump offsets for 06, 07, 0A, 2F, 58 tokens and the header reference to the end of script.
Insert new Hex code from the tool into the UPK file using HxD or any other hex editor.

Limitations and how it works

The tool is basically a data miner that looks at the world as a bunch of view token lines which may or may not contain references to each other.

When user has manually selected a new target for an invalid one, the same offset difference is automatically also applied to the next following jump reference all the way to the end of function. This chain effect is automatically halted whenever it would result in a new invalid jump reference.

The tool is intended to completely and intuitively save you the trouble of recalculating the jump offsets by hand and to repair mistakes you may have done when adding new code.

Known bugs

A bit buggy user interface, currently being improved

The intended way to use the tool is:

Export function (bytes) from UE Explorer into a separate mod file for the function you want to edit.
Modify the mod file using HxD or any other hex-editor.
Import the mod file back into UE Explorer to get new view tokens.
Copy view tokens into the repair tool.
Copy byte code into the repair tool (if you only want to verify valid header and jump offsets you can actually skip this step).
Fix anything broken and adjust jump offsets to liking.
Replace the byte code using HxD or any other hex-editor in the mod file.
Import the new byte code from the mod file back into UE Explorer again.

If you have already done the editing by hand you only have to do step 4 and nothing else.

If you have created a complicated else if/else statement, you can simply verify in the tool that all jump offsets are pointing to valid targets (you will be informed about any identified issue) and then afterwards verify in UE Explorer that the object code looks as intended.

The reason it's recommended exporting/importing to a mod file from within UE Explorer instead just modding the .UPK file directly is:

You don't have to bother about file offsets.
You can move code around using CTRL + X + V without worrying that you will break the entire .UPK if you make a mistake.
You can quickly save and restore different versions of your modifications.
You can use CTRL + A to quickly copy and replace the entire function.

For discussion about this tool, see this thread New tool available ....

Conditional statements

(With Wasteland Ghost's XCOM UPK Patcher & PatcherGUI compatible examples)

If statement / JumpIfNot token

if (something)
{
	<do something>;
}

07 <MemOff> <Expr>

07 JumpIfNot token
<MemOff> a two-byte offset
<Expr> that evaluates to True | False

Pseudo code example:
- 07 [@Label_endif] 81 00 <.bName_1> 16
  - 04 0B
- [#Label_endif]

If with Else statements / using Jump token

if (something)
{
	<do something>;
}
else
{
	<do something else>;
}

Requires: 06 <MemOff>

06 Jump | goto Token
<MemOff> a two-byte offset

Pseudo code example one:
- 07 [@Label_endif] 81 00 <.bLocal_Var> 16
  - 04 24 01
  - 06 [@Label_endelse]
- [#Label_endif]
  - 04 24 02
- [#Label_endelse]
If .bLocal_Var is False returns 1 else returns 2 ...

Pseudo code example two:
- 07 [@Label_endif1] 00 <.bParm> 16
  - 04 24 01
  - 06 [@Label_endelse]
- [#Label_endif1]
- 07 [@Label_endelse] 81 00 <.bLocal_Var> 16
  - 04 24 02
  - 06 [@Label_endelse]
  - 04 24 03
- [#Label_endelse]
If .bParm is true returns 1 else-if .bLocal_Var is False returns 2 else returns 3 ...

Switch case statement

Foreach loop / Iterator token

Data Structures

Arrays

Index accessing

Dynamic arrays

Struct

Objects

Object variables (member token)

Enums

An enum (also enumerated type or enumeration) is a primitive data type in UnrealScript. The possible values of an enum type are a list of identifiers, which can be compared not only for equality, but also for order.

See article Enum Expansion - XCOM:EU 2012.

Debugging Hex Code

[The following is derived from the Debugging Hex Code thread in the Nexus Forums XCOM Mod Talk section.]

One of the problems with making direct hex edits to existing code is debugging when it causes a "crash-to-desktop" (CTD). Here are some suggestions:

Check "launch.log". With luck, the last few lines will have something like "script call stack" giving you a pointer to the location of the problem.

Double-check that your modification decompiles correctly in UE Explorer.
- If it doesn't, a likely candidate for the error is mismatched jump offsets from control structures (i.e. 'if-then-else' or 'while' blocks). Both UE Explorer and UPK Modder can help with figuring out the proper offsets.

Try adding your modifications one line at a time.

The following procedure is used by Amineri (of the Long War mod team) when debugging more complex modifications, such as those in a modfile.

Narrow the problem down to a single function.
- Apply and Revert various combinations of modfile as needed to narrow down the scope of the CTD as best you can.
- When changes are "linked", use early <return> statements to skip executing particular functions or segments of code.

Once narrowed down to a function, skip executing particular pieces of code to narrow the CTD to a particular line.
- <Return> statements can be added to skip execution of code past a certain point in the fuction, e.g.:


<line1>
return
<line2>

will prevent <line2> from executing, so if the above lines fail to CTD it means that the problems lives in <line2> or later.

- For the purpose of narrowing down where the problem lies, try putting a <return> statement about mid-way in the suspect section. Then try halving again the remaining suspect section, and keep halving as you narrow down to a particular line.
- Conditional jumps (e.g. 0x07 token statements) can be converted to unconditional (i.e. 0x06) to skip all code within a conditional. If a CTD stops happening after such a change, then the CTD is within the body of the conditional block.

Once narrowed down to a particular line, figure out the problem with the line.
- Make sure both absolute and relative jump offsets are correct (UPKmodder v0.90+ checks for validity of absolute jump offsets, but not relative ones).
- Some composited lines can be broken apart to test pieces. For example:


<statement1> = <statement2>

can broken into:


<statement1>
<statement2>

by replacing the 0x0F Assignment token with a 0x0B null-op. Functionally the two statements won't do anything, but their code is still executed. Having broken them down into separate lines, the use of <return> statements can be used to determine which portion of code is causing the CTD.

These techniques are easier than commenting out large sections of code as you don't then need to repeatedly adjust jump references.

Recent Discoveries

There have been a couple of recent (April-May 2013) discoveries that affect the need for or use of some of XCOM Modding tools. They are summarized here for reference.

Phoning Home

When using mods, it is necessary to prevent each game from connecting to it's update servers behind your back. However, the DNS entry is clearly for a vendor specific address. This appears to be a vendor verifier (i.e. "Firaxis Verifier", in this case) independent of those patches distributed via Steam. They may not be 'patches' per se, but merely replacements of certain files to ensure consistency. This is informally called 'phoning home' and initially was believed to be intended to be a Steam mechanism to prevent cheating in Multi-Player games. However, it has now been determined to affect single-player games as well.

PatcherGUI (as of v5.2) includes options to "Enable INI loading" and "Disable phoning home" for you. This is the easiest and recommended method.

If you use PatcherGUI, you can skip the rest of this sub-section. It's intended for those who have to deal with this manually.

Those interested in the mechanism involved in this 'phone home' process are referred to the Class: IniLocPatcher - XCOM:EU 2012 article.

This edit to the hosts file is not affected by Steam delivered patches or re-installing Steam and does not affect your ability to get updated when you go back to 'Online' mode for ANY reason (such as connecting to the Steam Store). Note this should be used in conjunction with disabling 'auto-updates' for your XCOM game in the Steam Library. Be aware, however, that since November of 2013 XCOM patches seem to disregard this Steam 'auto-updates' setting and have been known to apply regardless. Having a backup copy of your complete modded XCOM game tree current and available elsewhere is recommended as well.

At present, the EU 'phone home' process is known to overwrite Armors, Characters, and Weapons arrays of the EXE internal resource cache version of the DefaultGameCore.INI file, and the ...\My Documents\My Games\XCOM - Enemy Unknown\ file XComGameCore.INI file which is a merged copy from the DGC.INI and DLC content.

My Games\XCOM - Enemy Unknown\XComGame\Logs\EMS\

XComDLC.ini (XComGame.XComDLCManager)
XComGame.int (XComGame.XComDLCManager)
XComGameCore.ini (overwriting Armors, Characters and Weapons arrays)
XComMPGame.ini (overwriting a lot of MP related arrays)

These may be in My Games (if you have DLCs installed) or the <Steam install path> equivalent path, or both. If you are having problems with mods that alter one of these files not seeming to 'stick', check that your hosts file has the correct servers disabled according to the Launch.log as described below.

To determine where your game attempts to communicate, you want to examine your \My Games\<Steam Game folder>\Logs\Launch.log file. You are looking for lines similar to these:

[0027.91] DevHTTP: FHttpDownload resolve complete to: 65.118.245.165:80

[0027.91] DevHTTP: FHttpDownload resolve complete to: 65.118.245.165:80

...

[0028.15] DevHTTP: FHttpDownload::StateParsingHeader: Got unknown status code 503 URL: http://prod.xcom.firaxis.com:80/News.ashx

[0028.19] DevHTTP: FHttpDownload::StateParsingHeader: Got unknown status code 503 URL: http://prod.xcom.firaxis.com:80/Dict.ashx

The entries marked like so are:

The Server IP address: 65.118.245.165
The Server DNS Name: prod.xcom.firaxis.com

These are what you will place into the hosts file to block attempts to connect.

Start Menu > Right click on 'Notepad' (or any text editor), select "Run as Administrator"
- File > Open > %systemroot%\system32\drivers\etc\hosts. The hosts file has no '.' suffix, so make sure you don't add .txt or anything else. Also, %systemroot% on most systems is C:\Windows, but the exact folder name may vary by OS version. The value of the global environmental variable %systemroot% on your system can be determined by entering 'set' on the Windows command line, or as it's shortcut is called, the Command Prompt. But the variable %systemroot% (including the percent "%" signs) can safely be used without knowing it's actual value.
- Add these three lines to the hosts file:
  (The first line is a comment. The next two lines do the actual work, but are specific to XCOM:Enemy Unknown 2012. Other games may have their own DNS names [i.e. prod.xcom.firaxis.com] and IP addresses [i.e. 65.118.245.165]. The '127.0.0.1' IP address is a local loopback address the prevents attempts to connect to either of the other addresses from ever leaving your computer.)


# Prevent XCOM:EU from Phoning Home
127.0.0.1 prod.xcom.firaxis.com
127.0.0.1 65.118.245.165

Save the file.

NOTE: The "Enemy Within" (EW) expansion to XCOM:EU 2012 uses it's own DNS and IP address entries:


# Prevent XCOM:EW from Phoning Home
127.0.0.1 prod.xcom-ew.firaxis.com
127.0.0.1 65.118.245.139

However, early reports (13 Nov 2013) that disabling these addresses in hosts will prevent EW from launching have been attributed to other factors, such as failing to also disable 'auto-updates'. Save game synching with Cloud Storage still functions correctly, even with these addresses disabled.

The XCOM-EW expansion is proof that other DNS and IP addresses can be utilized by the same or different games. Care should be taken to check the Launch.log file as outlined above for changes with each patch or major release.

Disabling Hash checks

PatcherGUI (as of v5.2), part of the mod UPK Utils, includes options to "Enable INI loading" and "disable phoning home" for you. It can also find and open the correct config folder for you. So, to install INI modifications, one needs to select the correct game path (XCom or XCom/XEW), "enable INI loading" and "disable phoning home", and it will open the config folder and replace the vanilla file with the modified one. This is currently the recommended tool if you do not feel comfortable hex editing yourself.

It has recently (20 Apr 2013) been discovered that the launcher does not check every UPK file, and that it uses a table in the executable to look-up the files it checks. It then compares the hash with that in the companion "<UPK filename>.uncompressed_size" file. By simply changing the names of the UPK files in this look-up table so they are not recognized or found, it will happily load regardless of any other files hash. This appears to eliminate the need for removal of the companion ".uncompressed_size" file (which consequentially causes the game to use it's internally stored hash value) and XSHAPE (which updates the internal hash), or any other hash fixing.

Simply look for the strings "xcomgame.upk" (XComGame.upk) and "xcomstrategygame.upk" (XComStrategyGame.upk) in the executable using a hex editor (such as HxD, linked below) and change them to some other string. The current version of ToolBoks is now using this technique on both of those files, as well as "xcomshell.upk" (XComShell.upk).

Mac OSX users need to make these changes to the Hashes.sha file in Binaries\Win32, to the filenames located at 0x00D0-0x0120.

NOTE: The Enemy Within (EW) expansion does not include these UPK hash checks any longer, and does not need this patch to read in the loose decompressed versions of these files. The original EU EXE with patch 5 still does.

The following is taken from the documentation of the XCOMModHelper tool available at Github.com. That tool is recommended for Windows environments instead of manual edits.

Bear in mind: Search for the following in hex mode for unicode values.

Enemy Unknown patch 5 (changelist 382855).
Target Path: Binaries\Win32\XComGame.exeBinaries\Win32\XComGame.exe
Description: Disable XComGame.UPK hash check.
Find:


78 63 6f 6d 67 61 6d 65 2e 75 70 6b

Replace:


77 63 6f 6d 67 61 6d 65 2e 75 70 6b

Target Path: Binaries\Win32\XComGame.exeBinaries\Win32\XComGame.exe
Description: Disable XComStrategyGame.UPK hash check.
Find:


78 63 6f 6d 73 74 72 61 74 65 67 79 67 61 6d 65 2e 75 70 6b

Replace:


77 63 6f 6d 73 74 72 61 74 65 67 79 67 61 6d 65 2e 75 70 6b

Enabling INI loading

PatcherGUI (as of v5.2) includes options to "Enable INI loading" and "Disable phoning home" for you. This is the easiest and recommended method. See also the INI Loading Test entry.

The following is provided for background and those interested in the manual method.

A recent (24 May 2013) discovery makes it possible to force the game to load the DGC.INI config file from the "<Steam install path>\XComGame\Config" folder, thereby eliminating the need to use ModPatcher or make DGC.INI changes to the EXE at all, and avoiding the size limitation on the embedded file. (See the entry note on ModPatcher here about size limitations.)

Bear in mind: Loose INI files get read, and thus override mod changes to the internal 'resource cache', after the game EXE starts.

Choose your mods intelligently and with understanding. Not all will work together.

These hex offsets are for EXE version 1.0.0.28586; patch 4, Changelist 356266, unless otherwise noted.

In xcomgame.exe: starting at location 0x157D93A.
For Mac OS XCOM Elite Edition: starting at location 0x1D86A70.
For Mac OSX Steam version, patch 3 (Jan 8, 2013): starting at location 0x1E7E5E0.
For Mac App Store version 1.0: around offset 0x1BD58D0. Search for "72 00 00 00 65 00 00 00 2E 00 00 00 69 00 00 00 6E 00 00 00 69 00 00 00" (which is "re.ini" in UTF32 Little-Endian).

NOTE that these strings are also found as part of disabled debug code, at offset range: 0x156BA82-156BF38.

For Mac App Store version 1.0.1: starting at 0x1D86B50. If searching in a hex editor, look for "44 00 00 00 65 00 00 00 66 00 00 00 61 00 00 00 75 00 00 00 6C 00 00 00 74 00 00 00 47 00 00 00 61 00 00 00 6D 00 00 00 65 00 00 00 43 00 00 00 6F 00 00 00 72 00 00 00 65 00 00 00 2E 00 00 00 69 00 00 00 6E 00 00 00 69 00 00 00". This is "DefaultGameCore.ini" in UTF32LE.

If you edited version App Store version 1.0, the App Store will not update the game and you will need to delete it and re-download. After editing, the app will need to be code-signed, again (as usual...).
Take care with the location of the strings you disable. As always, these locations may change after official game patches are released.

There are the three unicode text strings:

\.X.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.G.a.m.e.C.o.r.e...i.n.i...........\.....\.X.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.M.P.G.a.m.e...i.n.i.......\.....\.X.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.L.o.a.d.o.u.t.s...i.n.i.

It appears that these three strings are checked to see which files to NOT load from the Config directory.

Change the first string to something else:

\.C.C.o.m.G.a.m.e.\.C.o.n.f.i.g.\.D.e.f.a.u.l.t.G.a.m.e.C.o.r.e...i.n.i.

And behold: the game loads from the DefaultGameCore.ini (DGC.INI) file in the "<Steam install path>\XComGame\Config\" directory, even if larger than the embedded version.

All the other INI files are read by default. Only these three have to be modded to load.

This means that no longer are BALANCE Mode settings the only ones taking effect from the text editable file in "\Config", and that additional items can be added to the DGC.INI file. Of the other files explicitly denied from loading, DefaultMPGame.ini (believed to be "multiplayer' related), and DefaultLoadouts.ini (initial equipment for classes of Soldiers and Aliens), are similarly enabled in the same manner.

NOTE: The Enemy Within (EW) expansion appears to still need this patch to read in the loose INI files. The original EU EXE definitely still does.

For instructions on how to search for the above filenames as unicode strings, with screenshots, see the wiki article How to DIY common mod solutions.

Bear in mind: Search for the following in hex mode for unicode values.

The following patches are taken from the documentation of the XCOMModHelper tool available at Github.com. That tool is recommended for Windows environments instead of manual edits.

Enemy Unknown patch 5 (changelist 382855). Target Path: Binaries\Win32\XComGame.exeBinaries\Win32\XComGame.exe
Find:


25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 58 00 43 00

Replace:


25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 57 00 43 00

Enemy Unknown patch 6 (changelist 398179). Same change, at offset 22535260 (decimal) / 0x157dc5c (hex).

Enemy Within patch 0 (initial release). Target Path: XEW\Binaries\Win32\XComEW.exe Find:


25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 58' 00 43 00

Replace:


25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 57 00 43 00

Enemy Within game file XComEW.EXE hex offset for Mac/OSX version: around 0x1FE7EB3.

Most likely reason for any failure to load the DGC.INI is that (assuming you have the EW version of the game) you modified the DGC.INI in the wrong folder tree. EU is under "XCom-Enemy-Unknown", while EW is under "XCom-Enemy-Unknown\XEW".

INI Loading Test

Try this to test that the loose DGC.INI file is loading:

Edit the DGC.INI as follows:
Replace the Original 'eItem_Shotgun' line 'Properties[1]=eWPAssault' entry with '=eWP_Anyclass' as shown below. You can just copy and paste this SINGLE line with any text editor. (Save your '.BAK' file to restore the original DGC.INI.)

Weapons=(iType=eItem_Shotgun,ABILITIES[0]=eAbility_ShotStandard,ABILITIES[1]=eAbility_Overwatch,ABILITIES[2]=eAbility_NONE,ABILITIES[3]=eAbility_NONE,ABILITIES[4]=eAbility_NONE,ABILITIES[5]=eAbility_NONE,Properties[0]=eWP_AnyClass,Properties[1]=eWP_AnyClass,Properties[2]=eWP_None,Properties[3]=eWP_None,Properties[4]=eWP_None,Properties[5]=eWP_None,iDamage=6,iEnvironmentDamage=30,iRange=22,iReactionRange=8,iReactionAngle=360,iRadius=0,iCritical=40,iOffenseBonus=10,iSuppression=10,iSize=eItemSize_Large,iHPBonus=0,iWillBonus=5)

Load any previous save where you are in the XCOM base (the Strategy game phase) or play a new game to that point, go into the Barracks, View Soldiers, and change the Loadout of a Support class (which couldn't without this change and doesn't have any other class restrictions that might prevent it, as is the case with a Heavy). The 'vanilla' DGC.INI that's in the embedded 'Resource cache' only permits an Assault to equip a Shotgun. With this change, any class can that doesn't have other built-in restrictions.
Anyone other than an Assault class equipping a Shotgun proves you are loading the DGC from the correct '\config' folder.

INI file size limitations

There appears to be a minimum and maximum size to the combined total of the three INI files that are prevented by default from being loaded by the game engine: DefaultGameCore.ini, DefaultMPGame.ini, and DefaultLoadouts.ini. If the combined total size of these three INIs is either smaller or larger than these limits, the game won't load (not a CTD) with the message: "Failed to start game (app already running)". This may apply to the combined size of ALL INI files, but has not been tested nor have the combined minimums and maximums been determined. (See the discussion at Difficulty adding more items to DGC.INI.)

However, interestingly this implies reducing the size of one of these files enables expanding the size of another of them by the same amount. There has been some success with exploiting this for expanding the DGC.INI, but there may be other checks that have not yet been identified. There appears to be an upper limit to the DGC.INI of very close to 101648 bytes.

Increased Load Times

01 Oct 2013: The root cause of this problem has been identified as the format of all the "Package" entries in the [Engine.PackagesToFullyLoadForDLC] section of DLC's XComEngine.ini file. The 'dot' preceeding the keyword "Package" is a UE3 Special Character and needs to be replaced with a 'plus', as in:

[Engine.PackagesToFullyLoadForDLC]
  {change}
.Package=Zhang_ANIM
  {to}
+Package=Zhang_ANIM

Work is underway to alter the cleanup script below to incorporate this fix so it will only need to be run after a game update or verification of game cache files.

It's been determined (see the Forcing game to load from DefaultGameCore.ini thread) that DLC content is getting merged repeatedly whenever the game loads or restarts. This increases the size of the respective INI files, and causes the same DLC content to be reloaded multiple times per session. If the game has been loaded 100 times, those DLC assets are now being loaded 100 times. This is due to a bug in the process by which DLC content is merged.

This issue has been confirmed to exist with a vanilla, verified cache copy of the game without any mods.

The issue appears to be in the My Games folder XComEngine.ini / [Engine.PackagesToFullyLoadForDLC] section. In that section the following 37 lines are repeated <loaded> times:

MapName=Command1
Package=DLC_Day060
Package=Deco_Archangel0
Package=Deco_Carapace1
Package=Deco_Ghost0
Package=Deco_Kevlar1
Package=Deco_Psi0
Package=Deco_Skeleton1
Package=Deco_Titan0
Package=Helmet_Archangel0
Package=Helmet_Carapace1
Package=Helmet_Ghost0
Package=Helmet_Kevlar1
Package=Helmet_Psi0
Package=Helmet_Skeleton1
Package=Helmet_Titan0
Package=Head_AsnMale_F
Package=Hair_MaleHair_N
LoadForAllGameTypes=LoadForAllGameTypes
Package=Zhang_ANIM
MapName=CIN_LoadScreen
Package=UILibrary_MapImages_DLC1
MapName=DLC1_1_LowFriends
Package=DLC_Day060
MapName=DLC1_2_CnfndLight
Package=DLC_Day060
MapName=DLC1_3_Gangplank
Package=DLC_Day060
MapName=Command1
Package=Deco_Kevlar0
Package=Helmet_Kevlar0
Package=Deco_Skeleton0
Package=Helmet_Skeleton0
Package=Deco_Carapace0
Package=Helmet_Carapace0
Package=Hair_OriginalSoldier

The current workaround solution is to periodically remove the excess lines from the XComEngine.ini / [Engine.PackagesToFullyLoadForDLC] section. A VB script has been created to do this available here, but here are the instructions to recreate it for yourself if your antivirus program balks at it.

Open notepad.
Copy & Paste below text into notepad.

Const MY_DOCUMENTS = &H5&
Const ForReading = 1
Const ForWriting = 2
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objShell = CreateObject("Shell.Application")
Set objFolder = objShell.Namespace(MY_DOCUMENTS)
Set objFolderItem = objFolder.Self 

ConfigFolder = objFolderItem.Path & "\My Games\XCOM - Enemy Unknown\XComGame\Config\"
Set OrgFile = objFSO.OpenTextFile(ConfigFolder & "XComEngine.ini", ForReading)

strOrgText = OrgFile.ReadAll
OrgFile.Close

arrMapName = Split(strOrgText, "MapName=Command1")
if Ubound(arrMapName) >2 then
  strOutput = arrMapName(0) & "MapName=Command1" & arrMapName(1) & "MapName=Command1" & arrMapName(Ubound(arrMapName))
  timestamp = year(Now()) & "-" & Right(Cstr(month(now()) + 100),2) & "-" & Right(Cstr(day(now()) + 100),2) & "_" & Right(Cstr(hour(now()) + 100),2) & Right(Cstr(Minute(now()) + 100),2) & Right(Cstr(Second(now()) + 100),2)
  objFSO.CopyFile ConfigFolder & "XComEngine.ini", ConfigFolder & "XComEngine.bak" & timestamp
  Set destFile = objFSO.OpenTextFile(ConfigFolder & "XComEngine.ini", ForWriting)
  destFile.WriteLine(strOutput)
  destFile.Close
  msgbox (Ubound(arrMapName)/2)-1 & " redundant sets of engine DLC packages has been removed from XComEngine.ini." & vbnewline & vbnewline & "A backup of your original file has been saved here:" & vbnewline & ConfigFolder & "XComEngine.bak" & timestamp
else
  msgbox "Everything looks fine. No action taken."
end if

File --> Save As... --> CleanUpXComEngineIni.vbs (make sure to not save it as CleanUpXComEngineIni.vbs.txt by mistake).

NOTE: If your "My Games" folder is called something strange like "Meinen Spielen" you will have to change that on line 9.

When you run the script it will first check that <My Documents>\My Games\XCOM - Enemy Unknown\XComGame\Config\XComEngine.ini contains more than two "MapName=Command1". It only removes lines between the second and the last "MapName=Command1".

Enemy Within

This is a purchased expansion to XCOM:EU 2012, released in the USA on 12 Nov 2013. It installs to a XEW subfolder under the main XCOM install path. Most of the loose files from EU are replicated in this subfolder, which means EU and EW are basically separate games. There is some sharing of movie (.BIK) files. EW adds a launcher app XComLauncher.exe in the \XCom-Enemy-Unknown\XEW\Binaries\Win32 folder which is started by Steam and provides the option of which game (EU or EW) to launch.

The actual game executables this launcher runs are:

\XCom-Enemy-Unknown\Binaries\Win32\XComGame.exe -- launches Enemy Unknown
\XCom-Enemy-Unknown\XEW\Binaries\Win32\XComEW.exe -- launches Enemy Within

EW no longer makes a hash check of the compressed UPK files. So they can simply be decompressed and will be used after modification once the related .uncompressed_size files are removed.

However, the mod to force the game to use the loose XEW\XCOMGame\Config\DefaultGameCore.ini file is still needed. Hex edit the XComEW.EXE file as follows:

Find: (i.e. key is the 58 in the last line)


  25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
  65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
  2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 58 00 43 00

Replace: (i.e. replace the 58 in the last line with 57.)


  25 00 64 00 00 00 00 00 49 00 6e 00 69 00 56 00
  65 00 72 00 73 00 69 00 6f 00 6e 00 00 00 00 00
  2e 00 2e 00 5c 00 2e 00 2e 00 5c 00 57 00 43 00

EW Update server (i.e. HOSTS file):

127.0.0.1 prod.xcom-ew.firaxis.com
127.0.0.1 65.118.245.139

These 'phone home' locations update the XComGamecore.INI file in config/loc on game launch, and are needed in addition to the entries for EU. However, early reports that disabling these connections cause the game to freeze are now attributed to other causes, such as not disabling 'auto-updates'. Cloud Synching of save game files still works with these addresses disabled. (While sometimes referred to as "the silent patching mechanism", so far there is no evidence that these servers are involved in actual changes to the game code. They still appear to only restore certain vanilla files involved in Multiplayer games, which can disrupt mods touching those same files.)

The EW 'phone home' addresses seem to only update XComGameCore.INI in My Games\XCOM - Enemy Within\XComGame\Logs\EMS\ at this time.

Extending/Replacing Functions

In early Dec 2013, wghost81 discovered it was possible to alter the UPK Package information so the game would utilize an expanded or replacement function of any size. This frees the mod creator from the constraint of keeping their code changes to within the original byte size limitations of the vanilla function, as well as providing the possibility to add new functions exclusive to your own mods. See the Hex editing UPK files article entry Hex editing III: Extending/Replacing Functions for details.

References

Referred to by this article:

That refer to this article:

Hex editing UPK files

Contents

Introduction

Content

Hex Editing Overview

Programs & Tools

UE Explorer

UPK Decompressor

PatcherGUI/PatchUPK

XSHAPE

HxD HEX Editor

Notepad++

UPK Extractor

WinMerge

JPEXS Flash Decompiler

Hex editing I: changing single values

A word of advice

Preparing the ground

Browsing UPK files with UE Explorer

Searching for the code

Understanding the numbers

Applying the changes

Keeping notes

Constructing BEFORE and AFTER hex

Using modding tools

Hex editing II: Changing a small part of the script

Understanding script’s file and memory sizes

Planning the replacement

Using modding tools II

Hex editing III: Changing a big part of the script

Making changed functions compatible with other mods

How to locate a script’s file and memory sizes

Managing the skip tokens

Ternary condition token

Context tokens

Default Parameter token

Array Find, AddItem, RemoveItem, InsertItem and Sort tokens

AddItem example:

InsertItem example:

Comparison operator tokens

Using PatcherGUI/PatchUPK pseudo-code to calculate skip size

Related Topics

UnrealScript

Programing basic concepts

Data Types

Hex values

Sentences & Operators

Prefix'd operators

Multiple operators

EndFunctionParms

Hex code for an expression with multiple operators

Lessons Learned

Super Functions

Latent Functions

UPKUtils Modfile

Some examples

UPKUtils Pseudo-Code

Control Structures

Jump / Goto

Conditional statements

If statement / JumpIfNot token

If with Else statements / using Jump token

Switch case statement

Foreach loop / Iterator token

Data Structures

Arrays

Index accessing

Dynamic arrays

Struct

Objects

Object variables (member token)

Enums

Debugging Hex Code

Recent Discoveries

Phoning Home

Disabling Hash checks

Enabling INI loading

INI Loading Test

INI file size limitations

Increased Load Times