File information

Last updated

Original upload

Created by

Galenmacil

Uploaded by

Galenmacil

Virus scan

Safe to use

About this mod

A complete patching system for Grim Dawn: Decompress game files, patch what needs to be patched/edited and reconstruct the game back to a fully playable state.

Permissions and credits
Changelogs
A Data Patching System for Grim Dawn


Forewarning: This system, and the accompanying mod, was designed to work with the fully expanded game: Ashes of Malmouth and Forgotten Gods are mandatory. Crucible is not. While you need the complete game to install the Grim Dawn EU Lite MOD, you can still benefit from what the patching system can offer, providing you are willing to modify part of it yourself.
The creator of this MOD and accompanying windows batch files take no responsibility should nefarious usage comes of it either by malevolent modifications to it of simply by mistakes...
You can edit, modify and redistribute parts or the whole "patching system" without asking for explicit permission.
Backing up your save files before starting a newly modded game for the first time is highly recommended.



Update procedure if you want to install the latest version over a previous version:

See the included readme file for instructions on how to update.


Simple installation procedures for those that just want to quickly install/test the mod:

You should not play this mod with an existing character if it is above level 50. A fresh start is recommended.

A - Download and extract the whole content of the MOD ZIP file to the Grim Dawn working folder. This is the folder that contain the "Grim Dawn.exe" 32 bit file. NOT the "x64" folder! That one as the 64 bit executable.

B - Parts of the mod can be deactivated should you not want them. Check out the list of .AHK files freshly extracted from the above step. The filenames describe what they do. If you want to deactivate a specific patch, simply delete that file or change the extension to something else like ".unused".

C - From the game working folder, browse to and select the file named "Install GD-DPS MOD.bat". Double-click it to launch it.

  • The installation process will most likely take more than 10 minutes. Read the message that appears and navigate your way thru the choices, if any.
  • The installation operates in a console windows. All inputs are made with the keyboard.
  • If you click the inside of the console window, anywhere, it may prevent the window from updating. The process is still going on but it is not showing. You need to press a key with the window focused to unpause it. This is very annoying, I know, so be careful, please. :)


***************************************


This is the result of a few months of intense fiddling around with Grim Dawn mod tools. It started as a simple question: "Can Grim Dawn data be easily and automatically modified between game update using a 'patching' mechanism"? So this premise led the foundation for an elaborate modding mechanism that function as follow:

1 - It uses only readily available free tools from the Internet. Currently, it uses some of the tools Crate graciously distributed as part of the game as well as some third party ones:
Crate's AssetManager
Crate's MapCompiler
Crate's ArchiveTool
Custom programmed utilities for handling archives and map decompiling: ARZARC and unMap
Microsoft's windows batch scripting language, part of any Microsoft OS since MS-DOS
The one and only AutoHotkey (https://www.autohotkey.com/download/)
The indispensable MD5 Command Line Message Digest Utility (https://www.fourmilab.ch/md5/)
The lightning fast FFMPEG (https://ffmpeg.org/)
The noble 7-Zip Archiver (https://www.7-zip.org/)
(Optional but highly recommended) NotePad++ (https://notepad-plus-plus.org/)

2 - The process is completely automated. It flows somewhat like this:
- Starting from a fully up to date "Forgotten Gods" expanded working, moded or not, copy of the game. A backup of all the game ARZ and ARC compressed archives is created in a sub-folder of the game working folder.
- This backup is then fully extracted, including the GDX2 .MAP file, in parallel executing threads when possible . From there,  it serves as a restore point, so to speak, for the game. The backup is never altered in any way whatsoever.
- Using AutoHotkey powerful regular expression evaluator, an arbitrary numbers of .AHK scripts files that are designed to target and modify specific data in the game database records are ran, again in parallel when possible. The scripts fetch data files from the backup, patches them in memory, then rewrites them in the game working folder. Leaving the original untouched.
- Along side that, to handles the game various resources file types, like .WAV, .TEX, .MSH, .CNV, etc.., advanced window batch scripts are ran to, again, fetch the data from the source, apply modification to them and rewrites them in the game working folder. Leaving the source completely intact.
- Once all these modifications to the game are made, the game data sets (all the various archives for resources, world file and levels, databases and templates) are reconstructed, automatically, from the game working folder. This process is bottle-necked mostly by the storage subsystem I/O speed, but, if it is fast enough like an M.2 drive or RAM drive, this stage can benefit from multi-threading allowing for simultaneous reconstruction of all the ARC and ARZ files (more on that later on).

3 - The game can then be played in a modded state determine by whatever the patching scripts can do. Since it uses patching, it is possible to start from an already modded game and build from there or easily re-patch after an official update. Also, since the patching does not depend on the local game working folder data, it uses the backup source data instead, the whole process is fully repeatable, lets say, for testing purposes, without any issues.



***************************************


To demonstrate the process as proof of concept, it is distributed as a MOD that I called "Grim Dawn EU Lite MOD". Initially, the goal was to mod the game and make it slightly more feasible, at least for me, on the Elite and, more specifically, Ultimate levels. The whole idea was to "slightly modify" core game mechanics to lean toward that goal. It is not a complete redesign of the game nor is it a game expander. No new class or items are added to the game for example. The game retains a very vanilla flavor.


List of the mod's many changes: Note that since all these modification are handled via patching scripts, they can all be disabled or modified by simply editing the corresponding script file in a text editor!


- Maximum player level cap increase to 200: With the (not so) recent announcement by Crate that they are now supporting mod with an increased player level cap, it is is now possible to continue leveling your character past level 100!
- Devotion point cap increase to 200: Coupled with the Tribute points system from 'The Crucible' add-on, this allows lower level character to quickly gets some form of Constellation build while still making it a long process to get past the default 55 devotion points for high level character.
- Always 3 attributes points gained per new level: Even at level above 50.
- Class skills maximum level set to "Ultimate Level": Makes it possible to level up a skill past its default maximum level up to that skill ultimate level.
- Inclusive exclusive skills: Convert all "exclusive skill" to normal skill so they can be used with one another.
- Unlocked devotion skills: Assign any devotion skills to any character or item skill independently of types.
- Monsters spawning multiplier: Adjust the number of monsters present in mobs. This include regular mobs, factions specifics mobs, corrupted shrines monsters, totems mobs and non-unique bosses and heroes monsters. Does not get applied to 'The Crucible' add-on'.
- "Rogue Dungeon" merchant will buy items: Just like regular merchant but with a 3% bonus payment on items.
- Faster merchant merchandise refreshing: Set to a fixed 5 minutes instead of the default 10 to 45 minutes.
- Player run speed caps changes: Starting at 20% on all difficulty for the minimum and up to 200% at the maximum from the default value of 135%. Faster movement with the correct gears and bonus equals easier getaways from fast enemies.
- No artificial penalties on any resistance types on Elite and Ultimate difficulty:
In the default game, the penalty is 25% on Elite and 50% on Ultimate. This remove that.
- Adjustable player summonable pets movement speed increase: Pets can now follow you around easily even when moving at ridiculous speed!
- Constitution giving item rework: Untouched food and ration only gives back constitution, no health. However, vital essence now gives back some health and energy but no constitution. Better balanced that way in combat since tougher enemies are more likely to drop vital essence on death. Also, one bowl of any cook's tasty creation is now enough to completely refill constitution.
- Non linear attributes progression for class mastery: This makes it more viable to have a dual class non maxed masteries character. Imagine that, instead of a straight line, the attributes (health, energy, physique, cunning and spirit) bonus gained by increasing a mastery level follows a logarithmic style curved progression. Starts and end points are the same, all nine classes have the same 2400 total, split between the 5 types, as they did before. Only the progression follows a curves so, for example, the bonus gained from having two masteries at 25 is proportionally higher than having only one mastery at 50. Also possible is the adjustment of the total attributes limit: Set it lower or higher then the default 2400 total. All attributes progression are scaled accordingly.
- Items stacks limit increased to 9999 for components and 999 for potions and other stackables: A much needed increased from the default stacks sizes of 1000 and 99 respectively.
- Change to the camera maximum and minimum distance and angle: Configurable. Top down 90° as well as closeup character inspection angles are now possible.
- Focus items, AKA caster off-hands, skills cooldown bonus multiplied by a factor of 2: More incentive to use them for what they are good at.
- Less restrictive component fusing on gears: Any component that can be fused on at least one armor parts gets expanded to be fused on all armor types. Same for amulet, ring and medal in the jewelries category. More possibilities. This does not affect augments powders.
- Varied and more accurate tags for items: Tired of seeing "Redeemer", "Preserver" or "Exalted" named gear? This patch give more detailed and logical item names to all equipable items below 'Epic' (blue) rarity. Monster infrequent 'Rare' (green) are not affected either. See the "Readme.txt" file for importable compatibility data for Wanez GD Tools colored tags features.
- Better item dismantling mechanism: Fixed dismantling cost at 2000 iron bits but every components can now be obtained, even rare level 75 ones. The higher the level of the components, the less chances it will drop from dismantling an item. Rare items like 'Epic' and 'Legendary' also gives higher chances of obtaining rare crafting materials (Cultist Seal, Wendigo Spirit, Eldritch Essence, etc.) instead of components.
- Side quest and faction item level restriction removal: Once you get the standing to buy an item from a faction, there won't be any level requirement restricting you further from using it. Again, this does not affect augment powders. As for side quest item like "Fortified Doublet", "Skinner Family ring", "Hart's Amulet" and the like, keep them for a future new build as kick starter gears!
- Constellation Power Tweaks: The starting constellation 'Crossroad' is now actually interesting for high level characters too! A percentage bonus as been applied on top of the linear bonus already in place.
- Enemies mesh rescaler: Modify the size of two bloatly oversized enemies to more adequate stature: 'Korvaak Ascended' and 'Eldritch Wraith'. The heroes and boss category for these two foes types will still appears proportionally bigger.
- Get all five inventory bags in normal or veteran difficulty: The last quest of the main game, "The Bane of Cairn", now grant the player that 5th bag on completion.
- Fix for structural objects, decorations and some entities that do no cast shadows: Crate forgot to set the corresponding flag for a bunch of them.
- "Unmuted" storms and lightning: Thunder, wind and rain will come to life!
- Increased chances of bad weather: It is the apocalypse after all...
- Potions duration and effects tweaks: All 'oil' type potions now last for 450 seconds. All 'regeneration' type potions, like all royal jelly derivatives, have their effect increased but not their duration. 'Elixir of the Hunt' now last for 900 seconds, like all other elixir, but the speed bonus was reduced to 10% from 30%. Time to bring all these stashed potions on your next adventure... and quaff them!
- Light radius bonus on non special items changed to racial damage bonus: Convert "Luminous", "Celestial" and "Light Bringer's" prefixed item light radius bonus to Chthonic/Undead damage bonus. 'Epic' (blue) and 'Legendary' (pink) gears with the light radius bonus are not affected.
- Iron bits loot adjustments: Multiply by a fixed value the minimum and maximum amount of iron bits that drop as loot. The base game, "The Crucible" and "Forgotten Gods Endless Dungeons" can all be adjusted independently.
- Player, devotion and item "skills" tweaking: Not so useful skills are given an uplift: 'Aether Tendril', 'Demon Breath', 'Beronath's Fury' and 'Glyph of Disorder Rune' are now more interesting to use. Duration, range, energy cost and cooldown can all be manipulated.
- Lots of sounds improvements:  Unique sounds for different water effect, fire sizes, acid pool and organic structures among others. Positional ambient sounds feel more natural, new and more diverse sound for ranged weapons. Many skills get new and expanded sound effects. Other changes include sounds sanitation like loudness normalizing, fix for badly constructed looping sounds, unnecessary over sampling at 96khz/32 bit... Yes, sounds in this game have been neglected a lot by Crate but, using FFMPEG, pretty much anything can be done sound wise. Except for new sounds, a lot of existing ones are modified, by scripts, on the fly.
- Complete music removal: For those that do not like music in this game. Only the main menu theme is preserved.
- Loading screen and tips disabling: Replaced by a black screen with only the loading animation shown in the lower right.
- 'Grim UI X-Mod Hoarder' integration: This enlarged UI is a part of the 'EU Lite MOD' and can be integrated seamlessly into the base game, including using it with 'The Crucible' add-on.
- Support for 'Dawn of Masteries': All the above changes can also be automatically integrated in this custom game mode via compatibility patching if it is detected during the installation.



***************************************


Going Deeper: What is needed to create a mod with this system?
The mod ZIP files comes with all the third party tools listed above at step one. But one, obviously, needs to have the game with both expansion installed. Crucible is not mandatory but can be patched as well if present. Having a good understanding of the windows batch scripting language as well as some level of AutoHotkey and regular expression syntax comprehension is going to help a lot.

The system can be expanded easily by supplying new AHK scripts and taking example from using the 50+ windows batch file (.BAT files) included. Just by opening these files in a text editor, a lot can be figured out. The installation process execute any AHK scripts (within some parameters) it find in the game working folder. There are two easy way to disable a script: Edit it with a text editor and set the "Disabled:=True" variable or simply change the extension to something else like "Increase maximum player level to 200.ahk.unused"

A lot of care have been taken in the creation of the installation process to be as Murphy proof as possible, so it should be quite safe. The file named "Install GD-DPS MOD.bat" is the starting point for everything else. The file "Setup.bat" handle the installer configuration settings. Once initial installation of the MOD is completed two uninstallation files are generated "Uninstall GD-DPS MOD.bat" and "CleanUp GD-DPS MOD.bat". The first completely revert the game to the source backup and then ask if the mod files themselves should be removed. The second one perform manual post installation removal of no longer needed files like the source backup and the mod files themselves should you want to free some (~40GB!) storage space.

Of course, unpacking, patching and repacking a complete game is not going to be the fastest nor the cheapest way to do things: On a very slow system, from an uncached game file sets, the whole process is going to take more than 40 minutes. On a decent one with the game on an SSD, I would assume the process to be in the 15-20 minutes range. Furthermore, expect a total data written count, for the first run only, to be close to 60GB... Once the source is created, each complete rebuild will write over the same data each time: Approximately that of the complete game ~10GB. The repacked GD-DPS moded game is substantially smaller than the original one simply because a lot of sound files, musics mostly, as well as the unneeded "GDX0 and GDX1 levels archives", gets removed from it.

GD-DPS can take advantage of multi-threaded execution, particularly for the deconstruction and reconstruction phases. It is, however, disabled by default, the reason being that having these stages handle files seemingly "out of order" can be confusing if errors or problems arises. Should you want to activate it, in the "Setup.bat" file, locate the two lines, near the end of the file, "SET FastDeconstruction" and "SET FastReconstruction" and replace 'False' with 'True'. This should only be done when you are fairly confident you understand how the deconstruction and reconstruction processes work.

Once you get the hang of it, you can skip large parts of the reconstruction if some data do not changes between rebuilds of the game archives. Executing just the sequence you need can be done via a specialized windows console, the GD-DPS console. To launch it, run the batch script file named "GD-DPS.bat". The GD-DPS console is responsible for setting the proper background parameters so that typed commands and scripts unfold in a planned manner. Any attempt to launch either a command batch file or one of the mod .AHK script from an explorer folder view won't work. That is by design: to prevent miss execution of potentially destructive, for the game state at least, actions.

There is a lot to cover and this is just the basics. If you are interested in learning more, I invite you to inspect the mods provided files. You can also check this mod thread on the Grim Dawn official forum.


***************************************

If you made your way here, then congratulation, and thank you for your interest. Here is a special something for you: Notepad++ AutoHotkey and Grim Dawn DBR language.

In Notepad++, under the language tab, click on the last item, "User Defined Language" > "Define your language..." In the window that appears,  near the top section, there is an "Import..." button. Click it and browse to the extracted XML files from the archive you just downloaded then click the "Open" button.

Lastly, put perhaps this should have been firstly, English is not my native language so you'll excuse any weird turns of phrases and the plethora of grammar errors...