Help with a new GUI Element Hook

[WizzardMaker]

I want to hook a new function, that draws the GUI Elements (as defined in the Menu/GUISetXXX.DAT files)

.text:0027240D                 mov     edx, [esp+508h+objectA] ; objectA (?)
.text:00272411                 mov     ecx, [esp+508h+objectB] ; objectB (?)
.text:00272415                 push    ebx             ; guiElement 
.text:00272416                 push    eax             ; surfaceHeight
.text:00272417                 movzx   eax, word ptr [esi+6] ; Move with Zero-Extend
.text:0027241B                 push    eax             ; surfaceWidth
.text:0027241C                 call    DrawGUIElement  ; <-- this function needs the hook (Original Address is 0x272600)
.text:00272421                 movzx   eax, byte ptr [ebx+18h] ; Move with Zero-Extend
.text:00272425                 add     esp, 0Ch        ; Add
.text:00272428                 add     eax, 0FFFFFFFCh ; switch 18 cases

But I can't quite figure out how to implement a hook in the style of the S4ModApi. Could you help me with that?

[The game does not render any GUI Elements when nop'ing that function call: ]

For reference, ebx points to the GUI Element

A GUI Element looks like this:

struct GUIElement
{
  WORD x;
  WORD y;
  WORD width;
  WORD height;
  WORD mainTexture;
  WORD valueLink;
  WORD unknown;
  WORD buttonPressedTexture;
  WORD unknown1;
  WORD unknown2;
  WORD tooltipLink;
  WORD tooltipLinkExtra;
  WORD currentEffects; //enum where the first 4 bits define which font style to use and last 4 bits define effects (Like pressed etc)
  WORD unknownData[5]
  //sizeof == 36
};

And what would be the easiest way to get the value of a static address?
The current .gfx file of the GUI Engine is located at [S4_Main.exe+0xE94814]+0xC98 - how should I retrieve that value?

[nyfrk]

Sorry for my late answer. I planned in writing a proper tutorial but looks like I don't have enough time doing that so I will give you a few hints now.

First, have a look at this commit. It shows all the necessary changes that are required for adding a new Hook.

The good thing is, that the instruction you want to hook is already 5 bytes large. So you can just replace the instruction with a CallPatch. You can see example code here at these lines: hlib::CallPatch(addr, (DWORD)__hook);

The __hook function is a __declspec(naked) function that has no calling convention. However I can see from the code you showed that it is a __fastcall function with 5 arguments. I usually add naked functions as they allow for easy stack hacking. However in your case it is probably easier to just not create a naked function and implement it like that:

hlib::CallPatch(addr, (DWORD)OnDraw);

Just create the OnDraw function:

void __fastcall CGuiBltHook::OnDraw(DWORD _0, DWORD _1, DWORD _2, DWORD _3, DWORD _4) {
	//TRACE; // we do not log this as it will be a mayor performance hit
	mutex.lock();
	auto observers = GetInstance().observers; // obtain a copy of all the observers since the callbacks may modify the vector
	mutex.unlock();
	BOOL discard = false;
	S4GuiDrawParams params; // create a struct at S4ModApi.h 

        // copy parameters to struct 
	params._0 = _0; 
        params._1 = _1;
        // ...

       // _4 is probably the GUIElement pointer. So you would unpack that too like this
       params.x = _4.x;
       params.y = _4.y;
       // ...

        // iterate observers
	for (auto& observer : observers) {
		discard |= ((LPS4GUIDRAWCALLBACK)observer.callback)(&params, discard);
	}

        // this will be the tricky part. You will now have to call the original function that the game would have called. 
        // You also must return its return value. Luckily the function you hooked does not appear to have a return 
        // value, so we don't have to think about that.
        if (!discard) {
              void (__fastcall *DrawFunc)(DWORD _0, DWORD _1, DWORD _2, DWORD _3, DWORD _4)
                  = 0x272600; // better use the S4_Main + 0xXXXXX notation since that will make it more robust if windows decides to load the Image at a different address. 
              DrawFunc(_0,_1,_2,_3,_4);
        } 
}

You can name the arguments properly as you seem to already found out what they do.

A GUI Element looks like this

Previously I added game structs to s4.h or directly to the .cpp File of the hook code. You can choose where you want to put it. Don't forget to put #pragma pack(push, 1) before the definition of the struct and add a #pragma pack(pop) after it. Please do not add game structs to S4ModApi.h, since that will make it more difficult for people importing the DLL from non-C contexts (eg C#). Especially if you require #pragma pack. That's the reason I always do the unpacking of game structs:

       params.x = _4.x;
       params.y = _4.y;
       // ...

The current .gfx file of the GUI Engine is located at [S4_Main.exe+0xE94814]+0xC98 - how should I retrieve that value?

I think I added a READ_AT function somewhere. You can use it to safely read arbitrary memory addresses. If you want to read more than 4 Bytes (eg a string), have a look at memget_s and memset_s. See here for details. These functions will not crash the game if you attempt to write/read at an invalid address. So [S4_Main.exe+0xE94814]+0xC98 would probably be implemented as:

auto ptr = READ_AT(S4_Main + 0xE94814);
auto val = auto ptr = READ_AT(ptr + 0xC98); 

how should I retrieve that value

I added these kind of functions to s4.h. It is probably the best place for now. However, you cannot really make that wrong. Its up to you, where you want to place/implement it :)

[WizzardMaker]

The game uses the ebx pointer from before the function call - how do I preserve that register?

[nyfrk]

From your excerpt i think that ebx is already pushed onto the stack. So it should be in your argument list. (The last argument)

void __fastcall CGuiBltHook::OnDraw(DWORD ecx, DWORD edx, DWORD surfaceWidth, DWORD surfaceHeight, DWORD ebx)

You do not have to preserve that register. __fastcall functions (by specification) are only allowed to change the eax, ecx and edx registers. All others are untouched or automatically preserved (the compiler does that for you when you decorate a function with __fastcall).

It gets tricky when you create a hook at a non-function call (replacing ordinary opcodes). Then we would also have to preserve xmm/fpu registers. But that's not needed here since you set your hook at an already existing function call. This is also the reason you do not have to preserve the eax, ecx and edx registers. Because the previous function (that you overwrite) is also allowed to change these registers.

Edit: Here are some more details on the spec. There you can also find a sentence about the register preservation:

The compiler generates prolog and epilog code to save and restore the ESI, EDI, EBX, and EBP registers, if they are used in the function.

[WizzardMaker]

Yeah, you're right, but I run into the problem, that the game seems to deadlock when my patch is applied. It has something to do with my function, because if I nop the original call, the game runs fine (just without the gui elements)

The hook gets correctly called; all values seem fine (The struct is filled with the correct values) but it just seems like somethings wrong.

It could be the stack pointer by the looks of it
Before the call:
ESP = 004FD64C
After the call:
ESP = 004FD658

Everything runs fine if I manually change the ESP to the previous value - but that seems like the incorrect way of doing it

[nyfrk]

hmm, thats 3 dwords you are off. I think we cannot use __fastcall because the game does expect the caller to clean the stack - but we do it in the callee. The caller must clean the stack.

.text:00272425 add esp, 0Ch ; Add

There is no build-in calling convention that allows us to easily do that (there would be cdecl, but that would not pass ecx and edx as arguments). We need a naked function and manually call your hook procedure.

The calling convention the game uses here is a "compiler invented convention". Meaning the compiler invented it on the fly for optimization purposes. Unfortunately this means, that we cannot make assumptions about xmm / fpu registers too. It is probably best to preserve them too. The rules for the compiler invented convention appear to be as follows:

• Arguments passed from right to left onto the stack
• ecx and edx being additional first arguments (similar to fastcall)
• Must preserve all registers except ecx, edx and eax
• Callee cleans the stack (similar to cdecl)

I suggest changing the calling convention of your hook procedure to stdcall. This is a little easier to call directly from assembler. Note the changed return value that is used to control whether to call the original function. Do not call the original function from within OnDraw (you actually could do this using an __asm block and preparing, executing and cleaning the call yourself within the OnDraw function, but I prefer doing this in a naked function).

BOOL __stdcall CGuiBltHook::OnDraw(DWORD ecx, DWORD edx, DWORD surfaceWidth, DWORD surfaceHeight, DWORD ebx);

DWORD OrigFunction; // you store the original function address here. Populate it before you install the call patch.

Then you create a naked function like this:

static void __declspec(naked) __onHook() {
	__asm {
                // first we preserve all the xmm registers onto the stack. That are 128 bytes. 
		sub     esp, 16
		movdqu[esp], xmm0
		sub     esp, 16
		movdqu[esp], xmm1
		sub     esp, 16
		movdqu[esp], xmm2
		sub     esp, 16
		movdqu[esp], xmm3
		sub     esp, 16
		movdqu[esp], xmm4
		sub     esp, 16
		movdqu[esp], xmm5
		sub     esp, 16
		movdqu[esp], xmm6
		sub     esp, 16
		movdqu[esp], xmm7

                // second we preserve the fpu registers onto the stack. That are another 128 bytes.
		sub     esp, 128
		fsave[esp]
                
               // at this point we increased the stack by 260 bytes (don't forget to consider the pushed return address for the call instruction). We also did not change any registers.

                // preserve the ecx and edx registers. We need them to later call the original function that we replaced. We
                // cannot call it from c context since we cannot declare it as fastcall or any other calling convention.
                push edx 
		push ecx

                // now we start preparing our hook procedure invocation. That means, we push all arguments right-to-left onto the stack. At this point we increased the stack by 268 bytes

		push [esp + 276]
		push [esp + 276] // note that the offset does not change as push already subtracts 4 from the esp register
		push [esp + 276]
		push edx  // the additional registers get passed as arguments too
		push ecx
		call CGuiBltHook::OnDraw  // we call your stdcall hook procedure, this is allowed to change eax, ecx and edx as well as the xmm / fpu registers

                // since we cannot use fastcall to call the original function in the hook procedure, we must call it from assembler. 
                // first restore the edx and ecx registers we preserved before
                pop ecx
		pop edx

                // restore the fpu and mmu registers. 
		frstor[esp]
		add     esp, 128
		movdqu  xmm7, [esp]
		add     esp, 16
		movdqu  xmm6, [esp]
		add     esp, 16
		movdqu  xmm5, [esp]
		add     esp, 16
		movdqu  xmm4, [esp]
		add     esp, 16
		movdqu  xmm3, [esp]
		add     esp, 16
		movdqu  xmm2, [esp]
		add     esp, 16
		movdqu  xmm1, [esp]
		add     esp, 16
		movdqu  xmm0, [esp]
		add     esp, 16

                // now we test the return value from our hook procedure. If it is zero we call the original function now.
                // otherwise we skip it (so that an observer can basically decide to nop it). Return values are passed in the eax 
                // register. al is just the least significant byte of the eax register. 

		test al, al

                // if the return value is non-zero, we skip calling the original function
		jnz lbl_skip_original

                // the hook procedure decided that we want to call the original function. Note that
                // we cannot just call it now since there is the return address of the last call pushed
                // onto the stack. So we must either pop it temporary or repush all the arguments. 
                // when poping it, we must decide a register that we are allowed to modify without
               // the game noticing it. So we just repush all the arguments directly from stack.
		push[esp + 12]
		push[esp + 12]
		push[esp + 12]
                // ecx and edx are already prepared
		call OrigFunction

                // since this function is not really a fastcall, we must (as the caller) clean the stack. We do this by increasing the stack
                // pointer by 12 bytes
                add esp, 12

		lbl_skip_original:

                // at this point the stack is off by 4 bytes, which is because of the initially pushed return value. 
                // since the game wants to clean the stack, we can just ret now, which will pop the address from
                // the stack and copy it to the IP register. The game will then pop the 3 dwords it previously 
                // pushed onto the stack.

		ret
	}
}

Make sure to let the call patch call your naked function:

OnHook = hlib::CallPatch(addr, (DWORD)__hook); 

I hope i did not miscount any of the push [esp + xxx] offsets. If bogus values are pushed onto the stack, try adjusting it.

Edit: If you happen to find documentation about the "compiler invented calling conventions" feel free to share it with me. Maybe there is a decorator that allows us to specify functions with custom calling conventions. Then we would not need many of the naked functions.