Because it's confusing to me right now, I'd rather just do this in inline asm if it's possible. I'm needing to declare a 5 byte variable (arrayed db?). So can you do this in inline asm?

[code]
void func() {
  __asm {
          my_var db 5 dup(?)
          ....
        }
}
[/code]

and use that variable in the following code? I'm just not sure what the C equiv of that would be.

It'd just be char[5] or u8[5].

I think you can use a label and "emit". like something like

[code]__asm {
  my_var:
    __asm emit 0x00
    __asm emit 0x00
    __asm emit 0x00
    __asm emit 0x00
    __asm emit 0x00
}[/code]

if you don't care about its physical location inside your assembly, just declare it as a C char[] outside the __asm block.

Ah, well, I've been workin on this WriteMem function, and I'm getting an error saying "illegal operand type" on the two lines I have pointed out below...

[code]
void __fastcall WriteMem(DWORD AddressToPatch, DWORD AddressToUse, DWORD PaddingSize) {
    PaddingSize = PaddingSize == NULL ? 0 : PaddingSize;

DWORD OldProtect;
    VirtualProtect((LPVOID)AddressToPatch, 5 + PaddingSize, PAGE_EXECUTE_READWRITE, &OldProtect);

BYTE  CallOperand = 0xE8;
DWORD DataBuffer  = (AddressToUse - (AddressToPatch + 5));
    __asm {
        MOV BYTE PTR AddressToPatch, CallOperand  <-- improper operand type?
        MOV AddressToPatch+1, DataBuffer                <-- improper operand type?
    }

    for (DWORD x = PaddingSize; x > 0; x--) {
        __asm {
            MOV BYTE PTR [AddressToPatch+1+x], 90h
        }
    }
    VirtualProtect((LPVOID)AddressToPatch, 5 + PaddingSize, OldProtect, &OldProtect);
}
[/code]

How's this an improper operand type?

Why not just do:[code]
*(uint8_t*)AddressToPatch = 0xe8;
*(uint32_t*)AddressToPatch = DataBuffer;
memset(reinterpret_cast<uint8_t*>(AddressToPatch) + 5, 0x90, PaddingSize);
[/code]

because ive been told that that extra call to memset is sexually abusing the stack.

Do you know anything about "trampoline" functions, or something? So I can automatically determine the length of the memory address I am going to overwrite, and if its longer - NOP the remaining data, or if its shorter I can repair the overwritten instruction? i guess i know my jmp or call with always be 5 bytes, how can i read the length of the memoryaddress im overwriting?

Who fed you that line about memset?

Trampolines typically refer to tiny temporary functions created on the stack, and as such should be avoided wherever possible since they're incompatible with NX.  What you want is to disassemble the instruction(s) in the area you're overwriting, so that you can identify their length and thus make appropriate repairs.  Unless you need this to be very general purpose, it's usually cheaper just to look up the lengths by hand in a disassembler and hardcode the results in the program.

Yeah, that's what I have been doing. Just hardcoding in lengths. Seems to be working fine - was just wondering if there was a way to make this function of mine a little more universal

I guess I'll reply because it's another question.

I'm receiving an error related to my asm, telling me it's detecting a newline? Why? :-(

[code]
void DisplayStatInfo(void) {
DWORD offset = Offsets.PrintText;
char *text = "leeeeeeet";

__asm {
    xor edx, edx
          mov ecx, text
          call offset                      <-- new line?
}
}
[/code]

and my definition of PrinText...

[code]
struct MemoryAddresses {
const static DWORD PrintText = 0x004e5d80;
const static DWORD PrintTextPatch = 0x00473b68;
};

MemoryAddresses Offsets;
[/code]

I don't recognize the calling convention you're using (ecx and edx but no eax?), but you could just as easily typedef a function pointer and avoid using assembly like this at all.

something like:

[code]
typedef void( __fastcall *pfFoo)(int, int);

unsigned int address = 0xf00;

pfFoo foo = reinterpret_cast<pfFoo>(address);
foo(12, 23);
[/code]

I guess I don't know much about asm, and calling convetions.

This is just how I see Brood War doing it.

Brood War uses _fastcall, which is rather inefficient about register usage.  It'd be much better to do it the gcc way, which is: eax, edx, ecx.

So just about every instruction should use eax, edx and ecx?

[quote author=warz link=topic=14916.msg151879#msg151879 date=1146642218]
So just about every instruction should use eax, edx and ecx?
[/quote]

No :p  Maybe calls to other functions or addresses, probably

Well, this is exactly how brood war calls its print text function

[code]
__asm {
          PUSH text
          MOV BYTE PTR DS:[0x6CB51D], 11
          MOV WORD PTR DS:[0x6CB544], CX
          MOV WORD PTR DS:[0x6CB548], 276
          MOV WORD PTR DS:[0x6CB546], DI
          CALL offset
}
[/code]

It has compiled before, but it now saying it is finding a 'newline' at my call instruction. I don't know why :-(

Edit: Haha, nevermind. 'offset' is a keyword.

[quote author=Kp link=topic=14916.msg151850#msg151850 date=1146623235]
Brood War uses _fastcall, which is rather inefficient about register usage.  It'd be much better to do it the gcc way, which is: eax, edx, ecx.
[/quote]

It's also worth noting that in many places, __fastcall and register calling conventions for x86 are worse than just stack calling conventions.

In most non-leaf functions you will end up having to spill parameter values to the stack to preserve them across subsequent function values anyway (because eax/ecx/edx are volatile for most x86 calling conventions).  So, you don't really gain anything in the common case of a non-leaf function that uses its parameters across function calls.

This is why if you look at highly optimized x86 code (like, say, ntoskrnl) then you'll find that mostly you *don't* use register callling conventions except for leaf functions or very simple functions.

For better designed architectures and calling conventions (like x64), however, register parameter passing is a much better idea.

True.  Just to give some cases where _fastcall is useful:

C++ member functions which make extensive use of member variables often benefit from _thiscall (which is a variant of _fastcall) by pushing a stable register to the stack and keeping this in the stable register for the life of the function.  Procedural code which followed the same concept (having 1 or 2 high importance objects that're referenced often) would also benefit from _fastcall.

Redirection functions, such as the one below, also benefit from _fastcall.
[code]
int _fastcall foo(int x, int y)
{
    return bar(x, y, 3);
}[/code]

My original comment was mainly about the silliness of the number and order of registers used in _fastcall.  Since ecx is used as the counting register for repeatable instructions, it's best to allocate it last among _fastcall registers.  By doing so, 1 and 2 argument register-called functions keep ecx clear for immediate use in loops.  Incidentally, gcc allows you to specify anywhere from 0 to 3 arguments to be passed in registers.  If you expect the compiler will want immediate use of ecx, you could opt to pass only two arguments in registers (even if you had many more arguments), so that the called function would not need to spill ecx to the stack.

Aside from spilling concerns, there's no reason not to use eax as an additional argument register.  It's unstable anyway, so functions that benefit from register-call will do better having eax available.  If they need many arguments quickly, consuming eax reduces the number of stack references required.  If they only need to pass 2 arguments quickly, eax can be used in preference to ecx, thereby avoiding consuming the counting register.

Just FYI: If you use a library like libdisasm, disassembling the instructions to determine their length would become pretty easy, as would making appropriate repairs. Might make your program a little more bulky though -- uses lots of big tables in memory to decode instructions.