I'm just messing around with inline assembly in MS VC++ 6. Here is the code I am working on:

[code] __asm {
mov eax, 0
push eax
call GetModuleHandle
mov [hInstance], eax
mov eax, [hInstance]
push eax
mov eax, IDD_MAINFORM
push eax
mov eax, NULL
push eax
mov eax, MainDlgProc
push eax
call DialogBoxParam
}[/code]

Now, the error I get is:

[quote]error C2400: inline assembler syntax error in 'second operand'; found ')'[/quote]

Now, I'm not VC++ expert or asm expert, so I really have no clue why this would happen. For all I know the answer to this is very simple. I'm also not sure where this ')' is located when the error refers to it. Hopefully someone can help me out with this.

Can you tell us at least which line?

I would guess IDD_MAINFORM since it's #defined in a resource file.  But telling us the line of the error may be helpful...

Sorry about that. I meant to put which line, I just forgot. Here it is:

[code]mov eax, NULL[/code]

This is just a wild guess but instead of NULL maybe 'False' ?

[quote author=Hell-Lord link=topic=16593.msg167666#msg167666 date=1175913756]
This is just a wild guess but instead of NULL maybe 'false'
?
[/quote]

It just tells me that the label 'false' is undefined. Keep in mind that this is the first time I've ever tried messing with inline assembly, so I'm probably not doing something else that I should have done.

Replace it with 0.  I'm thinking that NULL is in stdio.h:

[code]
#define NULL    ((void *)0)
[/code]

Yep it is in the stdio.h header.

Just to my knowledge :
[code]__asm ("assembly code");[/code]

Not sure what difference () will make instead of {}

Also i believe you can entirely remove 'NULL'
so it will look like

[code] mov      eax[/code]

Or as is a really common practice:

[code]
xor eax, eax
[/code]

Replacing NULL with 0 causes the application to crash upon execution. Doing xor eax, eax also causes the application to crash upon execution. I'm beginning to think that I've done something else wrong? I'll let you know if I figure anything out.

By the way is that an Intel Processor you have right?

[Intel-based/X86/x64. Intel and AMD have the same base system, so saying Intel isn't quite the correct terminology.]

But i gather he knew.

It's not about knowing, Paul, it's about terminology.

Thanks for the lesson Andy :D

Try:
[code] mov      eax, edx[/code]

Well..he could of been asking it he was using an intel processor, never mind the architecture. :)

But I am just being dense on purpose.

Thats exactly what i was asking. Didn't sound right? :P

Correct me if I'm wrong, but isn't the processor brand unimportant?

There are processor brand specific features in x86 and certain things different processor models from either Intel or AMD support.

setting something to null doesn't sound very much like a feature...

[quote author=Yegg link=topic=16593.msg167663#msg167663 date=1175912210]
I'm just messing around with inline assembly in MS VC++ 6. Here is the code I am working on:

[code] __asm {
mov eax, 0
push eax
call GetModuleHandle
mov [hInstance], eax
mov eax, [hInstance]
push eax
mov eax, IDD_MAINFORM
push eax
mov eax, NULL
push eax
mov eax, MainDlgProc
push eax
call DialogBoxParam
}[/code]

Now, the error I get is:

[quote]error C2400: inline assembler syntax error in 'second operand'; found ')'[/quote]

Now, I'm not VC++ expert or asm expert, so I really have no clue why this would happen. For all I know the answer to this is very simple. I'm also not sure where this ')' is located when the error refers to it. Hopefully someone can help me out with this.
[/quote]

Yegg, there's a few things I see here. DialogBoxParam is most likely stdcall, meaning that when parameters are pushed onto the stack, they're pulled off in the same stack-like fashion - first on, last off. So, essentially, you have to push the parameters in a reverse order. Currently, you are passing the parameters backwards. See this page, it's very helpful while learning asm (or i thought, anyways).

Another thing you might want to try is, if the calling convention correction doesn't fix anything, to push the value stored inside hInstance, rather than the memory address pointed to by that value. :P

Last, using null should be ok, but i'd get into the practice of xor'ing eax by eax.

I would also attempt using the AT&T syntax instead of the Intel one.

You can't set an WORD to null. You set it to 0. If you set it to null, literally, a chunk of your processor would disappear, because it'd be turned to null. Think about it. :)

[tt]xor eax, eax[/tt] is the proper way to execute that instruction. If you're crashing, it's something else.

Note that:

[code]mov [hInstance], eax
mov eax, [hInstance]

.. is equivalent to ..

*hInstance = eax;
eax = *hInstance;[/code]

I can't see if you use hInstance anywhere else, but if you don't the first line is unnessary, and the second line is unnecessary no matter what you do. I'm not sure if you know (I figure you do, but what's it hurt?), but the return value of a function is put to eax, so..

[code]call GetModuleHandle

.. is equivalent to ..

int eax = GetModuleHandle();[/code]

EDIT -

I took the liberty of some personal investigation of this. Here's your code, edited up a bit and commented with current stack state:

[code]XOR  EAX, EAX
PUSH EAX ; stack: 0
CALL GetModuleHandle
PUSH EAX ; stack: hInstance, 0
MOV  EAX, IDD_MAINFORM
PUSH EAX ; stack: IDD_MAINFORM, hInstance, 0
XOR  EAX, EAX
PUSH EAX ; stack: 0, IDD_MAINFORM, hInstance, 0
MOV  EAX, MainDlgProc
PUSH EAX ; stack: MainDlgProc, 0, IDD_MAINFORM, hInstance, 0
CALL DialogBoxParam ; DialogBoxParam(MainDlgProc, 0, IDD_MAINFORM, hInstance, 0);[/code]

According to this MSDN page, DialogBoxParam is called as [tt]INT_PTR DialogBoxParam(HINSTANCE hInstance, LPCTSTR lpTemplateName, HWND hWndParent, DLGPROC lpDialogFunc, LPARAM dwInitParam);[/tt] however you appear to mixing up argument's being passed to it. You're sending it the right number of arguments, so I don't think you destroyed the stack, but calling it incorectly is still reason for the program to crash, I believe.

AT & T syntax ain't bad so what do you have against it?

Maybe I'm thinking of a different syntax, whatever GNU asm uses, but I tried it out once and could never understand it.

Yea it is a little different.....

Intel Snytax:
[code] mov        eax, edx [/code]   

AT & T:
[code] mov        %edx, %eax[/code]

Yeah, I don't know, I guess it's an aquired taste, but I just don't like it.

"Men are always in a rut, and they like it that way. If you want to progress in life, then there goes your nice comfortable rut."
- Dan Conner (paraphrased), Rosanne.

[quote author=Joe[x86] link=topic=16593.msg167746#msg167746 date=1176093611]
Yeah, I don't know, I guess it's an aquired taste, but I just don't like it.

"Men are always in a rut, and they like it that way. If you want to progress in life, then there goes your nice comfortable rut."
- Dan Conner (paraphrased), Rosanne.
[/quote]

lol :)

anyway was bored and found this.....
[quote] +------------------------------+------------------------------------+
|      Intel Code                |      AT&T Code                    |
+------------------------------+------------------------------------+
| mov    eax,1                  |  movl    $1,%eax                  | 
| mov    ebx,0ffh              |  movl    $0xff,%ebx                | 
| int    80h                        |  int    $0x80                    | 
| mov    ebx, eax              |  movl    %eax, %ebx                |
| mov    eax,[ecx]            |  movl    (%ecx),%eax              |
| mov    eax,[ebx+3] |  movl    3(%ebx),%eax              |
| mov    eax,[ebx+20h]      |  movl    0x20(%ebx),%eax          |
| add    eax,[ebx+ecx*2h] |  addl    (%ebx,%ecx,0x2),%eax      |
| lea    eax,[ebx+ecx]        |  leal    (%ebx,%ecx),%eax          |
| sub    eax,[ebx+ecx*4h-20h] |  subl    -0x20(%ebx,%ecx,0x4),%eax |
+------------------------------+------------------------------------+ [/quote]


***Edit***
fixed the quote a little.

[quote author=Joe[x86] link=topic=16593.msg167739#msg167739 date=1176092182]
You can't set an WORD to null. You set it to 0. If you set it to null, literally, a chunk of your processor would disappear, because it'd be turned to null. Think about it. :)[/quote]
WTF?  Null is another name for 0.  You can xor dx, dx, or xor dl, dl.  You can set any size of a value in a register or memory/stack variable to null.

Joe, really man, try to not spout false information.

Joe, also, please, look at the MSDN page for DialogBoxParam, and check out the parameters. You're not even passing the correct parameters at the correct location. Did you say you tested that code? How did this work? The hInstance variable should be the last item pushed onto the stack, not the dialog box procedure function pointer - it seems like you have them out of order completely, because the dialog proc function pointer shouldn't be first onto the stack or last for that matter.

[quote author=ŖėåłïťŷŔĩρρŀě link=topic=16593.msg167707#msg167707 date=1176056246]
setting something to null doesn't sound very much like a feature...
[/quote]

I never said it was.

[quote author=♥ link=topic=16593.msg167754#msg167754 date=1176107135]
Joe, also, please, look at the MSDN page for DialogBoxParam, and check out the parameters. You're not even passing the correct parameters at the correct location. Did you say you tested that code? How did this work? The hInstance variable should be the last item pushed onto the stack, not the dialog box procedure function pointer - it seems like you have them out of order completely, because the dialog proc function pointer shouldn't be first onto the stack or last for that matter.
[/quote]

That's not my code. I simply commented his, said what was on the stack at the time, and what he was doing. I then went on to link the MSDN page for DialogBoxParam and say he needed to check out the parameters. :P

[quote author=MyndFyre[vL] link=topic=16593.msg167753#msg167753 date=1176103118]
[quote author=Joe[x86] link=topic=16593.msg167739#msg167739 date=1176092182]
You can't set an WORD to null. You set it to 0. If you set it to null, literally, a chunk of your processor would disappear, because it'd be turned to null. Think about it. :)[/quote]
WTF?  Null is another name for 0.  You can xor dx, dx, or xor dl, dl.  You can set any size of a value in a register or memory/stack variable to null.

Joe, really man, try to not spout false information.
[/quote]

L2joke. <3

oh, well, way to restate what i said right before u n00b ^^

Oh. I didn't even read your post, btw. I replied from the first page. :P

[quote author=MyndFyre[vL] link=topic=16593.msg167673#msg167673 date=1175930337]
Or as is a really common practice:

[code]
xor eax, eax
[/code]
[/quote]
I wouldn't recommend optimizing assembly unless there's a really good reason to do so.

mov eax, 0

is the best way to do it. When you start adding your own optimizations, your code becomes less readable.

If you're trying to write optimized assembly, it's generally better to just write C and ask your compiler to optimize, it'll do a better job than you ever would anyways.

i don't feel like xor eax, eax and mov eax, 0 are too different, as far as code recognition goes. it's not tough to see both, and recognize what both are doing.

I agree with warz(?), everytime I see xor eax, eax I usually think zero.

[quote author=Dr. ♥ link=topic=16593.msg168226#msg168226 date=1177223412]
i don't feel like xor eax, eax and mov eax, 0 are too different, as far as code recognition goes. it's not tough to see both, and recognize what both are doing.
[/quote]
That's true, because you're accustomed to it. But that's not necessarily true for everybody (like the guy in this thread), and I definitely wouldn't recommend that somebody does it.

Well if he knew bitwise operations well he would know xoring a number by itself results in 0. And it gets stuck on one of the two registers... both are eax... real hard choice to pick... :P

Now, am I an idiot to think that [tt]move eax, 0[/tt] and [tt]xor eax, eax[/tt] both use a single processor cycle and are exactly the same stress-wise? I can see code size being different, but I think I'm right about the first part.

[quote author=Newby link=topic=16593.msg168248#msg168248 date=1177262433]
Well if he knew bitwise operations well he would know xoring a number by itself results in 0. And it gets stuck on one of the two registers... both are eax... real hard choice to pick... :P
[/quote]
So why spend the extra human time figuring it out? One of the things you're taught if you ever want to be a real programmer is NOT to optimize your own code. Readability is far more important than optimization.

[quote author=Joe[x86] link=topic=16593.msg168252#msg168252 date=1177264561]
Now, am I an idiot to think that [tt]move eax, 0[/tt] and [tt]xor eax, eax[/tt] both use a single processor cycle and are exactly the same stress-wise? I can see code size being different, but I think I'm right about the first part.
[/quote]
No Intel instruction uses one processor cycle. And xor uses less than mov, but not a significant amount.

[quote author=Joe[x86] link=topic=16593.msg168252#msg168252 date=1177264561]
Now, am I an idiot to think that [tt]move eax, 0[/tt] and [tt]xor eax, eax[/tt] both use a single processor cycle and are exactly the same stress-wise? I can see code size being different, but I think I'm right about the first part.
[/quote]

But they're not exactly the same stress-wise.  [tt]mov eax, 0[/tt] is 5 bytes because the 0 is stored four bytes wide, [tt]xor eax, eax[/tt] is one.  Otherwise it wouldn't be much of an optimization.

Wouldn't xor eax, eax be slower? the value of xor eax, eax needs to be calculated where mov eax, 0 moves a constant value into eax. Maybe not by very much (if anything at all) but yes, xor eax, eax is far more efficient anyways.

*sigh*

[quote author=brew link=topic=16593.msg168276#msg168276 date=1177281370]
Wouldn't xor eax, eax be slower? the value of xor eax, eax needs to be calculated where mov eax, 0 moves a constant value into eax. Maybe not by very much (if anything at all) but yes, xor eax, eax is far more efficient anyways.
[/quote]Uhh...an operation can't be slower AND simultaneously more efficient than another operation...

[quote author=iago link=topic=16593.msg168244#msg168244 date=1177254107]
[quote author=Dr. ♥ link=topic=16593.msg168226#msg168226 date=1177223412]
i don't feel like xor eax, eax and mov eax, 0 are too different, as far as code recognition goes. it's not tough to see both, and recognize what both are doing.
[/quote]
That's true, because you're accustomed to it. But that's not necessarily true for everybody (like the guy in this thread), and I definitely wouldn't recosmmend that somebody does it.
[/quote]

xor reg, reg is pretty much the de-facto way to zero out a register as far as x86 goes, so I would tend to disagree in this case.

I wouldn't advise being overly "tricky" with assembler without good reason, but there are a couple of cases where some instructions have "well-understood" implied connotations beyond their "original intended" use.

To be realistic, the "xor reg, reg" construct is so widely used that all but the most inexperienced at x86 assembler will recognize what it means.

[quote author=iago link=topic=16593.msg168258#msg168258 date=1177267272]
No Intel instruction uses one processor cycle. And xor uses less than mov, but not a significant amount.
[/quote]

Ah.. how's that so? I've never known that.

[quote author=brew link=topic=16593.msg168276#msg168276 date=1177281370]
Wouldn't xor eax, eax be slower? the value of xor eax, eax needs to be calculated where mov eax, 0 moves a constant value into eax. Maybe not by very much (if anything at all) but yes, xor eax, eax is far more efficient anyways.
[/quote]

You can't do mov eax, 0 with shellcode, can you? I would personally use xor eax, eax over mov eax, 0 just because of that.

Most nontrivial shellcode today will be prefixed by a small decoder program, so that the bulk of it need not be written with the constraints of the buffer (e.g. no nulls, no uppercase, and soforth).

[quote author=Skywing link=topic=16593.msg168287#msg168287 date=1177293782]
xor reg, reg is pretty much the de-facto way to zero out a register as far as x86 goes, so I would tend to disagree in this case.

I wouldn't advise being overly "tricky" with assembler without good reason, but there are a couple of cases where some instructions have "well-understood" implied connotations beyond their "original intended" use.

To be realistic, the "xor reg, reg" construct is so widely used that all but the most inexperienced at x86 assembler will recognize what it means.
[/quote]
That may be true, but if I was teaching somebody assembly, I definitely wouldn't teach them shortcuts like that. I might teach them to recognize it, because you're right that it's very common, but I wouldn't encourage them to do it that way.

[quote author=Joe[x86] link=topic=16593.msg168295#msg168295 date=1177298758]
Ah.. how's that so? I've never known that.
[/quote]
How? By taking up more than one cycle. There really isn't much more to say on that.

[quote author=Newby link=topic=16593.msg168296#msg168296 date=1177299200]
You can't do mov eax, 0 with shellcode, can you? I would personally use xor eax, eax over mov eax, 0 just because of that.
[/quote]
First of all, who cares about shellcode? Shellcode is special code that has special limitations, which are often bypassed with, like Skywing said, a decoder (aka, polymorphic shellcode).

But in trivial shellcode, it depends on the buffer. Often, you can have a null character in shellcode if you want, as long as it isn't required that your code is used like a null-terminated string somewhere. In a simple buffer overflow, you're jumping straight to the shellcode that was entered, so having null characters within the string is actually fine.