I'm trying to find out where the default operand size for an image is recorded in the .exe.

Background:

Instructions with immediate data generally have either 8, 16 or 32 bits of data encoded into the instruction. Instructions with 8-bits are easy to deal with, but 16/32 bit stuff is controlled by looking at a default operand size. This can be 16 or 32 bits. The size for the instruction is determined by checking for a prefix (0x66) which toggles the default.

Problem: the default size can be 16 or 32 bits, and I don't know where the default is specified. I suspect that IMAGE_NT_HEADERS.FileHeader.Characteristics (IMAGE_FILE_32BIT_MACHINE) is the flag that controls this, but I'm not sure: when I change that flag and disassemble the file in IDA, it still says "use32" in the segment data. It is of course possible that IDA means something else by 'use32'.

Anyway, if anyone can clarify, that would be splendid :)

Default operand size is set in the descriptor of the segment the code is in. I would think it would be an attribute on sections in an exe, not on the exe as a whole.

Although, I also know that 16-bit exes for DOS used different headers. They did not have the extended header that windows later added. I do not think there should be any IMAGE_NT_HEADERS at all in a 16-bit DOS executable. Those headers indicate a PE executable.

Well, there's IMAGE_SECTION_HEADER.Characteristics too. That has a flag (IMAGE_SCN_MEM_16BIT) but MSDN just lists it as reserved and changing it doesn't change IDA's output either. Pedump says 'purgeable' when I turn that flag on.

The best I've found by googling is:

[quote]
In protected mode, the processor determines the default operand
    size of the instruction it is executing by examining the "D-bit"
    in the CS segment descriptor. If D=0, all operand lengths and
    effective addresses are assumed to be 16 bits long. If D=1, the
    operands and addresses are 32 bits long.
    In real mode and in virtual 8086 mode, the default operand and
    address size is 16 bits (no descriptors in real mode).
[/quote]

But I'm not sure what or where the D bit is, and this applies to protected mode, and I'm not sure how that affects things.

I don't know if you've resolved your issue already or not (being about a 4-month-old topic), but I'll add my two cents anyways (if your issue is already resolved, then hopefully this'll be helpful to anyone else as well, since you didn't post that you've resolved your issue).

[list]
[li]I think the simplest way to find how to detect the operand size for the environment through a PE file is to compile a single project twice -- once for x86, and once more for a 64-bit platform, such as x64 and then compare the headers of the two files.
[/li][li] IDA 32-bit refuses to disassemble AMD64 (eg, x64 rather than IA64) PE files (version 4.8 says "Please use 64-bit IDA to load AMD64 files"). I seem to be unable to compile for IA64 at the moment, but I'm sure it would say a similar message.
[/li][/list]

Since I do not have IDA 64-bit (do you?), it seems like we'll have to use other tools. I've found PE Tools (use google) to be very helpful with ... other ... PE issues, and it seems to have a lot of capabilities. So, I opened it up and opened up a PE compiled for x86 and told it to compare it against a PE compiled for x64. It said that it does not currently support that function for "PE+" files, which could give you another thing to search for on google: PE+, aka extended portable executable. I did a brief search on google, and the PE+ format does seem to be rather underdocumented.

That's only a little trouble anyways, since it's still capable of opening and displaying a PE+ file, having dual screen does help here, because it allows me to see the contents of both PE listings, while still having a fair amount of room to type up this reply...  ;)

In any case, the first data that I see is the DOS header, which appears to be the same for both... with one minor difference: the PE Address pointer (or as you may know it, IMAGE_DOS_HEADER.e_lfanew) is at 0xE0 for the 32-bit file, but is at 0xF0 for the 64-bit file... there's obviously some new data that I probably cannot see.

The NT headers are also fairly identical, with a few more exceptions... the IMAGE_FILE_HEADER.Machine is 0x014C for the 32-bit PE and is 0x8664 for the 64-bit one. Both are defined in winnt.h, one as IMAGE_FILE_MACHINE_I386 and the other as IMAGE_FILE_MACHINE_AMD64. Another difference is that the IMAGE_FILE_HEADER.SizeOfOptionalHeader is 0xE0 for the 32-bit header and is 0xF0 for the 64-bit header (odd that they're the same values as IMAGE_DOS_HEADER.e_lfanew... or is it? You'll note that in winnt.h, IMAGE_SIZEOF_NT_OPTIONAL32_HEADER is defined as 224 and IMAGE_SIZEOF_NT_OPTIONAL64_HEADER is defined as 240, which are exactly the same as those hexadecimal numbers). In the characteristics of the two, the 32-bit PE file's is (IMAGE_FILE_RELOCS_STRIPPED | IMAGE_FILE_EXECUTABLE_IMAGE | IMAGE_FILE_32BIT_MACHINE), whereas the 64-bit's is only IMAGE_FILE_EXECUTABLE_IMAGE. The last difference between the two is that the 32-bit PE file has 4 sections, whereas the 64-bit one has 5 (looking at the sections, the 64-bit PE file's extra section is a relocation section).

The IMAGE_OPTIONAL_HEADER is where things get sticky: in winnt.h, IMAGE_NT_HEADER has two different versions: IMAGE_NT_HEADERS64 and IMAGE_NT_HEADERS32, the only difference between the two is that the OptionalHeader variable is defined as IMAGE_OPTIONAL_HEADER64 and IMAGE_OPTIONAL_HEADER32, respectively. Windows' PE loader does use that section to define where executable code should be based at, so that's not really that surprising. In any case, I'll list the differring variables between the 32-bit and the 64-bit IMAGE_OPTIONAL_HEADER structures:

[list]
[li].ImageBase
[/li][li].SizeOfStackReserve
[/li][li].SizeOfStackCommit
[/li][li].SizeOfHeapReserve
[/li][li].SizeOfHeapCommit
[/li][/list]

Only the above variables are different between the two, and they are all DWORDs in the 32-bit version, whereas they are ULONLONGs in the 64-bit version (which is again, unsurprising).

In any case, that's only the size differences... the difference in values are that the 32-bit IMAGE_OPTIONAL_HEADER.Magic is 0x10B, whereas the 64-bit one is 0x20B. In the winnt.h header, those are defined as IMAGE_NT_OPTIONAL_HDR32_MAGIC and IMAGE_NT_OPTIONAL_HDR64_MAGIC, respectively. If I were to just the maximum operand size on a single thing available in the header so far, it would be this magic value, since the magic value also defines the sizes of the variables I've already listed. There is a bunch of other information which is different between the two IMAGE_OPTIONAL_HEADERs, but I'd say that the most relevant is that the Major SubSystem Version and Minor SubSystem Version is 4.0 for the 32-bit one and 5.2 for the 64-bit one. In addition, the DllCharacteristics variable is zero for the 32-bit one, whereas it's 0x0200 (IMAGE_DLLCHARACTERISTICS_WDM_DRIVER, see here) for the 64-bit header... which is rather odd, because I the 64-bit PE is an EXE (not a DLL)... I did not set /DRIVER:WDM in the linker command line and in fact the "Driver" option in the configuration (Project Properties -> Configuration Properties -> Linker -> System -> Driver) is specifically set to "Not Set". That's probably a bug in Visual Studio 2005.

The doesn't seem to be anything of interest in the IMAGE_DATA_DIRECTORY arrays.

All of that side, I would rather use something a little more reliable for detecting the operand size, which assuming that you're writing in assembly it shouldn't be too hard (although I'm putting my foot in my mouth, being that I haven't tried it): simply set up an invalid instruction exception handler, and then execute a 64-bit-only instruction.

As for your reference to the "D bit", there's very little references as to exactly what the D bit is, let alone how to access it. I did find this, however: http://www.osdever.net/tutorials/pdf/pmode.pdf
[quote]For stack segments, the Default Size bit is also known as the B (Big) bit, and controls whether 16- or 32-bit
values are pushed and popped. For code segments, the D bit indicates whether instructions will operate on
16-bit (D=0) or 32-bit (D=1) quantities by default.[/quote]To me, that would indicate that the "D bit" is shortform of the "Default Size" bit. The PDF goes on to describe the descriptors to which I believe your quote is referring.

I do hope this helps.

Nice post, I was going to respond with this (With that same article, wtf) but opted not to due to how old it was. :)

To quote from the PECOFF specification(found here: http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx )

The characteristics field has the following 2 flags:

[quote]
IMAGE_FILE_16BIT_MACHINE 0x0040 Use of this flag is reserved for future use.
IMAGE_FILE_32BIT_MACHINE 0x0100 Machine based on 32-bit-word architecture.
[/quote]

So, the reason that IDA still dis-assembled your binary as 32-bit, is because even though you unset the 32-bit flag you didn't set the 16 bit one...and I would guess that IDA, in the absence of explicit specification, defaults to 32-bit disassembly.