I've been told it's the player's character file (D2S), but it doesn't appear to be a strict carbon-copy of it. Anyone have any ideas?

(This relates to Open Battle.net, btw)


Cheers,

Matt.

I think there is a 6 byte header, 1st byte been the packet message ID (0x6C), 2nd byte being the lengh of the data appending the header, then some dword.
The server would then insert the character data into the game, when the client sucessfully 0x6B's to enter the game.
Then every 5min or so, the server should ecco the updated character data to the client (with 0xB0's I think) so that the client can auto save the character. (there is also somthing along these lines you must do in responce to a player quiting the game (0x69) other wise there character wont save)
This is pretty much exacly how closed D2GS's function,but they request and save the data to a character server on the network.

Just a side note: Most messages in the D2GS protocol are fixed lengh, and the ones that are not, normaly have a number in the message to calculate the lengh.

hope this helps

Ah, thanks for the input. I spoke to Shadow Dancer on MSN, he cleared some things up aswell. Heh, yeah I figured the same that closed works in a similiar way except it's the Battle.net server uploading the character.


Hmm, do you think it's possible to drop the auto-saving? I mean, not in a technical sense (kernel-mode filter driver seems sensible) but would the client have a 'fit', so to speak, if it diden't receive the updated character at those 5 minute intervals?

The plan is that I'm writing (or have actually already written) an administrative tool for D2 open games, initially because an old friend asked for it, and it seems like a good excuse to get back to playing some D2 :) and also instigated the idea for a uni project (d2 game-server).

Anyway, as I was saying I just figured it seems like a serious waste of bandwidth uploading possibly 8 player's 4KB characters every now and again. So maybe it would be a good feature to enable it's disablement.


EDIT:
Oh, by the way, you wouldn't happen to know if there's any documentation on the .D2S (1.11b) file format do you? After a long time of searching I've only found a few tidbits, and are actually pretty incorrect.

[quote author=Win32 link=topic=16810.msg170386#msg170386 date=1182659249]
Oh, by the way, you wouldn't happen to know if there's any documentation on the .D2S (1.11b) file format do you? After a long time of searching I've only found a few tidbits, and are actually pretty incorrect.
[/quote]

I did some work on this a while ago.  A lot of the documentation I found online was pretty good for the most part -- the things that are wrong were related to some items and also the way stat points are stored.

If you are interested, I can send you some C# code that I wrote to read/parse most of the file.

Wow, yeah that would be fantastic YaBB. Would it also be possible for you to direct me to some documentation on the file format? So I know what I'm working with.


Thanks a bunch.


EDIT: Oh and, one last thing, you wouldn't happen to know where the documentation (thread I believe it's meant to be) went on C->S D2GS messages? The link within the thread (https://davnit.net/bnet/vL/index.php?topic=11756.0) is broken.

I think the webpages I used were mostly:
http://www.xmission.com/~trevin/DiabloIIv1.09_File_Format.shtml
and
http://www.ladderhall.com/ericjwin/html/udietooinfo/udietood2s.html

The source code for my project is available here.  It's not particularly well written or well commented, but it worked ok last time I tried it.  I had some trouble with the UI since I think the version of GTK that monodevelop uses has changed I wrote it with originally, but you're probably not interested in that stuff :)

[quote author=Win32 link=topic=16810.msg170386#msg170386 date=1182659249]
Hmm, do you think it's possible to drop the auto-saving? I mean, not in a technical sense (kernel-mode filter driver seems sensible) but would the client have a 'fit', so to speak, if it diden't receive the updated character at those 5 minute intervals?

The plan is that I'm writing (or have actually already written) an administrative tool for D2 open games, initially because an old friend asked for it, and it seems like a good excuse to get back to playing some D2 :) and also instigated the idea for a uni project (d2 game-server).

Anyway, as I was saying I just figured it seems like a serious waste of bandwidth uploading possibly 8 player's 4KB characters every now and again. So maybe it would be a good feature to enable it's disablement.
[/quote]
Ye, the client has no problems with this :)
Ages ago, when I was writeing a kind of closed UMS/Custom D2GS, but testing it on open, I just ignored the C>S 0x6C's, and loaded defalt character data for the player.
But I didnt want to make the players over-write there existing character with a crappy lvl 1 defalt'ed character, so i didnt send any 5min save updates, and just disconnected them on 0x69, so that nothing was saved and there character was as it was before they enterd the game :)

Ah, that's good to hear :)

A question I should've found the answer to before I started my D2GS project -- Is there enough information available to actually pull-off writing an open D2 game-server?

Due to the server is going to be strictly for PvP purposes I would assume there will be no need to support alot of the game's functionality, such as Quests.

Or am I missing something?

hmm yes and no, alot of the more complex stuff is doted around.
The main packets that may cause some problems will be packets like 0x18, 0x27, 0x95, 0x96, 0x9C, 0x9D, 0xA8, 0xA9, 0xAA etc, as they have most of the infomation saved in bitfields.

I think as a PvP server would go, the 1st big problem would be generating the map. (with S>C 0x03, then 0x07 and 0x08's)
(useing a few ready-generated combinations is always a good/quick way to do this)
Then the next big problem would be emulating all 500 - 600 differnt skills and there missiles/effects (about 300 - 400 if you disclude NPC skills -- see skills.txt and Missiles.txt in d2data.mpq in the \excel\ folder)
Then the next big task would be to manage all the players stats and calculating damage done from 1 player to another.

It really can get as complex and deep as you want it to :)

I strongly advise making a gateway to sit between D2 and an open game, so you can monitor traffic from S>C and C>S as it will proove invalueable later on, when you need to check somthing, or figger somthing out.
Kind of like setting a socket to listen on 127.0.0.2:4000, then loading up d2, hosting a lan game, then loading up another d2, connect it to the lan game 127.0.0.2:4000 (your gateway/proxying program) and then have the gateway connect to 127.0.0.1:4000 where the 1st d2 (host) is listening.
Then anything you do on the 2nd client, your gateway would beable to see all, before its fawarded on.
You will probly need somthing like this mainly to abserve the order of things, as events happen in the game.

Also on another note, d2 ladder is resetting tomorow, so theres a good chance d2 may patch tomorow as well. So i think you have picked a good time to build a open D2GS, if its now about to change :)

Hmm interesting :)

I was told by Shadow Dancer, to my surprise, that damage calculations were done by the client. Hence, Player A attacks Player B, Player A would calculate the damage and send it off. I found this strange since it would mean that the server would have to verify the damage is 'legit'. But I'm thinking your saying things go the other way?

Does the server define the map layout as whole?

Yeah, I can imagine emulating missiles and such are going to take awhile to perfect properly; a hell of alot of testing.

I tried monitoring traffic locally but since it's just on loopback I'm never actually able to capture any packets, even tried installing a virtual network adapter in an attempt to force routing. Guess I'll just have to wait till I get a new computer.

[quote author=Win32 link=topic=16810.msg170422#msg170422 date=1182745267]
Hmm interesting :)

I was told by Shadow Dancer, to my surprise, that damage calculations were done by the client. Hence, Player A attacks Player B, Player A would calculate the damage and send it off. I found this strange since it would mean that the server would have to verify the damage is 'legit'. But I'm thinking your saying things go the other way?

Does the server define the map layout as whole?

Yeah, I can imagine emulating missiles and such are going to take awhile to perfect properly; a hell of alot of testing.

I tried monitoring traffic locally but since it's just on loopback I'm never actually able to capture any packets, even tried installing a virtual network adapter in an attempt to force routing. Guess I'll just have to wait till I get a new computer.
[/quote]
Yeah, in a open game, player A (the host) will calculate everything.
But its important to know that, player A is a client on its own server, it connects to its self.
So really, the server has to emulate as much of the game engine thats needed.
What defines a host, is when a host connects to its self, it will use the logon packet 0x67 rather than 0x68.
If the game was single player (no network/internet connections) then the client will calculate stuff, where as a online type game, the server calculates everything and notifys the client.
For example:
Player A and B stand next to each other.
Player A sends a message to melee attack player B
When the server recv's player A's attack message, it will check the distance between A and B to insure A is inrange to melee attack, then notify player B (and any other inview players) that player A has used melee attack (0x4C) on player B, then calculates the chances of A landing a hit on B (attack rating vs defence/block rate etc) and how much dmg will be done to player B on success.
Then on successfull attack:
Server notifys all inview players that player B has taken dmg (this will make the clients hear the *smack* of the attack, and show some dmg effects on player B)
Server notifys all inview players that player B is stuned for a few seconds (to stop the client responding why being stuned etc) as well as any other effects that have a chance to trigger.
On failing to land the attack:
Nothing. Or at least the server would tell all inrange players that player B dodged (if it had passive skills etc)
After success/failed:
After player A's attack time has passed, server notifys all other inrange players, that player A has stoped at location (this would generaly be longer if player A was coverd by frozen or cold etc)

This is why d2-lan/open/closed games screw up in lag, like monsters get to there location, but remain walking on the spot (because the server hasnt said "monster stops at X/Y")

As for the map, the server manages this to, as well as the client, for collision detection.
It can be often that the server thinks a client is in a differnt loction to what the client thinks it is, and if the distance gets to out-of-synq, the server normaly updates there location with 0x15 (or the client can do this with 0x5F i think)

Aside, have you tryed connecting d2#2 to 127.0.0.1:4000, and connecting the gateway/packet monitor to your net-address:4000?

But anyway to cut along story short, the server is the boss/manager/core of everything :)

Ah, thanks for clarifying that.

I would think that, for example:

Player A messages the server to melee-attack Player B

The server would figure out the result first (attack rating vs defense) then compress the 0x4C along with  (depending on the result) either 'notify Player B (and others in range) of damage taken to Player A'  and 'notify that Player B has been stunned', OR (on failure) 'inform players in range Player B dodged (if the player has such a skill)'.

My setup at the moment is: Laptop (192.168.0.2) -> Router (WAN Adapter) (192.168.0.1). I've tried capturing on 192.168.0.2, but the packets take a shortcut since it's destination is the same address, pretty much just inter-process comm. I've also tried installing a virtual network adapter with a different IP, trying to force routing, still no luck. I don't seem to be able to capture on 127.0.0.1 with my network monitor (Ethereal).

By the way, don't suppose you know anything about packet 0x22? Or 0x94? I'm having a look at the messages S->C and alot don't seem to be documented. (Had to do this by getting a friend to join my game)

[quote author=Win32 link=topic=16810.msg170431#msg170431 date=1182757765]
Ah, thanks for clarifying that.

I would think that, for example:

Player A messages the server to melee-attack Player B

The server would figure out the result first (attack rating vs defense) then compress the 0x4C along with  (depending on the result) either 'notify Player B (and others in range) of damage taken to Player A'  and 'notify that Player B has been stunned', OR (on failure) 'inform players in range Player B dodged (if the player has such a skill)'.
[/quote]
ye, it was 7AM when I replyed :), but the point I was making was, that the server manages everything after 1 player sends the message to attack another player.

[quote author=Win32 link=topic=16810.msg170431#msg170431 date=1182757765]
My setup at the moment is: Laptop (192.168.0.2) -> Router (WAN Adapter) (192.168.0.1). I've tried capturing on 192.168.0.2, but the packets take a shortcut since it's destination is the same address, pretty much just inter-process comm. I've also tried installing a virtual network adapter with a different IP, trying to force routing, still no luck. I don't seem to be able to capture on 127.0.0.1 with my network monitor (Ethereal).
[/quote]
Hm, when you fire up D2, and goto *Other Multiplayer > TCP/IP Game*, does it display your internet IP address or the network IP Address?
It should be possible to get a gateway between D2 and its server.
Im not useing a router, but i just tested having my gateway listening on MyInternetIP:4000, then loading up d2, and hosting a TCP/IP game.
But because the address/port were inuse, d2 uses 127.0.0.1:4000, which is where the gateway connects and fawards to.
This enabled me to get between d2 and its own server. (D2 connects to Gateway, gateway connects to D2)
Have you tryed it this way, but with 192.168.0.1/192.168.0.2 rather than 127.x.x.x?
Im pretty sure it should be possible with 2 network IP's, if you dont have access to your internet IP.

[quote author=Win32 link=topic=16810.msg170431#msg170431 date=1182757765]
By the way, don't suppose you know anything about packet 0x22? Or 0x94? I'm having a look at the messages S->C and alot don't seem to be documented. (Had to do this by getting a friend to join my game)
[/quote]
ye, 0x94 is in the research topic (lists a chars skill tree) and 0x22 (below) states the amount one has on a skill, for example, 10 on tomb of townportal etc.
0x22 mainly relates to what you see when you press "S" to bring up the skill tree i think.
[code]
0x22
(BYTE) Entity Type(?)
(BYTE) Unknown - 0x26
(DWORD) Entity ID
(WORD) Skill Code
(BYTE) Stats Amount
(WORD) Unknown
[/code]
The amount of scrolls in a tomb book, is in the 0x9C/0x9D, and is updated with 0x3E (item/stuff quantity/duriblity change) every time you use one, as well as a 0x22 tagging along side.

[EDIT]: Just incase you ment 0x95 or 0x96, and not 0x94:
[code]
0x95
(BITS[15]) Current Life
(BITS[15]) Current Mana
(BITS[15]) Current Stamina
(BITS[15]) Location X
(BITS[1])  Unknown
(BITS[15]) Location Y
(BITS[20]) Unknown
[/code]

[code]
0x96
(BITS[15]) Current Stamina
(BITS[15]) Location X
(BITS[1])  Unknown
(BITS[15]) Location Y
(BITS[18]) Unknown
[/code]

Hmm, sorry I actually meant 0x5E (And possibly 0x97). Here's a dump of a section of the decompressed packet that is troubling me:


2100 0002 0000 00A4 0000 3F00 975E 0100 0000 0000 0001 0001 0000 0000 0101 0100 0000 0000
0000 0100 0000 0101 0000 0000 0000 B128 0600 0000 0000 0100 FD9F FD9F FD9F FD9F FD9F FD9F
0100 0100 1D90 791C FD9F FD9F FD9F E51F 0100 0100 FD9F FD9F FD9F FD9F FD9F FD9F 0100 0100
FD9F FD9D FD9F 0000 0000 0000 0000 0000 0000 0000 FD9F ED9F 0000 FD9F FD9F FD9F 00       


0x97 is said to be 'blank', which I presume means there's just the message ID. Then we have 0x5E where I couldn't find any information on.

Any thoughts? :)

port forward the server port and then connect to your real ip not your router ip, that should work with ethereal.

[quote author=Win32 link=topic=16810.msg170434#msg170434 date=1182778123]
Hmm, sorry I actually meant 0x5E (And possibly 0x97). Here's a dump of a section of the decompressed packet that is troubling me:


2100 0002 0000 00A4 0000 3F00 975E 0100 0000 0000 0001 0001 0000 0000 0101 0100 0000 0000
0000 0100 0000 0101 0000 0000 0000 B128 0600 0000 0000 0100 FD9F FD9F FD9F FD9F FD9F FD9F
0100 0100 1D90 791C FD9F FD9F FD9F E51F 0100 0100 FD9F FD9F FD9F FD9F FD9F FD9F 0100 0100
FD9F FD9D FD9F 0000 0000 0000 0000 0000 0000 0000 FD9F ED9F 0000 FD9F FD9F FD9F 00       


0x97 is said to be 'blank', which I presume means there's just the message ID. Then we have 0x5E where I couldn't find any information on.

Any thoughts? :)
[/quote]
Ye 0x97 is blank :)
It means that you switched wepons. Bydefalt d2 is set to side 1, but if there char was last saved useing side 2, it will recv it on joining, so that d2 switchs.

0x5E is some quest nonsence I think. (never really looked at it or seen an importance)
Its always a binary looking (1 bit per byte) message with some byte at the end. (0xB1 in this case) Its also fixed lengh.

Then you have 0x28 (quest tree) after that and by the looks of things you done most of your quests :P

A few things I forgot the mention about the mapping in d2, is S>C 0x03 contains seeds that are used to generate the act and populate it with objects/npc's/gold piles etc. (nothing but the walkable floor/walls are really important)
Then each section of an area is added/removed with S>C 0x07/0x08 as the player moves around it.
The server populates these sections as they are 1st-time scouted and saves them along with everything on them to file when they are unscouted (no player is inview with it) so that when a player goes back, everything can be reloaded as the player last scouted it and the time idle can be applyed to any vanishing items/healing npcs etc.

Ah, I see.

Um, is there any documentation for these packets? As they aren't listed in the usual D2GS thread. Since I seem to be running into quite a few messages that there doesn't appear to be any info on lying around.

[quote author=Win32 link=topic=16810.msg170467#msg170467 date=1182847847]
Ah, I see.

Um, is there any documentation for these packets? As they aren't listed in the usual D2GS thread. Since I seem to be running into quite a few messages that there doesn't appear to be any info on lying around.
[/quote]
Ye, if you cant find any info on them on the web, just say the ID's of the messages your stuck on figgering out and/or what bit your stuck on.
Most of the research i posted in the d2gs thread, was with in a few weeks of me discovering D2 had a ingame protocol :P so it is pretty basic and doesnt list every single message.
I didnt have any pre-documentation about the protocol or any packet lenghs, just a function to decompress and a function to return some sort of packet header, so its suppriseing I listed as many packets as i did up there :)
The main idea was for it to be a stepping stone to those wanting to learn about the protocol, so they could get a basic idea of how it worked and could go on to teach them selfs/research the rest.
Abit like the "give a man a fish, or teach a man how to fish" saying :)
But what goes with out saying is, teach everyone everything there is to know about fishing, and the ocean starts to run out of fish ;)

Somthing that might come in handy, is the packet lenghs. (somone posted them on the 4th - 6th page in the D2GS research thread)

Ah :)

Hmm, I think 0x22 might've changed? The specification you gave me was:

[quote]
0x22
(BYTE) Entity Type(?)
(BYTE) Unknown - 0x26
(DWORD) Entity ID
(WORD) Skill Code
(BYTE) Stats Amount
(WORD) Unknown
[/quote]

Here's an example of what I've encountered:

2200 0002 0000 00DB 0001 8600 2200 0002 0000 00D9 0001 8600                             

It would seem the dword at offset+0x2 is the player's identifier (I believe)


I've noticed alot of fields are tagged as 'unknown', I'm thinking, will these holes in the protocol possibly result in being unable to properly implement the protocol?