Valhalla Legends Forums Archive | Battle.net Bot Development | Help Extracting uncompressed D2GS Packets

AuthorMessageTime
Wokket
G'day guys,

Firstly, a huge thank you to everyone that's contributed to this forum, a combination of living on here and downloading the BN# code has let me build my bot up through the battle.net logon, realm logon, char selection, game listing, and game server logon... I'm at the point where I've sent my 0x6b packet to the game server, my char appears in the game, and I'm receiving compressed game data.

I'm using the decompression code from the last post here.

My problem I guess is that I start decoding the packet's, and everything is going swimmingly, but after a couple it seems I'm getting out of whack somewhere.

[code]

                  f1 37  17 47 b1 f8 76 03 ac 14  .......7 .G..v...
86 53 d1 b4 6b 1b 95 96  bf fc 10 c7 b1 f8 76 03  .S..k... ......v.
a6 4a c5 d0 cc d9 d0 df  1e c7 e1 d8 0e 81 10 d8  .J...... ........
7b 1f 87 60 3b ba f7 7b  84 bb 84 cb 84 db 84 eb  {..`;..{ ........
9f b8 51 b9 0f 70 a5 60  88 b8 54 b8 55 a2 0a d7  ..Q..p.` ..T.U...
22 6e 16 2e 16 6d 05 a9  f0 bb 70 bd 72 2a e1 82  "n...m.. ..p.r*..
e1 8a d0 63 b8 66 b4 19  ec 07 30 f0 7b 1f 87 60  ...c.f.. ..0.{..`
3a 13 a8 8e a1 cc 3c 1e  c7 e1 d8 0e 84 c9 89 d4  :.....<. ........
3d 70 4a 22 96 82 8b 9d  1f 2a 8e 89 40 50 c6 05  =pJ".... .*..@P..
86 90 14 31 81 61 a4 56  08 a0 a1 14 22 82 84 40  ...1.a.V ...."..@
84 0a 80 51 a6 12 85 14  08 78 14 0d 27 c2 10 2c  ...Q.... .x..'..,
c3 a8 f6 3f 0e c0 76 18  ad b6 db 63 0b bb bb bd  ...?..v. ...c....
dd fd dd de ef bb bd dd  dd dd ef 25 21 fb dc d0  ........ ...%!...
37 8b 0b 4e 4d 10 4e 4d  1e f3 52 e0 e4 8d 83 41  7..NM.NM ..R....A
e0 d5 83 42 49 93 de e6  84 93 40 c6 cb 9a 18 1a  ...BI... ..@.....
3a 67 bd cb 8e cb 97 1f  ff ff e6 06 9f ff e1 f7  :g...... ........
ff d6 f3 bf 90 12 02 b6  b7 fa df 90 12 03 fa df  ........ ........
ff ff fa df ff f3 18 f6  3f 0e c0 74 62 7e 73 02  ........ ?..tb~s.
a7 2e 26 1c ae 2a 1c 9f  08 a7 48 a3 12 f3 a3 0d  ..&..*.. ..H.....
e7 46 31 85 e7 26 47 87  46 65 71 31 1c            .F1..&G. Feq1. 
[/code]

This is the original, compressed TCP data from Wireshark of the first packet received after the 0x6b, TCP data was 311 bytes, the f137 should mean 311bytes, so all the data is in teh one packet, no need to group in the next packet.

[code]
59 89 93 7e a6 05 4d 65  6c 74 72 75 69 64 00 00 
00 00 00 00 00 00 00 00  00 00|aa 00 89 93 7e a6 
0c 69 59 f9 ff 1f 76 00  89 93 7e a6 94|1b 89 93| 
7e a6 00 00 01 02 00 01  01 00 01 d9 00 01 da 00 
01 db 00 01 dc 00 01 03  00 01 df 00 01 e0 00 01 
e1 00 05 e2 00 01 e4 00  01 e5 00 07 e6 00 01 e8 
00 01 e9 00 01 ea 00 14  eb 00 03 ee 00 01 ef 00 
01 f0 00 01 f2 00 01 f4  00 14 f5 00 01 f9 00 14 
fa 00 05 22 00 26 89 93  7e a6 dc 00 07 1f 00 22 
00 26 89 93 7e a6 da 00  0b 1f 00 27 01 ae 33 64 
df 01 1f 03 55 7c 0d 88  60 9d 6f 88 60 9d 6f 48 
d3 b2 07 00 d3 b2 07 aa  9c 7b 6f 00 d8 41 04 06 
00 00 00 9b fc 03 d0 b6  00 00 23 00 89 93 7e a6 
00 f4 00 ff ff ff ff 5e  01 01 01 01 01 01 00 01 
01 01 00 00 00 00 01 01  01 01 00 01 01 00 00 01 
01 01 00 01 01 01 01 01  01 01 00 01 00 28 06 00 
00 00 00 00 01 00 01 10  1c 00 4a 80 1d 10 04 00 
1d 10 01 00 01 00 11 90  79 08 05 10 81 11 05 10 
25 0c 01 00 01 00 01 10  25 10 f5 13 01 10 09 10 
01 12 01 00 01 00 01 90  01 13 01 90 01 00 00 00 
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
00 00 09 10 00 00 00 00  00 00 00 00 00 00 00 00 
00 00 00 00 29 00 00 00  00 00 00 00 00 00 00 00 
00 00 80 00 00 00 20 00  00 00 00 00 a0 00 a0 00 
80 00 80 00 00 00 00 00  00 00 80 00 00 00 00 00 
a0 00 a0 00 00 00 00 00  00 00 80 00 00 00 00 00 
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
00 00 00 00 80 00 00 00  00 00 00 00 00 00 00 00 
00 00 00 00 00 0b 00 89  93 7e a6 5f 01 00 00 00 
1d 00 9c 1d 01 3a 1d 02  46 1d 03 d3 1f 07 00 5f 
02 00 1f 09 00 00 01 00  1f 0b 00 5e 01 00 1d 0c 
51 1f 0f 58 3a 08 00 2f 
[/code]

This is the decompressed version of that data using the code linked above.  I've included some pipe's to delimit the D2GS packets as extracted below


[PlayerAssign 0x59, datalen=25]
89 93 7e a6 05 4d 65 6c  74 72 75 69 64 00 00 00 
00 00 00 00 00 00 00 00  00

My char has joined, my char name, null co-ords, everything looks great.

[ CompInfoAdd 0xaa, len=18] 6 bytes original, plus 0c extras
00 89 93 7e a6 0c 69 59  f9 ff 1f 76 00 89 93 7e 
a6 94

[ WordToExperience 0x1b, len=2]  This seems a bit weird, low value for experience, and I haven't killed anything.
89 93


That parsing sequence leaves me on a 0x7e packet, which I can't find referenced anywhere?? Other runs have left me on a 0x00 value which also seems wrong.

I suspect it's the 0xaa parsing I'm getting wrong, but I'm really not sure.  If I treat the 0x0c value as total packet length, rather than extra bytes, it leaves me on a  0x76 (D2GS_OVERHEADCLEAR), which also seems wrong, and it's sizing is such I end up on the 0x1b as well, leaving me on the same byte.


I'm hesitant to blame the compression code, purely because it's far more likely I'm wrong, but I'm not sure how else to interpret that info.

Any and all help appreciated :)

Cheers,
Wokket




January 16, 2009, 11:26 PM
xpeh
Does sombody want to write a sniffer packet decoder plugin for bnet-related protocols? In particular, for Commview and Wireshark.

This would make bot devel much easies.
January 16, 2009, 11:33 PM
FrostWraith
To answer zephs question:
https://davnit.net/bnet/vL/index.php?board=17;action=display;threadid=4594
too bad its dead, but im sure somebody here still has it.  I just use WPE Pro
January 17, 2009, 12:13 AM
xpeh
I personally prefer sniffer plugin.

[quote author=FrostWraith link=topic=17790.msg181282#msg181282 date=1232151195]
I just use WPE Pro
[/quote]
?
January 17, 2009, 2:02 AM
FrostWraith
[quote author=xpeh link=topic=17790.msg181286#msg181286 date=1232157772]
[quote author=FrostWraith link=topic=17790.msg181282#msg181282 date=1232151195]
I just use WPE Pro
[/quote]
?
[/quote]
google it.
and by the way stop hijacking all these topics, damn man

Original poster, you stated "If I treat the 0x0c value as total packet length".  Are you looking at single bytes as the length of the data, or a WORD sized variable.  Most of the bnet protocol uses WORD (2 byte, 16-bit) variables to describe length.

Edit: nvm seems that for d2gs, the lengths are stored in 1 byte.
January 17, 2009, 2:14 AM
xpeh
[quote author=FrostWraith link=topic=17790.msg181288#msg181288 date=1232158464]
[quote author=xpeh link=topic=17790.msg181286#msg181286 date=1232157772]
[quote author=FrostWraith link=topic=17790.msg181282#msg181282 date=1232151195]
I just use WPE Pro
[/quote]
?
[/quote]
google it.
and by the way stop hijacking all these topics, damn man
[/quote]
LOLWUT? Do you want to say i'm offtoping?
WPE is another sniffer?
January 17, 2009, 2:31 AM
Sixen
[quote author=xpeh link=topic=17790.msg181289#msg181289 date=1232159463]
LOLWUT? Do you want to say i'm offtoping?
[/quote]

That's exactly what he's saying. You've done this at least 4 times now. Make your own damn threads.
January 17, 2009, 6:39 AM
Quarantine
Stop feeding the troll.
January 17, 2009, 1:57 PM
BreW
I agree. This troll sucks. Perhaps if he were a bit more subtle...
January 17, 2009, 2:14 PM
xpeh
I don't think it was a big offtop.

If you are searching for a troll, go to 4chan e.g.
January 17, 2009, 2:25 PM
Ringo
Here's the orginal topic of the D2GS decompression code:
https://davnit.net/bnet/vL/index.php?topic=585.0
most/all others are spawned from that, iirc.

The getpacketsize() function, will return the lengh of the appending data (after the header)
GamePacketSize(Data, Size, Offset)
In you're above test dump, "f1 37 17 47 b1....."
Offset returns the lengh of the header (in this case, 2 bytes)
Size returns the lengh of the data appending the header. (309 bytes in this case)
so, for example:
GamePacketSize(data[pos], Size, Offset)
pos=+offset
packet = data[pos, size]
pos=+size
pos should then be alined with the next packet (if there is one)
It sounded like you were not acounting for the offset+size as the total lengh.

hope this helps.
January 17, 2009, 3:02 PM
Wokket
Thanks for the constructive help Ringo  ;D

I used the version of the code you linked me to, and it returned effectively the same output as the code I was using (there is an extra trailing byte at the bottom of the uncompressed output with my old code compared to yours).... the first 60 bytes or so where I'm having trouble converting the packet data into game data are identical.

GamePacketSize() returns size=0x135, offset=0x02
GamePacketDecode() returns size=0x1f7


I'm not sure I made this clear enough in my first post, I haven't successfully parsed the entire 1st TCP packet received from the server, so I'm not (yet?) having the TCP Packet accumulation problems other people have reported...  I'm having problems extracting the Game-packets from the uncompressed data.

The first game-packet in the data (0x59) looks spot on according to the docs, and the data contained within it... I'm not sure how it goes pearshaped after that though..

Any chance someone could markup the decompressed output above with the game packet delimiters similar to mine so I can see where I'm going wrong?   If the decompressed data looks wrong, would someone mind pumping that TCP data through their decompression routine in case I've made the same mistake with the 2 different code bases??

In the meantime, I'll keep trying different ways to interpret it.

Thanks heaps
Wokket

Edit: I did have the length of the 0xaa packet wrong, actually reading the info in the D2GS Packet Research thread cleared that up for me, so I'm now getting the following sequence extracted:
[ PlayerAssign 0x59, len=25]
[ CompInfoAdd 0xaa, len=11]
[ OverheadClear 0x76, len=5] (Seems weird, but what the hey)
[ SkillsLog 0x94, len=86] Seems accurate, the skill numbers match up (for the most part), as do the skill points assigned... there's a few skills in the packet I can't find on bntedocs (eg db00, dc00), but I'm happy for the most part.

That leaves me on a 0x22 though, which I can't find referenced anywhere.

btw, is there a reason the awesome info in the Packet Research Thread isn't included on bnetdocs?


January 17, 2009, 10:24 PM
Barabajagal
[BNetDocs is in need of work. Lots of it. Everyone sort of got lazy halfway through transitioning it. And Don was going to rewrite the system to use an XML doctype system (I tried the same thing with my news and updates file for RCB, it was sort of fun making the doctype and making it W3C valid.), but I don't know what happened next...]
January 18, 2009, 12:17 AM
Ringo
[quote author=Wokket link=topic=17790.msg181319#msg181319 date=1232231049]
I'm not sure I made this clear enough in my first post, I haven't successfully parsed the entire 1st TCP packet received from the server, so I'm not (yet?) having the TCP Packet accumulation problems other people have reported...  I'm having problems extracting the Game-packets from the uncompressed data.

The first game-packet in the data (0x59) looks spot on according to the docs, and the data contained within it... I'm not sure how it goes pearshaped after that though..

Any chance someone could markup the decompressed output above with the game packet delimiters similar to mine so I can see where I'm going wrong?   If the decompressed data looks wrong, would someone mind pumping that TCP data through their decompression routine in case I've made the same mistake with the 2 different code bases??
[/quote]

umm, I broke these up by hand, its been awhile, so might be some mistakes in here;
[code]
59 89 93 7e a6 05 4d 65  6c 74 72 75 69 64 00 00 
00 00 00 00 00 00 00 00  00 00|aa 00 89 93 7e a6 
0c 69 59 f9 ff 1f 76 00  89 93 7e a6 94|1b 89 93| 
7e a6 00 00 01 02 00 01  01 00 01 d9 00 01 da 00 
01 db 00 01 dc 00 01 03  00 01 df 00 01 e0 00 01 
e1 00 05 e2 00 01 e4 00  01 e5 00 07 e6 00 01 e8 
00 01 e9 00 01 ea 00 14  eb 00 03 ee 00 01 ef 00 
01 f0 00 01 f2 00 01 f4  00 14 f5 00 01 f9 00 14 
fa 00 05 22 00 26 89 93  7e a6 dc 00 07 1f 00 22 
00 26 89 93 7e a6 da 00  0b 1f 00 27 01 ae 33 64 
df 01 1f 03 55 7c 0d 88  60 9d 6f 88 60 9d 6f 48 
d3 b2 07 00 d3 b2 07 aa  9c 7b 6f 00 d8 41 04 06 
00 00 00 9b fc 03 d0 b6  00 00 23 00 89 93 7e a6 
00 f4 00 ff ff ff ff 5e  01 01 01 01 01 01 00 01 
01 01 00 00 00 00 01 01  01 01 00 01 01 00 00 01 
01 01 00 01 01 01 01 01  01 01 00 01 00 28 06 00 
00 00 00 00 01 00 01 10  1c 00 4a 80 1d 10 04 00 
1d 10 01 00 01 00 11 90  79 08 05 10 81 11 05 10 
25 0c 01 00 01 00 01 10  25 10 f5 13 01 10 09 10 
01 12 01 00 01 00 01 90  01 13 01 90 01 00 00 00 
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
00 00 09 10 00 00 00 00  00 00 00 00 00 00 00 00 
00 00 00 00 29 00 00 00  00 00 00 00 00 00 00 00 
00 00 80 00 00 00 20 00  00 00 00 00 a0 00 a0 00 
80 00 80 00 00 00 00 00  00 00 80 00 00 00 00 00 
a0 00 a0 00 00 00 00 00  00 00 80 00 00 00 00 00 
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 
00 00 00 00 80 00 00 00  00 00 00 00 00 00 00 00 
00 00 00 00 00 0b 00 89  93 7e a6 5f 01 00 00 00 
1d 00 9c 1d 01 3a 1d 02  46 1d 03 d3 1f 07 00 5f 
02 00 1f 09 00 00 01 00  1f 0b 00 5e 01 00 1d 0c 
51 1f 0f 58 3a 08 00 2f 
[/code]

[code]
59 89 93 7e a6 05 4d 65 6c 74 72 75 69 64 00 00
00 00 00 00 00 00 00 00  00 00

aa 00 89 93 7e a6 0c 69 59 f9 ff 1f

76 00 89 93 7e a6

94 1b 89 93 7e a6 00 00 01 02 00 01 01 00 01 d9
00 01 da 00 01 db 00 01 dc 00 01 03 00 01 df 00
01 e0 00 01 e1 00 05 e2 00 01 e4 00 01 e5 00 07
e6 00 01 e8 00 01 e9 00 01 ea 00 14 eb 00 03 ee
00 01 ef 00 01 f0 00 01 f2 00 01 f4 00 14 f5 00
01 f9 00 14 fa 00 05 22 00 26 89 93 7e a6 dc 00
07 1f 00

22 00 26 89 93 7e a6 da 00 0b 1f 00

27 01 ae 33 64 df 01 1f 03 55 7c 0d 88 60 9d 6f
88 60 9d 6f 48 d3 b2 07 00 d3 b2 07 aa 9c 7b 6f
00 d8 41 04 06 00 00 00 9b fc 03 d0 b6 00 00

23 00 89 93 7e a6 00 f4 00 ff ff ff ff

5e 01 01 01 01 01 01 00 01 01 01 00 00 00 00 01
01 01 01 00 01 01 00 00 01 01 01 00 01 01 01 01
01 01 01 00 01 00

28 06 00 00 00 00 00 01 00 01 10 1c 00 4a 80 1d
10 04 00 1d 10 01 00 01 00 11 90 79 08 05 10 81
11 05 10 25 0c 01 00 01 00 01 10 25 10 f5 13 01
10 09 10 01 12 01 00 01 00 01 90 01 13 01 90 01
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 09 10 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00

29 00 00 00 00 00 00 00 00 00 00 00 00 00 80 00
00 00 20 00 00 00 00 00 a0 00 a0 00 80 00 80 00
00 00 00 00 00 00 80 00 00 00 00 00 a0 00 a0 00
00 00 00 00 00 00 80 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00

0b 00 89 93 7e a6

5f 01 00 00 00

1d 00 9c

1d 01 3a

1d 02 46

1d 03 d3

1f 07 00 5f 02 00

1f 09 00 00 01 00

1f 0b 00 5e 01 00

1d 0c 51

1f 0f 58 3a 08 00

2f 
[/code]
that 0x2F at the end, is bad bad bad, you need to get rid of that tbh.
Aside, you can figger out most packet lenghs from just looking at a few dumps (when they pop up), it doesn;t take to long to figger out, plus its good practiss :p
Some lenghs have/do change each patch (altho, I dont think anything changed from 1.11 to 1.12 tho)
iirc, page 3 in the D2GS research thread, is some packet dumps, that might help you with some packets that have lenghs unknown to you.

Also, the documentation in the d2gs research thread, is what I figgerd out in the 1st few weeks of probing D2GS, so its bland and some what basic, at best.
Ive been meaning to refresh it, but never got around to it.

hope this helps.
January 18, 2009, 3:08 AM
Wokket
That is fantastically awesome.  Thanks Ringo!

I think I'm on a roll now.

Cheers,
Wokket
January 18, 2009, 10:30 PM

Search