I was trying to make a program that would read a file, create a hex dump of it, and put it in a .txt file. I tested it on storm.dll, and it works... in 21 minutes...

[code]
Const FILE_READ As String = "C:\Program Files\Starcraft\storm.dll"
Const FILE_DUMP As String = "C:\WINDOWS\DESKTOP\hexdump.txt"
Const READ_SIZE As Long = 16

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (pDst As Any, pSrc As Any, ByVal ByteLen As Long)
Private Declare Function GetTickCount Lib "kernel32" () As Long

Dim startdump As Long


Private Sub dumpfile(ByVal filename As String, ByVal outfile As String)
Dim readcnt As Long
Dim inbuf As String
Dim tmpbuf As String
Dim readtime As Double

' This is to figure out how long the
' whole proccess takes.
startdump = GetTickCount


Close #2
Open filename For Binary Access Read As #2

If LOF(2) = 0 Then Exit Sub

pbar.Max = LOF(2)
inbuf = Space$(LOF(2))

Get #2, , inbuf
While LenB(inbuf) <> 0
tmpbuf = Left$(inbuf, 16)
inbuf = Right$(inbuf, Len(inbuf) - 16)
fappend hexdump(tmpbuf, readcnt), outfile
readcnt = readcnt + 16

pbar.Value = readcnt
Label1.Caption = readcnt & "/" & pbar.Max
pbar.Refresh
DoEvents
Wend

readtime = (GetTickCount - startdump) / 60000
Close #2
Label2.Caption = "Finished: ~" & readtime & " mins."
End Sub



Private Function hexdump(ByVal data As String, ByRef loffset As Long) As String
Dim tmpbuf As String
Dim spacebuf As String
Dim perbuf As String
Dim tmpoffset As String * 2
Dim offset As String

spacebuf = Space$(16)
perbuf = String(16, ".")

CopyMemory ByVal tmpoffset, loffset, 2

tmphex = Hex(Asc(Mid$(tmpoffset, 2, 1)))
offset = IIf(Val("&H" & tmphex) < 16, "0" & tmphex, tmphex)
tmphex = Hex(Asc(Mid$(tmpoffset, 1, 1)))
offset = IIf(Val("&H" & tmphex) < 16, offset & "0" & tmphex, offset & tmphex)

tmpbuf = strtohex(data)
For i = 1 To Len(data)
If Asc(Mid$(data, i, 1)) <> 32 Then
If Asc(Mid$(data, i, 1)) < 48 Or Asc(Mid$(data, i, 1)) > 122 Then Mid$(data, i, 1) = "."
End If
Next i

hexdump = offset & ": " & tmpbuf & Left$(spacebuf, 16 - Len(data)) & " " & data & Left$(perbuf, 16 - Len(data))
End Function


Private Function strtohex(ByVal data As String, Optional addspace As Byte = 0) As String
Dim buffer As String
Dim tmphex As String

For i = 1 To Len(data)
tmphex = Hex(Asc(Mid$(data, i, 1)))
If Val("&H" & tmphex) < 16 Then tmphex = "0" & tmphex

buffer = IIf(addspace = 0, buffer & tmphex & " ", buffer & tmphex)
Next i

strtohex = Left$(buffer, Len(buffer) - 1)
End Function

Private Sub fappend(ByVal data As String, ByVal filename As String)
Close #1
Open filename For Append As #1
Print #1, data
Close #1
End Sub

Private Sub fclear(ByVal filename As String)
Close #1
Open filename For Output As #1
Close #1
End Sub

Private Sub Form_Activate()
fclear FILE_DUMP
dumpfile FILE_READ, FILE_DUMP
End Sub
[/code]


Sample output:
[quote]0000: 4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00 MZ..............
0010: B8 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00 ........@.......
0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0030: 00 00 00 00 00 00 00 00 00 00 00 00 E0 00 00 00 ................
0040: 0E 1F BA 0E 00 B4 09 CD 21 B8 01 4C CD 21 54 68 ...........L..Th
0050: 69 73 20 70 72 6F 67 72 61 6D 20 63 61 6E 6E 6F is program canno
0060: 74 20 62 65 20 72 75 6E 20 69 6E 20 44 4F 53 20 t be run in DOS
0070: 6D 6F 64 65 2E 0D 0D 0A 24 00 00 00 00 00 00 00 mode............
0080: 24 F7 6A E0 60 96 04 B3 60 96 04 B3 60 96 04 B3 ..j.`...`...`...
0090: 73 9E 59 B3 62 96 04 B3 60 96 05 B3 A2 96 04 B3 s.Y.b...`.......
00A0: E3 9E 59 B3 6B 96 04 B3 E3 8A 0A B3 64 96 04 B3 ..Y.k.......d...
00B0: 60 96 04 B3 43 96 04 B3 34 B5 35 B3 7C 96 04 B3 `...C...4.5.....
00C0: A7 90 02 B3 61 96 04 B3 9F B6 00 B3 61 96 04 B3 ....a.......a...
00D0: 52 69 63 68 60 96 04 B3 00 00 00 00 00 00 00 00 Rich`...........
00E0: 50 45 00 00 4C 01 06 00 A2 FB AB 40 00 00 00 00 PE..L......@....

. . .

FFF0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[/quote]


Any idea why it takes so long (~21 minutes)?

[quote author=Eli_1 link=board=31;threadid=7242;start=0#msg65077 date=1087165814]
[code]
..
inbuf = Right$(inbuf, Len(inbuf) - 16)
..
[/code]
[/quote]

This is a very slow operation, because it will copy the entire file contents to a new buffer. It's makes the algorithm O(n^2). Reading the input file 16 bytes at a time would be faster, or using Mid to extract the data from inbuf.


[quote author=Eli_1 link=board=31;threadid=7242;start=0#msg65077 date=1087165814]
[code]
...
pbar.Refresh
DoEvents
...
[/code]
[/quote]

This is slow, don't do it on every iteration.


[quote author=Eli_1 link=board=31;threadid=7242;start=0#msg65077 date=1087165814]
[code]
Private Sub fappend(ByVal data As String, ByVal filename As String)
Close #1
Open filename For Append As #1
Print #1, data
Close #1
End Sub
[/code]
[/quote]

Do you have On Error Resume Next? I don't see how the first Close succeeds. Opening and closing files is typically slow. Don't open and close files unnecessarily.

Thanks, Adron. I'll try some of your suggestions and get back to you.

[quote]
Reading the input file 16 bytes at a time would be faster ...
[/quote]

That was how the program was origionally designed. Both ways seem equally slow.

[quote]
[code]
..
inbuf = Right$(inbuf, Len(inbuf) - 16)
..
[/code]

This is a very slow operation, because it will copy the entire file contents to a new buffer. It's makes the algorithm O(n^2). Reading the input file 16 bytes at a time would be faster, or using Mid to extract the data from inbuf.
[/quote]

I'm using Mid to extract the data now (this made the huge difference).


[quote]
[code]
...
pbar.Refresh
DoEvents
...
[/code]

This is slow, don't do it on every iteration.
[/quote]

I now do it on every 100th iteration.



The read/dump time on storm.dll is now ~1.59 minutes. This is a HUGE difference from the origional 21 minutes. I'm gonna try and only write to 'hexdump.txt' on every 100th iteration, along with the progressbar update - I'll post again if it makes a significant change. Thanks, Adron.

The main changes made are here:

[code]
...

While readcnt <> LOF(2)
tmpbuf = Mid$(inbuf, readcnt + 1, 16)
buffer = buffer & hexdump(tmpbuf, readcnt) & vbCrLf
readcnt = readcnt + 16

If readcnt Mod (16 * 50) = 0 Or readcnt = LOF(2) Then
pbar.Value = readcnt
Label1.Caption = readcnt & "/" & pbar.Max
pbar.Refresh
fappend buffer, FILE_DUMP: buffer = vbNullString
DoEvents
End If
Wend

...
[/code]

The new time is ~.8 minutes, but that's still a long time. How is it some of the hex editors like HexWorkshop seem to read a file and create identical output almost instantly?

This may not make a big difference, but every function call has some overhead associated with it, and i'm not sure what exactly goes on inside LOF() -- it might be a very expensive function. Try storing the length of the file in a variable when the program first runs and using variable that in place of the LOF() calls.

[quote author=Eli_1 link=board=31;threadid=7242;start=0#msg65077 date=1087165814]
[code]
Private Sub fappend(ByVal data As String, ByVal filename As String)
Close #1
Open filename For Append As #1
Print #1, data
Close #1
End Sub
[/code]


Do you have On Error Resume Next? I don't see how the first Close succeeds. Opening and closing files is typically slow. Don't open and close files unnecessarily.
[/quote]

That wouldn't cause an error and would execute fine but it's bad practice. He should should just do something like.
[code]Dim fFile As Byte
fFile = FreeFile 'Get unused file[/code]

[quote author=K link=board=31;threadid=7242;start=0#msg65101 date=1087175896]
... and i'm not sure what exactly goes on inside LOF() -- it might be a very expensive function. Try storing the length of the file in a variable when the program first runs and using variable that in place of the LOF() calls.
[/quote]

It did make a small difference. The time is at .7 minutes now.

[quote author=Eli_1 link=board=31;threadid=7242;start=0#msg65093 date=1087172380]
The new time is ~.8 minutes, but that's still a long time. How is it some of the hex editors like HexWorkshop seem to read a file and create identical output almost instantly?
[/quote]

Hex editors typically don't read the entire file. They throw up a window with a scrollbar large enough for the entire file (needs to check lof), and then read as much of the file as they need to display what you see (needs to read less than 1 kb).

You may want to pass some of your larger variables in as ByRef so that it doesn't need to create a copy of one each time. You need to make sure that you don't change it in the function though.

[quote author=drivehappy link=board=31;threadid=7242;start=0#msg65211 date=1087233218]
You may want to pass some of your larger variables in as ByRef so that it doesn't need to create a copy of one each time. You need to make sure that you don't change it in the function though.
[/quote]

[s]I'll try that in just a little bit.[/s] It didn't make a noticable difference. :(


I got the time down to .4 minutes by using SetPriorityClass.

[quote author=Eli_1 link=board=31;threadid=7242;start=0#msg65232 date=1087237579]
I got the time down to .4 minutes by using SetPriorityClass.
[/quote]

SetPriorityClass is probably a bit the same as reducing the frequency of DoEvents calls.