Entwickler-Ecke
Sonstiges (Delphi) - Suchen Array von Bytes in Filestream
RamiroCruzo - Mi 15.06.16 17:36
Titel: Suchen Array von Bytes in Filestream
Grüße Verehrte Masters,
Ich war der Bau eine Anwendung, die Array von Bytes in einem Filestream finden muss. Ich brauche seinen Ausgang in einem Array von ganzen Zahlen oder String-Liste.
Kannst du mir bitte helfen. Sein Teil meiner Semesterarbeit so brauchen sie dringend abzuschließen.
Danke und viele Grüße,
Ramiro Cruzo
erfahrener Neuling - Do 16.06.16 09:19
Hi and :welcome:
Kannst du bitte nochmal etwas genauer deine Situation/dein Problem erläutern. Dann kann dir hier bestimmt auch jemand helfen.
Falls du es auf deutsch noch nicht so gut kannst, gern auch auf Englisch ;)
Gruß Julian
RamiroCruzo - Do 16.06.16 18:20
Thanks @Julian..Seems my German skills are not good :D
Actually, our professor gave us files to analyse & those files contain chunks of raw data marked with some Bytes. So, I need to build a function or procedure so as to search each byte in that array in that file & return its position in form of array of int64 or stringlist.
Thanks again :D
Delete - Fr 17.06.16 11:13
- Nachträglich durch die Entwickler-Ecke gelöscht -
RamiroCruzo - Fr 17.06.16 19:18
Thanks for the help Sir :) , but the one you described is to find position of Byte in the ByteArray whereas I need to search for the byte in File i.e. Filestream.
Moderiert von
Narses: Zitat gekürzt.
Delete - Fr 17.06.16 19:49
- Nachträglich durch die Entwickler-Ecke gelöscht -
RamiroCruzo - Sa 18.06.16 06:12
No problemo... :D Well, basically the files are Binary, containing hex data. Here, I've attached the Hex View.
Delete - Sa 18.06.16 10:17
- Nachträglich durch die Entwickler-Ecke gelöscht -
RamiroCruzo - Do 23.06.16 18:54
Thanks for the previous reply :D ...I wrote a function to exactly search for 2 bytes (78 9C used for example) in a buffer & add its position in TIntList. But, I'm a bit confused how to implement that function, as, I need to read a file in buffers & for each buffers run this:
Delphi-Quelltext
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38: 39: 40: 41: 42: 43: 44: 45: 46: 47: 48: 49: 50: 51: 52: 53: 54: 55: 56: 57: 58: 59: 60: 61: 62: 63: 64: 65: 66: 67: 68: 69: 70: 71: 72: 73: 74: 75: 76: 77: 78: 79: 80: 81: 82: 83: 84: 85: 86: 87:
| function Find(szSubStr: PChar; buf: PByteArray; iBufSize: integer; iOffset: integer = 0): integer;
var iSubStrLen: integer; skip: array [byte] of integer; found: boolean; iMaxSubStrIdx: integer; iSubStrIdx: integer; iBufIdx: integer; iScanSubStr: integer; mismatch: boolean; iBufScanStart: integer; ch: byte; begin found := False; result := -1;
iSubStrLen := StrLen(szSubStr); if iSubStrLen = 0 then begin result := 0; Exit end; iMaxSubStrIdx := iSubStrLen - 1;
for ch := Low(skip) to High(skip) do skip[ch] := iSubStrLen; for iSubStrIdx := 0 to (iSubStrLen - 1) do skip[ord(szSubStr[iSubStrIdx])] := Max(iMaxSubStrIdx - iSubStrIdx, 1);
iBufScanStart := iOffset + iMaxSubStrIdx; while (not found) and (iBufScanStart < iBufSize) do begin iBufIdx := iBufScanStart; iScanSubStr := iMaxSubStrIdx; repeat mismatch := (ord(szSubStr[iScanSubStr]) <> buf[iBufIdx]); if not mismatch then if iScanSubStr > 0 then begin Dec(iBufIdx); Dec(iScanSubStr) end else found := True; until mismatch or found; if found then result := iBufIdx else iBufScanStart := iBufScanStart + skip[buf[iBufScanStart]]; end; end;
function findInBuffer(buffer: PByteArray; bufSize: integer): TIntList; var tmpRes: integer; memBuf: TMemoryStream; szFind: array [0 .. 255] of char; begin
result := TIntList.Create; memBuf := TMemoryStream.Create(); try memBuf.Write(buffer^, bufSize); tmpRes := 0; while (tmpRes <> -1) and (tmpRes <= bufSize) do begin strPCopy(szFind, $78 + $9C); tmpRes := Find(szFind, buffer, bufSize, tmpRes); if (tmpRes <> -1) then begin result.Add(tmpRes); inc(tmpRes, 15); end; end; finally FreeAndNil(memBuf); end; end; |
Can ya please tell how to implement it with FileStream? :)
Moderiert von
Christian S.: Replaced Code with Delphi tags
Delete - Fr 24.06.16 01:36
- Nachträglich durch die Entwickler-Ecke gelöscht -
jaenicke - Fr 24.06.16 06:59
For this task I would suggest the boyer-moore algorithm. It is a string search algorithm, but you can adapt it easily to using bytes, because it is based on comparing char by char.
Here you find an example:
http://www.entwickler-ecke.de/viewtopic.php?p=502137
RamiroCruzo - Fr 24.06.16 18:30
Frühlingsrolle hat folgendes geschrieben : |
You mean something like this:
|
Thanks for the function :D But the file I need to run this on is a big file so can we do the scan with small buffers which will increase the speed?
Moderiert von
Narses: Komplettzitat gekürzt.
Delete - Fr 24.06.16 19:40
- Nachträglich durch die Entwickler-Ecke gelöscht -
Sinspin - Fr 24.06.16 22:10
We talk about binary data, not about text files.
In case you do that with String should you use AnsiString.
However, i think it is better to use a byte array.
How large can the files be? If we talk about gigabyte should you use a buffered file stream instead of one big buffer for the full file content.
Here is a well working version which uses a byte array as Buffer for the file content. My biggest test file was close to 500MB. It took only a few seconds to scan this file.
Delphi-Quelltext
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38: 39: 40: 41: 42: 43: 44: 45: 46: 47: 48: 49:
| procedure FindByteSequenceInFile(FileName: string; Sequence: array of byte; var List: TStringList); var Buffer: array of byte; Fs: TFileStream; l, n, sl: Integer; ScanStopped: boolean;
begin List.Clear; sl := Length(Sequence); if sl = 0 then Exit;
Fs := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite); try SetLength(Buffer, Fs.Size); Fs.Read(Buffer[0], Fs.Size); for l := 0 to Fs.Size-1 do begin if Buffer[l] = Sequence[0] then begin if sl = 1 then List.Add(IntToStr(l)) else begin n := 1; ScanStopped := false; while n < sl do begin if Buffer[l+n] <> Sequence[n] then begin ScanStopped := true; Break; end; Inc(n); end; if not ScanStopped then List.Add(IntToStr(l)); end; end; end; finally Finalize(buffer); Fs.Free; end; end; |
For larger files is it better to use a buffered file stream, i am sure you are able to implement that based on my example.
RamiroCruzo - So 26.06.16 18:43
Sir, I'm continuously getting this error:
Zitat: |
Exception EAccessViolation in module project.exe at 000CDC40.
Access violation at address 004CDC40 in module 'project.exe'. Read of address 00000000.
|
I'm calling it like this:
Delphi-Quelltext
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11:
| var Test: array [0 .. 1] of byte = ( $78, $9C ); pos: TStringList;
begin FindByteSequenceInFile(ParamStr(1), Test, pos); pos.SaveToFile('log.txt'); end. |
jaenicke - So 26.06.16 21:03
You did not initialize your variable pos. You have to create the TStringList before you hand it over to the procedure.
Sinspin - Mo 27.06.16 00:16
I'm sorry, "Sir" is to strong for me. I feel the cold steel of a sword in my neck if i read that.
My call for the test looks a bit differend, it should work like that with D7 and newer.
For visualization have i a ListBox on my form which copies the results from the StringList before i the StringList destroy.
Delphi-Quelltext
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14:
| procedure Test; var sl: TStringList;
begin sl := TStringList.Create; try FindByteSequenceInFile(ParamStr(1), [13,10], sl); sl.SaveToFile('log.txt'); finally sl.Free; end; end; |
RamiroCruzo - Do 07.07.16 18:31
Sorry for the late reply guys..Was in hospital due to allergies..Am back now, I already solved that problem, also, changing Integer to int64 will remove 2 GB stackoverflow limit.
While doing the bufferedfilestream, define the buffer size otherwise with big files, it will assign so big buffer that will cause memory error.
:D :D Thanks again
Entwickler-Ecke.de based on phpBB
Copyright 2002 - 2011 by Tino Teuber, Copyright 2011 - 2025 by Christian Stelzmann Alle Rechte vorbehalten.
Alle Beiträge stammen von dritten Personen und dürfen geltendes Recht nicht verletzen.
Entwickler-Ecke und die zugehörigen Webseiten distanzieren sich ausdrücklich von Fremdinhalten jeglicher Art!