Autor Beitrag
RamiroCruzo
Hält's aus hier
Beiträge: 13

Verfasst: Sa 16.07.16 09:24
Greetings Masters,

I was trying to Multithread an app which uses stdin & stdout. But, I'm constantly having problems with output writing as the streams need to be written in correct order in order to make the CRC proper. I tried "Critical Sections", they limit one thread thread at a time but not the order.

Without the MT part, normal code looks like this:

Delphi-Quelltext
 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20: for I := 1 to SL1.Count do        begin          if Length(SL1[I - 1]) > 12 then          begin      ms1 := TMemoryStream.Create;            ms1.CopyFrom(mystream1,                StrToInt64('$' + DecodeRefStr(SL1[(I - 1)])[1])); ms2 := TMemoryStream.Create; ms2.CopyFrom(mystream1, StrToInt64('$' + DecodeRefStr(SL1[(I - 1)])[2]));      Output := TMemoryStream.Create;      processstream(ms1,ms2,Output);            mystream2.CopyFrom(Output, 0);            ms1.Free;      ms2.Free;      Output.Free;          end          else            mystream2.CopyFrom(mystream1, StrToInt64('\$' + SL1[(I - 1)]));        end;

SO, is there anyway that we can define the position while copying like we do in arrays so that we can define the position of buffer in integers not offsets?
jaenicke

Beiträge: 18726
Erhaltene Danke: 1628

W10 x64 (Chrome, IE11)
Delphi 10.2 Ent, Oxygene, C# (VS 2015), JS/HTML, Java (NB), PHP, Lazarus
Verfasst: Sa 16.07.16 11:37
If the size of the data is identical, you could calculate the position and set it before you write to the output stream.

If this is not the case, I would give the threads an ascending number to know which result should be written next. Then I would simply give the results combined with the numbers of the results to an output class, which writes the next result to be written as it gets it and stores all other results until the missing results have arrived.

You did not write which version of Delphi you are using. In newer versions I would use a TDictionary to store the results until all parts between have been written.

So in pseudocode this would be:
Delphi-Quelltext
 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20:21:22:23:24:25:26:27:28:29:30:31:32: Results: TObjectDictionary;procedure ExecuteThread(Number, Data);begin  // in the thread  WorkWithData;  HandleResult(Number, WorkResult);end;procedure HandleResult(Number, WorkResult);var  CurrentResult;begin  TMonitor.Enter(Results);  try    if NextNumber = Number then    begin      WriteResult(WorkResult);      Inc(NextNumber);      while Results.TryGetValue(NextNumber, CurrentResult) do      begin        WriteResult(CurrentResult);        Results.Remove(NextNumber);        Inc(NextNumber);      end;    end    else      Results.Add(Number, WorkResult);  finally    TMonitor.Exit(Results);  end;end;

If this wasn't detailed enough, I can of course explain more detailed.

TMonitor is more lightweight than a full critical section. If you have a large number of chunks and threads, I would recommend to separate storing the results and writing them to the output. So you only lock Results, write the results to it and unlock again. Another thread could check periodically for results (where of course it has to enter the lock too) and write them to the output. This way the periods, in which one thread is inside the lock, are not this long as the actual writing to the output takes place outside.

Für diesen Beitrag haben gedankt: RamiroCruzo
RamiroCruzo
Hält's aus hier
Beiträge: 13

Verfasst: So 17.07.16 21:22
Thanks for the reply I've both XE8 & 10.1 Berlin...

Well, TObjectDictionary is totally new to me, can ya please explain how it works?

Also, I'm using a thread declared as TRestoreThread with parameters of I, InputStream, OutputStream. I use Semaphore to control the number of threads.