Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.
 

Automatic cleanup of log files


#1

Hi,

Is there a way to automatically clean up log files that were created by setting the connection to rotate?


#2

Hello Cesar,

This functionality is currently not built-in, but already on our feature request list.

You could also achieve the same functionality with a simple background thread which periodically checks if there are too many log files and then deletes unnecessary logs. We have an example (for Delphi) which does exactly this. If you are interested, I can send you the example snippet via email.


#3

Yes, I would be interested in the code snippet. Thanks.


#4

Sent! Please let me know if you have any questions about the example.


#5

Hello,
Can I ask for the cleanup code too?


#6

Hello Piotr,

Sure. Since several other people have also asked for this, I will post the unit right here.

[code]unit LogCleanup;

interface

uses
SiAuto, SmartInspect, SysUtils, Classes, SyncObjs;

const
CFileName = ‘log.sil’;
CMaxFiles = 10;
CCleanupInterval = 1000 * 60 * 60;

type
TLogCleanupThread = class(TThread)
private
FMaxFiles: Integer;
FEvent: TEvent;
FFileName: String;
FCleanupInterval: Integer;
procedure Cleanup;
protected
procedure Execute; override;
public
constructor Create(const AFileName: String = CFileName;
const AMaxFiles: Integer = CMaxFiles;
const ACleanupInterval: Integer = CCleanupInterval);
destructor Destroy; override;
procedure Cancel;
property MaxFiles: Integer read FMaxFiles;
property CleanupInterval: Integer read FCleanupInterval;
end;

implementation

var
GSession: TSiSession;

{ TLogCleanupThread }

procedure TLogCleanupThread.Cancel;
begin
GSession.EnterMethod(Self, ‘Cancel’);
GSession.LogWarning(‘Cancelling!’);
Terminate;
FEvent.SetEvent;
GSession.LeaveMethod(Self, ‘Cancel’);
end;

procedure TLogCleanupThread.Cleanup;
var
LSearchRec: TSearchRec;
LErrorCode: Integer;
LMask, LPath: String;
LLogFile: String;
LFiles: TStringList;
I: Integer;
begin
GSession.EnterMethod(Self, ‘Cleanup’);

LMask := ChangeFileExt(FFileName, ‘’) + ‘*’ + ExtractFileExt(FFileName);
LErrorCode := FindFirst(LMask, 0, LSearchRec);

if LErrorCode = 0 then
begin
try
LFiles := TStringList.Create;
try
LFiles.Sorted := True;
LPath := IncludeTrailingPathDelimiter(ExtractFilePath(FFileName));

    repeat
      LLogFile := LPath + LSearchRec.Name;
      LFiles.Add(LLogFile);
    until FindNext(LSearchRec) <> 0;

    GSession.LogStringList('LFiles', LFiles);

    if LFiles.Count > FMaxFiles then
    begin
      for I := 0 to Pred(LFiles.Count - FMaxFiles) do
      begin
        GSession.LogMessage(LFiles[I]);
        if not DeleteFile(LFiles[I]) then
        begin
          GSession.LogLastError('DeleteFile');
        end;
      end;
    end;
  finally
    LFiles.Free;
  end;
finally
  FindClose(LSearchRec);
end;

end else
begin
GSession.LogString(‘LMask’, LMask);
GSession.LogWin32Error(‘FindFirst’, LErrorCode);
end;

GSession.LeaveMethod(Self, ‘Cleanup’);
end;

constructor TLogCleanupThread.Create(const AFileName: String;
const AMaxFiles: Integer; const ACleanupInterval: Integer);
begin
inherited Create(True);
FFileName := AFileName;
FMaxFiles := AMaxFiles;
FCleanupInterval := ACleanupInterval;
FEvent := TEvent.Create(nil, False, False, ‘’);
end;

destructor TLogCleanupThread.Destroy;
begin
FreeAndNil(FEvent);
inherited;
end;

procedure TLogCleanupThread.Execute;
begin
GSession.EnterThread(‘Log Cleanup’);
while not Terminated do
begin
Cleanup;
GSession.LogMessage(‘Waiting…’);
FEvent.WaitFor(FCleanupInterval);
end;
GSession.LeaveThread(‘Log Cleanup’);
end;

initialization
GSession := Si.AddSession(‘Log Cleanup’);
end.[/code]

This unit can be used as follows:

[code]type
TForm1 = class(TForm)
procedure FormCreate(Sender: TObject);
procedure FormDestroy(Sender: TObject);
private
FLogCleanup: TLogCleanupThread;
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
FLogCleanup := TLogCleanupThread.Create(‘c:\Test\log.sil’);
FLogCleanup.Resume;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
FLogCleanup.Cancel;
FLogCleanup.WaitFor;
FreeAndNil(FLogCleanup);
end;[/code]

Most of the time the background worker thread is put to sleep and thus does not use any unnecessary resources. In a customizable interval, the background thread wakes up and checks if old log files should be deleted.

The filename parameter for TLogCleanupThread specifies the base name for the log files to delete. In the example above, the thread is responsible for deleting files with the mask “c:\Test\log*.sil”. Moreover, the example uses the default settings for the delete interval (one hour) as well as the maximum allowed log files (10).


#7

Hi Tobias,

I have just started using this code but found it unnecessary to have a background thread waiting for doing a cleanup. I just extracted the actual cleanup code and execute it at program startup. That’s enough for my purpose and reduces the overhead.

Since I guess you will be adding this code to the smart inspect libraries, maybe you want to also offer this option.

twm


#8

Hello,
I think that cleanup at startup is not enough for long time running server applications. After months or years of running, it may result in GBs of log files.

best regards
Piotr Rezmer


#9

I agree. But many are not writing server apps but programs that get restarted at least once a day. Those might not want to have a background thread running all the time which does nothing. It’s just a matter of having a choice.

twm


#10

Hello all,

I think we will probably integrate the cleanup of old log files directly into the file protocol without using additional threads. On startup and whenever a new log file is created, the file protocol could then automatically delete unwanted/unneeded log files.


#11

Hello,

Just to clarify this: we won’t just add threads for new features to the libraries without a way to disable them. Some features like asynchronous logging require independent threads, but this is an optional feature (asynchronous logging is planned for SmartInspect 3.0). Just like twm said, we also believe that having a choice to disable such features is important.