Logstash : No sincedb_path set, generating one based on the file path while reading csv file

No sincedb_path set, generating one based on the file path

Suppose you have a file at location $PATH/a.csv and we want to read and export data( elastic search server).Let's say you are trying to read this log file through the following configuration

input {
  file {
    path => ["/some/path/a.csv"]
    start_position => "beginning"
  }
}

So, when you run this for the first time, this will work perfectly. The point to note here is that, when you have run this for the first time, logstash will automatically create a sincedb_path file at your home directory and it might look like .sincedb_1a892d89f927eb484cb12a7b7385ab72. This file tells the logstash that it has already processed this file.

If you want to reprocess this file, all you need to do is just delete this file.

OR

A better solution would be to use the following configurations

input {
  file {
    path => ["/some/path/a.csv"]
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

The above configuration will always write the .sincedb_path file to the /dev/null path.

/dev/null

/dev/null is a simple device (implemented in software and not corresponding to any hardware device on the system).

/dev/null looks empty when you read from it. Writing to /dev/null does nothing: data written to this device simply "disappear."