What logstash input plugins can you set sincedb_path for?

I have seen some Logstash configuration online like this:

input {
  file {
    start_position => "beginning"
    sincedb_path => "/pathToSincedb/sincedb"
    path => "/pathToYourData/*"

As we can see, there is the sincedb_path that is set for the File input plugin.


  1. What if the input plugin is http_poller or cloudwatch_logs, can I still use sincedb_path?
  2. If yes, what should I set my sincedb_path as if I am going to use a docker? With dockers, things are a bit different from usual path setup. I've heard I need to mount the path in my docker file.
  3. If no, how can I tell my program where it left off when reading the logs again?


  • I understand that sincedb_path is by default set to null, but I don't want that. I want my program to keep track of what logs have been read so far, so the next time I restart my program again, it can start from where it left off.
  • Setup sincedb_path in my Logstash configuration that is used in a docker file.

I appreciate any feedback/suggestions.

No, by default the file input will generate a valid (but obscure) sincedb_path and persist the in-memory sincedb across restarts. Only a file input uses a sincedb to persist state. A jdbc input can use sql_last_value. It is possible that other inputs use other options, but I have no experience of those.

1 Like

@Badger, I don't know what else to use other than sincedb_path for cloudwatch_logs input plugin.

According to lukewaite, sincedb_path can be used:

Could you please tell me what should I use for http_poller? Is that considered jdbc?

An http_poller input does not maintain state, and it is not jdbc.

1 Like

I see, I guess I can just create document_id for my output, so in that case I can overwrite the previous logs again with the same document_id.

For more information:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.