# Sincedb is not created **URL:** \ **Category:** Logstash\ **Created:** [June 13, 2016, 6:47pm UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648 "2016-06-13T18:47:50Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![cokeroman](https://avatars.discourse-cdn.com/v4/letter/c/cc9497/32.png) [@cokeroman](https://discuss.elastic.co/u/cokeroman)\ **Post date:** [June 13, 2016, 6:47pm UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648/1 "2016-06-13T18:47:50Z")
Hello all, I just update my logstash server from 1.5.4 to 2.3.2. I have to update some config parameters and I am collect log from several file and indexing to elasticsearch. My problem is when the logstash processs starts it doesn't create sincedb files and this cause data lose. For instance, if i stop the deamon for a while and i start it again few minutes later, logstash doesn't continue in the line where the process stop, it starts with lastest lines. This is my logstash configuration: input { file { path =\> "/var/log/varnish/varnishncsa.log" type =\> "varnish" sincedb\_write\_interval =\> "5" sincedb\_path =\> "/opt/logstash/logs/.since\_varnishncsa.log" } ``` file { path => "/var/log/varnish/recovery.log" type => "varnish" sincedb_write_interval => "5" sincedb_path => "/opt/logstash/logs/.since_recovery.log" } file { path => "/var/log/nginx/access.log" type => "https" sincedb_write_interval => "5" sincedb_path => "/opt/logstash/logs/.since_access_nginx.log" } ``` } filter { if [type] == "varnish" { grok { patterns\_dir =\> "/opt/logstash/patterns" match =\> { "message" =\> "(%{IP:clientip}|-) (%{USER:ident}|-) (%{USER:auth}|-) [%{HTTPDATE:timestamp}] "%{WORD:method} http://%{HOSTNAME:vhost}%{URIPATHPARAM:request}([%{WORD:garbage}])? HTTP/%{NUMBER:httpversion}" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:agent} (%{WORD:cached}|-) %{QS:content\_type} %{QS:node} %{NUMBER:response\_time} (%{QS:clientid}|-) (%{QS:referer}|-)"} } ``` date { match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"] add_tag => ["dated"] } } if [type] == "https" { grok { patterns_dir => "/opt/logstash/patterns" match => { "message" => "(%{IP:clientip}|-) (%{USER:ident}|-) (%{USER:auth}|-) \[%{HTTPDATE:timestamp}\] %{HOSTNAME:vhost} \"%{WORD:method} %{URIPATHPARAM:request}(\[%{WORD:garbage}\])? HTTP/%{NUMBER:httpversion}\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) (?:\"(?:%{URI:referrer}|-)\"|%{QS:referrer}) %{QS:agent} (%{QS:clientid}|-)"} } date { match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"] add_tag => ["dated"] } } ``` } output { elasticsearch { hosts =\> ["[http://localhost:9200](http://localhost:9200)"] flush\_size =\> "5000" idle\_flush\_time =\> "1" #index =\> "logstash-%{index\_date}" index =\> "logstash-%{+YYYY.MM.dd.HH}" } } Any clue??? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 14, 2016, 5:45am UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648/2 "2016-06-14T05:45:08Z")
What user does Logstash run as? What are the file permissions of /opt/logstash/logs and /opt/logstash/logs/.since\_recovery.log? Are there any clues in the Logstash log is you crank up the log level with `--verbose` or `--debug`? ---
**Author:** ![cokeroman](https://avatars.discourse-cdn.com/v4/letter/c/cc9497/32.png) [@cokeroman](https://discuss.elastic.co/u/cokeroman)\ **Post date:** [June 14, 2016, 6:15am UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648/3 "2016-06-14T06:15:04Z")
Hey Magnus, thanks for your reply, Logstash is running as root user, so is not a privileges problem, in fact its so weird because i have 3 input file (/var/log/varnish/varnishncsa.log, /var/log/nginx/access.log and /var/log/varnish/recovery.log) and only I hae problem with the first one (the other are created and updated right). Curiously, /var/log/varnish/varnishncsa.log is the most updated file that is because it's so odd this behavior. I run logstash with --verbose options and all i can see was this error but I don't what that means: {:timestamp=\>"2016-06-13T20:00:54.898000+0200", :message=\>"Ruby exception occurred: can't dup NilClass", :level=\>:error} Jorge ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 14, 2016, 6:16am UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648/4 "2016-06-14T06:16:31Z")
There's no mention of any of the sincedb files in the `--debug` output? Is there any more context to the "can't dup NilClass" error? ---
**Author:** ![cokeroman](https://avatars.discourse-cdn.com/v4/letter/c/cc9497/32.png) [@cokeroman](https://discuss.elastic.co/u/cokeroman)\ **Post date:** [June 14, 2016, 7:20am UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648/5 "2016-06-14T07:20:36Z")
yes i can see this when i run logstash in debug mode: {:timestamp=\>"2016-06-14T09:09:42.537000+0200", :message=\>"writing sincedb (delta since last write = 5)", :level=\>:debug, :file=\>"filewatch/observing\_tail.rb", :line=\>"102", :method=\>"observe\_read\_file"} however the file /opt/logstash/logs/.since\_varnishncsa.log is not created. i have seen a very interesting thing. Sometimes, when I have less events to send (no peak time), the file is created and updated frequently. However in peak time the some time the sincedb file is not created and when is created is not updated frequently. In logstash 1.5 this configuration was working fine. I really don't know what is happend. ---
**Author:** ![L4rS6](https://avatars.discourse-cdn.com/v4/letter/l/c2a13f/32.png) [@L4rS6](https://discuss.elastic.co/u/L4rS6)\ **Post date:** [December 22, 2016, 3:52pm UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648/6 "2016-12-22T15:52:10Z")
I got the same problem with Logstash 2.3.2. Since-DB file won't be updated if there are a lot of events... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:29am UTC](https://discuss.elastic.co/t/sincedb-is-not-created/52648/7 "2017-07-06T04:29:43Z")