Logstash does not creates nor updates index on elasticsearch

[Quentin_Moisy]

Hello, I new to the ELK flow and I have some issues with Logstash. Sometime my index will be populated sometime not. Furthermore it seems that logstash does not create index on elasticsearch. Can you help on that

My .conf file

input { 
     file {
   path => "C:/Work/elastic/log.csv"
   start_position => "beginning"
 }
} 

filter {
	csv {
		separator => ","
		columns => [
			"_time",
			"uid",
			"level"	
		]
	}
}
output { 
    
elasticsearch { 
        hosts => ["https://localhost:9200"]
        user => "elastic" 
        password => "password" 
	ssl_certificate_verification => false
	}

}

[jba (Jan Bruun Andersen)]

Read the documentation for the logstash-output-elasticsearch plugin. I think that you need to add a index => "......" somewhere in the elasticsearch {...} part to send the documents to your chosen index.

[Rios]

If you don't specify index name your data will end up in index:

• ECS Compatibility disabled: "logstash-%{+yyyy.MM.dd}"
• ECS Compatibility enabled: "ecs-logstash-%{+yyyy.MM.dd}"
  Default is v8 in 8.x version.

As Jan said, if you want separated index, just set:

elasticsearch { 
        hosts => ["https://localhost:9200"]
        index => "csvlog-%{+YYYY.MM.dd}"
        user => "elastic" 
        password => "password" 
	ssl_certificate_verification => false
	}

[Quentin_Moisy]

My apologies I didn't sent the right .conf. In the right one I have

index => "idx"

but I still have the issues

[Rios]

And again...
Since you have not set sincedb_path, it will be created and LS keeps tracking about read lines.

Option 1 Add sincedb_path => "NUL" (on Windows) bellow start_position, LS will not use the sincedb database file, on every LS restart will read from the beginning.

Option 2 Delete sincedb before everyrestart.

[Quentin_Moisy]

I tried it again today and it works, thanks Rios.
The solution was to add sincedb_path => "NUL" bellow start_position as you mentioned.

input { 
     file {
       path => "C:/Work/elastic/log.csv"
       start_position => "beginning"
       sincedb_path => "NUL"
 }
}