Where is my elasticsearch data?

Elastic Stack Elasticsearch
1

i have csv file and using logstash i ingest this file to elasticsearch and viewing the count process on kibana .
two things i want to know

  1. where is the csv file i just ingested in elasticsearch in windows. i have checked the data folder of elasticsearch but its not there
  2. in kibana i was showing the same number of count as that in csv file and later after some time i checked the count becomes twice . i didnt ingest the file again.
2

Can you show your logstash configuration file?

3

input {
file {
path => "F:\Data\cars.csv"
start_position => "beginning"

     }
   }

filter {
csv {
separator => ","
columns => ["car_model","car_name","manufacturer,"plant_location"]
}
}

output {
elasticsearch {
hosts => "localhost:9200"
index => "cars"
document_type => "car_record"
}
stdout { }
}

4

Hi sheryl,

Answring about the question number 2:

When you set the "start_position" to "beginning" all the lines will be read each new line added to the file...not only the new lines...So a suggest that you change this to "end" so logstash will read only the new lines that will be appended in the file.... or you can also put an uuid/fingerprint filter to eliminate the duplicates.
see the docs: https://www.elastic.co/guide/en/logstash/6.x/plugins-filters-fingerprint.html

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.