I have a 10MB file in size with 500000 lines. Each line has this text:
text_entry,$i
where $i = line number
My logstash.conf file looks like this:
input { file {
path => "C:/Users/anasdima/Workspace/data-tests/10MB-data.csv"
mode => "read"
} }
}
output {
elasticsearch { hosts => ["localhost:9200"] }
}
This takes 9 minutes to finish, which to me means that something is not right.
On the other hand, when I upload 10MB-data.csv from Kibana, it takes a couple of seconds.
What is making such a huge difference?
pjanzen
January 29, 2019, 8:24am
2
Hi,
How and where in kibana do you upload the file? Can you elaborate on that a bit more please.
What version are you using for logstash and elasticsearch. What hardware specs do you have (cpu, mem, disk type).
Paul.
I am using this ui from Kibana to upload the file:
I am using 6.5.4 Elastic, 6.5.4 Kibana and 6.5.4 logstash. 64 bit versions, running on Windows 10 Pro.
My CPU is i5-6600, RAM 16GB, Hard Drive (not SSD)
pjanzen
January 29, 2019, 9:44am
4
When you use the this interface (which I cannot find in my kibana btw) does it end up in a index somewhere?
I navigate like this:
yes it ends up in an index
system
February 26, 2019, 9:52am
6
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
