I did try running it straight from the GCE VM. I was actually getting slower indexing rates (~300-400/sec).
I am using out of the box configurations. I'm assuming I should still see better results.
@javadevmtl I like the idea of disabling replicas. I'm going to try that. Here is my logstash config. See anything wrong?
input {
file {
path => "/elasticsearch/data.csv"
start_position => "beginning"
type => "data"
}
}
filter {
csv{
separator => "|"
}
}
output {
elasticsearch {
action => "index"
host => "localhost"
port => "9200"
index => "indextest-data1"
workers => 2
#cluster => "elasticsearch-cluster"
protocol => "http"
cluster => "elasticsearch-cluster"
}
#stdout { codec => json }
}
Thanks for the help so far.