Hello,
I am using Logstash to ingest CSV files into Elasticsearch. I am using a Linux Machine (CentOS7).
I have Latitude and Longitude information in my csv.
Hence I have created a mapping like below:
PUT mks_locdata/_mapping/loc_data
{
"loc_data": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
Then I am using the below logstash config file for data ingestion:
input {
file {
path => "/opt/elastic/elasticsearch/Data/Locations.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["City", "State", "Lat", "Long"]
}mutate { convert => { "Long" => "float" } convert => { "Lat" => "float" } } mutate { rename => { "Long" => "[location][lon]" "Lat" => "[location][lat]" }}
output {
elasticsearch {
hosts => "localhost"
index => "mks_locdata"
document_type => "loc_data"
}
stdout {}
}
I am getting the below error:
Sending Logstash logs to /opt/elastic/elasticsearch/logstash-6.4.1/logs which is now configured via log4j2.properties
[2018-10-03T06:17:42,756][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-10-03T06:17:43,710][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.1"}
[2018-10-03T06:17:44,891][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 26, column 23 (byte 575) after filter {\r\n csv {\r\n separator => ","\r\n columns => ["City", "State", "Lat", "Long"]\r\n }\r\n\r\n mutate {\r\n\t\t\tconvert => { "Long" => "float" }\r\n\t\t\tconvert => { "Lat" => "float" }\r\n\t\t}\r\n\r\n mutate {\r\n\t\t\trename => {\r\n\t\t\t\t"Long" => "[location][lon]"\r\n\t\t\t\t"Lat" => "[location][lat]"\r\n\t\t}\r\n}\r\noutput {\r\n elasticsearch ", :backtrace=>["/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:49:in compile_graph'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2486:in map'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:149:in initialize'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/pipeline.rb:22:in initialize'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/pipeline.rb:90:in initialize'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/pipeline_action/create.rb:38:in execute'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/agent.rb:309:in block in converge_state'"]}
[2018-10-03T06:17:45,331][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
What am I missing here?