Logstash Configuration Error: "Expected one of #,"


(Rohini Basu) #1

Hello,

I am using Logstash to ingest CSV files into ElasticSearch. I am using a Linux Machine (CentOS7).

I have Latitude and Longitude information in my csv.

Hence I have created a mapping like below:
PUT mks_locdata/_mapping/loc_data
{
"loc_data": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}

Then I am using the below logstash config file for data ingestion:

input {
file {
path => "/opt/elastic/elasticsearch/Data/Locations.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["City", "State", "Lat", "Long"]
}

    mutate {
  	convert => { "Long" => "float" }
  	convert => { "Lat" => "float" }
  }

    mutate {
  	rename => {
  		"Long" => "[location][lon]"
  		"Lat" => "[location][lat]"
  }

}
output {
elasticsearch {
hosts => "localhost"
index => "mks_locdata"
document_type => "loc_data"
}
stdout {}
}

I am getting the below error:
Sending Logstash logs to /opt/elastic/elasticsearch/logstash-6.4.1/logs which is now configured via log4j2.properties
[2018-10-03T06:17:42,756][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-10-03T06:17:43,710][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.4.1"}
[2018-10-03T06:17:44,891][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 26, column 23 (byte 575) after filter {\r\n csv {\r\n separator => ","\r\n columns => ["City", "State", "Lat", "Long"]\r\n }\r\n\r\n mutate {\r\n\t\t\tconvert => { "Long" => "float" }\r\n\t\t\tconvert => { "Lat" => "float" }\r\n\t\t}\r\n\r\n mutate {\r\n\t\t\trename => {\r\n\t\t\t\t"Long" => "[location][lon]"\r\n\t\t\t\t"Lat" => "[location][lat]"\r\n\t\t}\r\n}\r\noutput {\r\n elasticsearch ", :backtrace=>["/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:41:in compile_imperative'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:11:in block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:149:ininitialize'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/pipeline.rb:22:in initialize'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/pipeline.rb:90:ininitialize'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/pipeline_action/create.rb:38:in execute'", "/opt/elastic/elasticsearch/logstash-6.4.1/logstash-core/lib/logstash/agent.rb:309:inblock in converge_state'"]}
[2018-10-03T06:17:45,331][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

What am I missing here?


(Christian Dahlqvist) #2

Looks like you may be missing a closing } for the last mutate filter. This type of issues is usually a lot easier to spot if you indent and format your config properly.


(Rohini Basu) #4

Thanks. That is the issue.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.