[ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=>“filter”, :name=>“CSV”, :path=>“logstash/filters/CSV”, :error _message=>“NameError”

[vikaskataria]

Hi,

I am new to ELK stack. I am using Logstash 5.5.2 and ElasticSearch 5.5.2 and Kibana 5.5.2. I am trying to push data from a CSV file into Elastic search using Logstash. I am getting Syntax Error: Unexpected tLT from CSV filter plugin when i run the command from my Windows Command Processor (CMD or DOS prompt).

logstash -f Jmeterogstash.conf

l

logstash -f Jmeterlogstash.conf

Copy pasting the contents of Jmeterlogstash.conf file below:

input {
file {
path => "D:/ApacheJmeter31/bin/MyOutput/output.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {
CSV {
separator => ","
columns => ["Timestamp", "Elapsed"]
}
mutate {convert => ["Timestamp", "integer"]}
mutate {convert => ["Elapsed", "integer"]}

}
output {
elasticsearch {
hosts => "localhost"
index => "jmeter"
document_type => "ELIntegration"
}
stdout { codec => rubydebug }
}

I have downloaded the CSV filter from the below location in Github:

logstash-plugins/logstash-filter-csv

Do I need ti install this filter? if yes then how?
Please help me in completing the task i am trying to do?strong text

Screenshot of CSV file

[magnusbaeck (Magnus Bäck)]

See https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html#installing-plugins for how to install Logstash plugins.

Please don't post screnshots if you can copy/paste regular text.

[vikaskataria]

Thank you very much Magnus for your reply.

I downloaded the logstash-filter-csv 3.0.5 from the https://rubygems.org/gems/logstash-filter-csv. Then i gave the below command
logstash-plugin install logstash-filter-csv-3.0.3.gem

I saw the message that the filter is installed successfully.
Then I gave the below command to verify the the list of installed filters with below command
logstash-plugin list

I saw that a filter with the name "logstash-filter-csv" is there in the list. Now i gave the below command to run logstash
logstash -f Jmeterogstash.conf

I was hoping that now logstash will read the values from the CSV file but it didn't happen instead i saw the below error messages:
D:\ELKStack\logstash552\bin>logstash -f Jmeterlogstash.conf
Picked up _JAVA_OPTIONS: -Xmx512M -Xms256M
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.Sending Logstash's logs to D:/ELKStack/logstash552/logs which is now configured via log4j2.properties [2017-08-26T01:58:54,475][ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=>"filter", :name=>"CSV", :path=>"logstash/filters/CSV", :error _message=>"NameError", :error_class=>NameError, :error_backtrace=>["D:/ELKStack/logstash552/logstash-re/lib/logstash/plugins/registry.rb:226:in namespace_lookup'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/plugins/registry.rb:162:inlegacy_lookup'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/plugins/registry.rb:138:in lookup'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/plugins/registry.rb:180:inlookup_pipeline_plugin'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/plugin.rb:140:in lookup'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/pipeline.rb:100:inplugin'", "(eval):12:in initialize'", "org/jruby/RubyKernel.java:1079:ineval'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/pipeline.rb:72:in initialize'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/pipeline.rb:156:ininitialize'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/agent.rb:286:in create_pipeline'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/agent.rb:95:inregister_pipeline'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/runner.rb:314:in execute'", "D:/ELKStack/logstash552/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:inrun'", "D:/ELKStack/logstash552/logstash-core/lib/logstash/runner.rb:209:in run'", "D:/ELKStack/logstash552/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:inrun'", "D:\ELKStack\logstash552\lib\bootstrap\environment.rb:71:in `(root)'"]}[2017-08-26T01:58:54,491][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Couldn't find any filter plugin named 'CSV'. Are you sure this is correct? Trying to load the CSV filter plugin resulted in this error: Problems loading the requested plugin named CSV of type filter. Error: NameError NameError"}

D:\ELKStack\logstash552\bin>

Please help me to resolve this issue.

Pasting the config file once again

input {
file {
path => "D:/ApacheJmeter31/bin/MyOutput/output.csv"
start_position => "beginning"
sincedb_path => “/dev/null”
}
}

filter {
CSV {
separator => ","
columns => [“Timestamp”, “Elapsed”]
}
mutate {convert => [“Timestamp”, “integer”]}
mutate {convert => [“Elapsed”, “integer”]}

}
output {
elasticsearch {
hosts => "localhost"
index => "jmeter"
document_type => “ELIntegration”
}
stdout { codec => rubydebug }
}

Is there any problem i this config file? Please help.

[magnusbaeck (Magnus Bäck)]

The filter is named csv, not CSV.

[vikaskataria]

Thank you very much Magnus for pointing out the mistake.

I have corrected the same and now I can see that Logstash is picking data from the CSV file and and transferring into Elasticsearch.

Really appriciate your help, it means a lot to me, Thank you very much once again.

This thread can be closed now.