Hello community,
I am trying to use translate filter where I add a yaml file with list of ips. When the ips in the csv don't match with IP in yaml file a new field is created "Added_asset" with value as new. Please help debugging the error
input {
file {
path => "C:/Users/kumar/Downloads/Sample_HPAM data.csv"
start_position => "beginning"
sincedb_path => "NULL"
}
}
filter {
csv {
separator => ","
columns => ["Devicetype","Assettag","AssignmentStatus","Makemodel","IP","Hostname","Serialnumber","OS","OSversion","NEID","Location","Business_Domain","BusinessDomianOwner","Application","AppLead","OPerationalSPOC","Platform","platformlead","HWMGMT","OSMGMT","OOBIP","Environment","Assignment Date","LastScanDt","AdditionalIP","","OperationalStatus"
]
}
translate{
field => "IP"
destination => "Added_asset"
fallback => "new"
dictionary_path => { 'C:/Users/kumar/Downloads/ip.yml'
}
}
}
output {
elasticsearch{
hosts => "http://localhost:9200/"
index => "hpam"
user => "elastic"
password => "XibKsoGe4p0uMhib6FdT"
}
stdout{}
}
Error:
Microsoft Windows [Version 10.0.19042.1083]
(c) Microsoft Corporation. All rights reserved.
C:\Users\kumar>cd C:\Users\kumar\Downloads\
C:\Users\kumar\Downloads>logstash -f logstash.conf
"Using bundled JDK: ""
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to C:/elastic_stack/logstash-7.12.1-windows-x86_64/logstash-7.12.1/logs which is now configured via log4j2.properties
[2021-07-16T17:17:37,440][INFO ][logstash.runner ] Log4j configuration path used is: C:\elastic_stack\logstash-7.12.1-windows-x86_64\logstash-7.12.1\config\log4j2.properties
[2021-07-16T17:17:37,460][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.12.1", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.10+9 on 11.0.10+9 +indy +jit [mswin32-x86_64]"}
[2021-07-16T17:17:37,774][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-07-16T17:17:39,407][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2021-07-16T17:17:39,827][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "=>" at line 21, column 3 (byte 726) after filter {\r\n csv {\r\n separator => ","\r\n columns => ["Devicetype","Assettag","AssignmentStatus","Makemodel","IP","Hostname","Serialnumber","OS","OSversion","NEID","Location","Business_Domain","BusinessDomianOwner","Application","AppLead","OPerationalSPOC","Platform","platformlead","HWMGMT","OSMGMT","OOBIP","Environment","Assignment Date","LastScanDt","AdditionalIP","","OperationalStatus"\r\n]\r\n }\r\n\r\n translate{\r\n field => "IP"\r\n destination => "Added_asset"\r\n fallback => "new"\r\n dictionary_path => { 'C:/Users/kumar/Downloads/ip.yml'\r\n \r\n ", :backtrace=>["C:/elastic_stack/logstash-7.12.1-windows-x86_64/logstash-7.12.1/logstash-core/lib/logstash/compiler.rb:32:in compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:184:in initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in initialize'", "C:/elastic_stack/logstash-7.12.1-windows-x86_64/logstash-7.12.1/logstash-core/lib/logstash/java_pipeline.rb:47:in initialize'", "C:/elastic_stack/logstash-7.12.1-windows-x86_64/logstash-7.12.1/logstash-core/lib/logstash/pipeline_action/create.rb:52:in execute'", "C:/elastic_stack/logstash-7.12.1-windows-x86_64/logstash-7.12.1/logstash-core/lib/logstash/agent.rb:389:in block in converge_state'"]}
[2021-07-16T17:17:39,993][INFO ][logstash.runner ] Logstash shut down.
[2021-07-16T17:17:40,009][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
at C_3a_.elastic_stack.logstash_minus_7_dot_12_dot_1_minus_windows_minus_x86_64.logstash_minus_7_dot_12_dot_1.lib.bootstrap.environment.(C:\elastic_stack\logstash-7.12.1-windows-x86_64\logstash-7.12.1\lib\bootstrap\environment.rb:89) ~[?:?]
C:\Users\kumar\Downloads>