Everything looks fine but still the data is not imported and not displayed on the console

Elastic Stack Logstash
1

C:\CSV Files>logstash -f summaryerrorbyfrequency.conf
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; support was removed in 14.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option CMSInitiatingOccupancyFraction; support was removed in 14.0
Java HotSpot(TM) 64-Bit Server VM warning: Ignoring option UseCMSInitiatingOccupancyOnly; support was removed in 14.0
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/C:/Users/chandana.k/AppData/Local/Temp/jruby-10312/jruby6722910307155946873jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to C:/ELK/logstash-7.9.0/logstash-7.9.0/logs which is now configured via log4j2.properties
[2020-08-25T16:09:32,361][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.9.0", "jruby.version"=>"jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 Java HotSpot(TM) 64-Bit Server VM 14.0.2+12-46 on 14.0.2+12-46 +indy +jit [mswin32-x86_64]"}
[2020-08-25T16:09:32,500][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-08-25T16:09:33,349][INFO ][org.reflections.Reflections] Reflections took 21 ms to scan 1 urls, producing 22 keys and 45 values
[2020-08-25T16:09:34,256][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[http://localhost:9200/]}}
[2020-08-25T16:09:34,372][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2020-08-25T16:09:34,407][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2020-08-25T16:09:34,412][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>7}
[2020-08-25T16:09:34,449][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]}
[2020-08-25T16:09:34,484][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2020-08-25T16:09:34,509][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/CSV Files/summaryerrorbyfrequency.conf"], :thread=>"#<Thread:0x27c522b8 run>"}
[2020-08-25T16:09:34,542][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2020-08-25T16:09:35,016][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.5}
[2020-08-25T16:09:35,294][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-08-25T16:09:35,318][INFO ][filewatch.observingtail ][main][2de158a34d87370e4dfe99272c7b5e2d8f4db7e0124461db740371758c1cfb95] START, creating Discoverer, Watch with file and sincedb collections
[2020-08-25T16:09:35,331][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2020-08-25T16:09:35,515][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

2

Can you post your pipeline in .conf file?

3

summaryerrorbyfrequency.conf file

input {
file {
path => "C:\CSV Files\SummaryErrorByFrequencyLog1.csv"
start_position => "beginning"
}
}

filter {
csv {
separator => ","
columns => ["ErrorCode","Retort Id","Is Overrideable","Severity","Error Count","1st Occurrence Date","1st Occurrence Time","Last Occurrence Date","Last Occurrence Time","Last reset date","Last reset time","English Text","Local Text"]
}
}

output {
elasticsearch {
hosts => "http://localhost:9200"
index => "frequencylog"
}
stdout {}
}

4

Do I have to change anything in pipelines.yml file?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.