Input json file to logstash issues


(Tal Bar Or) #1

Hi,

I am trying to input an tshark json output and output it "ruby_debug" for first stage facing issues
Tried several ways inout i searched but keep getting errors showed below , any i idea hot input correctly such file format?
please advice
Thanks

=====console error============
C:\ELK\logstash-5.4.0\bin>logstash --verbose -f loginput.conf
JAVA_OPTS was set to [ -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="C:\ELK\logstash-5.4.0/heapdump.hprof"]. Logstash will trust these options, and not set any defaults that it might usually set
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Could not find log4j2 configuration at path /ELK/logstash-5.4.0/config/log4j2.properties. Using default config which logs to console
22:12:33.968 [LogStash::Runner] ERROR logstash.plugins.registry - Problems loading a plugin with {:type=>"codec", :name=>"ruby_debug", :path=>"logstash/codecs/ruby_debug", :error_message=>"NameError", :error_class=>NameError, :error_backtrace=>["C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/plugins/registry.rb:221:in namespace_lookup'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/plugins/registry.rb:157:inlegacy_lookup'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/plugins/registry.rb:133:in lookup'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/plugins/registry.rb:175:inlookup_pipeline_plugin'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/plugin.rb:139:in lookup'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/config/mixin.rb:448:invalidate_value'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/config/mixin.rb:369:in process_parameter_value'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/config/mixin.rb:388:invalidate_check_parameter_values'", "org/jruby/RubyArray.java:1613:in each'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/config/mixin.rb:382:invalidate_check_parameter_values'", "org/jruby/RubyHash.java:1342:in each'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/config/mixin.rb:381:invalidate_check_parameter_values'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/config/mixin.rb:271:in validate'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/config/mixin.rb:129:inconfig_init'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/outputs/base.rb:63:in initialize'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/output_delegator_strategies/single.rb:3:ininitialize'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/output_delegator.rb:23:in initialize'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/pipeline.rb:94:inplugin'", "(eval):12:in initialize'", "org/jruby/RubyKernel.java:1079:ineval'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/pipeline.rb:63:in initialize'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/pipeline.rb:145:ininitialize'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/agent.rb:286:in create_pipeline'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/agent.rb:95:inregister_pipeline'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/runner.rb:274:in execute'", "C:/ELK/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:inrun'", "C:/ELK/logstash-5.4.0/logstash-core/lib/logstash/runner.rb:185:in run'", "C:/ELK/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:inrun'", "C:\ELK\logstash-5.4.0\lib\bootstrap\environment.rb:71:in `(root)'"]}
22:12:34.372 [LogStash::Runner] ERROR logstash.agent - Cannot create pipeline {:reason=>"Couldn't find any codec plugin named 'ruby_debug'. Are you sure this is correct? Trying to load the ruby_debug codec plugin resulted in this error: Problems loading the requested plugin named ruby_debug of type codec. Error: NameError NameError"}

====the input config file========
input{
file{
codec => json_lines
sincedb_path => "dev/null"
path => "file.json"
start_position => "beginning"
}
}
output{
stdout{codec => ruby_debug}
}
======the jsonfile=========
[
{
"_index": "packets-2017-05-22",
"_type": "pcap_file",
"_score": null,
"_source": {
"layers": {
"frame.number": ["26"],
"frame.time_relative": ["1.560480000"],
"ip.src": ["172.23.136.73"],
"ip.dst": ["172.23.254.88"],
"tcp.flags.syn": ["1"],
"tcp.flags.ack": ["0"]
}
}
}

,
{
"_index": "packets-2017-05-22",
"_type": "pcap_file",
"_score": null,
"_source": {
"layers": {
"frame.number": ["27"],
"frame.time_relative": ["1.560558000"],
"ip.src": ["10.10.1.1"],
"ip.dst": ["172.23.254.88"],
"tcp.flags.syn": ["1"],
"tcp.flags.ack": ["0"]
}
}
}

,
{
"_index": "packets-2017-05-22",
"_type": "pcap_file",
"_score": null,
"_source": {
"layers": {
"frame.number": ["28"],
"frame.time_relative": ["1.560616000"],
"ip.src": ["172.23.136.73"],
"ip.dst": ["172.23.254.88"],
"tcp.flags.syn": ["1"],
"tcp.flags.ack": ["0"]
}
}
}

,
{
"_index": "packets-2017-05-22",
"_type": "pcap_file",
"_score": null,
"_source": {
"layers": {
"frame.number": ["29"],
"frame.time_relative": ["1.560707000"],
"ip.src": ["172.23.254.88"],
"ip.dst": ["172.23.136.73"],
"tcp.flags.syn": ["1"],
"tcp.flags.ack": ["1"],
"tcp.analysis.acks_frame": ["26"],
"tcp.analysis.ack_rtt": ["0.000227000"]
}
}
}

,
{
"_index": "packets-2017-05-22",
"_type": "pcap_file",
"_score": null,
"_source": {
"layers": {
"frame.number": ["30"],
"frame.time_relative": ["1.560713000"],
"ip.src": ["172.23.136.73"],
"ip.dst": ["172.23.254.88"],
"tcp.flags.syn": ["0"],
"tcp.flags.ack": ["1"],
"tcp.analysis.acks_frame": ["29"],
"tcp.analysis.ack_rtt": ["0.000006000"]
}
}
}
]


(Mark Walkom) #2

You want rubydebug - https://www.elastic.co/guide/en/logstash/current/plugins-codecs-rubydebug.html


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.