[Logstash] Plugin split filter

Elastic In Your Native Tongue Discussions en français
1

Bonjour,

Lorsque je split un xml avec Logstash, j'ai l'erreur suivante:

Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>#<LogStash::ConfigurationError: Only String and Array types are splittable.  
field:[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId] is of type = Hash>

, "backtrace"=>
["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-split-2.0.5/lib/logstash/filters/split.rb:49:in `filter'", 
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/filters/base.rb:151:in 
`multi_filter'", "org/jruby/RubyArray.java:1613:in `each'", 
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/filters/base.rb:148:in 
`multi_filter'", "(eval):325:in `initialize'", "org/jruby/RubyArray.java:1613:in `each'", "(eval):316:in `initialize'", 
"org/jruby/RubyProc.java:281:in `call'", "(eval):302:in `filter_func'", 
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:267:in 
`filter_batch'", "org/jruby/RubyArray.java:1613:in `each'", "org/jruby/RubyEnumerable.java:852:in `inject'", 
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:265:in 
`filter_batch'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-
 java/lib/logstash/pipeline.rb:223:in `worker_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-
core-
2.4.1-java/lib/logstash/pipeline.rb:201:in `start_workers'"], :level=>:error}
LogStash::ConfigurationError: Only String and Array types are splittable. field:[parsed][Body]
[FlightListByAerodromeReply][data][flights][flight][flightId] is of type = Hash
filter at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-split-2.0.5/lib/logstash/filters/split.rb:49
multi_filter at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-
java/lib/logstash/filters/base.rb:151
       each at org/jruby/RubyArray.java:1613
multi_filter at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-
java/lib/logstash/filters/base.rb:148
 initialize at (eval):325
       each at org/jruby/RubyArray.java:1613
 initialize at (eval):316
       call at org/jruby/RubyProc.java:281
filter_func at (eval):302
 filter_batch at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-
java/lib/logstash/pipeline.rb:267
       each at org/jruby/RubyArray.java:1613
     inject at org/jruby/RubyEnumerable.java:852
filter_batch at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:265
worker_loop at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:223
start_workers at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:201

Voici ma configuration Logstash:

filter {
    if [type] == "FlightListByAerodromeReply" {
xml {
    source => "message"
    #store_xml => false
    target => "parsed"
    #force_array => false
}
split {
   field => "[parsed][Body]"
}
split {
   field => "[parsed][Body][FlightListByAerodromeReply]"
}
split {
  field => "[parsed][Body][FlightListByAerodromeReply][data]"
}
split {
  field => "[parsed][Body][FlightListByAerodromeReply][data][effectiveTrafficWindow]"
}
split {
  field => "[parsed][Body][FlightListByAerodromeReply][data][flights]"
}
split {
  field => "[parsed][Body][FlightListByAerodromeReply][data][flights][flight]"
}
split {
  field => "[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId]"
}
split {
  field => "[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys]"
}
mutate {
  add_field => { flightId => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][id]}" }
  add_field => { aircraftId => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys]
 [aircraftId]}" }
add_field => { aerodromeOfDestination => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys][aerodromeOfDestination]}" }
add_field => { estimatedOffBlockTime => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys][estimatedOffBlockTime]}" }
add_field => { airFiled => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys][airFiled]}" }
add_field => { nonICAOAerodromeOfDestination => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys][nonICAOAerodromeOfDestination]}" }
add_field => { nonICAOAerodromeOfDeparture => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys][nonICAOAerodromeOfDeparture]}" }
add_field => { aerodromeOfDeparture => "%{[parsed][Body][FlightListByAerodromeReply][data][flights][flight][flightId][keys][aerodromeOfDeparture]}" }
add_field => { wef => "%{[parsed][Body][FlightListByAerodromeReply][data][effectiveTrafficWindow][wef]}" }
add_field => { unt => "%{[parsed][Body][FlightListByAerodromeReply][data][effectiveTrafficWindow][unt]}" }
add_field => { requestReceptionTime => "%{[parsed][Body][FlightListByAerodromeReply][requestReceptionTime]}" }
add_field => { sendTime => "%{[parsed][Body][FlightListByAerodromeReply][sendTime]}" }
add_field => { status => "%{[parsed][Body][FlightListByAerodromeReply][status]}" }
remove_field => [ "message" ]
remove_field => [ "parsed" ]
    }
  }
}
  • Version Logstash (Docker): 'logstash:2.4.1-alpine'.
  • Version plugin 'logstash-filter-split': '2.0.5'.

Merci de votre aide.
Benjamin

3

l'erreur semble apparaitre au dernier split donc ca devrait etre specifique au fichier xml puisque flightid est un hash au lieu d'un array ou String. Pouvez-vous joindre le xml ?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.