# Data ingestion error

**URL:** <https://discuss.elastic.co/t/data-ingestion-error/365467>\
**Category:** Logstash\
**Created:** [August 23, 2024, 3:25pm UTC](https://discuss.elastic.co/t/data-ingestion-error/365467 "2024-08-23T15:25:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jaime\_solas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaime_solas/32/136699_2.png) [@jaime\_solas](https://discuss.elastic.co/u/jaime_solas)\
**Post date:** [August 23, 2024, 3:25pm UTC](https://discuss.elastic.co/t/data-ingestion-error/365467/1 "2024-08-23T15:25:47Z")

</div>

When I perform the ingestion everything seems to be fine, the information is displayed correctly in kibana but during the process the following errors appear

```auto
[ERROR] 2024-08-23 15:08:25.370 [[main]>worker2] ruby - Ruby exception occurred: undefined method `each' for nil:NilClass {:class=>"NoMethodError", :backtrace=>["(ruby filter code):4:in `block in register'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:96:in `inline_script'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:89:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:158:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:176:in `block in multi_filter'", "org/jruby/RubyArray.java:1981:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:173:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:133:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:304:in `block in start_workers'"]}
[ERROR] 2024-08-23 15:08:25.373 [[main]>worker2] ruby - Ruby exception occurred: undefined method `each' for nil:NilClass {:class=>"NoMethodError", :backtrace=>["(ruby filter code):4:in `block in register'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:96:in `inline_script'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:89:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:158:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:176:in `block in multi_filter'", "org/jruby/RubyArray.java:1981:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:173:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:133:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:304:in `block in start_workers'"]}
[ERROR] 2024-08-23 15:08:25.376 [[main]>worker2] ruby - Ruby exception occurred: undefined method `each' for nil:NilClass {:class=>"NoMethodError", :backtrace=>["(ruby filter code):4:in `block in register'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:96:in `inline_script'", "/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:89:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:158:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:176:in `block in multi_filter'", "org/jruby/RubyArray.java:1981:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:173:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:133:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:304:in `block in start_workers'"]}

```

It seems that everything is fine but I don't understand these errors

The .conf file that I use for ingestion into logstash is the following

```auto
input {
  file {
    path => "/home/lab/BFSALIDA/*.xml"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => multiline {
	pattern => "^<\?xml"
        negate => "true"
        what => "previous"
	auto_flush_interval => 5
    }
  }
}

filter {
  grok { match => { "message" => "<body>%{GREEDYDATA:theHTML}</body>" } }

  mutate { gsub => ["message", '<html xmlns="[^"]*"', '<html' ] }
  xml { 
    source => "message" 
    store_xml => false 
    remove_field => ["message"] 
    xpath => {
      '/html/head/meta[@name="dcterms:created"]/@content' => "Create_Date"
      '/html/head/meta[@name="Message:From-Email"]/@content' => "Message_from"
      '/html/head/meta[@name="Message-To"]/@content' => "Message_To"
      '/html/head/meta[@name="dc:title"]/@content' => "Titulo_email"
      '/html/head/meta[@name="resourceName"]/@content' => "eml_origen"
    }
  
  
  }
    ruby {
        code => '
            meta = event.remove("[parsed][head][meta]")
            meta.each { |x|
                key = x["name"]
                if key =~ /^Message:Raw-Header:/
                    newKey = key.sub(/^Message:Raw-Header:/, "")
                    newKey = "[headers][#{newKey}]"
                else
                    newKey = key.sub(/^.*:/, "")
                    newKey = "[otherStuff][#{newKey}]"
                end
                event.set(newKey, x["content"])
            }
        '
    }
}

output {
  stdout { codec => rubydebug }
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "emails"
  }
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 23, 2024, 3:45pm UTC](https://discuss.elastic.co/t/data-ingestion-error/365467/2 "2024-08-23T15:45:48Z")

</div>

> Ruby exception occurred: undefined method `each' for nil:NilClass {:class=>"NoMethodError", :backtrace=>["(ruby filter code):4:in `block in register'",

> [@jaime\_solas](#):
>
> ```auto
>  
> }
> ruby {
> code => '
> meta = event.remove("[parsed][head][meta]")
> meta.each { |x|
> 
> ```

The error message says that you are calling .each for a nil value. So meta must be nil -- there are events for which [parsed][head][meta] does not exist.

In the error message `"(ruby filter code):4:in` tells you it is somewhere close to the fourth line in the `code =>` block (although it never seems to get that number quite right).

ETA: You could avoid the exception using `meta&.each { |x|`, or the more verbose `meta && meta.each { |x|`, or even a begin/rescue block.

---

<div class="post-metadata">

**Author:** ![jaime\_solas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaime_solas/32/136699_2.png) [@jaime\_solas](https://discuss.elastic.co/u/jaime_solas)\
**Post date:** [August 23, 2024, 4:04pm UTC](https://discuss.elastic.co/t/data-ingestion-error/365467/3 "2024-08-23T16:04:46Z")

</div>

Just entering an & between meta and .each. Is it enough? All the xml files are uploaded. I see that this only happens with some xml files, not with all of them. I imagine that some will have empty fields and others will not. Am I right?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 23, 2024, 4:16pm UTC](https://discuss.elastic.co/t/data-ingestion-error/365467/4 "2024-08-23T16:16:12Z")

</div>

> [@jaime\_solas](#):
>
> Just entering an & between meta and .each. Is it enough?

In this case, yes. Both `meta&.each` and `meta && meta.each` are ways of saying "If meta is not nil then call meta.each". If meta.each is not called then nothing will get populated within [headers] or [otherStuff].