# How to load emails into ES instance

**URL:** <https://discuss.elastic.co/t/how-to-load-emails-into-es-instance/364730>\
**Category:** Logstash\
**Created:** [August 10, 2024, 11:48am UTC](https://discuss.elastic.co/t/how-to-load-emails-into-es-instance/364730 "2024-08-10T11:48:39Z")\
**Posts on this page:** 2\
**Page:** 2

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 14, 2024, 4:42pm UTC](https://discuss.elastic.co/t/how-to-load-emails-into-es-instance/364730/21 "2024-08-14T16:42:05Z")

</div>

The store\_xml and xpath options use different XML parsers (one uses NokoGiri, one uses XmlSimple). The xpath option does not like the xmlns attribute on the html element (and I'll never get back the hour and a half it took me to figure that out ☹ )

If you use

```
    mutate { gsub => ["message", '<html xmlns="[^"]*"', '<html' ] }
    xml {
        source => "message"
        store_xml => false
        remove_field => ["message"]
        xpath => {
            '/html/head/meta[@name="resourceName"]/@content' => "bar"
            '/html/head/meta[@name="dcterms:created"]/@content' => "foo"
        }
    }

```

then you will get

```
       "foo" => [
    [0] "2024-07-31T13:18:36Z"
],
       "bar" => [
    [0] "S12317 - Ciberseguridad - LeadsSpain (LeadsSpain@yourshortlist.com) - 2024-07-31 1518.eml"
]

```

Note that xpath always returns arrays, and setting `force_array => false` does not stop it doing so.

---

<div class="post-metadata">

**Author:** ![jaime\_solas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaime_solas/32/136699_2.png) [@jaime\_solas](https://discuss.elastic.co/u/jaime_solas)\
**Post date:** [August 15, 2024, 8:40am UTC](https://discuss.elastic.co/t/how-to-load-emails-into-es-instance/364730/22 "2024-08-15T08:40:11Z")

</div>

I am happy to report that now everything has turned out 100%, thank you very much for everything.  
I put the code in case someone else has the same problem as me.

```auto
input {
  file {
    path => "/home/jaime/PRUEBA_CORREOS/*.xml"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => multiline {
	pattern => "^<\?xml"
        negate => "true"
        what => "previous"
	auto_flush_interval => 5
    }
  }
}

filter {
  grok { match => { "message" => "<body>%{GREEDYDATA:theHTML}</body>" } }

  mutate { gsub => ["message", '<html xmlns="[^"]*"', '<html' ] }
  xml { 
    source => "message" 
    store_xml => false 
    remove_field => ["message"] 
    xpath => {
      '/html/head/meta[@name="dcterms:created"]/@content' => "Create_Date"
      '/html/head/meta[@name="Message:From-Email"]/@content' => "Message_from"
      '/html/head/meta[@name="Message-To"]/@content' => "Message_To"
      '/html/head/meta[@name="dc:title"]/@content' => "Titulo_email"
      '/html/head/meta[@name="resourceName"]/@content' => "eml_origen"
    }
  
  
  }
    ruby {
        code => '
            meta = event.remove("[parsed][head][meta]")
            meta.each { |x|
                key = x["name"]
                if key =~ /^Message:Raw-Header:/
                    newKey = key.sub(/^Message:Raw-Header:/, "")
                    newKey = "[headers][#{newKey}]"
                else
                    newKey = key.sub(/^.*:/, "")
                    newKey = "[otherStuff][#{newKey}]"
                end
                event.set(newKey, x["content"])
            }
        '
    }
}

output {
  stdout { codec => rubydebug }
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "emails"
  }
}

```

[Previous page](https://discuss.elastic.co/t/how-to-load-emails-into-es-instance/364730.md?page=1)