# Elasticsearch xml file

**URL:** <https://discuss.elastic.co/t/elasticsearch-xml-file/134468>\
**Category:** Logstash\
**Created:** [June 4, 2018, 5:37pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468 "2018-06-04T17:37:23Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 4, 2018, 5:37pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/1 "2018-06-04T17:37:23Z")

</div>

Hi i need index xml file to elasticsearch  
my xml file is like this

```
<Talk Speaker = "Alastair Parvin" Title= " Architecture for the people by the people" >
	<Segment id ="1" >
		<Time-slot>00:00:12,884 --> 00:00:16,053</Time-slot>
		<Original_text lang="en"></Original_text>
		<Translation lang="ar"></Translation>
		<Translation lang="fr"></Translation>
	</Segment>
</Talk>
</MulTed>

```

I need help please

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 4, 2018, 5:56pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/2 "2018-06-04T17:56:11Z")

</div>

You need to transform it to JSON document first.

You can use logstash if needed or do that by yourself depending what is the real source of this content.

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 4, 2018, 6:04pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/3 "2018-06-04T18:04:21Z")

</div>

Ok i try use logstash thanks

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 4, 2018, 6:16pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/4 "2018-06-04T18:16:55Z")

</div>

i try use logstash but i have error

```
input {
    file {
		path => "C:/Users/Dev/Desktop/file1.xml"
		start_position => "beginning"
		sincedb_path => "/dev/null"
		type => "xml"
		codec => multiline {
             pattern => "^<\?Multed .*\>"
             negate => "true"
             what => "previous"
}
	}
}
filter {
	xml {
    source => "message"
    target => "Multed"
	xpath =>["/Multed/Talk/Segment/@id","id",
		"/Multed/Talk/Segment/Original_text/text()","original_text"
		
	]
  }

       mutate { 
            remove_field => ["message"] 
        
            add_field => ["IDIndexed", "%{id}"] 
            add_field => ["Original_text", "%{original_text}"]           
                         
						}}
output{
    elasticsearch{
        hosts => ["localhost:9200"]
        index => "indexXml"
    }
    stdout{
	codec => rubydebug

    }
}

```

![er](https://us1.discourse-cdn.com/elastic/original/3X/a/b/ab97f020cce78a7a8146a4f538cce6023e73c0a4.PNG)  
Error in %{id} and %{original\_text}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 4, 2018, 6:38pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/5 "2018-06-04T18:38:41Z")

</div>

Please don't post images of text as they are hardly readable and not searchable.

Instead paste the text and format it with `</>` icon. Check the preview window.

I moved your question to #logstash

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 4, 2018, 6:43pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/6 "2018-06-04T18:43:34Z")

</div>

Ok  
{  
"type" =\> "xml",  
"IDIndexed" =\> "%{id}",  
"@timestamp" =\> 2018-06-04T18:16:49.466Z,  
"host" =\> "Dev-PC",  
"path" =\> "C:/Users/Dev/Desktop/file1.xml",  
"@version" =\> "1",  
"Original\_text" =\> "%{original\_text}",  
"tags" =\> [  
[0] "multiline",  
[1] "multiline\_codec\_max\_lines\_reached",  
[2] "\_xmlparsefailure"  
]  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 4, 2018, 8:17pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/7 "2018-06-04T20:17:10Z")

</div>

The multiline codec is incorrectly configured. Which line from the XML file is `^<\?Multed .*\>` supposed to match?

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 5, 2018, 12:38am UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/8 "2018-06-05T00:38:41Z")

</div>

```
`^<\?Multed .*\>` is root of document
        <Multed>
        <Talk Speaker = "Alastair Parvin" Title= " Architecture for the people by the people" >
        	<Segment id ="1" >
        		<Time-slot>00:00:12,884 --> 00:00:16,053</Time-slot>
        		<Original_text lang="en"></Original_text>
        		<Translation lang="ar"></Translation>
        		<Translation lang="fr"></Translation>
        	</Segment>
        </Talk>
        </MulTed>
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 5, 2018, 10:47am UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/9 "2018-06-05T10:47:15Z")

</div>

The regular expression `<\?Multed .*\>` does not match any of the lines in your example document.

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 5, 2018, 4:28pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/10 "2018-06-05T16:28:35Z")

</div>

what's regular expression is correct

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2018, 5:18pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/11 "2018-06-05T17:18:56Z")

</div>

You could try

```auto
        codec => multiline {
            pattern => "^<MulTed>"
            negate => "true"
            what => "previous"
            auto_flush_interval => 2
        }

```

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 5, 2018, 5:37pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/12 "2018-06-05T17:37:30Z")

</div>

The same problem

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2018, 5:47pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/13 "2018-06-05T17:47:25Z")

</div>

If the XML is indented then get rid of the start-of-line anchor and use

```
pattern => "<MulTed>"
```

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 5, 2018, 5:56pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/14 "2018-06-05T17:56:38Z")

</div>

The same error value

```
        input {
            file {
        		path => "C:/Users/Dev/Desktop/file1.xml"
        		start_position => "beginning"
        		sincedb_path => "/dev/null"
        		type => "xml"
        		   codec => multiline {
                    pattern => "<MulTed>"
                    negate => "true"
                    what => "previous"
                    auto_flush_interval => 2
                }
        	}
        }
        filter {
    		
    	xml {
        source => "Talk"
        target => "MulTed"
    	xpath =>["MulTed/Talk/Segment/@id","id",
    		"MulTed/Talk/Segment/Original_text/text()","original_text"]
      }

           mutate { 
                remove_field => ["message"] 
            
                add_field => ["IDIndexed", "%{id}"] 
                add_field => ["Original_text", "%{original_text}"]           
                             
    						}}
    output{
        elasticsearch{
            hosts => ["localhost:9200"]
            index => "senind"
        }
        stdout{
    	codec => rubydebug

        }
    }

```

{  
"tags" =\> [  
[0] "multiline"  
],  
"Original\_text" =\> "%{original\_text}",  
"@version" =\> "1",  
"path" =\> "C:/Users/Dev/Desktop/file1.xml",  
"type" =\> "xml",  
"host" =\> "Dev-PC",  
"@timestamp" =\> 2018-06-05T17:54:47.115Z,  
"IDIndexed" =\> "%{id}"  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2018, 6:27pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/15 "2018-06-05T18:27:20Z")

</div>

There is no error there. The multiline codec worked.

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 5, 2018, 6:29pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/16 "2018-06-05T18:29:13Z")

</div>

But the value of% {id} and %{original\_text} is not insert

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2018, 6:33pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/17 "2018-06-05T18:33:18Z")

</div>

That's because your xpath expressions are wrong. They refer to Multed, but the XML has MulTed. Or perhaps the other way around. Either way, it is case sensitive. Also, Original\_text/text() is empty.

Note also that xpath always returns arrays, so you might want to

```
if [id] { mutate { replace => { "id" => "%{[id][0]}" } } }
```

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 5, 2018, 6:44pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/18 "2018-06-05T18:44:08Z")

</div>

Ok thanks i try use it

---

<div class="post-metadata">

**Author:** ![Hana\_Ne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hana_ne/32/31903_2.png) [@Hana\_Ne](https://discuss.elastic.co/u/Hana_Ne)\
**Post date:** [June 5, 2018, 7:08pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/19 "2018-06-05T19:08:02Z")

</div>

the same error Thanks

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2018, 9:51pm UTC](https://discuss.elastic.co/t/elasticsearch-xml-file/134468/20 "2018-06-05T21:51:06Z")

</div>

OK, so comment out 'remove\_field =\> ["message"]' and show us what an event looks like, either using stdout { codec =\> rubydebug }, or copy and paste from the JSON event in Kibana.

[Next page](https://discuss.elastic.co/t/elasticsearch-xml-file/134468.md?page=2)