# Help with logstash and XML

**URL:** <https://discuss.elastic.co/t/help-with-logstash-and-xml/47501>\
**Category:** Logstash\
**Created:** [April 15, 2016, 1:13pm UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501 "2016-04-15T13:13:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![funkisjazz](https://avatars.discourse-cdn.com/v4/letter/f/3bc359/32.png) [@funkisjazz](https://discuss.elastic.co/u/funkisjazz)\
**Post date:** [April 15, 2016, 1:13pm UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501/1 "2016-04-15T13:13:12Z")

</div>

I've been searching the internet for the last 6 hours trying to find a guide, tutorial, hint or any kind of help to upload the content of some .xml files into ES using logstash.

For the life of me I haven't been able to do so or find anything to shed some light on the matter, so I come here... asking for anything, a link to some forgotten "how to for dummies guide", some help... provided that my code is at all "right"... anything really.

I have these XML files called log.xml, it looks something like this:

It is saved inisde: c:\Users\my.user\Desktop\TestLogsFolder\

I run logstash using the command (in cmd)

logstash -f pathtoconfigfile/configfile.conf

The configuration file (configfile.conf) has the following text inside it:

input {

```
file {
  type => "file"
	path => ["c:\Users\george.hosu\Desktop\TestLogsFolder\log.xml"]
	start_position => beginning
}

```

}

filter {

```
 xml {
      source => "message"
      target => "message_parsed"
      add_tag => ["xml_parsed"]
      remove_namespaces => true
      xpath => [
        "/testsuites/testsuite/testcase[1]", "testcase1names"
        ]
 }

```

}

output {

stdout { codec =\> "rubydebug"}

elasticsearch {

```
hosts => ["http://localhost:9200"]

manage_template => false

index => "QAlogs-dc-%{+YYYY.MM.dd.hh.mm}"

#protocol => "http"

document_type => "QAlogs"

#document_id => "%{[id]}"

```

}  
}

I do this while ES is running, logstash give me the message "main pipline started" and then... nothing, the ES database doesn't recieve any new data/indexes... etc.

I've added data to said databse with stdin{} and it worked just fine.

If anyone could help me figure how the hell should I go about this I would be beyond greatfull.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 17, 2016, 6:01am UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501/2 "2016-04-17T06:01:51Z")

</div>

It's likely to be a `sincedb` thing, find the existing file and then remove it.

It's better to either use stdin, or set sincedb to a null file, if you are testing.

---

<div class="post-metadata">

**Author:** ![funkisjazz](https://avatars.discourse-cdn.com/v4/letter/f/3bc359/32.png) [@funkisjazz](https://discuss.elastic.co/u/funkisjazz)\
**Post date:** [April 18, 2016, 10:32am UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501/3 "2016-04-18T10:32:22Z")

</div>

Ok, sorry for the late reply, I was away this whole weekend  
I tried deleting all the .sincedb.... files I had (I tried reading multiple filex)

Right now I have an xml filter and an output that works perfectly as long as I introduce XMLs from the concolse (as in copy paste them in and press eneter) using the stdin{} output.

I've tried various ways to "fix" the file input and I'm not sure what to do

Basically what I have right now (I've tried to keep it as bare bones as possible) is:

input {

```
file {
   path => ["c:\Users\my.user\Desktop\qalog\log2.xml"]
   start_position => beginning
}

```

}

I've tried reading .txt files instead... nothing (curiously enough it worked when I tried the example from the website with an appache server log)

.... yet with the above config a .sincedb... etc file is generated but NEVER updated, it stays at 0 0 0 0.

I've tried using " " around beginning, I've tried changing it to "end", I've tried adding info to the file while the pipeline is running and nothing, my console just stays at:

Settings: Default pipeline workers: 4  
Pipeline main started

and chills there for an idefinite amount of time

I'm quite sure I have commited a very obvious newby mistakes but for the life of me I can't figure out what it is

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 19, 2016, 9:17am UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501/4 "2016-04-19T09:17:50Z")

</div>

Are the input files older than 24 hours? See the file input's `ignore_older` option.

---

<div class="post-metadata">

**Author:** ![funkisjazz](https://avatars.discourse-cdn.com/v4/letter/f/3bc359/32.png) [@funkisjazz](https://discuss.elastic.co/u/funkisjazz)\
**Post date:** [April 19, 2016, 10:17am UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501/5 "2016-04-19T10:17:13Z")

</div>

I've tried updating the files with more info, creating new files and setting ignore\_older =\> 0 but It didn't seem to affect anything

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:01am UTC](https://discuss.elastic.co/t/help-with-logstash-and-xml/47501/6 "2017-07-06T05:01:36Z")

</div>