# Why "?" character appearance in output message via filebeat input?

**URL:** <https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260>\
**Category:** Logstash\
**Created:** [June 29, 2017, 9:41am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260 "2017-06-29T09:41:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sonnguyenit](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@sonnguyenit](https://discuss.elastic.co/u/sonnguyenit)\
**Post date:** [June 29, 2017, 9:41am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260/1 "2017-06-29T09:41:10Z")

</div>

I have log files with lines in xml format like that:

```auto
<?xml version="1.0" encoding="utf-8"?><UrlTrackingObj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Url>http://vietnamnet.vn/</Url><Action>view</Action><Ip>123.24.12.229</Ip><Os>Windows 10</Os><Browser>Chrome 58.0</Browser></UrlTrackingObj>
<?xml version="1.0" encoding="utf-8"?><UrlTrackingObj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Url>http://vietnamnet.vn/</Url><Action>view</Action><Ip>171.249.131.220</Ip><Os>Windows XP</Os><Browser>Chrome 50.3</Browser></UrlTrackingObj>
<?xml version="1.0" encoding="utf-8"?><UrlTrackingObj xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Url>http://vietnamnet.vn/</Url><Action>view</Action><Ip>27.2.64.174</Ip><Os>Windows 7</Os><Browser>Chrome 59.0</Browser></UrlTrackingObj>

```

then i use Filebeat to read these log files and put to Logstash.  
Here Filebeat and Logstash config:

```
filebeat.prospectors:
- input_type: log
  paths:
    - D:\works\logs\*.log

```

and:

```
input {
	beats {
        port => "5043"
    }
}
filter {
  xml {
    source => "message"
    store_xml => false
  }
}
output {
	stdout { codec => json }
}

```

But the output console like that:

```auto
{
    	"@timestamp":"2017-06-29T09:22:44.536Z",
    	"offset":1353334,
    	"@version":"1",
    	"input_type":"log",
    	"beat":{
    		"hostname":"sonnt-pc","name":"sonnt-pc","version":"5.4.3"
    	},
    	"host":"sonnt-pc",
    	"source":"D:\\works\\logs\\2017062901.log",
    	"message":"?<?xml version=\"1.0\" encoding=\"utf-8\"?><UrlTrackingObj xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"><Url>http://vietnamnet.vn/</Url><Action>view</Action><Ip>140.118.138.195</Ip><Os>Windows 8.1</Os><Browser>Firefox 53.0</Browser></UrlTrackingObj>",
    	"type":"log",
    	"tags":["beats_input_codec_plain_applied"]
    }

```

The problems are:

1. Why in message return contains "?" at the first character? (But in log line not has it).

2. I want to the output only like that:

```auto
{
	"@timestamp":"2017-06-29T09:22:44.536Z",
	"Url": "http://vietnamnet.vn/",
	"Action": "view",
	"Ip": "140.118.138.195",
	"Os": "Windows 8.1",
	"Browser": "Firefox 53.0",
}

```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 29, 2017, 9:43am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260/2 "2017-06-29T09:43:15Z")

</div>

Have you looked at your XML file in a hex editor and verified that the file _really_ begins with `<?xml`? I suspect the question mark might be a byte order mark.

---

<div class="post-metadata">

**Author:** ![sonnguyenit](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@sonnguyenit](https://discuss.elastic.co/u/sonnguyenit)\
**Post date:** [June 29, 2017, 10:09am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260/3 "2017-06-29T10:09:19Z")

</div>

@magnusbaeck: Yes, i've check these file, this line begins with some characters: `` in ANSI encode, but in UTF-8 encode not had.  
And i want to output json like:

```auto
{
	"@timestamp":"2017-06-29T09:22:44.536Z",
	"Url": "http://vietnamnet.vn/",
	"Action": "view",
	"Ip": "140.118.138.195",
	"Os": "Windows 8.1",
	"Browser": "Firefox 53.0",
}

```

What are the properties that i need to add to the config?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 29, 2017, 10:19am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260/4 "2017-06-29T10:19:05Z")

</div>

That sounds like byte order marks. You might be able to use a mutate filter's gsub option to remove those bytes. Otherwise use a ruby filter.

---

<div class="post-metadata">

**Author:** ![sonnguyenit](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@sonnguyenit](https://discuss.elastic.co/u/sonnguyenit)\
**Post date:** [June 30, 2017, 7:30am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260/5 "2017-06-30T07:30:39Z")

</div>

Thanks @magnusbaeck. I fixed it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 28, 2017, 7:30am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260/6 "2017-07-28T07:30:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.