# Logstash output changing field location randomly

**URL:** <https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448>\
**Category:** Logstash\
**Created:** [June 12, 2019, 2:45pm UTC](https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448 "2019-06-12T14:45:35Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mdavis00](https://avatars.discourse-cdn.com/v4/letter/m/c77e96/32.png) [@mdavis00](https://discuss.elastic.co/u/mdavis00)\
**Post date:** [June 12, 2019, 2:45pm UTC](https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448/1 "2019-06-12T14:45:36Z")

</div>

I am pulling in logs from Kafka and sending them out to Elasticsearch. I have been getting this set up over the last few weeks and everything seems to be working as expected. Today I noticed every time I start the service ( .../bin/logstash -f .../conf.d/kafka.conf ) I see that logstash is interpreting the fields in a different order.

input {  
kafka {  
bootstrap\_servers =\> ["kafka\_server\_ip:9092"]  
topics =\> ["topic1"]  
add\_field =\> { "topic" =\> "topic1" }  
codec =\> json {  
charset =\> "ISO-8859-1"  
}  
}  
}  
output {  
#I have a few conf files this places the right log into the right index  
if [topic] == "topic1" {  
elasticsearch {  
hosts =\> ["[http://1.1.1.1:9200](http://1.1.1.1:9200)"]  
index =\> "index1"  
}  
}  
#for testing  
stdout {}  
#also sending a copy to Splunk  
tcp {  
host =\> "2.2.2.2"  
port =\> 5514  
codec =\> "json"

```
    }

```

}

Raw Log going in:  
{"logDateTime":"06/12/2019 09:17:59:143","eventDateTime":"06/12/2019 09:17:06:247","sourceIp":"127.0.0.1","applicationIdentifier":"1234567","userIdentity":"Matt\_Test","eventType":"eventType","eventSeverity":"6","action":"action","result":"SUCCESS","reason":"reason"}

Logstash stdout:  
{  
"result" =\> "SUCCESS",  
"reason" =\> "reason",  
"eventDateTime" =\> "06/12/2019 09:17:06:247",  
"eventType" =\> "eventType",  
"applicationIdentifier" =\> "1234567",  
"topic" =\> "topic1",  
"userIdentity" =\> "Matt\_Test",  
"@timestamp" =\> 2019-06-12T14:18:06.448Z,  
"sourceIp" =\> "127.0.0.1",  
"logDateTime" =\> "06/12/2019 09:17:59:143",  
"eventSeverity" =\> "6",  
"@version" =\> "1",  
"action" =\> "action"  
}  
Restart the service and I see:  
{  
"reason" =\> "reason",  
"@timestamp" =\> 2019-06-12T14:41:03.771Z,  
"eventSeverity" =\> "6",  
"logDateTime" =\> "06/12/2019 09:40:59:143",  
"eventDateTime" =\> "06/12/2019 09:40:06:247",  
"@version" =\> "1",  
"topic" =\> "topic1",  
"action" =\> "action",  
"result" =\> "SUCCESS",  
"sourceIp" =\> "127.0.0.1",  
"eventType" =\> "eventType",  
"userIdentity" =\> "Matt\_Test",  
"applicationIdentifier" =\> "1234567"  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 12, 2019, 3:25pm UTC](https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448/2 "2019-06-12T15:25:04Z")

</div>

This is also true for a file input, but not a generator input. It also occurs if you replace the codec with a json filter. Not sure why it happens. It is not as simple as saying a hash is not ordered, because Ruby should iterate over a hash in the order of insertion, and I would expect a json parser to insert fields in the order of appearance.

---

<div class="post-metadata">

**Author:** ![mdavis00](https://avatars.discourse-cdn.com/v4/letter/m/c77e96/32.png) [@mdavis00](https://discuss.elastic.co/u/mdavis00)\
**Post date:** [June 12, 2019, 3:42pm UTC](https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448/3 "2019-06-12T15:42:20Z")

</div>

So you would consider this normal behavior? Nothing I can do about it?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 12, 2019, 4:01pm UTC](https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448/4 "2019-06-12T16:01:41Z")

</div>

I am not aware of any way to prevent it.

---

<div class="post-metadata">

**Author:** ![mdavis00](https://avatars.discourse-cdn.com/v4/letter/m/c77e96/32.png) [@mdavis00](https://discuss.elastic.co/u/mdavis00)\
**Post date:** [June 12, 2019, 4:24pm UTC](https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448/5 "2019-06-12T16:24:44Z")

</div>

Ok I'll have to work around it. Thanks for helping!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 10, 2019, 4:24pm UTC](https://discuss.elastic.co/t/logstash-output-changing-field-location-randomly/185448/6 "2019-07-10T16:24:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.