# Logstash and dynamic fields in logs

**URL:** <https://discuss.elastic.co/t/logstash-and-dynamic-fields-in-logs/86091>\
**Category:** Logstash\
**Created:** [May 17, 2017, 11:27am UTC](https://discuss.elastic.co/t/logstash-and-dynamic-fields-in-logs/86091 "2017-05-17T11:27:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmccormack](https://avatars.discourse-cdn.com/v4/letter/j/3ab097/32.png) [@jmccormack](https://discuss.elastic.co/u/jmccormack)\
**Post date:** [May 17, 2017, 11:27am UTC](https://discuss.elastic.co/t/logstash-and-dynamic-fields-in-logs/86091/1 "2017-05-17T11:27:02Z")

</div>

Hey All

So I have a log file that has dynamic content depending on the specific machine it is running on.

For example:  
0170517 00:07:20 name 00 2 3 6 0 5 name1 00 0 2 3 6 0 5 name2 00 0 2 3 6 0 5  
0170517 00:07:30 name 00 0 2 3 6 0 5 name1 00 0 2 3 6 0 5 name2 00 0 2 3 6 0 5 name3 00 0 2 3 6 0 5

Is there a way for logstash to dynamically figure out the new fields using the same timestamp. Or should I write a script to change the log format? (that would be extra overhead)

Something like this:  
0170517 00:07:20 name 00 0 2 3 6 0 5  
0170517 00:07:20 name1 00 0 2 3 6 0 5  
0170517 00:07:20 name2 00 0 2 3 6 0 5  
0170517 00:07:30 name 00 0 2 3 6 0 5  
0170517 00:07:30 name1 00 0 2 3 6 0 5  
0170517 00:07:30 name2 00 0 2 3 6 0 5  
0170517 00:07:30 name3 00 0 2 3 6 0 5

For example, this should work with the second format

input {  
file {  
path =\> "/var/log/name.log"  
start\_position =\> beginning  
}  
}

filter {  
if ([message] =~ /^#/) {  
drop{}  
}

```
 grok { 
        match => { "message" => "(?<timestamp>%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}) %{WORD:name} %{NUMBER:number1:int} %{NUMBER:number2:int} %{NUMBER:number3:int} %{NUMBER:number4:int} %{NUMBER:number5:int} %{NUMBER:number6:int} %{NUMBER:number7:int} "}

   date {
        match => ["timestamp", "yyyyMMdd HH:mm:ss"]
        target => "@timestamp"
          }

```

}

output {  
elasticsearch {  
hosts =\> "localhost:9200"  
index =\> "logstash-naming"  
}  
}

Thanks  
Any ideas or suggestions would be great.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 14, 2017, 11:27am UTC](https://discuss.elastic.co/t/logstash-and-dynamic-fields-in-logs/86091/2 "2017-06-14T11:27:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.