# How can I extract a sub-field from a field and print it as a separate field in filebeat?

**URL:** https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103
**Category:** Beats
**Tags:** filebeat
**Created:** [May 23, 2023, 10:32am UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103 "2023-05-23T10:32:46Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![varunsingla](https://avatars.discourse-cdn.com/v4/letter/v/a183cd/32.png) [@varunsingla](https://discuss.elastic.co/u/varunsingla)
#### Post date: [May 23, 2023, 10:32am UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/1 "2023-05-23T10:32:46Z")

</div>

"msg":"{"appName":"abc","eventCategory":"Authentication event","eventType":"Operator record change","id":"12345","ipAddress":"0.0.1.1","nodeID":"nodeabc","operation":"update","operatorID":"admin","operatorRecID":"DATAADMIN","operatorRecName":"Administrator Copy","requestorIdentity":"121212","tenantID":"shared","timeStamp":"Tue 2023 May 23, 10:17:47:593"}"

I need to extract the "timeStamp" field from this whole msg object, and print it as an independent field.

---

<div class="post-metadata">

### Author: ![kcreddy](https://avatars.discourse-cdn.com/v4/letter/k/6f9a4e/32.png) [@kcreddy](https://discuss.elastic.co/u/kcreddy)
#### Post date: [May 24, 2023, 8:19am UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/2 "2023-05-24T08:19:27Z")

</div>

Have you tried [decode\_json\_fields](https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html) processor?

---

<div class="post-metadata">

### Author: ![eMitch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emitch/32/93607_2.png) [@eMitch](https://discuss.elastic.co/u/eMitch)
#### Post date: [May 24, 2023, 1:54pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/3 "2023-05-24T13:54:01Z")

</div>

Hi @varunsingla and welcome to the community!

Additional context would be helpful so that the community can best provide insight.

How are you ingesting these logs?

As @kcreddy suggested, these appear to be JSON logs so you may want to look into the following:

- [Logstash JSON Filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html#plugins-filters-json)
- [Ingest Pipeline JSON Processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/json-processor.html#json-processor)
- [Filebeat Decode JSON Fields](https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html#decode-json-fields) (per @kcreddy )

---

<div class="post-metadata">

### Author: ![varunsingla](https://avatars.discourse-cdn.com/v4/letter/v/a183cd/32.png) [@varunsingla](https://discuss.elastic.co/u/varunsingla)
#### Post date: [May 24, 2023, 3:50pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/4 "2023-05-24T15:50:01Z")

</div>

I tried:

- decode\_json\_fields:  
fields: ["msg.timeStamp"]  
target: ""  
max\_depth: 1  
overwrite\_keys: false  
process\_array: false  
add\_error\_key: true

but got these errors:

"log.logger":"truncate\_fields","log.origin":{"file.name":"actions/decode\_json\_fields.go","file.line":109},"message":"Error trying to GetValue for field : msg.timeStamp in event : &{2023-05-24 10:46:27.702919726 -0500 CDT m=+14.072969149 {}

decode\_json\_fields=msg.timeStamp}: expected map but type is string",

---

<div class="post-metadata">

### Author: ![varunsingla](https://avatars.discourse-cdn.com/v4/letter/v/a183cd/32.png) [@varunsingla](https://discuss.elastic.co/u/varunsingla)
#### Post date: [May 24, 2023, 4:06pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/5 "2023-05-24T16:06:36Z")

</div>

I tried:

- decode\_json\_fields:  
fields: ["msg.timeStamp"]  
target: ""  
max\_depth: 1  
overwrite\_keys: false  
process\_array: false  
add\_error\_key: true

But, got these errors:

"log.logger":"truncate\_fields","log.origin":{"file.name":"actions/decode\_json\_fields.go","file.line":109},"message":"Error trying to GetValue for field : msg.timeStamp in event : &{2023-05-24 10:46:27.702919726 -0500 CDT m=+14.072969149 {}

decode\_json\_fields=msg.timeStamp}: expected map but type is string","service.name":"filebeat","ecs.version":"1.6.0"}

---

<div class="post-metadata">

### Author: ![varunsingla](https://avatars.discourse-cdn.com/v4/letter/v/a183cd/32.png) [@varunsingla](https://discuss.elastic.co/u/varunsingla)
#### Post date: [May 24, 2023, 4:12pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/6 "2023-05-24T16:12:16Z")

</div>

Hello @eMitch , I am not using Logstash or Elasticsearch.  
Just ingesting logs through filebeat.inputs

filebeat.inputs:

- type: filestream

---

<div class="post-metadata">

### Author: ![eMitch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emitch/32/93607_2.png) [@eMitch](https://discuss.elastic.co/u/eMitch)
#### Post date: [May 24, 2023, 4:28pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/7 "2023-05-24T16:28:10Z")

</div>

try `fields: ["msg"]` - then see if you can access `timeStamp` as a field.

---

<div class="post-metadata">

### Author: ![varunsingla](https://avatars.discourse-cdn.com/v4/letter/v/a183cd/32.png) [@varunsingla](https://discuss.elastic.co/u/varunsingla)
#### Post date: [May 24, 2023, 4:43pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/8 "2023-05-24T16:43:15Z")

</div>

Thanks @eMitch after you suggested, I used ["msg"] under fields, and it gave me the desired result 🙂

Now I am looking help on converting the timeStamp to EPOCH format and ipAddress as well to a particular format.

Could you please suggest how can that be done?

I tried using convert processor to convert ipAddress to "long" type, but it started returning a blank value instead:

- convert:  
fields:  
- {from: "ipAddress", to: "device.ip4", type: "long"}

Please suggest.

---

<div class="post-metadata">

### Author: ![eMitch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emitch/32/93607_2.png) [@eMitch](https://discuss.elastic.co/u/eMitch)
#### Post date: [May 26, 2023, 1:28pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/9 "2023-05-26T13:28:04Z")

</div>

what is your expected output for the IP if converting to "long"? For example, if the IP is 192.168.1.1, what do you expect the final output to be?

If you're not moving this data into elasticsearch, then you may want to [check out the script processor](https://www.elastic.co/guide/en/beats/filebeat/current/processor-script.html) and write some custom javascript to do those conversions.

---

<div class="post-metadata">

### Author: ![varunsingla](https://avatars.discourse-cdn.com/v4/letter/v/a183cd/32.png) [@varunsingla](https://discuss.elastic.co/u/varunsingla)
#### Post date: [May 26, 2023, 5:46pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/10 "2023-05-26T17:46:03Z")

</div>

192.168.1.1 should be converted to 3232235777

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 23, 2023, 7:46pm UTC](https://discuss.elastic.co/t/how-can-i-extract-a-sub-field-from-a-field-and-print-it-as-a-separate-field-in-filebeat/334103/11 "2023-06-23T19:46:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
