# How to create a new field by extracting data from already generated field?

**URL:** <https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386>\
**Category:** Logstash\
**Created:** [October 8, 2018, 6:08am UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386 "2018-10-08T06:08:53Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![pramodchoudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pramodchoudhari/32/126902_2.png) [@pramodchoudhari](https://discuss.elastic.co/u/pramodchoudhari)\
**Post date:** [October 8, 2018, 6:08am UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386/1 "2018-10-08T06:08:53Z")

</div>

This is how my source field looks like,

![Capture](https://us1.discourse-cdn.com/elastic/original/3X/9/8/9882341067fda5d3daf95fd3d2e11eb84bbe3332.png)

the filename is in the format YYYYMMDD.HHMM.min  
so the above file was created in the **year** :- 2045 , **month** :- 02 , **date** :- 18 , and **time** :- 09:54

what i want to do is from the above source field extract date and time and create a new field which contains only date and time,  
can anybody please tell my how can i do it?  
Please Help!!

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [October 8, 2018, 8:04pm UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386/2 "2018-10-08T20:04:02Z")

</div>

You can use a grok filter with source as the input field and then just create a grok pattern to match.

---

<div class="post-metadata">

**Author:** ![pramodchoudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pramodchoudhari/32/126902_2.png) [@pramodchoudhari](https://discuss.elastic.co/u/pramodchoudhari)\
**Post date:** [October 9, 2018, 11:20am UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386/3 "2018-10-09T11:20:30Z")

</div>

that i know but,  
this is my logstash.conf

```
input {
  beats {
    port => 5044
  }
}

filter {
  if [fields][log_type] == "syslog" {
    grok {
      match => { "message" => "(#!TOT:)?320:%{NUMBER:q}:%{NUMBER:q1}:%{NUMBER:q2}:%{NUMBER:q3}:%{NUMBER:q4}:%{NUMBER:q5}:%{NUMBER:st2} %{NUMBER:st3} %{NUMBER:st4} %{NUMBER:st5} %{NUMBER:st6} %{NUMBER:st7} %{NUMBER:st8} %{NUMBER:st9} %{NUMBER:st10} %{NUMBER:st11} %{NUMBER:st12} %{NUMBER:st13} %{NUMBER:st14} %{NUMBER:st15} %{NUMBER:st16} %{NUMBER:st17} %{NUMBER:st18} %{NUMBER:st19} %{NUMBER:st20} %{NUMBER:st21} %{NUMBER:st22} %{NUMBER:st23} %{NUMBER:st24} %{NUMBER:st25} %{NUMBER:st26} %{NUMBER:st27} %{NUMBER:st28} %{NUMBER:st29} :" }
      match => { "message" => "(#!TOT:)?83:%{NUMBER:q}:%{NUMBER:q1}:%{NUMBER:q2}:%{NUMBER:q3}:%{NUMBER:q4}:%{NUMBER:q5}:%{NUMBER:st2} %{NUMBER:st3} %{NUMBER:st4} %{NUMBER:st5} %{NUMBER:st6} %{NUMBER:st7} %{NUMBER:st8} %{NUMBER:st9} %{NUMBER:st10} %{NUMBER:st11} :" }
    }
  }
  if ("_grokparsefailure" in [tags]) { drop {} }
  else {
    mutate {
      remove_field => ["[q]","[q1]","[q2]","[q3]","[q4]","[st1]","[st3]","[st5]","[st6]","[st7]","[st8]","[st9]","[st10]","[st11]","[st12]","[st14]","[st15]","[st17]","[st18]","[st19]","[st20]","[st21]","[st22]","[st23]","[st24]","[st25]","[st26]","[st28]","[st29]"]
      convert => {
        "st4" => "integer"
        "st2" => "integer"
        "st16" => "integer"
        "st13" => "integer"
        "st27" => "integer"
      }
    }
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

i already have two filters inside the filter if the logs match to any of the matches it skips the other one,where should i put the filter to extract keywords from source field in my code?can you suggest me any edit or solution?

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [October 9, 2018, 5:19pm UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386/4 "2018-10-09T17:19:47Z")

</div>

Just add another grok filter:

```
grok{
       match => { "source" => "%{UNIXPATH}/%{YEAR}%{MONTHNUM}%{MONTHDAY}.%{HOUR}%{MINUTE}.min"
}

```

This should give you output that looks like this:

```
{
  "UNIXPATH": [
    [
      "/var/log/1min"
    ]
  ],
  "YEAR": [
    [
      "2045"
    ]
  ],
  "MONTHNUM": [
    [
      "02"
    ]
  ],
  "MONTHDAY": [
    [
      "18"
    ]
  ],
  "HOUR": [
    [
      "09"
    ]
  ],
  "MINUTE": [
    [
      "54"
    ]
  ]
}
```

---

<div class="post-metadata">

**Author:** ![pramodchoudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pramodchoudhari/32/126902_2.png) [@pramodchoudhari](https://discuss.elastic.co/u/pramodchoudhari)\
**Post date:** [October 10, 2018, 4:11am UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386/5 "2018-10-10T04:11:35Z")

</div>

thanks a lot!!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 7, 2018, 4:11am UTC](https://discuss.elastic.co/t/how-to-create-a-new-field-by-extracting-data-from-already-generated-field/151386/6 "2018-11-07T04:11:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.