# Removing fields from logstash

**URL:** <https://discuss.elastic.co/t/removing-fields-from-logstash/341782>\
**Category:** Logstash\
**Created:** [August 28, 2023, 7:16am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782 "2023-08-28T07:16:58Z")\
**Posts on this page:** 8\
**Page:** 4

<div class="post-metadata">

**Author:** ![chouben](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@chouben](https://discuss.elastic.co/u/chouben)\
**Post date:** [September 8, 2023, 6:32am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/63 "2023-09-08T06:32:14Z")

</div>

> [@bharti](#):
>
> My logs' timestamp format is DD-MM-YYYY HH:MM:SS (08-09-2023 10:38:43AM)
> 
> so what will be the type

Parsing of dates is done after you split your input message into fields:

> **[Date filter plugin | Logstash Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html)**

> [@bharti](#):
>
> so I want to fetch each value in their respective names

I still would strongly advise you to use the json filter to parse the json message in the input:

> **[JSON filter plugin | Logstash Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html)**

You get all your fields for free (instead of manually parsing them via grok:

```auto
    filter {
      json {
        source => "cutJsonMessage"
      }
    }

```

---

<div class="post-metadata">

**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\
**Post date:** [September 8, 2023, 6:37am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/64 "2023-09-08T06:37:19Z")

</div>

what will cutJsonMessage do?

---

<div class="post-metadata">

**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\
**Post date:** [September 8, 2023, 6:58am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/65 "2023-09-08T06:58:15Z")

</div>

will it give the output as  
txn\_no FT23217XT7DM  
txn\_channel AGENT  
debit\_account\_number  
credit\_account\_number

??

---

<div class="post-metadata">

**Author:** ![chouben](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@chouben](https://discuss.elastic.co/u/chouben)\
**Post date:** [September 8, 2023, 8:07am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/66 "2023-09-08T08:07:25Z")

</div>

> [@bharti](#):
>
> [Debug] : 2023-09-05T08:59:40 -\> [Response - {"abc":{"Service":{"Channel":{"HostIP":{"Status":"APPROVED" ,"Type":"AGENT"},  
> "Name":"[abc.COM](http://abc.com/)","Type":"AGENT"}}}}]

cutJsonMessage is a field you should create via grok parsing - something similar like:

```auto
match => { "message" => '^\[%{LOGLEVEL}\].*\[%{GREEDYDATA:cutJsonMessage }\]$' }
      json {
        source => "cutJsonMessage"
      }

```

> [@bharti](#):
>
> will it give the output as  
> txn\_no FT23217XT7DM  
> txn\_channel AGENT  
> debit\_account\_number  
> credit\_account\_number

As indicated multiple times, it will split your message in fields..

---

<div class="post-metadata">

**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\
**Post date:** [September 8, 2023, 10:05am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/67 "2023-09-08T10:05:54Z")

</div>

working...thanks chouben 😀

---

<div class="post-metadata">

**Author:** ![bharti](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@bharti](https://discuss.elastic.co/u/bharti)\
**Post date:** [September 8, 2023, 10:07am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/68 "2023-09-08T10:07:20Z")

</div>

and if I want to add filters in multiple log files ..then what will be the syntax

```auto
input {
  beats {
    port => 5044
  }
}
filter {
    if "/var/log/BAOBABAPIModule.log" in [log][file][path] {
      grok {
         match => { "message" => '^%{GREEDYDATA}"Status":"%{WORD:Status}"%{GREEDYDATA}$' }
      }

     grok {
        match => { "message" => '^%{GREEDYDATA}"TransactionNumber":"%{NUMBER:TransactionNumber}"%{GREEDYDATA}$' }

     }
}

```

currently using this

---

<div class="post-metadata">

**Author:** ![chouben](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@chouben](https://discuss.elastic.co/u/chouben)\
**Post date:** [September 8, 2023, 11:06am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/69 "2023-09-08T11:06:15Z")

</div>

I give up.. You are not even trying the solution suggested. We are +- 50 messages further.

I suggest you try "google" or search this forum 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 6, 2023, 11:06am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782/70 "2023-10-06T11:06:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/removing-fields-from-logstash/341782.md?page=3)