# Logstash filters for log file which is included some raw data and json data **URL:** \ **Category:** Logstash\ **Created:** [April 27, 2023, 12:03pm UTC](https://discuss.elastic.co/t/logstash-filters-for-log-file-which-is-included-some-raw-data-and-json-data/330950 "2023-04-27T12:03:31Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Harish1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harish1/32/119311_2.png) [@Harish1](https://discuss.elastic.co/u/Harish1)\ **Post date:** [April 27, 2023, 12:03pm UTC](https://discuss.elastic.co/t/logstash-filters-for-log-file-which-is-included-some-raw-data-and-json-data/330950/1 "2023-04-27T12:03:31Z")
Hi Elastic team, I'm new to ELK, I'm trying to find out the filters for below log file but I'm not able to find the proper Logstash filter for below data _2023-01-19 15:38:31 INFO VCIPDownstreamController:138 - {"timestamp":"2023-19-01T15:38:31","module":"VCIPDownstreamInitRequest","requestURI":"[http://sample.url.com/sam\_bnk\_vcip\_ws/v1/bbl/saveCustomerVCIPDetails","appRefNum":"Test29Dec0006","requestDateTime":"2023-19-01T15:38:31","responseDataTime":"","responseTime":"","requestData":"0dJiseXXhoHZJSoEeLqGkQ\u003d\u003d","responseData":"","responseCode":"","accessLog":"](http://sample.url.com/sam_bnk_vcip_ws/v1/bbl/saveCustomerVCIPDetails%22,%22appRefNum%22:%22Test29Dec0006%22,%22requestDateTime%22:%222023-19-01T15:38:31%22,%22responseDataTime%22:%22%22,%22responseTime%22:%22%22,%22requestData%22:%220dJiseXXhoHZJSoEeLqGkQ%5Cu003d%5Cu003d%22,%22responseData%22:%22%22,%22responseCode%22:%22%22,%22accessLog%22:%22){"remoteHost":"192.11.13.133","remoteAddr":"192.11.13.133","localaadr":"192.11.8.118","x-forwarded-for":["152.170.4.37"],"x-forwarded-proto":["https"],"x-forwarded-port":["443"],"host":["sample.url.com"],"x-amzn-trace-id":["Root\u003d1-63c9169e-01c9501a04f1239777d95ae7"],"content-length":["121799"],"authorization":["Basic VmNpcHVzZXI6VmNpcEtMQHA123\u003d\u003d"],"content-type":["application\/json"],"user-agent":["PostmanRuntime\/7.30.0"],"accept":["\/"],"postman-token":["7554nfbf-1d8f-4793-8258-d27c2ccf4cae"],"accept-encoding":["gzip, deflate, br"],"cookie":["AWSALB\u003d1Apfla0u4SSa7aQ3vAIGAv6wKbc5KH2eUhQ1lXCVr8SIzswx2vAWIK3e1gzob2FNspNYJA+aOjtwnXQoTFbQAgE0duYlEyGHIYk\/b\/byEdTGu0yeuCZ5roEM9706; AWSALBCORS\u003d1Apfla0u4SSa7aQ3vAIGAv6wKbc5KH2eUhQ1lXCVr8SIzswx2vAWIK3e1gzob2FNspNYJA+aOjtwnXQoTFbQAgE0duYlEyGHIYk\/b\/byEdTGu0yeuCZ5roEM9706"]}"}_\*\* I have used below grok pattern _%{TIMESTAMP\_ISO8601: date} %{LOGLEVEL:log-level} %{DATA:data}:%{INT:int} - %{GREEDYDATA:message}_ but my team wants that GREEDYDATA message also in a each separated filed in kabana because it having lot of key value pair of json data. kindly help me on this as soon as possible Thank you in advance. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 27, 2023, 2:48pm UTC](https://discuss.elastic.co/t/logstash-filters-for-log-file-which-is-included-some-raw-data-and-json-data/330950/2 "2023-04-27T14:48:22Z")
> [@Harish1](#): > > my team wants that GREEDYDATA message also in a each separated filed Have you tried using a [json](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html) filter? ---
**Author:** ![Harish1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harish1/32/119311_2.png) [@Harish1](https://discuss.elastic.co/u/Harish1)\ **Post date:** [April 28, 2023, 3:34pm UTC](https://discuss.elastic.co/t/logstash-filters-for-log-file-which-is-included-some-raw-data-and-json-data/330950/3 "2023-04-28T15:34:14Z")
Yes I'm tried with below filter but total data not showing in kibana ```auto filter { grok { match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} %{DATA:data}:%{INT:int} - %{GREEDYDATA:json_data}" } } } filter { json{ source => "json_data" target => "parsed_json" } } ``` actually in my logfile I have another json type of key value is there, in below log file you can see that in the _ **6th line** _ it is started as _ **"accesslog"** _ key pls look into that and help me out from that. `023-01-19 15:38:31 INFO VCIPDownstreamController:138 - {"timestamp":"2023-19-01T15:38:31","module":"VCIPDownstreamInitRequest","requestURI":"http://sample.url.com/sam_bnk_vcip_ws/v1/bbl/saveCustomerVCIPDetails","appRefNum":"Test29Dec0006","requestDateTime":"2023-19-01T15:38:31","responseDataTime":"","responseTime":"","requestData":"0dJiseXXhoHZJSoEeLqGkQ\u003d\u003d","responseData":"","responseCode":"","accessLog":"{"remoteHost":"192.11.13.133","remoteAddr":"192.11.13.133","localaadr":"192.11.8.118","x-forwarded-for":["152.170.4.37"],"x-forwarded-proto":["https"],"x-forwarded-port":["443"],"host":["sample.url.com"],"x-amzn-trace-id":["Root\u003d1-63c9169e-01c9501a04f1239777d95ae7"],"content-length":["121799"],"authorization":["Basic VmNpcHVzZXI6VmNpcEtMQHA123\u003d\u003d"],"content-type":["application\/json"],"user-agent":["PostmanRuntime\/7.30.0"],"accept":["\/"],"postman-token":["7554nfbf-1d8f-4793-8258-d27c2ccf4cae"],"accept-encoding":["gzip, deflate, br"],"cookie":["AWSALB\u003d1Apfla0u4SSa7aQ3vAIGAv6wKbc5KH2eUhQ1lXCVr8SIzswx2vAWIK3e1gzob2FNspNYJA+aOjtwnXQoTFbQAgE0duYlEyGHIYk\/b\/byEdTGu0yeuCZ5roEM9706; AWSALBCORS\u003d1Apfla0u4SSa7aQ3vAIGAv6wKbc5KH2eUhQ1lXCVr8SIzswx2vAWIK3e1gzob2FNspNYJA+aOjtwnXQoTFbQAgE0duYlEyGHIYk\/b\/byEdTGu0yeuCZ5roEM9706"]}"}` i'm struggling with that _ **"accesslog"** _ key valude data, i'm not able to find the perfect filter to show all variables and all data in kibana pls help me out from this ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 28, 2023, 5:24pm UTC](https://discuss.elastic.co/t/logstash-filters-for-log-file-which-is-included-some-raw-data-and-json-data/330950/4 "2023-04-28T17:24:50Z")
Please edit your post, select the configuration and click on \ in the toolbar above the edit pane, then do the same for the log entry. That will change the format from filter { grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:log-level} %{DATA:data}:%{INT:int} - %{GREEDYDATA:json\_data}" } } } to ``` filter { grok { match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} %{DATA:data}:%{INT:int} - %{GREEDYDATA:json_data}" } } } ``` and prevent the forum software from consuming parts of the message as formatting. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 26, 2023, 5:25pm UTC](https://discuss.elastic.co/t/logstash-filters-for-log-file-which-is-included-some-raw-data-and-json-data/330950/5 "2023-05-26T17:25:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.