# Need to store the value in cache memory and compare while parse in logstash

**URL:** <https://discuss.elastic.co/t/need-to-store-the-value-in-cache-memory-and-compare-while-parse-in-logstash/181791>\
**Category:** Logstash\
**Created:** [May 20, 2019, 10:29am UTC](https://discuss.elastic.co/t/need-to-store-the-value-in-cache-memory-and-compare-while-parse-in-logstash/181791 "2019-05-20T10:29:20Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ganesh2303](https://avatars.discourse-cdn.com/v4/letter/g/57b2e6/32.png) [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\
**Post date:** [May 20, 2019, 10:29am UTC](https://discuss.elastic.co/t/need-to-store-the-value-in-cache-memory-and-compare-while-parse-in-logstash/181791/1 "2019-05-20T10:29:20Z")

</div>

HI Team,  
I have some tricky challenge, Anyone can help me on this concept,

I have one restricted field and that will extract from my log. That keyword will not occur on every message. If my log contain restricted value(Group Field) i need to store that event id(DCN field Value) in cache memory/some index/some file to compare that id with up coming events. If my DCN id gets matched mean, need to add one field like restricted\_data: "yes". All this operations are happen in same index.

I have tried various steps but its all helpless and i need your help to achieve this. I share my code snippet here for your reference,

Sample log(which contain the restricted data field and normal log)

```
2019-05-06 22:41:09.174 [WMQJCAResourceAdapter : 4] [DEBUG] [DCN 0201912617621990C] ClaimTranslatorUtil - <?xml ><cts:GroupNumber>000PE1513</cts:GroupNumber>

2019-05-06 22:41:09.174 [WMQJCAResourceAdapter : 4] [DEBUG] [DCN 0201912617621990C] VenAdjUtil - Splunk log for Request : Starts

```

I need to use DCN value to compare each events

```
input {
 beats{
	port => 5047
 }
}
filter{
	grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} \[%{DATA:resource}\] \[?(?<loglevel>[a-zA-Z]+)\] \[DCN %{DATA:dcn_id}\] %{DATA:info} - ?(?<description>[a-zA-Z0-9\n -`!@#$%^&*':\".,(){}\[\]~]+)" }    
  }
  grok {
    match => { "description" => "<cts:GroupNumber>%{DATA:grp_id}<" }    
  }
   date {
    match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
	target => ["timestamp"]
  }
   
   if![restricted_data]
   {
	#translate {
	#	field => "dcn_id"
	#	destination => "restricted_data"
	#	override => true
	#	exact => true
	#	dictionary_path => "C:/Ganesh/ELK/Latest/check.yaml"
	#}
	 elasticsearch {
		  hosts => ["localhost:9200"]
		  index => "validate"
		  user => "elastic"
		  password => "elastic"
		  query => "dcn_id:%{[dcn_id]}"
		  fields => { "restricted_data" => "restricted" }
	   }
	    
	}
    
   if [grp_id]
   {
	   elasticsearch {
		  hosts => ["localhost:9200"]
		  index => "restricted_data"
		  user => "elastic"
		  password => "elastic"
		  query => "type:restricted AND grp_number:%{[grp_id]}"
		  fields => { "restricted_status" => "restricted_data" }
	   }
	 
	# mutate{
	# add_field => {"test" => "%{dcn_id}: Success"}
    #
	# }
	# mutate{
	# gsub => ["test", "[\\]", ""]
	# }
   } 
}
output {
	if "test" in [tags]{
	  elasticsearch {
		hosts => ["http://localhost:9200"]
		index => "test_log"
		user => "elastic"
		password => "elastic"
	}
	if [grp_id] 
	{
		# file { 
        # codec => line { format => "%{test}"}
        # path => "C:\Ganesh\ELK\Latest\check.yaml" 
        #}
		elasticsearch {
		hosts => ["http://localhost:9200"]
		index => "validate"
		user => "elastic"
		password => "elastic"
	}

	}	
  }
  stdout{
	codec => rubydebug
	}
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 17, 2019, 10:29am UTC](https://discuss.elastic.co/t/need-to-store-the-value-in-cache-memory-and-compare-while-parse-in-logstash/181791/2 "2019-06-17T10:29:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.