# Logstash parent-child event configuration

**URL:** <https://discuss.elastic.co/t/logstash-parent-child-event-configuration/58117>\
**Category:** Logstash\
**Created:** [August 16, 2016, 9:29am UTC](https://discuss.elastic.co/t/logstash-parent-child-event-configuration/58117 "2016-08-16T09:29:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aqiao](https://avatars.discourse-cdn.com/v4/letter/a/e274bd/32.png) [@aqiao](https://discuss.elastic.co/u/aqiao)\
**Post date:** [August 16, 2016, 9:29am UTC](https://discuss.elastic.co/t/logstash-parent-child-event-configuration/58117/1 "2016-08-16T09:29:00Z")

</div>

`2016-08-16 14:00:14,655 ABC BETA P1 DecodeFieldList Refresh <6>,TRDPRC_1,Price, <15>,CURRENCY,String,0 <22>,BID,Price, <23>,BID_1,Price, <24>,BID_2,Price, <25>,ASK,Price, <26>,ASK_1,Price, <27>,ASK_2,Price,`  
Above is my log sample, the first line is parent info and the rest is the children info. I have tested the parent pattern and children pattern via grok debuger as below:

- Parent pattern

`%{DATESTAMP:EventTime},%{NUMBER:Mil:INT} %{WORD:Type} %{GREEDYDATA:Item} %{GREEDYDATA:RIC} %{GREEDYDATA:Detail} %{GREEDYDATA:Category}`

- Children pattern  
`\<%{NUMBER:FID:INT}\>,%{GREEDYDATA:FName},%{WORD:FType},%{GREEDYDATA:FValue}`

How can i store the sample to two ES types via logstash configuration file

hope your help ! many many thanks

---

<div class="post-metadata">

**Author:** ![aqiao](https://avatars.discourse-cdn.com/v4/letter/a/e274bd/32.png) [@aqiao](https://discuss.elastic.co/u/aqiao)\
**Post date:** [August 18, 2016, 2:36pm UTC](https://discuss.elastic.co/t/logstash-parent-child-event-configuration/58117/2 "2016-08-18T14:36:33Z")

</div>

solved it!! here is my thinking:

1. define a pattern for parent event firstly,since the children events will not match the pattern, Logstash will generate a \_grokparsefailure tag for children events,then we can identify the current event is parent or child via the tag
2. use a filter-\>ruby block to generate document\_id and keep it in a global variable,then children events can access it.  
3.add a field such as doc\_id for both parent events and children events, which stores the document\_id in step 2 and add a field such as parent\_id only for children events to store the parent document\_id

below is the entire Lostash configuration:

input {  
beats {  
port =\> 5044  
}  
}

filter {

# remove the empty lines

if [message] =~ /^\s\*$/ {  
drop { }  
}

# define parent event pattern

grok {  
match =\> {"message" =\> "%{DATESTAMP:EventTime},%{NUMBER:Mil:INT} %{WORD:Type} %{GREEDYDATA:Item} %{GREEDYDATA:RIC} %{GREEDYDATA:Detail} %{GREEDYDATA:Category}"}  
}

# children events

if "\_grokparsefailure" in [tags] {  
grok {  
match =\> {"message" =\> "\<%{NUMBER:FID:INT}\>,%{GREEDYDATA:FName},%{WORD:FType},%{GREEDYDATA:FValue}"}  
add\_field =\> {"DocID" =\> '' "ParentID" =\> ''}  
add\_tag =\> ["%{FType}"]  
remove\_tag =\> ["\_grokparsefailure"]  
}  
ruby {  
code =\> "require 'digest/md5';  
event['ParentID'] = @@parentid;  
event['DocID'] = Digest::MD5.hexdigest(@@parentdate+event['FID'])"  
}  
}  
else{  
mutate {  
add\_field =\> {"DocID" =\> ''}  
add\_tag =\> ["parent"]  
}  
# define a global variable to keep the parent id  
# must set the default value for the variables in ruby -\> init block, or it will raise exception  
ruby {  
init =\> "@@parentid = '';@@parentdate=''"  
code =\> "require 'digest/md5';  
@@parentid = Digest::MD5.hexdigest(event['EventTime']+event['Mil']);  
event['DocID'] = @@parentid;  
@@parentdate = event['EventTime']+event['Mil']"  
}  
}  
#remove the redundant fields created by filebeat. you can ignore it if you don't use filebeat as shipper  
mutate {  
remove\_field =\> ["[beat][hostname]","[beat][name]","count","fields","input\_type","offset","type","beat","@version"]  
}  
}

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
#set the document\_id  
document\_id =\> %{"DocID"}  
document\_type =\> "%{[@metadata][type]}"  
#template =\> "/appserver/ELK/logstash-2.3.4/conf/template\_tolreport.json"  
#template\_name =\>"template\_tolreport"  
#template\_overwrite =\> true  
}

# file {

# path =\> "./test-%{+YYYY-MM-dd}.txt"

# }

}

at last,thanks for this thread [keep global variable in logstash configuration](https://discuss.elastic.co/t/keeping-global-variables-in-ls/39908/18) which let me know how use ruby syntax to define the global variable

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:42am UTC](https://discuss.elastic.co/t/logstash-parent-child-event-configuration/58117/3 "2017-07-06T04:42:47Z")

</div>