# Logstash JSON conversion can we control

**URL:** https://discuss.elastic.co/t/logstash-json-conversion-can-we-control/107379
**Category:** Logstash
**Created:** [November 13, 2017, 11:49am UTC](https://discuss.elastic.co/t/logstash-json-conversion-can-we-control/107379 "2017-11-13T11:49:47Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Shekhar\_Pandey](https://avatars.discourse-cdn.com/v4/letter/s/7ab992/32.png) [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)
#### Post date: [November 13, 2017, 11:49am UTC](https://discuss.elastic.co/t/logstash-json-conversion-can-we-control/107379/1 "2017-11-13T11:49:48Z")

</div>

When logstash fetch data from mysql....then it convert into json kv form....Can I control this conversion and what field I want it make in nested json form....is this possible in logstash....

Bcz I want some fields of mysql in nested json form....

Thanks in advance.

---

<div class="post-metadata">

### Author: ![krishnakumarag](https://avatars.discourse-cdn.com/v4/letter/k/a6a055/32.png) [@krishnakumarag](https://discuss.elastic.co/u/krishnakumarag)
#### Post date: [November 13, 2017, 1:45pm UTC](https://discuss.elastic.co/t/logstash-json-conversion-can-we-control/107379/2 "2017-11-13T13:45:50Z")

</div>

You have filter plugin and format the data structure you want  
[filter plugin](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html)

---

<div class="post-metadata">

### Author: ![Shekhar\_Pandey](https://avatars.discourse-cdn.com/v4/letter/s/7ab992/32.png) [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)
#### Post date: [November 13, 2017, 2:28pm UTC](https://discuss.elastic.co/t/logstash-json-conversion-can-we-control/107379/3 "2017-11-13T14:28:58Z")

</div>

Thanks Krishna. If any problem, I will face...I will be back...

---

<div class="post-metadata">

### Author: ![Shekhar\_Pandey](https://avatars.discourse-cdn.com/v4/letter/s/7ab992/32.png) [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)
#### Post date: [November 14, 2017, 7:45am UTC](https://discuss.elastic.co/t/logstash-json-conversion-can-we-control/107379/4 "2017-11-14T07:45:00Z")

</div>

Hi Krishna!

I follow the filter topic...but some where I stuck....I given full error details in the below link...

Kindly see [Logstash Aggregate plugin](https://discuss.elastic.co/t/logstash-aggregate-plugin/107501)

And same here also..  
This example from elastic documentation:  
Given that you have this SQL query : SELECT country\_name, town\_name FROM town

{ "country\_name": "France", "town\_name": "Paris" }  
{ "country\_name": "France", "town\_name": "Marseille" }  
{ "country\_name": "USA", "town\_name": "New-York" }

And you would like these 2 result events to push them into elasticsearch :  
{ "country\_name": "France", "towns": [{"town\_name": "Paris"}, {"town\_name": "Marseille"}] }  
{ "country\_name": "USA", "towns": [{"town\_name": "New-York"}] }

They mention we have to follow this filter  
You can do that using push\_previous\_map\_as\_event aggregate plugin option :  
filter {  
aggregate {  
task\_id =\> "%{country\_name}"  
code =\> "  
map['country\_name'] = event.get('country\_name')  
map['towns'] ||= []  
map['towns'] \<\< {'town\_name' =\> event.get('town\_name')}  
event.cancel()  
"  
push\_previous\_map\_as\_event =\> true  
timeout =\> 3  
}  
}

And with respect to above filter...I have sql query like this..  
Select pid, fsid, title, brand from table1;  
But I don't want to aggregate it...but I want to put all the above fields inside of one another name like Shekhar...

So ..I used the following method...  
filter {  
aggregate {  
task\_id =\> "%{id}"  
code =\> "

```
 map['shekhar'] ||= []
 map['shekhar'] << {'brand' => event.get('brand')}
map['shekhar'] << {'pid' => event.get('pid')}
map['shekhar'] << {'title' => event.get('title')}

```

map['shekhar'] \<\< {'brand\_id' =\> event.get('brand\_id')}  
event.cancel()  
"  
push\_previous\_map\_as\_event =\> true  
timeout =\> 3  
}  
}

I got output as I want all fields inside of the shekhar[].....but only one output showing because of aggregation....it showing last value....so how I can edit the above filter to get all the documents.....

Thank you.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 12, 2017, 7:45am UTC](https://discuss.elastic.co/t/logstash-json-conversion-can-we-control/107379/5 "2017-12-12T07:45:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
