# How to create sub indexes for an index in config file of logstash
**URL:** https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881
**Category:** Logstash
**Created:** [July 19, 2016, 1:31pm UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881 "2016-07-19T13:31:46Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@pavana](https://discuss.elastic.co/u/pavana)
#### Post date: [July 19, 2016, 1:31pm UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/1 "2016-07-19T13:31:46Z")
I have QueueMessage and MessageID as two fields which i defined using csv as shown below.
input {
log4j {
mode =\> "server"
host =\> "0.0.0.0"
port =\> 3456
type =\> "log4j"
}
}
filter{
csv{
columns=\> ["QueueMessage","MessageID"]
separator =\> ","
}
}
output {
stdout {}
elasticsearch { hosts =\> ["localhost:9200"] }
}
Example :
QueueMessage="John,A001,$3000"
MessageID= ID:E4EMS-SERVER.1F0578DF8CA60:114
Now my QueueMessage has 3 more sub fields (say Name,PersonID,Salary). How should I change the config file of logstash accordingly to reflect these sub fields in kibana.
---
### Author:  [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)
#### Post date: [July 19, 2016, 2:44pm UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/2 "2016-07-19T14:44:08Z")
Hi @pavana,
I think the best solution at this point is to parse out the remaining values using the [Grok](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) filter.
Assuming the QueueMessage has double quotes surrounding the string, the Grok filter would look like this:
```auto
grok {
match => ["QueueMessage","\"%{DATA:Name},%{DATA:PersonID},\$%{NUMBER:Salary:int}\""]
}
```
Note the `int` keyword after the salary, and also that we are not including the dollar sign in the parsed variable. This way, you will be able to use the salary value as a number in Kibana.
In my test, the output produced was like this:
```auto
{
"message" => "\"John,A001,$3000\"",
"@version" => "1",
"@timestamp" => "2016-07-19T14:40:32.103Z",
"host" => "MacBook-Pro.local",
"name" => "John",
"PersonID" => "A001",
"Salary" => 3000
}
```
Let me know if this works for you!
---
### Author:  [@pavana](https://discuss.elastic.co/u/pavana)
#### Post date: [July 19, 2016, 4:27pm UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/3 "2016-07-19T16:27:24Z")
If you meant that the filter should be like this then i am not getting the required output.
QueueMessage=John
MessageID=A001
This is what i am getting. Also Name,PersonID and Salary fields are not created in my kibana.
filter{
csv{
columns=\> ["QueueMessage","MessageID"]
separator =\> ","
}
grok {
match =\> ["QueueMessage",""%{DATA:Name},
%{DATA:PersonID},$%{NUMBER:Salary:int}""]
}
}
---
### Author:  [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)
#### Post date: [July 19, 2016, 4:34pm UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/4 "2016-07-19T16:34:57Z")
@pavana,
Can you please share with me a sample of the source input message?
---
### Author:  [@pavana](https://discuss.elastic.co/u/pavana)
#### Post date: [July 19, 2016, 4:57pm UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/5 "2016-07-19T16:57:49Z")
Here "message" has 4 fields....John, A0001, $5000, ID:E4EMS-SERVER.E18578E50F61B:32
I am retrieving first 3 data values from Queue and 4th one is automatically generated.
Indirectly my "message" is made of only 2 fields "QueueMessage"( which has John, A0001, $5000) and "MessageID" (which is ID:E4EMS-SERVER.E18578E50F61B:32).
The whole point is there are sub fields in a field.
message:John,A0001,$5000,ID:E4EMS-SERVER.E18578E50F61B:32 @version:1 @timestamp:July 19th 2016, 21:54:07.050 timestamp:1468945447050 path:QueueReader priority:DEBUG logger\_name:QueueReader thread:main class:? file:?:? method:? application:playground host:127.0.0.1:50357 type:log4j tags:\_grokparsefailure QueueMessage:John MessageID:A0001 column3:$5000 column4:ID:E4EMS-SERVER.E18578E50F61B:32 \_id:AVYD-Lw07ZpGBU8wlvJY \_type:log4j \_index:logstash-2016.07.19 \_score:
---
### Author:  [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)
#### Post date: [July 19, 2016, 6:45pm UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/6 "2016-07-19T18:45:49Z")
@pavana,
If the original message is always 4 fields, then why bother using the CSV parser? Perhaps just using Grok like this will work?
```auto
filter{
grok {
match => ["message","%{DATA:Name}, %{DATA:PersonID}, \$%{NUMBER:Salary:int}, %{DATA:MessageID}"]
}
}
```
---
### Author:  [@pavana](https://discuss.elastic.co/u/pavana)
#### Post date: [July 20, 2016, 9:38am UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/7 "2016-07-20T09:38:05Z")
But this wont work if i want to select the whole message excluding MessageID.
I cant keep on select the fields present in my data right?
MessageID is generated by the system and other 3 fields are sent by an application.
MessageID is required only if there is error in the data sent.
This is my requirement.
Hope u got me. It's just like Message is composite attribute which has Name,PersonID,Salary.
First of all is it possible to include complex attributes and divide them in logstash config file?
---
### Author:  [@PhaedrusTheGreek](https://discuss.elastic.co/u/PhaedrusTheGreek)
#### Post date: [July 20, 2016, 11:48am UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/8 "2016-07-20T11:48:36Z")
@pavana,
I do not understand what you are trying to accomplish. What do you mean by "Select the whole message" ?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:47am UTC](https://discuss.elastic.co/t/how-to-create-sub-indexes-for-an-index-in-config-file-of-logstash/55881/9 "2017-07-06T04:47:19Z")