# How to split index by field value? **URL:** \ **Category:** Logstash\ **Created:** [March 23, 2021, 8:12am UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055 "2021-03-23T08:12:39Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Rahmat\_Agung\_W](https://avatars.discourse-cdn.com/v4/letter/r/c0e974/32.png) [@Rahmat\_Agung\_W](https://discuss.elastic.co/u/Rahmat_Agung_W)\ **Post date:** [March 23, 2021, 8:12am UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/1 "2021-03-23T08:12:39Z")
I have a VM. There are 2 similiar apps (docker apps) inside it, staging and development. They use 1 filebeat to push log to logstash. On these log, I have field `container.labels.com_docker_compose_project`. I want to create 2 index. with condition: First index, will collect log if there is value start with `stag` on `container.labels.com_docker_compose_project` field. Second index, will collect log if there is value start with `dev` on `container.labels.com_docker_compose_project` field. Is it possible? If possible how? I want to try something like this: `if "^aeon-piccolo" in [container.labels.com_docker_compose_project]` But I think it didnt work. ---
**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\ **Post date:** [March 23, 2021, 9:29am UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/2 "2021-03-23T09:29:31Z")
Hi you could change the metadata fields for target index like this in a filter ```auto filter { if "^dev" in [tags] { mutate{ add_field =>{"[@metadata][target_index]"=> "devindex-%{+YYYY-MM-dd}"}} } } output { index => "%{[@metadata][target_index]}" } ``` 😉 ---
**Author:** ![Rahmat\_Agung\_W](https://avatars.discourse-cdn.com/v4/letter/r/c0e974/32.png) [@Rahmat\_Agung\_W](https://discuss.elastic.co/u/Rahmat_Agung_W)\ **Post date:** [March 23, 2021, 11:33am UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/3 "2021-03-23T11:33:45Z")
Is `[tags]` mean `[container.labels.com_docker_compose_project]` in my case? Or it is literraly `[tags]`? ---
**Author:** ![Rahmat\_Agung\_W](https://avatars.discourse-cdn.com/v4/letter/r/c0e974/32.png) [@Rahmat\_Agung\_W](https://discuss.elastic.co/u/Rahmat_Agung_W)\ **Post date:** [March 23, 2021, 11:46am UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/4 "2021-03-23T11:46:26Z")
Could you explain what this syntax did? 😅 ---
**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\ **Post date:** [March 23, 2021, 12:53pm UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/5 "2021-03-23T12:53:28Z")
Yes yes just adapt it to the fields you want to check on, sure i did use tags in my example but it's not mandatory ---
**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\ **Post date:** [March 23, 2021, 12:54pm UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/6 "2021-03-23T12:54:04Z")
Use your field not my example 😉 ---
**Author:** ![Rahmat\_Agung\_W](https://avatars.discourse-cdn.com/v4/letter/r/c0e974/32.png) [@Rahmat\_Agung\_W](https://discuss.elastic.co/u/Rahmat_Agung_W)\ **Post date:** [March 23, 2021, 1:21pm UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/7 "2021-03-23T13:21:43Z")
So my field is something look like this: ```auto - [container.image.name]: dev-example-nginx - [container.image.name]: stag-example-nginx - [container.image.name]: dev-example-mysql ``` ```auto if "^dev-example" in [container.image.name] { mutate { add_field =>{"[@metadata][target_index]"=> "example-dev"} } } else { mutate { add_field =>{"[@metadata][target_index]"=> "example-stag"} } } } output{...} ``` With above config, I only retrieve index example-stag. Did my regex wrong? ---
**Author:** ![Rahmat\_Agung\_W](https://avatars.discourse-cdn.com/v4/letter/r/c0e974/32.png) [@Rahmat\_Agung\_W](https://discuss.elastic.co/u/Rahmat_Agung_W)\ **Post date:** [March 23, 2021, 1:47pm UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/8 "2021-03-23T13:47:29Z")
And my index become like this %{[@metadata][target\_index]}-example-dev-mysql-2021.03` My config: ```auto index => "%{[@metadata][target_index]}-%{[container][image][name]}-%{+YYYY.MM}" ``` ---
**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\ **Post date:** [March 25, 2021, 2:13pm UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/9 "2021-03-25T14:13:50Z")
You might want to try =~ with the regexp but yeah that's the idea. Always try to match full strings when you can ( less cpu and more reliable ) ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 25, 2021, 6:15pm UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/10 "2021-03-25T18:15:07Z")
> [@Rahmat\_Agung\_W](#): > > `if "^dev-example" in [container.image.name] {` "in" here is a substring match, not a regular expression. You cannot use an anchor. So ``` if "v-ex" in [container.image.name] { ``` would match. I think you want ``` if [container.image.name] =~ /^dev-example/ ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 22, 2021, 6:15pm UTC](https://discuss.elastic.co/t/how-to-split-index-by-field-value/268055/11 "2021-04-22T18:15:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.