# Unable to push data into multiple indices

**URL:** <https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685>\
**Category:** Beats\
**Tags:** winlogbeat\
**Created:** [October 29, 2020, 11:58am UTC](https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685 "2020-10-29T11:58:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![bhaskar\_todi](https://avatars.discourse-cdn.com/v4/letter/b/6bbea6/32.png) [@bhaskar\_todi](https://discuss.elastic.co/u/bhaskar_todi)\
**Post date:** [October 29, 2020, 11:58am UTC](https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685/1 "2020-10-29T11:58:34Z")

</div>

I am trying to push different streams of windows events into different elastic indices based on the event.provider

But unfortunately, all my data is pushed into the default index.

I am not sure if the filtering is not working or the indices block is not set up properly

Here is my config

```auto
setup.template.name: "sysmonbeat-%{[agent.version]}"

setup.template.pattern: "sysmonbeat-%{[agent.version]}-*"

setup.ilm.rollover_alias: "sysmonbeat-%{[agent.version]}"

output.elasticsearch:

    hosts: ["URL"]

    username: "USERNAME"

    password: "PASSWORD"

    max_retries: 3

    indices:

        - index: 'sysmonbeat-%{[agent.version]}-%{+yyyy.MM.dd}'       

          when.equals:

            event.provider: 'Microsoft-Windows-Sysmon'

        - index: 'winlogbeat-%{[agent.version]}-%{+yyyy.MM.dd}' # default winlogbeat index

processors:

    - add_tags:

        tags:

            - "TAG"

    - add_labels:

        labels:

            tag: "TAG"

            datacenter: "NONPROD"

    - drop_fields:

        fields:

            - event.kind

            - process # drop the whole process subobject

            - agent # drop the whole process subobject

            - winlog.api

            - winlog.channel

            - winlog.opcode

            - winlog.provider_guid

            - winlog.record_id

            - winlog.task

            - winlog.computer_name

            - winlog_event_data.Binary

            - winlog.event_data.Image

            - winlog.event_data.ProcessGuid

            - winlog.event_data.ProcessId

            - winlog.event_data.RuleName

            - winlog.event_data.UtcTime

            - winlog.event_id

            - winlog.process.pid

            - winlog.process.thread.id

            - winlog.record_id

            - winlog.user.domain

            - winlog.user.identifier

            - winlog.user.name

            - winlog.user.type

            - winlog.version

            - winlog.user.type

            - winlog.user.identifier

            - winlog.user.name

            - winlog.user.domain

winlogbeat.event_logs:

    - name: Application

        ignore_older: 1s # do not push previously existing events

    - name: System

        ignore_older: 1s # do not push previously existing events

    - name: Microsoft-Windows-Sysmon/Operational

        event_id: 3

        ignore_older: 1s # do not push previously existing events

        provider:

            - Microsoft-Windows-Sysmon

        processors:

            - drop_fields:

                fields:

                    - event.kind

                    - log.level

                    - message

                    - winlog.event_data.User

    - name: Microsoft-Windows-Sysmon/Operational

        event_id: 22

        ignore_older: 1s # do not push previously existing events

        provider:

            - Microsoft-Windows-Sysmon

        processors:

            - drop_fields:

                fields:

                    - event.kind

                    - log.level

                    - message

                    - winlog.event_data.User

```

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [October 29, 2020, 2:15pm UTC](https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685/2 "2020-10-29T14:15:43Z")

</div>

> [@bhaskar\_todi](#):
>
> ```auto
> when.equals:
> event.provider: 'Microsoft-Windows-Sysmon'
> 
> ```

Winlogbeat doesn't use a field named `event.provider`. There is a `winlog.provider_name` that should work.

---

<div class="post-metadata">

**Author:** ![bhaskar\_todi](https://avatars.discourse-cdn.com/v4/letter/b/6bbea6/32.png) [@bhaskar\_todi](https://discuss.elastic.co/u/bhaskar_todi)\
**Post date:** [October 30, 2020, 10:43am UTC](https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685/3 "2020-10-30T10:43:43Z")

</div>

Nope, that doesn't work for me

Updated indices block

```auto
     indices:
        - index: 'sysmonbeat-%{[agent.version]}-%{+yyyy.MM.dd}' # More like an if else ladder
          when.equals:
            winlog.provider_name: 'Microsoft-Windows-Sysmon'
        - index: 'winlogbeat-%{[agent.version]}-%{+yyyy.MM.dd}' # default winlogbeat index

```

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [October 30, 2020, 6:56pm UTC](https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685/4 "2020-10-30T18:56:31Z")

</div>

I think you need to turn off [ILM](https://www.elastic.co/guide/en/beats/winlogbeat/7.7/ilm.html#ilm) to be able to customize the target index like this. Try it with

`setup.ilm.enabled: false`

---

<div class="post-metadata">

**Author:** ![bhaskar\_todi](https://avatars.discourse-cdn.com/v4/letter/b/6bbea6/32.png) [@bhaskar\_todi](https://discuss.elastic.co/u/bhaskar_todi)\
**Post date:** [November 2, 2020, 6:46am UTC](https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685/6 "2020-11-02T06:46:50Z")

</div>

All data is now going to the default index rather the custom index

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 30, 2020, 8:46am UTC](https://discuss.elastic.co/t/unable-to-push-data-into-multiple-indices/253685/7 "2020-11-30T08:46:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.