# Custom Module Issue - Ingest Pipeline Alpha Sorted?

**URL:** https://discuss.elastic.co/t/custom-module-issue-ingest-pipeline-alpha-sorted/92845
**Category:** Beats
**Tags:** filebeat
**Created:** [July 12, 2017, 4:54pm UTC](https://discuss.elastic.co/t/custom-module-issue-ingest-pipeline-alpha-sorted/92845 "2017-07-12T16:54:36Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![DangMA](https://avatars.discourse-cdn.com/v4/letter/d/49beb7/32.png) [@DangMA](https://discuss.elastic.co/u/DangMA)
#### Post date: [July 12, 2017, 4:54pm UTC](https://discuss.elastic.co/t/custom-module-issue-ingest-pipeline-alpha-sorted/92845/1 "2017-07-12T16:54:36Z")

</div>

I have created a custom module using `make create-fileset`. I've populated the various files, run `make collect`, and copied the results into my filebeat module directory. ALMOST everything is working as expected.

When filebeat executes the first time, it loads the ingest pipeline in alphabetical order rather than the order set in the `pipeline.json` file\*. This causes the grok processor, that needs to be processed first, to execute after the other processors.

Has anyone seen this before? is there a way to force filebeat to respect the ordering in the pipeline.json file when pushing the pipeline to ES?

Configuration:  
elastic search & kibana 5.4.1 (via docker-compose up)  
filebeat 5.5.0 (installed via brew)

- I've confirmed the pipeline processor ordering is correct in the module's pipeline.json and that filebeat, with debug out, is reading the pipeline and pushing it in alphabetical order.

---

<div class="post-metadata">

### Author: ![DangMA](https://avatars.discourse-cdn.com/v4/letter/d/49beb7/32.png) [@DangMA](https://discuss.elastic.co/u/DangMA)
#### Post date: [July 13, 2017, 8:59pm UTC](https://discuss.elastic.co/t/custom-module-issue-ingest-pipeline-alpha-sorted/92845/2 "2017-07-13T20:59:42Z")

</div>

Answering my own post... This took quite a while to figure out. (Probably more than it should have ☹ )

When initially loading a pipeline from a module, filebeat alphabetizes the processor entries within each ingest sub-block. Filebeat doesn't adhere to the ordering within the `pipeline.json` file.

To ensure proper ordering and execution, place any directives to be processed in sequential order in discrete `{}` blocks (nested at the same level).

In my case, the initial `pipeline.json` file was formatted as:

```auto
{
  "description" : "my pipeline",
  "processors" : [
    {
      "grok": {
          ...
      },
      "date" : {
        ...
      }
    }
  ]
}

```

After isolating each processor in a discrete block, filebeat loaded the pipeline as intended :

```auto
{
  "description" : "my pipeline",
  "processors" : [
    {
      "grok": {
          ...
      }
    },
    {
      "date" : {
        ...
      }
    }
  ]
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 10, 2017, 8:59pm UTC](https://discuss.elastic.co/t/custom-module-issue-ingest-pipeline-alpha-sorted/92845/3 "2017-08-10T20:59:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
