# Filebeat to elasticsearch's ingest node in Kubernetes

**URL:** <https://discuss.elastic.co/t/filebeat-to-elasticsearchs-ingest-node-in-kubernetes/159103>\
**Category:** Elasticsearch\
**Created:** [December 3, 2018, 7:25am UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearchs-ingest-node-in-kubernetes/159103 "2018-12-03T07:25:20Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![zeusro](https://avatars.discourse-cdn.com/v4/letter/z/57b2e6/32.png) [@zeusro](https://discuss.elastic.co/u/zeusro)\
**Post date:** [December 3, 2018, 7:25am UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearchs-ingest-node-in-kubernetes/159103/1 "2018-12-03T07:25:20Z")

</div>

I followed the guide as [running-on-kubernetes](https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html) say,everything worked fine,but one day I would like to spilt the message field by using ES's ingrest node.So I created three pipelines to handle the original docker's container message .But after I created them,there are no newer log inserting to the elastic searh any more.

So,my problem is how to set something to make the log system work properly.

here is my filebeat.yml

```auto
    filebeat.config:
        inputs:
          # Mounted `filebeat-inputs` configmap:
          path: ${path.config}/inputs.d/*.yml
          # Reload inputs configs as they change:
          reload.enabled: false
        modules:
          path: ${path.config}/modules.d/*.yml
          # Reload module configs as they change:
          reload.enabled: false

    # To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
    #filebeat.autodiscover:
    # providers:
    # - type: kubernetes
    # hints.enabled: true
    processors:
      - add_cloud_metadata:

    cloud.id: ${ELASTIC_CLOUD_ID}
    cloud.auth: ${ELASTIC_CLOUD_AUTH}

    output:
      elasticsearch:
        hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
        # username: ${ELASTICSEARCH_USERNAME}
        # password: ${ELASTICSEARCH_PASSWORD}
        pipelines:          
          - pipeline: "nginx"
            when.contains:
              kubernetes.container.name: "nginx-"
          - pipeline: "java"
            when.contains:
              kubernetes.container.name: "java-"              
          - pipeline: "default"  
            when.contains:
              kubernetes.container.name: ""         

```

Here are my pipeline which I declared

```auto

PUT /_ingest/pipeline/java

{

 "description": "[0]java[1]nginx[last]通用规则",

 "processors": [{

  "grok": {

   "field": "message",

   "patterns": [

    "\\[%{LOGLEVEL:level}\\s+?\\]\\[(?&lt;date&gt;\\d{4}-\\d{2}-\\d{2}\\s\\d{2}:\\d{2}:\\d{2},\\d{3})\\]\\[(?&lt;thread&gt;[A-Za-z0-9/-]+?)\\]\\[%{JAVACLASS:class}\\]\\[(?&lt;msg&gt;[\\s\\S]*?)\\]\\[(?&lt;stack&gt;.*?)\\]"

   ]

  },"remove": {

              "field": "message"

            }

 }]

}

PUT /_ingest/pipeline/nginx

{

 "description": "[0]java[1]nginx[last]通用规则",

 "processors": [{

  "grok": {

   "field": "message",

   "patterns": [

    "%{IP:client} - - \\[(?&lt;date&gt;.*?)\\] \"(?&lt;method&gt;[A-Za-z]+?) (?&lt;url&gt;.*?)\" %{NUMBER:statuscode} %{NUMBER:duration} \"(?&lt;refer&gt;.*?)\" \"(?&lt;user-agent&gt;.*?)\"" 

   ]

  },"remove": {

              "field": "message"

            }

 }]

}

PUT /_ingest/pipeline/default

{

 "description": "[0]java[1]nginx[last]通用规则",

 "processors": []

}

```

Thanks for any help 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 31, 2018, 7:38am UTC](https://discuss.elastic.co/t/filebeat-to-elasticsearchs-ingest-node-in-kubernetes/159103/2 "2018-12-31T07:38:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.