# Filebeat not using new pipeline
**URL:** https://discuss.elastic.co/t/filebeat-not-using-new-pipeline/218223
**Category:** Beats
**Tags:** filebeat
**Created:** [February 6, 2020, 4:58pm UTC](https://discuss.elastic.co/t/filebeat-not-using-new-pipeline/218223 "2020-02-06T16:58:14Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@ethranes](https://discuss.elastic.co/u/ethranes)
#### Post date: [February 6, 2020, 4:58pm UTC](https://discuss.elastic.co/t/filebeat-not-using-new-pipeline/218223/1 "2020-02-06T16:58:14Z")
Hi, I have created the following pipeline in the dev tools menu
```
PUT _ingest/pipeline/filebeat-ssologin
{
"processors": [
{
"dissect": {
"field": "message",
"pattern": "%{date} [%{log.logger}] %{log.level} %{log.origin.function} - %{message}"
}
},
{
"date": {
"field": "date",
"formats": [
"HH:mm:ss,SSS"
]
}
},
{
"remove": {
"field": "date"
}
}
]
}
```
This is working fine when I simulate with the following log
```
POST _ingest/pipeline/filebeat-ssologin/_simulate
{"docs":[{"_source":{"message":"15:55:32,864 [12] INFO SsoBrowserPopUpLib.SsoBrowserPopUp+<>c - Time Out in 290 seconds."}}]}
```
And I have set this pipleine in my filebeat.yml file, and stopped and restarted filebeat
`pipeline: "filebeat-ssologin"`
But after this, I'm not getting any logs coming through at all. I hope that I can get some help here getting my logs working with my new pipeline.
Thanks.
---
### Author:  [@ethranes](https://discuss.elastic.co/u/ethranes)
#### Post date: [February 7, 2020, 4:33pm UTC](https://discuss.elastic.co/t/filebeat-not-using-new-pipeline/218223/2 "2020-02-07T16:33:45Z")
Some additional info. From the filebeat running on my local machine, I can't see any obvious error on why it's not sending the data
```
2020-02-07T16:28:45.645Z DEBUG [input] input/input.go:152 Run input
2020-02-07T16:28:45.645Z DEBUG [input] log/input.go:187 Start next scan
2020-02-07T16:28:45.646Z DEBUG [input] log/input.go:417 Check file for harvesting: C:\something\something.log
2020-02-07T16:28:45.646Z DEBUG [input] log/input.go:507 Update existing file for harvesting: C:\something\something.log, offset: 164405
2020-02-07T16:28:45.646Z DEBUG [input] log/input.go:559 Harvester for file is still running: C:\something\something.log
2020-02-07T16:28:45.646Z DEBUG [input] log/input.go:208 input states cleaned up. Before: 1, After: 1, Pending: 0
2020-02-07T16:28:45.675Z DEBUG [multiline] multiline/multiline.go:175 Multiline event flushed because timeout reached.
2020-02-07T16:28:45.675Z DEBUG [processors] processing/processors.go:183 Publish event: {
"@timestamp": "2020-02-07T16:28:40.669Z",
"@metadata": {
"beat": "",
"type": "_doc",
"version": "",
"pipeline": "filebeat-ssologin"
},
"log": {
"offset": 164405,
"file": {
"path": "C:\\something\\something.log"
}
},
"message": "something ",
"input": {
"type": "log"
},
"ecs": {
"version": "1.0.0"
},
"host": {
"name": "something"
},
"agent": {
"hostname": "something",
"id": "47268521-67cf-4b2b-a24c-f17020bc8cb8",
"version": "7.1.1",
"type": "filebeat",
"ephemeral_id": "301f8140-10ce-4716-9ca9-8e4830e2e873"
}
}
2020-02-07T16:28:46.775Z DEBUG [elasticsearch] elasticsearch/client.go:338 PublishEvents: 1 events have been published to elasticsearch in 98.2554ms.
2020-02-07T16:28:46.776Z DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [211: 0, 1]
2020-02-07T16:28:46.777Z DEBUG [publisher] memqueue/eventloop.go:535 broker ACK events: count=1, start-seq=328, end-seq=328
2020-02-07T16:28:46.777Z DEBUG [publisher] memqueue/ackloop.go:128 ackloop: return ack to broker loop:1
2020-02-07T16:28:46.777Z DEBUG [publisher] memqueue/ackloop.go:131 ackloop: done send ack
2020-02-07T16:28:46.777Z DEBUG [acker] beater/acker.go:64 stateful ack {"count": 1}
2020-02-07T16:28:46.777Z DEBUG [registrar] registrar/registrar.go:356 Processing 1 events
2020-02-07T16:28:46.777Z DEBUG [registrar] registrar/registrar.go:326 Registrar state updates processed. Count: 1
2020-02-07T16:28:46.777Z DEBUG [registrar] registrar/registrar.go:411 Write registry file: C:\Users\something\Desktop\Elastic\filebeat-7.1.1-windows-x86_64\data\registry\filebeat\data.json
2020-02-07T16:28:46.795Z DEBUG [registrar] registrar/registrar.go:404 Registry file updated. 4 states written.
2020-02-07T16:28:47.677Z DEBUG [harvester] log/log.go:102 End of file reached: C:\something\something.log; Backoff now.
2020-02-07T16:28:55.647Z DEBUG [input] input/input.go:152 Run input
```
---
### Author:  [@ethranes](https://discuss.elastic.co/u/ethranes)
#### Post date: [February 11, 2020, 4:17pm UTC](https://discuss.elastic.co/t/filebeat-not-using-new-pipeline/218223/3 "2020-02-11T16:17:51Z")
After some reading I thought the issue may be the location where I've set the pipeline in the yml file, was originally
```
filebeat.inputs:
pipeline: "filebeat-ssologin"
```
I've now moved it to
```
output.elasticsearch:
pipeline: "filebeat-ssologin"
```
But it doesn't look like that has fixed the issue, or changed anything in the filebeat logs, issue remains.
---
### Author:  [@ethranes](https://discuss.elastic.co/u/ethranes)
#### Post date: [February 12, 2020, 12:14pm UTC](https://discuss.elastic.co/t/filebeat-not-using-new-pipeline/218223/4 "2020-02-12T12:14:08Z")
Wasn't getting anywhere using the dissect. So switched to using grok instead. And that works
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 11, 2020, 12:14pm UTC](https://discuss.elastic.co/t/filebeat-not-using-new-pipeline/218223/5 "2020-03-11T12:14:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.