"
pipeline: "qradar-eps"
```
Pipeline:
> {
> "qradar-eps": {
> "processors": [
> {
> "grok": {
> "field": "message",
> "patterns": [
> """(%{NUMBER:time1}s: %{NUMBER:eps1} eps), (%{NUMBER:time2}s: %{NUMBER:eps2} eps), (%{NUMBER:time3}s: %{NUMBER:eps3} eps), (%{NUMBER:time4}s: %{NUMBER:eps4} eps), (%{NUMBER:time5}s: %{NUMBER:eps5} eps), (%{NUMBER:time6}s: %{NUMBER:eps6} eps), (%{NUMBER:time7}s: %{NUMBER:eps7} eps)"""
> ]
> }
> }
> ]
> }
> }
Example log:
```auto
{
"_index": ".ds-syslog-testing-2023.05.10-000001",
"_id": "9lgdB4gBYplAgGeV2-L4",
"_version": 1,
"_score": 0,
"_source": {
"@timestamp": "2023-05-10T19:22:56.000Z",
"input": {
"type": "syslog"
},
"ecs": {
"version": "8.0.0"
},
"agent": {
"ephemeral_id": "f6f23923-e832-459e-8824-20a0e0b79751",
"id": "e7faf270-e51f-4fae-9a04-720d54908df8",
"name": "filebeat.log",
"type": "filebeat",
"version": "8.7.1"
},
"hostname": "dlc-1056",
"syslog": {
"facility": 16,
"facility_label": "local0",
"priority": 134,
"severity_label": "Informational"
},
"event": {
"severity": 6
},
"log": {
"source": {
"address": "10.1.1.1:1000"
}
},
"message": "abc123-log 2023-05-10 15:22:49,873 [3a01ef7b-7961-4bc9-a73a-3ab9fec2b36b/SequentialEventDispatcher] com.q1labs.sem.monitors.SourceMonitor: [INFO] [NOT:0000006000][10.1.1.1/- -] [-/- -]Incoming raw event rate (5s: 0.40 eps), (10s: 0.40 eps), (15s: 0.40 eps), (30s: 0.40 eps), (60s: 0.47 eps), (300s: 0.48 eps), (900s: 0.48 eps). Peak in the last 60s: 1.00 eps. Max Seen 18.40 eps. DLC Throttles/5s (60s: 0.00). Total DLC Throttles in the last 60s: 0. Total DLC Throttles: 0. DLC Threshold: 1.0E8. ",
"tags": [
"abc456_testing"
],
"host": {
"os": {
"version": "7.9 (Maipo)",
"family": "redhat",
"name": "Red Hat Enterprise Linux Server",
"kernel": "3.10.0-1160.83.1.el7.x86_64",
"codename": "Maipo",
"type": "linux",
"platform": "rhel"
},
"id": "4ad693cbe0e9404bad767e038a80d05f",
"containerized": false,
"ip": [
"10.115.59.173"
],
"mac": [
"00-50-56-A6-53-15"
],
"name": "abc456",
"hostname": "abc456",
"architecture": "x86_64"
}
},
"fields": {
"@timestamp": [
"2023-05-10T19:22:56.000Z"
]
}
}
```
Part of the response:
```auto
GET _nodes/stats/ingest?filter_path=nodes.*.ingest
"qradar-eps": {
"count": 0,
"time_in_millis": 0,
"current": 0,
"failed": 0,
"processors": [
{
"grok": {
"type": "grok",
"stats": {
"count": 0,
"time_in_millis": 0,
"current": 0,
"failed": 0
}
}
}
]
}
```
---
### Author:  [@Anton\_H](https://discuss.elastic.co/u/Anton_H)
#### Post date: [May 12, 2023, 8:49am UTC](https://discuss.elastic.co/t/syslog-ingest-pipeline-not-targeting-data/333204/2 "2023-05-12T08:49:05Z")
What happens when you add a on\_failure section to your ingest pipeline?
```auto
"on_failure": [
{
"append": {
"field": "error.message",
"value": "{{{_ingest.on_failure_message}}}"
}
}
]
```
---
### Author:  [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)
#### Post date: [May 12, 2023, 6:07pm UTC](https://discuss.elastic.co/t/syslog-ingest-pipeline-not-targeting-data/333204/3 "2023-05-12T18:07:49Z")
No luck so far.
```auto
"qradar-eps": {
"count": 0,
"time_in_millis": 0,
"current": 0,
"failed": 0,
"processors": [
{
"grok": {
"type": "grok",
"stats": {
"count": 0,
"time_in_millis": 0,
"current": 0,
"failed": 0
}
}
}
]
}
```
```auto
GET _ingest/pipeline/qradar-eps
{
"qradar-eps": {
"processors": [
{
"grok": {
"field": "message",
"patterns": [
"""\(%{NUMBER:time1}s: %{NUMBER:eps1} eps\), \(%{NUMBER:time2}s: %{NUMBER:eps2} eps\), \(%{NUMBER:time3}s: %{NUMBER:eps3} eps\), \(%{NUMBER:time4}s: %{NUMBER:eps4} eps\), \(%{NUMBER:time5}s: %{NUMBER:eps5} eps\), \(%{NUMBER:time6}s: %{NUMBER:eps6} eps\), \(%{NUMBER:time7}s: %{NUMBER:eps7} eps\)"""
],
"on_failure": [
{
"append": {
"field": "error.message",
"value": "{{{_ingest.on_failure_message}}}"
}
}
]
}
}
]
}
}
```
---
### Author:  [@Anton\_H](https://discuss.elastic.co/u/Anton_H)
#### Post date: [May 13, 2023, 10:16am UTC](https://discuss.elastic.co/t/syslog-ingest-pipeline-not-targeting-data/333204/4 "2023-05-13T10:16:51Z")
@Ryan_Downey I see that your grok pattern matches your example but only for the part of the EPS values. So there is no "match" for the entire message field.
Could you try and use the following grok pattern and then just ignore/drop the drop\_start and drop\_end fields?
```auto
(%{GREEDYDATA:drop_start} )\(%{NUMBER:time1}s: %{NUMBER:eps1} eps\), \(%{NUMBER:time2}s: %{NUMBER:eps2} eps\), \(%{NUMBER:time3}s: %{NUMBER:eps3} eps\), \(%{NUMBER:time4}s: %{NUMBER:eps4} eps\), \(%{NUMBER:time5}s: %{NUMBER:eps5} eps\), \(%{NUMBER:time6}s: %{NUMBER:eps6} eps\), \(%{NUMBER:time7}s: %{NUMBER:eps7} eps\)(.%{GREEDYDATA:drop_end})
```
Something else, have you tried linking the ingest pipeline to the index template? So events get processed by that ingest pipeline without you having to name the pipeline in your filebeat output.
---
### Author:  [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)
#### Post date: [May 17, 2023, 12:30pm UTC](https://discuss.elastic.co/t/syslog-ingest-pipeline-not-targeting-data/333204/5 "2023-05-17T12:30:13Z")
Anton,
Sorry for the delay, I forgot to hit the reply button. I've update the template and the grok pattern so we're now getting the data in the new fields populating. I appreciate the help getting this sorted out.
Ryan
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 31, 2023, 12:30pm UTC](https://discuss.elastic.co/t/syslog-ingest-pipeline-not-targeting-data/333204/6 "2023-05-31T12:30:43Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.