# Filebeat regexp

**URL:** https://discuss.elastic.co/t/filebeat-regexp/191792
**Category:** Beats
**Tags:** filebeat
**Created:** [July 23, 2019, 10:56am UTC](https://discuss.elastic.co/t/filebeat-regexp/191792 "2019-07-23T10:56:08Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![O.Shulha](https://avatars.discourse-cdn.com/v4/letter/o/e5b9ba/32.png) [@O.Shulha](https://discuss.elastic.co/u/O.Shulha)
#### Post date: [July 23, 2019, 10:56am UTC](https://discuss.elastic.co/t/filebeat-regexp/191792/1 "2019-07-23T10:56:08Z")

</div>

I have some log files in folder. I tried to send files to the logstash.

filebeat.yml

```auto
filebeat.inputs:
- input_type: log
  enabled: true
  paths:
    - /opt/nso/ncs-run/logs/rollback*
  multiline.pattern: 'ncs:*'
  multiline.negate: true
  multiline.match: after
  multiline.flush_pattern: '^/s}'

output:
  logstash:
    hosts: ["host:5002"]

```

Log file example:

```auto
# Created by: admin
# Date: 2019-07-18 17:10:35
# Via: rest
# Type: delta
# Label: 
# Comment: 
# No: 10087

ncs:services {
    ap:attachment {
        ap:pe asr9006a {
            ap:vrf {
                ap:vrf 9af34f86ec7448f6b223ab0ee77b8413 {
                    delete:
                    ap:bgp-peer 172.101.0.54;
                }
             }
         }
     }
 }

```

I want to send part of this log. Start from ncs: and end of the file "}"

logstash.conf

```auto

input {
  beats {
    port => 5002
  }
}

filter {}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "logstash-beats-%{+YYYY.MM.dd}"
  }
}

```

Errors:

```auto
[2019-07-23T09:20:55,280][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-syslog-2019.07.23", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x34fc23b>], :response=>{"index"=>{"_index"=>"logstash-syslog-2019.07.23", "_type"=>"_doc", "_id"=>"9MciHmwBv7eJVn6lOJd9", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [host] of type [text] in document with id '9MciHmwBv7eJVn6lOJd9'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:75"}}}}}

```

```auto
[2019-07-23T09:20:55,281][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-syslog-2019.07.23", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x6cde57c2>], :response=>{"index"=>{"_index"=>"logstash-syslog-2019.07.23", "_type"=>"_doc", "_id"=>"-sciHmwBv7eJVn6lOJd9", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [host] of type [text] in document with id '-sciHmwBv7eJVn6lOJd9'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:75"}}}}}

```

All versions are = 7.2  
And filebeat don`t sends the whole message. Just some parts. How to solve it??

Thank you a lot

---

<div class="post-metadata">

### Author: ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)
#### Post date: [July 25, 2019, 6:23pm UTC](https://discuss.elastic.co/t/filebeat-regexp/191792/2 "2019-07-25T18:23:24Z")

</div>

Hello @O.Shulha,

Looking the regular expression you are using I think it's almost correct.  
The pattern should be:

```auto
  multiline.pattern: '^ncs:.*'

```

looking a the other errors which come from the elasticsearch output, I think its a mapping issues, and there is some work around and solution in [Logstash errors after upgrading to filebeat-6.3.0](https://discuss.elastic.co/t/logstash-errors-after-upgrading-to-filebeat-6-3-0/135984/6)

```auto
[2019-07-23T09:20:55,281][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-syslog-2019.07.23", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x6cde57c2>], :response=>{"index"=>{"_index"=>"logstash-syslog-2019.07.23", "_type"=>"_doc", "_id"=>"-sciHmwBv7eJVn6lOJd9", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [host] of type [text] in document with id '-sciHmwBv7eJVn6lOJd9'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:75"}}}}}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 23, 2019, 7:38am UTC](https://discuss.elastic.co/t/filebeat-regexp/191792/4 "2019-08-23T07:38:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
