# Multiline txt log files not parsing correctly in ingest pipeline

**URL:** <https://discuss.elastic.co/t/multiline-txt-log-files-not-parsing-correctly-in-ingest-pipeline/229440>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 23, 2020, 10:39am UTC](https://discuss.elastic.co/t/multiline-txt-log-files-not-parsing-correctly-in-ingest-pipeline/229440 "2020-04-23T10:39:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nigel.piggott](https://avatars.discourse-cdn.com/v4/letter/n/94ad74/32.png) [@nigel.piggott](https://discuss.elastic.co/u/nigel.piggott)\
**Post date:** [April 23, 2020, 10:39am UTC](https://discuss.elastic.co/t/multiline-txt-log-files-not-parsing-correctly-in-ingest-pipeline/229440/1 "2020-04-23T10:39:25Z")

</div>

[on windows] I have a log file that can contain multiple lines.

The filebeat yml is correctly configured for multi line

The source appearing in elastic contains "\n"  
e.g. source file content "  
ababasd  
asfjklasjdflkjas  
asljdflkajs]"

will appear in the source as  
"ababasd\nasfjklasjdflkjas\nasljdflkajs]"

and in kibana would appear as  
"ababasd"

The ingest pipeline parses the source as greedy data  
e.g  
"%{GREEDYDATA:msg}"

The field msg then appears within kibana truncated at the first "\n" and not the full message with newlines

The exact same yml was used previously in a very early version of filebeat (1.1) and ELK (i.e. logstash) and worked fine

It would appear that FileBeat is encoding the newline in the log file as "\n" for json  
but then not decoding it when parsing it on elastic pipeline

is this supposed to work  
or only with LogStash

note i have tried to use a painless script in the pipeline but painless/pipeline and backslashs just do no work very well  
e.g. this fails and any alteratives (for example using "\")

```auto
     "source": "ctx.msg = ctx.msg.replace('\\', 'anything)"

```

no point in supplying exact full logs and yml as have reproduced this using console \_simulate with very basic details as above

```auto
POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "pipeline to test newline",
    "processors": [
      {
        "grok" : {
                "field" : "message",
                        "ignore_missing" : true,
                        "patterns" : [
                                "%{TIMESTAMP_ISO8601:timestamp} %{GREEDYDATA:msg}"
                        ]
                }
        , "script":
        {
            "lang": "painless",
            "inline": "ctx.msg = ctx.msg.replace('\\', 'anything')"
        }

    }
    ]
    }
    , "docs:" [
            {
                    "_source" : { "message" : """2020-04-23 11:33:00.0000 TEST message\nnextline""" }
            {
    ]
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 21, 2020, 10:45am UTC](https://discuss.elastic.co/t/multiline-txt-log-files-not-parsing-correctly-in-ingest-pipeline/229440/2 "2020-05-21T10:45:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.