# Multiline and Ingest Node

**URL:** <https://discuss.elastic.co/t/multiline-and-ingest-node/58529>\
**Category:** Elasticsearch\
**Created:** [August 21, 2016, 9:57am UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529 "2016-08-21T09:57:04Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\
**Post date:** [August 21, 2016, 9:57am UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/1 "2016-08-21T09:57:04Z")

</div>

Hi,

I am having Filebeat 5 sending multiline events to an Elastic Ingest node, which in turn, runs the grok processor.  
However the grok processor, takes only the first line of the data.  
The end of each line is a stack trace start message, which I would like to have all data, and using GREEDYDATA pattern.

Is it possible to ingest multiline events with the Ingest node ?

Thanks,

Ori

---

<div class="post-metadata">

**Author:** ![talevy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/talevy/32/44896_2.png) [@talevy](https://discuss.elastic.co/u/talevy)\
**Post date:** [August 26, 2016, 6:47pm UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/2 "2016-08-26T18:47:20Z")

</div>

@ori.rubinfeld. just to be sure, you are receiving documents of this nature?

```auto
{
"message": "first line\nsecond line\nthird line"
}

```

---

<div class="post-metadata">

**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\
**Post date:** [August 27, 2016, 6:13am UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/3 "2016-08-27T06:13:32Z")

</div>

Hi,

I am receiving lines like this:

27/08/2016 1:00:23 PM|some text1|some text2|first line  
second line  
third line  
.....

Using the GREEDYDATA at the end, I was able to extract only the first line.

I found a workaround for now, but would like to have another solution:  
I am replacing using gsub all the followings, before the GROK:

\n --\> @@n@@  
\r --\> @@r@@  
\t --\> @@t@@

Then, after extracting the message, I am doing another replace to the original character using again the gsub processor.

@@n@@ --\> \n  
@@r@@ --\> \r  
@@t@@ --\> \t

For now it is working great!!!  
Data is being presented correctly in Kibana.

But I would like to be able just to use the GREEDYDATA like in Logstash to have all the Stack trace, and not adding these uses of gsub processor.

Ori

---

<div class="post-metadata">

**Author:** ![talevy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/talevy/32/44896_2.png) [@talevy](https://discuss.elastic.co/u/talevy)\
**Post date:** [August 27, 2016, 5:12pm UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/4 "2016-08-27T17:12:03Z")

</div>

since GREEDYDATA does not span new-lines, you can introduce a new pattern that does

```auto
{
  "grok" : {
    "field" : "myField",
    "patterns" : ["%{GREEDYMULTILINE:allMyData}"]
    "pattern_definitions" : {
      "GREEDYMULTILINE" : "(.|\n)*"
    }
  }
}

```

This is pretty much a catch-all though, no different than the original value. I do not fully know the goal, so this may or may not help. Regardless, glad you were able to work around it with the help from other processors!

---

<div class="post-metadata">

**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\
**Post date:** [August 27, 2016, 7:36pm UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/5 "2016-08-27T19:36:46Z")

</div>

Thanks a lot!!!!!  
I will test it and update.  
Will take a while, since currently not on it.

Ori

---

<div class="post-metadata">

**Author:** ![ori.rubinfeld](https://avatars.discourse-cdn.com/v4/letter/o/7c8e57/32.png) [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\
**Post date:** [August 27, 2016, 7:38pm UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/6 "2016-08-27T19:38:51Z")

</div>

By the way, with the Logstash, GREEDYDATA takes all lines.  
Why isn't it the same here ?  
Shouldn't it just take the whole string (Including newline characters and carriage return) ?

Ori

---

<div class="post-metadata">

**Author:** ![talevy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/talevy/32/44896_2.png) [@talevy](https://discuss.elastic.co/u/talevy)\
**Post date:** [August 29, 2016, 4:16pm UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/7 "2016-08-29T16:16:14Z")

</div>

I suppose that is how the regex engine treats `.*`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:24pm UTC](https://discuss.elastic.co/t/multiline-and-ingest-node/58529/8 "2017-07-05T22:24:24Z")

</div>