# How to feed entire logfile to elasticsearch as a message

**URL:** <https://discuss.elastic.co/t/how-to-feed-entire-logfile-to-elasticsearch-as-a-message/54518>\
**Category:** Logstash\
**Created:** [July 1, 2016, 8:39am UTC](https://discuss.elastic.co/t/how-to-feed-entire-logfile-to-elasticsearch-as-a-message/54518 "2016-07-01T08:39:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sharath3185](https://avatars.discourse-cdn.com/v4/letter/s/2bfe46/32.png) [@sharath3185](https://discuss.elastic.co/u/sharath3185)\
**Post date:** [July 1, 2016, 8:39am UTC](https://discuss.elastic.co/t/how-to-feed-entire-logfile-to-elasticsearch-as-a-message/54518/1 "2016-07-01T08:39:01Z")

</div>

'm trying to use logstash to feed a whole text file as a message into ElasticSearch. I have used multiline filter now as below.

multiline {  
pattern =\> "/.\*./gm"  
negate =\> true  
what =\> "previous"  
}

This works for me. But my concern is:  
The multiline filter document says "This filter has been deprecated in favor of multiline-codec. Multiline filter is not thread-safe and cannot handle messages from multiple streams"

I tried using multiline-codec with no luck.  
Could you please let me know the alternative solution for multiline filter for the above scenario?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 2, 2016, 4:36am UTC](https://discuss.elastic.co/t/how-to-feed-entire-logfile-to-elasticsearch-as-a-message/54518/2 "2016-07-02T04:36:44Z")

</div>

> [@sharath3185](#):
>
> I tried using multiline-codec with no luck.

What do you mean?  
Do you have a config and sample log you can show?

---

<div class="post-metadata">

**Author:** ![sharath3185](https://avatars.discourse-cdn.com/v4/letter/s/2bfe46/32.png) [@sharath3185](https://discuss.elastic.co/u/sharath3185)\
**Post date:** [July 7, 2016, 1:41pm UTC](https://discuss.elastic.co/t/how-to-feed-entire-logfile-to-elasticsearch-as-a-message/54518/3 "2016-07-07T13:41:56Z")

</div>

sorry for the delayed reply.  
input  
{  
file  
{  
codec =\> multiline  
{  
pattern =\> "/.\*./gm"  
negate =\> true  
what =\> "previous"  
}  
path =\> ["path\_for\_logs"]  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
type =\> "LOG"

```
}

```

}

filter  
{  
if [type] == "LOG"{

```
 ruby 
    {
        code => "
        
        event['logfile_path'] = event['path'].strip
        "
    }
 
 mutate
    {
        gsub => ["message", "\n", ""]
        gsub => ["message", "\r", ""]
        add_field => ["logdata", "%{message}"]
        remove_field => ["@version", "path", "host", tags, "message"]
    }

```

}

```
}

```

output  
{  
if [type] == "LOG"  
{  
stdout  
{  
codec =\> rubydebug  
}  
elasticsearch  
{  
#hosts =\> "127.0.0.1:9200"  
#template\_overwrite =\> true  
template\_name =\> "template\_abc"  
manage\_template =\> true  
template =\> "/opt/elasticsearch/config/templates/template\_abc.json"  
hosts =\> "127.0.0.1:9200"  
index =\> "abc"  
document\_type =\> "logsearch"  
document\_id =\> "%{logfile\_path}"  
}  
}  
}

Sample Log:  
This is any unstructured log(literally any file) that i want to index into elasticsearch docuement as a field.

---

<div class="post-metadata">

**Author:** ![sharath3185](https://avatars.discourse-cdn.com/v4/letter/s/2bfe46/32.png) [@sharath3185](https://discuss.elastic.co/u/sharath3185)\
**Post date:** [July 13, 2016, 6:53am UTC](https://discuss.elastic.co/t/how-to-feed-entire-logfile-to-elasticsearch-as-a-message/54518/4 "2016-07-13T06:53:57Z")

</div>

Request help on this ASAP.  
Whether is it possible to use multiline codec to push entire logfile(any file with no specific pattern) as a field to elasticsearch?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:48am UTC](https://discuss.elastic.co/t/how-to-feed-entire-logfile-to-elasticsearch-as-a-message/54518/5 "2017-07-06T04:48:20Z")

</div>