# Codec multiline in output section

**URL:** <https://discuss.elastic.co/t/codec-multiline-in-output-section/196985>\
**Category:** Logstash\
**Created:** [August 27, 2019, 6:02pm UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985 "2019-08-27T18:02:46Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![uma\_rengasamy](https://avatars.discourse-cdn.com/v4/letter/u/e56c9b/32.png) [@uma\_rengasamy](https://discuss.elastic.co/u/uma_rengasamy)\
**Post date:** [August 27, 2019, 6:02pm UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/1 "2019-08-27T18:02:47Z")

</div>

Can we use codec multiline in o/p section with amazon\_es plugin ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 27, 2019, 6:10pm UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/2 "2019-08-27T18:10:26Z")

</div>

Multiline codec is as far as I know for inputs only as any ordering is lost part that point. What is it you are trying to achieve?

---

<div class="post-metadata">

**Author:** ![uma\_rengasamy](https://avatars.discourse-cdn.com/v4/letter/u/e56c9b/32.png) [@uma\_rengasamy](https://discuss.elastic.co/u/uma_rengasamy)\
**Post date:** [August 28, 2019, 4:55am UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/3 "2019-08-28T04:55:36Z")

</div>

1. I cannot use multiline in input section because we have mixed log patterns and input is from s3 with all log pattern in single prefix s3 bucket .  
2.I cannot use filter multiline since its deprecated  
3.I choosed output since based on the field value I have to apply codec .

So in this case is it better to go with GROK?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 28, 2019, 5:03am UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/4 "2019-08-28T05:03:09Z")

</div>

> [@uma\_rengasamy](#):
>
> I cannot use multiline in input section because we have mixed log patterns and input is from s3 with all log pattern in single prefix s3 bucket .

A codec should be applied per file even if your plugin matches many files, so I do not understand why this is not possible. If the log entries that need to be grouped are not consecutive within a file it is not multiline processing you are after.

> [@uma\_rengasamy](#):
>
> I cannot use filter multiline since its deprecated

Correct.

> [@uma\_rengasamy](#):
>
> I choosed output since based on the field value I have to apply codec .

At this point each line has been processed independently in different batches and multiple threads so any ordering has been lost. You can not apply multiline as an output codec.

---

<div class="post-metadata">

**Author:** ![uma\_rengasamy](https://avatars.discourse-cdn.com/v4/letter/u/e56c9b/32.png) [@uma\_rengasamy](https://discuss.elastic.co/u/uma_rengasamy)\
**Post date:** [August 28, 2019, 5:26am UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/5 "2019-08-28T05:26:38Z")

</div>

How can I selectively apply codec in input section , s3 prefix is not possible since it doesnt support regex .

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 28, 2019, 6:21pm UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/6 "2019-08-28T18:21:53Z")

</div>

Can you provide an example of what your logs look like and what you are looking to achieve?

---

<div class="post-metadata">

**Author:** ![uma\_rengasamy](https://avatars.discourse-cdn.com/v4/letter/u/e56c9b/32.png) [@uma\_rengasamy](https://discuss.elastic.co/u/uma_rengasamy)\
**Post date:** [September 5, 2019, 11:00am UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/7 "2019-09-05T11:00:19Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> xample of what your logs look like and

We have different type pf logs in s3 bucket, in that certain logs are multiline , So codec in input should be selective. Since s3 input doesnt support regex for prefix , do we have any alternative approach

---

<div class="post-metadata">

**Author:** ![uma\_rengasamy](https://avatars.discourse-cdn.com/v4/letter/u/e56c9b/32.png) [@uma\_rengasamy](https://discuss.elastic.co/u/uma_rengasamy)\
**Post date:** [September 27, 2019, 8:08am UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/8 "2019-09-27T08:08:43Z")

</div>

> [@uma\_rengasamy](#):
>
> rent type pf logs i

auto\_flush\_interval doesn't seems to work for s3 input. Is this supported for s3 plugin?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 25, 2019, 8:08am UTC](https://discuss.elastic.co/t/codec-multiline-in-output-section/196985/9 "2019-10-25T08:08:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
