# Start/Stop pattern for multiline

**URL:** <https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 31, 2016, 1:38am UTC](https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861 "2016-03-31T01:38:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![erikstephens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erikstephens/32/5430_2.png) [@erikstephens](https://discuss.elastic.co/u/erikstephens)\
**Post date:** [March 31, 2016, 1:38am UTC](https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861/1 "2016-03-31T01:38:09Z")

</div>

I can't figure out how to use the filebeat multiline functionality to aggregate output with a start & stop pattern:

```auto
2016-03-29T09:45:01 process-A starting
2016-03-29T09:45:02 process-A foo
2016-03-29T09:45:03 process-A bar
2016-03-29T09:45:04 process-A finished
2016-03-29T09:45:05 process-B baz

```

Is that possible with current filebeat (1.2.0)? If not, is that on the road map? Thanks!

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 31, 2016, 12:32pm UTC](https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861/2 "2016-03-31T12:32:06Z")

</div>

start/stop is not supported yet, but has been originally considered. Check [ticket #461](https://github.com/elastic/beats/issues/461).

In your sample, is 'process-B' a single line event? If log file only consists of multiline events, it's possible to workaround this limitation by clever configs.

E.g. if process-A is always multiline but process-B is not one can filter for process-A.

---

<div class="post-metadata">

**Author:** ![erikstephens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erikstephens/32/5430_2.png) [@erikstephens](https://discuss.elastic.co/u/erikstephens)\
**Post date:** [March 31, 2016, 1:35pm UTC](https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861/3 "2016-03-31T13:35:05Z")

</div>

It's syslog, where most lines/msgs stand on their own. process-A is the output from a cron job where individual lines as elasticsearch documents don't make sense (boot msgs might be another example). And a 3rd case is a log line that bursts and doesn't have any interesting per-msg data:

```auto
2016-03-29T09:45:06 process-C some generic error msg
...
2016-03-29T09:45:08 process-C some generic error msg

```

Would be nice to be able to roll those up into a single document and avoid "spamming" the elasticsearch index.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 31, 2016, 9:53pm UTC](https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861/4 "2016-03-31T21:53:55Z")

</div>

sounds like a mumbo-jumbo of everything. One can try to get some multiline support by using the regex OR-operator `|`, but then match order (`before/after`) and `negate` parameter must be same for all multiline patterns.

In general this use-case is not fully supported by multiline yet. Having a look at the actual logs, maybe (only maybe) I could come up with some regex pattern.

One problem with mutliline and all syslog output into one file might be interleaving multiline-events. Even if we were to support start/end patterns, you might still run into more subtle problems. Log routing as supported by some syslog daemons might help untangling the logs before forwarding logs via some shipper.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 31, 2016, 9:55pm UTC](https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861/5 "2016-03-31T21:55:40Z")

</div>

You may want to look at the aggregate filter in LS to handle this instead.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:53pm UTC](https://discuss.elastic.co/t/start-stop-pattern-for-multiline/45861/6 "2017-07-05T21:53:59Z")

</div>


