# Multiline negation

**URL:** https://discuss.elastic.co/t/multiline-negation/55447
**Category:** Logstash
**Created:** [July 13, 2016, 10:03pm UTC](https://discuss.elastic.co/t/multiline-negation/55447 "2016-07-13T22:03:24Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Mormaii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mormaii/32/10790_2.png) [@Mormaii](https://discuss.elastic.co/u/Mormaii)
#### Post date: [July 13, 2016, 10:03pm UTC](https://discuss.elastic.co/t/multiline-negation/55447/1 "2016-07-13T22:03:24Z")

</div>

This is the data I have (this is one entry each).

```
"MAIN_cache_hit" => 1472612747
"MAIN_client_req" => 1524049030
"MAIN_fetch_1xx" => 0
"MAIN_fetch_204" => 0

```

I'm trying to group MAIN\_cache\_hit and MAIN\_client\_req on a single logstash entry and leave the rest how they are. However this is grouping all of the fields together instead of creating one event with two fields and then one event per field. What am I doing wrong?

```
    multiline {
        pattern => "^MAIN_(cache|client)_(\w+)\z"
        negate => "true"
        what => "next"
    }
```

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 14, 2016, 6:17am UTC](https://discuss.elastic.co/t/multiline-negation/55447/2 "2016-07-14T06:17:37Z")

</div>

I think you're better off using the aggregate filter.

---

<div class="post-metadata">

### Author: ![Mormaii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mormaii/32/10790_2.png) [@Mormaii](https://discuss.elastic.co/u/Mormaii)
#### Post date: [July 14, 2016, 9:34pm UTC](https://discuss.elastic.co/t/multiline-negation/55447/3 "2016-07-14T21:34:30Z")

</div>

I don't understand why the multiline isn't working. This is what my data looks like:

[Data](http://puu.sh/q1QxV/7f19f0510d.png)

```
    multiline {
        pattern => "^MAIN"
        what => "next"
    }

```

I just want to merge all events together that start with MAIN since I want to do math operations between them and I can't if they're separate events.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 15, 2016, 5:32am UTC](https://discuss.elastic.co/t/multiline-negation/55447/4 "2016-07-15T05:32:17Z")

</div>

What does your input plugin (where you use the multiline filter) look like? Codecs receive a payload in the form of a string and applies the pattern to that string. In the examples of events you showed (for some odd reason as a screenshot; please use copy/paste next time) it's not clear what the original payload looks like.

---

<div class="post-metadata">

### Author: ![Mormaii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mormaii/32/10790_2.png) [@Mormaii](https://discuss.elastic.co/u/Mormaii)
#### Post date: [July 15, 2016, 5:55am UTC](https://discuss.elastic.co/t/multiline-negation/55447/5 "2016-07-15T05:55:09Z")

</div>

I'm using the multiline plugin in the **filter** not on the input. The input looks like this:

```
input {
    redis {
        host => "127.0.0.1"
        port => "6379"
        data_type => "list"
        key => "filebeat"
    }
}

```

I posted the data as a screenshot because I didn't want to create a long post. And here is the data I receive (there's two types of them, I want to merge **one type together to operate between events** )

```
{
    "@timestamp" => "2016-07-15T05:52:40.402Z",
          "type" => "vc_server",
     "parameter" => "/diario/hoy/um/sumariorss_xml",
         "value" => 5,
      "hostname" => "agslx-hpclavc05"
}
{
    "@timestamp" => "2016-07-15T05:52:40.402Z",
          "type" => "vc_server",
     "parameter" => "/destacados/externos/dx_zonales_2_html",
         "value" => 4,
      "hostname" => "agslx-hpclavc05"
}
{
         "@timestamp" => "2016-07-15T05:52:40.402Z",
               "type" => "vc_server",
           "hostname" => "agslx-hpclavc05",
    "MAIN_client_req" => 1695530932,
               "tags" => [
        [0] "varnishstat"
    ]
}
{
        "@timestamp" => "2016-07-15T05:52:40.402Z",
              "type" => "vc_server",
          "hostname" => "agslx-hpclavc05",
    "MAIN_cache_hit" => 1637393101,
              "tags" => [
        [0] "varnishstat"
    ]
}

```

The goal is to group the **MAIN** one togethers since I want to divide **MAIN\_cache\_hit** by **MAIN\_client\_req**.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 15, 2016, 5:59am UTC](https://discuss.elastic.co/t/multiline-negation/55447/6 "2016-07-15T05:59:33Z")

</div>

How do you know that the two MAIN events will always be consecutive with nothing else inbetween?

Either way, the multiline filter acts on the values of fields. I don't think you can get it to merge events based on the presence of certain fields.

Again, I think you should use the aggregate filter.

---

<div class="post-metadata">

### Author: ![Mormaii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mormaii/32/10790_2.png) [@Mormaii](https://discuss.elastic.co/u/Mormaii)
#### Post date: [July 15, 2016, 6:16am UTC](https://discuss.elastic.co/t/multiline-negation/55447/7 "2016-07-15T06:16:45Z")

</div>

You're completely right, I was thinking of how I'm setting it up right now and I know the order of the logs I'm sending from one server but when I start scaling horizontally then there's no way to determine the two events will arrive together. I'll look into the aggregate filter and see how can I do it.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:47am UTC](https://discuss.elastic.co/t/multiline-negation/55447/8 "2017-07-06T04:47:55Z")

</div>


