# Multiline Codec: Message mismatch between ES and ruby plugin logger

**URL:** https://discuss.elastic.co/t/multiline-codec-message-mismatch-between-es-and-ruby-plugin-logger/170194
**Category:** Logstash
**Created:** [February 27, 2019, 3:49pm UTC](https://discuss.elastic.co/t/multiline-codec-message-mismatch-between-es-and-ruby-plugin-logger/170194 "2019-02-27T15:49:18Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)
#### Post date: [February 27, 2019, 3:49pm UTC](https://discuss.elastic.co/t/multiline-codec-message-mismatch-between-es-and-ruby-plugin-logger/170194/1 "2019-02-27T15:49:18Z")

</div>

Hi all,  
I'm processing some NMAP XML output using multiline input plugin and XML filter plugin.  
And either I'm doing something wrong or an strange behavior is taking place.  
In order to check if multiline codec were splinting messages correctly I put

```auto
            ruby {
                    code => "
                            logger.info('new_msg')
                            logger.info(event.get('message'))
                    "
            }

```

in my LS configuration.  
As a result I get the message not printed completed. Instead of starting as `<host starttime="1551275965" endtime="1551275986"><status state="up" reason="reset" reason_ttl="63"/>`) I got

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/b/9bae1065625d2df4e1e652ab6997d3e561b3e0fc.jpeg)

but when I go the the index, message is ok.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/2/c214604ca81f6c0334fb9c1d49aceed4699a6ff3.png)  
What I'm doing wrong?  
Below my entire LS configuration  
Thank you!  
Regards  
Ana

```auto
input {
  file {
    path => "/data/xml_test/xml_sample_3.xml"
    sincedb_path => "/dev/null"
    start_position => "beginning"
        codec => multiline {
      pattern => "^<host starttime="
      auto_flush_interval => 1
      negate => "true"
      what => "previous"
      max_lines => 1000000000
     max_bytes => "50 MiB"

    }
      tags => "xml_test_nmap"
      type => "xml_test_nmap"
  }
}

filter {

                ruby {
                        code => "
                                logger.info('new_msg')
                                logger.info(event.get('message'))
                        "
                }

                xml { source => "message"
                        target => poc
                        xpath => ["host/@name", hostname]

                }
                split {
                        field => "[poc][ports][0][port]"
                }

}

output {
         elasticsearch {
        hosts => ["localhost:9200"]
        index => "xml_test_nmap_2"
      }
      stdout { codec => rubydebug }
}

```

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [February 27, 2019, 7:39pm UTC](https://discuss.elastic.co/t/multiline-codec-message-mismatch-between-es-and-ruby-plugin-logger/170194/2 "2019-02-27T19:39:02Z")

</div>

It looks like the default `log4j.properties` configuration for Logstash only outputs the last 1000 bytes of the given message; IIRC, this was a safety measure to prevent accidental flooding of disk:

> ```auto
> appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %-.10000m%n
> 
> ```
> 
> -- [https://github.com/elastic/logstash/blob/master/config/log4j2.properties#L24](https://github.com/elastic/logstash/blob/master/config/log4j2.properties#L24)

---

<div class="post-metadata">

### Author: ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)
#### Post date: [February 28, 2019, 9:44am UTC](https://discuss.elastic.co/t/multiline-codec-message-mismatch-between-es-and-ruby-plugin-logger/170194/3 "2019-02-28T09:44:44Z")

</div>

Hi Ry,  
Many thanks! It was that setting  
Regards  
Ana

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 28, 2019, 9:44am UTC](https://discuss.elastic.co/t/multiline-codec-message-mismatch-between-es-and-ruby-plugin-logger/170194/4 "2019-03-28T09:44:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.