# Filebeat multiline timeout

**URL:** <https://discuss.elastic.co/t/filebeat-multiline-timeout/110946>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 9, 2017, 10:27pm UTC](https://discuss.elastic.co/t/filebeat-multiline-timeout/110946 "2017-12-09T22:27:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shreya\_Chatterjee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreya_chatterjee/32/24948_2.png) [@Shreya\_Chatterjee](https://discuss.elastic.co/u/Shreya_Chatterjee)\
**Post date:** [December 9, 2017, 10:27pm UTC](https://discuss.elastic.co/t/filebeat-multiline-timeout/110946/1 "2017-12-09T22:27:58Z")

</div>

excerpt from log file:

```
2017-11-06 11:54:43,333; [LOG_LEVEL=ALWAYS, CMPNT_NM=com.fmr.ifeb.alayer.cache.ehcache.ApplicationCacheImpl, MESSAGE=Initialized cache named 'oscarJdbcDaxCache']
2017-11-06 11:54:43,344; [LOG_LEVEL=ALWAYS, CMPNT_NM=com.fmr.ifeb.alayer.cache.ehcache.ApplicationCacheImpl, MESSAGE=Initialized cache named 'beneaggrJdbcDaxCache']
2017-11-06 11:54:53,509; [LOG_LEVEL=ALWAYS, CMPNT_NM=com.fmr.ifeb.alayer.cache.ehcache.ApplicationCacheImpl, MESSAGE=Initialized cache named 'footerDaxCache']

```

Filebeat.yml

```
filebeat.prospectors:

- input_type: log
  paths:
    - C:\Users\a617744\NewData11\data8.log 
  document_type: springlog

 #tail_files: true
  multiline.pattern: '^\[[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 500
  multiline.timeout: 1m
  backoff: 5m
  max_backoff: 10m

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3

output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

logging.level: debug

```

logstash config file:

```
input {
	 beats {
        port => 5044
  }
}

filter {
  mutate{
		gsub=>["message","\r",""]
	}
	grok {
  
	id => "myspringlogfilter"  
	match => { "message" => ["%{TIMESTAMP_ISO8601:timestamp}; \[LOG_LEVEL=%{WORD:log-level}, CMPNT_NM=%{GREEDYDATA:component_name}, MESSAGE=%{GREEDYDATA:message}\]"]} 
	overwrite => ["message"]
	}
}
output {
	elasticsearch {
		hosts => "localhost:9200" 
		#manage_template => false
    	index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
		#index => "filebeat"
   		document_type => "%{[@metadata][type]}" 	
	}
	stdout {
         codec => rubydebug
  }
}

```

logstaconfig file:

```
input {
	 beats {
        port => 5044
  }
}

filter {
  mutate{
		gsub=>["message","\r",""]
	}
	grok {
  
	id => "myspringlogfilter"  
	match => { "message" => ["%{TIMESTAMP_ISO8601:timestamp}; \[LOG_LEVEL=%{WORD:log-level}, CMPNT_NM=%{GREEDYDATA:component_name}, MESSAGE=%{GREEDYDATA:message}\]"]} 
	overwrite => ["message"]
	}
}
output {
	elasticsearch {
		hosts => "localhost:9200" 
		#manage_template => false
    	index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
		#index => "filebeat"
   		document_type => "%{[@metadata][type]}" 	
	}
	stdout {
         codec => rubydebug
  }
}

```

filebeat log:

```
2017-12-10T03:49:47+05:30 INFO Stopping 1 prospectors
2017-12-10T03:49:47+05:30 INFO Dynamic config reloader stopped
2017-12-10T03:49:47+05:30 INFO Prospector ticker stopped
2017-12-10T03:49:47+05:30 INFO Stopping Prospector: 500517288269846499
2017-12-10T03:49:47+05:30 DBG Multiline event flushed because timeout reached.

```

output looks like:

```
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 3,
    "successful": 3,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "filebeat-6.0.0-2017.12.09",
        "_type": "doc",
        "_id": "99xYPWABfws4q82EV5qr",
        "_score": 1,
        "_source": {
          "@timestamp": "2017-12-09T22:13:46.070Z",
          "offset": 329,
          "log-level": "ALWAYS",
          "component_name": "com.fmr.ifeb.alayer.cache.ehcache.ApplicationCacheImpl, MESSAGE=Initialized cache named 'oscarJdbcDaxCache']\n2017-11-06 11:54:43,344; [LOG_LEVEL=ALWAYS, CMPNT_NM=com.fmr.ifeb.alayer.cache.ehcache.ApplicationCacheImpl",
          "@version": "1",
          "beat": {
            "name": "PCCV050921",
            "hostname": "PCCV050921",
            "version": "6.0.0"
          },
          "host": "INDV050921",
          "source": """C:\Users\a617744\NewData11\data8.log""",
          "message": "Initialized cache named 'beneaggrJdbcDaxCache'",
          "tags": [
            "beats_input_codec_plain_applied"
          ],
          "timestamp": "2017-11-06 11:54:43,333"
        }
      }
    ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [December 11, 2017, 1:01pm UTC](https://discuss.elastic.co/t/filebeat-multiline-timeout/110946/2 "2017-12-11T13:01:39Z")

</div>

Your sample logs does not seem multiline to me. Could you provide multiline samples?

Also, in you attached output `component_name` does not seem right. I am guessing that you only want to put `com.fmr.ifeb.alayer.cache.ehcache.ApplicationCacheImpl` into the field. It happens because you have `CMPNT_NM=%{GREEDYDATA:component_name},` in your logstash config. `GREEDYDATA` mathes eveything until EOL. You would need to use `JAVACLASS` or `NOTSPACE`. Otherwise your message would be added to the `component_name`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 8, 2018, 1:01pm UTC](https://discuss.elastic.co/t/filebeat-multiline-timeout/110946/3 "2018-01-08T13:01:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.