# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=816 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 817 --- ## [Struggling with netflow](https://discuss.elastic.co/t/struggling-with-netflow/120952)
**Author:** [@tombox](https://discuss.elastic.co/u/tombox)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 11:46pm UTC](https://discuss.elastic.co/t/struggling-with-netflow/120952 "2018-02-21T23:46:39Z")
Really struggling to get netflow going here, fresh centos 7 install not much configured and getting the following which goes beyond my comprehension. I get the below when I run "bin/logstash --modules netflow --setup" … --- ## [\]\[logstash.outputs.elasticsearch\] Encountered a retryable error](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-encountered-a-retryable-error/119320)
**Author:** [@Jose\_Ramierez](https://discuss.elastic.co/u/Jose_Ramierez)\ **Replies:** 1\ **Last updated:** [February 21, 2018, 11:28pm UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-encountered-a-retryable-error/119320 "2018-02-21T23:28:25Z")
I have been fighting this issue off and on for some time. Logstash: 5.6.4 Elastic: 5.6.4 CentOS 7 Logstash filters: "name" : "logstash-output-elasticsearch", "version" : "7.4.2" "name" : "logstash-input-tcp", // … --- ## [Logstash connection to Cassandra](https://discuss.elastic.co/t/logstash-connection-to-cassandra/120739)
**Author:** [@jcoughlin](https://discuss.elastic.co/u/jcoughlin)\ **Replies:** 1\ **Last updated:** [February 21, 2018, 11:02pm UTC](https://discuss.elastic.co/t/logstash-connection-to-cassandra/120739 "2018-02-21T23:02:47Z")
I am trying to access a Cassandra database to connect to Elasticsearch/Kibana. I have the latest versions 6.2.1 of all and installed the logstash-input-jdbc I am getting the following error: \[2018-02-20T16:19:47,915\]\[E… --- ## [Issue with Multiple Web Socket Inputs](https://discuss.elastic.co/t/issue-with-multiple-web-socket-inputs/120945)
**Author:** [@manojporwal](https://discuss.elastic.co/u/manojporwal)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 10:51pm UTC](https://discuss.elastic.co/t/issue-with-multiple-web-socket-inputs/120945 "2018-02-21T22:51:19Z")
Using logstash v6.2.1 I have setup 2 pipelines each getting input using a web socket input plugin. My problem is at a time only one pipeline seems to be making connection to web socket server, meaning data is collected … --- ## [Question about mapping types in 6.X (and next versions)](https://discuss.elastic.co/t/question-about-mapping-types-in-6-x-and-next-versions/120478)
**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 2\ **Last updated:** [February 21, 2018, 9:02pm UTC](https://discuss.elastic.co/t/question-about-mapping-types-in-6-x-and-next-versions/120478 "2018-02-21T21:02:58Z")
Hello, I recently updated from 5.6.2 to 6.2.1 and one the problems I had was related to the mapping types. I was using filebeat to send data and setting the 'type' field in filebeat.yml, in Logstash I use the type valu… --- ## [How can I see what command the logstash container is running?](https://discuss.elastic.co/t/how-can-i-see-what-command-the-logstash-container-is-running/120930)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 3\ **Last updated:** [February 21, 2018, 8:57pm UTC](https://discuss.elastic.co/t/how-can-i-see-what-command-the-logstash-container-is-running/120930 "2018-02-21T20:57:21Z")
https://www.elastic.co/guide/en/logstash/6.2/docker-config.html I know where the container expects pipelines to be but I need to know exactly what logstash commands its running to execute them so I can debug The Docker… --- ## [JMX input plugin install problems](https://discuss.elastic.co/t/jmx-input-plugin-install-problems/120920)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 7:02pm UTC](https://discuss.elastic.co/t/jmx-input-plugin-install-problems/120920 "2018-02-21T19:02:46Z")
I have the JMX plugin running properly on our Windows development env. However, trying to move over to the Linux testing env I cannot get the plugin installed. The plugin install ends up: roacamu03:/opt/app/logstash: bi… --- ## [Logstash Elasticsearch](https://discuss.elastic.co/t/logstash-elasticsearch/120918)
**Author:** [@Ajit\_Kumar1](https://discuss.elastic.co/u/Ajit_Kumar1)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 6:55pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch/120918 "2018-02-21T18:55:44Z")
Hi expert, I have a data like \[ { "user": { "\_id": "5a7bec992e5521389cd4c8d5", "email": "debiprasad199211@gmail.com", "name": "debi prasad das" }, "location": { "id": "ef5aa… --- ## [How to identify/display set of sequence data In the log file](https://discuss.elastic.co/t/how-to-identify-display-set-of-sequence-data-in-the-log-file/120043)
**Author:** [@rajkamalkool6](https://discuss.elastic.co/u/rajkamalkool6)\ **Replies:** 7\ **Last updated:** [February 21, 2018, 6:08pm UTC](https://discuss.elastic.co/t/how-to-identify-display-set-of-sequence-data-in-the-log-file/120043 "2018-02-21T18:08:41Z")
I got one different use case from my client which I need to identify a set of log data which got recorded in sequence at any line in the log file and they want to view the identified sequence in Kibana and its related co… --- ## [Log4net stops passing through to logstash](https://discuss.elastic.co/t/log4net-stops-passing-through-to-logstash/119518)
**Author:** [@peter1992](https://discuss.elastic.co/u/peter1992)\ **Replies:** 3\ **Last updated:** [February 21, 2018, 5:05pm UTC](https://discuss.elastic.co/t/log4net-stops-passing-through-to-logstash/119518 "2018-02-21T17:05:35Z")
We are experiencing a problem where logstash stops collecting data from log4net but there are no errors to indicate why. Has anyone seen this before? --- ## [Limit number of events](https://discuss.elastic.co/t/limit-number-of-events/120898)
**Author:** [@umapada](https://discuss.elastic.co/u/umapada)\ **Replies:** 1\ **Last updated:** [February 21, 2018, 5:03pm UTC](https://discuss.elastic.co/t/limit-number-of-events/120898 "2018-02-21T17:03:16Z")
Is there a way, I can limit the number of events in logstash. e.g. I want filtered 5 events in every one second --- ## [Logstash needs restarting ,once kafka is down and live back again](https://discuss.elastic.co/t/logstash-needs-restarting-once-kafka-is-down-and-live-back-again/120888)
**Author:** [@Pratheek](https://discuss.elastic.co/u/Pratheek)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 4:05pm UTC](https://discuss.elastic.co/t/logstash-needs-restarting-once-kafka-is-down-and-live-back-again/120888 "2018-02-21T16:05:34Z")
Currently we are facing some issues with logstash .When kafka is down and back live after sometime,logstash is not resuming to get data from the kafka topic ,its still throwing some errors and resumes loading data only w… --- ## [How to create a parent using the new join type with logstash](https://discuss.elastic.co/t/how-to-create-a-parent-using-the-new-join-type-with-logstash/119523)
**Author:** [@mmoxam](https://discuss.elastic.co/u/mmoxam)\ **Replies:** 1\ **Last updated:** [February 21, 2018, 3:29pm UTC](https://discuss.elastic.co/t/how-to-create-a-parent-using-the-new-join-type-with-logstash/119523 "2018-02-21T15:29:31Z")
With the introduction of the new join datatype, parent child relationships have been updated to a new format. How can I use Logstash to specify the field that will be used as the join between the parent and the child. D… --- ## [Mapper\_parsing\_exception & failed to parse](https://discuss.elastic.co/t/mapper-parsing-exception-failed-to-parse/120540)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 2\ **Last updated:** [February 21, 2018, 3:43pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-failed-to-parse/120540 "2018-02-21T15:43:15Z")
I'm trying to use Json filter plugin | Logstash Reference: filter { json { skip\_on\_invalid\_json =\> true source =\> "message" } } logstash log - getting bombarded with similar messages: logstash11 | \[2018-0… --- ## [Logstash - Accessing Event Data and Fields in the Configuration - Conditionals](https://discuss.elastic.co/t/logstash-accessing-event-data-and-fields-in-the-configuration-conditionals/120727)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 2\ **Last updated:** [February 21, 2018, 3:36pm UTC](https://discuss.elastic.co/t/logstash-accessing-event-data-and-fields-in-the-configuration-conditionals/120727 "2018-02-21T15:36:24Z")
I'm trying to follow Accessing Event Data and Fields in the Configuration | Logstash Reference \[6.2\] | Elastic and I have a question: output: output { if \[beats\_input\_raw\_event\] in \[tags\] or \[beats\_input\_codec\_plain\_… --- ## [X\_pack basic logstash no longer imports data](https://discuss.elastic.co/t/x-pack-basic-logstash-no-longer-imports-data/120694)
**Author:** [@fug](https://discuss.elastic.co/u/fug)\ **Replies:** 2\ **Last updated:** [February 21, 2018, 3:20pm UTC](https://discuss.elastic.co/t/x-pack-basic-logstash-no-longer-imports-data/120694 "2018-02-21T15:20:14Z")
I am running 6.1.1 across the board, and had been using elastic search with no issues (POC). I wanted csv reporting so installed x\_pack and got the license. Installed no issues and everything seemed to work. However, aft… --- ## [Modules and pipelines - Ignoring the 'pipelines.yml'](https://discuss.elastic.co/t/modules-and-pipelines-ignoring-the-pipelines-yml/120852)
**Author:** [@collse](https://discuss.elastic.co/u/collse)\ **Replies:** 1\ **Last updated:** [February 21, 2018, 2:55pm UTC](https://discuss.elastic.co/t/modules-and-pipelines-ignoring-the-pipelines-yml/120852 "2018-02-21T14:55:17Z")
Hi, still trying to get a grip on this but I want to use the netflow plugin of logstash but surely at the same time not sacrifice my other configuration but I cant find any information if this is even possible or how th… --- ## [Group apache org.apache.http request into a single object](https://discuss.elastic.co/t/group-apache-org-apache-http-request-into-a-single-object/120684)
**Author:** [@shacoshe](https://discuss.elastic.co/u/shacoshe)\ **Replies:** 5\ **Last updated:** [February 21, 2018, 1:34pm UTC](https://discuss.elastic.co/t/group-apache-org-apache-http-request-into-a-single-object/120684 "2018-02-21T13:34:40Z")
hello all :slight\_smile: i have a log file the look like this \[DEBUG\]\[DefaultHttpClient\]\[2018-02-20 18:29:50,348\] Attempt 1 to execute request \[DEBUG\]\[DefaultClientConnection\]\[2018-02-20 18:29:50,348\] Sending request… --- ## [Logstash error for the multiline json file](https://discuss.elastic.co/t/logstash-error-for-the-multiline-json-file/120474)
**Author:** [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\ **Replies:** 8\ **Last updated:** [February 21, 2018, 12:13pm UTC](https://discuss.elastic.co/t/logstash-error-for-the-multiline-json-file/120474 "2018-02-21T12:13:52Z")
I am getting below logstash error for the multiline json file. Error: \[2018-02-19T19:59:01,332\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :e… --- ## [Increase Logstash Java heap-size on Ubuntu 16.10](https://discuss.elastic.co/t/increase-logstash-java-heap-size-on-ubuntu-16-10/120825)
**Author:** [@Ryan\_Grannell](https://discuss.elastic.co/u/Ryan_Grannell)\ **Replies:** 1\ **Last updated:** [February 21, 2018, 11:35am UTC](https://discuss.elastic.co/t/increase-logstash-java-heap-size-on-ubuntu-16-10/120825 "2018-02-21T11:35:48Z")
Hi, We're currently running into problems where our Logstash VM is hitting the Java heap-size limit, despite having a large amount of free RAM on the VM. \[image\] {"level":"ERROR","loggerName":"logstash.filters.grok","… --- ## [Grok on source field](https://discuss.elastic.co/t/grok-on-source-field/120800)
**Author:** [@Dykan\_P](https://discuss.elastic.co/u/Dykan_P)\ **Replies:** 2\ **Last updated:** [February 21, 2018, 10:33am UTC](https://discuss.elastic.co/t/grok-on-source-field/120800 "2018-02-21T10:33:21Z")
Hello, I'm beginner on stack ELK 5.6 and I have a problem : I try to recover my virtualhost and put it in a new field but my grok is false. The data I want is in source but she has different form : /var/www/clients/cl… --- ## [Alternate to document\_type](https://discuss.elastic.co/t/alternate-to-document-type/120813)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 10:33am UTC](https://discuss.elastic.co/t/alternate-to-document-type/120813 "2018-02-21T10:33:06Z")
Since document\_type is going to be deprecated what is the alternate to it which we can use --- ## [Parsing Log4j](https://discuss.elastic.co/t/parsing-log4j/120779)
**Author:** [@Eswar\_Kumar\_Musiboin](https://discuss.elastic.co/u/Eswar_Kumar_Musiboin)\ **Replies:** 3\ **Last updated:** [February 21, 2018, 10:13am UTC](https://discuss.elastic.co/t/parsing-log4j/120779 "2018-02-21T10:13:47Z")
To parse log4j logs in the logstash .i need a standard configuration file using different filter plugins. if anybody have the file please help me out --- ## [Log stash filter for Nagios service-performdata](https://discuss.elastic.co/t/log-stash-filter-for-nagios-service-performdata/120609)
**Author:** [@charan.gandra](https://discuss.elastic.co/u/charan.gandra)\ **Replies:** 3\ **Last updated:** [February 21, 2018, 10:10am UTC](https://discuss.elastic.co/t/log-stash-filter-for-nagios-service-performdata/120609 "2018-02-21T10:10:44Z")
Hi, My Nagios service-perfdata is in below format and I am using filebeat to ship this log to Logstash. DATATYPE::SERVICEPERFDATA TIMET::1519123537 HOSTNAME::DXCSRVMON11014 SERVICEDESC::Zombie Proces… --- ## [Grouping data by id with aggregate](https://discuss.elastic.co/t/grouping-data-by-id-with-aggregate/120803)
**Author:** [@sabdoul](https://discuss.elastic.co/u/sabdoul)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 9:51am UTC](https://discuss.elastic.co/t/grouping-data-by-id-with-aggregate/120803 "2018-02-21T09:51:59Z")
Hello, I want to group software by an id (pc name). For that I use the aggregate filter. But the grouping does not work. Maybe I miss something (below my code): input { elasticsearch { hosts =\> \["localhost:9200"\] … --- ## [How to make a custom grok filter](https://discuss.elastic.co/t/how-to-make-a-custom-grok-filter/120765)
**Author:** [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\ **Replies:** 11\ **Last updated:** [February 21, 2018, 9:17am UTC](https://discuss.elastic.co/t/how-to-make-a-custom-grok-filter/120765 "2018-02-21T09:17:35Z")
Hello I have to create a custom grok filter for my logs so please let me know the procedure to do that as in the pattern which I want where it should be saved the extension of the file in which i write my custom pattern… --- ## [Decode a asn1 encoded string](https://discuss.elastic.co/t/decode-a-asn1-encoded-string/120776)
**Author:** [@shubham\_kumar1](https://discuss.elastic.co/u/shubham_kumar1)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 7:17am UTC](https://discuss.elastic.co/t/decode-a-asn1-encoded-string/120776 "2018-02-21T07:17:58Z")
Hi all, I am parsing a xml log file, inside that i have field which show encoded data, that data is encoded in asn1. I want to decode that field and then parse it to elasticsearch to show the decoded value as an event. … --- ## [Requirement to clear the index data and load fresh data](https://discuss.elastic.co/t/requirement-to-clear-the-index-data-and-load-fresh-data/120768)
**Author:** [@ranganath\_nangineni](https://discuss.elastic.co/u/ranganath_nangineni)\ **Replies:** 3\ **Last updated:** [February 21, 2018, 6:39am UTC](https://discuss.elastic.co/t/requirement-to-clear-the-index-data-and-load-fresh-data/120768 "2018-02-21T06:39:00Z")
I have a requirement to clear the existing index data and load the data into the same index whenever we run the logstash config. How can we achieve this through the config file ? --- ## [Logstash Elasticsearch Output plugin Error](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-error/120766)
**Author:** [@Amit\_Hora](https://discuss.elastic.co/u/Amit_Hora)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 6:12am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-error/120766 "2018-02-21T06:12:32Z")
Hi, Everything was working fine, i updated my logstash configuration to add one more index in output. But now i am seeing below log messages, though they are not very frequent. \[ERROR\]\[logstash.outputs.elasticsearch\] E… --- ## [Sensitive information parsing](https://discuss.elastic.co/t/sensitive-information-parsing/120685)
**Author:** [@mohank44](https://discuss.elastic.co/u/mohank44)\ **Replies:** 3\ **Last updated:** [February 21, 2018, 5:33am UTC](https://discuss.elastic.co/t/sensitive-information-parsing/120685 "2018-02-21T05:33:32Z")
Hello All, I am trying to mask a sensitive information from a log message. I have given the sample log file and the config file as below. i was trying to do this using gsub with no success. Please review and advise at … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=815) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=817)