# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=649 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 650 --- ## [Collecting =\> parsing ansible outputs into elasticsearch](https://discuss.elastic.co/t/collecting-parsing-ansible-outputs-into-elasticsearch/156622)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 0\ **Last updated:** [November 14, 2018, 9:47am UTC](https://discuss.elastic.co/t/collecting-parsing-ansible-outputs-into-elasticsearch/156622 "2018-11-14T09:47:06Z")
Dear community, my colleague collects ansible outputs (in case of errors) 1:1 into elasticsearch. Does someone have / share some valid parsing config for logstash for this kind of source so that the results in Kibana g… --- ## [Logstash creates pipeline but index is not created](https://discuss.elastic.co/t/logstash-creates-pipeline-but-index-is-not-created/156620)
**Author:** [@rishabh1](https://discuss.elastic.co/u/rishabh1)\ **Replies:** 0\ **Last updated:** [November 14, 2018, 9:41am UTC](https://discuss.elastic.co/t/logstash-creates-pipeline-but-index-is-not-created/156620 "2018-11-14T09:41:57Z")
I am trying to create an index on elasticsearch cloud using a json file. I have created the configuration as given below: input { file { path =\> \["/root/leads.json"\] start\_positio… --- ## [Unable to use multiple output plugin](https://discuss.elastic.co/t/unable-to-use-multiple-output-plugin/156618)
**Author:** [@ravikt](https://discuss.elastic.co/u/ravikt)\ **Replies:** 0\ **Last updated:** [November 14, 2018, 9:39am UTC](https://discuss.elastic.co/t/unable-to-use-multiple-output-plugin/156618 "2018-11-14T09:39:23Z")
I am trying to send data from cloudwatch logs to both amazon elasticsearch and S3 bucket from the same logstash instance. I see data going to amazon ES but not to S3 bucket. Below is the configuration I am using for logs… --- ## [Run curl/shell script with arguments in Logstash config file](https://discuss.elastic.co/t/run-curl-shell-script-with-arguments-in-logstash-config-file/156599)
**Author:** [@dx123](https://discuss.elastic.co/u/dx123)\ **Replies:** 0\ **Last updated:** [November 14, 2018, 8:04am UTC](https://discuss.elastic.co/t/run-curl-shell-script-with-arguments-in-logstash-config-file/156599 "2018-11-14T08:04:52Z")
Here I have a bash shell script containing a curl command that runs an Elasticsearch query. curl -XGET "http://localhost:9200/index\_name/\_search?" -H 'Content-Type: application/json' -d' { "size": 0, "aggs": { "… --- ## [Grok Pattern for Specific words in side message field](https://discuss.elastic.co/t/grok-pattern-for-specific-words-in-side-message-field/156414)
**Author:** [@satyaban](https://discuss.elastic.co/u/satyaban)\ **Replies:** 1\ **Last updated:** [November 14, 2018, 4:34am UTC](https://discuss.elastic.co/t/grok-pattern-for-specific-words-in-side-message-field/156414 "2018-11-14T04:34:11Z")
I have some log file having pattern like Platform = iOS || iOSRegdID = 1034 || Status = Login || LoggedIn successfully Platform = Android || AndroidRegdID = 1024 || Status = Login || LoggedIn successfully Platform =… --- ## [Logstash-s3-plugin not importing data or creating sincedb file](https://discuss.elastic.co/t/logstash-s3-plugin-not-importing-data-or-creating-sincedb-file/155245)
**Author:** [@tom.isaacson](https://discuss.elastic.co/u/tom.isaacson)\ **Replies:** 1\ **Last updated:** [November 14, 2018, 4:16am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-not-importing-data-or-creating-sincedb-file/155245 "2018-11-14T04:16:13Z")
I have an S3 bucket containing logs from other S3 buckets showing downloads, etc. I pulled Docker then installed S3 input plugin | Logstash Reference \[8.11\] | Elastic. I created a logstast.conf file: input { s3 { … --- ## [HIVE Jdbc input](https://discuss.elastic.co/t/hive-jdbc-input/156574)
**Author:** [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\ **Replies:** 0\ **Last updated:** [November 14, 2018, 4:06am UTC](https://discuss.elastic.co/t/hive-jdbc-input/156574 "2018-11-14T04:06:38Z")
Can we use jdbc input plug in to connect to Hive? I haven't seen any official documentation. Please provide a sample config if it could be done. What jars would be used? I don't want to go down the route to create exter… --- ## [Logstash Cloudwatch input plugin not creating the index on Kibana and Error log\_group in tail](https://discuss.elastic.co/t/logstash-cloudwatch-input-plugin-not-creating-the-index-on-kibana-and-error-log-group-in-tail/156557)
**Author:** [@Shahid\_Chaudhary](https://discuss.elastic.co/u/Shahid_Chaudhary)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 10:31pm UTC](https://discuss.elastic.co/t/logstash-cloudwatch-input-plugin-not-creating-the-index-on-kibana-and-error-log-group-in-tail/156557 "2018-11-13T22:31:55Z")
Hi Teach Team, I have one ELK single node server installed on the EC2 instance. Now i need to get logs/metrics from the Cloudwatch and other AWS manage services , did these steps. 1 - ELK stack is installed and workin… --- ## [Logstash-syslog-output failed to load plugin code](https://discuss.elastic.co/t/logstash-syslog-output-failed-to-load-plugin-code/156552)
**Author:** [@chris\_a.k](https://discuss.elastic.co/u/chris_a.k)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 10:12pm UTC](https://discuss.elastic.co/t/logstash-syslog-output-failed-to-load-plugin-code/156552 "2018-11-13T22:12:56Z")
Hello, I am a newbie trying to write standard output into my syslog server, I am having issues to load Syslog output plugin code. Find below the error received: \[2018-11-14T01:07:00,302\]\[WARN \]\[logstash.config.source.m… --- ## [Cant get multiple logstash pipelines to work](https://discuss.elastic.co/t/cant-get-multiple-logstash-pipelines-to-work/156530)
**Author:** [@tlenya](https://discuss.elastic.co/u/tlenya)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 6:56pm UTC](https://discuss.elastic.co/t/cant-get-multiple-logstash-pipelines-to-work/156530 "2018-11-13T18:56:32Z")
Here is a snippet from my pipeline.yml pipeline.id: my\_csv config.string: "input { generator {} } filter { sleep { time =\> 1 } } output { stdout { codec =\> dots } }" path.config: "D:\\ELK\\logstash-6.4.2\\config\*.conf" … --- ## [Custom grok pattern/regex help](https://discuss.elastic.co/t/custom-grok-pattern-regex-help/155636)
**Author:** [@jcartwright](https://discuss.elastic.co/u/jcartwright)\ **Replies:** 0\ **Last updated:** [November 7, 2018, 12:03am UTC](https://discuss.elastic.co/t/custom-grok-pattern-regex-help/155636 "2018-11-07T00:03:50Z")
Hello All, I have an Apache log where the clientip field will occasionally be just "." rather than a IP address or hostname. This is causing the HOSTNAME pattern to fail and the grok to fail to parse the message. So m… --- ## [S3-plugin logstash "plugin had an unrecoverable error. Will restart this plugin."](https://discuss.elastic.co/t/s3-plugin-logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/156526)
**Author:** [@rpendela](https://discuss.elastic.co/u/rpendela)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 6:14pm UTC](https://discuss.elastic.co/t/s3-plugin-logstash-plugin-had-an-unrecoverable-error-will-restart-this-plugin/156526 "2018-11-13T18:14:08Z")
I am using s3-input plugin to process the logs to logstash, I am receiving some logs of logfiles from s3 but not all of them. As far as I know, my configuration is correct because I am receiving some logs but something … --- ## [Trouble getting XML into usable format](https://discuss.elastic.co/t/trouble-getting-xml-into-usable-format/153009)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 8\ **Last updated:** [November 13, 2018, 5:17pm UTC](https://discuss.elastic.co/t/trouble-getting-xml-into-usable-format/153009 "2018-11-13T17:17:57Z")
Hello, I am not sure if this would be better for Logstash or Beats, but I am using filebeat to send an XML file of windows events. Some of the data looks neat and can be pretty easy to implement a KV with a value split o… --- ## [S3 Output - Minio Connection Error](https://discuss.elastic.co/t/s3-output-minio-connection-error/156360)
**Author:** [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Replies:** 2\ **Last updated:** [November 13, 2018, 5:14pm UTC](https://discuss.elastic.co/t/s3-output-minio-connection-error/156360 "2018-11-13T17:14:57Z")
Hi! We're trying to output to Minio via the S3 output plugin. We specify the name of the server via endpoint: and we're getting this error: \[ERROR\]\[logstash.outputs.s3 \] Uploading failed, retrying. {:exception=\>S… --- ## [Parsing logs with stack trace](https://discuss.elastic.co/t/parsing-logs-with-stack-trace/156514)
**Author:** [@vvn84](https://discuss.elastic.co/u/vvn84)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 4:39pm UTC](https://discuss.elastic.co/t/parsing-logs-with-stack-trace/156514 "2018-11-13T16:39:00Z")
Hi, I have a situation where I am parsing logs that usually are in a structured format and am able to parse it and index them. But when there is an error, there is stacktrace which spans multiple lines and hence the gro… --- ## [Logstash geoip filter error: \_geoip\_lookup\_failure](https://discuss.elastic.co/t/logstash-geoip-filter-error-geoip-lookup-failure/156484)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 1:49pm UTC](https://discuss.elastic.co/t/logstash-geoip-filter-error-geoip-lookup-failure/156484 "2018-11-13T13:49:36Z")
I'm trying to use a custom MaxMind private IP .mmdb geoip2 database with Logstash. My config is pretty simple, and I'm trying to decode netflow: filter{ geoip { database =\> "/usr/share/logstash/geolite2/tes… --- ## [How can i get the hostname of linux server using logstash](https://discuss.elastic.co/t/how-can-i-get-the-hostname-of-linux-server-using-logstash/156480)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 1\ **Last updated:** [November 13, 2018, 1:44pm UTC](https://discuss.elastic.co/t/how-can-i-get-the-hostname-of-linux-server-using-logstash/156480 "2018-11-13T13:44:59Z")
HI, My logstash is running on 2 different linux server and i want to find which logstash server logs are getting into index by adding hostname field. I tried with below ruby code and getting error on that. Anyone can he… --- ## [How many connection can Logstash handle:](https://discuss.elastic.co/t/how-many-connection-can-logstash-handle/154827)
**Author:** [@somu\_p](https://discuss.elastic.co/u/somu_p)\ **Replies:** 5\ **Last updated:** [November 13, 2018, 1:15pm UTC](https://discuss.elastic.co/t/how-many-connection-can-logstash-handle/154827 "2018-11-13T13:15:34Z")
in my environment i want to know how many connection can one logstash handle. i will have 25000 machine running on beats. can all those 25000 contact logstash for processing then store data to elastic search. --- ## [Dissect Filter - Catching a sub field](https://discuss.elastic.co/t/dissect-filter-catching-a-sub-field/155371)
**Author:** [@vishnuduttpv](https://discuss.elastic.co/u/vishnuduttpv)\ **Replies:** 3\ **Last updated:** [November 13, 2018, 12:39pm UTC](https://discuss.elastic.co/t/dissect-filter-catching-a-sub-field/155371 "2018-11-13T12:39:20Z")
Hi, Following is my logstash configuration, filter { dissect { mapping =\> { '\[exec.stdout\]' =\> "%{host} %{up} %{users} %{load} %{IP} %{Memoryfree} %{MemoryTotal} %{SwapTotal} %{SwapFree} %{Psql\_active} %{psql\_est… --- ## [Logstash Syslog input crashes with a malformed request](https://discuss.elastic.co/t/logstash-syslog-input-crashes-with-a-malformed-request/156449)
**Author:** [@Roberto\_1](https://discuss.elastic.co/u/Roberto_1)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 10:43am UTC](https://discuss.elastic.co/t/logstash-syslog-input-crashes-with-a-malformed-request/156449 "2018-11-13T10:43:20Z")
Hello, I see that my Logstash, set with a simply Syslog input configuration, crash anytime I test the listening port with Netcat. Below my Logstash Input configuration. input { syslog { port =\> 5000 } } output… --- ## [Stuck at "Successfully started Logstash API endpoint {:port=\>9600}"](https://discuss.elastic.co/t/stuck-at-successfully-started-logstash-api-endpoint-port-9600/155023)
**Author:** [@Mustapha\_MJ](https://discuss.elastic.co/u/Mustapha_MJ)\ **Replies:** 7\ **Last updated:** [November 13, 2018, 9:32am UTC](https://discuss.elastic.co/t/stuck-at-successfully-started-logstash-api-endpoint-port-9600/155023 "2018-11-13T09:32:58Z")
Hello guys i get stuck at "Successfully started Logstash API endpoint {:port=\>9600}" while trying to ingest txt log file into elasticsearch here is my config file : input { file { path =\> "C:\\Users\\mustapha\\Desktop\\… --- ## [Logstash configuration geo\_point mapping not apply](https://discuss.elastic.co/t/logstash-configuration-geo-point-mapping-not-apply/156374)
**Author:** [@GibbsKim](https://discuss.elastic.co/u/GibbsKim)\ **Replies:** 4\ **Last updated:** [November 13, 2018, 9:28am UTC](https://discuss.elastic.co/t/logstash-configuration-geo-point-mapping-not-apply/156374 "2018-11-13T09:28:04Z")
hi guys, I'm ELK 6.3.1 user. I have question about geo\_mapping in logstash. I was using logstash .conf file for CSV file and Json template file. (was in 5.2.2) Below is a part of configuration file. input { file … --- ## [How to zip and push the logstash logs to S3 bucket?](https://discuss.elastic.co/t/how-to-zip-and-push-the-logstash-logs-to-s3-bucket/156409)
**Author:** [@arun5635](https://discuss.elastic.co/u/arun5635)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 8:23am UTC](https://discuss.elastic.co/t/how-to-zip-and-push-the-logstash-logs-to-s3-bucket/156409 "2018-11-13T08:23:58Z")
Hi I am very new to the Logstash , I need to zip the docker logs(var/log/logstash/docker.log) and push it to s3 bucket. can anyone please help me how to zip and push the logstash logs? --- ## [How to create a new Date format - GROK](https://discuss.elastic.co/t/how-to-create-a-new-date-format-grok/156401)
**Author:** [@rotem\_lom](https://discuss.elastic.co/u/rotem_lom)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 7:37am UTC](https://discuss.elastic.co/t/how-to-create-a-new-date-format-grok/156401 "2018-11-13T07:37:19Z")
Hi There, Unfortunately I don't find timestamp pattern fits to my log: 2018 Aug 04 04:27:49.756 BLA BLA BLA... So far I using the following pattern to handle this part: %{YEAR} %{MONTH} %{MONTHDAY} %{TIME} It do… --- ## [In HTTP input plugin, curl command is halted when thread value "0 or -1" sets in logstash config file](https://discuss.elastic.co/t/in-http-input-plugin-curl-command-is-halted-when-thread-value-0-or-1-sets-in-logstash-config-file/156387)
**Author:** [@garima](https://discuss.elastic.co/u/garima)\ **Replies:** 1\ **Last updated:** [November 13, 2018, 7:34am UTC](https://discuss.elastic.co/t/in-http-input-plugin-curl-command-is-halted-when-thread-value-0-or-1-sets-in-logstash-config-file/156387 "2018-11-13T07:34:31Z")
i am trying to validate the http request for thread value zero. Execution Steps: Logstash-http.conf input { http { port =\> 8080 threads =\> 0 } } output { stdout { codec =\> rubydebug} } 2. Run logstash by bin… --- ## [Curator for logstash](https://discuss.elastic.co/t/curator-for-logstash/156324)
**Author:** [@Eva](https://discuss.elastic.co/u/Eva)\ **Replies:** 4\ **Last updated:** [November 13, 2018, 6:34am UTC](https://discuss.elastic.co/t/curator-for-logstash/156324 "2018-11-13T06:34:56Z")
Hello there I have query on some of the curator actions, ex: 'Close' action. Would like to know what is the use of it . Not sure if this is the right place to ask about curator, i couldn't find any curator specific foru… --- ## [Can I add metadata from java?](https://discuss.elastic.co/t/can-i-add-metadata-from-java/156391)
**Author:** [@xodn0812](https://discuss.elastic.co/u/xodn0812)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 6:04am UTC](https://discuss.elastic.co/t/can-i-add-metadata-from-java/156391 "2018-11-13T06:04:43Z")
It is a structure that is sent to logstash using logback in java. Is it possible to add metadata to java and filter it in logstash? --- ## [Problems with the Netflow Module - Possible ElasticSearch 6 Regression?](https://discuss.elastic.co/t/problems-with-the-netflow-module-possible-elasticsearch-6-regression/156385)
**Author:** [@Zamiell](https://discuss.elastic.co/u/Zamiell)\ **Replies:** 0\ **Last updated:** [November 13, 2018, 4:32am UTC](https://discuss.elastic.co/t/problems-with-the-netflow-module-possible-elasticsearch-6-regression/156385 "2018-11-13T04:32:48Z")
Hello ElasticSearch community, I'm attempting to to use the Logstash Netflow Module, as documented here: https://www.elastic.co/guide/en/logstash/current/netflow-module.html I have a vanilla CentOS 7 installation with… --- ## [Unable to update data from CSV file to Elasticsearch cluster](https://discuss.elastic.co/t/unable-to-update-data-from-csv-file-to-elasticsearch-cluster/156354)
**Author:** [@rasheed\_shaik](https://discuss.elastic.co/u/rasheed_shaik)\ **Replies:** 0\ **Last updated:** [November 12, 2018, 9:45pm UTC](https://discuss.elastic.co/t/unable-to-update-data-from-csv-file-to-elasticsearch-cluster/156354 "2018-11-12T21:45:27Z")
Hi, I am trying to upload data from CSV file to Elasticsearch, and here is my logstash config file: logstash.conf input { file { path =\> \["/home/ubuntu/logstash/logstash-6.4.3/auto.csv"\] start\_position =\> "beginning… --- ## [Java Illegal State Exception: Logstash stopped processing](https://discuss.elastic.co/t/java-illegal-state-exception-logstash-stopped-processing/156353)
**Author:** [@slightlyjaded](https://discuss.elastic.co/u/slightlyjaded)\ **Replies:** 0\ **Last updated:** [November 12, 2018, 9:42pm UTC](https://discuss.elastic.co/t/java-illegal-state-exception-logstash-stopped-processing/156353 "2018-11-12T21:42:10Z")
Hello, Im trying to use ELK + Elastiflow to analyze netflow and I'm running into an issue with logstash. I followed this tutorial: http://blogs.catapultsystems.com/mdowst/archive/2018/06/18/install-elastiflow-on-ubuntu… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=648) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=650)