# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=862 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 863 --- ## [Ping input plugin for logstash](https://discuss.elastic.co/t/ping-input-plugin-for-logstash/107717)
**Author:** [@davidecavestro](https://discuss.elastic.co/u/davidecavestro)\ **Replies:** 0\ **Last updated:** [November 15, 2017, 10:12am UTC](https://discuss.elastic.co/t/ping-input-plugin-for-logstash/107717 "2017-11-15T10:12:30Z")
I've just published logstash-input-ping. It is meant to acquire ping metrics directly from logstash: the same can be done using PingBeat or other stuff... in fact I did it just to learn some ruby :slight\_smile: Howeve… --- ## [Split source value and create a custom field with splitted one](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334)
**Author:** [@platso588](https://discuss.elastic.co/u/platso588)\ **Replies:** 4\ **Last updated:** [December 5, 2017, 1:37pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334 "2017-12-05T13:37:54Z")
Hi All, I have merged few Yarn logs in elasticsearch and in Kibana. source=/data06/yarn/logs/application\_1510757389739\_0634/container\_e15\_1510757554319\_0034\_01\_000001/hive2kafka-export29.log From source, I want to ext… --- ## [Global variables in an external file](https://discuss.elastic.co/t/global-variables-in-an-external-file/109378)
**Author:** [@ea1987](https://discuss.elastic.co/u/ea1987)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 1:29pm UTC](https://discuss.elastic.co/t/global-variables-in-an-external-file/109378 "2017-12-05T13:29:33Z")
Hi guys, I would like logstash to use global variables defined in an external file and use them in each event elaboration for doing something like this: external file: field1 = \[value1,value2\] meta logstash Language: … --- ## [Elastic search filter plugin](https://discuss.elastic.co/t/elastic-search-filter-plugin/110015)
**Author:** [@Sai\_Jana\_Selvakumar](https://discuss.elastic.co/u/Sai_Jana_Selvakumar)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 1:03pm UTC](https://discuss.elastic.co/t/elastic-search-filter-plugin/110015 "2017-12-05T13:03:43Z")
Do we need to install Elastic search filter plugin externally or do we need to install.. When I try to run logstash with elasticsearch filter plugin .I am getting this error Problems loading a plugin with {:type=\>"filt… --- ## [Unable to configure filebeat when output is set to logstash](https://discuss.elastic.co/t/unable-to-configure-filebeat-when-output-is-set-to-logstash/110349)
**Author:** [@Devendra\_Date](https://discuss.elastic.co/u/Devendra_Date)\ **Replies:** 0\ **Last updated:** [December 5, 2017, 12:54pm UTC](https://discuss.elastic.co/t/unable-to-configure-filebeat-when-output-is-set-to-logstash/110349 "2017-12-05T12:54:37Z")
Hi, I am creating 3 node cluster with ELK also installing filebeat on another server with predefined nginx logs to pass. While using Elasticsearch as an output in filebeat.yml file I am able to pass all logs successfull… --- ## [Replace field content](https://discuss.elastic.co/t/replace-field-content/110023)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 3\ **Last updated:** [December 4, 2017, 3:42pm UTC](https://discuss.elastic.co/t/replace-field-content/110023 "2017-12-04T15:42:35Z")
hi i\*m trying to replace the field content , currently a number/integer to a field with letters. basically i want to change the logontype to something more understandable than just the number "2" i first tried this as i… --- ## [Trying to ingest MongoDB collection to ES with mongo id](https://discuss.elastic.co/t/trying-to-ingest-mongodb-collection-to-es-with-mongo-id/110285)
**Author:** [@lorenzoc](https://discuss.elastic.co/u/lorenzoc)\ **Replies:** 1\ **Last updated:** [December 5, 2017, 11:34am UTC](https://discuss.elastic.co/t/trying-to-ingest-mongodb-collection-to-es-with-mongo-id/110285 "2017-12-05T11:34:12Z")
Hello, I am trying to use logstash to ingest a mongodb collection into ES6, but I'm facing a problem with mongo's \_Id field. I've found that many people just solve this issue by removing that id and letting ES generate… --- ## [Drop documents if certain field values are the same as the previous document](https://discuss.elastic.co/t/drop-documents-if-certain-field-values-are-the-same-as-the-previous-document/110154)
**Author:** [@lukebigum](https://discuss.elastic.co/u/lukebigum)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 11:01am UTC](https://discuss.elastic.co/t/drop-documents-if-certain-field-values-are-the-same-as-the-previous-document/110154 "2017-12-05T11:01:50Z")
Hello, I've got some software that generates status information every second, however I only want to record state changes of certain fields, not every single document. An example of what it would look like is: { times… --- ## [Logstash 6.0.0: Unknown setting 'index\_type' for elasticsearch](https://discuss.elastic.co/t/logstash-6-0-0-unknown-setting-index-type-for-elasticsearch/110318)
**Author:** [@gasmi](https://discuss.elastic.co/u/gasmi)\ **Replies:** 0\ **Last updated:** [December 5, 2017, 10:38am UTC](https://discuss.elastic.co/t/logstash-6-0-0-unknown-setting-index-type-for-elasticsearch/110318 "2017-12-05T10:38:24Z")
I'm trying to run logstash config files on ELK 6.0.0 docker image but I got this error: Unknown setting 'index\_type' for elasticsearch I am retrieving Spark streaming stats and send it to elasticsearch, but even this … --- ## [Multiple Grok filters with Logstash 6 for a single logfile?](https://discuss.elastic.co/t/multiple-grok-filters-with-logstash-6-for-a-single-logfile/110297)
**Author:** [@burnersk](https://discuss.elastic.co/u/burnersk)\ **Replies:** 3\ **Last updated:** [December 5, 2017, 10:02am UTC](https://discuss.elastic.co/t/multiple-grok-filters-with-logstash-6-for-a-single-logfile/110297 "2017-12-05T10:02:22Z")
I want to parse Debian's auth.log using multiple Grok match patterns and assign individual tags to them. Here is an exemplary auth.log: Dec 4 13:39:23 deb sshd\[970\]: Server listening on 0.0.0.0 port 22. Dec 4 13:39:2… --- ## [Urgent :ELK version compatible for Windows 10 to install as a service](https://discuss.elastic.co/t/urgent-elk-version-compatible-for-windows-10-to-install-as-a-service/110270)
**Author:** [@richagautam](https://discuss.elastic.co/u/richagautam)\ **Replies:** 12\ **Last updated:** [December 5, 2017, 10:02am UTC](https://discuss.elastic.co/t/urgent-elk-version-compatible-for-windows-10-to-install-as-a-service/110270 "2017-12-05T10:02:24Z")
Hi Team, I have to install ELK for my project. Kindly advice which ELK version will be compatible for Windows 10 to install as windows service Thanks Richa --- ## [Logstash beats SSL file read problem](https://discuss.elastic.co/t/logstash-beats-ssl-file-read-problem/110283)
**Author:** [@bodi4](https://discuss.elastic.co/u/bodi4)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 9:42am UTC](https://discuss.elastic.co/t/logstash-beats-ssl-file-read-problem/110283 "2017-12-05T09:42:46Z")
Hi, recently installed Logstash 6.0. Migrating pipelines from 5.x version. I have secured connection Filebeat -\> Logstash But loading pipeline Logstash cannot read certificates: \[2017-12-05T11:24:58,829\]\[ERROR\]\[logs… --- ## [Error while running logstash in linux](https://discuss.elastic.co/t/error-while-running-logstash-in-linux/110265)
**Author:** [@shivam\_bhagat](https://discuss.elastic.co/u/shivam_bhagat)\ **Replies:** 3\ **Last updated:** [December 5, 2017, 9:33am UTC](https://discuss.elastic.co/t/error-while-running-logstash-in-linux/110265 "2017-12-05T09:33:31Z")
Hi Iam getting following error when i execute following command in linux -sudo bin/logstash -f /etc/logstash/conf.d/dataLoader1.conf Error: NameError: cannot link Java class org.apache.logging.log4j.core.config.Logge… --- ## [Secure communication with Elasticsearch (SSL)](https://discuss.elastic.co/t/secure-communication-with-elasticsearch-ssl/59569)
**Author:** [@Tsury](https://discuss.elastic.co/u/Tsury)\ **Replies:** 4\ **Last updated:** [December 5, 2017, 9:14am UTC](https://discuss.elastic.co/t/secure-communication-with-elasticsearch-ssl/59569 "2017-12-05T09:14:51Z")
What are the best practices to secure the communication between Logstash and Elasticsearch? I'm looking for a non-commercial solution. It is possible to direct Logstash to NGINX with SSL? If so, how would you go about… --- ## [Windows event forward collector server -\> ELK](https://discuss.elastic.co/t/windows-event-forward-collector-server-elk/110291)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 0\ **Last updated:** [December 5, 2017, 8:52am UTC](https://discuss.elastic.co/t/windows-event-forward-collector-server-elk/110291 "2017-12-05T08:52:57Z")
Hi i hope this is the correct place to post this. we have a Windows Event forward infrastructure in place (WEF) sending the logs to a central server. does anyone have a config for this, my plan is that i will only nee… --- ## [Logstash-output-http in batch](https://discuss.elastic.co/t/logstash-output-http-in-batch/110098)
**Author:** [@bijay](https://discuss.elastic.co/u/bijay)\ **Replies:** 3\ **Last updated:** [December 5, 2017, 8:44am UTC](https://discuss.elastic.co/t/logstash-output-http-in-batch/110098 "2017-12-05T08:44:38Z")
Hi, Is there any work around for the http-output in batched format? It seems the below githib repo is too old and not in working state. Tried with logstash version 2.4.0 and 5.6.4 Any kind of help will be appreciat… --- ## [Combining Netscaler Appflow request and response within Logstash](https://discuss.elastic.co/t/combining-netscaler-appflow-request-and-response-within-logstash/110043)
**Author:** [@scherma](https://discuss.elastic.co/u/scherma)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 8:40am UTC](https://discuss.elastic.co/t/combining-netscaler-appflow-request-and-response-within-logstash/110043 "2017-12-05T08:40:54Z")
Hi all, when using Logstash to parse Netscaler Appflow, I have noted that the Appflow output places request and response information different records. I would like to be able to combine the request and response (they ar… --- ## [Logstash version missing in monitoring](https://discuss.elastic.co/t/logstash-version-missing-in-monitoring/109257)
**Author:** [@ericrichter](https://discuss.elastic.co/u/ericrichter)\ **Replies:** 3\ **Last updated:** [December 5, 2017, 8:36am UTC](https://discuss.elastic.co/t/logstash-version-missing-in-monitoring/109257 "2017-12-05T08:36:42Z")
Why isn't the version of the logstash node visible in the list with nodes? In the 5.x.x versions of x-pack, this version was visible and it was even possible to sort on the version number. From that list, it was easy to … --- ## [Invalid timestamp format, but the date is formed correctly](https://discuss.elastic.co/t/invalid-timestamp-format-but-the-date-is-formed-correctly/110092)
**Author:** [@Mojster](https://discuss.elastic.co/u/Mojster)\ **Replies:** 15\ **Last updated:** [December 5, 2017, 8:28am UTC](https://discuss.elastic.co/t/invalid-timestamp-format-but-the-date-is-formed-correctly/110092 "2017-12-05T08:28:51Z")
My logstash: ... date { locale =\> "en" match =\> \["date", "dd.MM.YYYY;HH:mm:ss"\] timezone =\> "Europe/Ljubljana" target =\> "date" } ... I've proced with this 352327606 records. 352327595 were ok, but 11 thrown an err… --- ## [Logstash input configure](https://discuss.elastic.co/t/logstash-input-configure/110275)
**Author:** [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Replies:** 1\ **Last updated:** [December 5, 2017, 8:24am UTC](https://discuss.elastic.co/t/logstash-input-configure/110275 "2017-12-05T08:24:30Z")
cat /etc/logstash/conf.d/02-beats-input.conf input { beats { path =\> "/var/log/messages" type =\> syslog } beats { path =\> "/var/log/apache/access.log" type =\> apache } port =\> 5044 ssl =\> true ssl\_certificate… --- ## [Logstash - port closed](https://discuss.elastic.co/t/logstash-port-closed/109774)
**Author:** [@Tade](https://discuss.elastic.co/u/Tade)\ **Replies:** 3\ **Last updated:** [December 5, 2017, 7:33am UTC](https://discuss.elastic.co/t/logstash-port-closed/109774 "2017-12-05T07:33:36Z")
Hi, I have problem with logstash - service is running, but port 10515 isnt open :frowning: Here is my logstash.conf: input { udp { host =\> "192.168.201.30" port =\> 10514 codec =\> "json" type =\> "rsyslog" } } f… --- ## [Unable to send data to ES 6.0 using Logstash 6.0](https://discuss.elastic.co/t/unable-to-send-data-to-es-6-0-using-logstash-6-0/109977)
**Author:** [@Pratiksha\_Tewary](https://discuss.elastic.co/u/Pratiksha_Tewary)\ **Replies:** 1\ **Last updated:** [December 5, 2017, 7:32am UTC](https://discuss.elastic.co/t/unable-to-send-data-to-es-6-0-using-logstash-6-0/109977 "2017-12-05T07:32:42Z")
Hi, I wanted to know if there have been any changes to the Logstash configuration process from the previous versions. I created a mapping in ES 6.0 and wanted to send data to it using Logstash. So, my pipeline was as fo… --- ## [Geoip create everything but the geoip.location](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900)
**Author:** [@Adesfire](https://discuss.elastic.co/u/Adesfire)\ **Replies:** 13\ **Last updated:** [December 5, 2017, 7:23am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900 "2017-12-05T07:23:55Z")
Good morning everyone, today I'm trying to implement the geoip filter, using this input file : filter { if \[host\] =~ /10\\.0\\.0\\.1/ { grok { match =\> \[ "message", "\\A%{SYSLOG… --- ## [Bulk insert of tables](https://discuss.elastic.co/t/bulk-insert-of-tables/110065)
**Author:** [@pragadeesh\_dharsha](https://discuss.elastic.co/u/pragadeesh_dharsha)\ **Replies:** 3\ **Last updated:** [December 5, 2017, 6:54am UTC](https://discuss.elastic.co/t/bulk-insert-of-tables/110065 "2017-12-05T06:54:31Z")
how to import a complete database into elasticsearch at once? --- ## [Logstash - Syslog Help - I am a total pleb](https://discuss.elastic.co/t/logstash-syslog-help-i-am-a-total-pleb/109957)
**Author:** [@MalwareSandbox](https://discuss.elastic.co/u/MalwareSandbox)\ **Replies:** 5\ **Last updated:** [December 5, 2017, 6:24am UTC](https://discuss.elastic.co/t/logstash-syslog-help-i-am-a-total-pleb/109957 "2017-12-05T06:24:53Z")
Hey Guys, I am a noob when it comes to ELK but am really eager to get this set up. I am currently using ELK to store syslog from multiple firewalls. I am using a fortinet (which is seeming to be not that fun to work wit… --- ## [\_geoip\_lookup\_failure question](https://discuss.elastic.co/t/geoip-lookup-failure-question/110237)
**Author:** [@peterch](https://discuss.elastic.co/u/peterch)\ **Replies:** 0\ **Last updated:** [December 5, 2017, 2:11am UTC](https://discuss.elastic.co/t/geoip-lookup-failure-question/110237 "2017-12-05T02:11:15Z")
Dear All, I tried to use geoip but show \_geoip\_lookup\_failure. Here's the my logstash config and sample data. Please advice. Thanks \[sample data\] geoip { source =\> "Source IP" } --- ## [Cannot receive data after using grok](https://discuss.elastic.co/t/cannot-receive-data-after-using-grok/110086)
**Author:** [@peterch](https://discuss.elastic.co/u/peterch)\ **Replies:** 2\ **Last updated:** [December 5, 2017, 1:57am UTC](https://discuss.elastic.co/t/cannot-receive-data-after-using-grok/110086 "2017-12-05T01:57:50Z")
Dear all, The following is my logstash config. However, I find cannot receive data. I use netstat -an find the port of 5044 and 514 are not listening. If i remove grok, can receive data. Any idea? Thanks input{ beats … --- ## [Docker logstash is not processing input](https://discuss.elastic.co/t/docker-logstash-is-not-processing-input/110079)
**Author:** [@bizmate](https://discuss.elastic.co/u/bizmate)\ **Replies:** 2\ **Last updated:** [December 4, 2017, 10:46pm UTC](https://discuss.elastic.co/t/docker-logstash-is-not-processing-input/110079 "2017-12-04T22:46:05Z")
I have been using a single solution container for running elastic, logstash and kibana and I am trying the images at elastic.co So I am trying with \> FROM docker.elastic.co/logstash/logstash:6.0.0 \> COPY ./config/logst… --- ## [How to aggregate a value repeatedly](https://discuss.elastic.co/t/how-to-aggregate-a-value-repeatedly/110213)
**Author:** [@magdieldominguez](https://discuss.elastic.co/u/magdieldominguez)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 9:21pm UTC](https://discuss.elastic.co/t/how-to-aggregate-a-value-repeatedly/110213 "2017-12-04T21:21:54Z")
Hi I have a log file and in each line it contains a number that I need to keep adding to get the total produced per log. So some of the lines in the log file could be... 10.05.17 20:06:39.79 UPDATE/Add: Legs:15 10.05.… --- ## [Logstash Filter - Elasticserach Query](https://discuss.elastic.co/t/logstash-filter-elasticserach-query/110210)
**Author:** [@arnydo](https://discuss.elastic.co/u/arnydo)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 8:29pm UTC](https://discuss.elastic.co/t/logstash-filter-elasticserach-query/110210 "2017-12-04T20:29:15Z")
Good afternoon, I am working on a Logstash filter and am in need of assitance. Basically, I have two indexes; one is a list of known malware domains (domainlist) and another is an index of DNS queries fed by Packetbeat… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=861) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=863)