# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1042 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1043 --- ## [Monitor api](https://discuss.elastic.co/t/monitor-api/73980)
**Author:** [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Replies:** 3\ **Last updated:** [February 6, 2017, 10:06am UTC](https://discuss.elastic.co/t/monitor-api/73980 "2017-02-06T10:06:12Z")
Hi All, Logstash 5 has now a monitor api but I cannot seems to find any project / dashboard / plugin for this functionality. I have install x-pack and I seen the elasticsearch stats and such but no logstash info.. Do… --- ## [Geo point in logstash](https://discuss.elastic.co/t/geo-point-in-logstash/73851)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 8\ **Last updated:** [February 6, 2017, 9:52am UTC](https://discuss.elastic.co/t/geo-point-in-logstash/73851 "2017-02-06T09:52:53Z")
My input as CSV and output as elastic search. in my csv file having one column location which is holding value latitude and longitude information. for example-72.8799,32.46567(attitude,longitude) i am using below fil… --- ## [Logstash Pipline aborted error](https://discuss.elastic.co/t/logstash-pipline-aborted-error/73946)
**Author:** [@mliu1212](https://discuss.elastic.co/u/mliu1212)\ **Replies:** 2\ **Last updated:** [February 6, 2017, 7:14am UTC](https://discuss.elastic.co/t/logstash-pipline-aborted-error/73946 "2017-02-06T07:14:40Z")
I am trying to run Suricata with the ELK stack - I cannot get my info to visualize on Kibana bc the Logstash pipline keeps breaking down: {:timestamp=\>"2017-02-04T13:50:29.731000+0000", :message=\>"Pipeline aborted due … --- ## [How to ship Logstash API results to](https://discuss.elastic.co/t/how-to-ship-logstash-api-results-to/73822)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 3\ **Last updated:** [February 6, 2017, 6:55am UTC](https://discuss.elastic.co/t/how-to-ship-logstash-api-results-to/73822 "2017-02-06T06:55:33Z")
Hi, I am able to run Logstash API calls using my Linux command line using curl like; curl -XGET "http://localhost:9600/\_node/stats/jvm" is there any way to poll this in regular intervals and push it to elasticsearch… --- ## [How to set up: Synology -\> Logstash RFC3164](https://discuss.elastic.co/t/how-to-set-up-synology-logstash-rfc3164/73756)
**Author:** [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Replies:** 1\ **Last updated:** [February 6, 2017, 6:53am UTC](https://discuss.elastic.co/t/how-to-set-up-synology-logstash-rfc3164/73756 "2017-02-06T06:53:12Z")
Hi, I'm very new to the complete ELK stack. I set up on a ESXI 6.5 a CentOS 7 (minimal installation) the ELK stack. Everything is working so far (I think so). I used for that: http://www.tecmint.com/install-elasticsear… --- ## [Logstash timestamp issue](https://discuss.elastic.co/t/logstash-timestamp-issue/73843)
**Author:** [@saurabhT](https://discuss.elastic.co/u/saurabhT)\ **Replies:** 3\ **Last updated:** [February 6, 2017, 6:41am UTC](https://discuss.elastic.co/t/logstash-timestamp-issue/73843 "2017-02-06T06:41:00Z")
I have logs which have got time stamp which are in the format 17/01/30 07:55:03 i have got it parsed using Grok filters but when i want to assing the same value to @timestamp it does not turn up. My filters look like g… --- ## [How to use twitter plugin in Logstash 5.x](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176)
**Author:** [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Replies:** 6\ **Last updated:** [February 6, 2017, 5:17am UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176 "2017-02-06T05:17:22Z")
Hi I want to pull tweets matching particular keywords into ELK, adding a field to show what was matched. I can do this, but solution does not scale and I think there is likely a better way. Here's my conf files, I t… --- ## [Problems with geoip configuration](https://discuss.elastic.co/t/problems-with-geoip-configuration/28767)
**Author:** [@dfaropennetwork](https://discuss.elastic.co/u/dfaropennetwork)\ **Replies:** 35\ **Last updated:** [February 5, 2017, 9:12pm UTC](https://discuss.elastic.co/t/problems-with-geoip-configuration/28767 "2017-02-05T21:12:28Z")
Hi All I've problems with mi geoip ip configuration, i've kibana 4.1.1 with logstash 1.5.3 and apache 2.4. This is my personalized geoip file configuration on logstash, i called 12-geoip.conf : filter { if \[type\] =… --- ## [Remembering Field Values](https://discuss.elastic.co/t/remembering-field-values/73870)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 12\ **Last updated:** [February 5, 2017, 6:18pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870 "2017-02-05T18:18:24Z")
Hi, Does anyone have any solutions for how to store and re-use field values between events? For example if we have a log file that contains something like:- Counter 1 stuff.... stuff.... Counter 2 stuff... stuff... … --- ## [Logstash aggregate doesn't run at all!](https://discuss.elastic.co/t/logstash-aggregate-doesnt-run-at-all/63763)
**Author:** [@xiaozhuqq48](https://discuss.elastic.co/u/xiaozhuqq48)\ **Replies:** 1\ **Last updated:** [February 5, 2017, 6:10pm UTC](https://discuss.elastic.co/t/logstash-aggregate-doesnt-run-at-all/63763 "2017-02-05T18:10:50Z")
Down below is my code. I have even set the task\_id to 1, and used add\_tag. The ruby code had executed, but aggregate doesn't work. Can someone help? if \[operation\] == "receive a request" { ruby { code =\> " … --- ## [Logstash pattern or dsl query](https://discuss.elastic.co/t/logstash-pattern-or-dsl-query/71203)
**Author:** [@Lucifer\_hbp](https://discuss.elastic.co/u/Lucifer_hbp)\ **Replies:** 4\ **Last updated:** [February 5, 2017, 6:01pm UTC](https://discuss.elastic.co/t/logstash-pattern-or-dsl-query/71203 "2017-02-05T18:01:13Z")
Hi, I have following kind of logs: DEBUG 20161219T172835.101-05:00 59 Client start DEBUG 20161219T172835.101-05:00 59 Client v=1 DEBUG 20161219T172835.101-05:00 59 Client v=2 DEBUG 20161219T172835.461-05:00 59 Clien… --- ## [Logstash http\_poller plugin help](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957)
**Author:** [@FrankC](https://discuss.elastic.co/u/FrankC)\ **Replies:** 5\ **Last updated:** [February 5, 2017, 3:37pm UTC](https://discuss.elastic.co/t/logstash-http-poller-plugin-help/73957 "2017-02-05T15:37:00Z")
Hello, I am trying to use the http\_poller plugin to pull data from an API end point such as the Quandl service. First I tested the http\_poller using ES url: http://localhost:9200. That worked. Then I changed url t… --- ## [Problem when log stash is executed as a service (Ubuntu)](https://discuss.elastic.co/t/problem-when-log-stash-is-executed-as-a-service-ubuntu/70837)
**Author:** [@caleneo](https://discuss.elastic.co/u/caleneo)\ **Replies:** 7\ **Last updated:** [February 5, 2017, 2:50pm UTC](https://discuss.elastic.co/t/problem-when-log-stash-is-executed-as-a-service-ubuntu/70837 "2017-02-05T14:50:19Z")
Hello, I'm having several issues trying to execute it as a service. Using the unzipped files everything is OK but when I copy the configuration file into the conf.d/ directory I get always the same error: \[2017-01-07T0… --- ## [Offline plugin pack - http proxy support](https://discuss.elastic.co/t/offline-plugin-pack-http-proxy-support/73798)
**Author:** [@Pradyumna\_Joshi](https://discuss.elastic.co/u/Pradyumna_Joshi)\ **Replies:** 2\ **Last updated:** [February 5, 2017, 11:03am UTC](https://discuss.elastic.co/t/offline-plugin-pack-http-proxy-support/73798 "2017-02-05T11:03:10Z")
I am trying to use "translate" plugin (logstash-filter-translate) on one of the intranet system. Since "translate" plugin is not part of logstash standard plugins, I am trying to prepare an offline plugin pack for trans… --- ## [Is it possible to index only the matched log lines of grok in Logstash?](https://discuss.elastic.co/t/is-it-possible-to-index-only-the-matched-log-lines-of-grok-in-logstash/73624)
**Author:** [@Kulasangar\_Gowrisang](https://discuss.elastic.co/u/Kulasangar_Gowrisang)\ **Replies:** 5\ **Last updated:** [February 5, 2017, 10:00am UTC](https://discuss.elastic.co/t/is-it-possible-to-index-only-the-matched-log-lines-of-grok-in-logstash/73624 "2017-02-05T10:00:33Z")
I'm having a log file, which actually has INFOs' and ERRORs. So I tried to match only the needful INFOs by using the grok filter. So this is how my log lines look like. Few of them from the file. And this is how my gro… --- ## [Architectural advise requested](https://discuss.elastic.co/t/architectural-advise-requested/73972)
**Author:** [@MarvinD](https://discuss.elastic.co/u/MarvinD)\ **Replies:** 3\ **Last updated:** [February 5, 2017, 9:59am UTC](https://discuss.elastic.co/t/architectural-advise-requested/73972 "2017-02-05T09:59:46Z")
Hi all, I'm new to ESK and I could really use some help in figuring out the best way to set up what I'm trying to achieve. I have an SQL database that contains sales data, one row for each sold item. That table is a co… --- ## [/opt/logstash/bin/logstash](https://discuss.elastic.co/t/opt-logstash-bin-logstash/73889)
**Author:** [@ELKSERVER](https://discuss.elastic.co/u/ELKSERVER)\ **Replies:** 1\ **Last updated:** [February 5, 2017, 9:25am UTC](https://discuss.elastic.co/t/opt-logstash-bin-logstash/73889 "2017-02-05T09:25:29Z")
sudo /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/ Thats the line digitalocean has me putting in when building my ELK server with Ubuntu 16.04. That line is suppose to return "Configuration OK" but m… --- ## [Logstash.inputs.udp - EOFError: End of file reached](https://discuss.elastic.co/t/logstash-inputs-udp-eoferror-end-of-file-reached/73212)
**Author:** [@drolfe](https://discuss.elastic.co/u/drolfe)\ **Replies:** 1\ **Last updated:** [February 5, 2017, 1:32am UTC](https://discuss.elastic.co/t/logstash-inputs-udp-eoferror-end-of-file-reached/73212 "2017-02-05T01:32:56Z")
HI, I"m getting this error after 30 minutes or so, udp port is open but nothing gets indexed until I restart logstash load looks to only be 100 events a second app versions elasticsearch 5.1.1 all kibana … --- ## [Logstash sflow codec](https://discuss.elastic.co/t/logstash-sflow-codec/73116)
**Author:** [@drolfe](https://discuss.elastic.co/u/drolfe)\ **Replies:** 1\ **Last updated:** [February 5, 2017, 1:32am UTC](https://discuss.elastic.co/t/logstash-sflow-codec/73116 "2017-02-05T01:32:06Z")
HI all, I've just started using the below codec https://www.elastic.co/guide/en/logstash/current/plugins-codecs-sflow.html It works well, however doing a straight export into elasticsearch finds 95% of all the fields … --- ## [KV filter trim with regex](https://discuss.elastic.co/t/kv-filter-trim-with-regex/73437)
**Author:** [@flalar](https://discuss.elastic.co/u/flalar)\ **Replies:** 3\ **Last updated:** [February 4, 2017, 11:29am UTC](https://discuss.elastic.co/t/kv-filter-trim-with-regex/73437 "2017-02-04T11:29:08Z")
Given the example filter inputs "\[Key1:/app/segment/confirm.jsp\] 21:50:00 \[Key2:3\] \[Key3:93\] \[Key4:392\] \[Key5:210\] \[Key6:144\] \[Key7:93\] \[Key8:392\] Some more text that should not be included in key value" I need to ext… --- ## [JDBC, Putting result of SQL query into one field](https://discuss.elastic.co/t/jdbc-putting-result-of-sql-query-into-one-field/73817)
**Author:** [@Alexander1](https://discuss.elastic.co/u/Alexander1)\ **Replies:** 4\ **Last updated:** [February 4, 2017, 10:56am UTC](https://discuss.elastic.co/t/jdbc-putting-result-of-sql-query-into-one-field/73817 "2017-02-04T10:56:46Z")
Hello everyone! I have a problem with configuring the logstash 5.2. I use jdbc-input plugin, And I want to put the result of the SQL-query in to the one field called "query\_result". But I don't know how. Can you help m… --- ## [Kafka input unable to create consumer after setting session timeout](https://discuss.elastic.co/t/kafka-input-unable-to-create-consumer-after-setting-session-timeout/61956)
**Author:** [@retzkek](https://discuss.elastic.co/u/retzkek)\ **Replies:** 3\ **Last updated:** [February 4, 2017, 1:48am UTC](https://discuss.elastic.co/t/kafka-input-unable-to-create-consumer-after-setting-session-timeout/61956 "2017-02-04T01:48:51Z")
I'm trying to setup logstash (5.0.0-beta1) to process events from a Kafka (0.10.0.0) topic and index them into a fairly busy Elasticsearch cluster (we've been using Logstash and Elasticsearch for a while, Kafka is the ne… --- ## [Logstash 5.1.1 kafka input doesn't pick up existing messages on topic](https://discuss.elastic.co/t/logstash-5-1-1-kafka-input-doesnt-pick-up-existing-messages-on-topic/73921)
**Author:** [@Fizi](https://discuss.elastic.co/u/Fizi)\ **Replies:** 0\ **Last updated:** [February 3, 2017, 11:18pm UTC](https://discuss.elastic.co/t/logstash-5-1-1-kafka-input-doesnt-pick-up-existing-messages-on-topic/73921 "2017-02-03T23:18:17Z")
I have the following logstash config with a kafka input input { kafka { bootstrap\_servers =\> "localhost:9092" topics =\> \["mytopic"\] } } filter { json { source =\> "m… --- ## [\[ERROR\]\[org.logstash.beats.BeatsHandler\] Exception: not an SSL/TLS record](https://discuss.elastic.co/t/error-org-logstash-beats-beatshandler-exception-not-an-ssl-tls-record/71289)
**Author:** [@r1chardp](https://discuss.elastic.co/u/r1chardp)\ **Replies:** 8\ **Last updated:** [February 3, 2017, 10:47pm UTC](https://discuss.elastic.co/t/error-org-logstash-beats-beatshandler-exception-not-an-ssl-tls-record/71289 "2017-02-03T22:47:03Z")
my logstash config: input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt" ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key" } } My filebeat c… --- ## [Logstash not indexing in Solr](https://discuss.elastic.co/t/logstash-not-indexing-in-solr/73243)
**Author:** [@Samuel\_Lima](https://discuss.elastic.co/u/Samuel_Lima)\ **Replies:** 8\ **Last updated:** [February 3, 2017, 9:48pm UTC](https://discuss.elastic.co/t/logstash-not-indexing-in-solr/73243 "2017-02-03T21:48:00Z")
I'm trying to implement the following architecture: Apache Log --\> Filebeat --\> Logstash --\> Solr Apache and Filebeat run in the same VM. Logstash and Solr run each one on its own VM. I know by the logs that Filebeat… --- ## [Filebeat 5.1 to logstash 5.1 SSL setup, filebeat CRIT exiting](https://discuss.elastic.co/t/filebeat-5-1-to-logstash-5-1-ssl-setup-filebeat-crit-exiting/73903)
**Author:** [@robertchen117](https://discuss.elastic.co/u/robertchen117)\ **Replies:** 0\ **Last updated:** [February 3, 2017, 7:01pm UTC](https://discuss.elastic.co/t/filebeat-5-1-to-logstash-5-1-ssl-setup-filebeat-crit-exiting/73903 "2017-02-03T19:01:06Z")
I generate the crt and key using openssl, logstash configuration: input { beats { host =\> "10.8.8.153" port =\> "5044" ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/filebeat2logst… --- ## [Elasticsearch message format](https://discuss.elastic.co/t/elasticsearch-message-format/73856)
**Author:** [@Oleksii\_Mazilov](https://discuss.elastic.co/u/Oleksii_Mazilov)\ **Replies:** 4\ **Last updated:** [February 3, 2017, 3:37pm UTC](https://discuss.elastic.co/t/elasticsearch-message-format/73856 "2017-02-03T15:37:28Z")
I need to remove several fields before pushing data to Elasticsearch. I can't use mutate filter because in aggregate filter I'm using field which I need to remove Conf example: input { jdbc { bla\_bla\_bla } } filte… --- ## [Logstash activity is not recording in the database](https://discuss.elastic.co/t/logstash-activity-is-not-recording-in-the-database/73864)
**Author:** [@ggp](https://discuss.elastic.co/u/ggp)\ **Replies:** 0\ **Last updated:** [February 3, 2017, 1:57pm UTC](https://discuss.elastic.co/t/logstash-activity-is-not-recording-in-the-database/73864 "2017-02-03T13:57:49Z")
i have different versions of logstash process running with postgresql database as a source, with older version of logstash i can see its activity getting tracked in postgresql pg\_stat\_activity table with client ip , que… --- ## [No output when using multiline codec from stdin](https://discuss.elastic.co/t/no-output-when-using-multiline-codec-from-stdin/73547)
**Author:** [@mhooper](https://discuss.elastic.co/u/mhooper)\ **Replies:** 6\ **Last updated:** [February 3, 2017, 1:14pm UTC](https://discuss.elastic.co/t/no-output-when-using-multiline-codec-from-stdin/73547 "2017-02-03T13:14:11Z")
I have recently started using logstash and am trying to build a sandbox environment to test different parsing. I have built a test.config file using stdin and cat a data file through it to get stdout display of outcome… --- ## [Failed to send event to Redis](https://discuss.elastic.co/t/failed-to-send-event-to-redis/73581)
**Author:** [@111126](https://discuss.elastic.co/u/111126)\ **Replies:** 2\ **Last updated:** [February 3, 2017, 10:47am UTC](https://discuss.elastic.co/t/failed-to-send-event-to-redis/73581 "2017-02-03T10:47:50Z")
I use the next logstash config: input{ tcp { port =\> "7101" type =\> "228-pur-plan-tcp" codec =\> "json\_lines" } tcp { host =\> "0.0.0.0" port =\> "7102" type =\> "228-pur-plan-tcp" codec =\> "json\_lines" } tcp { … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1041) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1043)