# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=870 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 871 --- ## [InfluxDB input plugin for logstash](https://discuss.elastic.co/t/influxdb-input-plugin-for-logstash/107670)
**Author:** [@kiranilla](https://discuss.elastic.co/u/kiranilla)\ **Replies:** 1\ **Last updated:** [November 22, 2017, 1:47pm UTC](https://discuss.elastic.co/t/influxdb-input-plugin-for-logstash/107670 "2017-11-22T13:47:10Z")
Hi, As there is a output plugin for influxDB in Logstash (https://www.elastic.co/guide/en/logstash/current/plugins-outputs-influxdb.html), but not able to see any input plugin for it. Could you please provide an way or… --- ## [JournalBeat Logstash configuration](https://discuss.elastic.co/t/journalbeat-logstash-configuration/107427)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 2\ **Last updated:** [November 22, 2017, 1:23pm UTC](https://discuss.elastic.co/t/journalbeat-logstash-configuration/107427 "2017-11-22T13:23:16Z")
I am trying to get the JournalBeat installed and working, so it can take my systemd logs and send them to ElasticSearch. I am running Centos 7.3. The steps I completed: I have downloaded the zip file file from https:/… --- ## [Duration\_in\_millis shows too high values](https://discuss.elastic.co/t/duration-in-millis-shows-too-high-values/108563)
**Author:** [@mathias](https://discuss.elastic.co/u/mathias)\ **Replies:** 2\ **Last updated:** [November 22, 2017, 1:22pm UTC](https://discuss.elastic.co/t/duration-in-millis-shows-too-high-values/108563 "2017-11-22T13:22:49Z")
Hi, I am using ELK 5.6.3 I am looking at the delay of events and I can't understand the output values. curl -s logstash:9600/\_node/stats/pipeline | jq .pipeline.plugins.outputs \[ { "id": "events\_output\_to\_elasti… --- ## [JDBC tracking a string column for unique ingestion](https://discuss.elastic.co/t/jdbc-tracking-a-string-column-for-unique-ingestion/108662)
**Author:** [@ericohtake](https://discuss.elastic.co/u/ericohtake)\ **Replies:** 8\ **Last updated:** [November 22, 2017, 1:07pm UTC](https://discuss.elastic.co/t/jdbc-tracking-a-string-column-for-unique-ingestion/108662 "2017-11-22T13:07:43Z")
I'm trying to ingest just new rows from our database (existent rows never change), but I'm facing some problems on keeping track of what have been already ingested. This fact causes data duplication when sent to ES. I'm… --- ## [Aggregate inactivity\_timeout must be lower than timeout error](https://discuss.elastic.co/t/aggregate-inactivity-timeout-must-be-lower-than-timeout-error/108697)
**Author:** [@coda](https://discuss.elastic.co/u/coda)\ **Replies:** 0\ **Last updated:** [November 22, 2017, 11:19am UTC](https://discuss.elastic.co/t/aggregate-inactivity-timeout-must-be-lower-than-timeout-error/108697 "2017-11-22T11:19:55Z")
Hi! Could someone please help me on this? The following aggregate: aggregate { task\_id =\> "%{postfix\_queueid}" code =\> " if event.get('@timestamp') map\['@timestamp'\] = event.get('@timestamp'… --- ## [Issues with sending events to elasticsearch on Elastic 6.0](https://discuss.elastic.co/t/issues-with-sending-events-to-elasticsearch-on-elastic-6-0/108687)
**Author:** [@sumeshms](https://discuss.elastic.co/u/sumeshms)\ **Replies:** 0\ **Last updated:** [November 22, 2017, 9:44am UTC](https://discuss.elastic.co/t/issues-with-sending-events-to-elasticsearch-on-elastic-6-0/108687 "2017-11-22T09:44:36Z")
Hi I am facing some issues with passing data to Elasticsearch from logstash with the version 6.0. Logstash stop sending events to ES after a while (say 10 minutes) and If I restart logstash, again it starts sending. Re… --- ## [Logstash output same data to Elasticsearch repeatedly](https://discuss.elastic.co/t/logstash-output-same-data-to-elasticsearch-repeatedly/108680)
**Author:** [@HelloUniverse](https://discuss.elastic.co/u/HelloUniverse)\ **Replies:** 0\ **Last updated:** [November 22, 2017, 9:20am UTC](https://discuss.elastic.co/t/logstash-output-same-data-to-elasticsearch-repeatedly/108680 "2017-11-22T09:20:32Z")
Hi There, We are facing this kind of problem occasionaly that once some ES nodes failed, the Logstash output plugin will insert some data repeatedly to this ES cluster. The data were extracted from kafka by Logstash t… --- ## [Extracting a key value pair in which the key contains certain string](https://discuss.elastic.co/t/extracting-a-key-value-pair-in-which-the-key-contains-certain-string/108384)
**Author:** [@abhilash3011](https://discuss.elastic.co/u/abhilash3011)\ **Replies:** 1\ **Last updated:** [November 22, 2017, 9:14am UTC](https://discuss.elastic.co/t/extracting-a-key-value-pair-in-which-the-key-contains-certain-string/108384 "2017-11-22T09:14:22Z")
Hi all, I am parsing my data as key value pair using kv filter. I want to extract only the key which contain a certain string. For example if my string is "name". then i want to extract keys which are "myname", yourname… --- ## [How to pass date from messages to output filename in logstash](https://discuss.elastic.co/t/how-to-pass-date-from-messages-to-output-filename-in-logstash/108653)
**Author:** [@adi\_info](https://discuss.elastic.co/u/adi_info)\ **Replies:** 1\ **Last updated:** [November 22, 2017, 7:57am UTC](https://discuss.elastic.co/t/how-to-pass-date-from-messages-to-output-filename-in-logstash/108653 "2017-11-22T07:57:11Z")
I have a requirement to name the output file based on the date present in input messages. I want to assign the incoming date to metadata and then use it in the output like path =\> /path/location/filename\_%{\[@metadata\]\[… --- ## [Grokparse failure mikrotik](https://discuss.elastic.co/t/grokparse-failure-mikrotik/108237)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 6\ **Last updated:** [November 22, 2017, 6:29am UTC](https://discuss.elastic.co/t/grokparse-failure-mikrotik/108237 "2017-11-22T06:29:13Z")
Hi I am trying to grok an event log and it looks like following- has\_log: http dstnat: in:\ out:(none), proto TCP (SYN), 10.0.1.215:45306-\>162.235.200.2:80, len 60 I have tested the grok for the above i… --- ## [Trouble with Logstash JSON parsing](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426)
**Author:** [@tuxnader](https://discuss.elastic.co/u/tuxnader)\ **Replies:** 9\ **Last updated:** [November 22, 2017, 6:17am UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426 "2017-11-22T06:17:42Z")
I'm calling an API from a script. The script then runs the results through "jq" to rearrange the fields properly which then outputs to a .json file. In Logstash I'm using File input to read the file and trying to output … --- ## [Merging JSON files](https://discuss.elastic.co/t/merging-json-files/108626)
**Author:** [@NPNP](https://discuss.elastic.co/u/NPNP)\ **Replies:** 1\ **Last updated:** [November 22, 2017, 1:59am UTC](https://discuss.elastic.co/t/merging-json-files/108626 "2017-11-22T01:59:21Z")
Hello everyone, 100% newbie here. Thank you for your patience. I'm trying to merge two simple json files under certain condition. Is that possible? Any direction is useful. JSON\_1 { "name\_1" : "John", "age\_1"… --- ## [Getting key value from nested "event" element](https://discuss.elastic.co/t/getting-key-value-from-nested-event-element/107852)
**Author:** [@danielmcquillen](https://discuss.elastic.co/u/danielmcquillen)\ **Replies:** 4\ **Last updated:** [November 22, 2017, 12:42am UTC](https://discuss.elastic.co/t/getting-key-value-from-nested-event-element/107852 "2017-11-22T00:42:29Z")
This one has me stumped, and I wonder if there's a subtlety I'm missing. I have an (unfortunately structured) log event that records an 'assessment' attempt in our learning management system. It looks something like thi… --- ## [Date parsing with am:pm and timezone](https://discuss.elastic.co/t/date-parsing-with-am-pm-and-timezone/108616)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 4\ **Last updated:** [November 21, 2017, 10:49pm UTC](https://discuss.elastic.co/t/date-parsing-with-am-pm-and-timezone/108616 "2017-11-21T22:49:50Z")
The date in my logs looks like that: Nov 5, 2017 11:06:30 PM CST I tried to parse it like that: date { match =\> \[ "timestamp" , "MMM dd, yyyy KK:mm:ss aa Z" \] } or like that: date { … --- ## [Sleep Each Individual Event](https://discuss.elastic.co/t/sleep-each-individual-event/108621)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 0\ **Last updated:** [November 21, 2017, 10:17pm UTC](https://discuss.elastic.co/t/sleep-each-individual-event/108621 "2017-11-21T22:17:19Z")
Logstash v5.6.2. I am encountering a race condition where a network monitor I'm building is passing alerts to Logstash. The issue is that I also query an Icinga Monitor every 60s in an attempt to generate a list of know… --- ## [Ruby Code - Converting Socket Address Error](https://discuss.elastic.co/t/ruby-code-converting-socket-address-error/108425)
**Author:** [@kazul33](https://discuss.elastic.co/u/kazul33)\ **Replies:** 3\ **Last updated:** [November 21, 2017, 8:55pm UTC](https://discuss.elastic.co/t/ruby-code-converting-socket-address-error/108425 "2017-11-21T20:55:45Z")
I'm working with Auditd data. I have the SADDR within a log message and I'm trying to convert it using some ruby code. Here is my snippet: filter { ruby { code =\> " require 'socket' … --- ## [Elastic output plugin: result \_type in Elastic is always "doc"](https://discuss.elastic.co/t/elastic-output-plugin-result-type-in-elastic-is-always-doc/108382)
**Author:** [@Andrey\_Tomilin](https://discuss.elastic.co/u/Andrey_Tomilin)\ **Replies:** 2\ **Last updated:** [November 21, 2017, 8:12pm UTC](https://discuss.elastic.co/t/elastic-output-plugin-result-type-in-elastic-is-always-doc/108382 "2017-11-21T20:12:37Z")
I am using the latest (6.0) ElasticStack for clear setup (no migrations) I have the following type mapping: PUT my\_index { "mappings": { "my\_type": { "dynamic": "strict", "properties": { … --- ## [Import of huge quantity of csv files in elasticsearch with logstash](https://discuss.elastic.co/t/import-of-huge-quantity-of-csv-files-in-elasticsearch-with-logstash/108092)
**Author:** [@Wills1](https://discuss.elastic.co/u/Wills1)\ **Replies:** 8\ **Last updated:** [November 21, 2017, 7:12pm UTC](https://discuss.elastic.co/t/import-of-huge-quantity-of-csv-files-in-elasticsearch-with-logstash/108092 "2017-11-21T19:12:47Z")
Hi ! I'm new in ELK and have some questions concerning the import of multiple csv-files in elasticsearch via logstash. I have a folder with more than 425 csv-files within and each csv file have different column header;… --- ## [Misscounting lines when importing csv file to logstash](https://discuss.elastic.co/t/misscounting-lines-when-importing-csv-file-to-logstash/108604)
**Author:** [@afonso.ribeiro](https://discuss.elastic.co/u/afonso.ribeiro)\ **Replies:** 0\ **Last updated:** [November 21, 2017, 6:11pm UTC](https://discuss.elastic.co/t/misscounting-lines-when-importing-csv-file-to-logstash/108604 "2017-11-21T18:11:43Z")
Is there any bug counting lines of csv file when importing data through logstash to elastic? I have a csv file with 1000 lines and 200 columns. Some of the columns have missing values and logstash is importing more line… --- ## [UNable to install logstash on windows 10](https://discuss.elastic.co/t/unable-to-install-logstash-on-windows-10/107306)
**Author:** [@Korrapati\_Lokesh\_Kum](https://discuss.elastic.co/u/Korrapati_Lokesh_Kum)\ **Replies:** 2\ **Last updated:** [November 21, 2017, 4:22pm UTC](https://discuss.elastic.co/t/unable-to-install-logstash-on-windows-10/107306 "2017-11-21T16:22:34Z")
I have tried creating a config file and I get an error cant find java1.5 or higher. java 8 is installed in my laptop \[image\] --- ## [Logstash HTTP Input and REDIS Ouput](https://discuss.elastic.co/t/logstash-http-input-and-redis-ouput/108130)
**Author:** [@charlot\_Attard](https://discuss.elastic.co/u/charlot_Attard)\ **Replies:** 2\ **Last updated:** [November 21, 2017, 2:33pm UTC](https://discuss.elastic.co/t/logstash-http-input-and-redis-ouput/108130 "2017-11-21T14:33:27Z")
I have installed http plugin on my logstash 5.3.2, and am running the plugin with the below syntax. input{ http { id =\> "190190" port =\> "1000" } } The problem is that when I specify output as redis the logs are n… --- ## [What is this log nginx grok pattern](https://discuss.elastic.co/t/what-is-this-log-nginx-grok-pattern/108560)
**Author:** [@omid](https://discuss.elastic.co/u/omid)\ **Replies:** 1\ **Last updated:** [November 21, 2017, 2:31pm UTC](https://discuss.elastic.co/t/what-is-this-log-nginx-grok-pattern/108560 "2017-11-21T14:31:51Z")
Hi what is this log nginx grok pattern !? 100.100.100.100 - - \[21/Nov/2017:11:04:13 +0330\] "GET /bookmarklet.js?1511247 HTTP/1.0" 200 16077 "http://mywebsite.ir/video/3911/خانه" "Mozilla/5.0 (Windows NT 6.1; Win64; x64… --- ## [Logstash sns plugin error after restart](https://discuss.elastic.co/t/logstash-sns-plugin-error-after-restart/108558)
**Author:** [@Akaren](https://discuss.elastic.co/u/Akaren)\ **Replies:** 0\ **Last updated:** [November 21, 2017, 12:29pm UTC](https://discuss.elastic.co/t/logstash-sns-plugin-error-after-restart/108558 "2017-11-21T12:29:30Z")
Hi, I have installed sns plugin without any error. And here is my output part: if \[type\] == 'message-dropped' { sns { region =\> "eu-west-1" arn =\> "arn:aws:sns:eu-west-1:1111111111111:sip\_pillory"… --- ## [Multiple outputs and filebeat back pressure management](https://discuss.elastic.co/t/multiple-outputs-and-filebeat-back-pressure-management/108553)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 2\ **Last updated:** [November 21, 2017, 11:25am UTC](https://discuss.elastic.co/t/multiple-outputs-and-filebeat-back-pressure-management/108553 "2017-11-21T11:25:50Z")
In case of multiple outputs, how the back pressure management will be done at input say filebeat as the performance of each of the outputs will vary. Also what will happen if one of the output will fail. Whether the eve… --- ## [New logstash server in Azure elastic 3 node cluster](https://discuss.elastic.co/t/new-logstash-server-in-azure-elastic-3-node-cluster/108297)
**Author:** [@HeadScratcher](https://discuss.elastic.co/u/HeadScratcher)\ **Replies:** 11\ **Last updated:** [November 21, 2017, 11:12am UTC](https://discuss.elastic.co/t/new-logstash-server-in-azure-elastic-3-node-cluster/108297 "2017-11-21T11:12:59Z")
this weekend i've been attempting to get my cluster running. I chose the Azure market place Elastic/Kibana/X-PACK setup and configured it like this Single Subnet 10.0.0.0/24 3 data nodes 10.0.0.6, 10.0.0.7, 10.0.0.8 … --- ## [Logstash HTTP input with SSL keystore](https://discuss.elastic.co/t/logstash-http-input-with-ssl-keystore/108550)
**Author:** [@metju312](https://discuss.elastic.co/u/metju312)\ **Replies:** 0\ **Last updated:** [November 21, 2017, 11:00am UTC](https://discuss.elastic.co/t/logstash-http-input-with-ssl-keystore/108550 "2017-11-21T11:00:12Z")
Hello, What I want: I want to send logs using "Postman" to http logstash input with SSL. What I already done: I added SSL configuration in my logstash.conf file using http plugin: https://www.elastic.co/blog/introd… --- ## [Logstash config - elasticsearch to elasticsearch](https://discuss.elastic.co/t/logstash-config-elasticsearch-to-elasticsearch/108546)
**Author:** [@giangra](https://discuss.elastic.co/u/giangra)\ **Replies:** 3\ **Last updated:** [November 21, 2017, 10:54am UTC](https://discuss.elastic.co/t/logstash-config-elasticsearch-to-elasticsearch/108546 "2017-11-21T10:54:30Z")
Goodmorning everyone, I need to know if there is a possibility to set elasticsearch as input (also as an output), parsing all the logs I had previously parsed with logstash. For example: I have parsed all my akamai lo… --- ## [Logstash-output-mongodb - bad file descriptor](https://discuss.elastic.co/t/logstash-output-mongodb-bad-file-descriptor/108542)
**Author:** [@blueren](https://discuss.elastic.co/u/blueren)\ **Replies:** 0\ **Last updated:** [November 21, 2017, 10:06am UTC](https://discuss.elastic.co/t/logstash-output-mongodb-bad-file-descriptor/108542 "2017-11-21T10:06:48Z")
I have my logstash config as below: output { if \[type\] == "netflow" { elasticsearch { hosts =\> \[ "${ELASTIFLOW\_ES\_HOST:127.0.0.1:9200}" \] user =\> "${ELASTIFLOW\_ES\_USER:elastic}" … --- ## [Multiple logstash instances for ecommerce](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515)
**Author:** [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\ **Replies:** 4\ **Last updated:** [November 21, 2017, 8:31am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-for-ecommerce/108515 "2017-11-21T08:31:27Z")
hi, i want to implement ELK for an ecommerce. Project is ready on php and mysql. I want to search product using EL. so i have to migrate mysql data to logstash index right. But there are lots of other conditions and co… --- ## [How to achieve Clusters in Logstash?](https://discuss.elastic.co/t/how-to-achieve-clusters-in-logstash/108387)
**Author:** [@akshay](https://discuss.elastic.co/u/akshay)\ **Replies:** 5\ **Last updated:** [November 21, 2017, 8:21am UTC](https://discuss.elastic.co/t/how-to-achieve-clusters-in-logstash/108387 "2017-11-21T08:21:23Z")
Hello, Following is the deployment scenario. We are using ELK stack (ElasticSearch,Logstash and Kibana for UI) Elastic Search will be deployed on 3 nodes which form one Cluster. We also want to Cluster Logstash which… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=869) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=871)