# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=505 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 506 --- ## [Output to an Elasticsearch cluster - Details?](https://discuss.elastic.co/t/output-to-an-elasticsearch-cluster-details/192058)
**Author:** [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Replies:** 0\ **Last updated:** [July 24, 2019, 2:47pm UTC](https://discuss.elastic.co/t/output-to-an-elasticsearch-cluster-details/192058 "2019-07-24T14:47:19Z")
There seems to be some ambiguity in relation to the concept of a logstasher outputting to an ES cluster. There is very little mention of the concept in the logstash documentation that I could find. The "hosts" setting … --- ## [Custom plugin with intermittent uploading](https://discuss.elastic.co/t/custom-plugin-with-intermittent-uploading/192056)
**Author:** [@Ram\_Sharan\_Mittal](https://discuss.elastic.co/u/Ram_Sharan_Mittal)\ **Replies:** 0\ **Last updated:** [July 24, 2019, 2:41pm UTC](https://discuss.elastic.co/t/custom-plugin-with-intermittent-uploading/192056 "2019-07-24T14:41:53Z")
I have a file input and push to our elasticsearch custom plugin which oauth and some extra header which our elasticSearch service need. Like to reduce load on elasticserach so logstash should push file's content after a … --- ## [Elasticsearch to SQL](https://discuss.elastic.co/t/elasticsearch-to-sql/191853)
**Author:** [@mroldao](https://discuss.elastic.co/u/mroldao)\ **Replies:** 6\ **Last updated:** [July 24, 2019, 2:18pm UTC](https://discuss.elastic.co/t/elasticsearch-to-sql/191853 "2019-07-24T14:18:13Z")
Hi, Im new in Elasticsearch. I have a project to collect syslogs from some servers using custom grok filter. I can write different logs to different indexes and works fine. Kibana reporting is a great tool, but my go… --- ## [JSON Array - Logstash](https://discuss.elastic.co/t/json-array-logstash/191819)
**Author:** [@shagun](https://discuss.elastic.co/u/shagun)\ **Replies:** 5\ **Last updated:** [July 24, 2019, 1:30pm UTC](https://discuss.elastic.co/t/json-array-logstash/191819 "2019-07-24T13:30:02Z")
I am trying to parse the nested JSON. I tried to use the Split filter for it but it results in a matric form. { "School": { "Details1": \[{ "Name": "a", "Class": "1" }, { "Name": "b", … --- ## [Unable to install logstash-kinesis-input](https://discuss.elastic.co/t/unable-to-install-logstash-kinesis-input/191965)
**Author:** [@uma\_rengasamy](https://discuss.elastic.co/u/uma_rengasamy)\ **Replies:** 1\ **Last updated:** [July 24, 2019, 12:32pm UTC](https://discuss.elastic.co/t/unable-to-install-logstash-kinesis-input/191965 "2019-07-24T12:32:30Z")
'logstash-input-kinesis' plugin Installation hangs on RHEL 7 . On listing plugin it throws below error Bundler::GemNotFound: Could not find gem 'logstash-input-kinesis java' in any of the gem sources listed in your Gem… --- ## [Using aggregate filter for multi task\_id](https://discuss.elastic.co/t/using-aggregate-filter-for-multi-task-id/191564)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 4\ **Last updated:** [July 24, 2019, 12:28pm UTC](https://discuss.elastic.co/t/using-aggregate-filter-for-multi-task-id/191564 "2019-07-24T12:28:24Z")
hi all I am using jmx plugin to ship WebLogic MBean into ELK. for this purpose, following script is using in logstash. input { jmx { path =\> "C:\\logstash\\jmxconf" polling\_frequency =\> 5 type =\> "jmx" nb\_thread … --- ## [Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@satyam2593](https://discuss.elastic.co/u/satyam2593)\ **Replies:** 1\ **Last updated:** [July 24, 2019, 11:53am UTC](https://discuss.elastic.co/t/pipeline-aborted-due-to-error-pipeline-id-main-exception-regexperror-empty-char-class-duplicate-tcp-syn-failed-to-locate-egress-interface-invalid-transport-field-no-matching-connection-dns-response-dns-query/192000 "2019-07-24T11:53:57Z") \[ERROR\] 2019-07-24 10:52:19.002 \[\[main\]-pipeline-manager\] javapipeline - Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@elitzur\_e](https://discuss.elastic.co/u/elitzur_e)\ **Replies:** 1\ **Last updated:** [July 24, 2019, 11:47am UTC](https://discuss.elastic.co/t/output-plugin-for-sql-server-for-latest-version-of-logstash/192011 "2019-07-24T11:47:01Z") Hi, ive been looking to use logstash to output to sql server. i cant seem to find an offcial plugin or maybe im misunderstanding something. would appreciate some directions here, thanks :slight\_smile: --- ## [Logstash Email error : email - Something happen while delivering an email {:exception=\>#\ **Author:** [@irobot678](https://discuss.elastic.co/u/irobot678)\ **Replies:** 2\ **Last updated:** [July 24, 2019, 11:40am UTC](https://discuss.elastic.co/t/logstash-email-error-email-something-happen-while-delivering-an-email-exception-net-must-issue-a-starttls-command-first/191861 "2019-07-24T11:40:48Z") Hi , I am using elk stack of version 7.1.1 + X-pack on unbuntu 14.04 . i want to trigger a alert whenever the message contains "Exception " keyword in it. logstash.yml input { file { path =\> "/home/a.log" s… --- ## [How to use translate plugin with ES index for lookup data](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743)
**Author:** [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Replies:** 5\ **Last updated:** [July 24, 2019, 8:02am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743 "2019-07-24T08:02:23Z")
I have read about translate filter functionality in logstash which could be used for lookup event data with that of a file( csv, json) which we can define in destination\_path yml file. Can I use an elastisearch index ( … --- ## [Logstash jdbc connection with Windows Authentication](https://discuss.elastic.co/t/logstash-jdbc-connection-with-windows-authentication/191963)
**Author:** [@AlexisC](https://discuss.elastic.co/u/AlexisC)\ **Replies:** 0\ **Last updated:** [July 24, 2019, 7:00am UTC](https://discuss.elastic.co/t/logstash-jdbc-connection-with-windows-authentication/191963 "2019-07-24T07:00:24Z")
Hello, I'm trying to index data from a SQL Server to Elasticsearch via the plugin Logstash JDBC. I'm working on a Windows Environment with Elastic version 6.4.2 Here you can find the configuration file for Logstash: i… --- ## [CSV Output not what expected](https://discuss.elastic.co/t/csv-output-not-what-expected/191629)
**Author:** [@Ruben\_Aguilar](https://discuss.elastic.co/u/Ruben_Aguilar)\ **Replies:** 4\ **Last updated:** [July 24, 2019, 6:52am UTC](https://discuss.elastic.co/t/csv-output-not-what-expected/191629 "2019-07-24T06:52:26Z")
Hello, I have to parse an XML file and extract certain variables from it. The parsing part is solved. And from the XML filter I get three variables in this way: "DX\_METERNAME" =\> \[ \[0\] "DX\_KILOM", \[1\] "DX\_KILOM", \[2… --- ## [Output data to elasticsearch with null\_pointer\_exception](https://discuss.elastic.co/t/output-data-to-elasticsearch-with-null-pointer-exception/191756)
**Author:** [@zxyyxzzxy001](https://discuss.elastic.co/u/zxyyxzzxy001)\ **Replies:** 2\ **Last updated:** [July 24, 2019, 2:59am UTC](https://discuss.elastic.co/t/output-data-to-elasticsearch-with-null-pointer-exception/191756 "2019-07-24T02:59:31Z")
I want to use logstash transfer data from mysql to elasticsearch. I use dynamic template to define IK tokenizer which is used for chinese, but some data transfer successfully, others failed with : response code: 500 ({"t… --- ## [Only String and Array types are splittable. field:splitedJson is of type = NilClass](https://discuss.elastic.co/t/only-string-and-array-types-are-splittable-field-splitedjson-is-of-type-nilclass/191725)
**Author:** [@TsuWeiQuan](https://discuss.elastic.co/u/TsuWeiQuan)\ **Replies:** 2\ **Last updated:** [July 24, 2019, 1:53am UTC](https://discuss.elastic.co/t/only-string-and-array-types-are-splittable-field-splitedjson-is-of-type-nilclass/191725 "2019-07-24T01:53:09Z")
Hello, I have an array of JSON and i can parse them out nicely with logstash. However, i realized that the logs are flooded with \[2019-07-23T10:35:06,859\]\[WARN \]\[logstash.filters.split \] Only String and Array types … --- ## [Filebeat to logstash not working](https://discuss.elastic.co/t/filebeat-to-logstash-not-working/191926)
**Author:** [@nila](https://discuss.elastic.co/u/nila)\ **Replies:** 0\ **Last updated:** [July 24, 2019, 12:10am UTC](https://discuss.elastic.co/t/filebeat-to-logstash-not-working/191926 "2019-07-24T00:10:49Z")
I have configured ELK cluster basic version and enabled TLS/SSL. it works fine until logstash. The communication between filebeat to logstash is not working. I dont see any errors in filebeat. Please help if SSL config… --- ## [Logstash 7.2 can not match date on JSON data from Cisco telemetry](https://discuss.elastic.co/t/logstash-7-2-can-not-match-date-on-json-data-from-cisco-telemetry/191557)
**Author:** [@tlittirut](https://discuss.elastic.co/u/tlittirut)\ **Replies:** 7\ **Last updated:** [July 23, 2019, 11:59pm UTC](https://discuss.elastic.co/t/logstash-7-2-can-not-match-date-on-json-data-from-cisco-telemetry/191557 "2019-07-23T23:59:55Z")
Hello ! I'm a newbie on logstash and trying to decode cisco telemetry data in json format. My code in logstash are the following: filter { if \[type\] == "xjson" { json { source =\> "message" } mutate { remove\_field… --- ## [Trouble with conditionals in aggregate filter](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910)
**Author:** [@N\_Antech](https://discuss.elastic.co/u/N_Antech)\ **Replies:** 6\ **Last updated:** [July 23, 2019, 11:09pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910 "2019-07-23T23:09:04Z")
I am working to pull my data into elasticsearch using logstash and running into some trouble. I'll do my best to explain the situation: I'm pulling data via jdbc and everything seems to be working with the exception of… --- ## [Only one of the 2 TCP inputs is working](https://discuss.elastic.co/t/only-one-of-the-2-tcp-inputs-is-working/191112)
**Author:** [@nsm](https://discuss.elastic.co/u/nsm)\ **Replies:** 5\ **Last updated:** [July 23, 2019, 5:52pm UTC](https://discuss.elastic.co/t/only-one-of-the-2-tcp-inputs-is-working/191112 "2019-07-23T17:52:18Z")
I have two logstash configuration files under one folder. Following are the configurations for both files: File 1: input { tcp { port =\> 50002 add\_field =\> { log\_type =\> 'panw' }… --- ## [TCp issue logstash](https://discuss.elastic.co/t/tcp-issue-logstash/191868)
**Author:** [@alfred\_vikas\_paul](https://discuss.elastic.co/u/alfred_vikas_paul)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 4:38pm UTC](https://discuss.elastic.co/t/tcp-issue-logstash/191868 "2019-07-23T16:38:05Z")
Hi Every one, I am sending json data in loop and send it via tcp . but my logstash misses some data for example . if in loop of 500 logs . it recieve 100 and that to random value. i have no clue please let me know how … --- ## [How to configure pipeline-to-pipeline communication in central pipeline management?](https://discuss.elastic.co/t/how-to-configure-pipeline-to-pipeline-communication-in-central-pipeline-management/191867)
**Author:** [@ReneSperling](https://discuss.elastic.co/u/ReneSperling)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 4:35pm UTC](https://discuss.elastic.co/t/how-to-configure-pipeline-to-pipeline-communication-in-central-pipeline-management/191867 "2019-07-23T16:35:06Z")
Hello What I want to do: I have multiple programs sending their logs via Filebeat to logstash on one single port (using multiple ports is not an option for me, sorry). The Input shall only work like a switch where I e… --- ## ["KILLED" response](https://discuss.elastic.co/t/killed-response/191864)
**Author:** [@Kevin\_Juliano](https://discuss.elastic.co/u/Kevin_Juliano)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 4:09pm UTC](https://discuss.elastic.co/t/killed-response/191864 "2019-07-23T16:09:59Z")
I've been working on indexing a huge number of rows on my database, I've adjusted the heap size up to 45GB, it is running but I am getting: \[INFO \] 2019-07-23 14:00:23.706 \[Api Webserver\] agent - Successfully started Lo… --- ## [Logstash fails to create netflow dashboard in kibana](https://discuss.elastic.co/t/logstash-fails-to-create-netflow-dashboard-in-kibana/191850)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 3:05pm UTC](https://discuss.elastic.co/t/logstash-fails-to-create-netflow-dashboard-in-kibana/191850 "2019-07-23T15:05:11Z")
STGRESZC-M-F2Q5:~ stgreszc$ logstash --modules netflow --setup -M netflow.var.input.udp.port=9996 -M netflow.var.kibana.host="elk-alln-001:5601" -M netflow.var.elasticsearch.hosts="elk-alln-001:9200" Thread.exclusive is… --- ## [Logstash is not writing latest value in jdbc Sincedb](https://discuss.elastic.co/t/logstash-is-not-writing-latest-value-in-jdbc-sincedb/186306)
**Author:** [@Tabish0009](https://discuss.elastic.co/u/Tabish0009)\ **Replies:** 10\ **Last updated:** [July 23, 2019, 2:58pm UTC](https://discuss.elastic.co/t/logstash-is-not-writing-latest-value-in-jdbc-sincedb/186306 "2019-07-23T14:58:12Z")
I am using 5.x version of Logstash and 7.x version of Elasticsearch and Kibana, the issue I am facing is even after setting the path for the sincedb, logstash is not writing the timestamp till the point reading has been … --- ## [Logstash fails when running the service by systemctl](https://discuss.elastic.co/t/logstash-fails-when-running-the-service-by-systemctl/191838)
**Author:** [@mhsh64](https://discuss.elastic.co/u/mhsh64)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 2:30pm UTC](https://discuss.elastic.co/t/logstash-fails-when-running-the-service-by-systemctl/191838 "2019-07-23T14:30:09Z")
Hello, I get the following line when running logstash: error=\>#\, :back… --- ## [Uberagent](https://discuss.elastic.co/t/uberagent/191832)
**Author:** [@jonathan01](https://discuss.elastic.co/u/jonathan01)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 2:08pm UTC](https://discuss.elastic.co/t/uberagent/191832 "2019-07-23T14:08:44Z")
We are wanting to utilize the uberagent feature but we are a little confused on how to set up. We looked at the documentation and it says to run code curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/… --- ## [Use filebeat added field for logstash conditions](https://discuss.elastic.co/t/use-filebeat-added-field-for-logstash-conditions/191799)
**Author:** [@Wellguys](https://discuss.elastic.co/u/Wellguys)\ **Replies:** 2\ **Last updated:** [July 23, 2019, 2:07pm UTC](https://discuss.elastic.co/t/use-filebeat-added-field-for-logstash-conditions/191799 "2019-07-23T14:07:24Z")
Hello. I'm having a problem understanding how exactly I am supposed to write the conditions in logstash with regards to fields I added to filebeat. Here is my filebeat inputs filebeat.inputs: - type: log enabled: tr… --- ## [Logstash not outputting to file](https://discuss.elastic.co/t/logstash-not-outputting-to-file/191809)
**Author:** [@a.gavric](https://discuss.elastic.co/u/a.gavric)\ **Replies:** 1\ **Last updated:** [July 23, 2019, 1:43pm UTC](https://discuss.elastic.co/t/logstash-not-outputting-to-file/191809 "2019-07-23T13:43:45Z")
Hello everyone, im triying to set up to logstash outputs one that will output to elastic (which runs fine) and one that would simply output to a json file on file storage, i have attempted to different configs Config 1: … --- ## [Netflow one-time dashboard setup via docker?](https://discuss.elastic.co/t/netflow-one-time-dashboard-setup-via-docker/191824)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 0\ **Last updated:** [July 23, 2019, 1:26pm UTC](https://discuss.elastic.co/t/netflow-one-time-dashboard-setup-via-docker/191824 "2019-07-23T13:26:33Z")
I'm trying to run the new netflow module in logstash version 7.2 using docker. It is working and I can see the index and data in kibana, however how do I pass the one-time "--setup" to kibana via logstash on docker cont… --- ## [Filter IP addresses by subnet](https://discuss.elastic.co/t/filter-ip-addresses-by-subnet/191663)
**Author:** [@Nadir006](https://discuss.elastic.co/u/Nadir006)\ **Replies:** 4\ **Last updated:** [July 23, 2019, 12:29pm UTC](https://discuss.elastic.co/t/filter-ip-addresses-by-subnet/191663 "2019-07-23T12:29:54Z")
Hi all, I need to enrich my firewall logs with the subnet address, subnet name and subnet location. I have a csv file with all these informations. Here is a similar case : https://discuss.elastic.co/t/cidr-subnet-relati… --- ## [WinlogBeat to Logstash integration](https://discuss.elastic.co/t/winlogbeat-to-logstash-integration/191669)
**Author:** [@akhilsharma.in](https://discuss.elastic.co/u/akhilsharma.in)\ **Replies:** 3\ **Last updated:** [July 23, 2019, 12:02pm UTC](https://discuss.elastic.co/t/winlogbeat-to-logstash-integration/191669 "2019-07-23T12:02:09Z")
Hi Team - Winlogbeat is sending events to Logstash at 5044. I have below questions: In winlogbeat, if I set ignore\_older: 10s Does this mean that Winlogbeat will check if there are any new events before 10 seconds, … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=504) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=506)