# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=452 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 453 --- ## [Problem in my conf file](https://discuss.elastic.co/t/problem-in-my-conf-file/206412)
**Author:** [@lgllrd](https://discuss.elastic.co/u/lgllrd)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 1:20pm UTC](https://discuss.elastic.co/t/problem-in-my-conf-file/206412 "2019-11-04T13:20:41Z")
Hi, I'm trying to start my logstash pipe configuration file but it tell me "command not found" : ''' /home/logstash/test.conf: ligne 3: \[/home/logstash/testdata.log\]: Aucun fichier ou dossier de ce type /home/logstash/… --- ## [Getting error when bulk json files trasfer from logstash to ELK](https://discuss.elastic.co/t/getting-error-when-bulk-json-files-trasfer-from-logstash-to-elk/206409)
**Author:** [@namali](https://discuss.elastic.co/u/namali)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 1:13pm UTC](https://discuss.elastic.co/t/getting-error-when-bulk-json-files-trasfer-from-logstash-to-elk/206409 "2019-11-04T13:13:50Z")
transferring more than 12000 same structured json files to ELK once. when I run logstash config file getting following error. \[2019-11-04T17:47:36,249\]\[ERROR\]\[org.logstash.Logstash \] java.lang.InternalError: java.io.… --- ## [Read JSON array file](https://discuss.elastic.co/t/read-json-array-file/206376)
**Author:** [@MMH](https://discuss.elastic.co/u/MMH)\ **Replies:** 2\ **Last updated:** [November 4, 2019, 12:58pm UTC](https://discuss.elastic.co/t/read-json-array-file/206376 "2019-11-04T12:58:48Z")
I am using Logstash 6.5. I want to read the JSON file, process it and end the execution of Logstash. The format of the input file is similar to the following: \[ { "id":"1", "value":"v1" }, { "id":"2", "v… --- ## [Logstash start parsing old data when restart happened](https://discuss.elastic.co/t/logstash-start-parsing-old-data-when-restart-happened/206347)
**Author:** [@Suresh\_Pal](https://discuss.elastic.co/u/Suresh_Pal)\ **Replies:** 1\ **Last updated:** [November 4, 2019, 12:42pm UTC](https://discuss.elastic.co/t/logstash-start-parsing-old-data-when-restart-happened/206347 "2019-11-04T12:42:03Z")
Hi Team, I'm facing the issue as logstash parses the old data whenever i restart the logstash service. I don't know why it happened but it impact my use case and logs. Can somebody help me to figure out the issue. I… --- ## [How to separate arriving syslog messages](https://discuss.elastic.co/t/how-to-separate-arriving-syslog-messages/206379)
**Author:** [@fim](https://discuss.elastic.co/u/fim)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 10:53am UTC](https://discuss.elastic.co/t/how-to-separate-arriving-syslog-messages/206379 "2019-11-04T10:53:23Z")
Logstash instance is listening on a single port. I tried to separate incoming syslog messages as follows: Two ways with "IN" and "==" operator but logstash doesn't catch one of the IFs. input { syslog { po… --- ## [Picked up from multi-line logs using grok pattern](https://discuss.elastic.co/t/picked-up-from-multi-line-logs-using-grok-pattern/206365)
**Author:** [@kavindu\_manaram](https://discuss.elastic.co/u/kavindu_manaram)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 10:19am UTC](https://discuss.elastic.co/t/picked-up-from-multi-line-logs-using-grok-pattern/206365 "2019-11-04T10:19:10Z")
Here is my one log segment which file beat picked. 2019-10-23 09:53:43.2218 - \[yyy\] \[EF\] Opened connection at 10/23/2019 9:53:43 AM +05:30 2019-10-23 09:53:43.2218 - \[yyy\] \[EF\] \*SELECT Count() FROM INFORMATION\_SCHEMA … --- ## [Connection not establishing between logstash and elastic search](https://discuss.elastic.co/t/connection-not-establishing-between-logstash-and-elastic-search/206333)
**Author:** [@Gowtham\_Gandham](https://discuss.elastic.co/u/Gowtham_Gandham)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 6:14am UTC](https://discuss.elastic.co/t/connection-not-establishing-between-logstash-and-elastic-search/206333 "2019-11-04T06:14:13Z")
version: '3.5' services: elasticsearch: build: context: elasticsearch/ volumes: - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro ports: - '9200:9200' - '9300:9300' … --- ## [Any solutions ? Please share me](https://discuss.elastic.co/t/any-solutions-please-share-me/206263)
**Author:** [@pbona](https://discuss.elastic.co/u/pbona)\ **Replies:** 3\ **Last updated:** [November 3, 2019, 11:18pm UTC](https://discuss.elastic.co/t/any-solutions-please-share-me/206263 "2019-11-03T23:18:03Z")
\[root@elastic logstash\]# tail -f /var/log/logstash/logstash-plain.log \[2019-10-30T04:39:22,142\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\ **Author:** [@mbrown007](https://discuss.elastic.co/u/mbrown007)\ **Replies:** 12\ **Last updated:** [November 3, 2019, 11:05pm UTC](https://discuss.elastic.co/t/date-matching/205333 "2019-11-03T23:05:39Z") I have a CSV and I have got it indexed in ELASTICSEARCH. The issue is that I have the following set up and how can I verify that the dates are timestamps? # Sample Logstash configuration for creating a simple # Beats -\> … --- ## [Configure logstash input jdbc plugin to disable autocommit for postgresql connection](https://discuss.elastic.co/t/configure-logstash-input-jdbc-plugin-to-disable-autocommit-for-postgresql-connection/206307)
**Author:** [@turntwo463](https://discuss.elastic.co/u/turntwo463)\ **Replies:** 0\ **Last updated:** [November 3, 2019, 11:02pm UTC](https://discuss.elastic.co/t/configure-logstash-input-jdbc-plugin-to-disable-autocommit-for-postgresql-connection/206307 "2019-11-03T23:02:55Z")
I've come across the issue where the fetch size cannot be applied for postgresql jdbc connections since the connections default to autocommit, as reported here: Looking at the documentation I see that under the sequel… --- ## [Query on Grok & Mutate plugins with Logstash](https://discuss.elastic.co/t/query-on-grok-mutate-plugins-with-logstash/206293)
**Author:** [@angryninja](https://discuss.elastic.co/u/angryninja)\ **Replies:** 0\ **Last updated:** [November 3, 2019, 2:43pm UTC](https://discuss.elastic.co/t/query-on-grok-mutate-plugins-with-logstash/206293 "2019-11-03T14:43:55Z")
Hello, I am trying to match a pattern ES document which is being ingested via Logstash. Pattern is duration defined in hours like PT(Hours)H. I am using following as part of the logstash conf but I don't see any field… --- ## [Parsing problem - one or more space](https://discuss.elastic.co/t/parsing-problem-one-or-more-space/206285)
**Author:** [@Jan\_Kaspar](https://discuss.elastic.co/u/Jan_Kaspar)\ **Replies:** 1\ **Last updated:** [November 3, 2019, 2:18pm UTC](https://discuss.elastic.co/t/parsing-problem-one-or-more-space/206285 "2019-11-03T14:18:25Z")
Hi All, I would have a question about parsing the logs from DNS server logs. Sample of DNS logs: message: 3. 11. 2019 12:33:14 0958 PACKET 000000C97B8371C0 UDP Snd 192.168.5.202 4cbf Q \[0000 NOERROR\] SOA … --- ## [Monitoring templates](https://discuss.elastic.co/t/monitoring-templates/206290)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 0\ **Last updated:** [November 3, 2019, 1:51pm UTC](https://discuss.elastic.co/t/monitoring-templates/206290 "2019-11-03T13:51:37Z")
I have Logstash running version 7.4.0. It is connecting to an Elasticsearch cluster running 7.4.0. I am not sure why the following issue is showing up. \[2019-11-03T05:34:11,762\]\[WARN \]\[logstash.outputs.elasticsearch\] Yo… --- ## [How to improve the speed of Logstash output to Elasticsearch?](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254)
**Author:** [@StruggleYang](https://discuss.elastic.co/u/StruggleYang)\ **Replies:** 7\ **Last updated:** [November 3, 2019, 10:10am UTC](https://discuss.elastic.co/t/how-to-improve-the-speed-of-logstash-output-to-elasticsearch/206254 "2019-11-03T10:10:46Z")
I have a lot of application logs to collect, and data conversion is like, filebeat =\> kafka =\> logstash =\> elasticsearch ,Everything is smooth, But sometimes the application log will suddenly increase, such as when the u… --- ## [Logs from hp-ux - logstash or 3rd party solution?](https://discuss.elastic.co/t/logs-from-hp-ux-logstash-or-3rd-party-solution/206279)
**Author:** [@Abderaouf\_Omari](https://discuss.elastic.co/u/Abderaouf_Omari)\ **Replies:** 0\ **Last updated:** [November 3, 2019, 10:10am UTC](https://discuss.elastic.co/t/logs-from-hp-ux-logstash-or-3rd-party-solution/206279 "2019-11-03T10:10:01Z")
Hi all, i hope that one of you could help me, my need is to centralize logs, so iam using elasticsearch and kibana ( 7.3.1 version ). and from clients side iam using filebeat to transfer the logs directly to Elasticsear… --- ## [Output](https://discuss.elastic.co/t/output/206253)
**Author:** [@Jan\_Kaspar](https://discuss.elastic.co/u/Jan_Kaspar)\ **Replies:** 0\ **Last updated:** [November 2, 2019, 6:02pm UTC](https://discuss.elastic.co/t/output/206253 "2019-11-02T18:02:40Z")
Hi, I have a question to Output plugin. Curently I am using filebat to parse mulitple rsyslog files. Rsyslog files are divided by device. every device logs in to separate directory. What is necesary to split index fi… --- ## [Where is rest of the memory?](https://discuss.elastic.co/t/where-is-rest-of-the-memory/205537)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 0\ **Last updated:** [October 28, 2019, 6:52pm UTC](https://discuss.elastic.co/t/where-is-rest-of-the-memory/205537 "2019-10-28T18:52:58Z")
I am trying to install Logstash 7.4.0 from the RPM. I have a 6.8.3 RPM already installed and the process is running. I am getting the following error: Downloading packages: Delta RPMs disabled because /usr/bin/applydelt… --- ## [S3 input \[ERROR\]\[logstash.pipeline \] A plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/s3-input-error-logstash-pipeline-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/206234)
**Author:** [@prabhakar\_talari](https://discuss.elastic.co/u/prabhakar_talari)\ **Replies:** 0\ **Last updated:** [November 2, 2019, 10:33am UTC](https://discuss.elastic.co/t/s3-input-error-logstash-pipeline-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/206234 "2019-11-02T10:33:36Z")
All, we are getting below frequent errors in logstash log file Error: couldn't parse document: Premature end of file. Exception: RuntimeError Stack: nokogiri/XmlSaxParserContext.java:258:in parse\_with' /usr/share/log… --- ## [Configuring Logstash for Elasticsearch Services in the Cloud](https://discuss.elastic.co/t/configuring-logstash-for-elasticsearch-services-in-the-cloud/206215)
**Author:** [@sarahvo](https://discuss.elastic.co/u/sarahvo)\ **Replies:** 0\ **Last updated:** [November 1, 2019, 8:43pm UTC](https://discuss.elastic.co/t/configuring-logstash-for-elasticsearch-services-in-the-cloud/206215 "2019-11-01T20:43:03Z")
Hi, I'm using Elasticsearch Services on Azure and I'm trying to configure logstash to point to my elasticsearch instance in the cloud. I was reading the documentation given here: https://www.elastic.co/guide/en/logstash/… --- ## [How to send same data to multiple elastic clusters with logstash output](https://discuss.elastic.co/t/how-to-send-same-data-to-multiple-elastic-clusters-with-logstash-output/206196)
**Author:** [@prabhakar\_talari](https://discuss.elastic.co/u/prabhakar_talari)\ **Replies:** 4\ **Last updated:** [November 1, 2019, 5:39pm UTC](https://discuss.elastic.co/t/how-to-send-same-data-to-multiple-elastic-clusters-with-logstash-output/206196 "2019-11-01T17:39:07Z")
All, We have requirement where we need to read data from kafka topics from logstash and send all data to cluster 1 and few data to cluster 2. Please let us know is it possible and it would really help us if any one prov… --- ## [How can i connect filebeat with logstash?](https://discuss.elastic.co/t/how-can-i-connect-filebeat-with-logstash/205954)
**Author:** [@Zuleyma\_Espinoza](https://discuss.elastic.co/u/Zuleyma_Espinoza)\ **Replies:** 3\ **Last updated:** [November 1, 2019, 5:24pm UTC](https://discuss.elastic.co/t/how-can-i-connect-filebeat-with-logstash/205954 "2019-11-01T17:24:48Z")
Hi, I'm new on these topics. I'm working with ELK+Filebeat, I follow the steps of documentation but I can't connect filebeat with logstash, can some person help me, with tips or something else? The version of ELK+Fileb… --- ## [How extract data with logstash filter xml from a complex xml](https://discuss.elastic.co/t/how-extract-data-with-logstash-filter-xml-from-a-complex-xml/205905)
**Author:** [@Leandro1](https://discuss.elastic.co/u/Leandro1)\ **Replies:** 2\ **Last updated:** [November 1, 2019, 5:17pm UTC](https://discuss.elastic.co/t/how-extract-data-with-logstash-filter-xml-from-a-complex-xml/205905 "2019-11-01T17:17:13Z")
Hello, I need to extract the data returned from an xml in an http input The xml Returned is: \ \ **Author:** [@jalu](https://discuss.elastic.co/u/jalu)\ **Replies:** 2\ **Last updated:** [November 1, 2019, 4:21pm UTC](https://discuss.elastic.co/t/logstash-to-convert-all-numeric-field-to-float/206100 "2019-11-01T16:21:04Z") Hi, I recently start to work on a JSON\> logstash \> influx pipeline , I wonder if there is a way to make logstash to read every numeric value from JSON into float and write them into Influx. For now, some of the fields ar… --- ## [grok/KV filter for multiple values](https://discuss.elastic.co/t/grok-kv-filter-for-multiple-values/206109)
**Author:** [@Senthil\_ak](https://discuss.elastic.co/u/Senthil_ak)\ **Replies:** 6\ **Last updated:** [November 1, 2019, 4:17pm UTC](https://discuss.elastic.co/t/grok-kv-filter-for-multiple-values/206109 "2019-11-01T16:17:15Z")
For the nginx extended logs, I am using below grok pattern and KV pattern. But the value sometimes comes as '-' , some time real values and hypen + value. %{GREEDYDATA:\[nginx\]\[access\]\[message\]}' Log - 3 types rt=0.006 … --- ## [Logstash \<-\> Logstash using lumberjack. Beats input can not be passed through?](https://discuss.elastic.co/t/logstash-logstash-using-lumberjack-beats-input-can-not-be-passed-through/206184)
**Author:** [@Dars](https://discuss.elastic.co/u/Dars)\ **Replies:** 0\ **Last updated:** [November 1, 2019, 3:10pm UTC](https://discuss.elastic.co/t/logstash-logstash-using-lumberjack-beats-input-can-not-be-passed-through/206184 "2019-11-01T15:10:06Z")
Hi, I have an issues with lumberjack and Logstash to Logstash communication. I have 2 sites, and have a logstash on each site. They are connected with a VPN connection over the internet. I can get the Lumberjack connec… --- ## [Using stdin to ingest multiple files at once](https://discuss.elastic.co/t/using-stdin-to-ingest-multiple-files-at-once/205314)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 2\ **Last updated:** [November 1, 2019, 2:26pm UTC](https://discuss.elastic.co/t/using-stdin-to-ingest-multiple-files-at-once/205314 "2019-11-01T14:26:13Z")
I have added stdin { } into the input of my logstash config file. When running it I've been trying to run it with one config file, one call but for 6 files since it is all being ingested into one index. How can I do this… --- ## [Logstash-output-google\_cloud\_storage plugin does not fail when expected](https://discuss.elastic.co/t/logstash-output-google-cloud-storage-plugin-does-not-fail-when-expected/204500)
**Author:** [@Niki\_Maslarski](https://discuss.elastic.co/u/Niki_Maslarski)\ **Replies:** 3\ **Last updated:** [November 1, 2019, 2:12pm UTC](https://discuss.elastic.co/t/logstash-output-google-cloud-storage-plugin-does-not-fail-when-expected/204500 "2019-11-01T14:12:44Z")
Hi all, I'm using logstash to store events/logs in to google cloud storage via the logstash. Using this plugin https://www.elastic.co/guide/en/logstash/current/plugins-outputs-google\_cloud\_storage.html However recentl… --- ## [Shutdown Logstash when using pipeline-to-pipelines communication](https://discuss.elastic.co/t/shutdown-logstash-when-using-pipeline-to-pipelines-communication/205431)
**Author:** [@glenacota](https://discuss.elastic.co/u/glenacota)\ **Replies:** 1\ **Last updated:** [November 1, 2019, 1:22pm UTC](https://discuss.elastic.co/t/shutdown-logstash-when-using-pipeline-to-pipelines-communication/205431 "2019-11-01T13:22:59Z")
Hi! I would like to refactor my pipelines by using one of the pipeline-to-pipeline architecture patterns (the forked path pattern). The input of the upstream pipeline is a one-time query to an Elasticsearch cluster, usi… --- ## [Dynamic Prefix with s3 input plugin](https://discuss.elastic.co/t/dynamic-prefix-with-s3-input-plugin/206167)
**Author:** [@rhishi127](https://discuss.elastic.co/u/rhishi127)\ **Replies:** 0\ **Last updated:** [November 1, 2019, 1:04pm UTC](https://discuss.elastic.co/t/dynamic-prefix-with-s3-input-plugin/206167 "2019-11-01T13:04:54Z")
Hi, I am using logstash version 6.6.0 I want to pass current year value as variable in a prefix of s3 input plugin as we can pass in s3 output plugin prefix can anyone help --- ## [Ambiguous field reference for object field](https://discuss.elastic.co/t/ambiguous-field-reference-for-object-field/206166)
**Author:** [@bruce289](https://discuss.elastic.co/u/bruce289)\ **Replies:** 0\ **Last updated:** [November 1, 2019, 1:01pm UTC](https://discuss.elastic.co/t/ambiguous-field-reference-for-object-field/206166 "2019-11-01T13:01:14Z")
Hi, I'm attempting to split a field which looks like: { "VERSION" : "version-uk-v2.1.0-global-v2.2.0" } into the following: { "host" : { "version" : { "uk" : "v2.1.0", "global" : "v2.2.0" } … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=451) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=453)