# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=398 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 399 --- ## [Offline Plugin update installation fails](https://discuss.elastic.co/t/offline-plugin-update-installation-fails/221255)
**Author:** [@Christian\_Hubinger](https://discuss.elastic.co/u/Christian_Hubinger)\ **Replies:** 0\ **Last updated:** [February 27, 2020, 2:27pm UTC](https://discuss.elastic.co/t/offline-plugin-update-installation-fails/221255 "2020-02-27T14:27:37Z")
Hi, I've developed an extended version of the output-file-plugin (see this PR https://github.com/logstash-plugins/logstash-output-file/pull/85) and i want to ship this with our logstash configuration RPM we are building… --- ## [How to convert comma to ; in field](https://discuss.elastic.co/t/how-to-convert-comma-to-in-field/220869)
**Author:** [@vpolizki](https://discuss.elastic.co/u/vpolizki)\ **Replies:** 2\ **Last updated:** [February 27, 2020, 11:23am UTC](https://discuss.elastic.co/t/how-to-convert-comma-to-in-field/220869 "2020-02-27T11:23:02Z")
Hi i want to collect csv file, but some field include comma in text i want to change it to ; --- ## [Handling / dropping files according to previously handled files](https://discuss.elastic.co/t/handling-dropping-files-according-to-previously-handled-files/220799)
**Author:** [@AxelR](https://discuss.elastic.co/u/AxelR)\ **Replies:** 5\ **Last updated:** [February 27, 2020, 10:25am UTC](https://discuss.elastic.co/t/handling-dropping-files-according-to-previously-handled-files/220799 "2020-02-27T10:25:35Z")
I'm having an issue on how to deal with a batch of files. The lines of each file contain different pieces of information: the first line acts as a header (with a recognizable prefix) and includes (among other things)… --- ## [Logstash shuts down after a few seconds](https://discuss.elastic.co/t/logstash-shuts-down-after-a-few-seconds/221195)
**Author:** [@PRAV01](https://discuss.elastic.co/u/PRAV01)\ **Replies:** 3\ **Last updated:** [February 27, 2020, 10:19am UTC](https://discuss.elastic.co/t/logstash-shuts-down-after-a-few-seconds/221195 "2020-02-27T10:19:53Z")
Hi, i'm using Logstash 7.6.0 and i am trying to make a function where i read logs on port 3003. to do so i installed the http output plugin and created a file called plugin.conf in /usr/share/logstash/config as so: outp… --- ## [Getting "Failed to flush outgoing items {exception=\>WebHDFS::ServerError}" while using output plugin 'logstash.outputs.webhdfs'](https://discuss.elastic.co/t/getting-failed-to-flush-outgoing-items-exception-webhdfs-servererror-while-using-output-plugin-logstash-outputs-webhdfs/221162)
**Author:** [@hits.elastic](https://discuss.elastic.co/u/hits.elastic)\ **Replies:** 0\ **Last updated:** [February 27, 2020, 7:50am UTC](https://discuss.elastic.co/t/getting-failed-to-flush-outgoing-items-exception-webhdfs-servererror-while-using-output-plugin-logstash-outputs-webhdfs/221162 "2020-02-27T07:50:19Z")
Hi, I am using 'logstash.outputs.webhdfs' logstash output plugin. Below is my logstash configuration input { file { type =\> "file1" path =\> "/var/log/logstash/logstash-1/\*" start\_position =\> "beginni… --- ## [How sincedb works when using read mode for the file input plugin](https://discuss.elastic.co/t/how-sincedb-works-when-using-read-mode-for-the-file-input-plugin/221163)
**Author:** [@akwoprosper](https://discuss.elastic.co/u/akwoprosper)\ **Replies:** 0\ **Last updated:** [February 27, 2020, 7:51am UTC](https://discuss.elastic.co/t/how-sincedb-works-when-using-read-mode-for-the-file-input-plugin/221163 "2020-02-27T07:51:16Z")
Hello everyone, I just want to find out the behavior of the sincedb when using the read mode of the file input plugin. As a matter of fact, I just want to confirm that the current read position is logged only after a f… --- ## [Split multi- nested XML in Logstash](https://discuss.elastic.co/t/split-multi-nested-xml-in-logstash/220077)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 11\ **Last updated:** [February 27, 2020, 6:40am UTC](https://discuss.elastic.co/t/split-multi-nested-xml-in-logstash/220077 "2020-02-27T06:40:06Z")
Hi All, Here's a sample of my xml data \ \ \120778\ \ Average Response Time (ms) \ \… --- ## [Logstash shutdown automatically](https://discuss.elastic.co/t/logstash-shutdown-automatically/221139)
**Author:** [@Seetharaman\_K](https://discuss.elastic.co/u/Seetharaman_K)\ **Replies:** 0\ **Last updated:** [February 27, 2020, 4:07am UTC](https://discuss.elastic.co/t/logstash-shutdown-automatically/221139 "2020-02-27T04:07:50Z")
Hi All , Our Logstash got shutdown automatically. Logstash Log above is the Log of the logstash before it got shut down. i could not see much information . can any one help after seeing the log. --- ## [Logstash pipeline systax errors after 7.6.0](https://discuss.elastic.co/t/logstash-pipeline-systax-errors-after-7-6-0/220903)
**Author:** [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Replies:** 13\ **Last updated:** [February 27, 2020, 1:09am UTC](https://discuss.elastic.co/t/logstash-pipeline-systax-errors-after-7-6-0/220903 "2020-02-27T01:09:21Z")
Working logstash 7.5.2 upgraded to 7.6.0 causes 3 of my pipelines to fail with syntax error messages. Or course, since they were working, I can't really find any syntax errors. Has anyone else seen this? Thanks --- ## [How to grok complex patterns with logstash 7.1](https://discuss.elastic.co/t/how-to-grok-complex-patterns-with-logstash-7-1/218480)
**Author:** [@keshavl](https://discuss.elastic.co/u/keshavl)\ **Replies:** 13\ **Last updated:** [February 27, 2020, 1:03am UTC](https://discuss.elastic.co/t/how-to-grok-complex-patterns-with-logstash-7-1/218480 "2020-02-27T01:03:26Z")
HI. I am aware that DATA and GREEDYDATA are expensive patterns but those are most simple to use. I am using following pattern to extract data out of JMS XML . I am pasting both Input as well as Pattern which I have confi… --- ## [Logstash not connecting to anything in docker](https://discuss.elastic.co/t/logstash-not-connecting-to-anything-in-docker/221121)
**Author:** [@thickathanasnika](https://discuss.elastic.co/u/thickathanasnika)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 11:36pm UTC](https://discuss.elastic.co/t/logstash-not-connecting-to-anything-in-docker/221121 "2020-02-26T23:36:35Z")
I am basically having an issue with logstash exiting. The logs I am getting are: log1 | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a fut… --- ## [Authorization in HTTP\_POLLER](https://discuss.elastic.co/t/authorization-in-http-poller/221117)
**Author:** [@RunningSmurf](https://discuss.elastic.co/u/RunningSmurf)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 10:07pm UTC](https://discuss.elastic.co/t/authorization-in-http-poller/221117 "2020-02-26T22:07:08Z")
Hello, I can run a CURL command to verify a website. For the login, I add the following: --NTLM --negotiate -u \: Does anyone know how to translate this into HTTP\_POLLER? Thanks! Sincerely, RS --- ## [An unknown error occurred sending a bulk request to Elasticsearch. We will retry indefinitely {:error\_message=\>"\\"\\\\xB0\\" from ASCII-8BIT to UTF-8"](https://discuss.elastic.co/t/an-unknown-error-occurred-sending-a-bulk-request-to-elasticsearch-we-will-retry-indefinitely-error-message-xb0-from-ascii-8bit-to-utf-8/220906)
**Author:** [@m.a\_farazuddin](https://discuss.elastic.co/u/m.a_farazuddin)\ **Replies:** 1\ **Last updated:** [February 26, 2020, 9:42pm UTC](https://discuss.elastic.co/t/an-unknown-error-occurred-sending-a-bulk-request-to-elasticsearch-we-will-retry-indefinitely-error-message-xb0-from-ascii-8bit-to-utf-8/220906 "2020-02-26T21:42:07Z")
I am trying to query v$active\_session\_history table from logstash with below config file and I bump into the error due to column XID of RAW Type - I am also using container to set up my ELK configuration I am trying to… --- ## [Unable to send data from logstash to aws elastic service using amazon\_es plugin](https://discuss.elastic.co/t/unable-to-send-data-from-logstash-to-aws-elastic-service-using-amazon-es-plugin/221110)
**Author:** [@Shubham\_Dangare](https://discuss.elastic.co/u/Shubham_Dangare)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 8:35pm UTC](https://discuss.elastic.co/t/unable-to-send-data-from-logstash-to-aws-elastic-service-using-amazon-es-plugin/221110 "2020-02-26T20:35:45Z")
I have setup logstash config to send postgres data to aws elastic and have policy set to public for now output { stdout {codec =\> rubydebug} amazon\_es { hosts =\> \["https://.us-east-2.es.amazonaws.com"\] region =\> "us… --- ## [Logstash JDBC and Firebird error jdbc\_driver](https://discuss.elastic.co/t/logstash-jdbc-and-firebird-error-jdbc-driver/221101)
**Author:** [@tborgesvieira](https://discuss.elastic.co/u/tborgesvieira)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 7:47pm UTC](https://discuss.elastic.co/t/logstash-jdbc-and-firebird-error-jdbc-driver/221101 "2020-02-26T19:47:19Z")
I'm trying to use logstash (version 7.6.0) with firebird on the docker, put the jar in logstash-core / lib / jars / jaybird-jdk18-3.0.3.jar and I'm setting up the input correctly, but I'm having an "org.firebirdsql" err… --- ## [Aggregate messages in sequence order?](https://discuss.elastic.co/t/aggregate-messages-in-sequence-order/220251)
**Author:** [@jfro](https://discuss.elastic.co/u/jfro)\ **Replies:** 4\ **Last updated:** [February 26, 2020, 7:26pm UTC](https://discuss.elastic.co/t/aggregate-messages-in-sequence-order/220251 "2020-02-26T19:26:11Z")
I have vendor appliance that I'm sending syslog data from to logstash v6.5.4. The appliance splits messages that are over 1kB into separate messages and then sends them out via UDP. We had kafka buffering messages befor… --- ## [Logstash Kafka Input Plugin delivery gurantee](https://discuss.elastic.co/t/logstash-kafka-input-plugin-delivery-gurantee/219645)
**Author:** [@danielyahn](https://discuss.elastic.co/u/danielyahn)\ **Replies:** 1\ **Last updated:** [February 26, 2020, 7:18pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-delivery-gurantee/219645 "2020-02-26T19:18:37Z")
Can Logstash Kafka input plug-in make any delivery guarantee? Per Tips and Best Practices | Logstash Reference \[7.6\] | Elastic, we don't get at-least-once guarantee. "Does Kafka Input commit offsets only after the eve… --- ## [Logstash to pull logs from Elasticsearch input](https://discuss.elastic.co/t/logstash-to-pull-logs-from-elasticsearch-input/221095)
**Author:** [@rmukkara](https://discuss.elastic.co/u/rmukkara)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 7:01pm UTC](https://discuss.elastic.co/t/logstash-to-pull-logs-from-elasticsearch-input/221095 "2020-02-26T19:01:20Z")
Logstash is supposed to pull logs from elasticsearch input, but somehow its failing to pull. I have used below configuration. let me know if i am missing something ........... input { beats { port =\> 5044 } elastic… --- ## [Config very repetitive, possible to be more DRY?](https://discuss.elastic.co/t/config-very-repetitive-possible-to-be-more-dry/221088)
**Author:** [@McPo](https://discuss.elastic.co/u/McPo)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 5:41pm UTC](https://discuss.elastic.co/t/config-very-repetitive-possible-to-be-more-dry/221088 "2020-02-26T17:41:53Z")
Hi, I have noticed that logstash config appears to be very repetitive, for example we make use of "jdbc\_streaming" filter 21 times. This result in having to define jdbc\_user, jdbc\_password, jdbc\_driver\_class, jdbc\_driver… --- ## [Log flooded with error messages](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897)
**Author:** [@stefan\_schumacher](https://discuss.elastic.co/u/stefan_schumacher)\ **Replies:** 5\ **Last updated:** [February 26, 2020, 3:48pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897 "2020-02-26T15:48:17Z")
Hi, I am trying to setup the ELK-Stack using this - https://pawelurbanek.com/elk-nginx-logs-setup - Blog article here. Filebeat is running and sending data to logstash. My logstash configuration looks like this (I rem… --- ## [Logstash not seeing elastic?](https://discuss.elastic.co/t/logstash-not-seeing-elastic/221069)
**Author:** [@thickathanasnika](https://discuss.elastic.co/u/thickathanasnika)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 3:34pm UTC](https://discuss.elastic.co/t/logstash-not-seeing-elastic/221069 "2020-02-26T15:34:58Z")
So I am getting this message in docker-compose... log1 | \[2020-02-26T12:40:38,174\]\[WARN \]\[logstash.licensechecker.licensereader\] Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>"http:… --- ## [Unable parse multiline pattern](https://discuss.elastic.co/t/unable-parse-multiline-pattern/221043)
**Author:** [@bivaswap](https://discuss.elastic.co/u/bivaswap)\ **Replies:** 2\ **Last updated:** [February 26, 2020, 3:25pm UTC](https://discuss.elastic.co/t/unable-parse-multiline-pattern/221043 "2020-02-26T15:25:49Z")
Hello, I am trying to parse multiline logs Log sample 06/Feb/2020:09:20 +0000 ACCESS { "Total alerts subscribed for email only": 0, "Total alerts subscribed for In-App only": 59579 filebeat configuration filebea… --- ## [SNMP Trap listener died Address already in use caused by problems with files in \`\`yamlmibdir\`\`](https://discuss.elastic.co/t/snmp-trap-listener-died-address-already-in-use-caused-by-problems-with-files-in-yamlmibdir/221029)
**Author:** [@mjuenema](https://discuss.elastic.co/u/mjuenema)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 11:32am UTC](https://discuss.elastic.co/t/snmp-trap-listener-died-address-already-in-use-caused-by-problems-with-files-in-yamlmibdir/221029 "2020-02-26T11:32:40Z")
This issue has been discussed several times but it seems that this can be caused when there are problems with the MIB files in yamlmibdir. The error message is completely misleading in this case. References: https://… --- ## [Connecting logstash to AWS elasticsearch endpoint](https://discuss.elastic.co/t/connecting-logstash-to-aws-elasticsearch-endpoint/221024)
**Author:** [@vidhyadhara](https://discuss.elastic.co/u/vidhyadhara)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 11:22am UTC](https://discuss.elastic.co/t/connecting-logstash-to-aws-elasticsearch-endpoint/221024 "2020-02-26T11:22:13Z")
Hello Folks, I've my data in a server which i want to push into AWS elasticsearch with logstash. I've logstash version 7.5 installed on WINDOWS 10. AWS elasticsearch service also. I'm able to push data from PY scripts … --- ## [Grok fails in logstash with grokparsefailure, but OK in grokdebugger](https://discuss.elastic.co/t/grok-fails-in-logstash-with-grokparsefailure-but-ok-in-grokdebugger/221015)
**Author:** [@ifishy](https://discuss.elastic.co/u/ifishy)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 10:47am UTC](https://discuss.elastic.co/t/grok-fails-in-logstash-with-grokparsefailure-but-ok-in-grokdebugger/221015 "2020-02-26T10:47:11Z")
Hello, i've the following text to parse, there are some newlinies and i masked them: Category: RabbitMqMonitorWorker.IRmqMonitor\\nEventId: 15670\\n\\nTest-Cluster: Die Verbindung 'StadtKoeln\_TokenProvider (0c1d52e0-67cc-… --- ## [Grok pattern OK in debugger, but parse failure in logstash](https://discuss.elastic.co/t/grok-pattern-ok-in-debugger-but-parse-failure-in-logstash/220405)
**Author:** [@argnist](https://discuss.elastic.co/u/argnist)\ **Replies:** 4\ **Last updated:** [February 26, 2020, 9:30am UTC](https://discuss.elastic.co/t/grok-pattern-ok-in-debugger-but-parse-failure-in-logstash/220405 "2020-02-26T09:30:52Z")
Hello everyone. I wrote grok pattern, checked it in debugger in Kibana. Grok pattern OK in debugger, but in logstash i got parse failure. I tried to debug pattern in logstash with ~/logstash/bin/logstash --path.setti… --- ## [Pipeline aborted due to error help me](https://discuss.elastic.co/t/pipeline-aborted-due-to-error-help-me/220838)
**Author:** [@samargh](https://discuss.elastic.co/u/samargh)\ **Replies:** 7\ **Last updated:** [February 25, 2020, 4:43pm UTC](https://discuss.elastic.co/t/pipeline-aborted-due-to-error-help-me/220838 "2020-02-25T16:43:16Z")
i'm working with logstash the current version 7.6 i have 44 columns , 4 of them are date but contains nothing i declared them as a data it's not accepted i didn't dclare them still not accepted ( cuz i know that it's … --- ## [Logstash pipline elasticsearch input with ssl on ECE 2.4](https://discuss.elastic.co/t/logstash-pipline-elasticsearch-input-with-ssl-on-ece-2-4/220989)
**Author:** [@tazikm](https://discuss.elastic.co/u/tazikm)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 9:17am UTC](https://discuss.elastic.co/t/logstash-pipline-elasticsearch-input-with-ssl-on-ece-2-4/220989 "2020-02-26T09:17:17Z")
Hi, i want to use logstash elasticsearch input plugin with ssl but i get this error on logstash logs: size=\>1000, scroll=\>"1m", docinfo=\>false, docinfo\_target=\>"@metadata", docinfo\_fields=\>\["\_index", "\_type", "\_id"\]\> … --- ## [Facing issue with logstash services throwing "\[logstash.instrument.periodicpoller.cgroup\] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu"](https://discuss.elastic.co/t/facing-issue-with-logstash-services-throwing-logstash-instrument-periodicpoller-cgroup-one-or-more-required-cgroup-files-or-directories-not-found-proc-self-cgroup-sys-fs-cgroup-cpuacct-sys-fs-cgroup-cpu/220981)
**Author:** [@stbwit.demo](https://discuss.elastic.co/u/stbwit.demo)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 8:42am UTC](https://discuss.elastic.co/t/facing-issue-with-logstash-services-throwing-logstash-instrument-periodicpoller-cgroup-one-or-more-required-cgroup-files-or-directories-not-found-proc-self-cgroup-sys-fs-cgroup-cpuacct-sys-fs-cgroup-cpu/220981 "2020-02-26T08:42:15Z")
This is the data in the config file: input { file { path =\> "C:/Automation/application07012020.log" start\_position =\> "beginning" sincedb\_path =\> "NUL" } } output { elasticsearch { hosts =\> "localhost:9200" in… --- ## [How to index a SQL Server table which contains foreign keys?](https://discuss.elastic.co/t/how-to-index-a-sql-server-table-which-contains-foreign-keys/220973)
**Author:** [@LaaKii](https://discuss.elastic.co/u/LaaKii)\ **Replies:** 0\ **Last updated:** [February 26, 2020, 8:11am UTC](https://discuss.elastic.co/t/how-to-index-a-sql-server-table-which-contains-foreign-keys/220973 "2020-02-26T08:11:10Z")
I already indexed a simple SQL Table with an id and a couple of other values like name, etc. But when it comes to indexing a table with foreign keys, im quite lost how to configure logstash. My database is similar to t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=397) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=399)