# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=229 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 230 --- ## [Grokparsefailure on logstash](https://discuss.elastic.co/t/grokparsefailure-on-logstash/273047)
**Author:** [@srk1](https://discuss.elastic.co/u/srk1)\ **Replies:** 8\ **Last updated:** [May 18, 2021, 5:12pm UTC](https://discuss.elastic.co/t/grokparsefailure-on-logstash/273047 "2021-05-18T17:12:29Z")
Hi ES users, i'm trying to send a simple log using filebeat on to ES via logstash and i'm experiencing grokparsefailure errors on kibana. Can someone please assist on what type of grok filter needs to be used? Below is t… --- ## [Listen loop error: #\](https://discuss.elastic.co/t/listen-loop-error-ioerror-closed-stream/273314)
**Author:** [@Carlos\_Velasquez](https://discuss.elastic.co/u/Carlos_Velasquez)\ **Replies:** 0\ **Last updated:** [May 18, 2021, 3:49pm UTC](https://discuss.elastic.co/t/listen-loop-error-ioerror-closed-stream/273314 "2021-05-18T15:49:51Z")
Hi, When I start the logstash (manual invocation), I receive the folowing error. the version i'm using is 7.12.1 and Centos 8 Operating System: CentOS Linux 8 CPE OS Name: cpe:/o:centos:centos:8 K… --- ## [Having problem getting syslog to show on Elasticsearch](https://discuss.elastic.co/t/having-problem-getting-syslog-to-show-on-elasticsearch/273179)
**Author:** [@Praewpun](https://discuss.elastic.co/u/Praewpun)\ **Replies:** 4\ **Last updated:** [May 18, 2021, 1:41pm UTC](https://discuss.elastic.co/t/having-problem-getting-syslog-to-show-on-elasticsearch/273179 "2021-05-18T13:41:24Z")
Hi, I'm having problems trying to get syslog sent by external sources to logstash, then to elasticsearch (Kibana). I think the problem is that I don't understand what should the host and port should be specified. Here … --- ## [Grok hhtp](https://discuss.elastic.co/t/grok-hhtp/273292)
**Author:** [@delacoche28](https://discuss.elastic.co/u/delacoche28)\ **Replies:** 0\ **Last updated:** [May 18, 2021, 1:28pm UTC](https://discuss.elastic.co/t/grok-hhtp/273292 "2021-05-18T13:28:06Z")
hello with this log line: HTTP/1.0" 200 285 "-" "-" 30.853 I want to have a word to have paterns like : protocole : HTPP HTTP\_VERSION: 1.0 THX --- ## [Logstash Syslog input plugin | @timestamp issue](https://discuss.elastic.co/t/logstash-syslog-input-plugin-timestamp-issue/273244)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 1\ **Last updated:** [May 18, 2021, 11:04am UTC](https://discuss.elastic.co/t/logstash-syslog-input-plugin-timestamp-issue/273244 "2021-05-18T11:04:13Z")
Hi, I faced a brick wall and I seem to not be able to overcome it. My scenario FW logs coming to a server where they are caught by Syslog-ng Syslog-ng passes the logs to Logstash on the same server Currently, as a test… --- ## [Geo\_point automatically](https://discuss.elastic.co/t/geo-point-automatically/273183)
**Author:** [@Emi\_lie](https://discuss.elastic.co/u/Emi_lie)\ **Replies:** 4\ **Last updated:** [May 18, 2021, 8:40am UTC](https://discuss.elastic.co/t/geo-point-automatically/273183 "2021-05-18T08:40:00Z")
Hello, I struggle to add the geo\_point to my index, automatically. All the solutions I found, official and unofficial, indicate that it is necessary to pass by the devTools to make a PUT of the index. But I use elastic… --- ## [Failed to parse date field](https://discuss.elastic.co/t/failed-to-parse-date-field/272657)
**Author:** [@KarlWolf](https://discuss.elastic.co/u/KarlWolf)\ **Replies:** 4\ **Last updated:** [May 18, 2021, 6:09am UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/272657 "2021-05-18T06:09:10Z")
Hello, I'm new in topic ELK stuff and I try to solve my problem with date parsing. I have two fields with date and time stamp which I connected together. Then I want mark that pool as Date and send to Elastic: My Log l… --- ## [Can we connect one Logstash node with Both Licensed & Open source elastic stack?](https://discuss.elastic.co/t/can-we-connect-one-logstash-node-with-both-licensed-open-source-elastic-stack/273162)
**Author:** [@muralikrishna](https://discuss.elastic.co/u/muralikrishna)\ **Replies:** 2\ **Last updated:** [May 18, 2021, 2:51am UTC](https://discuss.elastic.co/t/can-we-connect-one-logstash-node-with-both-licensed-open-source-elastic-stack/273162 "2021-05-18T02:51:51Z")
Hi All, I would like to understand, If we can connect one Logstash node with Both Licensed & Open source elasticsearch nodes at a time ? currently we are using licensed elasticsearch for centralized pipelined managemen… --- ## [Logstash install and startup error](https://discuss.elastic.co/t/logstash-install-and-startup-error/273208)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 7\ **Last updated:** [May 17, 2021, 7:46pm UTC](https://discuss.elastic.co/t/logstash-install-and-startup-error/273208 "2021-05-17T19:46:19Z")
Good Afternoon; I am new to the ELK tools and am in the midst of a proof of concept using them. I am testing on a Windows 10 Professional box and have been able to install and get running both ElasticSearch and Kibana… --- ## [DatabaseError: driver.new.connect returned nil \>\> Logstash doesn´t connect to Postgresql Database](https://discuss.elastic.co/t/databaseerror-driver-new-connect-returned-nil-logstash-doesn-t-connect-to-postgresql-database/271288)
**Author:** [@Eng.Lucy](https://discuss.elastic.co/u/Eng.Lucy)\ **Replies:** 3\ **Last updated:** [May 17, 2021, 4:39pm UTC](https://discuss.elastic.co/t/databaseerror-driver-new-connect-returned-nil-logstash-doesn-t-connect-to-postgresql-database/271288 "2021-05-17T16:39:02Z")
I try to connect Logstash to Postgresql. This database is located in a remote server and Logstash runs in a docker container. My logstash.conf is: # input{ jdbc{ jdbc\_connection\_string =\> "jdbc:postgresql://000.00.0… --- ## [Metricbeat 7.12.1 mapping error (cannot be changed from type)](https://discuss.elastic.co/t/metricbeat-7-12-1-mapping-error-cannot-be-changed-from-type/272559)
**Author:** [@Flavio\_Pompermaier](https://discuss.elastic.co/u/Flavio_Pompermaier)\ **Replies:** 1\ **Last updated:** [May 17, 2021, 1:49pm UTC](https://discuss.elastic.co/t/metricbeat-7-12-1-mapping-error-cannot-be-changed-from-type/272559 "2021-05-17T13:49:14Z")
Hi to all, similarly to Metricbeat mapper error in 7.9.2 I have a problem when sending data from metricbeat to Logstash/Elasticsearch (7.12.1). I have enabled 3 modules: system, elasticsearch-xpack and logstash-xpack. … --- ## [Starting error caused by content of config file that it doesn't actually include](https://discuss.elastic.co/t/starting-error-caused-by-content-of-config-file-that-it-doesnt-actually-include/273133)
**Author:** [@111475](https://discuss.elastic.co/u/111475)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 6:54am UTC](https://discuss.elastic.co/t/starting-error-caused-by-content-of-config-file-that-it-doesnt-actually-include/273133 "2021-05-17T06:54:53Z")
I have this config file (/etc/logstash/conf.g/my.config). ... output { elasticsearch { hosts =\> \["XXXX"\] index =\> "search\_log-%{+YYYY.MM.dd}" user =\> "XXXX" password =\> "XXXX" ssl =\> true cacert =\> "/etc/logstas… --- ## [Logstash Secret Password ERROR](https://discuss.elastic.co/t/logstash-secret-password-error/273156)
**Author:** [@JAACNTT](https://discuss.elastic.co/u/JAACNTT)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 10:05am UTC](https://discuss.elastic.co/t/logstash-secret-password-error/273156 "2021-05-17T10:05:19Z")
Hello, I am trying to implement secret keystore from logstash, but I always get the same error: sudo /usr/share/logstash/bin/logstash-keystore create Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM … --- ## [Ruby pipeline permission denied sending to logstash](https://discuss.elastic.co/t/ruby-pipeline-permission-denied-sending-to-logstash/272584)
**Author:** [@Stan3005](https://discuss.elastic.co/u/Stan3005)\ **Replies:** 3\ **Last updated:** [May 17, 2021, 9:49am UTC](https://discuss.elastic.co/t/ruby-pipeline-permission-denied-sending-to-logstash/272584 "2021-05-17T09:49:27Z")
Hi, So I've set up an ELK stack. First time setting one up but installed elastic, kibana and logstash. When I go to localhost:5601 it takes me to the elastic dashboard. The problem I'm having is when I'm trying to sen… --- ## [Need Help with Logstash Multiline Codec](https://discuss.elastic.co/t/need-help-with-logstash-multiline-codec/273134)
**Author:** [@Pemitha\_Randiya](https://discuss.elastic.co/u/Pemitha_Randiya)\ **Replies:** 0\ **Last updated:** [May 17, 2021, 7:01am UTC](https://discuss.elastic.co/t/need-help-with-logstash-multiline-codec/273134 "2021-05-17T07:01:39Z")
Here is my sample log data : 2021/05/17 12:17:41.866 | 382DC9789F284D3C9D7CC269F1851094|0000SUMMARY:\[INFO \]: 2021/05/17 12:17:41.866 |--------------------------------------------------------------------- 2021/05/1… --- ## [Setup logstash to stop reading a file after some time](https://discuss.elastic.co/t/setup-logstash-to-stop-reading-a-file-after-some-time/273117)
**Author:** [@Amit\_Singh3](https://discuss.elastic.co/u/Amit_Singh3)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 10:33pm UTC](https://discuss.elastic.co/t/setup-logstash-to-stop-reading-a-file-after-some-time/273117 "2021-05-16T22:33:09Z")
We are using logstsash to read a folder to read logs, due to log rotation tons of files are being created in the folder, what if I want logstash to stop reading from a file if there is no new data in a file for 5 minute… --- ## [Ingest-convert.sh output incorrect for PaloAlto (pawn) Ingest Pipeline](https://discuss.elastic.co/t/ingest-convert-sh-output-incorrect-for-paloalto-pawn-ingest-pipeline/273076)
**Author:** [@metalshanked](https://discuss.elastic.co/u/metalshanked)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 4:38pm UTC](https://discuss.elastic.co/t/ingest-convert-sh-output-incorrect-for-paloalto-pawn-ingest-pipeline/273076 "2021-05-16T16:38:10Z")
Hi, I am planning to use Logstash instead of Filebeats for loading and parsing palo alto logs to Elastic. I setup the Filebeats as a one-time to generate the Ingest Pipeline Definition using the "pawn" Filebeats module… --- ## [Beats not able to connect with Logstash](https://discuss.elastic.co/t/beats-not-able-to-connect-with-logstash/273093)
**Author:** [@robinhood\_lko](https://discuss.elastic.co/u/robinhood_lko)\ **Replies:** 2\ **Last updated:** [May 16, 2021, 4:29pm UTC](https://discuss.elastic.co/t/beats-not-able-to-connect-with-logstash/273093 "2021-05-16T16:29:09Z")
Hi All, I have deployed ELK Stack using OSS versions Elasticsearch and Kibana. While connecting with Beats using Logstash, I am seeing following error on the system. Please can someone help me on this. Thanks \[2021-05-… --- ## [Input file plugin](https://discuss.elastic.co/t/input-file-plugin/273109)
**Author:** [@hsalim](https://discuss.elastic.co/u/hsalim)\ **Replies:** 0\ **Last updated:** [May 16, 2021, 12:14pm UTC](https://discuss.elastic.co/t/input-file-plugin/273109 "2021-05-16T12:14:16Z")
Question 1: Has anyone seen following message with input File plugin? \[2021-05-15T23:26:23,609\]\[DEBUG\]\[logstash.javapipeline \] Shutdown waiting for worker thread {:pipeline\_id=\>"rawdiameter1200", :thread=\>"#\ **Author:** [@Anoop\_kumar\_SRIVASTA](https://discuss.elastic.co/u/Anoop_kumar_SRIVASTA)\ **Replies:** 1\ **Last updated:** [May 16, 2021, 10:42am UTC](https://discuss.elastic.co/t/unable-to-install-output-jdbc-plugin-on-logstash/273062 "2021-05-16T10:42:40Z") Hi I am facing issue while connecting logstash output send to mysql database..below are my configuration setting output { jdbc{ driver\_jar\_path =\> "/app/logstash-CBS/mysql-connector-java-8.0.25.jar" driver\_class =\> … --- ## [Determining custom timestamp for new index does not work](https://discuss.elastic.co/t/determining-custom-timestamp-for-new-index-does-not-work/273106)
**Author:** [@Roy\_Levy](https://discuss.elastic.co/u/Roy_Levy)\ **Replies:** 0\ **Last updated:** [May 16, 2021, 10:27am UTC](https://discuss.elastic.co/t/determining-custom-timestamp-for-new-index-does-not-work/273106 "2021-05-16T10:27:45Z")
Hey, I have an index which is loaded with aggregated data each day at same time, like so: I'm trying to use logstash in order to migrate the data to another ES cluster in order to use machine learning capabilities t… --- ## [dentityMapCodec has reached 100% capacity {:current\_size=\>20000, :upper\_limit=\>20000} warning: thread "\[main\]\ **Author:** [@Bugsbee](https://discuss.elastic.co/u/Bugsbee)\ **Replies:** 4\ **Last updated:** [May 16, 2021, 4:08am UTC](https://discuss.elastic.co/t/dentitymapcodec-has-reached-100-capacity-current-size-20000-upper-limit-20000-warning-thread-main-file-terminated-with-exception-report-on-exception-is-true/273097 "2021-05-16T04:08:15Z") Hi Team, Community A.Error Need your assistance in this error im getting: dentityMapCodec has reached 100% capacity {:current\_size=\>20000, :upper\_limit=\>20000} warning: thread "\[main\]\ **Author:** [@Felipe\_Fuller](https://discuss.elastic.co/u/Felipe_Fuller)\ **Replies:** 2\ **Last updated:** [May 15, 2021, 8:40pm UTC](https://discuss.elastic.co/t/running-python-script-without-printing-results/271678 "2021-05-15T20:40:28Z") Hi Community! I hope that you're doing great! I've been involved in a project, and we have to process data with a python script and then insert it with Logstash into Elasticsearch. Having that said, I use the exec plugi… --- ## [FATAL 470 Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/fatal-470-logstash-stopped-processing-because-of-an-error-systemexit-exit/273091)
**Author:** [@Selcuk\_Benter](https://discuss.elastic.co/u/Selcuk_Benter)\ **Replies:** 1\ **Last updated:** [May 15, 2021, 7:12pm UTC](https://discuss.elastic.co/t/fatal-470-logstash-stopped-processing-because-of-an-error-systemexit-exit/273091 "2021-05-15T19:12:44Z")
Very basic setup, however keep getting 470 error. any suggestions OS: Windows 10 \[2021-05-15T20:49:31,983\]\[ERROR\]\[logstash.config.sourceloader\] No configuration found in the configured sources. \[2021-05-15T20:49:32,23… --- ## [Logstash-input-kinesis compression](https://discuss.elastic.co/t/logstash-input-kinesis-compression/273046)
**Author:** [@jvboyle](https://discuss.elastic.co/u/jvboyle)\ **Replies:** 2\ **Last updated:** [May 15, 2021, 3:17pm UTC](https://discuss.elastic.co/t/logstash-input-kinesis-compression/273046 "2021-05-15T15:17:44Z")
When running the logstash-input-kinesis plugin , the logs coming from cloudwatch are zipped and encoded 64. Adding the option to the input ( compression =\> "gzip" ) causes the plugin to no load correct \[2021-05-14T… --- ## [Mongodb output plugin connection error](https://discuss.elastic.co/t/mongodb-output-plugin-connection-error/273043)
**Author:** [@Roberto\_B](https://discuss.elastic.co/u/Roberto_B)\ **Replies:** 2\ **Last updated:** [May 15, 2021, 10:30am UTC](https://discuss.elastic.co/t/mongodb-output-plugin-connection-error/273043 "2021-05-15T10:30:33Z")
Hi there, i'm tryng to connect on MongoDB Atlas cluster using logstash output. I reach so many errors like this: \[WARN \] 2021-05-14 17:56:55.616 \[Ruby-0-Thread-12: :1\] mongodb - MONGODB | Error running ismaster on clu… --- ## [How to individualize the data that has an array objects](https://discuss.elastic.co/t/how-to-individualize-the-data-that-has-an-array-objects/272836)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 10\ **Last updated:** [May 14, 2021, 7:30pm UTC](https://discuss.elastic.co/t/how-to-individualize-the-data-that-has-an-array-objects/272836 "2021-05-14T19:30:07Z")
I need to be able to extract the data that it brings me in logstash, to be able to graph I have to extract the metrics, but when it sends them to me to kibana I cannot graph because they are in an array, I need help as I… --- ## [Ruby filter](https://discuss.elastic.co/t/ruby-filter/273055)
**Author:** [@paulbrown4](https://discuss.elastic.co/u/paulbrown4)\ **Replies:** 2\ **Last updated:** [May 14, 2021, 6:52pm UTC](https://discuss.elastic.co/t/ruby-filter/273055 "2021-05-14T18:52:40Z")
I am not very familiar with ruby, more so how it interacts with logstash, but I am trying to parse an array of CSVs. For whatever reason the ruby filter I am trying to use is not expanding the variables as a valid field … --- ## [Logstash Filter](https://discuss.elastic.co/t/logstash-filter/272734)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 6\ **Last updated:** [May 14, 2021, 4:15pm UTC](https://discuss.elastic.co/t/logstash-filter/272734 "2021-05-14T16:15:41Z")
\<166\>1 2021-05-11T17:50:40+0000 flowIdLog, applianceName=nameofsight, app=app1 How do i parse this, the KV section works but i dont know how to deal with this section " \<166\>1 2021-05-11T17:50:40+0000 flowIdLog " fil… --- ## [Logstash elasticsearch input plugin disable ssl verification](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin-disable-ssl-verification/272979)
**Author:** [@vikasp](https://discuss.elastic.co/u/vikasp)\ **Replies:** 2\ **Last updated:** [May 14, 2021, 3:54pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin-disable-ssl-verification/272979 "2021-05-14T15:54:19Z")
Hi Team, I was wondering if we have an option to disable ssl verification for logstash elasticsearch input plugin. Since the stack is deployed with certs created by elasticsearch cert util, just by adding cacert is not… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=228) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=230)