# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=337 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 338 --- ## [Numeric field (long) shown as string in kibana discovery](https://discuss.elastic.co/t/numeric-field-long-shown-as-string-in-kibana-discovery/237656)
**Author:** [@ndg](https://discuss.elastic.co/u/ndg)\ **Replies:** 3\ **Last updated:** [June 25, 2020, 4:15pm UTC](https://discuss.elastic.co/t/numeric-field-long-shown-as-string-in-kibana-discovery/237656 "2020-06-25T16:15:22Z")
Hi! We are installing and testing the ELK stack in a test env in out company. I am working towards migrating rsyslog logs from Graylog to ELK. But even when some fields are mapped as long in elasticsearch when i go to … --- ## [I'm linking ELK to Snort. but there's been an error with the logstash](https://discuss.elastic.co/t/im-linking-elk-to-snort-but-theres-been-an-error-with-the-logstash/238563)
**Author:** [@arse\_nal](https://discuss.elastic.co/u/arse_nal)\ **Replies:** 4\ **Last updated:** [June 25, 2020, 4:06pm UTC](https://discuss.elastic.co/t/im-linking-elk-to-snort-but-theres-been-an-error-with-the-logstash/238563 "2020-06-25T16:06:47Z")
I'm linking ELK to Snort. there's been an error with the logstash But I can't figure out a way to deal with this error. Help me. I Use ELK 6.2.2, Centos 7, and snort 2.9.16. configure file code! --- ## [Logstash-GemNotFound-belzebuth](https://discuss.elastic.co/t/logstash-gemnotfound-belzebuth/238709)
**Author:** [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Replies:** 0\ **Last updated:** [June 25, 2020, 3:27pm UTC](https://discuss.elastic.co/t/logstash-gemnotfound-belzebuth/238709 "2020-06-25T15:27:39Z")
Hello team, input{ file { path =\> "/tmp/date.log" start\_position =\> beginning sincedb\_path =\> "/dev/null" } } filter {} output { … --- ## [Cannot retrieve keystore values in pipeline](https://discuss.elastic.co/t/cannot-retrieve-keystore-values-in-pipeline/238703)
**Author:** [@bodi4](https://discuss.elastic.co/u/bodi4)\ **Replies:** 0\ **Last updated:** [June 25, 2020, 2:54pm UTC](https://discuss.elastic.co/t/cannot-retrieve-keystore-values-in-pipeline/238703 "2020-06-25T14:54:13Z")
Faced a problem with logstash getting started after upgrade to 7.8 release. Using different variables for storing passwords in keystore. like for elasticsearch users, cert paswords etc.. This variables is used in many … --- ## [How not to overwrite duplicates? save old documents](https://discuss.elastic.co/t/how-not-to-overwrite-duplicates-save-old-documents/238656)
**Author:** [@lyubava](https://discuss.elastic.co/u/lyubava)\ **Replies:** 2\ **Last updated:** [June 25, 2020, 1:36pm UTC](https://discuss.elastic.co/t/how-not-to-overwrite-duplicates-save-old-documents/238656 "2020-06-25T13:36:20Z")
Hi all! I use this filter to prevent duplicates: fingerprint { source =\> "name" target =\> "\[@metadata\]\[fingerprint\]" method =\> "MURMUR3" } And it works. I don't see documents with the same 'name' field in … --- ## [Kibana : Unexpected token \< in JSON at position 1](https://discuss.elastic.co/t/kibana-unexpected-token-in-json-at-position-1/238674)
**Author:** [@krpatel19](https://discuss.elastic.co/u/krpatel19)\ **Replies:** 0\ **Last updated:** [June 25, 2020, 12:53pm UTC](https://discuss.elastic.co/t/kibana-unexpected-token-in-json-at-position-1/238674 "2020-06-25T12:53:53Z")
We are getting below exceptions while accessing Elasticsearch through GCP https load balancer. It is happening for KIbana endpoint as well as for elasticsearch endpoint made for application configured as internal http LB… --- ## [Pull alerts from Solar Winds](https://discuss.elastic.co/t/pull-alerts-from-solar-winds/238493)
**Author:** [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\ **Replies:** 3\ **Last updated:** [June 25, 2020, 10:54am UTC](https://discuss.elastic.co/t/pull-alerts-from-solar-winds/238493 "2020-06-25T10:54:53Z")
Code Share. I use powershell to pull all new alerts from Solarwinds and here is the code. First I check to see what the max alertid is in elasticsearch then I query solarwinds db to find all alerts after this max aler… --- ## [Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 69](https://discuss.elastic.co/t/handling-exception-org-logstash-beats-invalidframeprotocolexception-invalid-version-of-beats-protocol-69/238253)
**Author:** [@priyashree](https://discuss.elastic.co/u/priyashree)\ **Replies:** 1\ **Last updated:** [June 25, 2020, 4:38am UTC](https://discuss.elastic.co/t/handling-exception-org-logstash-beats-invalidframeprotocolexception-invalid-version-of-beats-protocol-69/238253 "2020-06-25T04:38:14Z")
I have 2 logstash machines and both the machines shows the following error in /var/log/logstash. \[2020-06-23T12:12:08,031\]\[INFO \]\[org.logstash.beats.BeatsHandler\]\[main\] \[local: logstash\_machine\_ip:5044, remote: filebeat… --- ## [Configure tcp input to accept syslog over ssl](https://discuss.elastic.co/t/configure-tcp-input-to-accept-syslog-over-ssl/237853)
**Author:** [@danona45](https://discuss.elastic.co/u/danona45)\ **Replies:** 5\ **Last updated:** [June 25, 2020, 4:30am UTC](https://discuss.elastic.co/t/configure-tcp-input-to-accept-syslog-over-ssl/237853 "2020-06-25T04:30:52Z")
This is my input config, but I don't get anything passed input in my pipeline or I would see something in my ruby debug output. No errors in the log. \> tcp { \> type =\> "syslog" \> port =\> "4000" \> … --- ## [Logstash shuts down after setting offset for kafka partition](https://discuss.elastic.co/t/logstash-shuts-down-after-setting-offset-for-kafka-partition/237233)
**Author:** [@Aliasgar\_Choolawala](https://discuss.elastic.co/u/Aliasgar_Choolawala)\ **Replies:** 1\ **Last updated:** [June 25, 2020, 3:43am UTC](https://discuss.elastic.co/t/logstash-shuts-down-after-setting-offset-for-kafka-partition/237233 "2020-06-25T03:43:54Z")
this is logstash.conf input { kafka { bootstrap\_servers =\> "kafka-app-logs:9092" topics =\> \["pm2-logs"\] codec =\> multiline { pattern =\> "^\\s" what =\> "previous" } } } filter { mutate {… --- ## [Elapsed filter doesn't work sometimes](https://discuss.elastic.co/t/elapsed-filter-doesnt-work-sometimes/238557)
**Author:** [@Ferdous\_Khan](https://discuss.elastic.co/u/Ferdous_Khan)\ **Replies:** 4\ **Last updated:** [June 25, 2020, 12:26am UTC](https://discuss.elastic.co/t/elapsed-filter-doesnt-work-sometimes/238557 "2020-06-25T00:26:48Z")
Hello, I am seeing Elapsed filter doesn't work all the times in our staging environment. In our development environment, we have single logstash and elasticsearch instance and Elapsed filter works fine there. But in sta… --- ## [Index management Logstash vs Elasticsearch](https://discuss.elastic.co/t/index-management-logstash-vs-elasticsearch/238408)
**Author:** [@envycz](https://discuss.elastic.co/u/envycz)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 10:39pm UTC](https://discuss.elastic.co/t/index-management-logstash-vs-elasticsearch/238408 "2020-06-24T22:39:26Z")
Hello, I have ELK stack deployed all on one host for testing purposes and two Filebeat klients sending data from other hosts to it. First one is sending Apache and Icinga monitoring logs from a linux machine directly d… --- ## [Logstash ES cert authenication is paid or free?](https://discuss.elastic.co/t/logstash-es-cert-authenication-is-paid-or-free/238535)
**Author:** [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 10:07pm UTC](https://discuss.elastic.co/t/logstash-es-cert-authenication-is-paid-or-free/238535 "2020-06-24T22:07:22Z")
Is certificate based authentication of Logstash output to elasticsearch is free or paid? --- ## [Logstash with syslog input doesn't send logs to Elasticsearch](https://discuss.elastic.co/t/logstash-with-syslog-input-doesnt-send-logs-to-elasticsearch/238553)
**Author:** [@jelocabral](https://discuss.elastic.co/u/jelocabral)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 7:43pm UTC](https://discuss.elastic.co/t/logstash-with-syslog-input-doesnt-send-logs-to-elasticsearch/238553 "2020-06-24T19:43:20Z")
Dear, I've configured Logstash 7.8.0 with a Syslog input, into an index called "syslog-514-xxxxx": input { udp { port =\> 514 type =\> syslog } } output { elasticsearch { hosts =\> \[ "https://172.31.2.2:9200" \] us… --- ## [Logstash connection failure to elasticsearch (deduplication pipeline)](https://discuss.elastic.co/t/logstash-connection-failure-to-elasticsearch-deduplication-pipeline/237181)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 4\ **Last updated:** [June 24, 2020, 7:17pm UTC](https://discuss.elastic.co/t/logstash-connection-failure-to-elasticsearch-deduplication-pipeline/237181 "2020-06-24T19:17:12Z")
I have created a pipeline to deduplicate logs in an index. I am facing issues while having logstash connect to the elasticsearch node. IP hosting elasticsearch is 192.168.0.2 and has a self-signed certificate. Followin… --- ## [Message: file too big for single read](https://discuss.elastic.co/t/message-file-too-big-for-single-read/238348)
**Author:** [@hilo21](https://discuss.elastic.co/u/hilo21)\ **Replies:** 2\ **Last updated:** [June 24, 2020, 6:55pm UTC](https://discuss.elastic.co/t/message-file-too-big-for-single-read/238348 "2020-06-24T18:55:28Z")
Hello, Logstash was working fine for me until this started happening. I can't start my logstash instance and I can't find some extra information on this error anywhere. \[2020-06-23T23:03:31,655\]\[INFO \]\[logstash.runner … --- ## [Logstash pipeline to pipeline losing events when queue is full](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-losing-events-when-queue-is-full/238545)
**Author:** [@wizardofc0dz](https://discuss.elastic.co/u/wizardofc0dz)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 6:34pm UTC](https://discuss.elastic.co/t/logstash-pipeline-to-pipeline-losing-events-when-queue-is-full/238545 "2020-06-24T18:34:06Z")
Hello, I have configured pipeline-to-pipeline with output isolator configuration in Logstash version 7.4.1 to send logs to local and remote Elasticsearch. Both pipelines, local and remote output, have been configured w… --- ## [Logstash mixing up two configuration files?!](https://discuss.elastic.co/t/logstash-mixing-up-two-configuration-files/238540)
**Author:** [@RonGros](https://discuss.elastic.co/u/RonGros)\ **Replies:** 7\ **Last updated:** [June 24, 2020, 6:26pm UTC](https://discuss.elastic.co/t/logstash-mixing-up-two-configuration-files/238540 "2020-06-24T18:26:06Z")
Hi, I swear to god I don't believe myself when I am writng this. I have a logstash configuration. it takes a CSV file we have and throws it into logstash. I have some ugly patches there that take information from the fi… --- ## [Could not index event to Elasticsearch.Rejecting mapping update to \[\] as the final mapping would have more than 1 type: \[\_doc,syslog\]](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-rejecting-mapping-update-to-as-the-final-mapping-would-have-more-than-1-type-doc-syslog/238326)
**Author:** [@cezar996](https://discuss.elastic.co/u/cezar996)\ **Replies:** 2\ **Last updated:** [June 24, 2020, 3:44pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-rejecting-mapping-update-to-as-the-final-mapping-would-have-more-than-1-type-doc-syslog/238326 "2020-06-24T15:44:52Z")
Hello everybody! I get the following error when I try to send syslog data to Elasticsearch Index: \[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :… --- ## [Centralize Kafka input settings](https://discuss.elastic.co/t/centralize-kafka-input-settings/238496)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 2\ **Last updated:** [June 24, 2020, 3:40pm UTC](https://discuss.elastic.co/t/centralize-kafka-input-settings/238496 "2020-06-24T15:40:43Z")
Hi, Is there a way to refer to one file in my Logstash configuration files where I put all Kafka input general settings and avoid flooding (20 rows that are the same) each pipeline? Thank you! --- ## [How to integrate ELK with ArcSight](https://discuss.elastic.co/t/how-to-integrate-elk-with-arcsight/236762)
**Author:** [@cezar996](https://discuss.elastic.co/u/cezar996)\ **Replies:** 4\ **Last updated:** [June 24, 2020, 3:36pm UTC](https://discuss.elastic.co/t/how-to-integrate-elk-with-arcsight/236762 "2020-06-24T15:36:32Z")
Hello everybody! I have installed ELK 7.6.2 and I want to get logs from ArcSight SIEM. In order to do this I configured a forward connector which sends all events in Syslog CEF format on port 5000 TCP, as described here… --- ## [How to use Logstash and ElasticSearch to Index 'transactionID' from 'message' while keeping original 'message' intact](https://discuss.elastic.co/t/how-to-use-logstash-and-elasticsearch-to-index-transactionid-from-message-while-keeping-original-message-intact/237503)
**Author:** [@jibitgeorge](https://discuss.elastic.co/u/jibitgeorge)\ **Replies:** 3\ **Last updated:** [June 24, 2020, 3:27pm UTC](https://discuss.elastic.co/t/how-to-use-logstash-and-elasticsearch-to-index-transactionid-from-message-while-keeping-original-message-intact/237503 "2020-06-24T15:27:00Z")
I already have ELK setup for my Java application and I'm able to see the logs in Kibana. I have a field called 'message' in which I have a string called transactionID and a value assoicated with it (eg: transactionID:73… --- ## [Logstash can't handle large messages](https://discuss.elastic.co/t/logstash-cant-handle-large-messages/238499)
**Author:** [@rzwarts](https://discuss.elastic.co/u/rzwarts)\ **Replies:** 3\ **Last updated:** [June 24, 2020, 2:52pm UTC](https://discuss.elastic.co/t/logstash-cant-handle-large-messages/238499 "2020-06-24T14:52:22Z")
Hi all, I'm having an issue with logstash and large syslog messages. I am using logstash 7.8 and a json template to get the files sent from rsyslog to logstash. But some messages are too long and completely crash the se… --- ## [Logstash ssl-tcp configuration syslog output](https://discuss.elastic.co/t/logstash-ssl-tcp-configuration-syslog-output/238349)
**Author:** [@bambam](https://discuss.elastic.co/u/bambam)\ **Replies:** 6\ **Last updated:** [June 24, 2020, 2:44pm UTC](https://discuss.elastic.co/t/logstash-ssl-tcp-configuration-syslog-output/238349 "2020-06-24T14:44:56Z")
Hello, I'm trying to forward messages from Logstash to an external syslog server (Qradar). I need to use ssl-tcp, for the external connection only not the connection to Elasticsearch. I can't find any documentation on h… --- ## [Get first two octets of IP address](https://discuss.elastic.co/t/get-first-two-octets-of-ip-address/238361)
**Author:** [@Leo\_Kwok](https://discuss.elastic.co/u/Leo_Kwok)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 2:15pm UTC](https://discuss.elastic.co/t/get-first-two-octets-of-ip-address/238361 "2020-06-24T14:15:20Z")
Hi, I only need the first two octet of IP address, for example, only 91.255 for IP 91.225.201.1. Please advise how to get this in logstash filter. Thanks, Leo --- ## [DLQ - Managing Dead Letter Queue Size- To /dev/null?](https://discuss.elastic.co/t/dlq-managing-dead-letter-queue-size-to-dev-null/238162)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 3\ **Last updated:** [June 24, 2020, 2:06pm UTC](https://discuss.elastic.co/t/dlq-managing-dead-letter-queue-size-to-dev-null/238162 "2020-06-24T14:06:39Z")
Looking through the DLQ docs https://www.elastic.co/guide/en/logstash/current/dead-letter-queues.html it looks like I can set a maximum size for each DLQ file, but it sounds like the rollover will just create a new file. … --- ## [Logstash 7.8.0 keystore via symlink not work](https://discuss.elastic.co/t/logstash-7-8-0-keystore-via-symlink-not-work/238485)
**Author:** [@zeleznypa](https://discuss.elastic.co/u/zeleznypa)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 1:30pm UTC](https://discuss.elastic.co/t/logstash-7-8-0-keystore-via-symlink-not-work/238485 "2020-06-24T13:30:27Z")
We have configured a shared folder between servers, that contain common configuration files for all of our logstash servers. This folder contains also the logstash.keystore file. When I copy the file logstash.keystore t… --- ## [Json parse error, original data now in message field ParserError: Unexpected end of input within/between array entries](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field-parsererror-unexpected-end-of-input-within-between-array-entries/238471)
**Author:** [@mbilal](https://discuss.elastic.co/u/mbilal)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 1:03pm UTC](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field-parsererror-unexpected-end-of-input-within-between-array-entries/238471 "2020-06-24T13:03:22Z")
JSOn data: { "EventID":1, "MachineName":''ABC', EntryType" :"Error", "Message" : "Stopped", "Source":"ABC", "TimeGenerated":"06/23/2020 06:50:40", "TimeWritten":"06/23/2020 06:50:40", "UserName": "", "Site": "" … --- ## [Logstash pipeline fails to start when log sending app is online](https://discuss.elastic.co/t/logstash-pipeline-fails-to-start-when-log-sending-app-is-online/238075)
**Author:** [@FanteG](https://discuss.elastic.co/u/FanteG)\ **Replies:** 1\ **Last updated:** [June 24, 2020, 12:25pm UTC](https://discuss.elastic.co/t/logstash-pipeline-fails-to-start-when-log-sending-app-is-online/238075 "2020-06-24T12:25:18Z")
Hello everyone. Recently I've stuck upon following issue - logstash cant start pipeline with following error Summary\[ERROR\]\[logstash.javapipeline \]\[strat\] Pipeline aborted due to error {:pipeline\_id=\>"strat&… --- ## [How to point logstash to my elasticcloud instance](https://discuss.elastic.co/t/how-to-point-logstash-to-my-elasticcloud-instance/238443)
**Author:** [@Martijn\_van\_Iersel](https://discuss.elastic.co/u/Martijn_van_Iersel)\ **Replies:** 0\ **Last updated:** [June 24, 2020, 11:22am UTC](https://discuss.elastic.co/t/how-to-point-logstash-to-my-elasticcloud-instance/238443 "2020-06-24T11:22:20Z")
Hello, I signed up for an elasticcloud trial and I'm trying to configure logstash to send my apache logs to my elasticcloud/elasticsearch instance. I'm super confused about how to configure logstash. The example here s… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=336) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=338)