# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=633 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 634 --- ## [Logstash json file on a different server](https://discuss.elastic.co/t/logstash-json-file-on-a-different-server/160497)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 3\ **Last updated:** [December 12, 2018, 8:34am UTC](https://discuss.elastic.co/t/logstash-json-file-on-a-different-server/160497 "2018-12-12T08:34:34Z")
my project and ES are on different server. my logstash conf file input { file { path =\> "/home/es/templetes/review\_data.json" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" codec =\> "json… --- ## [Loss Data while indexing from My sql to elastic via logstash](https://discuss.elastic.co/t/loss-data-while-indexing-from-my-sql-to-elastic-via-logstash/159113)
**Author:** [@Rajesh\_Verma](https://discuss.elastic.co/u/Rajesh_Verma)\ **Replies:** 3\ **Last updated:** [December 12, 2018, 6:10am UTC](https://discuss.elastic.co/t/loss-data-while-indexing-from-my-sql-to-elastic-via-logstash/159113 "2018-12-12T06:10:30Z")
I had a issue facing during the indexing the data via logstash, case : I am able to index my whole data count 150000 from mysql server using the default document ID , but when i put the document id by using 3 field of … --- ## [Logstash TCP/SSL | General OpenSslEngine problem](https://discuss.elastic.co/t/logstash-tcp-ssl-general-opensslengine-problem/158044)
**Author:** [@x4cker](https://discuss.elastic.co/u/x4cker)\ **Replies:** 1\ **Last updated:** [December 12, 2018, 5:44am UTC](https://discuss.elastic.co/t/logstash-tcp-ssl-general-opensslengine-problem/158044 "2018-12-12T05:44:13Z")
Hi, I'm forwarding logs from logstash to logstash using tcp output/input plugins over SSL. on the client side i use this config in the output: tcp{ host =\> "logstash01.domain.tld" port =\> 8443 codec =\> json\_l… --- ## [Logstash - tagging snort\[#####\] not working](https://discuss.elastic.co/t/logstash-tagging-snort-not-working/160022)
**Author:** [@kozoke](https://discuss.elastic.co/u/kozoke)\ **Replies:** 1\ **Last updated:** [December 12, 2018, 2:25am UTC](https://discuss.elastic.co/t/logstash-tagging-snort-not-working/160022 "2018-12-12T02:25:47Z")
Hi, I'm attempting to extract pfsense logs, specifically the snort plugin alert logs. I've pulled the following filter from this site: https://gist.githubusercontent.com/mrlesmithjr/0799cdc3710f2ae1e182/raw/cfad5b0740… --- ## [Out of order fields](https://discuss.elastic.co/t/out-of-order-fields/160452)
**Author:** [@alexsamad](https://discuss.elastic.co/u/alexsamad)\ **Replies:** 0\ **Last updated:** [December 12, 2018, 12:11am UTC](https://discuss.elastic.co/t/out-of-order-fields/160452 "2018-12-12T00:11:17Z")
Hi I have a input like 2018-12-12 10:34:51,357 \[SocketAcceptorIoProcessor-1.0\] INFO msg.incoming - FIX.4.4:xxxx-\>hhhh/hhhhfix: 8=FIX.4.4 where therest is in the form of Code=value\\x01 delimited by \\x1. Now the probl… --- ## [Failed to Create Monitoring event (Metric not found)](https://discuss.elastic.co/t/failed-to-create-monitoring-event-metric-not-found/160446)
**Author:** [@kb2295](https://discuss.elastic.co/u/kb2295)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 10:22pm UTC](https://discuss.elastic.co/t/failed-to-create-monitoring-event-metric-not-found/160446 "2018-12-11T22:22:48Z")
Hi, I'm trying to start a simple logstash instance with a config file to upload data from a csv file. Keep getting this error: \[ERROR\]\[logstash.inputs.metrics \] Failed to create monitoring event {:message=\>"For path… --- ## [Scheduled CSV File Update](https://discuss.elastic.co/t/scheduled-csv-file-update/160435)
**Author:** [@kb2295](https://discuss.elastic.co/u/kb2295)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 8:11pm UTC](https://discuss.elastic.co/t/scheduled-csv-file-update/160435 "2018-12-11T20:11:28Z")
Hello, I have a conf file which uploads a CSV to logstash. input { file { path =\> "/path/to/data/data.csv" start\_position =\> "beginning" sincedb\_path =\>"dev/null" } } filter { csv { separator =\> "," columns … --- ## [Logstash 6.5.0-1 throwing OPEN\_WARN\_INTERVAL warning](https://discuss.elastic.co/t/logstash-6-5-0-1-throwing-open-warn-interval-warning/160275)
**Author:** [@Raj\_S](https://discuss.elastic.co/u/Raj_S)\ **Replies:** 2\ **Last updated:** [December 11, 2018, 7:39pm UTC](https://discuss.elastic.co/t/logstash-6-5-0-1-throwing-open-warn-interval-warning/160275 "2018-12-11T19:39:49Z")
Hi, Currently I'm seeing some servers are shows this following wARN in logstash 6.5.0-1 Preformatted text\[WARN \]\[filewatch.tailmode.handlers.createinitial\] open\_file OPEN\_WARN\_INTERVAL is '300'Preformatted text Would … --- ## [Grokparsefailure tag - what does really it mean?](https://discuss.elastic.co/t/grokparsefailure-tag-what-does-really-it-mean/160266)
**Author:** [@nazgul](https://discuss.elastic.co/u/nazgul)\ **Replies:** 1\ **Last updated:** [December 11, 2018, 7:13pm UTC](https://discuss.elastic.co/t/grokparsefailure-tag-what-does-really-it-mean/160266 "2018-12-11T19:13:11Z")
Hello, I have next problem. Started to use Filter-\>Grok in my Logstash recently and at the beginning everything was ok. One day I noticed that to all messages from Logstash to Elastic... Logstash started to add a tag=… --- ## [Logstash sends multiple copies of data](https://discuss.elastic.co/t/logstash-sends-multiple-copies-of-data/160267)
**Author:** [@nazgul](https://discuss.elastic.co/u/nazgul)\ **Replies:** 2\ **Last updated:** [December 11, 2018, 7:07pm UTC](https://discuss.elastic.co/t/logstash-sends-multiple-copies-of-data/160267 "2018-12-11T19:07:00Z")
Hello everyone, need you help on next question: I have 5 logstash.conf file to run. They are equal and only "Input path/.../....log" is different. \[sudo\] password for maxim: input { file { path =\> "/var/log/test.l… --- ## [How not to keep track of commit offset with kafka](https://discuss.elastic.co/t/how-not-to-keep-track-of-commit-offset-with-kafka/160423)
**Author:** [@gunbird1](https://discuss.elastic.co/u/gunbird1)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 5:58pm UTC](https://discuss.elastic.co/t/how-not-to-keep-track-of-commit-offset-with-kafka/160423 "2018-12-11T17:58:36Z")
Hi, I'm doing mostly testing on a logstash that consumes events from kafka. I frequently stop the logstash and can restart it several hours or days later. I don't want to keep track of the event offset because when I … --- ## [Upgrade logstash-output-mongodb to use latest stable Ruby driver](https://discuss.elastic.co/t/upgrade-logstash-output-mongodb-to-use-latest-stable-ruby-driver/160422)
**Author:** [@smeisner](https://discuss.elastic.co/u/smeisner)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 5:58pm UTC](https://discuss.elastic.co/t/upgrade-logstash-output-mongodb-to-use-latest-stable-ruby-driver/160422 "2018-12-11T17:58:26Z")
Hi all, we encountered reconnection problems during failover scenarios in our MongoDB cluster. Investigation showed that the version of the Ruby driver in use by Logstash was very old and not certified to be compatible … --- ## [Secure Syslog with logstash](https://discuss.elastic.co/t/secure-syslog-with-logstash/159604)
**Author:** [@totuyim](https://discuss.elastic.co/u/totuyim)\ **Replies:** 3\ **Last updated:** [December 11, 2018, 5:58pm UTC](https://discuss.elastic.co/t/secure-syslog-with-logstash/159604 "2018-12-11T17:58:13Z")
I'm using tcp input plugin, on windows, to handle SSL for receiving secure syslog but I only have the .cer file for a certification. How can I use this .cer file? It's a client/server certificate issued by COMODO. --- ## [How to send logs from AWS Kinesis Firehose to logstash](https://discuss.elastic.co/t/how-to-send-logs-from-aws-kinesis-firehose-to-logstash/160410)
**Author:** [@vnallamothu](https://discuss.elastic.co/u/vnallamothu)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 4:25pm UTC](https://discuss.elastic.co/t/how-to-send-logs-from-aws-kinesis-firehose-to-logstash/160410 "2018-12-11T16:25:01Z")
Using Kinesis-input-plugin, I was able to send the logs from my Kinesis-data-streams to logstash but not able to send logs from Kinesis-Firehose to logstash. Can you please let me know if I can send logs from Kinesis Fir… --- ## [Logstash's Kinesis input plugin doesn't able to access aws credentials file](https://discuss.elastic.co/t/logstashs-kinesis-input-plugin-doesnt-able-to-access-aws-credentials-file/159804)
**Author:** [@vnallamothu](https://discuss.elastic.co/u/vnallamothu)\ **Replies:** 7\ **Last updated:** [December 11, 2018, 3:50pm UTC](https://discuss.elastic.co/t/logstashs-kinesis-input-plugin-doesnt-able-to-access-aws-credentials-file/159804 "2018-12-11T15:50:22Z")
I was trying to send my logs from AWS Kinesis Firehose to my self managed Elasticsearch cluster through logstash and logstash's kinesis input plugin. My conf file looks like the following: input { kinesis { … --- ## [Failed to execute action - logstash](https://discuss.elastic.co/t/failed-to-execute-action-logstash/160228)
**Author:** [@vnallamothu](https://discuss.elastic.co/u/vnallamothu)\ **Replies:** 1\ **Last updated:** [December 11, 2018, 3:48pm UTC](https://discuss.elastic.co/t/failed-to-execute-action-logstash/160228 "2018-12-11T15:48:40Z")
Hi, I'm trying to send logs from AWS Kinesis firehose to elasticsearch through logstash. My ".conf" file look like following: input { kinesis { kinesis\_stream\_name =\> "GuardDutyFindings-Delivery… --- ## [Change log level for logstash-jdbc-input plugin](https://discuss.elastic.co/t/change-log-level-for-logstash-jdbc-input-plugin/160359)
**Author:** [@Maxim\_Kholod](https://discuss.elastic.co/u/Maxim_Kholod)\ **Replies:** 4\ **Last updated:** [December 11, 2018, 3:33pm UTC](https://discuss.elastic.co/t/change-log-level-for-logstash-jdbc-input-plugin/160359 "2018-12-11T15:33:30Z")
Hi, how to access Log4j2 configuration for logstash-jdbc-input? I want to do the same as logger.elasticsearchoutput.level = debug but for logstash-jdbc-input plugin. What is the logger name for this plugin? --- ## [Logstash to elasticsearch via apache SSL causes connection failure in version 6.5.1](https://discuss.elastic.co/t/logstash-to-elasticsearch-via-apache-ssl-causes-connection-failure-in-version-6-5-1/160363)
**Author:** [@jeraldsm](https://discuss.elastic.co/u/jeraldsm)\ **Replies:** 2\ **Last updated:** [December 11, 2018, 2:37pm UTC](https://discuss.elastic.co/t/logstash-to-elasticsearch-via-apache-ssl-causes-connection-failure-in-version-6-5-1/160363 "2018-12-11T14:37:10Z")
We have been using Elasticsearch version 2.4.2 and logstash version 2.3.4 and I have been trying to set up fresh ELK docker containers with version 6.5.1, however, I'm currently blocked as the logstash docker stops wit… --- ## [Noob question: Can I use logstash to collect & send specific log lines (not the whole logs)?](https://discuss.elastic.co/t/noob-question-can-i-use-logstash-to-collect-send-specific-log-lines-not-the-whole-logs/160265)
**Author:** [@DaveR](https://discuss.elastic.co/u/DaveR)\ **Replies:** 1\ **Last updated:** [December 11, 2018, 12:54pm UTC](https://discuss.elastic.co/t/noob-question-can-i-use-logstash-to-collect-send-specific-log-lines-not-the-whole-logs/160265 "2018-12-11T12:54:11Z")
Greetings, I'd like to collect a set of log lines from remote systems. Does Logstash have this capability, and can the log line specification be configured on the fly? A bit more detail: my product emits massive logs.… --- ## [Logstash monitoring index is yellow in cluster](https://discuss.elastic.co/t/logstash-monitoring-index-is-yellow-in-cluster/160336)
**Author:** [@Mojster](https://discuss.elastic.co/u/Mojster)\ **Replies:** 2\ **Last updated:** [December 11, 2018, 12:52pm UTC](https://discuss.elastic.co/t/logstash-monitoring-index-is-yellow-in-cluster/160336 "2018-12-11T12:52:23Z")
I'm using Elastic stack 6.5.2. I have currently two identical production indexes: 1x ES, 1x KB, 1LS (all Win based - single server) 2x ES, 1x KB 0LS (CentOS based - 2x server) This is because we are slowly moving to … --- ## [Add field with logstash (using kafka input plugin)](https://discuss.elastic.co/t/add-field-with-logstash-using-kafka-input-plugin/160358)
**Author:** [@yehosef](https://discuss.elastic.co/u/yehosef)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 11:26am UTC](https://discuss.elastic.co/t/add-field-with-logstash-using-kafka-input-plugin/160358 "2018-12-11T11:26:23Z")
I'm using the kafka input plugin for Logstash and there are metadata fields described in https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html. I want to copy some or all the fields into my data for… --- ## [Configure Logstash to import json from REST and visualise a Time/Value-Chart in Kibana](https://discuss.elastic.co/t/configure-logstash-to-import-json-from-rest-and-visualise-a-time-value-chart-in-kibana/160355)
**Author:** [@BugRogers](https://discuss.elastic.co/u/BugRogers)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 11:16am UTC](https://discuss.elastic.co/t/configure-logstash-to-import-json-from-rest-and-visualise-a-time-value-chart-in-kibana/160355 "2018-12-11T11:16:02Z")
Hi, i'm trying to import a json-structure from a RESt-API into Elasticsearch. Then i want to create a line-chart from a time/value-pair. In detail: Json: { "self":{ "href":"https://anUrl" }, "version":2, "values"… --- ## [Logstash parsing date into JSON](https://discuss.elastic.co/t/logstash-parsing-date-into-json/160229)
**Author:** [@Frederic\_Gaerel](https://discuss.elastic.co/u/Frederic_Gaerel)\ **Replies:** 1\ **Last updated:** [December 11, 2018, 11:05am UTC](https://discuss.elastic.co/t/logstash-parsing-date-into-json/160229 "2018-12-11T11:05:21Z")
Hi. this problem seems solved for some but I still have it.... Sample data in log file : {"TIMESTAMP":"2018-12-10 11:28:20.753","USER":"username (AIX)","ACTION":"SEARCH\_SITE","CALLING\_METHOD":"postRechercheSites","FIL… --- ## [Configuration is invalid](https://discuss.elastic.co/t/configuration-is-invalid/159518)
**Author:** [@Ruthvik\_Sai](https://discuss.elastic.co/u/Ruthvik_Sai)\ **Replies:** 1\ **Last updated:** [December 11, 2018, 10:31am UTC](https://discuss.elastic.co/t/configuration-is-invalid/159518 "2018-12-11T10:31:11Z")
Hi friends, I am getting the following error: C:\\logstash\\bin\>logstash -f appname\_modifier.conf --config.test\_and\_exit Sending Logstash logs to C:/logstash/logs which is now configured via log4j2.properties \[2018-12-… --- ## [Unable to merge pipelines in logstash](https://discuss.elastic.co/t/unable-to-merge-pipelines-in-logstash/160344)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 10:18am UTC](https://discuss.elastic.co/t/unable-to-merge-pipelines-in-logstash/160344 "2018-12-11T10:18:32Z")
Hi I am using Logstash version 6.2.4 I am trying to merge pipelines as below: Master pipeline/Merged Pipeline: # config/pipelines.yml - pipeline.id: beats1 config.string: | input { file { path =\> "/XXXXX/… --- ## [Transferring data from one elastic to another](https://discuss.elastic.co/t/transferring-data-from-one-elastic-to-another/160339)
**Author:** [@chrillebile](https://discuss.elastic.co/u/chrillebile)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 9:57am UTC](https://discuss.elastic.co/t/transferring-data-from-one-elastic-to-another/160339 "2018-12-11T09:57:19Z")
Hey, I'm trying transfer data from one elastic to another with the help of logstash, using cron-job that runs every hour. But all the data from last run + new data gets added, so i get tons of duplicates. Is it possible … --- ## [Index pattern is not created for Centralized pipeline](https://discuss.elastic.co/t/index-pattern-is-not-created-for-centralized-pipeline/160286)
**Author:** [@Susmitha\_Devathi](https://discuss.elastic.co/u/Susmitha_Devathi)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 4:12am UTC](https://discuss.elastic.co/t/index-pattern-is-not-created-for-centralized-pipeline/160286 "2018-12-11T04:12:33Z")
Hi Team, I have created centralized pipeline with parent and child pipelines ,but i'm unable to create index pattern. I have written parent and child pipelines in below format parent: config/pipelines.yml pipeline.… --- ## [Remove part of string in Logstash](https://discuss.elastic.co/t/remove-part-of-string-in-logstash/160313)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 0\ **Last updated:** [December 11, 2018, 7:27am UTC](https://discuss.elastic.co/t/remove-part-of-string-in-logstash/160313 "2018-12-11T07:27:39Z")
I am getting this document from API, "name": "accounts/plgh/locations/986532/reviews/" but i need the 986532 in different key. How to get that form that string. --- ## [What is Event in Logstash](https://discuss.elastic.co/t/what-is-event-in-logstash/160307)
**Author:** [@Varun\_S](https://discuss.elastic.co/u/Varun_S)\ **Replies:** 2\ **Last updated:** [December 11, 2018, 7:12am UTC](https://discuss.elastic.co/t/what-is-event-in-logstash/160307 "2018-12-11T07:12:41Z")
What is a event in Logstash. Whether it is a 1 record or 1 file? --- ## [Ruby exception undefined method \`each'](https://discuss.elastic.co/t/ruby-exception-undefined-method-each/158727)
**Author:** [@Elango-mas](https://discuss.elastic.co/u/Elango-mas)\ **Replies:** 3\ **Last updated:** [December 11, 2018, 6:55am UTC](https://discuss.elastic.co/t/ruby-exception-undefined-method-each/158727 "2018-12-11T06:55:05Z")
i have to check the list of values h=\[2,3,7,12\] in below array details { a= \[1,2,3,4\] , b=\[5,6,7,8\] , c=\[9,10\] , d=\[11,12,13\] } and have to create a new array as below details\_new { a=\[2,3\] , b=\[7\], d=\[12\] } I am usi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=632) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=634)