# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=745 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 746 --- ## [Unable to modify roles or see logstash indices](https://discuss.elastic.co/t/unable-to-modify-roles-or-see-logstash-indices/133601)
**Author:** [@static](https://discuss.elastic.co/u/static)\ **Replies:** 4\ **Last updated:** [June 7, 2018, 8:58pm UTC](https://discuss.elastic.co/t/unable-to-modify-roles-or-see-logstash-indices/133601 "2018-06-07T20:58:57Z")
Hi all, I'm hoping someone can help, and if so that others who have this issue find this topic in the future. I desperately need help finalizing the x-pack installation / security. I followed all the online documentat… --- ## [Convert values in kv-pairs (using ruby filter)](https://discuss.elastic.co/t/convert-values-in-kv-pairs-using-ruby-filter/135035)
**Author:** [@oleg31337](https://discuss.elastic.co/u/oleg31337)\ **Replies:** 4\ **Last updated:** [June 7, 2018, 6:40pm UTC](https://discuss.elastic.co/t/convert-values-in-kv-pairs-using-ruby-filter/135035 "2018-06-07T18:40:31Z")
Hi there! There was a topic already for that: https://discuss.elastic.co/t/convert-values-in-kv-pairs/39904 but the solution no longer works due to syntax changes in new version. Could anyone please advise what it sho… --- ## [Index names for beats flowing through logstash](https://discuss.elastic.co/t/index-names-for-beats-flowing-through-logstash/134196)
**Author:** [@reeuwijk](https://discuss.elastic.co/u/reeuwijk)\ **Replies:** 1\ **Last updated:** [June 7, 2018, 5:40pm UTC](https://discuss.elastic.co/t/index-names-for-beats-flowing-through-logstash/134196 "2018-06-07T17:40:52Z")
I'm pretty sure I'm missing something obvious, but ths has taken far too much time already, and I haven't found anyone with the same problem, so I hope somebody is willing to help this beginner. I am setting up a monito… --- ## [Logstash configuration error when using filter](https://discuss.elastic.co/t/logstash-configuration-error-when-using-filter/135013)
**Author:** [@vkrishna91](https://discuss.elastic.co/u/vkrishna91)\ **Replies:** 2\ **Last updated:** [June 7, 2018, 4:25pm UTC](https://discuss.elastic.co/t/logstash-configuration-error-when-using-filter/135013 "2018-06-07T16:25:47Z")
Hi, I have installed logstash 6.2 and with the following configuration i am not able to start the logstash. I am getting error at the filter configuration, but i tried to change it and getting the same result. Can anyon… --- ## [Issue using Logstash JDBC Input Plugin with Multiple Dynamic Tables](https://discuss.elastic.co/t/issue-using-logstash-jdbc-input-plugin-with-multiple-dynamic-tables/134832)
**Author:** [@Joaquim\_Freitas](https://discuss.elastic.co/u/Joaquim_Freitas)\ **Replies:** 5\ **Last updated:** [June 7, 2018, 4:18pm UTC](https://discuss.elastic.co/t/issue-using-logstash-jdbc-input-plugin-with-multiple-dynamic-tables/134832 "2018-06-07T16:18:30Z")
Hi All ! I'm implementing a Logstash solution to fetch events from MSSQL database with the relevant data present: In a fixed set of well known database tables with fixed columns; BUT also have the most important … --- ## [Need help to get Key Value into Elastic](https://discuss.elastic.co/t/need-help-to-get-key-value-into-elastic/134882)
**Author:** [@paano](https://discuss.elastic.co/u/paano)\ **Replies:** 6\ **Last updated:** [June 7, 2018, 4:17pm UTC](https://discuss.elastic.co/t/need-help-to-get-key-value-into-elastic/134882 "2018-06-07T16:17:31Z")
ELK version 6.2.3 Sample data as below {'\_links': {'doc': {'href': '/mgmt/docs/status/MQSystemResources'}, 'self': {'href': '/mgmt/status/default/MQSystemResources'}}, 'MQSystemResources': {'HAStatus': 'Online', 'Tota… --- ## [Date Filter from DD/MMM/YYYY:hh:MM:ss to @timestamp not working](https://discuss.elastic.co/t/date-filter-from-dd-mmm-yyyymm-ss-to-timestamp-not-working/135007)
**Author:** [@nareshreddyp](https://discuss.elastic.co/u/nareshreddyp)\ **Replies:** 2\ **Last updated:** [June 7, 2018, 3:39pm UTC](https://discuss.elastic.co/t/date-filter-from-dd-mmm-yyyymm-ss-to-timestamp-not-working/135007 "2018-06-07T15:39:24Z")
I want to match the date and timestamp from the log file. Date from my logfile is below 07/Jun/2018:04:35:00.458 I have used grok filter to extract the information using (?%{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE… --- ## [Reading Data From Kafka and use filter json fails with ParserError](https://discuss.elastic.co/t/reading-data-from-kafka-and-use-filter-json-fails-with-parsererror/132179)
**Author:** [@paano](https://discuss.elastic.co/u/paano)\ **Replies:** 18\ **Last updated:** [June 7, 2018, 3:13pm UTC](https://discuss.elastic.co/t/reading-data-from-kafka-and-use-filter-json-fails-with-parsererror/132179 "2018-06-07T15:13:57Z")
Version: 6.2.3 Operating System: RHEL Config File : input { kafka { bootstrap\_servers =\> "127.0.0.1:9092" auto\_offset\_reset =\> earliest topics =\> \["test"\] gr… --- ## [How to make Logstash write internal logs to a file? (--path.logs fails)](https://discuss.elastic.co/t/how-to-make-logstash-write-internal-logs-to-a-file-path-logs-fails/126822)
**Author:** [@CDuv](https://discuss.elastic.co/u/CDuv)\ **Replies:** 5\ **Last updated:** [June 7, 2018, 1:53pm UTC](https://discuss.elastic.co/t/how-to-make-logstash-write-internal-logs-to-a-file-path-logs-fails/126822 "2018-06-07T13:53:29Z")
Context I am building a program that will pass an Apache log file into Logstash and output the result (after parsing and filtering) to an external database (Elastic, MongoDB, etc.). Basically, the program will execute f… --- ## [Issue with Ruby and logstash-input-imap](https://discuss.elastic.co/t/issue-with-ruby-and-logstash-input-imap/134935)
**Author:** [@Alexa](https://discuss.elastic.co/u/Alexa)\ **Replies:** 10\ **Last updated:** [June 7, 2018, 1:33pm UTC](https://discuss.elastic.co/t/issue-with-ruby-and-logstash-input-imap/134935 "2018-06-07T13:33:35Z")
Afer configuring the impa plugin on logstash and running the following command: bin/logstash -f email\_log.conf --path.data /var/lib/logstash/imap --config.reload.automatic I get the following error: \[ERROR\] 2018-06-06… --- ## [Logstash adds 9200 port to elastic host during healthcheck](https://discuss.elastic.co/t/logstash-adds-9200-port-to-elastic-host-during-healthcheck/134974)
**Author:** [@Tatiana\_Koroleva](https://discuss.elastic.co/u/Tatiana_Koroleva)\ **Replies:** 2\ **Last updated:** [June 7, 2018, 1:15pm UTC](https://discuss.elastic.co/t/logstash-adds-9200-port-to-elastic-host-during-healthcheck/134974 "2018-06-07T13:15:08Z")
I run elastic cluster in aws (as a service) , so its https endpoint does not have a 9200 ending. But logstash still adds :9200 at the end during healthcheck - \[2018-06-07T11:46:38,218\]\[INFO \]\[logstash.outputs.elasticsea… --- ## [Intermittent \_rubyexception Errors](https://discuss.elastic.co/t/intermittent-rubyexception-errors/134991)
**Author:** [@Krog](https://discuss.elastic.co/u/Krog)\ **Replies:** 2\ **Last updated:** [June 7, 2018, 12:58pm UTC](https://discuss.elastic.co/t/intermittent-rubyexception-errors/134991 "2018-06-07T12:58:38Z")
Can anyone tell me if the init configuration option is supported and/or necessary in ruby script files, and if so, how to use it? My script file (yesterday.rb) looks like this: def register(params) @f\_timestamp = para… --- ## [How to frame json object in Logstash?](https://discuss.elastic.co/t/how-to-frame-json-object-in-logstash/134944)
**Author:** [@kartheek91](https://discuss.elastic.co/u/kartheek91)\ **Replies:** 4\ **Last updated:** [June 7, 2018, 12:50pm UTC](https://discuss.elastic.co/t/how-to-frame-json-object-in-logstash/134944 "2018-06-07T12:50:35Z")
Can anyone tell me how to frame json object in logstash configuration. --- ## [Mapping ip address to specific network](https://discuss.elastic.co/t/mapping-ip-address-to-specific-network/132268)
**Author:** [@mmalewicz](https://discuss.elastic.co/u/mmalewicz)\ **Replies:** 8\ **Last updated:** [June 7, 2018, 12:10pm UTC](https://discuss.elastic.co/t/mapping-ip-address-to-specific-network/132268 "2018-06-07T12:10:52Z")
Hello, I have a problem preparing logstash script. I'm parsing a syslog, then I get a source IPv4 address. I would like to match this addres to a networks defined in file and add a field with network name. example of … --- ## [Unable to perform incremental load](https://discuss.elastic.co/t/unable-to-perform-incremental-load/134230)
**Author:** [@Deepak\_Dandotiya](https://discuss.elastic.co/u/Deepak_Dandotiya)\ **Replies:** 4\ **Last updated:** [June 7, 2018, 11:20am UTC](https://discuss.elastic.co/t/unable-to-perform-incremental-load/134230 "2018-06-07T11:20:00Z")
Hello, I am new to elasticsearch and logstash. Now I am running below emp-index-logstash.conf file using logstash to perform the incremental load using \<./logstash -f /data/deepak/logstash-6.2.4/bin/emp-index-logstash.… --- ## [Steps to Install and Configure SQL Server with Logstash](https://discuss.elastic.co/t/steps-to-install-and-configure-sql-server-with-logstash/134854)
**Author:** [@Minasys](https://discuss.elastic.co/u/Minasys)\ **Replies:** 2\ **Last updated:** [June 7, 2018, 10:52am UTC](https://discuss.elastic.co/t/steps-to-install-and-configure-sql-server-with-logstash/134854 "2018-06-07T10:52:32Z")
I'm trying to find a step by step tutorial to install and configure SQL Server with Logstash or Elasticsearch and I can't find full ones. I need help doing that as I'm new to Elasticsearch! --- ## [Logstash match value form external file](https://discuss.elastic.co/t/logstash-match-value-form-external-file/134367)
**Author:** [@Md\_Ghulam\_khaja](https://discuss.elastic.co/u/Md_Ghulam_khaja)\ **Replies:** 6\ **Last updated:** [June 7, 2018, 10:42am UTC](https://discuss.elastic.co/t/logstash-match-value-form-external-file/134367 "2018-06-07T10:42:11Z")
I want to filter out logs which contains specific IP mapped from external file. DefaultUdpTransportMapping@182e8b8peerAddress=192.168.4.x, processed=false I have a txt file which contains list of IP line by line. I wan… --- ## [Will logstash 6.2.4 support log rotation default](https://discuss.elastic.co/t/will-logstash-6-2-4-support-log-rotation-default/134957)
**Author:** [@shanmugaraj88](https://discuss.elastic.co/u/shanmugaraj88)\ **Replies:** 0\ **Last updated:** [June 7, 2018, 10:41am UTC](https://discuss.elastic.co/t/will-logstash-6-2-4-support-log-rotation-default/134957 "2018-06-07T10:41:20Z")
We faced issues that output syslog logs are not getting zipped neither rotated in 2.1.0 so we are planning to migrate to 6.2.4 . so do i need to configure any additional steps to get this working? Please share your input… --- ## [Kafka input plugin issues](https://discuss.elastic.co/t/kafka-input-plugin-issues/134951)
**Author:** [@anush\_jayan](https://discuss.elastic.co/u/anush_jayan)\ **Replies:** 0\ **Last updated:** [June 7, 2018, 10:16am UTC](https://discuss.elastic.co/t/kafka-input-plugin-issues/134951 "2018-06-07T10:16:07Z")
hi this is my input configuration for logstash but i can see lag on kafka server for this topic can anyone help me with this input{ kafka{ bootstrap\_servers =\> "xxx.xx.xx.xxxx:9092,xxx.xx.xx.xxx:9092,xxx.xx.xx.xxx:9… --- ## [Failed to execute action : Logstash : Pipeline action create](https://discuss.elastic.co/t/failed-to-execute-action-logstash-pipeline-action-create/134843)
**Author:** [@irfanjs](https://discuss.elastic.co/u/irfanjs)\ **Replies:** 8\ **Last updated:** [June 7, 2018, 9:04am UTC](https://discuss.elastic.co/t/failed-to-execute-action-logstash-pipeline-action-create/134843 "2018-06-07T09:04:18Z")
hi, have following setup : CentOS 7 machine Logstash version : 6.2.4 elasticsearch version : 6.2.4-1 kibana : 6.2.4 all the services (logstash , kibana and elasticsearch we working fine ) . following file is a inp… --- ## [Logstash input sqs error](https://discuss.elastic.co/t/logstash-input-sqs-error/134349)
**Author:** [@fafahei](https://discuss.elastic.co/u/fafahei)\ **Replies:** 2\ **Last updated:** [June 7, 2018, 7:42am UTC](https://discuss.elastic.co/t/logstash-input-sqs-error/134349 "2018-06-07T07:42:37Z")
\[ERROR\] 2018-06-04 01:44:22.008 \[\[main\]\#\ **Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 7\ **Last updated:** [June 7, 2018, 7:38am UTC](https://discuss.elastic.co/t/logstash-management-info-wrong/134836 "2018-06-07T07:38:52Z") In our kibana UI the following is displayed when going to Management -\> Logstash -\> Pipelines \[image\] However, when tailing: sudo tail -f /var/log/logstash/logstash-plain.log \[2018-06-06T15:46:44,686\]\[INFO \]\[logstash… --- ## [Grok patterns format in logstash](https://discuss.elastic.co/t/grok-patterns-format-in-logstash/134908)
**Author:** [@Leorxt](https://discuss.elastic.co/u/Leorxt)\ **Replies:** 0\ **Last updated:** [June 7, 2018, 7:01am UTC](https://discuss.elastic.co/t/grok-patterns-format-in-logstash/134908 "2018-06-07T07:01:39Z")
Mostly my patterns working but after "Framework Error in :\]" part does not see in the logs.How can print rest of it on logs. I tried at the last in patterns, but still does not work "%{GREEDYDATA:restofthem}" My grok … --- ## [Log4net \> Redis \> Logstash](https://discuss.elastic.co/t/log4net-redis-logstash/134899)
**Author:** [@cowensel](https://discuss.elastic.co/u/cowensel)\ **Replies:** 0\ **Last updated:** [June 7, 2018, 6:40am UTC](https://discuss.elastic.co/t/log4net-redis-logstash/134899 "2018-06-07T06:40:29Z")
I wonder if anyone can assist/advise. We currently have an application that is using Log4net for its logging which we currently udp to logstash which we have found to be unreliable. We have a redis instance we would lik… --- ## [Topbeat limit filter](https://discuss.elastic.co/t/topbeat-limit-filter/134815)
**Author:** [@andres\_c](https://discuss.elastic.co/u/andres_c)\ **Replies:** 1\ **Last updated:** [June 6, 2018, 11:10pm UTC](https://discuss.elastic.co/t/topbeat-limit-filter/134815 "2018-06-06T23:10:41Z")
hello i need help i am new I'm trying to add a threshold to the topbeat logstash filter, I want to add a message to tags but do not add it to me This is the filter I have: filter { if "topbeat" in \[tags\]{ if \[{mem}{… --- ## [Streaming events using logstash](https://discuss.elastic.co/t/streaming-events-using-logstash/134825)
**Author:** [@Prakash\_Punj](https://discuss.elastic.co/u/Prakash_Punj)\ **Replies:** 1\ **Last updated:** [June 6, 2018, 6:58pm UTC](https://discuss.elastic.co/t/streaming-events-using-logstash/134825 "2018-06-06T18:58:48Z")
Hi ! , I have question about streaming events to elastic search. For example say we have a file access.log which is continuously changes and in the logstash we have defined this file as an input. After I run logstash, … --- ## [File input plugin 4.1.2 \[filewatch.tailmode.handlers.shrink\] mismatch on sincedb\_value.watched\_file, this should have been handled by Discoverer](https://discuss.elastic.co/t/file-input-plugin-4-1-2-filewatch-tailmode-handlers-shrink-mismatch-on-sincedb-value-watched-file-this-should-have-been-handled-by-discoverer/134837)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 0\ **Last updated:** [June 6, 2018, 3:21pm UTC](https://discuss.elastic.co/t/file-input-plugin-4-1-2-filewatch-tailmode-handlers-shrink-mismatch-on-sincedb-value-watched-file-this-should-have-been-handled-by-discoverer/134837 "2018-06-06T15:21:05Z")
Hi Team, In the new version of file input plugin - when this warn happens \[WARN \]\[filewatch.tailmode.handlers.shrink\] mismatch on sincedb\_value.watched\_file, this should have been handled by Discoverer, it seems to stop… --- ## [Format object with key value](https://discuss.elastic.co/t/format-object-with-key-value/134735)
**Author:** [@arnaud\_beun](https://discuss.elastic.co/u/arnaud_beun)\ **Replies:** 2\ **Last updated:** [June 6, 2018, 2:49pm UTC](https://discuss.elastic.co/t/format-object-with-key-value/134735 "2018-06-06T14:49:44Z")
Hi, I am exporting AWS WAF samples to or elasticsearch and JSON WAF return look like this : { "@timestamp":"2018-06-01T05:05:48.792Z", "Headers":\[ { "Value":"xxx", "Name":"Host" }, { "Value":"Mozilla/5.0 (Windo… --- ## [Convert EPOCH time to Elapsed Time](https://discuss.elastic.co/t/convert-epoch-time-to-elapsed-time/134794)
**Author:** [@sahadevgd](https://discuss.elastic.co/u/sahadevgd)\ **Replies:** 0\ **Last updated:** [June 6, 2018, 11:41am UTC](https://discuss.elastic.co/t/convert-epoch-time-to-elapsed-time/134794 "2018-06-06T11:41:33Z")
Hi Team, I have an application which outputs the below data in csv format. The timeStamp format is the EPOCH time. We want this to be shown as Elapsed Timei in Kibana. (Example Below) Actual Output from the Application… --- ## [Not able to fetch logs from S3 using Logstash 6.2.4](https://discuss.elastic.co/t/not-able-to-fetch-logs-from-s3-using-logstash-6-2-4/134791)
**Author:** [@Deepak\_Jangid](https://discuss.elastic.co/u/Deepak_Jangid)\ **Replies:** 0\ **Last updated:** [June 6, 2018, 11:31am UTC](https://discuss.elastic.co/t/not-able-to-fetch-logs-from-s3-using-logstash-6-2-4/134791 "2018-06-06T11:31:16Z")
Hello all, I am using Logstash 6.2.4 to fetch logs from s3-location. These are my configurations: input { s3 { access\_key\_id =\> "#########" secret\_access\_key =\> "#########" bucket =\> "mybucket" region =\> "us-east-… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=744) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=746)