# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=581 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 582 --- ## [Separate production logs - logstash](https://discuss.elastic.co/t/separate-production-logs-logstash/173178)
**Author:** [@Habbiot](https://discuss.elastic.co/u/Habbiot)\ **Replies:** 1\ **Last updated:** [March 20, 2019, 3:42pm UTC](https://discuss.elastic.co/t/separate-production-logs-logstash/173178 "2019-03-20T15:42:49Z")
This is a to continue \[https://discuss.elastic.co/t/separate-production-logs/173146/2\]. I have an single ELK stack collecting both Production and Development logs. Logstash is parses logs to elasticsearch. How can I u… --- ## [Grok best practice](https://discuss.elastic.co/t/grok-best-practice/172871)
**Author:** [@kharvey](https://discuss.elastic.co/u/kharvey)\ **Replies:** 4\ **Last updated:** [March 20, 2019, 3:37pm UTC](https://discuss.elastic.co/t/grok-best-practice/172871 "2019-03-20T15:37:52Z")
I'm getting farther and farther in the weeds with building out custom grok stuff. And before I make it too far I would like to know what the best practice for handling grok patterns is. Is it best to make one giant gro… --- ## [Azure Event Hub & Logstash Clusters](https://discuss.elastic.co/t/azure-event-hub-logstash-clusters/173176)
**Author:** [@lgwapnitsky](https://discuss.elastic.co/u/lgwapnitsky)\ **Replies:** 0\ **Last updated:** [March 20, 2019, 2:54pm UTC](https://discuss.elastic.co/t/azure-event-hub-logstash-clusters/173176 "2019-03-20T14:54:30Z")
I am currently working with my developers to manage log and event ingestion using Azure Event Hub and ELK. Until this morning, my design included data going through the following path: App -\> AEH -\> Logstash Cluster -\> … --- ## [Multiline Logstash](https://discuss.elastic.co/t/multiline-logstash/172915)
**Author:** [@Sanriya\_Krimzn](https://discuss.elastic.co/u/Sanriya_Krimzn)\ **Replies:** 4\ **Last updated:** [March 20, 2019, 12:53pm UTC](https://discuss.elastic.co/t/multiline-logstash/172915 "2019-03-20T12:53:48Z")
Hi I am using multilint to extract data from Sample Log \[This \*'create' =\> \* array ( 'key1' =\> 'value1', 'key2' =\> 'value2', 'key6' =\> 'value3' ), ) \[2014-03-02 17:34:20\] - 127.0.0.1|DEBUG| flush\_multi\_line Co… --- ## [Logstash starts successfully but doesn't generate events](https://discuss.elastic.co/t/logstash-starts-successfully-but-doesnt-generate-events/173141)
**Author:** [@Yogendra\_Hada](https://discuss.elastic.co/u/Yogendra_Hada)\ **Replies:** 0\ **Last updated:** [March 20, 2019, 12:33pm UTC](https://discuss.elastic.co/t/logstash-starts-successfully-but-doesnt-generate-events/173141 "2019-03-20T12:33:55Z")
Hello, Logstash starts successfully but doesn't generate events when I change path to include subdirectories. My logstash.conf looks like:- input { file { path =\> "/local/kdmlogs/2019031905/\*/\*" exclude =\> "\*.s3lo… --- ## [Special characters in data preventing CSV import using logstash 6.6.2](https://discuss.elastic.co/t/special-characters-in-data-preventing-csv-import-using-logstash-6-6-2/173014)
**Author:** [@touchstonemark](https://discuss.elastic.co/u/touchstonemark)\ **Replies:** 4\ **Last updated:** [March 20, 2019, 11:13am UTC](https://discuss.elastic.co/t/special-characters-in-data-preventing-csv-import-using-logstash-6-6-2/173014 "2019-03-20T11:13:44Z")
Hi all, I am trying to import a CSV file which is an export from a Service Now ticketing system into ES using Logstash 6.6.2 - but it works if I use manually entered test data - but as soon as I use actual data it fails.… --- ## [Update operation doesn't update same columns](https://discuss.elastic.co/t/update-operation-doesnt-update-same-columns/173114)
**Author:** [@Varun\_Mahajan](https://discuss.elastic.co/u/Varun_Mahajan)\ **Replies:** 0\ **Last updated:** [March 20, 2019, 10:31am UTC](https://discuss.elastic.co/t/update-operation-doesnt-update-same-columns/173114 "2019-03-20T10:31:36Z")
Hi , I have a logstash in which in am using document\_id="update" operation , But in update operation the columns value are not getting overridden , Here's my logstash , Hope to receive some replies input { beats { po… --- ## [Logstash-output-email plugin installation Error](https://discuss.elastic.co/t/logstash-output-email-plugin-installation-error/171106)
**Author:** [@predator](https://discuss.elastic.co/u/predator)\ **Replies:** 1\ **Last updated:** [March 20, 2019, 8:55am UTC](https://discuss.elastic.co/t/logstash-output-email-plugin-installation-error/171106 "2019-03-20T08:55:41Z")
Hi, I am trying to install the Logstash-output-email but it showing the ERROR: File /usr/share/logstash/Gemfile does not exits Please help how to install Gemfile. Thanks Vishal --- ## [How extract specific fields from JSON input with Logstash filters?](https://discuss.elastic.co/t/how-extract-specific-fields-from-json-input-with-logstash-filters/173066)
**Author:** [@mdobrin](https://discuss.elastic.co/u/mdobrin)\ **Replies:** 0\ **Last updated:** [March 20, 2019, 1:54am UTC](https://discuss.elastic.co/t/how-extract-specific-fields-from-json-input-with-logstash-filters/173066 "2019-03-20T01:54:03Z")
I am currently trying to parse a JSON string using Logstash for sending to an Elastic Search DB. The input string looks like the following: "message": """{"resourceId":"201987-20190320-05307201","body":{"orderNumber":"… --- ## [How to put multiple hosts in one jmx conf file?](https://discuss.elastic.co/t/how-to-put-multiple-hosts-in-one-jmx-conf-file/173061)
**Author:** [@Stephen\_Lu](https://discuss.elastic.co/u/Stephen_Lu)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 11:03pm UTC](https://discuss.elastic.co/t/how-to-put-multiple-hosts-in-one-jmx-conf-file/173061 "2019-03-19T23:03:55Z")
I have a jmx input, with below jmx conf file: { "host" : "hostname", "port" : port, "queries" : \[ { "object\_name" : "java.lang:type=Memory", "object\_alias" : "M… --- ## [Grok pattern need](https://discuss.elastic.co/t/grok-pattern-need/173024)
**Author:** [@nagr](https://discuss.elastic.co/u/nagr)\ **Replies:** 1\ **Last updated:** [March 19, 2019, 7:33pm UTC](https://discuss.elastic.co/t/grok-pattern-need/173024 "2019-03-19T19:33:18Z")
Hi All, Could you share your thought to get the exact grok pattern for the below content, log file --\> "ServerManager~31bf3856ad364e35~amd64~~6.3.9600.16384, Remote Parent: IIS-HttpErrors, Intended State: Staged" so t… --- ## [Kafka AvroDesrializer - Logstash](https://discuss.elastic.co/t/kafka-avrodesrializer-logstash/172189)
**Author:** [@DSPN](https://discuss.elastic.co/u/DSPN)\ **Replies:** 4\ **Last updated:** [March 19, 2019, 6:10pm UTC](https://discuss.elastic.co/t/kafka-avrodesrializer-logstash/172189 "2019-03-19T18:10:32Z")
input { kafka{ group\_id =\> "group1" topics =\> \["Topic1"\] bootstrap\_servers =\> "192.168.0.1:9092" codec =\> avro { schema\_uri =\> "=/logstash-6.6.0/avrotest.avsc" } key\_deserializer\_class =\> "io.confluent.kafka.seri… --- ## [Aggregate filter logstash not working when the input is elasticsearch](https://discuss.elastic.co/t/aggregate-filter-logstash-not-working-when-the-input-is-elasticsearch/173016)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 4:44pm UTC](https://discuss.elastic.co/t/aggregate-filter-logstash-not-working-when-the-input-is-elasticsearch/173016 "2019-03-19T16:44:59Z")
Hello, I am wondering why I don't have the same result when I use elasticsearch (instead of a file) as an input in my logstash configuration. This my log file : {"@timestamp":"2019-03-11T17:12:39.929+01:00","@version"… --- ## [How to start multiple Configuration in logstash](https://discuss.elastic.co/t/how-to-start-multiple-configuration-in-logstash/172981)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 1\ **Last updated:** [March 19, 2019, 3:28pm UTC](https://discuss.elastic.co/t/how-to-start-multiple-configuration-in-logstash/172981 "2019-03-19T15:28:26Z")
HI, I'm running my ELK in windows and i want to run multiple conf in single command how can i perform it and i tried below command but it takes last command only, logstash.bat -f C:\\Ganesh\\ELK\\logstash-6.6.1\\logstash-6… --- ## [Logstash use environment variable in yml or pipeline conf](https://discuss.elastic.co/t/logstash-use-environment-variable-in-yml-or-pipeline-conf/172990)
**Author:** [@hubertn](https://discuss.elastic.co/u/hubertn)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 2:02pm UTC](https://discuss.elastic.co/t/logstash-use-environment-variable-in-yml-or-pipeline-conf/172990 "2019-03-19T14:02:45Z")
I have tried to use environment variable in yml and conf file for some reason the variable did not get resolve when i was runnig the pipeline as services. It complained that the variable was not declared. I tried to put… --- ## [Logstash docker fails to connect to Elasticsearch-docker](https://discuss.elastic.co/t/logstash-docker-fails-to-connect-to-elasticsearch-docker/172977)
**Author:** [@Sudhanshu\_Garg](https://discuss.elastic.co/u/Sudhanshu_Garg)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 1:27pm UTC](https://discuss.elastic.co/t/logstash-docker-fails-to-connect-to-elasticsearch-docker/172977 "2019-03-19T13:27:38Z")
I am running ELK 6.6.1 docker on my Ubuntu 16.04 host PC. I am trying to run docker for all 4 components - Elasticsearch, logstash, filebeat and kibana. But logstash docker is not able to connect to elasticsearch docker… --- ## [Logstash Pipeline Set-up and Architecture](https://discuss.elastic.co/t/logstash-pipeline-set-up-and-architecture/172948)
**Author:** [@robincher](https://discuss.elastic.co/u/robincher)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 11:04am UTC](https://discuss.elastic.co/t/logstash-pipeline-set-up-and-architecture/172948 "2019-03-19T11:04:40Z")
Hi all, Currently i have one cluster of LogStash nodes ,and fronted by a load-balancer with a fully qualified domain name. For the logstash configuration, i allows all beats to pipe their logs into a single port (by def… --- ## [Index files according to pattern or interval name](https://discuss.elastic.co/t/index-files-according-to-pattern-or-interval-name/172728)
**Author:** [@fadihaddad](https://discuss.elastic.co/u/fadihaddad)\ **Replies:** 0\ **Last updated:** [March 18, 2019, 8:45am UTC](https://discuss.elastic.co/t/index-files-according-to-pattern-or-interval-name/172728 "2019-03-18T08:45:35Z")
is there a way o can index files on a name interval, for example : I have these file 2015125201\_test.csv 2015125211\_test.csv 2015125221\_test.csv 2015125231\_test.csv 2015125241\_test.csv 2015125301\_test.csv 20151253… --- ## [Show only specific states in Pie Chart visualization](https://discuss.elastic.co/t/show-only-specific-states-in-pie-chart-visualization/172930)
**Author:** [@rhari](https://discuss.elastic.co/u/rhari)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 9:44am UTC](https://discuss.elastic.co/t/show-only-specific-states-in-pie-chart-visualization/172930 "2019-03-19T09:44:39Z")
New to LogStash, ElasticSearch, Kibana I am pulling data from Postgres DB table and updated records in LogStash. Now I wanted to create Piechart dashboard based on different states like Critical, Major and created new … --- ## [Logstash jdbc plugin, connection timed out](https://discuss.elastic.co/t/logstash-jdbc-plugin-connection-timed-out/172629)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 7\ **Last updated:** [March 19, 2019, 8:48am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-connection-timed-out/172629 "2019-03-19T08:48:27Z")
hi all, I am using logstash jdbc plugin to load data from database, my jdbc conf is as following: input { jdbc { jdbc\_driver\_library =\> "sqljdbc42.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServ… --- ## [Logstash service crash 6.6 (geoip) filter](https://discuss.elastic.co/t/logstash-service-crash-6-6-geoip-filter/172641)
**Author:** [@naftalin\_n](https://discuss.elastic.co/u/naftalin_n)\ **Replies:** 5\ **Last updated:** [March 19, 2019, 6:10am UTC](https://discuss.elastic.co/t/logstash-service-crash-6-6-geoip-filter/172641 "2019-03-19T06:10:02Z")
Hi, I using filebeat with elk stack 6.6.3 Logstash service crash when getting json log with remote address fo Geoip filter. here is my configuration file: input { beats { port =\> 5044 } } filter { … --- ## [LOGSTASH\_Pull JDBC logs from cloud server to local using SSH key](https://discuss.elastic.co/t/logstash-pull-jdbc-logs-from-cloud-server-to-local-using-ssh-key/171965)
**Author:** [@padma](https://discuss.elastic.co/u/padma)\ **Replies:** 2\ **Last updated:** [March 19, 2019, 6:01am UTC](https://discuss.elastic.co/t/logstash-pull-jdbc-logs-from-cloud-server-to-local-using-ssh-key/171965 "2019-03-19T06:01:46Z")
How to pull the JDBC logs from the cloud server to local using ssh key --- ## [Logstash jdbc pluging reconnecting attempts](https://discuss.elastic.co/t/logstash-jdbc-pluging-reconnecting-attempts/172897)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 5:19am UTC](https://discuss.elastic.co/t/logstash-jdbc-pluging-reconnecting-attempts/172897 "2019-03-19T05:19:35Z")
hi all, I am using jdbc pluging, my scheduling of logstash conf file is as following: input { jdbc { jdbc\_driver\_library =\> "sqljdbc42.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_c… --- ## [Logstash output differs in different runs](https://discuss.elastic.co/t/logstash-output-differs-in-different-runs/172630)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 2\ **Last updated:** [March 19, 2019, 5:03am UTC](https://discuss.elastic.co/t/logstash-output-differs-in-different-runs/172630 "2019-03-19T05:03:03Z")
hi all, I am using jdbc plugin to load data from database, my logstash config file is as following: input { jdbc { jdbc\_driver\_library =\> "sqljdbc42.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLSe… --- ## [Flush Pattern is Not working](https://discuss.elastic.co/t/flush-pattern-is-not-working/172886)
**Author:** [@Sanriya\_Krimzn](https://discuss.elastic.co/u/Sanriya_Krimzn)\ **Replies:** 0\ **Last updated:** [March 19, 2019, 3:18am UTC](https://discuss.elastic.co/t/flush-pattern-is-not-working/172886 "2019-03-19T03:18:10Z")
Hi I am using flush\_pattern in multiline to define the end of a multiline block. It is not working. Below is my code snippet. multiline { \* \* pattern =\> "^\[" \* what =\> "previous"\* \* … --- ## [Binding local port for Logstash http output plugin](https://discuss.elastic.co/t/binding-local-port-for-logstash-http-output-plugin/172877)
**Author:** [@Quiza12](https://discuss.elastic.co/u/Quiza12)\ **Replies:** 2\ **Last updated:** [March 19, 2019, 12:58am UTC](https://discuss.elastic.co/t/binding-local-port-for-logstash-http-output-plugin/172877 "2019-03-19T00:58:35Z")
Hi all, I'm using the logstash-http-output plugin to send logs (converted to a SOAP message during filtering) to an endpoint on another server. I'm having problems trying to find a way to bind a local port for the http… --- ## [Filter out beat.\*](https://discuss.elastic.co/t/filter-out-beat/172868)
**Author:** [@teamg](https://discuss.elastic.co/u/teamg)\ **Replies:** 2\ **Last updated:** [March 18, 2019, 11:03pm UTC](https://discuss.elastic.co/t/filter-out-beat/172868 "2019-03-18T23:03:25Z")
I have no use for the beat.hostname, beat.name or beat.version fields and would like to filter them out. This filter does not work: cat /etc/logstash/conf.d/input.conf filter { mutate { remove\_field =\> \[ "beat.name"… --- ## [Could not index event to Elasticsearch after upgrade 5.6.14 -\> 6.6](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-after-upgrade-5-6-14-6-6/172849)
**Author:** [@Slartibartfast](https://discuss.elastic.co/u/Slartibartfast)\ **Replies:** 1\ **Last updated:** [March 18, 2019, 10:27pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-after-upgrade-5-6-14-6-6/172849 "2019-03-18T22:27:28Z")
I have a very small Elastic Stack running one node and with LS and FB on the same server as ES. Seven other servers kick syslogs and audit logs to LS via FB. When I inherited the setup, it was running ELK 2.x and I succe… --- ## [He given configuration is invalid. Reason: Illegal character in scheme name at index 0: \["172.23.130.220:9200","172.23.130.164:9200"\]](https://discuss.elastic.co/t/he-given-configuration-is-invalid-reason-illegal-character-in-scheme-name-at-index-0-172-23-130-220-9200-172-23-130-164-9200/170152)
**Author:** [@vaibhavjain882](https://discuss.elastic.co/u/vaibhavjain882)\ **Replies:** 3\ **Last updated:** [March 18, 2019, 8:38pm UTC](https://discuss.elastic.co/t/he-given-configuration-is-invalid-reason-illegal-character-in-scheme-name-at-index-0-172-23-130-220-9200-172-23-130-164-9200/170152 "2019-03-18T20:38:14Z")
Hi all I am deploy logstash consumer on kubernetes with following specification: Logstash version logstash:6.6.1 I am picking up all the values from the environment variable, my config file looks like this: input { … --- ## [Logstash Netflow udp queue size](https://discuss.elastic.co/t/logstash-netflow-udp-queue-size/172845)
**Author:** [@Ryne\_Keel](https://discuss.elastic.co/u/Ryne_Keel)\ **Replies:** 1\ **Last updated:** [March 18, 2019, 8:09pm UTC](https://discuss.elastic.co/t/logstash-netflow-udp-queue-size/172845 "2019-03-18T20:09:46Z")
I am trying to find out how to view the queue so I can know whether or not I need to increase settings. Trying to fine tune this to make sure that we aren't missing any netflow data. I can compare it against our other ne… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=580) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=582)