# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=932 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 933 --- ## [Logstash appears to be outputting to ES, but nothing shows up in Kibana](https://discuss.elastic.co/t/logstash-appears-to-be-outputting-to-es-but-nothing-shows-up-in-kibana/95931)
**Author:** [@benjiwheeler](https://discuss.elastic.co/u/benjiwheeler)\ **Replies:** 1\ **Last updated:** [August 5, 2017, 5:46pm UTC](https://discuss.elastic.co/t/logstash-appears-to-be-outputting-to-es-but-nothing-shows-up-in-kibana/95931 "2017-08-05T17:46:51Z")
My logstash instance seems to report that it’s running fine, and connecting to ES. I also can confirm the ES endpoint is correct, and my AWS IAM role gives me access privs, by its response to a curl GET. Meanwhile, other… --- ## [Trying to add fields to CSV and fill them with data](https://discuss.elastic.co/t/trying-to-add-fields-to-csv-and-fill-them-with-data/95952)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 1\ **Last updated:** [August 5, 2017, 4:10pm UTC](https://discuss.elastic.co/t/trying-to-add-fields-to-csv-and-fill-them-with-data/95952 "2017-08-05T16:10:28Z")
Hi and thanks for reading and helping. Posting because my search got nothing I can use. I am uploading .csv files from a directory structure that represents countries. The filenames don’t give me anything but the date. … --- ## [Logstash kafka output: Failed to construct kafka producer and stuck after no connection](https://discuss.elastic.co/t/logstash-kafka-output-failed-to-construct-kafka-producer-and-stuck-after-no-connection/95983)
**Author:** [@ilyasim](https://discuss.elastic.co/u/ilyasim)\ **Replies:** 0\ **Last updated:** [August 5, 2017, 4:04pm UTC](https://discuss.elastic.co/t/logstash-kafka-output-failed-to-construct-kafka-producer-and-stuck-after-no-connection/95983 "2017-08-05T16:04:36Z")
Hi, i have a Logstash instance in a docker container and it sending logs to kafka using the kafka-output-plugin the instance configured with the default settings (including the kafka host and the topic ). Also i am us… --- ## [Connecting elasticsearch to a separate Logstash instance without editting any files](https://discuss.elastic.co/t/connecting-elasticsearch-to-a-separate-logstash-instance-without-editting-any-files/95904)
**Author:** [@jayunit100](https://discuss.elastic.co/u/jayunit100)\ **Replies:** 1\ **Last updated:** [August 5, 2017, 3:47pm UTC](https://discuss.elastic.co/t/connecting-elasticsearch-to-a-separate-logstash-instance-without-editting-any-files/95904 "2017-08-05T15:47:51Z")
I want to run elasticsearch from a docker container, and I’m going to run it on the same location I’m stashing my logs. Is there a way to simply launch elastic search with a --logstash=/var/logstash/mylogs/ option, so… --- ## [Multiple configuration files are indexing to all outputs](https://discuss.elastic.co/t/multiple-configuration-files-are-indexing-to-all-outputs/95914)
**Author:** [@gabron](https://discuss.elastic.co/u/gabron)\ **Replies:** 1\ **Last updated:** [August 5, 2017, 3:40pm UTC](https://discuss.elastic.co/t/multiple-configuration-files-are-indexing-to-all-outputs/95914 "2017-08-05T15:40:55Z")
Here’s the back story, I’m setting up a logstash server that will take data from file beats and ship it to the elastic cloud. I originally created a configuration file that would use a beats input, no filters and then o… --- ## [How to configure Logstash cluster](https://discuss.elastic.co/t/how-to-configure-logstash-cluster/95973)
**Author:** [@sesaravanan](https://discuss.elastic.co/u/sesaravanan)\ **Replies:** 1\ **Last updated:** [August 5, 2017, 10:36am UTC](https://discuss.elastic.co/t/how-to-configure-logstash-cluster/95973 "2017-08-05T10:36:14Z")
Hi, How to setup logstash cluster with 3 nodes? Any help appreciated. Thanks, Saravanan --- ## [Mapping filebeat index to geo-point](https://discuss.elastic.co/t/mapping-filebeat-index-to-geo-point/95969)
**Author:** [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\ **Replies:** 3\ **Last updated:** [August 5, 2017, 8:59am UTC](https://discuss.elastic.co/t/mapping-filebeat-index-to-geo-point/95969 "2017-08-05T08:59:42Z")
I used this mapping to my filebeat index template PUT /filebeat-\*/?pretty { “mappings”: { “syslog”: { “properties”: { “geoip.location”: { “type”: “geo\_point” } } } } } and i got this error { “error”: { “r… --- ## [Geoip.location](https://discuss.elastic.co/t/geoip-location/95632)
**Author:** [@elaair](https://discuss.elastic.co/u/elaair)\ **Replies:** 12\ **Last updated:** [August 5, 2017, 3:59am UTC](https://discuss.elastic.co/t/geoip-location/95632 "2017-08-05T03:59:35Z")
If geoip.location appears in the index as a geo\_point, why am I unable to use it in a heat map? Also, why do geoip.longitude and geoip.latitude appear as numbers? Shouldn’t they by default be float? I have a mapping lik… --- ## [There no grok ,geoip information](https://discuss.elastic.co/t/there-no-grok-geoip-information/95636)
**Author:** [@Joe\_Yang](https://discuss.elastic.co/u/Joe_Yang)\ **Replies:** 7\ **Last updated:** [August 4, 2017, 10:48pm UTC](https://discuss.elastic.co/t/there-no-grok-geoip-information/95636 "2017-08-04T22:48:24Z")
We followed this link to configure elasticsearch, filebeat and logslash(https://www.elastic.co/guide/en/logstash/5.4/advanced-pipeline.html), but there’s no ip and geo information after running curl as follows, any advic… --- ## [Logstash query template](https://discuss.elastic.co/t/logstash-query-template/95407)
**Author:** [@exocore123](https://discuss.elastic.co/u/exocore123)\ **Replies:** 10\ **Last updated:** [August 4, 2017, 4:47pm UTC](https://discuss.elastic.co/t/logstash-query-template/95407 "2017-08-04T16:47:21Z")
Change from original question { "query": { "bool": { "must": { "query\_string": { "query": "id=%{id}" } }, "filter": { "range" : { "timestamp" : { … --- ## [Logstash Performance Issue](https://discuss.elastic.co/t/logstash-performance-issue/95948)
**Author:** [@sarthak2016](https://discuss.elastic.co/u/sarthak2016)\ **Replies:** 0\ **Last updated:** [August 4, 2017, 7:42pm UTC](https://discuss.elastic.co/t/logstash-performance-issue/95948 "2017-08-04T19:42:54Z")
Hello All, Here are configuration details, Windows box Logstash 5.3.0 CSV will uploaded every 30 mins into S3 bucket, I am currently running logstash from command prompt and keep command prompt up and running so if … --- ## [Logstash + 31mA plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/logstash-31ma-plugin-had-an-unrecoverable-error-will-restart-this-plugin/95947)
**Author:** [@sarthak2016](https://discuss.elastic.co/u/sarthak2016)\ **Replies:** 0\ **Last updated:** [August 4, 2017, 7:22pm UTC](https://discuss.elastic.co/t/logstash-31ma-plugin-had-an-unrecoverable-error-will-restart-this-plugin/95947 "2017-08-04T19:22:59Z")
I am running logstash from command prompt, Logstash reads CSV files from S3 bucket, I keep up and running command prompt for logstash because we are uploading CSV files frequently in a day, I am getting below error after… --- ## [Logstash Debugging](https://discuss.elastic.co/t/logstash-debugging/95400)
**Author:** [@jfarrin1](https://discuss.elastic.co/u/jfarrin1)\ **Replies:** 6\ **Last updated:** [August 4, 2017, 6:01pm UTC](https://discuss.elastic.co/t/logstash-debugging/95400 "2017-08-04T18:01:03Z")
Hello, I was wondering if anyone had a good way to debug with logstash. Currently, I am using the stdout with rubydebug. In my error file it shows an error name but won’t give me any specific information, which makes th… --- ## [@metadata and regexp named groups](https://discuss.elastic.co/t/metadata-and-regexp-named-groups/93237)
**Author:** [@danielmotaleite](https://discuss.elastic.co/u/danielmotaleite)\ **Replies:** 2\ **Last updated:** [August 4, 2017, 5:35pm UTC](https://discuss.elastic.co/t/metadata-and-regexp-named-groups/93237 "2017-08-04T17:35:18Z")
in the url https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html we can see that we can use \[@metadata\]\[timestamp\] in a grok: filter { grok { match =\> \[ "message", "%{HTTPDATE:\[@metadat… --- ## [Don't understand why my query doens't return all value](https://discuss.elastic.co/t/dont-understand-why-my-query-doenst-return-all-value/95929)
**Author:** [@ntomioags](https://discuss.elastic.co/u/ntomioags)\ **Replies:** 0\ **Last updated:** [August 4, 2017, 4:53pm UTC](https://discuss.elastic.co/t/dont-understand-why-my-query-doenst-return-all-value/95929 "2017-08-04T16:53:24Z")
Hi All, I am new on this forum, and a new user of ELK. I will try to describe my problem. I’m trying to parse Zimbra log file and I begin with these two lines : Aug 4 06:44:30 ags-zimbraprox postfix/smtpd\[7440\]: D3F8… --- ## [Logstash not parsing multiple logs for different type](https://discuss.elastic.co/t/logstash-not-parsing-multiple-logs-for-different-type/93606)
**Author:** [@Ahmed.O](https://discuss.elastic.co/u/Ahmed.O)\ **Replies:** 2\ **Last updated:** [August 4, 2017, 4:38pm UTC](https://discuss.elastic.co/t/logstash-not-parsing-multiple-logs-for-different-type/93606 "2017-08-04T16:38:38Z")
Hi Team, I have been looking for logstash configuration that should parse two different log types and to create different indexes for types. in my case i have .log and .CSV file types, That i'm capturing through filebea… --- ## [How do I process a subset of files from an s3 path?](https://discuss.elastic.co/t/how-do-i-process-a-subset-of-files-from-an-s3-path/94537)
**Author:** [@nfplatzke](https://discuss.elastic.co/u/nfplatzke)\ **Replies:** 0\ **Last updated:** [July 25, 2017, 8:18pm UTC](https://discuss.elastic.co/t/how-do-i-process-a-subset-of-files-from-an-s3-path/94537 "2017-07-25T20:18:00Z")
I am trying to process a subset of elastic beanstalk logs with s3. The issue is that the path elastic beanstalk publishes to looks like this: resources/environments/logs/publish/e-environment/i-instance/log.files I h… --- ## [Tracking column failed](https://discuss.elastic.co/t/tracking-column-failed/95905)
**Author:** [@FR-TheS](https://discuss.elastic.co/u/FR-TheS)\ **Replies:** 1\ **Last updated:** [August 4, 2017, 3:25pm UTC](https://discuss.elastic.co/t/tracking-column-failed/95905 "2017-08-04T15:25:52Z")
Hello, I got an issue on the tracking column with sql server request. The tracking column is not increment, so i’m reloading the same data always. statement =\> "SELECT TOP 10000 \* FROM PWODecisionHistory WHER… --- ## [Field value filter](https://discuss.elastic.co/t/field-value-filter/95701)
**Author:** [@shaharz](https://discuss.elastic.co/u/shaharz)\ **Replies:** 6\ **Last updated:** [August 4, 2017, 2:20pm UTC](https://discuss.elastic.co/t/field-value-filter/95701 "2017-08-04T14:20:59Z")
Hello, i would like to know how can i do a field value filter? for example to do drop event by field value or length of a value, and how can i do a where in filter, for example. “field value” not in \[1,2,3\] etc. thanks… --- ## [KV filter is not working with FILE input](https://discuss.elastic.co/t/kv-filter-is-not-working-with-file-input/95767)
**Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 3\ **Last updated:** [August 4, 2017, 1:55pm UTC](https://discuss.elastic.co/t/kv-filter-is-not-working-with-file-input/95767 "2017-08-04T13:55:42Z")
input { file { path =\> "/prod/onic\_app\*.log" start\_position =\> beginning source =\> onic\_tomcat\_app } file { path =\> "/prod/onic\_\_perf\*.log" start\_position =\> beginning source =\> onic\_tomcat\_perf } file { path… --- ## [Logstash can't send logs to ES](https://discuss.elastic.co/t/logstash-cant-send-logs-to-es/95875)
**Author:** [@snah95](https://discuss.elastic.co/u/snah95)\ **Replies:** 4\ **Last updated:** [August 4, 2017, 1:32pm UTC](https://discuss.elastic.co/t/logstash-cant-send-logs-to-es/95875 "2017-08-04T13:32:34Z")
Hello everyone! I have a little problem with my ELK Stack. My Logstash container gathers logs from Filebeat, but it can't pass them to Elasticsearch. I tried many methods described in another topics, but with no success… --- ## [Logstash 5.5 input jdbc schedule problem](https://discuss.elastic.co/t/logstash-5-5-input-jdbc-schedule-problem/95202)
**Author:** [@eldiablo62](https://discuss.elastic.co/u/eldiablo62)\ **Replies:** 9\ **Last updated:** [August 4, 2017, 1:23pm UTC](https://discuss.elastic.co/t/logstash-5-5-input-jdbc-schedule-problem/95202 "2017-08-04T13:23:33Z")
Hi, I have configured logstash to get data from mysql server and i have scheduled to get data every minute but it’s not working. I think i have made a mistake on my configuration. If somebody can help me :slight\_smile: … --- ## [Logstash user in Xpack](https://discuss.elastic.co/t/logstash-user-in-xpack/95727)
**Author:** [@dencowboy](https://discuss.elastic.co/u/dencowboy)\ **Replies:** 3\ **Last updated:** [August 4, 2017, 1:20pm UTC](https://discuss.elastic.co/t/logstash-user-in-xpack/95727 "2017-08-04T13:20:31Z")
there are 2 configuration files for our logstash: config/logstash.yml pipeline/logstash.conf Now I want to know which roles and which user(s) we minimal need to define to get data to elasticsearch and to see it in kib… --- ## [One centos start multi logstash agents does not work](https://discuss.elastic.co/t/one-centos-start-multi-logstash-agents-does-not-work/95824)
**Author:** [@zhaochl](https://discuss.elastic.co/u/zhaochl)\ **Replies:** 3\ **Last updated:** [August 4, 2017, 12:45pm UTC](https://discuss.elastic.co/t/one-centos-start-multi-logstash-agents-does-not-work/95824 "2017-08-04T12:45:55Z")
one ceontos start multi logstash agents to consumer kafka but none works logstash version 2.4 --- ## [How to deal with optionals fields in Logstash/ElasticSearch](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965)
**Author:** [@NicolasHennion](https://discuss.elastic.co/u/NicolasHennion)\ **Replies:** 4\ **Last updated:** [August 4, 2017, 12:08pm UTC](https://discuss.elastic.co/t/how-to-deal-with-optionals-fields-in-logstash-elasticsearch/94965 "2017-08-04T12:08:33Z")
have a problem when i try to import the following XML file in ElasticSearch with Logstash: \ \ \ \ \… --- ## [Logstash - set @timestamp from sql date column with JDBC input](https://discuss.elastic.co/t/logstash-set-timestamp-from-sql-date-column-with-jdbc-input/95873)
**Author:** [@dlalex83](https://discuss.elastic.co/u/dlalex83)\ **Replies:** 3\ **Last updated:** [August 4, 2017, 12:00pm UTC](https://discuss.elastic.co/t/logstash-set-timestamp-from-sql-date-column-with-jdbc-input/95873 "2017-08-04T12:00:42Z")
Hi there, I've setup a JDBC input plugin to read off my SQL Server ErrorLog table with the output being an ES index. No particular mapping needed but the only requirement is to use the LogDate column as the main @times… --- ## [My ZXTM logs not shipping to Elastic Search From Logstash](https://discuss.elastic.co/t/my-zxtm-logs-not-shipping-to-elastic-search-from-logstash/94945)
**Author:** [@kolli\_dilip](https://discuss.elastic.co/u/kolli_dilip)\ **Replies:** 5\ **Last updated:** [August 4, 2017, 11:58am UTC](https://discuss.elastic.co/t/my-zxtm-logs-not-shipping-to-elastic-search-from-logstash/94945 "2017-08-04T11:58:12Z")
Dear All, Can some one help me to resolve the issue, where the ZXTM logs are not being shipped to ES. The logstash config is as below input { file { type =\> “ZXTMLogs” path =\>\[ “F:/UKBlogsstachconfiguration/Log… --- ## [Only String and Array types are splittable](https://discuss.elastic.co/t/only-string-and-array-types-are-splittable/95246)
**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\ **Replies:** 3\ **Last updated:** [August 4, 2017, 11:36am UTC](https://discuss.elastic.co/t/only-string-and-array-types-are-splittable/95246 "2017-08-04T11:36:06Z")
Or more specifically: logstash.filters.split \] Only String and Array types are splittable. field:event\_data.MemberName is of type = NilClass Hi, So, I’d like to extract the contents of the string field called: field:e… --- ## [JDBC SQL Server connection issue](https://discuss.elastic.co/t/jdbc-sql-server-connection-issue/95844)
**Author:** [@FR-TheS](https://discuss.elastic.co/u/FR-TheS)\ **Replies:** 2\ **Last updated:** [August 4, 2017, 11:07am UTC](https://discuss.elastic.co/t/jdbc-sql-server-connection-issue/95844 "2017-08-04T11:07:55Z")
Hello everyone, Since yesterday, I can't anymore connect to my sql server. I got a problem on method close connection. Logstash 5.4.3 SQL Server JDBC Driver Conf input { jdbc { jdbc\_driver\_library =\> "C:\\sof… --- ## [Best practice handling asynchron events](https://discuss.elastic.co/t/best-practice-handling-asynchron-events/95172)
**Author:** [@Sinmson](https://discuss.elastic.co/u/Sinmson)\ **Replies:** 2\ **Last updated:** [August 4, 2017, 10:18am UTC](https://discuss.elastic.co/t/best-practice-handling-asynchron-events/95172 "2017-08-04T10:18:29Z")
Hey guys, I am trying to index events that occur at different times in different invervalls. My situation There are 2 servers and between them is a middleware. Server A sends a request to do a certain job. Server B is… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=931) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=933)