# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=995 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 996 --- ## [Logstash seems successful. but nothing found on elasticsearch](https://discuss.elastic.co/t/logstash-seems-successful-but-nothing-found-on-elasticsearch/83484)
**Author:** [@sraghu1980](https://discuss.elastic.co/u/sraghu1980)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 6:19am UTC](https://discuss.elastic.co/t/logstash-seems-successful-but-nothing-found-on-elasticsearch/83484 "2017-04-25T06:19:26Z")
Hi, I have /etc/logstash/conf.d/logstash.conf: input { file { path =\> "/var/lib/collectd/csv/Gateway/cpu-0/cpu-idle-2017-04-24" start\_position =\> beginning sincedb\_path =\> "/dev/null" ignore\_older =\> 0 } } … --- ## [GROK filter for different events sourced fro the same input](https://discuss.elastic.co/t/grok-filter-for-different-events-sourced-fro-the-same-input/83485)
**Author:** [@evlonden](https://discuss.elastic.co/u/evlonden)\ **Replies:** 1\ **Last updated:** [April 25, 2017, 6:10am UTC](https://discuss.elastic.co/t/grok-filter-for-different-events-sourced-fro-the-same-input/83485 "2017-04-25T06:10:41Z")
Hi All, I'm puzzling a bit with some filter settings whereby events being sourced from a UDP input in (not really) syslog format have somewhat different characteristics. Based upon the event class generated by the Broc… --- ## [Global field gsub filter?](https://discuss.elastic.co/t/global-field-gsub-filter/83243)
**Author:** [@\_Ian](https://discuss.elastic.co/u/_Ian)\ **Replies:** 6\ **Last updated:** [April 24, 2017, 6:52pm UTC](https://discuss.elastic.co/t/global-field-gsub-filter/83243 "2017-04-24T18:52:58Z")
Is there a way to apply a gsub action to every field during filtering? I'm currently using dissect to split apart a message and want to do a more global replace of a null character to an actual null ES is happy with. C… --- ## [Kafka input plugin creates anywhere from 2-5x more documents than in topic](https://discuss.elastic.co/t/kafka-input-plugin-creates-anywhere-from-2-5x-more-documents-than-in-topic/83449)
**Author:** [@xrl](https://discuss.elastic.co/u/xrl)\ **Replies:** 0\ **Last updated:** [April 24, 2017, 6:18pm UTC](https://discuss.elastic.co/t/kafka-input-plugin-creates-anywhere-from-2-5x-more-documents-than-in-topic/83449 "2017-04-24T18:18:39Z")
I am trying to run a small pool of logstash with at-most-once delivery semantics on a kafka 0.10.2.1 cluster. The kafka input client is running with the new style of kafka consumer group, managed as metadata in the broke… --- ## [Feeding Logstash](https://discuss.elastic.co/t/feeding-logstash/28977)
**Author:** [@StivOstenberg](https://discuss.elastic.co/u/StivOstenberg)\ **Replies:** 2\ **Last updated:** [April 24, 2017, 5:55pm UTC](https://discuss.elastic.co/t/feeding-logstash/28977 "2017-04-24T17:55:01Z")
I am part of a team that manages a large number of AWS Accounts. I wrote a program that can read the credentials for all the accounts from the Windows Credential Store. It then uses those credentials to pull data from… --- ## [JDBC integrated security](https://discuss.elastic.co/t/jdbc-integrated-security/54157)
**Author:** [@tollef](https://discuss.elastic.co/u/tollef)\ **Replies:** 3\ **Last updated:** [April 24, 2017, 5:26pm UTC](https://discuss.elastic.co/t/jdbc-integrated-security/54157 "2017-04-24T17:26:57Z")
Hello. I'm currently in a POC stage in a project and have to collect data from a MS SQL Server. I am using jdbc-input-plugin to connect to the database. dbtest.conf \`input { jdbc { jdbc\_driver\_library =\> "C:\\… --- ## [Issues in installing gem file in logstash 5.2.1](https://discuss.elastic.co/t/issues-in-installing-gem-file-in-logstash-5-2-1/83256)
**Author:** [@rajkamalkool6](https://discuss.elastic.co/u/rajkamalkool6)\ **Replies:** 4\ **Last updated:** [April 24, 2017, 5:19pm UTC](https://discuss.elastic.co/t/issues-in-installing-gem-file-in-logstash-5-2-1/83256 "2017-04-24T17:19:56Z")
I am using logstash 5.2.1 and I am trying to install some ruby gems into my logstash. I have the gems already downloaded and its in my local directory. When I try to install it I am getting some connection refused error. … --- ## [Best Practices - Confused](https://discuss.elastic.co/t/best-practices-confused/83293)
**Author:** [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Replies:** 3\ **Last updated:** [April 24, 2017, 4:58pm UTC](https://discuss.elastic.co/t/best-practices-confused/83293 "2017-04-24T16:58:05Z")
Dear Logstash Team and Community, I'm a self trained enthusiast of logstash and we've been working to move over applications into pushing to ES. Some of our legacy applications need time to push directly to ES and th… --- ## [How to update GeoLiteCity.dat periodically](https://discuss.elastic.co/t/how-to-update-geolitecity-dat-periodically/83438)
**Author:** [@saad](https://discuss.elastic.co/u/saad)\ **Replies:** 0\ **Last updated:** [April 24, 2017, 4:32pm UTC](https://discuss.elastic.co/t/how-to-update-geolitecity-dat-periodically/83438 "2017-04-24T16:32:00Z")
Hi, Would you please send me bash code to update GeoLiteCity.dat periodically in logstash. --- ## [Logstash OOM](https://discuss.elastic.co/t/logstash-oom/83435)
**Author:** [@StoRRica](https://discuss.elastic.co/u/StoRRica)\ **Replies:** 0\ **Last updated:** [April 24, 2017, 4:08pm UTC](https://discuss.elastic.co/t/logstash-oom/83435 "2017-04-24T16:08:58Z")
Hello, i am quite new to logstash but with this config: https://pastebin.com/zg66Pvgr after like 5-10 minutes of running logstash and get no output to console. I already played with such a things as ignore\_older, max\_fi… --- ## [Logstash: unrecognized service](https://discuss.elastic.co/t/logstash-unrecognized-service/83144)
**Author:** [@shaonbean](https://discuss.elastic.co/u/shaonbean)\ **Replies:** 4\ **Last updated:** [April 24, 2017, 3:45pm UTC](https://discuss.elastic.co/t/logstash-unrecognized-service/83144 "2017-04-24T15:45:54Z")
CentOS 6.9 use elastic repo install logstash,but logstash daemon service can't not found .when execte sudo service logstash start ,error logstash: unrecognized service --- ## [Logstash within docker container trying to connect to elasticsearch](https://discuss.elastic.co/t/logstash-within-docker-container-trying-to-connect-to-elasticsearch/83335)
**Author:** [@holt](https://discuss.elastic.co/u/holt)\ **Replies:** 3\ **Last updated:** [April 24, 2017, 3:01pm UTC](https://discuss.elastic.co/t/logstash-within-docker-container-trying-to-connect-to-elasticsearch/83335 "2017-04-24T15:01:37Z")
There was a similar post on this, but the replies have been deleted: When I run a elasticsearch 5.3.1 from a docker container it tries to connect to elasticsearch even though my pipeline only has a udp port as the outp… --- ## [Logstash listens on IPV6 only](https://discuss.elastic.co/t/logstash-listens-on-ipv6-only/83410)
**Author:** [@X\_Calibur](https://discuss.elastic.co/u/X_Calibur)\ **Replies:** 1\ **Last updated:** [April 24, 2017, 12:45pm UTC](https://discuss.elastic.co/t/logstash-listens-on-ipv6-only/83410 "2017-04-24T12:45:18Z")
I have the following config file: input{ beats{ port =\> "5043" } } output{ elasticsearch { hosts =\> \["localhost:9200"\] index =\> "winlogbeat" … --- ## ["service logstash start" does not work](https://discuss.elastic.co/t/service-logstash-start-does-not-work/83002)
**Author:** [@X\_Calibur](https://discuss.elastic.co/u/X_Calibur)\ **Replies:** 14\ **Last updated:** [April 24, 2017, 12:12pm UTC](https://discuss.elastic.co/t/service-logstash-start-does-not-work/83002 "2017-04-24T12:12:38Z")
Hi, I have Ubuntu 16.10 and the latest Logstash installed. Running Logstash manually with /usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash" works just fine but I'm unable to run it as a service. Any h… --- ## [ES-to-ES reindexing: why documents are deleted in destination index?](https://discuss.elastic.co/t/es-to-es-reindexing-why-documents-are-deleted-in-destination-index/83389)
**Author:** [@KIVagant](https://discuss.elastic.co/u/KIVagant)\ **Replies:** 2\ **Last updated:** [April 24, 2017, 11:22am UTC](https://discuss.elastic.co/t/es-to-es-reindexing-why-documents-are-deleted-in-destination-index/83389 "2017-04-24T11:22:42Z")
Hello. I have the next configuration file for the AWS ES reindexing: input { elasticsearch { hosts =\> "search-.....es.amazonaws.com:443" index =\> "logstash-source-index\*" #query =\> "\*" size =\> 500 … --- ## [Logstash Sizing](https://discuss.elastic.co/t/logstash-sizing/83065)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 6\ **Last updated:** [April 24, 2017, 11:19am UTC](https://discuss.elastic.co/t/logstash-sizing/83065 "2017-04-24T11:19:27Z")
What is the recommended typical sizing configuration with respect to memory, disk and cpu for logstash. --- ## [Not able to send multiple date fields in logstash](https://discuss.elastic.co/t/not-able-to-send-multiple-date-fields-in-logstash/83359)
**Author:** [@SanC](https://discuss.elastic.co/u/SanC)\ **Replies:** 4\ **Last updated:** [April 24, 2017, 11:11am UTC](https://discuss.elastic.co/t/not-able-to-send-multiple-date-fields-in-logstash/83359 "2017-04-24T11:11:45Z")
My Log file- MachineName|FeedName|Version|InputFileName|LoadType|StartTime|NumOfRecordsinCEMP|NumOfRecordsinStage|Status|Message|EndTime|EventSeverity|Action|time\_taken\_ms EDTD-LYNXSQL01|Falcon|2017-04-13\_180858\_7480000|… --- ## [What is the grok pattern for this?](https://discuss.elastic.co/t/what-is-the-grok-pattern-for-this/83393)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 4\ **Last updated:** [April 24, 2017, 10:59am UTC](https://discuss.elastic.co/t/what-is-the-grok-pattern-for-this/83393 "2017-04-24T10:59:31Z")
Hi, Seems to be a easy question but i am quite new to this grok pattern.I have logs like this {"@timestamp": "2017-04-24T07:03:07.985Z", "function": "cli", "linenum": 204, "loglevel": "INFO", "message": "Job complete… --- ## [How to tweak logstash -\> elasticsearch indexing process](https://discuss.elastic.co/t/how-to-tweak-logstash-elasticsearch-indexing-process/83401)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 3\ **Last updated:** [April 24, 2017, 10:52am UTC](https://discuss.elastic.co/t/how-to-tweak-logstash-elasticsearch-indexing-process/83401 "2017-04-24T10:52:24Z")
Hello, my logstash configurations create a separate index for each log file, I want to add them all together in a single elasticsearch index, how can I do this? Here's how my logstash.conf looks like: input { tcp { … --- ## [Logstash not working, but says configuration ok](https://discuss.elastic.co/t/logstash-not-working-but-says-configuration-ok/83367)
**Author:** [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\ **Replies:** 6\ **Last updated:** [April 24, 2017, 10:17am UTC](https://discuss.elastic.co/t/logstash-not-working-but-says-configuration-ok/83367 "2017-04-24T10:17:56Z")
Logstash is creating problems, I am using it with filebeat to send logs to elastic. When I start it, it says every thing is ok, luvpreet@DHARI-Inspiron-3542:/usr/share/logstash$ sudo bin/logstash --path.settings /etc/l… --- ## [Issue Running ELK stack on Docker + Aws ECS](https://discuss.elastic.co/t/issue-running-elk-stack-on-docker-aws-ecs/83201)
**Author:** [@hardik.aws](https://discuss.elastic.co/u/hardik.aws)\ **Replies:** 1\ **Last updated:** [April 24, 2017, 8:27am UTC](https://discuss.elastic.co/t/issue-running-elk-stack-on-docker-aws-ecs/83201 "2017-04-24T08:27:36Z")
Hello I am having issue running docker on Amazon ECS below is the main config file for logstash input { http { host =\> "0.0.0.0" port =\> 5044 } } output { elasticsearch { … --- ## [Import one-to-many relation from SQL](https://discuss.elastic.co/t/import-one-to-many-relation-from-sql/83375)
**Author:** [@DOB](https://discuss.elastic.co/u/DOB)\ **Replies:** 0\ **Last updated:** [April 24, 2017, 8:14am UTC](https://discuss.elastic.co/t/import-one-to-many-relation-from-sql/83375 "2017-04-24T08:14:49Z")
I try to import job ads form two MySQL tables (job data and locations) but I'm facing a problem when a job ad has multiple locations. I'm using this MySQL query: SELECT id, company, jobtitle, description, priority, DAT… --- ## [Parsing with Grok filter](https://discuss.elastic.co/t/parsing-with-grok-filter/83224)
**Author:** [@Sam67000](https://discuss.elastic.co/u/Sam67000)\ **Replies:** 1\ **Last updated:** [April 24, 2017, 7:50am UTC](https://discuss.elastic.co/t/parsing-with-grok-filter/83224 "2017-04-24T07:50:51Z")
Hello everyone. I need your help to extract informations from a specific field. So I have a log with a lot of information and with a grok filter I only store log lines that interest me in a variable named "DAS". … --- ## [\[ELK\]logstash default timezone cause index splitting problem in different timezones](https://discuss.elastic.co/t/elk-logstash-default-timezone-cause-index-splitting-problem-in-different-timezones/26615)
**Author:** [@Weiwei\_Wang](https://discuss.elastic.co/u/Weiwei_Wang)\ **Replies:** 14\ **Last updated:** [April 24, 2017, 7:49am UTC](https://discuss.elastic.co/t/elk-logstash-default-timezone-cause-index-splitting-problem-in-different-timezones/26615 "2017-04-24T07:49:18Z")
I'm a Chinese developer, our timezone is +08:00，the problem using logstash is that @timestamp is always formatted as @timestamp" =\> "2015-07-25T16:00:30.000Z, the input time is 2015-07-26 00:00:30. This problem will caus… --- ## [Custom GROK Patterns not work](https://discuss.elastic.co/t/custom-grok-patterns-not-work/82727)
**Author:** [@davidS](https://discuss.elastic.co/u/davidS)\ **Replies:** 4\ **Last updated:** [April 24, 2017, 6:45am UTC](https://discuss.elastic.co/t/custom-grok-patterns-not-work/82727 "2017-04-24T06:45:28Z")
Hi all, we tried to implement some custom grok patterns into a seperate file and load it into logstash conf by using patterns\_dir. Each time running logstash with the following conf output a pattern not defined error m… --- ## [Unable to fetch mappings - BRO IDS + ELK Stack](https://discuss.elastic.co/t/unable-to-fetch-mappings-bro-ids-elk-stack/83128)
**Author:** [@adam2](https://discuss.elastic.co/u/adam2)\ **Replies:** 2\ **Last updated:** [April 24, 2017, 5:22am UTC](https://discuss.elastic.co/t/unable-to-fetch-mappings-bro-ids-elk-stack/83128 "2017-04-24T05:22:43Z")
Hello Everyone, i'll start by saying...newbe here and i'm lost. I'm trying to setup BRO and ELK stack. I'm following https://github.com/BrashEndeavours/bro-elk-IDS. I have my BRO ids and ELK stack on the same server … --- ## [Minimize logstash resources](https://discuss.elastic.co/t/minimize-logstash-resources/83336)
**Author:** [@holt](https://discuss.elastic.co/u/holt)\ **Replies:** 0\ **Last updated:** [April 23, 2017, 8:32pm UTC](https://discuss.elastic.co/t/minimize-logstash-resources/83336 "2017-04-23T20:32:53Z")
I'm using logstash in a somewhat unconventional way & I would like to minimize the resources it consumes. I will only have input from a single filebeat source & a single output. In particular, I'd like to reduce the amou… --- ## [Logs are sending to Elasticsearch without 'output' in 'logstash.conf'](https://discuss.elastic.co/t/logs-are-sending-to-elasticsearch-without-output-in-logstash-conf/83262)
**Author:** [@Steve\_1](https://discuss.elastic.co/u/Steve_1)\ **Replies:** 7\ **Last updated:** [April 23, 2017, 8:22pm UTC](https://discuss.elastic.co/t/logs-are-sending-to-elasticsearch-without-output-in-logstash-conf/83262 "2017-04-23T20:22:03Z")
I use Filebeat to forward logs from another server to Logstash and then Logstash sends them to Elasticsearch. In filebeat.yml I specify output as Logstash but in logstash.conf I don't specify output at all. But in Kibana… --- ## [Solved - Ruby exceptions - updating filter to new api](https://discuss.elastic.co/t/solved-ruby-exceptions-updating-filter-to-new-api/83320)
**Author:** [@AtomicWerks](https://discuss.elastic.co/u/AtomicWerks)\ **Replies:** 1\ **Last updated:** [April 23, 2017, 4:56pm UTC](https://discuss.elastic.co/t/solved-ruby-exceptions-updating-filter-to-new-api/83320 "2017-04-23T16:56:27Z")
Hi. I'm very new to the ELK stack and trying to get my logstash config updated from 4.x to 5.x and I'm having a problem with my ruby filter. I have already in the config: \>ruby { code =\> "if event\['event\_type'\] == 'f… --- ## [How to "tag" from which location data is coming from?](https://discuss.elastic.co/t/how-to-tag-from-which-location-data-is-coming-from/82795)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 17\ **Last updated:** [April 23, 2017, 8:50am UTC](https://discuss.elastic.co/t/how-to-tag-from-which-location-data-is-coming-from/82795 "2017-04-23T08:50:50Z")
Hello, Not sure if this should go here or in the elastic discussion, apologies. Lets say I got remote location A, B and C and each of them are sending their netflow, snmp and syslogs to a central ELK stack. The re… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=994) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=996)