# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=853 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 854 --- ## [\_geoip\_lookup\_failure but not specified](https://discuss.elastic.co/t/geoip-lookup-failure-but-not-specified/111348)
**Author:** [@teejayuu](https://discuss.elastic.co/u/teejayuu)\ **Replies:** 3\ **Last updated:** [December 20, 2017, 12:48am UTC](https://discuss.elastic.co/t/geoip-lookup-failure-but-not-specified/111348 "2017-12-20T00:48:59Z")
Using filebeat to send IIS logs from Windows System and to my logstash server and then to elastic search and Kibana. Elastic search and Kibana - 5.3.2 Logstash and filebeat - 5.3.2 filebeat.yml : #====================… --- ## [Reindex Elasticsearch indexes with Logstash from a continuous feed](https://discuss.elastic.co/t/reindex-elasticsearch-indexes-with-logstash-from-a-continuous-feed/112507)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 3\ **Last updated:** [December 19, 2017, 9:40pm UTC](https://discuss.elastic.co/t/reindex-elasticsearch-indexes-with-logstash-from-a-continuous-feed/112507 "2017-12-19T21:40:55Z")
Is there a way to pull an index from Elasticsearch with Logstash and reindex it with a continuous feed or a scheduled query as with the jdbc pulgin? --- ## [How to load nested documents in Elasticsearh using logstash](https://discuss.elastic.co/t/how-to-load-nested-documents-in-elasticsearh-using-logstash/112501)
**Author:** [@Geert](https://discuss.elastic.co/u/Geert)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 8:24pm UTC](https://discuss.elastic.co/t/how-to-load-nested-documents-in-elasticsearh-using-logstash/112501 "2017-12-19T20:24:12Z")
I have a product database with 100.000 products. Each product has its own specs. The number and type of specs differ for each product. So my database has 2 tables with a 1:N relation: a product table (about 100.000 rec… --- ## [Logstash - Extracting substring from CSV column](https://discuss.elastic.co/t/logstash-extracting-substring-from-csv-column/112472)
**Author:** [@AshishC](https://discuss.elastic.co/u/AshishC)\ **Replies:** 7\ **Last updated:** [December 19, 2017, 7:51pm UTC](https://discuss.elastic.co/t/logstash-extracting-substring-from-csv-column/112472 "2017-12-19T19:51:56Z")
Hi, I have a csv file with a column called "threadName", value of which varies with each record in csv. example CME\_MOC 15-1 CME\_MOC 15-2 CME\_MOC 15-3 PME\_MOC 15-1 KME\_MOC 15-2 I am sending csv records to elastics… --- ## ["Unable to fetch mapping. Do you have indices matching the pattern?"](https://discuss.elastic.co/t/unable-to-fetch-mapping-do-you-have-indices-matching-the-pattern/112393)
**Author:** [@Kristoffer\_Fagerlund](https://discuss.elastic.co/u/Kristoffer_Fagerlund)\ **Replies:** 8\ **Last updated:** [December 19, 2017, 6:29pm UTC](https://discuss.elastic.co/t/unable-to-fetch-mapping-do-you-have-indices-matching-the-pattern/112393 "2017-12-19T18:29:23Z")
logstash doesnt seem to forward logs to elastic correctly slices: yellow open .kibana RixJKa2bQ8iRBwDmHd18Yw 1 1 1 0 3.2kb 3.2kb /etc/logstash/logstash.yml: path.config: /etc/logstash/conf.d ... path.logs: /var/log… --- ## [Logstash elasticsearch to kafka stops after being started](https://discuss.elastic.co/t/logstash-elasticsearch-to-kafka-stops-after-being-started/112481)
**Author:** [@anon24174512](https://discuss.elastic.co/u/anon24174512)\ **Replies:** 2\ **Last updated:** [December 19, 2017, 5:32pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-to-kafka-stops-after-being-started/112481 "2017-12-19T17:32:15Z")
Dear all, I'am trying to export documents from an Elasticsearch index to a Kafka topic using Logstash. However, when I start Logstash with only the elk-to-kafka configuration file it starts and quickly stops right after… --- ## [Logstash 5.6.x to 6.1.0 and ES template update](https://discuss.elastic.co/t/logstash-5-6-x-to-6-1-0-and-es-template-update/112482)
**Author:** [@phr0gz](https://discuss.elastic.co/u/phr0gz)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 5:20pm UTC](https://discuss.elastic.co/t/logstash-5-6-x-to-6-1-0-and-es-template-update/112482 "2017-12-19T17:20:43Z")
Hello, Here is the list of changes/recommendations (I hope I don't miss anything): Logstash: ES output plugin will change the document\_type from logs to doc ElasticSearch: Indices created in Elasticsearch 6.0.0 … --- ## [Logstash pipelines isolation](https://discuss.elastic.co/t/logstash-pipelines-isolation/112443)
**Author:** [@Martin\_Duris](https://discuss.elastic.co/u/Martin_Duris)\ **Replies:** 1\ **Last updated:** [December 19, 2017, 3:32pm UTC](https://discuss.elastic.co/t/logstash-pipelines-isolation/112443 "2017-12-19T15:32:44Z")
Hi guys, i have logstash (v6) where i am using 2 pipelines, defined like this: - pipeline.id: pipe1 path.config: "/etc/logstash/conf.d/pipelines/pipe1.conf" pipeline.workers: 1 - pipeline.id: pipe2 path.config: "… --- ## [Logstash don't parse Date field properly](https://discuss.elastic.co/t/logstash-dont-parse-date-field-properly/112460)
**Author:** [@Wills1](https://discuss.elastic.co/u/Wills1)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 3:15pm UTC](https://discuss.elastic.co/t/logstash-dont-parse-date-field-properly/112460 "2017-12-19T15:15:17Z")
Hi ! I have a date field in my csv file that I try to parse. Following ist my date filter: date{ match =\> \[ "Date", "ISO8601" \] timezone =\> "Europe/Berlin" target =\> "Date" } my current configured time zone in my … --- ## [Conditional Problem during log aggregation](https://discuss.elastic.co/t/conditional-problem-during-log-aggregation/112445)
**Author:** [@Simon\_Devineau](https://discuss.elastic.co/u/Simon_Devineau)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 2:08pm UTC](https://discuss.elastic.co/t/conditional-problem-during-log-aggregation/112445 "2017-12-19T14:08:15Z")
Hello everyone, I am writing about a conditional problem I have. I am trying to aggregate Jenkins logs line using https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggrega… --- ## [How to give \\n as delimiter for dissect filter](https://discuss.elastic.co/t/how-to-give-n-as-delimiter-for-dissect-filter/112439)
**Author:** [@George\_Cherian](https://discuss.elastic.co/u/George_Cherian)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 1:15pm UTC](https://discuss.elastic.co/t/how-to-give-n-as-delimiter-for-dissect-filter/112439 "2017-12-19T13:15:44Z")
Hi Logstash Team, Need to filter out a field till the end of the line in dissect filter. Is it possible to do it using dissect. Came to know that from logstash 6.0, if the config in logstash.yml config.support\_escapes:… --- ## [Very high CPU load (how to interpret output of "top"?)](https://discuss.elastic.co/t/very-high-cpu-load-how-to-interpret-output-of-top/112269)
**Author:** [@Precision](https://discuss.elastic.co/u/Precision)\ **Replies:** 1\ **Last updated:** [December 19, 2017, 12:30pm UTC](https://discuss.elastic.co/t/very-high-cpu-load-how-to-interpret-output-of-top/112269 "2017-12-19T12:30:23Z")
I am using Logstash to send logfiles via the Gelf plugin to Graylog. My config is fairly complex, with lots of grok filters. When my system is under load the messages queue up for DAYS. I have attached the output of "… --- ## [\[SOLVED\] \[logstash 2.3\] ruby error using event.get](https://discuss.elastic.co/t/solved-logstash-2-3-ruby-error-using-event-get/112020)
**Author:** [@mobidyc](https://discuss.elastic.co/u/mobidyc)\ **Replies:** 2\ **Last updated:** [December 19, 2017, 11:35am UTC](https://discuss.elastic.co/t/solved-logstash-2-3-ruby-error-using-event-get/112020 "2017-12-19T11:35:54Z")
Hello, I try to do a substraction on a float number and i'm facing this issue: # echo '{"ts":"123124.094", "rq":"0.020"}' | \\ /opt/logstash/bin/logstash -e \\ "input { stdin { codec =\> json } } outp… --- ## [Prune + Fingerprint?](https://discuss.elastic.co/t/prune-fingerprint/112418)
**Author:** [@Tonni\_Hult](https://discuss.elastic.co/u/Tonni_Hult)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 11:26am UTC](https://discuss.elastic.co/t/prune-fingerprint/112418 "2017-12-19T11:26:47Z")
Hi Is anyone aware of an existing filter that combines parts of prune and fingerprint? My usecase is that I want to make a list of fields either by using a whitelist or a blacklist (which prune has support for) and the… --- ## [Logstash randomly down after upgrade](https://discuss.elastic.co/t/logstash-randomly-down-after-upgrade/112394)
**Author:** [@cwt](https://discuss.elastic.co/u/cwt)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 9:46am UTC](https://discuss.elastic.co/t/logstash-randomly-down-after-upgrade/112394 "2017-12-19T09:46:35Z")
Hi, I have issue after logstash upgrade to 6.1.0. Logstash stops accepting logs. Logstash logs: https://pastebin.com/5M2XnV8S Syslog: https://pastebin.com/Uw356wL6 --- ## [Several log formatting with ansible loop](https://discuss.elastic.co/t/several-log-formatting-with-ansible-loop/112387)
**Author:** [@Juliatzin\_Ixtekakati](https://discuss.elastic.co/u/Juliatzin_Ixtekakati)\ **Replies:** 0\ **Last updated:** [December 19, 2017, 9:17am UTC](https://discuss.elastic.co/t/several-log-formatting-with-ansible-loop/112387 "2017-12-19T09:17:18Z")
Hi, I have an ansible dictionary where I define all my logfiles to parse and what it the format of the message: logstash.conf grok { match =\> \[ "message", '%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:dockerhost} %{DAT… --- ## [How to remove fields with regex from json?](https://discuss.elastic.co/t/how-to-remove-fields-with-regex-from-json/111776)
**Author:** [@son](https://discuss.elastic.co/u/son)\ **Replies:** 5\ **Last updated:** [December 19, 2017, 8:48am UTC](https://discuss.elastic.co/t/how-to-remove-fields-with-regex-from-json/111776 "2017-12-19T08:48:45Z")
Hi, I am stucking in removing fields with regex. I have log in json. From filebeat i have configured it to push to logstash via config: filebeat.prospectors: input\_type: log paths: E:\\test\\app\\Logging\\test.json j… --- ## [Apply multiline codec and filter](https://discuss.elastic.co/t/apply-multiline-codec-and-filter/112156)
**Author:** [@anudeepmk](https://discuss.elastic.co/u/anudeepmk)\ **Replies:** 8\ **Last updated:** [December 19, 2017, 8:37am UTC](https://discuss.elastic.co/t/apply-multiline-codec-and-filter/112156 "2017-12-19T08:37:29Z")
I have the log files generated from one of our hybris as below, INFO | jvm 1 | main | 2017/12/18 04:13:28.985 | at org.zkoss.zk.ui.impl.EventProcessingThreadImpl.run(EventProcessingThreadImpl.java:446) \[zk-3.6.4-hybris… --- ## [How does Logstash handle multiple pipeline files?](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 3\ **Last updated:** [December 19, 2017, 7:47am UTC](https://discuss.elastic.co/t/how-does-logstash-handle-multiple-pipeline-files/111645 "2017-12-19T07:47:37Z")
I've tried looking in the documentation and Googling for this, but I couldn't find any answer. I'm unclear about how Logstash handles these. I know there is a /etc/logstash/conf.d/ directory which you can put your pipeli… --- ## [Catching/Logging Events That Break Parsing](https://discuss.elastic.co/t/catching-logging-events-that-break-parsing/112320)
**Author:** [@cb2015](https://discuss.elastic.co/u/cb2015)\ **Replies:** 1\ **Last updated:** [December 19, 2017, 6:28am UTC](https://discuss.elastic.co/t/catching-logging-events-that-break-parsing/112320 "2017-12-19T06:28:43Z")
Hi Everyone, So I am using ELK to parse logs with the help of filebeat. I ran into an issue today where ~900 log lines were dropped when they should not have been. I know which lines these were. However, when I looked … --- ## [Logstash in to es cloud when ssl is set](https://discuss.elastic.co/t/logstash-in-to-es-cloud-when-ssl-is-set/112339)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 2\ **Last updated:** [December 19, 2017, 6:15am UTC](https://discuss.elastic.co/t/logstash-in-to-es-cloud-when-ssl-is-set/112339 "2017-12-19T06:15:57Z")
logstash in to es cloud when ssl is set .how to setting the logstash.conf --- ## [Write parsed fields to redis and not the whole message](https://discuss.elastic.co/t/write-parsed-fields-to-redis-and-not-the-whole-message/112344)
**Author:** [@truthadjustr](https://discuss.elastic.co/u/truthadjustr)\ **Replies:** 1\ **Last updated:** [December 19, 2017, 4:54am UTC](https://discuss.elastic.co/t/write-parsed-fields-to-redis-and-not-the-whole-message/112344 "2017-12-19T04:54:54Z")
I am sending some output to redis but i learn that it is sending the message field content. How do i select which field to send to redis instead of the message field? --- ## [Date filter vs. accepting @timestamp as-is](https://discuss.elastic.co/t/date-filter-vs-accepting-timestamp-as-is/112311)
**Author:** [@jjensen](https://discuss.elastic.co/u/jjensen)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 10:41pm UTC](https://discuss.elastic.co/t/date-filter-vs-accepting-timestamp-as-is/112311 "2017-12-18T22:41:22Z")
Hi, This is something I've been curious about - I know that using the date filter you can basically replace what's in @timestamp with the actual date of the log entry itself. But is there a downside to just not using a … --- ## [Keep the "last" dump only in an elasticsearch index](https://discuss.elastic.co/t/keep-the-last-dump-only-in-an-elasticsearch-index/112318)
**Author:** [@mr\_chang7](https://discuss.elastic.co/u/mr_chang7)\ **Replies:** 0\ **Last updated:** [December 18, 2017, 10:06pm UTC](https://discuss.elastic.co/t/keep-the-last-dump-only-in-an-elasticsearch-index/112318 "2017-12-18T22:06:59Z")
Hi guys, Let's say I have an index "A" that needs to keep the last dump of data. Data come in dumps with a few days of interval (irregular). The events of entities have entity ID's and I put each event by its ID (as do… --- ## [Output data in multiple indices based on fields](https://discuss.elastic.co/t/output-data-in-multiple-indices-based-on-fields/112303)
**Author:** [@Harsh\_Verma](https://discuss.elastic.co/u/Harsh_Verma)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 9:37pm UTC](https://discuss.elastic.co/t/output-data-in-multiple-indices-based-on-fields/112303 "2017-12-18T21:37:18Z")
I have a logfile with some special logs and some general logs.I want the general logs to go in a general index and the special logs(logs which match any of my grok patterns) to go in a separate index and in the general i… --- ## [Error in parse Geo\_point in logstash](https://discuss.elastic.co/t/error-in-parse-geo-point-in-logstash/112122)
**Author:** [@yaser](https://discuss.elastic.co/u/yaser)\ **Replies:** 11\ **Last updated:** [December 18, 2017, 9:32pm UTC](https://discuss.elastic.co/t/error-in-parse-geo-point-in-logstash/112122 "2017-12-18T21:32:12Z")
i send this json to logstash : { "location" :45.5 } this is my conf.d/config file :: input { rabbitmq { user =\> 'user' password =\> '' exchange =\> '' queue =\> '\*\*\*\*' durable =\> true host =\> 'ip' subscriptio… --- ## [Error installing logstash plugin](https://discuss.elastic.co/t/error-installing-logstash-plugin/112265)
**Author:** [@diegopereiraeng](https://discuss.elastic.co/u/diegopereiraeng)\ **Replies:** 2\ **Last updated:** [December 18, 2017, 7:54pm UTC](https://discuss.elastic.co/t/error-installing-logstash-plugin/112265 "2017-12-18T19:54:02Z")
Hi, I have installed logstash 5.5.1 using .tar. The problem is only installing plug-ins. I'm geting this error: /logstash-5.5.1\> ./bin/logstash-plugin install file:///tmp/tcp-plugin.zip Installing file: /tmp/tcp-plu… --- ## [Two Logstash conf's with Ruby filter are modifying each others output](https://discuss.elastic.co/t/two-logstash-confs-with-ruby-filter-are-modifying-each-others-output/112287)
**Author:** [@tarunwx](https://discuss.elastic.co/u/tarunwx)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 7:42pm UTC](https://discuss.elastic.co/t/two-logstash-confs-with-ruby-filter-are-modifying-each-others-output/112287 "2017-12-18T19:42:14Z")
user\_profiles - main object ---educations - array of object (man can have many educational degrees) ---experiences - array of object (many can have many experiences) These both indexes are getting merged with final ob… --- ## [Logstash syslog TCP output breaking to parts](https://discuss.elastic.co/t/logstash-syslog-tcp-output-breaking-to-parts/110551)
**Author:** [@dorr](https://discuss.elastic.co/u/dorr)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 7:14pm UTC](https://discuss.elastic.co/t/logstash-syslog-tcp-output-breaking-to-parts/110551 "2017-12-18T19:14:33Z")
Hi, question I got from a customer regarding Logstash 2.3.4 we use there: I've noticed that the logs are sent in multiple syslog message. 1 application log item = X tcp syslog request. Is it possible to make 1 applic… --- ## [Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties](https://discuss.elastic.co/t/could-not-find-log4j2-configuration-at-path-usr-share-logstash-config-log4j2-properties/111857)
**Author:** [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Replies:** 1\ **Last updated:** [December 18, 2017, 7:11pm UTC](https://discuss.elastic.co/t/could-not-find-log4j2-configuration-at-path-usr-share-logstash-config-log4j2-properties/111857 "2017-12-18T19:11:37Z")
Hi everyone. I decided to sign up for this site after troubleshooting my issue and nothing seemed to help. I installed ELK Stack and Logstash from this website: http://www.itzgeek.com/how-tos/linux/ubuntu-how-tos/setup… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=852) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=854)