# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1109 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1110 --- ## [Creating/updating array of objects in elasticsearch logstash output](https://discuss.elastic.co/t/creating-updating-array-of-objects-in-elasticsearch-logstash-output/59139)
**Author:** [@Nandan\_Phadke](https://discuss.elastic.co/u/Nandan_Phadke)\ **Replies:** 2\ **Last updated:** [August 30, 2016, 4:30am UTC](https://discuss.elastic.co/t/creating-updating-array-of-objects-in-elasticsearch-logstash-output/59139 "2016-08-30T04:30:55Z")
I am facing an issue using elastic search output with logstash. Here is my sample event { "guid":"someguid", "nestedObject":{ "field1":"val1", "field2":"val2" } } Here is what I want to hav… --- ## [Redis cluster can communicate with logstash cluster](https://discuss.elastic.co/t/redis-cluster-can-communicate-with-logstash-cluster/59181)
**Author:** [@shashant](https://discuss.elastic.co/u/shashant)\ **Replies:** 4\ **Last updated:** [August 30, 2016, 4:15am UTC](https://discuss.elastic.co/t/redis-cluster-can-communicate-with-logstash-cluster/59181 "2016-08-30T04:15:50Z")
Hi, i want to communication b/w redis cluster and logstash cluster. is it possible is yes the how. --- ## [Problem formatting Log Timestamp](https://discuss.elastic.co/t/problem-formatting-log-timestamp/59262)
**Author:** [@RaviSidhu](https://discuss.elastic.co/u/RaviSidhu)\ **Replies:** 1\ **Last updated:** [August 30, 2016, 2:33am UTC](https://discuss.elastic.co/t/problem-formatting-log-timestamp/59262 "2016-08-30T02:33:55Z")
Hi All, As a test, I am querying a DB using logtash and printing the output on the prompt. I have two columns access\_date(29-AUG-16) and access\_time(11:30:45) in the DB. While querying, I concatenate the two fields. … --- ## [@timestamp always 8h exact more than the matching time?](https://discuss.elastic.co/t/timestamp-always-8h-exact-more-than-the-matching-time/59236)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 2\ **Last updated:** [August 29, 2016, 11:48pm UTC](https://discuss.elastic.co/t/timestamp-always-8h-exact-more-than-the-matching-time/59236 "2016-08-29T23:48:58Z")
Hi I know there are quite a few topics on this. But still i couldn't figure it out after reviewing them. THIS is how i date filtered date field to target @timestamp (tried with and without timezone). date{ matc… --- ## [Logstash Wildcard does not seem to be working?](https://discuss.elastic.co/t/logstash-wildcard-does-not-seem-to-be-working/59070)
**Author:** [@DustinVest](https://discuss.elastic.co/u/DustinVest)\ **Replies:** 6\ **Last updated:** [August 29, 2016, 6:09pm UTC](https://discuss.elastic.co/t/logstash-wildcard-does-not-seem-to-be-working/59070 "2016-08-29T18:09:05Z")
I cant seem to get a wildcard to work in my config file. I have a folder with a bunch of csv files, tab delimited. I got it to run once on a single file but I cant seem to get that to work either now. Here is my confi… --- ## [How to configure logstash to index json logs including all fields](https://discuss.elastic.co/t/how-to-configure-logstash-to-index-json-logs-including-all-fields/58874)
**Author:** [@lin-zhao](https://discuss.elastic.co/u/lin-zhao)\ **Replies:** 5\ **Last updated:** [August 29, 2016, 5:41pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-index-json-logs-including-all-fields/58874 "2016-08-29T17:41:47Z")
Say the input of my logstash job is already in line delimited json format. I want to have each json key indexed as a field to elasticsearch, is there to make this work? For example, the input looks like: { "a": "valu… --- ## [Referencing a field instead of Index](https://discuss.elastic.co/t/referencing-a-field-instead-of-index/59207)
**Author:** [@sombilat](https://discuss.elastic.co/u/sombilat)\ **Replies:** 0\ **Last updated:** [August 29, 2016, 4:18pm UTC](https://discuss.elastic.co/t/referencing-a-field-instead-of-index/59207 "2016-08-29T16:18:36Z")
Here's the scenario: We have a JDBC configuration, lets call it JDBC. We are not outputting an Index for this. Which extracts data every hour to an output file in a target dir. The File JDBC generates, is picked up by an… --- ## [Date filter failing with JDBC input](https://discuss.elastic.co/t/date-filter-failing-with-jdbc-input/59198)
**Author:** [@aethos](https://discuss.elastic.co/u/aethos)\ **Replies:** 0\ **Last updated:** [August 29, 2016, 2:59pm UTC](https://discuss.elastic.co/t/date-filter-failing-with-jdbc-input/59198 "2016-08-29T14:59:01Z")
I have JDBC collecting from a database and then I'm feeding one of the fields to date to store it in timestamp. However I ran into an issue and found a workaround. Wondering if there might be a bug in JDBC input? Basic… --- ## [Extract fields from JSON](https://discuss.elastic.co/t/extract-fields-from-json/59075)
**Author:** [@ggp](https://discuss.elastic.co/u/ggp)\ **Replies:** 9\ **Last updated:** [August 29, 2016, 2:54pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075 "2016-08-29T14:54:24Z")
I have this below JSON coming from RabbitMQ event": { "payloadContext": { "messageProfile": { "domain": "ERP", "process": "process-01", "serviceName": "service-01", "serviceVersion": "4.0.0.RELEASE" }, "applica… --- ## [Logstash is not creating Indexes - SOLVED](https://discuss.elastic.co/t/logstash-is-not-creating-indexes-solved/57984)
**Author:** [@Vishal\_Sharma1](https://discuss.elastic.co/u/Vishal_Sharma1)\ **Replies:** 20\ **Last updated:** [August 29, 2016, 2:22pm UTC](https://discuss.elastic.co/t/logstash-is-not-creating-indexes-solved/57984 "2016-08-29T14:22:53Z")
i have logstash running and its suddnely stopped creating indexes. I have installed logstash from the repository on Centos7. service logstash configtest is ok sudo service logstash start starts log stash properly. In… --- ## [Logs from journald wih Logstash and the journald plugin](https://discuss.elastic.co/t/logs-from-journald-wih-logstash-and-the-journald-plugin/58939)
**Author:** [@bnordbo](https://discuss.elastic.co/u/bnordbo)\ **Replies:** 2\ **Last updated:** [August 29, 2016, 1:44pm UTC](https://discuss.elastic.co/t/logs-from-journald-wih-logstash-and-the-journald-plugin/58939 "2016-08-29T13:44:05Z")
Hi, I'm trying to set up log forwarding from journald to Elasticsearch using Logstash and the journald input plugin, but I'm running into some (to me, anyway) odd issues related to the \_uid field. Using Logstash 2.3.4 … --- ## [Can i have 2-3 input in a config file](https://discuss.elastic.co/t/can-i-have-2-3-input-in-a-config-file/59176)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 1\ **Last updated:** [August 29, 2016, 12:12pm UTC](https://discuss.elastic.co/t/can-i-have-2-3-input-in-a-config-file/59176 "2016-08-29T12:12:20Z")
Hi All I know this is very basic question but i want avoid creating multiple conf files of LS I want to have 2-3 input and one standard output as ES in a single conf file Is it possible? --- ## [What is the significance of add\_tags in mutate](https://discuss.elastic.co/t/what-is-the-significance-of-add-tags-in-mutate/59157)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 2\ **Last updated:** [August 29, 2016, 11:52am UTC](https://discuss.elastic.co/t/what-is-the-significance-of-add-tags-in-mutate/59157 "2016-08-29T11:52:12Z")
Hi All I want to understand the significance of add\_tags in mutate filter Does it mean a description field has multiple keywords and we add tags to that For example : Description : This is really a good example S… --- ## [IP Field contained invalid IP address or hostname with geoip filter](https://discuss.elastic.co/t/ip-field-contained-invalid-ip-address-or-hostname-with-geoip-filter/49803)
**Author:** [@Duleendra](https://discuss.elastic.co/u/Duleendra)\ **Replies:** 4\ **Last updated:** [August 29, 2016, 11:46am UTC](https://discuss.elastic.co/t/ip-field-contained-invalid-ip-address-or-hostname-with-geoip-filter/49803 "2016-08-29T11:46:09Z")
Hi I am parsing a CSV file with the following configurations. This CSV file has a field which contains IP address. Problem is , Logstash gives the following error with geoip for IP addresses even though IP addresses… --- ## [Logstash not creating ES indexes](https://discuss.elastic.co/t/logstash-not-creating-es-indexes/59154)
**Author:** [@harpax](https://discuss.elastic.co/u/harpax)\ **Replies:** 1\ **Last updated:** [August 29, 2016, 11:06am UTC](https://discuss.elastic.co/t/logstash-not-creating-es-indexes/59154 "2016-08-29T11:06:19Z")
Hi, I have made an update from ES 1.4.x and Logstash 1.4.x to the current release versions (2.3.x) on a 3 node cluster. My old data is still available and with a few changes to the logstash.conf (deprecated functions/o… --- ## [Logstash issue with Beats Plugin](https://discuss.elastic.co/t/logstash-issue-with-beats-plugin/59166)
**Author:** [@Ashutosh](https://discuss.elastic.co/u/Ashutosh)\ **Replies:** 0\ **Last updated:** [August 29, 2016, 10:42am UTC](https://discuss.elastic.co/t/logstash-issue-with-beats-plugin/59166 "2016-08-29T10:42:04Z")
I have a 3 node setup of ELK Stack. For last few weeks I am facing a strange problem where Logstash is running without any issues but does not output any logs either to the Elasticsearch or Console. Especially with beats… --- ## [Categorize with keywords and assign value](https://discuss.elastic.co/t/categorize-with-keywords-and-assign-value/59121)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 4\ **Last updated:** [August 29, 2016, 7:35am UTC](https://discuss.elastic.co/t/categorize-with-keywords-and-assign-value/59121 "2016-08-29T07:35:35Z")
Hi All I have a Field called Description and using mutate i want add a new field called category Description has certain words like cat, dog If cat keyword comes then assign value Cat and with dog keyword with Categor… --- ## [Logstash rabbitmq output performance](https://discuss.elastic.co/t/logstash-rabbitmq-output-performance/59152)
**Author:** [@lkosz](https://discuss.elastic.co/u/lkosz)\ **Replies:** 0\ **Last updated:** [August 29, 2016, 6:44am UTC](https://discuss.elastic.co/t/logstash-rabbitmq-output-performance/59152 "2016-08-29T06:44:38Z")
Hi, I have ELK in following scheme: \[log-courier on hosts\] ~\> \[logstash-cache (15 instances)\] ~\> \[haproxy\] ~\> \[rabbitmq logstash-cache queue\] ~\> \[haproxy\] ~\> \[logstash-indexer (5 instances)\] ~\> \[haproxy\] ~\> \[rabbitmq l… --- ## [Prefix Preserving IP Anonymization](https://discuss.elastic.co/t/prefix-preserving-ip-anonymization/58709)
**Author:** [@Saurabh\_Jambhule](https://discuss.elastic.co/u/Saurabh_Jambhule)\ **Replies:** 5\ **Last updated:** [August 29, 2016, 5:27am UTC](https://discuss.elastic.co/t/prefix-preserving-ip-anonymization/58709 "2016-08-29T05:27:34Z")
Is there any algorithm that preserves prefix property of IP? --- ## [Logstash 2.3.4 getting stuck while attempting to install template in elasticsearch](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 7\ **Last updated:** [August 29, 2016, 5:26am UTC](https://discuss.elastic.co/t/logstash-2-3-4-getting-stuck-while-attempting-to-install-template-in-elasticsearch/59098 "2016-08-29T05:26:19Z")
Cross-posting from stackoverflow:- I am trying to upgradelogstash 1.5.3 to logstash 2.3.4 logstash config for 1.5.3 looks like below:- input { kafka { zk\_connect =\> "kafka:2181" group\_id =\> "log… --- ## [Reference "numbered" fields in sprintf format](https://discuss.elastic.co/t/reference-numbered-fields-in-sprintf-format/59064)
**Author:** [@kaung](https://discuss.elastic.co/u/kaung)\ **Replies:** 3\ **Last updated:** [August 29, 2016, 3:13am UTC](https://discuss.elastic.co/t/reference-numbered-fields-in-sprintf-format/59064 "2016-08-29T03:13:01Z")
I want to read the following json object in logstash. "aggregations": { "\_percentile": { "values": { "5.0": 10, "95.0": 10 } The following doesn't work: mutate { add\_field =\> { "target" =\> "%{\[aggregations\]\[\_… --- ## [Logstash Indexer to Elastcsearch Tunnings ( must go faster!)](https://discuss.elastic.co/t/logstash-indexer-to-elastcsearch-tunnings-must-go-faster/58171)
**Author:** [@eperry](https://discuss.elastic.co/u/eperry)\ **Replies:** 13\ **Last updated:** [August 29, 2016, 1:10am UTC](https://discuss.elastic.co/t/logstash-indexer-to-elastcsearch-tunnings-must-go-faster/58171 "2016-08-29T01:10:37Z")
Hi, I am trying to come up with tunning settings for my Logstash Indexer It sits between Kafka and Elasticsearch (Pulling off of 1 topic called logstash with 25 partitions) I have 12 nodes (2 client, 3 masters and t… --- ## [Define Action in http output](https://discuss.elastic.co/t/define-action-in-http-output/59112)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 6\ **Last updated:** [August 28, 2016, 3:10pm UTC](https://discuss.elastic.co/t/define-action-in-http-output/59112 "2016-08-28T15:10:18Z")
How to define action in http output? Here is my http output file output { http { url =\> "https://xyz,com?WSDL" content\_type =\> "text/xml;charset=UTF-8" http\_method =\> "post" format =\> "message" messa… --- ## [Logstash hangs with error sized\_queue\_timeout](https://discuss.elastic.co/t/logstash-hangs-with-error-sized-queue-timeout/59031)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 2\ **Last updated:** [August 28, 2016, 4:56am UTC](https://discuss.elastic.co/t/logstash-hangs-with-error-sized-queue-timeout/59031 "2016-08-28T04:56:29Z")
Cross-posting this from stackoverflow. We have a logstash pipeline in which numerous logstash-forwarders forward logs to a single logstash instance. Many times we have observed that the logstash hangs with the below er… --- ## [Http\_poller handling paged responses](https://discuss.elastic.co/t/http-poller-handling-paged-responses/59095)
**Author:** [@mkowalew3382](https://discuss.elastic.co/u/mkowalew3382)\ **Replies:** 1\ **Last updated:** [August 27, 2016, 11:36pm UTC](https://discuss.elastic.co/t/http-poller-handling-paged-responses/59095 "2016-08-27T23:36:14Z")
We are polling data from a rest api and it pulls back a response in batches. Is there any way for the poller to go grab the next batch? As of right now it is only consuming the 1st 2000 items and never getting past it. … --- ## [Grok filter for url in squid log](https://discuss.elastic.co/t/grok-filter-for-url-in-squid-log/58820)
**Author:** [@Saurabh\_Jambhule](https://discuss.elastic.co/u/Saurabh_Jambhule)\ **Replies:** 3\ **Last updated:** [August 27, 2016, 3:34pm UTC](https://discuss.elastic.co/t/grok-filter-for-url-in-squid-log/58820 "2016-08-27T15:34:08Z")
My current config is, grok { match =\> { "message" =\> "%{WORD:month}\\s\*%{INT:day} %{INT:tmp1}:%{INT:tmp2}:%{INT:tmp3} %{USERNAME:proxy\_server} %{WORD:squidnm}\[%{INT:proxy\_port}\]:\\s\*%{POSINT:timestamp}.%{WORD:t… --- ## [Logstash conditional "in" with single element list not working](https://discuss.elastic.co/t/logstash-conditional-in-with-single-element-list-not-working/58807)
**Author:** [@Thomas\_Larsson\_Kron](https://discuss.elastic.co/u/Thomas_Larsson_Kron)\ **Replies:** 2\ **Last updated:** [August 27, 2016, 2:45pm UTC](https://discuss.elastic.co/t/logstash-conditional-in-with-single-element-list-not-working/58807 "2016-08-27T14:45:06Z")
Hello. I'm using logstash 2.1.1 and I'm encountering a problem with using the "in" conditional expression to filter out log events that are not of a certain level. If I have more than one element in the list, everyth… --- ## [Duplicate documents](https://discuss.elastic.co/t/duplicate-documents/59002)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 2\ **Last updated:** [August 27, 2016, 2:33pm UTC](https://discuss.elastic.co/t/duplicate-documents/59002 "2016-08-27T14:33:58Z")
Hi, I have below configuration of LS. I don't know why it shows 2 JSON messages in debug on console as well as in ES, with same offset. I think there are 2 threads doing this duplication. How to debug thread information… --- ## [Logstash 2.2.2 stops sending messages](https://discuss.elastic.co/t/logstash-2-2-2-stops-sending-messages/58998)
**Author:** [@Dennis\_Au](https://discuss.elastic.co/u/Dennis_Au)\ **Replies:** 3\ **Last updated:** [August 26, 2016, 10:14pm UTC](https://discuss.elastic.co/t/logstash-2-2-2-stops-sending-messages/58998 "2016-08-26T22:14:25Z")
I'm having an issue with Logstash 2.2.2 with Oracle Java 1.8 just stopping and can't figure out what the issue is. There are multiple logstash processes running (one for windows, and one for syslog) listening on diff… --- ## [Logstash Redis and pattern\_channel question](https://discuss.elastic.co/t/logstash-redis-and-pattern-channel-question/59080)
**Author:** [@romspacenut](https://discuss.elastic.co/u/romspacenut)\ **Replies:** 1\ **Last updated:** [August 26, 2016, 10:04pm UTC](https://discuss.elastic.co/t/logstash-redis-and-pattern-channel-question/59080 "2016-08-26T22:04:57Z")
I'm in the middle of shipping our logs from beats to a Redis queue using pattern\_channel. I've read that the metadata is not not available through the Redis input. Currently our logs ship directly from beats to logstash.… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1108) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1110)