# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1186 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1187 --- ## [Curl to elasticsearch ELK cluster doesn't bring back current data whereas Kibana does](https://discuss.elastic.co/t/curl-to-elasticsearch-elk-cluster-doesnt-bring-back-current-data-whereas-kibana-does/42782)
**Author:** [@Stuart\_Cracraft1](https://discuss.elastic.co/u/Stuart_Cracraft1)\ **Replies:** 3\ **Last updated:** [February 26, 2016, 8:31am UTC](https://discuss.elastic.co/t/curl-to-elasticsearch-elk-cluster-doesnt-bring-back-current-data-whereas-kibana-does/42782 "2016-02-26T08:31:55Z")
On Kibana, I see all current data all the time. But when I do a curl -XGET 'http://elkcluster:9200/logstash-YYYY.MM.DD/\_search?host=somehost&type=heartbeat' it shows me out of date logstash heartbeats. I go b… --- ## [Logstash not preserving mappings when copying docs from one index to another](https://discuss.elastic.co/t/logstash-not-preserving-mappings-when-copying-docs-from-one-index-to-another/42815)
**Author:** [@srinivas](https://discuss.elastic.co/u/srinivas)\ **Replies:** 2\ **Last updated:** [February 26, 2016, 7:02am UTC](https://discuss.elastic.co/t/logstash-not-preserving-mappings-when-copying-docs-from-one-index-to-another/42815 "2016-02-26T07:02:48Z")
I am using the following pipeline to copy data from index to another. input { elasticsearch { hosts =\> \["sourcehost:9200"\] index =\> "source-index" query =\> '{"query":{"match\_all":{}}}' size =\> 500 scro… --- ## [Multiple logstash reading from a single kafka topic](https://discuss.elastic.co/t/multiple-logstash-reading-from-a-single-kafka-topic/27727)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 9\ **Last updated:** [February 26, 2016, 5:17am UTC](https://discuss.elastic.co/t/multiple-logstash-reading-from-a-single-kafka-topic/27727 "2016-02-26T05:17:50Z")
I want to have multiple logstash reading from a single kafka topic. The log message in a kafka topic should be read by only one of the logstash instances. I could not find any doc related to this. I have some doubts r… --- ## [How can I upgrade Logstash?](https://discuss.elastic.co/t/how-can-i-upgrade-logstash/42715)
**Author:** [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Replies:** 3\ **Last updated:** [February 26, 2016, 1:26am UTC](https://discuss.elastic.co/t/how-can-i-upgrade-logstash/42715 "2016-02-26T01:26:29Z")
Hi, Currently I am running Logstash 1.5.3, how can I upgrade it to the latest version? Is there any step by step procedure? Btw, I don't use Logstash forwarder. --- ## [Logstash Dies when Elasticsearch is restarted](https://discuss.elastic.co/t/logstash-dies-when-elasticsearch-is-restarted/42783)
**Author:** [@Bradly](https://discuss.elastic.co/u/Bradly)\ **Replies:** 1\ **Last updated:** [February 26, 2016, 12:27am UTC](https://discuss.elastic.co/t/logstash-dies-when-elasticsearch-is-restarted/42783 "2016-02-26T00:27:17Z")
Everytime I restart any of the elasticsearch nodes they logstash server stops sending logs to any of them. I have it setup to send data to all 5 host in my cluster from the netflow on our core router. The output lines … --- ## [s3 Input plugin is not sending ELB Data](https://discuss.elastic.co/t/s3-input-plugin-is-not-sending-elb-data/42693)
**Author:** [@aniamin](https://discuss.elastic.co/u/aniamin)\ **Replies:** 1\ **Last updated:** [February 25, 2016, 11:06pm UTC](https://discuss.elastic.co/t/s3-input-plugin-is-not-sending-elb-data/42693 "2016-02-25T23:06:54Z")
I want to send log of ELB to s3 bucket. ELB logs of different service will be in different directory of my main log bucket. When I tried to put that in my input s3 conf, I am not getting any log Here is my s3 input con… --- ## [s3 input plugin is not collecting ELB Logs](https://discuss.elastic.co/t/s3-input-plugin-is-not-collecting-elb-logs/42692)
**Author:** [@aniamin](https://discuss.elastic.co/u/aniamin)\ **Replies:** 1\ **Last updated:** [February 25, 2016, 11:06pm UTC](https://discuss.elastic.co/t/s3-input-plugin-is-not-collecting-elb-logs/42692 "2016-02-25T23:06:47Z")
I want to send log of ELB to s3 bucket. ELB logs of different service will be in different directory of my main log bucket. When I tried to put that in my input s3 conf, I am not getting any log Here is my s3 input con… --- ## [Logstash input (kafka) add\_field very slow](https://discuss.elastic.co/t/logstash-input-kafka-add-field-very-slow/28541)
**Author:** [@L8apex32](https://discuss.elastic.co/u/L8apex32)\ **Replies:** 4\ **Last updated:** [February 25, 2016, 10:31pm UTC](https://discuss.elastic.co/t/logstash-input-kafka-add-field-very-slow/28541 "2016-02-25T22:31:00Z")
We are working towards very fast ingest into elastic search and found an unexpected slow spot. We have LS pulling from kafka which it can do very fast (5 consumer threads) - over 50k eps. However, if we add a single "add… --- ## [Is there a Grok Pattern for Windows Event Log?](https://discuss.elastic.co/t/is-there-a-grok-pattern-for-windows-event-log/42762)
**Author:** [@eso](https://discuss.elastic.co/u/eso)\ **Replies:** 1\ **Last updated:** [February 25, 2016, 9:14pm UTC](https://discuss.elastic.co/t/is-there-a-grok-pattern-for-windows-event-log/42762 "2016-02-25T21:14:02Z")
Hello, I am new to Elastic Stack. and don't know much about Grok and how to construct patterns. Today is my second days to read about Elastic Stack, particularly LogStash. I would like to play with it in order to le… --- ## [How to use logstash filters](https://discuss.elastic.co/t/how-to-use-logstash-filters/42643)
**Author:** [@Aron\_Pedersen](https://discuss.elastic.co/u/Aron_Pedersen)\ **Replies:** 3\ **Last updated:** [February 25, 2016, 6:45pm UTC](https://discuss.elastic.co/t/how-to-use-logstash-filters/42643 "2016-02-25T18:45:00Z")
I am new to the whole BELK stack (or The Elastic Stack from 5.0.0) so sorry if the question is not well described. I have logs that looks like the example below that I ship with filebeat to logstash Log example: 2/24… --- ## [Rotate index every other day](https://discuss.elastic.co/t/rotate-index-every-other-day/42497)
**Author:** [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Replies:** 4\ **Last updated:** [February 25, 2016, 3:20pm UTC](https://discuss.elastic.co/t/rotate-index-every-other-day/42497 "2016-02-25T15:20:26Z")
Hello Guys, I'm using logstash 2.2, in a Centos 6. Logstash rotates the indices each day, is there any way to rotate index every 2 days or 3 days?. Or rotate the index by size?. I must save and keep open the index … --- ## [Logstash not working until I re-save the file](https://discuss.elastic.co/t/logstash-not-working-until-i-re-save-the-file/42559)
**Author:** [@Natarajan\_venkataram](https://discuss.elastic.co/u/Natarajan_venkataram)\ **Replies:** 3\ **Last updated:** [February 25, 2016, 10:27am UTC](https://discuss.elastic.co/t/logstash-not-working-until-i-re-save-the-file/42559 "2016-02-25T10:27:01Z")
The logstash is not parsing the csv logs files automatically. It works only after I reopen the file and save it again. Sometimes, if the log files are more than 1GB, opening the files with Excel ( or any other editor) is… --- ## [LS doesn't index to ES during tutorial](https://discuss.elastic.co/t/ls-doesnt-index-to-es-during-tutorial/42568)
**Author:** [@andrew.trobec](https://discuss.elastic.co/u/andrew.trobec)\ **Replies:** 7\ **Last updated:** [February 25, 2016, 8:44am UTC](https://discuss.elastic.co/t/ls-doesnt-index-to-es-during-tutorial/42568 "2016-02-25T08:44:15Z")
Hello! I am very very new to this platform so I have very little idea of what I'm doing. I'm following the LS tutorial and have reached the "Setting Up an Advanced Logstash Pipeline" section. I have carried out all… --- ## [Database request](https://discuss.elastic.co/t/database-request/42596)
**Author:** [@Justin\_V](https://discuss.elastic.co/u/Justin_V)\ **Replies:** 3\ **Last updated:** [February 25, 2016, 7:49am UTC](https://discuss.elastic.co/t/database-request/42596 "2016-02-25T07:49:45Z")
Hello, I didn't find the answer in other topics, so I post a new one. Here's the thing : In my logs, I have some id's, who are totally not representative of what they are exactly. So my question is : After having "gro… --- ## [Logstash 1.5.6 consuming CPU when no incoming traffic](https://discuss.elastic.co/t/logstash-1-5-6-consuming-cpu-when-no-incoming-traffic/40956)
**Author:** [@Karol\_Stojek](https://discuss.elastic.co/u/Karol_Stojek)\ **Replies:** 6\ **Last updated:** [February 25, 2016, 6:29am UTC](https://discuss.elastic.co/t/logstash-1-5-6-consuming-cpu-when-no-incoming-traffic/40956 "2016-02-25T06:29:01Z")
I have a logstash listening on a single port that has accepted traffic from filebeats. It was working fine for a few hours. Now I've stopped all the filebeats, there is no incomming traffic on the listened port. But … --- ## [Split row data to fields](https://discuss.elastic.co/t/split-row-data-to-fields/42586)
**Author:** [@chenbe2204](https://discuss.elastic.co/u/chenbe2204)\ **Replies:** 6\ **Last updated:** [February 25, 2016, 6:11am UTC](https://discuss.elastic.co/t/split-row-data-to-fields/42586 "2016-02-25T06:11:29Z")
Hi , When I load Json file to ElasticSearch , I sea the data in 1 column (message ). For example: {"Group": "abi", "ExTime": "", "LastSuccess": "", "PadDG%": "", "TestName": "PT\_FOTA/abi/SUPPORT/test132", "WI": "", "T… --- ## [Parsing DB data processes the fields as analyzed. Is there a way to change it?](https://discuss.elastic.co/t/parsing-db-data-processes-the-fields-as-analyzed-is-there-a-way-to-change-it/42633)
**Author:** [@RaviSidhu](https://discuss.elastic.co/u/RaviSidhu)\ **Replies:** 7\ **Last updated:** [February 25, 2016, 4:51am UTC](https://discuss.elastic.co/t/parsing-db-data-processes-the-fields-as-analyzed-is-there-a-way-to-change-it/42633 "2016-02-25T04:51:28Z")
Hi, I am connecting to a database in Logstash, querying a table, and inserting the data in ES. But the fields are all analyzed. Is there a way out for me to allow the strings as not\_analyzed? Although I see the full v… --- ## [Pulling logs into logstash without running an agent on every vm](https://discuss.elastic.co/t/pulling-logs-into-logstash-without-running-an-agent-on-every-vm/42605)
**Author:** [@cshawler](https://discuss.elastic.co/u/cshawler)\ **Replies:** 3\ **Last updated:** [February 24, 2016, 6:59pm UTC](https://discuss.elastic.co/t/pulling-logs-into-logstash-without-running-an-agent-on-every-vm/42605 "2016-02-24T18:59:33Z")
Is there away to pull logs into logstash without installing a shipper agent on every vm? --- ## [Logstash Architecture at Scale](https://discuss.elastic.co/t/logstash-architecture-at-scale/42495)
**Author:** [@misterjulien](https://discuss.elastic.co/u/misterjulien)\ **Replies:** 2\ **Last updated:** [February 24, 2016, 6:46pm UTC](https://discuss.elastic.co/t/logstash-architecture-at-scale/42495 "2016-02-24T18:46:36Z")
Regarding the doc "Deploying and Scaling Logstash", what would be an ideal way(s) to implement a mature Logstash cluster? The picture at the bottom of the doc shows multiple instances of Shippers, a message queue, and … --- ## [After convert csv to ES anyway to update index without change its name](https://discuss.elastic.co/t/after-convert-csv-to-es-anyway-to-update-index-without-change-its-name/42481)
**Author:** [@Neal\_Shan](https://discuss.elastic.co/u/Neal_Shan)\ **Replies:** 6\ **Last updated:** [February 24, 2016, 6:32pm UTC](https://discuss.elastic.co/t/after-convert-csv-to-es-anyway-to-update-index-without-change-its-name/42481 "2016-02-24T18:32:11Z")
Scenario 1. use logstash transfer csv to elasticsearch 2. csv update every week, have to create a new index for it, and index name will be changed, otherwise, the index will have old and new data in it, we only need the … --- ## [Kv variable with multiple types](https://discuss.elastic.co/t/kv-variable-with-multiple-types/42102)
**Author:** [@angelsmile](https://discuss.elastic.co/u/angelsmile)\ **Replies:** 6\ **Last updated:** [February 24, 2016, 6:31pm UTC](https://discuss.elastic.co/t/kv-variable-with-multiple-types/42102 "2016-02-24T18:31:43Z")
Hi, I have used kv filter in logstash configuration file. A variable "jobname" created by kv originally had the type of integer according to its value, "jobname=40000" but now in some documents the values of the v… --- ## [Timezone based on some field value](https://discuss.elastic.co/t/timezone-based-on-some-field-value/42618)
**Author:** [@tikejhya](https://discuss.elastic.co/u/tikejhya)\ **Replies:** 1\ **Last updated:** [February 24, 2016, 5:57pm UTC](https://discuss.elastic.co/t/timezone-based-on-some-field-value/42618 "2016-02-24T17:57:45Z")
there are multiple files parsed through logstash and those files are based on country and country has its specific timestamp ie.: 2016-02-24 14:13:04 \[without timezone\] And it looks like logstash only shows file in pr… --- ## [Logstash/Beats on non-Linux server](https://discuss.elastic.co/t/logstash-beats-on-non-linux-server/42621)
**Author:** [@jeurich](https://discuss.elastic.co/u/jeurich)\ **Replies:** 1\ **Last updated:** [February 24, 2016, 5:49pm UTC](https://discuss.elastic.co/t/logstash-beats-on-non-linux-server/42621 "2016-02-24T17:49:30Z")
The environment I am supporting is a mixed environment of Solaris, AIX and Linux. So far I am not finding any information on getting beats to run on non-linux UNIX servers. Has anyone been able to get this to work? An… --- ## [Geoip no longer working](https://discuss.elastic.co/t/geoip-no-longer-working/42417)
**Author:** [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Replies:** 10\ **Last updated:** [February 24, 2016, 1:50pm UTC](https://discuss.elastic.co/t/geoip-no-longer-working/42417 "2016-02-24T13:50:22Z")
Hi, all. Reecently, I moved from logstash-forwarder to filebeat on my linux nodes. Prior to the change, I had geoip working well. This is the stanza in logstash-forwarder.conf which, along with the input and filter c… --- ## [Snmptrap throwing errors](https://discuss.elastic.co/t/snmptrap-throwing-errors/42571)
**Author:** [@massimo](https://discuss.elastic.co/u/massimo)\ **Replies:** 2\ **Last updated:** [February 24, 2016, 12:21pm UTC](https://discuss.elastic.co/t/snmptrap-throwing-errors/42571 "2016-02-24T12:21:51Z")
Hello, I configured an input for receiving snmp traps. However non of them get stored in elasticsearch. When looking at the output of rubydebug errors showe up: ailed action. {:status=\>400, :action=\>\["index", {:id=\>n… --- ## [Responsetime doesnt come in JSON object in case of %{COMMONAPACHELOG}"](https://discuss.elastic.co/t/responsetime-doesnt-come-in-json-object-in-case-of-commonapachelog/42562)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 5\ **Last updated:** [February 24, 2016, 11:20am UTC](https://discuss.elastic.co/t/responsetime-doesnt-come-in-json-object-in-case-of-commonapachelog/42562 "2016-02-24T11:20:59Z")
Hi, I am using logstash 1.5.0 as indexer to filter apache access logs. I use match =\> { "message" =\> "%{COMMONAPACHELOG}" } in my groke filter. Sample log is : xx.xx.xx.xxx - - \[24/Feb/2016:07:45:54 +0200\] "GET /s… --- ## [Problem input file version 2.2.0](https://discuss.elastic.co/t/problem-input-file-version-2-2-0/41802)
**Author:** [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Replies:** 6\ **Last updated:** [February 24, 2016, 8:25am UTC](https://discuss.elastic.co/t/problem-input-file-version-2-2-0/41802 "2016-02-24T08:25:46Z")
Maurizio Maltese: Hello there everybody I recently updated to the last version of Logstash, taking it to the 2.2.0. I'm having some issues with the input file plugin. When I run Logstash and insert files that have to … --- ## [Geoip not working while taking input from lumberjack](https://discuss.elastic.co/t/geoip-not-working-while-taking-input-from-lumberjack/42390)
**Author:** [@shivam\_singh](https://discuss.elastic.co/u/shivam_singh)\ **Replies:** 4\ **Last updated:** [February 24, 2016, 7:44am UTC](https://discuss.elastic.co/t/geoip-not-working-while-taking-input-from-lumberjack/42390 "2016-02-24T07:44:23Z")
Hi, When i send nginx logs using logstash-forwarder i am getting following in response \`172.10.6.182 - - \[22/Feb/2016:02:06:53 -0500\] "GET /analytics.js HTTP/1.1" 301 184 "http://www.hjhjhjkk.com/searchResult" "Mozi… --- ## [How to filter object,float,string datatype values coming from single variable](https://discuss.elastic.co/t/how-to-filter-object-float-string-datatype-values-coming-from-single-variable/42461)
**Author:** [@naiktej13](https://discuss.elastic.co/u/naiktej13)\ **Replies:** 4\ **Last updated:** [February 24, 2016, 7:25am UTC](https://discuss.elastic.co/t/how-to-filter-object-float-string-datatype-values-coming-from-single-variable/42461 "2016-02-24T07:25:11Z")
I am trying to parse the data which is in the following format: { "node" =\> "10.0.1.136", "timestamp" =\> "2016-02-23 11:47:09", "namespace" =\> "statistics", "stats" =\> { "erro… --- ## [Parsing failed to due to \\r\\n in logstash](https://discuss.elastic.co/t/parsing-failed-to-due-to-r-n-in-logstash/42490)
**Author:** [@Rohan\_Jain](https://discuss.elastic.co/u/Rohan_Jain)\ **Replies:** 1\ **Last updated:** [February 23, 2016, 6:51pm UTC](https://discuss.elastic.co/t/parsing-failed-to-due-to-r-n-in-logstash/42490 "2016-02-23T18:51:02Z")
In my log files \\r\\n is coming which is windows newline and in unix system logstash is unable to parse this and i am getting grokparsefailure. How can i solve this error. Unix recognizes \\n as newline. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1185) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1187)