# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1177 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1178 --- ## [What does the filter metavent do?](https://discuss.elastic.co/t/what-does-the-filter-metavent-do/44937)
**Author:** [@jupp](https://discuss.elastic.co/u/jupp)\ **Replies:** 2\ **Last updated:** [March 21, 2016, 12:00am UTC](https://discuss.elastic.co/t/what-does-the-filter-metavent-do/44937 "2016-03-21T00:00:37Z")
I would like to understand what this plugin does . Can someone explain to me? --- ## [Mutate field with no format?](https://discuss.elastic.co/t/mutate-field-with-no-format/44928)
**Author:** [@Libfy](https://discuss.elastic.co/u/Libfy)\ **Replies:** 1\ **Last updated:** [March 20, 2016, 4:11pm UTC](https://discuss.elastic.co/t/mutate-field-with-no-format/44928 "2016-03-20T16:11:35Z")
Hi all, I want to put in my logstash config file a mutate instruction to change a field in a number type without format pattern. Could you please help me to do this? Many thanks for your help. Libfy --- ## [Wrong Timestamp](https://discuss.elastic.co/t/wrong-timestamp/44930)
**Author:** [@Libfy](https://discuss.elastic.co/u/Libfy)\ **Replies:** 3\ **Last updated:** [March 20, 2016, 2:58pm UTC](https://discuss.elastic.co/t/wrong-timestamp/44930 "2016-03-20T14:58:59Z")
Hi, I have an issue with logstash. all logs are prefixed with a wrong timestamp: the year is 2000. The rest of the date is good. here is my output section from logstash.conf: output { elasticsearch { … --- ## [Aggregation of log time-series - median per minute?](https://discuss.elastic.co/t/aggregation-of-log-time-series-median-per-minute/44587)
**Author:** [@marius](https://discuss.elastic.co/u/marius)\ **Replies:** 2\ **Last updated:** [March 20, 2016, 8:19am UTC](https://discuss.elastic.co/t/aggregation-of-log-time-series-median-per-minute/44587 "2016-03-20T08:19:39Z")
Hi, I am facing the problem that I have high throughput JSON logs. I don't want to store the logs with that accuracy, but the sender is not configurable. I wonder if Logstash can do something for me to calculate the… --- ## [Read aws cloud trail log from s3 bucket](https://discuss.elastic.co/t/read-aws-cloud-trail-log-from-s3-bucket/44820)
**Author:** [@mohankgm](https://discuss.elastic.co/u/mohankgm)\ **Replies:** 1\ **Last updated:** [March 20, 2016, 12:29am UTC](https://discuss.elastic.co/t/read-aws-cloud-trail-log-from-s3-bucket/44820 "2016-03-20T00:29:01Z")
Hi, I'm reading the cloud trail log s from s3 bucket using the blow code. But only I getting log adding this " prefix =\> "AWSLogs/xxxxxxxxx/CloudTrail/us-west-2/2016/03/18"" How get the log recursively from S3 subfo… --- ## [Getting sporadic MapperParsingException errors for elasticsearch output (logstash 2.2.2)](https://discuss.elastic.co/t/getting-sporadic-mapperparsingexception-errors-for-elasticsearch-output-logstash-2-2-2/44874)
**Author:** [@gerry](https://discuss.elastic.co/u/gerry)\ **Replies:** 1\ **Last updated:** [March 19, 2016, 5:40pm UTC](https://discuss.elastic.co/t/getting-sporadic-mapperparsingexception-errors-for-elasticsearch-output-logstash-2-2-2/44874 "2016-03-19T17:40:07Z")
Hi, I'm using logstash 2.2.2 to read from rabbitmq and post to my ES cluster (1.7.4), and I'm getting occasional "MapperParsingException\[object mapping for \[email\_queue\] tried to parse field \[payload\] as object, but go… --- ## [Rabbitmq plugin fails to validate certificates](https://discuss.elastic.co/t/rabbitmq-plugin-fails-to-validate-certificates/44628)
**Author:** [@sjsadowski](https://discuss.elastic.co/u/sjsadowski)\ **Replies:** 1\ **Last updated:** [March 19, 2016, 2:58pm UTC](https://discuss.elastic.co/t/rabbitmq-plugin-fails-to-validate-certificates/44628 "2016-03-19T14:58:05Z")
I was looking to see how to actually report this, and am not quite sure how since the rabbitmq input plugin has no maintainer listed here: https://discuss.elastic.co/t/logstash-plugins-community-maintainers/35953 Our t… --- ## [Need Ruby developer help on a logstash plugin](https://discuss.elastic.co/t/need-ruby-developer-help-on-a-logstash-plugin/44899)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 0\ **Last updated:** [March 19, 2016, 1:23pm UTC](https://discuss.elastic.co/t/need-ruby-developer-help-on-a-logstash-plugin/44899 "2016-03-19T13:23:02Z")
Hello, I am working on re-writing logstash plugin filter-useragent by using a Java library. This improve performance a lot and it's more accurate (in my tests). But I have a few problem with spec test, if a ruby guru c… --- ## [Cannot get log from RabbitMQ (get log very slow)](https://discuss.elastic.co/t/cannot-get-log-from-rabbitmq-get-log-very-slow/44787)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 6\ **Last updated:** [March 19, 2016, 1:20pm UTC](https://discuss.elastic.co/t/cannot-get-log-from-rabbitmq-get-log-very-slow/44787 "2016-03-19T13:20:10Z")
Hi, i'm ussing lastet verion ES, LS. I'm ussing Brocker RabbitMQ : LS Shipper -\> RabbitMQ -\> LS Indexer -\> ES Now I run, i check on RabbitMQ, i have 2M doc/event queue (Incomming 6-10k event/doc per second) and ack ver… --- ## [Testers wanted: Lumberjack integration with Hortonworks](https://discuss.elastic.co/t/testers-wanted-lumberjack-integration-with-hortonworks/44880)
**Author:** [@trixpan](https://discuss.elastic.co/u/trixpan)\ **Replies:** 2\ **Last updated:** [March 19, 2016, 5:06am UTC](https://discuss.elastic.co/t/testers-wanted-lumberjack-integration-with-hortonworks/44880 "2016-03-19T05:06:12Z")
Hi there, For those using logstash-forwarder but also using Hortonworks stack (HDP and HDF) I have baked an experimental listener that allows Hortonworks Dataflow (a.k.a. Apache NiFi) to accept connections from logst… --- ## [How to filter windows events logs events with newlines](https://discuss.elastic.co/t/how-to-filter-windows-events-logs-events-with-newlines/44245)
**Author:** [@s802645](https://discuss.elastic.co/u/s802645)\ **Replies:** 2\ **Last updated:** [March 19, 2016, 5:02am UTC](https://discuss.elastic.co/t/how-to-filter-windows-events-logs-events-with-newlines/44245 "2016-03-19T05:02:39Z")
Hi, I received windows events from another source in this format: EventCode=1234 EvenType= blah blah User=me Message= key1:value1 arbitrarty .... Each event entry that comes into log stash has a newline. I j… --- ## [Pipeline stalls and errors in logstash](https://discuss.elastic.co/t/pipeline-stalls-and-errors-in-logstash/41256)
**Author:** [@dan.henning](https://discuss.elastic.co/u/dan.henning)\ **Replies:** 3\ **Last updated:** [March 19, 2016, 4:59am UTC](https://discuss.elastic.co/t/pipeline-stalls-and-errors-in-logstash/41256 "2016-03-19T04:59:41Z")
We are currently testing out logstash, elastic search, kibana and filebeat to log some IIS log files, each thing can talk to each other correctly but when processing the logs in logstash to send to elastic search we get … --- ## [Logstash without SSL?](https://discuss.elastic.co/t/logstash-without-ssl/732)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 14\ **Last updated:** [March 19, 2016, 2:01am UTC](https://discuss.elastic.co/t/logstash-without-ssl/732 "2016-03-19T02:01:40Z")
Hi, I want to know whether we can ship logs to logstash without SSL? is there any way to disable SSL and then configure logstash and forwarder without SSL certificates? br, Sunil --- ## [Twitter::Error::Unauthorized](https://discuss.elastic.co/t/twitter-error-unauthorized/42136)
**Author:** [@jitendersaini](https://discuss.elastic.co/u/jitendersaini)\ **Replies:** 2\ **Last updated:** [March 19, 2016, 1:53am UTC](https://discuss.elastic.co/t/twitter-error-unauthorized/42136 "2016-03-19T01:53:38Z")
HI I'm new to EKL and trying to learn this. I've installed the logstash-2.2.2 and elasticsearch-2.1.1 in my machine. I'm trying to read the twitters feed as given in the logstash guide but it's throwing the Unauthorized … --- ## [Logs to logstash config file](https://discuss.elastic.co/t/logs-to-logstash-config-file/44735)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 6\ **Last updated:** [March 19, 2016, 12:16am UTC](https://discuss.elastic.co/t/logs-to-logstash-config-file/44735 "2016-03-19T00:16:46Z")
I have a simple log file consisting of some information...how do convert it to a logstash-configuration file for indexing in elasticsearch and visualizing in kibana? I am confused. please help! --- ## [Kafka output is very slow with dynamic fields (sprintf)](https://discuss.elastic.co/t/kafka-output-is-very-slow-with-dynamic-fields-sprintf/44836)
**Author:** [@palmerabollo](https://discuss.elastic.co/u/palmerabollo)\ **Replies:** 1\ **Last updated:** [March 18, 2016, 6:49pm UTC](https://discuss.elastic.co/t/kafka-output-is-very-slow-with-dynamic-fields-sprintf/44836 "2016-03-18T18:49:20Z")
I have a logstash 2.2.x process handling 6000 events per second. It has the following kafka output configured: kafka { bootstrap\_servers =\> "kafkaserver:9092" topic\_id =\> "mytopic" } When we change the topic\_id to… --- ## [Timezone offset problem](https://discuss.elastic.co/t/timezone-offset-problem/44845)
**Author:** [@DaveO](https://discuss.elastic.co/u/DaveO)\ **Replies:** 2\ **Last updated:** [March 18, 2016, 5:29pm UTC](https://discuss.elastic.co/t/timezone-offset-problem/44845 "2016-03-18T17:29:18Z")
I'm having problems setting @timestamp based on the date, time and time zone offset fields from System Center Configuration Manager client logs. After much trial and error, I have narrowed down the problem to the time … --- ## [Logstash input/output parameters?](https://discuss.elastic.co/t/logstash-input-output-parameters/44833)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [March 18, 2016, 11:41am UTC](https://discuss.elastic.co/t/logstash-input-output-parameters/44833 "2016-03-18T11:41:21Z")
Hi, Could anydoy explain below parameters in input and output plugin of logstash? batch\_count? threads? What is the trade off for better performance? What is relation between workers and threads for one input or multip… --- ## [Load balancing Shipper\\Indexer](https://discuss.elastic.co/t/load-balancing-shipper-indexer/44593)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 2\ **Last updated:** [March 18, 2016, 11:35am UTC](https://discuss.elastic.co/t/load-balancing-shipper-indexer/44593 "2016-03-18T11:35:29Z")
Hello, I am using below structure of event processing pipeline in my project. LSF--\>logstash-shipper(Scaled)---\>Redis---\>Indexer(Scaled)---\>NGINX---\>Multiple nodes ES Here, I want to make Shipper and indexer highly av… --- ## [How to manage logstash deployments?](https://discuss.elastic.co/t/how-to-manage-logstash-deployments/44768)
**Author:** [@Joshua\_P](https://discuss.elastic.co/u/Joshua_P)\ **Replies:** 4\ **Last updated:** [March 18, 2016, 11:20am UTC](https://discuss.elastic.co/t/how-to-manage-logstash-deployments/44768 "2016-03-18T11:20:41Z")
What's the best way to manage multiple Logstash deployments? I have a testing environment and a production environment and each has it's own configuration file pulling data from its respective data store and sending t… --- ## [Logstash service will not run (Ubuntu server 14.04)](https://discuss.elastic.co/t/logstash-service-will-not-run-ubuntu-server-14-04/44719)
**Author:** [@brayndasilva](https://discuss.elastic.co/u/brayndasilva)\ **Replies:** 2\ **Last updated:** [March 18, 2016, 10:53am UTC](https://discuss.elastic.co/t/logstash-service-will-not-run-ubuntu-server-14-04/44719 "2016-03-18T10:53:59Z")
Hi, I'm trying out the ELK stack on an environment of 1 ELK server, two client servers and one laptop to access Kabina. I have followed https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-lo… --- ## [Logstash to collect ES logs and visualize to Kibana](https://discuss.elastic.co/t/logstash-to-collect-es-logs-and-visualize-to-kibana/44526)
**Author:** [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Replies:** 2\ **Last updated:** [March 18, 2016, 10:25am UTC](https://discuss.elastic.co/t/logstash-to-collect-es-logs-and-visualize-to-kibana/44526 "2016-03-18T10:25:06Z")
Hi All, We are using Elasticsearch 1.7.3 for our application. Our ES cluster has 3 physical servers with 48 cores of CPU & 256 GB RAM. We are having 3 master nodes, 6 data nodes & 1 client node and having nearly 6 TB… --- ## [Logstash failing to parse json](https://discuss.elastic.co/t/logstash-failing-to-parse-json/44471)
**Author:** [@narengorla1](https://discuss.elastic.co/u/narengorla1)\ **Replies:** 20\ **Last updated:** [March 18, 2016, 9:12am UTC](https://discuss.elastic.co/t/logstash-failing-to-parse-json/44471 "2016-03-18T09:12:22Z")
Hi - I have been trying to parse a json string and i get jasonpraseer error.. below is the json string.. . "{reportSuite=geometrixxapibeta, timeGMT=1457955198, receivedTimeGMT=1457955198, hitIdHigh=3130934947308273664,… --- ## [Time stamp change](https://discuss.elastic.co/t/time-stamp-change/44696)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 2\ **Last updated:** [March 18, 2016, 8:44am UTC](https://discuss.elastic.co/t/time-stamp-change/44696 "2016-03-18T08:44:35Z")
Hi All , I am trying to feed in the log from past date using GROK but the date picked in @timestamp is getting changed by some month & year offset. Actual Date-2016-03-16 12:28:11AM Time stamp in elasticsearch - "@time… --- ## [Auto mapping a csv file](https://discuss.elastic.co/t/auto-mapping-a-csv-file/44695)
**Author:** [@Croos](https://discuss.elastic.co/u/Croos)\ **Replies:** 7\ **Last updated:** [March 18, 2016, 8:36am UTC](https://discuss.elastic.co/t/auto-mapping-a-csv-file/44695 "2016-03-18T08:36:31Z")
I'm uploading a csv file to elasticsearch by logstash befor that do the mapping manually. How to automate the mapping when the csv file is uploading? --- ## [Need a example with filter clones and script ruby](https://discuss.elastic.co/t/need-a-example-with-filter-clones-and-script-ruby/44305)
**Author:** [@patou](https://discuss.elastic.co/u/patou)\ **Replies:** 1\ **Last updated:** [March 18, 2016, 6:56am UTC](https://discuss.elastic.co/t/need-a-example-with-filter-clones-and-script-ruby/44305 "2016-03-18T06:56:59Z")
For a line of a file csv, I make an array with a script ruby. for this line, I wish to iterate on this array to make a clone for every element. Is it possible? You have a example ? Cdt patrick --- ## [Logstsh elastic search struggling](https://discuss.elastic.co/t/logstsh-elastic-search-struggling/41807)
**Author:** [@bubu.senapati](https://discuss.elastic.co/u/bubu.senapati)\ **Replies:** 9\ **Last updated:** [March 18, 2016, 6:55am UTC](https://discuss.elastic.co/t/logstsh-elastic-search-struggling/41807 "2016-03-18T06:55:57Z")
Hi Magnus, I have an ELK stack deployed in production. Its just a single node Log files from various servers (webserver and application server) are pumped to Elastic search through logstash. Suddenly with doubled traff… --- ## [Connecting FileBeats to Logstash](https://discuss.elastic.co/t/connecting-filebeats-to-logstash/44575)
**Author:** [@nateuni](https://discuss.elastic.co/u/nateuni)\ **Replies:** 3\ **Last updated:** [March 17, 2016, 4:17am UTC](https://discuss.elastic.co/t/connecting-filebeats-to-logstash/44575 "2016-03-17T04:17:58Z")
I have FileBeats running on an ubuntu webserver on EC2. I have opened up the ports, but still I cannot get them to talk to each other. filebeats starts without errors but I cannot get it to print the apache logs to th… --- ## [Groc Regexs in Logstash](https://discuss.elastic.co/t/groc-regexs-in-logstash/44706)
**Author:** [@nateuni](https://discuss.elastic.co/u/nateuni)\ **Replies:** 1\ **Last updated:** [March 17, 2016, 5:05pm UTC](https://discuss.elastic.co/t/groc-regexs-in-logstash/44706 "2016-03-17T17:05:35Z")
I have apache log that I have also added cookie information to. The cookie is a from a wordpress site and I want to get the username out of it, so that I can log users and the pages that they visit. I can use the extende… --- ## [Help with grok](https://discuss.elastic.co/t/help-with-grok/44772)
**Author:** [@Jorge](https://discuss.elastic.co/u/Jorge)\ **Replies:** 0\ **Last updated:** [March 17, 2016, 8:42pm UTC](https://discuss.elastic.co/t/help-with-grok/44772 "2016-03-17T20:42:07Z")
Hi all, I have the following log , as you can see , it consists of different lines and each of the lines may have different fields such system id : As you can see the titles of the fields carry ( : ) . The first line is… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1176) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1178)