# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1223 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1224 --- ## [Grok timestamp field](https://discuss.elastic.co/t/grok-timestamp-field/33883)
**Author:** [@beorn107](https://discuss.elastic.co/u/beorn107)\ **Replies:** 3\ **Last updated:** [November 5, 2015, 8:24pm UTC](https://discuss.elastic.co/t/grok-timestamp-field/33883 "2015-11-05T20:24:35Z")
Hi I am trying to import a JSON file into ElasticSearch to use with Kibana. I am able to pull the file into ElasticSearch using Logstash but I am getting stuck on this one issue. The json has the timestamp for the r… --- ## [Logstash init process in Debian Jessie](https://discuss.elastic.co/t/logstash-init-process-in-debian-jessie/33673)
**Author:** [@Don\_Pich](https://discuss.elastic.co/u/Don_Pich)\ **Replies:** 6\ **Last updated:** [November 5, 2015, 7:06pm UTC](https://discuss.elastic.co/t/logstash-init-process-in-debian-jessie/33673 "2015-11-05T19:06:06Z")
for the life of me, I cannot understand why this isn't working. If I start the logstash daemon manually with this one liner built off of the init, it works fine. /opt/logstash/bin/logstash agent -f /etc/logstash/con… --- ## [Shipping logs from logstash 1.5 to indexer running 2.0](https://discuss.elastic.co/t/shipping-logs-from-logstash-1-5-to-indexer-running-2-0/33907)
**Author:** [@mgirndt](https://discuss.elastic.co/u/mgirndt)\ **Replies:** 2\ **Last updated:** [November 5, 2015, 6:28pm UTC](https://discuss.elastic.co/t/shipping-logs-from-logstash-1-5-to-indexer-running-2-0/33907 "2015-11-05T18:28:37Z")
Can logstash ship logs from a server running logstash 1.5 through redis to my indexer which is running logstash 2.0? Thanks, Michael --- ## [\_grokparsefailure still happening for a small number of IIS log entries - how to track them down?](https://discuss.elastic.co/t/-grokparsefailure-still-happening-for-a-small-number-of-iis-log-entries-how-to-track-them-down/33476)
**Author:** [@mc1903](https://discuss.elastic.co/u/mc1903)\ **Replies:** 6\ **Last updated:** [November 5, 2015, 5:35pm UTC](https://discuss.elastic.co/t/-grokparsefailure-still-happening-for-a-small-number-of-iis-log-entries-how-to-track-them-down/33476 "2015-11-05T17:35:45Z")
So, moving on from my previous 2 posts. I am making good progress, slowly admittedly, but I am learning a heck of a lot as I go! I am still having a very small number (approx \<0.1%) of my IIS log entries failing with \_… --- ## [Shield Audit Logstash Pattern?](https://discuss.elastic.co/t/shield-audit-logstash-pattern/33887)
**Author:** [@Bobby\_Pilot](https://discuss.elastic.co/u/Bobby_Pilot)\ **Replies:** 0\ **Last updated:** [November 5, 2015, 4:04pm UTC](https://discuss.elastic.co/t/shield-audit-logstash-pattern/33887 "2015-11-05T16:04:31Z")
Before I go create my own, is there a published logstash grep pattern for Shield audit logs? The audit message is published like so for us at the INFO level: \[2015-11-05 09:33:58,959\]\[INFO \]\[shield.audit.logfile \] … --- ## [Multiple beats inputs](https://discuss.elastic.co/t/multiple-beats-inputs/33661)
**Author:** [@voipoclay](https://discuss.elastic.co/u/voipoclay)\ **Replies:** 9\ **Last updated:** [November 5, 2015, 3:49pm UTC](https://discuss.elastic.co/t/multiple-beats-inputs/33661 "2015-11-05T15:49:01Z")
Hello all! I was curious if adding more beats inputs (and adding the additional ports to filebeat and loadbalance:true) would increase throughput of logstash? --- ## [Creating a tag out of the extracted field from the message](https://discuss.elastic.co/t/creating-a-tag-out-of-the-extracted-field-from-the-message/33753)
**Author:** [@swetav](https://discuss.elastic.co/u/swetav)\ **Replies:** 5\ **Last updated:** [November 5, 2015, 3:13pm UTC](https://discuss.elastic.co/t/creating-a-tag-out-of-the-extracted-field-from-the-message/33753 "2015-11-05T15:13:42Z")
Hi I have parsed a log using "COMBINEDAPACHELOG". filter{ grok { match =\> \[ "message", "%{COMBINEDAPACHELOG}" \] } } It's working well but now i need to extract string from the message tag and crerate a new… --- ## [Trying to parse double colons with grok fails](https://discuss.elastic.co/t/trying-to-parse-double-colons-with-grok-fails/33846)
**Author:** [@daneel](https://discuss.elastic.co/u/daneel)\ **Replies:** 2\ **Last updated:** [November 5, 2015, 2:40pm UTC](https://discuss.elastic.co/t/trying-to-parse-double-colons-with-grok-fails/33846 "2015-11-05T14:40:17Z")
Hi, I'm trying to parse the following strings from a file input: ::10.10.86.13 \< auth 10.10.86.13 27/10/2015 13:57:01 \[test\] \[123456789132\] \[\] \[\] \[num|123456789456789|1234567894561234\] \> As soon as I try to parse t… --- ## [Logstash plugin unrecoverable error when using a mapping](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815)
**Author:** [@Iain\_Woolf](https://discuss.elastic.co/u/Iain_Woolf)\ **Replies:** 4\ **Last updated:** [November 5, 2015, 2:39pm UTC](https://discuss.elastic.co/t/logstash-plugin-unrecoverable-error-when-using-a-mapping/33815 "2015-11-05T14:39:38Z")
I'm trying to pull in UK Crime data to Elasticsearch so that I can experiment with the map visualizations in Kibana 4. When I create a mapping to show that the location field is a geo\_point, I then get errors when trying… --- ## [How to handle date related log files?](https://discuss.elastic.co/t/how-to-handle-date-related-log-files/33725)
**Author:** [@Yzord](https://discuss.elastic.co/u/Yzord)\ **Replies:** 2\ **Last updated:** [November 5, 2015, 1:34pm UTC](https://discuss.elastic.co/t/how-to-handle-date-related-log-files/33725 "2015-11-05T13:34:37Z")
I am installing an ELK server for all the logfiles we have in our network. Configuring logstash-forwarder isn't that hard. I use for example to push dhcpd.log to my ELK server. But now i have a server which has logfil… --- ## [Logstash consume a large amount of memory and CPU](https://discuss.elastic.co/t/logstash-consume-a-large-amount-of-memory-and-cpu/33861)
**Author:** [@Pierre\_Jacquot](https://discuss.elastic.co/u/Pierre_Jacquot)\ **Replies:** 0\ **Last updated:** [November 5, 2015, 12:03pm UTC](https://discuss.elastic.co/t/logstash-consume-a-large-amount-of-memory-and-cpu/33861 "2015-11-05T12:03:00Z")
Hello, We are making some implementation on our system of logstash in order to retrieve some information on differentes sources (BDD,FILE, JMX). We implemented 3 windows service that use the same logstash context with… --- ## [Logstash](https://discuss.elastic.co/t/logstash/33716)
**Author:** [@sylv](https://discuss.elastic.co/u/sylv)\ **Replies:** 3\ **Last updated:** [November 5, 2015, 11:09am UTC](https://discuss.elastic.co/t/logstash/33716 "2015-11-05T11:09:52Z")
hello I set new versions : elasticsearch kibana and logstach i have these errors with my config file txt who was ok with previous versions: l\\logstash\_titulaires\_config\_file.txt io/console not supported; tty will not b… --- ## [Kafka.consumer.RangeAssignor: No broker partitions consumed by consumer thread logstash\_logstash-indexer](https://discuss.elastic.co/t/kafka-consumer-rangeassignor-no-broker-partitions-consumed-by-consumer-thread-logstash-logstash-indexer/33012)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 9\ **Last updated:** [November 5, 2015, 10:31am UTC](https://discuss.elastic.co/t/kafka-consumer-rangeassignor-no-broker-partitions-consumed-by-consumer-thread-logstash-logstash-indexer/33012 "2015-11-05T10:31:13Z")
I have an ELK Set-up in which a logstash is pushing data to Kafka and another logstash is pulling data from Kafka. Below are my Kafka Input Config:- input { kafka { zk\_connect =\> "kafka:2181" grou… --- ## [Multiple logstash forwarder instances](https://discuss.elastic.co/t/multiple-logstash-forwarder-instances/29850)
**Author:** [@paulkeogh](https://discuss.elastic.co/u/paulkeogh)\ **Replies:** 11\ **Last updated:** [November 5, 2015, 7:42am UTC](https://discuss.elastic.co/t/multiple-logstash-forwarder-instances/29850 "2015-11-05T07:42:53Z")
Hi, I am running multiple LF instances on a single host - unfortunately I started them both from the same working directory so they are sharing a .logstash-forwarder file. I presume this is incorrect and I will recti… --- ## [Copying @timestamp in custom field not working](https://discuss.elastic.co/t/copying-timestamp-in-custom-field-not-working/33738)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 4\ **Last updated:** [November 5, 2015, 6:40am UTC](https://discuss.elastic.co/t/copying-timestamp-in-custom-field-not-working/33738 "2015-11-05T06:40:46Z")
Hi, I am trying to copy the timestamp field in logs to some custom field 'eventLogTime' on Kibana. This I want to use for sorting on basis of event time. I am using below configuration in logstash indexer grok { … --- ## [Add new input plugin when logstash is running](https://discuss.elastic.co/t/add-new-input-plugin-when-logstash-is-running/33823)
**Author:** [@kitchanw1209](https://discuss.elastic.co/u/kitchanw1209)\ **Replies:** 2\ **Last updated:** [November 5, 2015, 6:23am UTC](https://discuss.elastic.co/t/add-new-input-plugin-when-logstash-is-running/33823 "2015-11-05T06:23:47Z")
Does logstash have any way for user to add new input plugin when it is running? For instance, I'm just monitoring fileA, now I want to monitor fileB and a database also, but I don't want to shut down or restart logstash.… --- ## [Problems receiving data from large number of logstash-forwarders on logstash](https://discuss.elastic.co/t/problems-receiving-data-from-large-number-of-logstash-forwarders-on-logstash/33584)
**Author:** [@Medelwr](https://discuss.elastic.co/u/Medelwr)\ **Replies:** 8\ **Last updated:** [November 5, 2015, 2:50am UTC](https://discuss.elastic.co/t/problems-receiving-data-from-large-number-of-logstash-forwarders-on-logstash/33584 "2015-11-05T02:50:57Z")
Hi, I have 60+ clients that I am trying to connect to my logstash instance. They are sending logs on a short interval using LSF (Lumberjack). Versions: Logstash 2.0 Elasticsearch 2.0 Kibana 4.2 Logstash-forwarder 0.… --- ## [Logstash CEF codec](https://discuss.elastic.co/t/logstash-cef-codec/33783)
**Author:** [@us3r](https://discuss.elastic.co/u/us3r)\ **Replies:** 3\ **Last updated:** [November 4, 2015, 10:15pm UTC](https://discuss.elastic.co/t/logstash-cef-codec/33783 "2015-11-04T22:15:20Z")
Together with my collegue we found a few problems with this codec: The first one is described here -\> https://github.com/logstash-plugins/logstash-codec-cef/issues/9 Second one is related with the changes in Elasticsea… --- ## [How to start LS1.5 as a background task](https://discuss.elastic.co/t/how-to-start-ls1-5-as-a-background-task/33814)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 1\ **Last updated:** [November 4, 2015, 10:06pm UTC](https://discuss.elastic.co/t/how-to-start-ls1-5-as-a-background-task/33814 "2015-11-04T22:06:18Z")
We are using Linux with LogStash 1.5.4. We are trying to setup Logstash to run as a background process on this server. We ahve tried to start service as the regular background task like this: nohup ./bin/logstash -f… --- ## [Java version on forwarder machine](https://discuss.elastic.co/t/java-version-on-forwarder-machine/33798)
**Author:** [@weevil](https://discuss.elastic.co/u/weevil)\ **Replies:** 1\ **Last updated:** [November 4, 2015, 9:14pm UTC](https://discuss.elastic.co/t/java-version-on-forwarder-machine/33798 "2015-11-04T21:14:19Z")
I know it is highly recommended to use Java 8 for elasticsearch and logstash. Is it highly recommended on machines running just the logstash-forwarder also? --- ## [Visualize - splitting sentences](https://discuss.elastic.co/t/visualize-splitting-sentences/33721)
**Author:** [@zappe](https://discuss.elastic.co/u/zappe)\ **Replies:** 3\ **Last updated:** [November 4, 2015, 7:23pm UTC](https://discuss.elastic.co/t/visualize-splitting-sentences/33721 "2015-11-04T19:23:09Z")
When I'm trying to visualize a field, on the X axis, which has more then one word, then Kibana split every word into one, for example, bar. Why is that? --- ## [Best Way to create nested field](https://discuss.elastic.co/t/best-way-to-create-nested-field/26757)
**Author:** [@DavidL](https://discuss.elastic.co/u/DavidL)\ **Replies:** 9\ **Last updated:** [November 4, 2015, 7:22pm UTC](https://discuss.elastic.co/t/best-way-to-create-nested-field/26757 "2015-11-04T19:22:11Z")
I successfully parsed out some fields, and am trying to aggregate some of them into one fields. Anyone suggestions on how to to that would be really appreciated. I haven't found a way to do so, but I assume this must be … --- ## [Multi\_line codec vs filter](https://discuss.elastic.co/t/multi-line-codec-vs-filter/33782)
**Author:** [@marinoc](https://discuss.elastic.co/u/marinoc)\ **Replies:** 0\ **Last updated:** [November 4, 2015, 7:05pm UTC](https://discuss.elastic.co/t/multi-line-codec-vs-filter/33782 "2015-11-04T19:05:31Z")
I'm using Logstash to index files where separate lines relate to the same event, thus I want to use multi\_line. What is the difference between using a multi\_line codec in my input plugin versus using the multi\_line fil… --- ## [Logstash custom apache logs](https://discuss.elastic.co/t/logstash-custom-apache-logs/33761)
**Author:** [@mlopez](https://discuss.elastic.co/u/mlopez)\ **Replies:** 2\ **Last updated:** [November 4, 2015, 5:53pm UTC](https://discuss.elastic.co/t/logstash-custom-apache-logs/33761 "2015-11-04T17:53:12Z")
Hi, I've just setup a trial cluster and managed to configure logstash to send out data. However we are using a custom apache logs that include the vhost server's name as shown here below : cloudfront.jnstaging.crosskno… --- ## [Logstash read from kafka in a round-robin way](https://discuss.elastic.co/t/logstash-read-from-kafka-in-a-round-robin-way/33760)
**Author:** [@Chengcheng\_Pei](https://discuss.elastic.co/u/Chengcheng_Pei)\ **Replies:** 0\ **Last updated:** [November 4, 2015, 3:29pm UTC](https://discuss.elastic.co/t/logstash-read-from-kafka-in-a-round-robin-way/33760 "2015-11-04T15:29:51Z")
I have 3 partitions: 0, 1, 2. So the messages can be classified as 0, 1, 2. eg: 1 message in partition 0: 0 3 messages in partition 1: 111 2 messages in partition 2: 22 How to make the logstash to consume me… --- ## [Cant get Logstash to connect to ES ( localhost )](https://discuss.elastic.co/t/cant-get-logstash-to-connect-to-es-localhost/33592)
**Author:** [@Kyle\_Hartigan](https://discuss.elastic.co/u/Kyle_Hartigan)\ **Replies:** 7\ **Last updated:** [November 4, 2015, 2:20pm UTC](https://discuss.elastic.co/t/cant-get-logstash-to-connect-to-es-localhost/33592 "2015-11-04T14:20:55Z")
i am doing a brand new clean install of Logstash and Elasticsearch ( version 2.0) and i am having issues getting logstash to connect to the elasticsearch server on the same host, there is no firewall installed and i c… --- ## [LSF performance bottleneck with Logstash Multiline?](https://discuss.elastic.co/t/lsf-performance-bottleneck-with-logstash-multiline/33730)
**Author:** [@logilouis](https://discuss.elastic.co/u/logilouis)\ **Replies:** 0\ **Last updated:** [November 4, 2015, 11:04am UTC](https://discuss.elastic.co/t/lsf-performance-bottleneck-with-logstash-multiline/33730 "2015-11-04T11:04:37Z")
Hi, Currently we have ELK setup as below: LSF \[:arrow\_right:\] Logstash (indexer) \[:arrow\_right:\] Elasticsearch Each component is running on individual server hosted in same data center. The LSF is running together wi… --- ## [TLS handshake / read error looking for ack](https://discuss.elastic.co/t/tls-handshake-read-error-looking-for-ack/33254)
**Author:** [@incogniro](https://discuss.elastic.co/u/incogniro)\ **Replies:** 2\ **Last updated:** [November 4, 2015, 10:44am UTC](https://discuss.elastic.co/t/tls-handshake-read-error-looking-for-ack/33254 "2015-11-04T10:44:32Z")
Hi I'm having an issue with the forwarders in a deployment which fills the logstash-forwarder.err log with a combination of the messages shown below. Output in the forwarder log: 2015/10/29 13:06:53.594221 Connecting t… --- ## [Logstash and GeoIPASN](https://discuss.elastic.co/t/logstash-and-geoipasn/33726)
**Author:** [@sampasei](https://discuss.elastic.co/u/sampasei)\ **Replies:** 0\ **Last updated:** [November 4, 2015, 10:37am UTC](https://discuss.elastic.co/t/logstash-and-geoipasn/33726 "2015-11-04T10:37:53Z")
Hi all, i have a question about logstash and GeoIPASN. I'm using GeoIPASNum.dat and works well: i'm using the snipped code below geoip { source =\> "id\_orig\_h" target =\> "geoIPASN" … --- ## [Conditional grok](https://discuss.elastic.co/t/conditional-grok/33641)
**Author:** [@Tak\_MK](https://discuss.elastic.co/u/Tak_MK)\ **Replies:** 7\ **Last updated:** [November 4, 2015, 8:40am UTC](https://discuss.elastic.co/t/conditional-grok/33641 "2015-11-04T08:40:40Z")
Hi guys! I have a log with two totally different patterns (yeah, I should split them up), and I want to parse them with grok. Can I do something like "if the message has X word, use this grok pattern, else, use that … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1222) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1224)