# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=918 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 919 --- ## [Get only the date from timestamp](https://discuss.elastic.co/t/get-only-the-date-from-timestamp/98568)
**Author:** [@Sunil\_M](https://discuss.elastic.co/u/Sunil_M)\ **Replies:** 0\ **Last updated:** [August 28, 2017, 1:03pm UTC](https://discuss.elastic.co/t/get-only-the-date-from-timestamp/98568 "2017-08-28T13:03:27Z")
Hi, I have a timestamp in my input. so i convert it as follows date { match =\> \[“timestamp”, “UNIX\_MS”\] target =\> “mydate” } And the mydate is 2017-08-22T11:26:22.156Z I would like to extract only the 2017-08-2… --- ## [Logstash logs filling up disk space](https://discuss.elastic.co/t/logstash-logs-filling-up-disk-space/98561)
**Author:** [@n\_b](https://discuss.elastic.co/u/n_b)\ **Replies:** 1\ **Last updated:** [August 28, 2017, 1:35pm UTC](https://discuss.elastic.co/t/logstash-logs-filling-up-disk-space/98561 "2017-08-28T13:35:31Z")
I am using logstash version 1.5.4 to push logs from a file to elasticsearch. We are using ‘-l’ option and specifying a local file for storing logstash’s logs. I am not using --verbose or debug mode. Now, in case of err… --- ## [Versioning with version\_type external](https://discuss.elastic.co/t/versioning-with-version-type-external/98446)
**Author:** [@Lukas\_Tilch](https://discuss.elastic.co/u/Lukas_Tilch)\ **Replies:** 2\ **Last updated:** [August 28, 2017, 1:11pm UTC](https://discuss.elastic.co/t/versioning-with-version-type-external/98446 "2017-08-28T13:11:05Z")
Hello, I want to use an external versioning system with logstash so a document only gets updated when the version is GTE The config looks like this output { stdout { codec =\> rubydebug } elasticsearch { hosts =\> … --- ## [Timestamp not enough precision](https://discuss.elastic.co/t/timestamp-not-enough-precision/98531)
**Author:** [@Robert\_Kovacs](https://discuss.elastic.co/u/Robert_Kovacs)\ **Replies:** 4\ **Last updated:** [August 28, 2017, 12:59pm UTC](https://discuss.elastic.co/t/timestamp-not-enough-precision/98531 "2017-08-28T12:59:49Z")
I have logs with time precision of only seconds like this: 2017-08-09 14:54:38 INFO something 2017-08-09 14:54:38 INFO something 2017-08-09 14:54:38 DEBUG something 2017-08-09 14:54:38 INFO something I use the date … --- ## [Xml filter array](https://discuss.elastic.co/t/xml-filter-array/96055)
**Author:** [@Lukas\_Tilch](https://discuss.elastic.co/u/Lukas_Tilch)\ **Replies:** 16\ **Last updated:** [August 28, 2017, 12:09pm UTC](https://discuss.elastic.co/t/xml-filter-array/96055 "2017-08-28T12:09:17Z")
Hello I have a XML file wich contains 10k+ elements, I played around with the XML filter for logstash wich only gives me arrays like this "date" =\> \[ \[ 0\] "Jun 21, 2013 1:48:43 PM", \[ 1\] "Apr 22, 2013 12:16:00 PM", … --- ## [Multi input, multi output and aggregate filter](https://discuss.elastic.co/t/multi-input-multi-output-and-aggregate-filter/98549)
**Author:** [@M4nd76](https://discuss.elastic.co/u/M4nd76)\ **Replies:** 0\ **Last updated:** [August 28, 2017, 11:00am UTC](https://discuss.elastic.co/t/multi-input-multi-output-and-aggregate-filter/98549 "2017-08-28T11:00:06Z")
Hello, I’m trying to configure an instance of Logstash to read into 3 differents databases and to output, for each database, into a specific ElasticSearch index (one database \<-\> one index). For each databases, the str… --- ## [Message to List then to Object representation](https://discuss.elastic.co/t/message-to-list-then-to-object-representation/98527)
**Author:** [@panchicore](https://discuss.elastic.co/u/panchicore)\ **Replies:** 1\ **Last updated:** [August 28, 2017, 10:56am UTC](https://discuss.elastic.co/t/message-to-list-then-to-object-representation/98527 "2017-08-28T10:56:15Z")
I groked an event resulting with the field: "a5" =\> "...", "a6" =\> "KILL#11##1#Cameroon#CM#CM#6#12#CM;CRISISLEX\_T03\_DEAD#11##1#Cameroon#CM#CM#6#12#CM;" "a7" =\> "...", Then applied the split mutation to a6 and result wa… --- ## [Logstash having issues with bad interpreter: No such file or directory](https://discuss.elastic.co/t/logstash-having-issues-with-bad-interpreter-no-such-file-or-directory/98534)
**Author:** [@Raghunandan\_Sk](https://discuss.elastic.co/u/Raghunandan_Sk)\ **Replies:** 0\ **Last updated:** [August 28, 2017, 9:54am UTC](https://discuss.elastic.co/t/logstash-having-issues-with-bad-interpreter-no-such-file-or-directory/98534 "2017-08-28T09:54:34Z")
Hey guys , I had logstash running fine till 2 days ago but due to disk issue I had to reboot the server after that logstash was not working fine. I add new disk to this machine (its AWS EC2 machine ) I removed logstash… --- ## [Logstash Config Problem](https://discuss.elastic.co/t/logstash-config-problem/98522)
**Author:** [@avik\_ewu](https://discuss.elastic.co/u/avik_ewu)\ **Replies:** 2\ **Last updated:** [August 28, 2017, 9:25am UTC](https://discuss.elastic.co/t/logstash-config-problem/98522 "2017-08-28T09:25:34Z")
I have large dataset useing csv filter and found config error, can somenone help me to identify the problem. input { file { path =\> "/home/observer/COLLECTION/3PAR/VLUN/statvlun\_10seconds.csv" start\_position=\> “begin… --- ## [\[Solved\] Wrong time zone modification](https://discuss.elastic.co/t/solved-wrong-time-zone-modification/98372)
**Author:** [@Napsty](https://discuss.elastic.co/u/Napsty)\ **Replies:** 0\ **Last updated:** [August 25, 2017, 1:33pm UTC](https://discuss.elastic.co/t/solved-wrong-time-zone-modification/98372 "2017-08-25T13:33:32Z")
I have a strange case where a Docker container (system time runs in timezone UTC) sends HAProxy logs to a remote syslog server and from there into logstash. On the remote syslog server the log arrives: Aug 25 11:38:00 … --- ## [Logstash is not processing incoming events](https://discuss.elastic.co/t/logstash-is-not-processing-incoming-events/98516)
**Author:** [@jakubhajek](https://discuss.elastic.co/u/jakubhajek)\ **Replies:** 0\ **Last updated:** [August 28, 2017, 7:53am UTC](https://discuss.elastic.co/t/logstash-is-not-processing-incoming-events/98516 "2017-08-28T07:53:59Z")
Hello, We are using Logstash 5.2.0 connected with Elasticsearch. The application is sending the logs in XML via HTTP input format which than is parsed by XML filter to JSON document. The Logstash is configured to use de… --- ## [Reading from multiple files using wildcard](https://discuss.elastic.co/t/reading-from-multiple-files-using-wildcard/98439)
**Author:** [@daaakeee](https://discuss.elastic.co/u/daaakeee)\ **Replies:** 1\ **Last updated:** [August 28, 2017, 7:49am UTC](https://discuss.elastic.co/t/reading-from-multiple-files-using-wildcard/98439 "2017-08-28T07:49:04Z")
Hi! I want to read from multiple log files using file input plugin For ex, input { file { path =\> \["/var/log/search.log\*"\] type =\> "red\_searchlog" sincedb\_path =\> "/dev/null" start\_position =\> "beginning" } } … --- ## [How do we send output from logstash to cassandra instead of elasticsearch?](https://discuss.elastic.co/t/how-do-we-send-output-from-logstash-to-cassandra-instead-of-elasticsearch/98513)
**Author:** [@Joe\_Yang](https://discuss.elastic.co/u/Joe_Yang)\ **Replies:** 1\ **Last updated:** [August 28, 2017, 7:48am UTC](https://discuss.elastic.co/t/how-do-we-send-output-from-logstash-to-cassandra-instead-of-elasticsearch/98513 "2017-08-28T07:48:42Z")
we have configured the output from logstash to elasticsearch, however, we’d like to configure the output from logstash to cassandra for other purpose, any advice would be appreciated. --- ## [Can someone please help me with Geo filter on said logstash conf](https://discuss.elastic.co/t/can-someone-please-help-me-with-geo-filter-on-said-logstash-conf/98366)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 8\ **Last updated:** [August 28, 2017, 7:12am UTC](https://discuss.elastic.co/t/can-someone-please-help-me-with-geo-filter-on-said-logstash-conf/98366 "2017-08-28T07:12:24Z")
Hi Guys, I am not expert in logstash and I need help regarding indexing my opencanary data with geo tags. I already have below conf but I am sure it does not have geo tag. Can someone please help me with appropriate co… --- ## [Persistent Queue Error after hang up](https://discuss.elastic.co/t/persistent-queue-error-after-hang-up/95143)
**Author:** [@Shaoranlaos](https://discuss.elastic.co/u/Shaoranlaos)\ **Replies:** 4\ **Last updated:** [August 28, 2017, 7:03am UTC](https://discuss.elastic.co/t/persistent-queue-error-after-hang-up/95143 "2017-08-28T07:03:16Z")
Our Logstash has hung up(not terminated) on the 29.7. 13:55 so that no events where processed. Curiosly on the 30.7. 14:10 it has started again but threw only Errors(see under config decription) until i restarted it on … --- ## [Logstash aggregate filter on auditd logs](https://discuss.elastic.co/t/logstash-aggregate-filter-on-auditd-logs/98480)
**Author:** [@Nadavr](https://discuss.elastic.co/u/Nadavr)\ **Replies:** 1\ **Last updated:** [August 27, 2017, 4:50pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-on-auditd-logs/98480 "2017-08-27T16:50:48Z")
Hi Everyone, I’m trying to parse Linux audit.log files with logstash. Every execution of shell command (in this example: tail /etc/hosts) generates 5 different event types, relating to the same action, having the sam… --- ## [Logstash get data but Kibana can't show it!](https://discuss.elastic.co/t/logstash-get-data-but-kibana-cant-show-it/98006)
**Author:** [@hack3rcon](https://discuss.elastic.co/u/hack3rcon)\ **Replies:** 21\ **Last updated:** [August 27, 2017, 7:52am UTC](https://discuss.elastic.co/t/logstash-get-data-but-kibana-cant-show-it/98006 "2017-08-27T07:52:34Z")
Hello. I'm using "Winlogbeat for sending Windows event Log to my ELK. Logstash get data: $ curl -XGET 'http://localhost:9200/\_cat/indices?v' yellow open beat-2017.08.22 Ez89QEvaR8207QD06iY3jw 5 1 … --- ## [Remove some fields about logstash](https://discuss.elastic.co/t/remove-some-fields-about-logstash/98454)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 1\ **Last updated:** [August 27, 2017, 6:40am UTC](https://discuss.elastic.co/t/remove-some-fields-about-logstash/98454 "2017-08-27T06:40:04Z")
Hi everyone, I add three fields: day,month,year to adjust my index time. But I want to remove these three fields. How can I do . input { stdin { } } filter { grok { match =\> { "message" =\> '%{SYSLOG… --- ## [Redis output plugin "Redis::TimeoutError" never recovers](https://discuss.elastic.co/t/redis-output-plugin-redis-timeouterror-never-recovers/98462)
**Author:** [@nikhilo](https://discuss.elastic.co/u/nikhilo)\ **Replies:** 0\ **Last updated:** [August 27, 2017, 12:25am UTC](https://discuss.elastic.co/t/redis-output-plugin-redis-timeouterror-never-recovers/98462 "2017-08-27T00:25:40Z")
We have been trying to introduce redis in our logstash pipeline in order to spread the parsing load evenly among multiple logstash servers. Note: Tried this approach with logstash 1.4.2 and 5.1.1 Unfortunately some of … --- ## [Using Logstash 5.5 for reading .gz files](https://discuss.elastic.co/t/using-logstash-5-5-for-reading-gz-files/97947)
**Author:** [@shashwat](https://discuss.elastic.co/u/shashwat)\ **Replies:** 3\ **Last updated:** [August 26, 2017, 3:10pm UTC](https://discuss.elastic.co/t/using-logstash-5-5-for-reading-gz-files/97947 "2017-08-26T15:10:16Z")
Hi, I am trying to use logstash 5.5 for analyzing archived (.gz) files generating every minute. Each.gz file contains csv file in it. My .conf file looks like below: input { file { type =\> "gzip" path =\> \[ “C:\\data\*… --- ## [Dead letter queue](https://discuss.elastic.co/t/dead-letter-queue/98376)
**Author:** [@CeeDee](https://discuss.elastic.co/u/CeeDee)\ **Replies:** 1\ **Last updated:** [August 26, 2017, 12:34pm UTC](https://discuss.elastic.co/t/dead-letter-queue/98376 "2017-08-26T12:34:54Z")
Hi, I am trying to add some additional redundancy to my RELK environment with the dead letter queue in case the connection between logstash & elasticsearch is down. However, it seems that the dead letter queue is not w… --- ## [@timestamp and \_index question](https://discuss.elastic.co/t/timestamp-and--index-question/98448)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 6\ **Last updated:** [August 26, 2017, 10:50am UTC](https://discuss.elastic.co/t/timestamp-and--index-question/98448 "2017-08-26T10:50:17Z")
Hi everyone, I store syslog into elasticsearch and overview data on Kibana. I found that the date of @timestamp and \_index doesn’t match. The \_index is default logstash-\* and it seems would automatically adjust from @… --- ## [Analyse file in specific order](https://discuss.elastic.co/t/analyse-file-in-specific-order/98384)
**Author:** [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\ **Replies:** 2\ **Last updated:** [August 26, 2017, 10:49am UTC](https://discuss.elastic.co/t/analyse-file-in-specific-order/98384 "2017-08-26T10:49:28Z")
Do you know if with the path I can analyse file in order they were created? input { file { path =\> "/home/data/public\_html/datas/update/\*.csv" start\_position =\> beginning sincedb\_path =\> "/home/dat… --- ## [Check if logstash running](https://discuss.elastic.co/t/check-if-logstash-running/98383)
**Author:** [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\ **Replies:** 2\ **Last updated:** [August 26, 2017, 10:47am UTC](https://discuss.elastic.co/t/check-if-logstash-running/98383 "2017-08-26T10:47:33Z")
I need to be sure that logstash is running and my logstash process too. Do you know if I can build a cron (or something like that) in order to verify the well running? Thanks a lot --- ## [\[ERROR\]\[logstash.plugins.registry\] Problems loading a plugin with {:type=\>“filter”, :name=\>“CSV”, :path=\>“logstash/filters/CSV”, :error \_message=\>“NameError”](https://discuss.elastic.co/t/error-logstash-plugins-registry-problems-loading-a-plugin-with-type-filter-name-csv-path-logstash-filters-csv-error--message-nameerror/97974)
**Author:** [@vikaskataria](https://discuss.elastic.co/u/vikaskataria)\ **Replies:** 4\ **Last updated:** [August 26, 2017, 9:15am UTC](https://discuss.elastic.co/t/error-logstash-plugins-registry-problems-loading-a-plugin-with-type-filter-name-csv-path-logstash-filters-csv-error--message-nameerror/97974 "2017-08-26T09:15:45Z")
Hi, I am new to ELK stack. I am using Logstash 5.5.2 and ElasticSearch 5.5.2 and Kibana 5.5.2. I am trying to push data from a CSV file into Elastic search using Logstash. I am getting Syntax Error: Unexpected tLT from … --- ## [Clear Logstash cache](https://discuss.elastic.co/t/clear-logstash-cache/98440)
**Author:** [@hack3rcon](https://discuss.elastic.co/u/hack3rcon)\ **Replies:** 2\ **Last updated:** [August 26, 2017, 7:09am UTC](https://discuss.elastic.co/t/clear-logstash-cache/98440 "2017-08-26T07:09:18Z")
Hello. How can I clear Logstash cache? I got new data from Windows event log but I just see old Logs !!! I used below commands too but I guess they clear “Elastic” cache: $ curl -XPOST 'http://localhost:9200/\_cache/cl… --- ## [Configure "Winlogbeat" for Logstash](https://discuss.elastic.co/t/configure-winlogbeat-for-logstash/97870)
**Author:** [@hack3rcon](https://discuss.elastic.co/u/hack3rcon)\ **Replies:** 4\ **Last updated:** [August 26, 2017, 6:53am UTC](https://discuss.elastic.co/t/configure-winlogbeat-for-logstash/97870 "2017-08-26T06:53:06Z")
Hello. I installed “Logstash”, “Elasticsearch” and “Kibana” on my Linux Box. I install Winlogbeat on my Windows server and I like to forward Windows Event Log to my Linux Box. my Winlogbeat configuration is : #========… --- ## [Offline install gzip\_lines plugin not working on Linux](https://discuss.elastic.co/t/offline-install-gzip-lines-plugin-not-working-on-linux/98400)
**Author:** [@shashwat](https://discuss.elastic.co/u/shashwat)\ **Replies:** 2\ **Last updated:** [August 26, 2017, 6:03am UTC](https://discuss.elastic.co/t/offline-install-gzip-lines-plugin-not-working-on-linux/98400 "2017-08-26T06:03:52Z")
Hi, I am trying with logstash 5.5.1. When I run following command on my Linux machine(staging server) to prepare for offline install of this plugin, it fails. ./logstash-plugin prepare-offline-pack --output ./plugins\_g… --- ## [Apache logs not able to map geo IPs](https://discuss.elastic.co/t/apache-logs-not-able-to-map-geo-ips/98429)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 2\ **Last updated:** [August 26, 2017, 4:55am UTC](https://discuss.elastic.co/t/apache-logs-not-able-to-map-geo-ips/98429 "2017-08-26T04:55:40Z")
Hi Guys, I am trying to plot the geo for my apache logs but somehow I am getting below error on kibana, can someone please help me on this error? No Compatible Fields: The “apache-\*” index pattern does not contain any … --- ## [Logstash Flush pipeline to output](https://discuss.elastic.co/t/logstash-flush-pipeline-to-output/97837)
**Author:** [@arunvijayam](https://discuss.elastic.co/u/arunvijayam)\ **Replies:** 3\ **Last updated:** [August 26, 2017, 2:48am UTC](https://discuss.elastic.co/t/logstash-flush-pipeline-to-output/97837 "2017-08-26T02:48:39Z")
Hi All, I am using logstash 5.4.2 Persistence Queue. Where I have config file to get input throw JDBC and do some transformations and storing output to mongo db. But when I run this in logstash it inserts only few re… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=917) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=919)