# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=504 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 505 --- ## [Issue with regex pattern for prune filter plugin blacklist\_names](https://discuss.elastic.co/t/issue-with-regex-pattern-for-prune-filter-plugin-blacklist-names/191834)
**Author:** [@sai\_kiran1](https://discuss.elastic.co/u/sai_kiran1)\ **Replies:** 4\ **Last updated:** [July 25, 2019, 4:07pm UTC](https://discuss.elastic.co/t/issue-with-regex-pattern-for-prune-filter-plugin-blacklist-names/191834 "2019-07-25T16:07:30Z")
Hi There! I am trying to ignore the fields a4, a5, a6....., a1100 from my audit logs and tried applying prune filter plugin with setting blacklist\_names, but I see the fields a4, a5, a6..... are still incoming. Can som… --- ## [Timestamp changed by itself his way of working](https://discuss.elastic.co/t/timestamp-changed-by-itself-his-way-of-working/192270)
**Author:** [@patimothe](https://discuss.elastic.co/u/patimothe)\ **Replies:** 0\ **Last updated:** [July 25, 2019, 2:40pm UTC](https://discuss.elastic.co/t/timestamp-changed-by-itself-his-way-of-working/192270 "2019-07-25T14:40:22Z")
Hello, I have a weird problem: my pipelines file all work the same way about date: I get the date of the event, then I use the date plugin this way: date { match =\> \[ "date", "ISO8601" \] timezone =\> "Europe/Paris" … --- ## [CSV data to Elasticsearch using Logstash](https://discuss.elastic.co/t/csv-data-to-elasticsearch-using-logstash/192055)
**Author:** [@Fragan](https://discuss.elastic.co/u/Fragan)\ **Replies:** 8\ **Last updated:** [July 25, 2019, 1:59pm UTC](https://discuss.elastic.co/t/csv-data-to-elasticsearch-using-logstash/192055 "2019-07-25T13:59:27Z")
Hello, im trying to push csv data to Elasticsearch using Logstash, here's my logstash conf file : input{ file{ path =\> "C:\\Users\\zk3i\\Desktop\\ELK\\kpassdemo\_vue\_qsse.CSV" start\_position =\> "beginning" sincedb\_path… --- ## [How can i run data collect automatically without using command "logstrash -f" after restarting OS](https://discuss.elastic.co/t/how-can-i-run-data-collect-automatically-without-using-command-logstrash-f-after-restarting-os/192003)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 7\ **Last updated:** [July 25, 2019, 1:54pm UTC](https://discuss.elastic.co/t/how-can-i-run-data-collect-automatically-without-using-command-logstrash-f-after-restarting-os/192003 "2019-07-25T13:54:44Z")
i have many conf file to collect data from different sources, and after a shutdown or restart of the machien i am obliged to run the command "logstash -f test.conf", I want to automate this task and launch it automatical… --- ## [Syntax error with filter](https://discuss.elastic.co/t/syntax-error-with-filter/192162)
**Author:** [@mcoa](https://discuss.elastic.co/u/mcoa)\ **Replies:** 3\ **Last updated:** [July 25, 2019, 1:50pm UTC](https://discuss.elastic.co/t/syntax-error-with-filter/192162 "2019-07-25T13:50:10Z")
Hello, I'm use logstash for parser some postgress log's. But my filter section says syntax error: \[2019-07-25T00:08:47,536\]\[INFO \]\[logstash.runner \] Starting Logstash {"logstash.version"=\>"6.8.1"} \[2019-07-25T… --- ## [Spliting the field](https://discuss.elastic.co/t/spliting-the-field/192191)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 1\ **Last updated:** [July 25, 2019, 1:45pm UTC](https://discuss.elastic.co/t/spliting-the-field/192191 "2019-07-25T13:45:33Z")
Hi There, I have a field called SE\_Payload.SE\_SysStatsUtilizationDiskSpace and it gives me lot of useful information, but I dont know to how to split these in to different values like SE\_Payload.SE\_SysStatsUtilizationD… --- ## [Multiline Pattern for Logs in Kibana](https://discuss.elastic.co/t/multiline-pattern-for-logs-in-kibana/191757)
**Author:** [@sumit-kashyap](https://discuss.elastic.co/u/sumit-kashyap)\ **Replies:** 6\ **Last updated:** [July 25, 2019, 1:27pm UTC](https://discuss.elastic.co/t/multiline-pattern-for-logs-in-kibana/191757 "2019-07-25T13:27:36Z")
Can you please help me to how I can see my logs in different lines differ by minutes and seconds. My logs are as under: Jul 22, 2019 1:23:05 PM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying we… --- ## [How to configure the monitoring pipeline?](https://discuss.elastic.co/t/how-to-configure-the-monitoring-pipeline/192251)
**Author:** [@Extinguo](https://discuss.elastic.co/u/Extinguo)\ **Replies:** 0\ **Last updated:** [July 25, 2019, 1:22pm UTC](https://discuss.elastic.co/t/how-to-configure-the-monitoring-pipeline/192251 "2019-07-25T13:22:04Z")
Hey guys, I'm currently encrypting the communication between elasticsearch, logstash and kibana (and soon filebeat...) Whenever I start Logstash with the xpack.monitoring-feature enabled I get the following warning: lo… --- ## [Elastic search not picking up changes for updated conf files](https://discuss.elastic.co/t/elastic-search-not-picking-up-changes-for-updated-conf-files/192180)
**Author:** [@ghanshyam\_Sehgal](https://discuss.elastic.co/u/ghanshyam_Sehgal)\ **Replies:** 1\ **Last updated:** [July 25, 2019, 1:13pm UTC](https://discuss.elastic.co/t/elastic-search-not-picking-up-changes-for-updated-conf-files/192180 "2019-07-25T13:13:05Z")
I had created an index in elastic search,now when i tried to update/change the conf files i.e. I changed the index name but I cannot see a new index being created.Does the elastic search picks up the changes in conf file… --- ## [How to change Data-Type in a Field to only get an Ip-Adr?](https://discuss.elastic.co/t/how-to-change-data-type-in-a-field-to-only-get-an-ip-adr/191994)
**Author:** [@Moritz\_Kiesewetter](https://discuss.elastic.co/u/Moritz_Kiesewetter)\ **Replies:** 9\ **Last updated:** [July 25, 2019, 12:40pm UTC](https://discuss.elastic.co/t/how-to-change-data-type-in-a-field-to-only-get-an-ip-adr/191994 "2019-07-25T12:40:16Z")
Hey Community, so basically i'm totally new to ES. I set up an Elk-Cluster with 2 Worker Node's, and a Master Node running Kibana etc. I attached a security system to it via logstash, so now i get the syslogs and it al… --- ## [How to kill a single pipeline and restart it in a multiple pipelined logstash](https://discuss.elastic.co/t/how-to-kill-a-single-pipeline-and-restart-it-in-a-multiple-pipelined-logstash/192031)
**Author:** [@bharatwaj](https://discuss.elastic.co/u/bharatwaj)\ **Replies:** 2\ **Last updated:** [July 25, 2019, 11:46am UTC](https://discuss.elastic.co/t/how-to-kill-a-single-pipeline-and-restart-it-in-a-multiple-pipelined-logstash/192031 "2019-07-25T11:46:03Z")
I am running a multiple pipelined logstash and wish to kill a single pipeline if it throws error and restart after fixing it. How do I do that ? --- ## [Logstash does not deliver Logs anymore](https://discuss.elastic.co/t/logstash-does-not-deliver-logs-anymore/192199)
**Author:** [@Moritz\_Kiesewetter](https://discuss.elastic.co/u/Moritz_Kiesewetter)\ **Replies:** 2\ **Last updated:** [July 25, 2019, 9:30am UTC](https://discuss.elastic.co/t/logstash-does-not-deliver-logs-anymore/192199 "2019-07-25T09:30:18Z")
Hello! This is kind of an emergancy, i tried to change my /etc/Logstash/conf.d/10-syslog....config in my ELK-Cluster. As i was nearly finished, i noticed that it would not work the way i tried it. So i rolled back ( i p… --- ## [Logstash unable to authenticate and get data from AWS kinesis stream](https://discuss.elastic.co/t/logstash-unable-to-authenticate-and-get-data-from-aws-kinesis-stream/192203)
**Author:** [@mikolaj.janczak](https://discuss.elastic.co/u/mikolaj.janczak)\ **Replies:** 0\ **Last updated:** [July 25, 2019, 8:56am UTC](https://discuss.elastic.co/t/logstash-unable-to-authenticate-and-get-data-from-aws-kinesis-stream/192203 "2019-07-25T08:56:03Z")
Hello, I have an issue with logstash. I try to configure it to get data from AWS kinesis and upload to elastic. I tested multiple cofig files, all ended with error java.lang.IllegalArgumentException: profile file canno… --- ## [Logstash put all apache log entries into one index with the current date in kibana](https://discuss.elastic.co/t/logstash-put-all-apache-log-entries-into-one-index-with-the-current-date-in-kibana/186939)
**Author:** [@rohithsolomon](https://discuss.elastic.co/u/rohithsolomon)\ **Replies:** 11\ **Last updated:** [July 25, 2019, 8:06am UTC](https://discuss.elastic.co/t/logstash-put-all-apache-log-entries-into-one-index-with-the-current-date-in-kibana/186939 "2019-07-25T08:06:46Z")
Using the latest versions of ELK. Input data: 100.43.83.137 - - \[29/May/2014:02:12:13 -0500\] "GET / HTTP/1.1" 200 37932 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 100.43.83.137 - - \[29/May/… --- ## [Primary shard is not active](https://discuss.elastic.co/t/primary-shard-is-not-active/192172)
**Author:** [@dm1987](https://discuss.elastic.co/u/dm1987)\ **Replies:** 0\ **Last updated:** [July 25, 2019, 6:06am UTC](https://discuss.elastic.co/t/primary-shard-is-not-active/192172 "2019-07-25T06:06:58Z")
\[2019-07-25T08:56:04,372\]\[INFO \]\[logstash.outputs.elasticsearch\] retrying failed action with response code: 503 ({"type"=\>"unavailable\_shards\_exception", "reason"=\>"\[winlogbeat-2019.07.24\]\[0\] primary shard is not active … --- ## [First Time User Through Wazuh, setting up Logstash with Netflow for my Cisco Network](https://discuss.elastic.co/t/first-time-user-through-wazuh-setting-up-logstash-with-netflow-for-my-cisco-network/192169)
**Author:** [@oylkm](https://discuss.elastic.co/u/oylkm)\ **Replies:** 0\ **Last updated:** [July 25, 2019, 5:43am UTC](https://discuss.elastic.co/t/first-time-user-through-wazuh-setting-up-logstash-with-netflow-for-my-cisco-network/192169 "2019-07-25T05:43:56Z")
Hi there experts/gurus, I am getting the error below when I try to setup Logstash with the Netflow module on the latest Wazuh installation. At first I noticed it was struggling for control with Nfsen (Nfsen was currently… --- ## [GROK Filter fails and not able to find out the solution](https://discuss.elastic.co/t/grok-filter-fails-and-not-able-to-find-out-the-solution/192152)
**Author:** [@premkumar](https://discuss.elastic.co/u/premkumar)\ **Replies:** 2\ **Last updated:** [July 25, 2019, 3:09am UTC](https://discuss.elastic.co/t/grok-filter-fails-and-not-able-to-find-out-the-solution/192152 "2019-07-25T03:09:50Z")
Hi Team, I'm trying to use the grok filter in the logstash configuration pipeline. The grok pattern seem to work in the kibana provided grok debugger. But using the pattern in the filter throws the error that logstash c… --- ## [Using input-file-plugin, just work after modifying the file, and logstash is not closing after indexing](https://discuss.elastic.co/t/using-input-file-plugin-just-work-after-modifying-the-file-and-logstash-is-not-closing-after-indexing/192130)
**Author:** [@Romulo\_Ramos\_de\_Siqu](https://discuss.elastic.co/u/Romulo_Ramos_de_Siqu)\ **Replies:** 6\ **Last updated:** [July 24, 2019, 10:04pm UTC](https://discuss.elastic.co/t/using-input-file-plugin-just-work-after-modifying-the-file-and-logstash-is-not-closing-after-indexing/192130 "2019-07-24T22:04:25Z")
I'm trying to use logstash to upload a json with arrays to elasticsearch, but when I run the logstash with the specified .conf file, it successfully start at port 9200, and then... nothing. It only works after I get to .… --- ## [Testing for values and adding a value when value field is empty or null](https://discuss.elastic.co/t/testing-for-values-and-adding-a-value-when-value-field-is-empty-or-null/191703)
**Author:** [@Zorkmid](https://discuss.elastic.co/u/Zorkmid)\ **Replies:** 2\ **Last updated:** [July 24, 2019, 8:49pm UTC](https://discuss.elastic.co/t/testing-for-values-and-adding-a-value-when-value-field-is-empty-or-null/191703 "2019-07-24T20:49:57Z")
Hello All, I have an upstream system that will produce event data that hopefully will be parsed by Logstash and then sent to ES. I've work out grok patterns and basic filters working with some test events. This part se… --- ## [Logstash parsing and Watcher alerting](https://discuss.elastic.co/t/logstash-parsing-and-watcher-alerting/191806)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 15\ **Last updated:** [July 24, 2019, 8:45pm UTC](https://discuss.elastic.co/t/logstash-parsing-and-watcher-alerting/191806 "2019-07-24T20:45:03Z")
Hi everyone! I have kind of these log messages: Jul 23 09:24:16 mmr mmr-core\[4268\]: Processing file \[Aweg3AOMTs\_1563866656876839.mt\], passing to thread \[5147\] Jul 23 09:24:16 mmr mmr-core\[5147\]: Aweg3AOMTs\_15638666568… --- ## [Issues procesing fields with mutate](https://discuss.elastic.co/t/issues-procesing-fields-with-mutate/192124)
**Author:** [@humartinez](https://discuss.elastic.co/u/humartinez)\ **Replies:** 2\ **Last updated:** [July 24, 2019, 8:27pm UTC](https://discuss.elastic.co/t/issues-procesing-fields-with-mutate/192124 "2019-07-24T20:27:36Z")
Hello, Im having a hard time trying to do some simple task, copy a field to another name, split the value, and add a new field using one of the splited part. (Hope to give you some background with this) In other words … --- ## [Error installing Filter TLD](https://discuss.elastic.co/t/error-installing-filter-tld/190252)
**Author:** [@tellus83](https://discuss.elastic.co/u/tellus83)\ **Replies:** 1\ **Last updated:** [July 24, 2019, 6:11pm UTC](https://discuss.elastic.co/t/error-installing-filter-tld/190252 "2019-07-24T18:11:38Z")
I'm using Logstash 7.2.0 and get error installing logstash-filter-tld ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "public\_suffix": In snapshot (Gemfile.lock): pu… --- ## [Aggregate filter bug with java execution engine](https://discuss.elastic.co/t/aggregate-filter-bug-with-java-execution-engine/187445)
**Author:** [@kibit86](https://discuss.elastic.co/u/kibit86)\ **Replies:** 3\ **Last updated:** [July 24, 2019, 5:19pm UTC](https://discuss.elastic.co/t/aggregate-filter-bug-with-java-execution-engine/187445 "2019-07-24T17:19:55Z")
Hi all, I was getting crazy trying to debug the same aggregate filter that on other logstash instances worked smoothly. In my new elk cluster, v 7.1.1, this filter faced some kind of syncronization error because someti… --- ## [Most recent CSV file from S3 gets read indefinitely](https://discuss.elastic.co/t/most-recent-csv-file-from-s3-gets-read-indefinitely/191906)
**Author:** [@resuni](https://discuss.elastic.co/u/resuni)\ **Replies:** 3\ **Last updated:** [July 24, 2019, 4:47pm UTC](https://discuss.elastic.co/t/most-recent-csv-file-from-s3-gets-read-indefinitely/191906 "2019-07-24T16:47:54Z")
I'm trying to process CSV files stored in an S3 bucket using Logstash. Everything works fine until it gets to the last file, which it creates entries in Elasticsearch for endlessly. The data is in daily time buckets, an… --- ## [Date pattern match help needed](https://discuss.elastic.co/t/date-pattern-match-help-needed/192087)
**Author:** [@arshadazad](https://discuss.elastic.co/u/arshadazad)\ **Replies:** 2\ **Last updated:** [July 24, 2019, 4:46pm UTC](https://discuss.elastic.co/t/date-pattern-match-help-needed/192087 "2019-07-24T16:46:43Z")
Hi All , I have a date format like "Jul 22, 2019 4:54:10 PM CEST" . I am trying to find a pattern for a date match but i am unable to . Can someone please help ? --- ## [Logstash-filter-memcached and unsatisfactory GET perfomance](https://discuss.elastic.co/t/logstash-filter-memcached-and-unsatisfactory-get-perfomance/192088)
**Author:** [@Timsi](https://discuss.elastic.co/u/Timsi)\ **Replies:** 0\ **Last updated:** [July 24, 2019, 4:07pm UTC](https://discuss.elastic.co/t/logstash-filter-memcached-and-unsatisfactory-get-perfomance/192088 "2019-07-24T16:07:26Z")
I am trying to integrate this filter into my pipeline to first set keys in one pipeline (about 4m key =\> value objects, each about 1kb in size), and then in another step later on get these values in another pipeline to e… --- ## [Yii2 logs to es via logstash](https://discuss.elastic.co/t/yii2-logs-to-es-via-logstash/191960)
**Author:** [@manish\_kaushik](https://discuss.elastic.co/u/manish_kaushik)\ **Replies:** 3\ **Last updated:** [July 24, 2019, 4:02pm UTC](https://discuss.elastic.co/t/yii2-logs-to-es-via-logstash/191960 "2019-07-24T16:02:42Z")
\`I have been trying to index the logs to elasticsearch and making custom grok patterns. I have created some pattern but unable to complete the rest to match it. I would like to know whether i am dong right and if there i… --- ## [Jdbc input plugin add field when connection fails](https://discuss.elastic.co/t/jdbc-input-plugin-add-field-when-connection-fails/192073)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 0\ **Last updated:** [July 24, 2019, 3:18pm UTC](https://discuss.elastic.co/t/jdbc-input-plugin-add-field-when-connection-fails/192073 "2019-07-24T15:18:33Z")
Hi All, I'm new to Elastic and i'm configuring logstash with jdbc imput to get information from oracle database, however if for any reason (listener down, db down, etc) jdbc connection fails, it does not generate any in… --- ## [Logstash crash after \[FATAL\]\[logstash.runner\] An unexpected error occurred!](https://discuss.elastic.co/t/logstash-crash-after-fatal-logstash-runner-an-unexpected-error-occurred/192069)
**Author:** [@emiliano7437](https://discuss.elastic.co/u/emiliano7437)\ **Replies:** 0\ **Last updated:** [July 24, 2019, 3:10pm UTC](https://discuss.elastic.co/t/logstash-crash-after-fatal-logstash-runner-an-unexpected-error-occurred/192069 "2019-07-24T15:10:28Z")
Hello. I´m new with ELK. I have an ELK cluster with the following: Node1: elasticsearch, logstash, kibana || 32GB RAM || -Xms12g, -Xmx12g Node2: elasticsearch, logstash || 32GB RAM || -Xms12g, -Xmx12g Node3,4,5: maste… --- ## [SNMP/SNMPTRAP input with filter and output to CSV](https://discuss.elastic.co/t/snmp-snmptrap-input-with-filter-and-output-to-csv/191905)
**Author:** [@icecotorra](https://discuss.elastic.co/u/icecotorra)\ **Replies:** 2\ **Last updated:** [July 23, 2019, 10:07pm UTC](https://discuss.elastic.co/t/snmp-snmptrap-input-with-filter-and-output-to-csv/191905 "2019-07-23T22:07:07Z")
Hello, I want to be able to grab the output of any SNMP/SNMPTRAP and structure it so it can be exported to a CSV file. I will then take the file and import it to another box running Elasticsearch and Kibana. Can you p… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=503) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=505)