# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=343 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 344 --- ## [How to grok different format record lines in one single log file](https://discuss.elastic.co/t/how-to-grok-different-format-record-lines-in-one-single-log-file/236797)
**Author:** [@Eaven](https://discuss.elastic.co/u/Eaven)\ **Replies:** 4\ **Last updated:** [June 12, 2020, 3:02pm UTC](https://discuss.elastic.co/t/how-to-grok-different-format-record-lines-in-one-single-log-file/236797 "2020-06-12T15:02:29Z")
Dear Friend, I'm encountering an issue when configuring logstash, here is an example from my case that I need to grok filter to match - some lines are with IP fields and sesseionID fields but some are not (so they are n… --- ## [Restore rabbit message when elastic reject them](https://discuss.elastic.co/t/restore-rabbit-message-when-elastic-reject-them/236831)
**Author:** [@jamorales](https://discuss.elastic.co/u/jamorales)\ **Replies:** 1\ **Last updated:** [June 12, 2020, 2:53pm UTC](https://discuss.elastic.co/t/restore-rabbit-message-when-elastic-reject-them/236831 "2020-06-12T14:53:26Z")
Hello every one, i have the next scenario: In my logstash pipeline, in input I have configured the rabbitmq , so we take messages from Rabbitmq and then we send them to elasticsearch. But sometimes the message is reje… --- ## [Multiple logstash output configurations](https://discuss.elastic.co/t/multiple-logstash-output-configurations/236840)
**Author:** [@OatmealOgre](https://discuss.elastic.co/u/OatmealOgre)\ **Replies:** 1\ **Last updated:** [June 12, 2020, 2:46pm UTC](https://discuss.elastic.co/t/multiple-logstash-output-configurations/236840 "2020-06-12T14:46:33Z")
I've been trying to find information on this and started to wonder if it's even possible. Situation is as follows. We have several logstash pipeline configuration files. All of them have their output largely the same, … --- ## [Plugin cipher (Deterministic encryption + iv\_random\_length)](https://discuss.elastic.co/t/plugin-cipher-deterministic-encryption-iv-random-length/236856)
**Author:** [@Florent\_Fauvin](https://discuss.elastic.co/u/Florent_Fauvin)\ **Replies:** 2\ **Last updated:** [June 12, 2020, 2:33pm UTC](https://discuss.elastic.co/t/plugin-cipher-deterministic-encryption-iv-random-length/236856 "2020-06-12T14:33:58Z")
Hello, I would like to do "deterministic encryption" with the cipher plugin, which requires not to have a "random iv". Unfortunately, even if the official documentation says that the "iv\_random\_length" parameter is not… --- ## [Is it available logstash "environment" filter plugin in Windows?](https://discuss.elastic.co/t/is-it-available-logstash-environment-filter-plugin-in-windows/236897)
**Author:** [@eizquierdo](https://discuss.elastic.co/u/eizquierdo)\ **Replies:** 2\ **Last updated:** [June 12, 2020, 2:27pm UTC](https://discuss.elastic.co/t/is-it-available-logstash-environment-filter-plugin-in-windows/236897 "2020-06-12T14:27:10Z")
I have this simple configuration: input { stdin { } } filter { environment { add\_metadata\_from\_env =\> {"test" =\> "TEST\_ENV"} add\_field =\> \["my\_test", "%{\[@metadata\]\[test\]}"\] } } output { stdout { codec =\> ru… --- ## [Getting syslog messages into Elastic](https://discuss.elastic.co/t/getting-syslog-messages-into-elastic/236489)
**Author:** [@pwallace10](https://discuss.elastic.co/u/pwallace10)\ **Replies:** 2\ **Last updated:** [June 12, 2020, 12:35pm UTC](https://discuss.elastic.co/t/getting-syslog-messages-into-elastic/236489 "2020-06-12T12:35:59Z")
Hi, I hope someone can point me in a direction, I have been slogging away for weeks now, trying every tutorial I can find.. I want to get syslog messages from my ASA firewalls into Elastic. I have redone the setup using… --- ## [Publish same Log file logs to elastic with different index](https://discuss.elastic.co/t/publish-same-log-file-logs-to-elastic-with-different-index/236511)
**Author:** [@Vinutha\_A\_H](https://discuss.elastic.co/u/Vinutha_A_H)\ **Replies:** 4\ **Last updated:** [June 12, 2020, 11:40am UTC](https://discuss.elastic.co/t/publish-same-log-file-logs-to-elastic-with-different-index/236511 "2020-06-12T11:40:41Z")
Hi, We are trying to read a log file twice. Sending one copy as it is (just mapping the key to each field in logs using grok) to kafka/elastic with index xyz and second copy by removing some fields (we are also doing ke… --- ## [Logstash 7.7.0 installation failed with openjdk 11.0.2 + RHEL7.8](https://discuss.elastic.co/t/logstash-7-7-0-installation-failed-with-openjdk-11-0-2-rhel7-8/236826)
**Author:** [@bhavesh.vasoya](https://discuss.elastic.co/u/bhavesh.vasoya)\ **Replies:** 1\ **Last updated:** [June 12, 2020, 9:12am UTC](https://discuss.elastic.co/t/logstash-7-7-0-installation-failed-with-openjdk-11-0-2-rhel7-8/236826 "2020-06-12T09:12:14Z")
\[root@elk ~\]$ java --version openjdk 11.0.2 2019-01-15 OpenJDK Runtime Environment 18.9 (build 11.0.2+9) OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode) \[root@elk ~\]$ \[root@elk ~\]$ yum install logstash Loaded … --- ## [How to use Multiple if statements in filter section of logstash configuration file](https://discuss.elastic.co/t/how-to-use-multiple-if-statements-in-filter-section-of-logstash-configuration-file/234942)
**Author:** [@Matish\_Bhuyan](https://discuss.elastic.co/u/Matish_Bhuyan)\ **Replies:** 12\ **Last updated:** [June 12, 2020, 7:45am UTC](https://discuss.elastic.co/t/how-to-use-multiple-if-statements-in-filter-section-of-logstash-configuration-file/234942 "2020-06-12T07:45:37Z")
Hi All, I am a newbie to the elk. I am currently using logstash version 7.2. My issue here is to update and new values to the keywords, Below is my logstash configuration file input { http { … --- ## [Logstash-input{ stdin{} output{ stdout{} }](https://discuss.elastic.co/t/logstash-input-stdin-output-stdout/236642)
**Author:** [@111351](https://discuss.elastic.co/u/111351)\ **Replies:** 2\ **Last updated:** [June 12, 2020, 6:59am UTC](https://discuss.elastic.co/t/logstash-input-stdin-output-stdout/236642 "2020-06-12T06:59:36Z")
run logstash -e "input{ stdin{} output{ stdout{} }" show error : Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one… --- ## [Logstash not starting giving error when I run the command logstash -f logstash.conf in my command prompt](https://discuss.elastic.co/t/logstash-not-starting-giving-error-when-i-run-the-command-logstash-f-logstash-conf-in-my-command-prompt/236809)
**Author:** [@TUSHAR\_Sinha](https://discuss.elastic.co/u/TUSHAR_Sinha)\ **Replies:** 0\ **Last updated:** [June 12, 2020, 4:10am UTC](https://discuss.elastic.co/t/logstash-not-starting-giving-error-when-i-run-the-command-logstash-f-logstash-conf-in-my-command-prompt/236809 "2020-06-12T04:10:34Z")
C:\\elastic\_stack\\logstash-7.7.1\\logstash-7.7.1\\bin\>logstash.bat -f logstash.conf Exception in thread "main" java.lang.ExceptionInInitializerError at org.apache.logging.log4j.core.util.WatchManager.(WatchManager.java:53… --- ## [Cannot start logstash in command line](https://discuss.elastic.co/t/cannot-start-logstash-in-command-line/236804)
**Author:** [@azteker](https://discuss.elastic.co/u/azteker)\ **Replies:** 1\ **Last updated:** [June 12, 2020, 2:07am UTC](https://discuss.elastic.co/t/cannot-start-logstash-in-command-line/236804 "2020-06-12T02:07:06Z")
I got "Logstash could not be started because there is already another instance using the configured data directory" "systemctl status logstash" shows "Active: failed". Why I still got this message? Here is the message … --- ## [Logstash Errors mapper\_parsing\_exception vs illegal\_argument\_exception](https://discuss.elastic.co/t/logstash-errors-mapper-parsing-exception-vs-illegal-argument-exception/236783)
**Author:** [@akrzos](https://discuss.elastic.co/u/akrzos)\ **Replies:** 2\ **Last updated:** [June 11, 2020, 10:58pm UTC](https://discuss.elastic.co/t/logstash-errors-mapper-parsing-exception-vs-illegal-argument-exception/236783 "2020-06-11T22:58:23Z")
I am trying to understand the difference between these two errors: \[2020-06-05T13:51:34,267\]\[WARN \]\[logstash.outputs.elasticsearch\]\[filebeat\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:… --- ## [Logstash Crashes... Minimal Config, Default Pipeline!](https://discuss.elastic.co/t/logstash-crashes-minimal-config-default-pipeline/236766)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 4\ **Last updated:** [June 11, 2020, 10:41pm UTC](https://discuss.elastic.co/t/logstash-crashes-minimal-config-default-pipeline/236766 "2020-06-11T22:41:45Z")
Hi Logstash gurus, I have been running the Docker container instance of Logstash (ver 7.4.0, yes, I need to upgrade) for a while now. I had to tear down and rebuild the container, and I decided to take the opportunity … --- ## [Create new field from existing one](https://discuss.elastic.co/t/create-new-field-from-existing-one/236778)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 1\ **Last updated:** [June 11, 2020, 9:44pm UTC](https://discuss.elastic.co/t/create-new-field-from-existing-one/236778 "2020-06-11T21:44:53Z")
Create new field with proper url content. this path are sometime 10 level deep and sometime 20. this is what I have full\_file\_name: /home/sachin/Tests/noise/spread/xspread/xspread\_a\_10\_v1.tl.gz project: sachin file\_n… --- ## [Runout error when parsing with logstash](https://discuss.elastic.co/t/runout-error-when-parsing-with-logstash/236752)
**Author:** [@nb03briceno](https://discuss.elastic.co/u/nb03briceno)\ **Replies:** 1\ **Last updated:** [June 11, 2020, 8:38pm UTC](https://discuss.elastic.co/t/runout-error-when-parsing-with-logstash/236752 "2020-06-11T20:38:37Z")
Hello everyone, I'm trying to use logstash in order to parse some txt files. When doing it, one of the lines read from the txt file is really big, causing a problem in the grok parsing. A line of the txt file I'm talkin… --- ## [Cipher plugin binary key](https://discuss.elastic.co/t/cipher-plugin-binary-key/236765)
**Author:** [@JamKage](https://discuss.elastic.co/u/JamKage)\ **Replies:** 0\ **Last updated:** [June 11, 2020, 5:40pm UTC](https://discuss.elastic.co/t/cipher-plugin-binary-key/236765 "2020-06-11T17:40:40Z")
Hi all, I am trying to use the cipher plugin with a key that is not a string, but is a sequence of bytes. The key has been created using the following Java snippet: public byte\[\] getKey(String keyString) throws NoS… --- ## [Getting 403 error connecting Logstash to ElasticSearch](https://discuss.elastic.co/t/getting-403-error-connecting-logstash-to-elasticsearch/236750)
**Author:** [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Replies:** 1\ **Last updated:** [June 11, 2020, 4:47pm UTC](https://discuss.elastic.co/t/getting-403-error-connecting-logstash-to-elasticsearch/236750 "2020-06-11T16:47:58Z")
Hello. I'm attempting to connect Logstash to ElasticSearch in a secure manner for the first time. I'm following the instructions here: https://www.elastic.co/guide/en/logstash/current/ls-security.html I keep getting … --- ## [Migrating logs sources from Elasticsearch to Logstash](https://discuss.elastic.co/t/migrating-logs-sources-from-elasticsearch-to-logstash/236731)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 1\ **Last updated:** [June 11, 2020, 3:37pm UTC](https://discuss.elastic.co/t/migrating-logs-sources-from-elasticsearch-to-logstash/236731 "2020-06-11T15:37:05Z")
I have 12 log sources running Packetbeat, WinLogbeat while some additionally run Filebeat & Metricbeat. They currently feed directly into a single Elasticsearch instance that is running processor at ~95% I am thinkin… --- ## [Plugin cipher & Value of a field](https://discuss.elastic.co/t/plugin-cipher-value-of-a-field/236709)
**Author:** [@Florent\_Fauvin](https://discuss.elastic.co/u/Florent_Fauvin)\ **Replies:** 6\ **Last updated:** [June 11, 2020, 3:13pm UTC](https://discuss.elastic.co/t/plugin-cipher-value-of-a-field/236709 "2020-06-11T15:13:40Z")
Hello, I am trying to use the value of a field, but I do not know why, I have the impression that the cipher plugin cannot read it. Could you help me figure out what's wrong please? Below, I stored the value of "algori… --- ## [How create fields using json fileter in logstash](https://discuss.elastic.co/t/how-create-fields-using-json-fileter-in-logstash/236557)
**Author:** [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\ **Replies:** 10\ **Last updated:** [June 11, 2020, 3:06pm UTC](https://discuss.elastic.co/t/how-create-fields-using-json-fileter-in-logstash/236557 "2020-06-11T15:06:48Z")
Hi I am getting json formated data I can able to split the data into fields using json plugin in logstash,but with in the fields i am some more fields.Please help me out to get those fields also. Below is the example f… --- ## [Migrating packetbeat ingestion from Elasticsearch to Logstash](https://discuss.elastic.co/t/migrating-packetbeat-ingestion-from-elasticsearch-to-logstash/236215)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 9\ **Last updated:** [June 11, 2020, 3:04pm UTC](https://discuss.elastic.co/t/migrating-packetbeat-ingestion-from-elasticsearch-to-logstash/236215 "2020-06-11T15:04:14Z")
Hello, I currently push packetbeat logs from hosts directly to elasticsearch with a geo-pipeline. I want to migrate this compute load to a dedicated logstash pipeline. Stack is running at 7.7.1. I need help with: Geo… --- ## [Logstash throwing error after enabling the xpack security](https://discuss.elastic.co/t/logstash-throwing-error-after-enabling-the-xpack-security/236256)
**Author:** [@mkbmanas87](https://discuss.elastic.co/u/mkbmanas87)\ **Replies:** 2\ **Last updated:** [June 11, 2020, 2:16pm UTC](https://discuss.elastic.co/t/logstash-throwing-error-after-enabling-the-xpack-security/236256 "2020-06-11T14:16:41Z")
Hi Team after enabling the xpack to create user in elasticsearch. The logstash is throwing error. Error: logstash\[9213\]\[WARN \]\[logstash.outputs.elasticsearch\] Attempted to resurrect connection to dead ES instance, but… --- ## [Deadletter queue](https://discuss.elastic.co/t/deadletter-queue/236684)
**Author:** [@Pankaj\_Jadhav](https://discuss.elastic.co/u/Pankaj_Jadhav)\ **Replies:** 1\ **Last updated:** [June 11, 2020, 2:15pm UTC](https://discuss.elastic.co/t/deadletter-queue/236684 "2020-06-11T14:15:19Z")
Hi have to add event,metadata in dead letter queue when will get any exception in http filter. am using pipelines.yml as follows ================================================================= - pipeline.id: pipelin… --- ## [Replacing empty string with dynamic value from the same field](https://discuss.elastic.co/t/replacing-empty-string-with-dynamic-value-from-the-same-field/236686)
**Author:** [@guas5](https://discuss.elastic.co/u/guas5)\ **Replies:** 2\ **Last updated:** [June 11, 2020, 12:39pm UTC](https://discuss.elastic.co/t/replacing-empty-string-with-dynamic-value-from-the-same-field/236686 "2020-06-11T12:39:07Z")
Hi everyone, I would be grateful for your help regarding how to replace an empty string in a field witha dynamic value. I've already tried different filters such as replace, copy, etc. but it doesn't work as needed. Her… --- ## [A field referenced by another field](https://discuss.elastic.co/t/a-field-referenced-by-another-field/236668)
**Author:** [@Florent\_Fauvin](https://discuss.elastic.co/u/Florent_Fauvin)\ **Replies:** 2\ **Last updated:** [June 11, 2020, 12:56pm UTC](https://discuss.elastic.co/t/a-field-referenced-by-another-field/236668 "2020-06-11T12:56:32Z")
Hello, I would like to copy the value of a 1st field, into a 2nd field which is referenced by a 3rd field. Example: field1 = "OK" field2 = "" field3 = "field2" How to copy the value of field1 to field2 using field3… --- ## [Using %{MONTHDAY:MONTHDAY}-%{MONTH:MONTH}-%{YEAR:YEAR} %{HOUR:HOUR}:%{MINUTE:MINUTE}](https://discuss.elastic.co/t/using-monthday-monthday-month-month-year-year-hour-hour-minute-minute/236658)
**Author:** [@Burga](https://discuss.elastic.co/u/Burga)\ **Replies:** 1\ **Last updated:** [June 11, 2020, 11:46am UTC](https://discuss.elastic.co/t/using-monthday-monthday-month-month-year-year-hour-hour-minute-minute/236658 "2020-06-11T11:46:07Z")
Hi , in my grok I'm parsing date column : match =\> { "message" =\> "%{MONTHDAY:MONTHDAY}-%{MONTH:MONTH}-%{YEAR:YEAR} %{HOUR:HOUR}:%{MINUTE:MINUTE}:%{SECOND:SECOND}%{SPACE} %{DATA:loglevel}%{SPACE} \\\[%{DATA:thread}\\\]%{GR… --- ## [Xml attribute PARSING IN LOGSTASH](https://discuss.elastic.co/t/xml-attribute-parsing-in-logstash/236307)
**Author:** [@Puneeth\_S\_B\_Gowda1](https://discuss.elastic.co/u/Puneeth_S_B_Gowda1)\ **Replies:** 1\ **Last updated:** [June 11, 2020, 8:51am UTC](https://discuss.elastic.co/t/xml-attribute-parsing-in-logstash/236307 "2020-06-11T08:51:58Z")
Hi I am looking for xml attribute parsing through Logstash + file beat This is my xml \\M1S\\ from above one i need M2S attribute value as new colum… --- ## [File\_completed\_action not deleting files](https://discuss.elastic.co/t/file-completed-action-not-deleting-files/236424)
**Author:** [@zfmoto](https://discuss.elastic.co/u/zfmoto)\ **Replies:** 0\ **Last updated:** [June 9, 2020, 10:34pm UTC](https://discuss.elastic.co/t/file-completed-action-not-deleting-files/236424 "2020-06-09T22:34:11Z")
I enabled file\_completed\_action to delete, however the files still exist & are not logged after parsing. I've tried enabling permissions for these files as well to allow write. Here's a view of my config + permissions. i… --- ## [Join two log files in logstash conf](https://discuss.elastic.co/t/join-two-log-files-in-logstash-conf/236306)
**Author:** [@Rushabh\_Doshi](https://discuss.elastic.co/u/Rushabh_Doshi)\ **Replies:** 15\ **Last updated:** [June 10, 2020, 8:57pm UTC](https://discuss.elastic.co/t/join-two-log-files-in-logstash-conf/236306 "2020-06-10T20:57:21Z")
i have an activity log like: 2020-06-02 03:30:50,712 USERID="(not logged in)" IP="103.226.146.203" SESSION="" HOST="h2t57-hrm.gh.pri" HOMEDIR="8.57" SITE="ps" METHOD="GET" HTTP\_USER\_AGENT="Mozilla/5.0 (Macintosh; Intel … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=342) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=344)