# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=153 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 154 --- ## [Mutate or grok product/price that keep changing everyday](https://discuss.elastic.co/t/mutate-or-grok-product-price-that-keep-changing-everyday/300134)
**Author:** [@travlest](https://discuss.elastic.co/u/travlest)\ **Replies:** 0\ **Last updated:** [March 20, 2022, 3:02pm UTC](https://discuss.elastic.co/t/mutate-or-grok-product-price-that-keep-changing-everyday/300134 "2022-03-20T15:02:01Z")
Hi, I have a question about logstash for CSV. My situation is, I have multiple CSV files being uploaded into logstash. the log will continue to update for product or price but the domain will remain the same. my questio… --- ## [Split json fields of similar type in logstash](https://discuss.elastic.co/t/split-json-fields-of-similar-type-in-logstash/300133)
**Author:** [@zubair\_aftab](https://discuss.elastic.co/u/zubair_aftab)\ **Replies:** 1\ **Last updated:** [March 20, 2022, 1:48pm UTC](https://discuss.elastic.co/t/split-json-fields-of-similar-type-in-logstash/300133 "2022-03-20T13:48:40Z")
I have a json file converted from pcap using tshark. There are different layers inside the json and each layer has different fields. Some fields are repeating inside the same message for example "bicc\_bicc\_cic" as shown … --- ## [Logstash concatenate two field values from metricbeat](https://discuss.elastic.co/t/logstash-concatenate-two-field-values-from-metricbeat/300060)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 3\ **Last updated:** [March 18, 2022, 5:11pm UTC](https://discuss.elastic.co/t/logstash-concatenate-two-field-values-from-metricbeat/300060 "2022-03-18T17:11:10Z")
Hello All, Can someone help with this,I need to concatenate two fileds from metricbeat and create other filed with this concatenated value.Tried to use mutate filter didn't worked. filter{ mutate { add\_field =\> { "moun… --- ## [Logstash 8.1.0-1 fails to start](https://discuss.elastic.co/t/logstash-8-1-0-1-fails-to-start/300059)
**Author:** [@lmalossi](https://discuss.elastic.co/u/lmalossi)\ **Replies:** 0\ **Last updated:** [March 18, 2022, 3:13pm UTC](https://discuss.elastic.co/t/logstash-8-1-0-1-fails-to-start/300059 "2022-03-18T15:13:12Z")
Hi all, I'm trying to start logstash with systemctl but in log i get the following \[2022-03-18T16:07:32,888\]\[INFO \]\[logstash.runner \] Log4j configuration path used is: /etc/logstash/log4j2.properties \[2022-03-… --- ## [Logs appear in multi indexes](https://discuss.elastic.co/t/logs-appear-in-multi-indexes/300006)
**Author:** [@Mi\_Kmi](https://discuss.elastic.co/u/Mi_Kmi)\ **Replies:** 0\ **Last updated:** [March 18, 2022, 6:26am UTC](https://discuss.elastic.co/t/logs-appear-in-multi-indexes/300006 "2022-03-18T06:26:31Z")
Hi I am using two pipelines on different port to handle multi type logs port 5044 \> log A / index name "aaa" port 5045 \> log B / index name "bbb" And Filebeat on servers is sending log B to Logstash port 5045 on the … --- ## [Errors installing logstash-mixin-ecs\_compatibility\_support plugin](https://discuss.elastic.co/t/errors-installing-logstash-mixin-ecs-compatibility-support-plugin/299374)
**Author:** [@mikespharss](https://discuss.elastic.co/u/mikespharss)\ **Replies:** 4\ **Last updated:** [March 17, 2022, 7:28pm UTC](https://discuss.elastic.co/t/errors-installing-logstash-mixin-ecs-compatibility-support-plugin/299374 "2022-03-17T19:28:50Z")
I started having a weird problem where using logstash-plugin to install logstash-output-opensearch was leaving my plugin environment in a broken state. Logstash version: logstash 7.16.1 using bundled JDK version I sta… --- ## [Logstash blocked when using Regex filter](https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313)
**Author:** [@valm-dili13](https://discuss.elastic.co/u/valm-dili13)\ **Replies:** 2\ **Last updated:** [March 17, 2022, 5:55pm UTC](https://discuss.elastic.co/t/logstash-blocked-when-using-regex-filter/299313 "2022-03-17T17:55:45Z")
Dear community, When I add a specific filter in my pipeline, few Kb of data are processed then everything is blocked. (No more output when debugging logstash). When I check my beats logs, it says that the remote port i… --- ## [Remove field after parsing json](https://discuss.elastic.co/t/remove-field-after-parsing-json/299973)
**Author:** [@raspi](https://discuss.elastic.co/u/raspi)\ **Replies:** 2\ **Last updated:** [March 17, 2022, 5:27pm UTC](https://discuss.elastic.co/t/remove-field-after-parsing-json/299973 "2022-03-17T17:27:35Z")
Hello i need help please this is my logstash.conf \< filter { if "jaeger" in \[message\] { drop { } } mutate { gsub =\> \["message", '"message":', '"containerlogs":'\] remove\_tag =\> \[ "pfd", "middle", "fluentd", "bea… --- ## [Index data getting deleted automatically after some time,parsing oracle data from logstash to elastic](https://discuss.elastic.co/t/index-data-getting-deleted-automatically-after-some-time-parsing-oracle-data-from-logstash-to-elastic/299979)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 0\ **Last updated:** [March 17, 2022, 5:13pm UTC](https://discuss.elastic.co/t/index-data-getting-deleted-automatically-after-some-time-parsing-oracle-data-from-logstash-to-elastic/299979 "2022-03-17T17:13:50Z")
Hello All, I'm trying to get data from oracle db and parsing it through logstash pipeline and finally to elastic. while checking the data in discover in kibana ,observed that the data gets automaticaly deleted after fe… --- ## [I do not receive the logs in Logstash (Heroku)](https://discuss.elastic.co/t/i-do-not-receive-the-logs-in-logstash-heroku/299895)
**Author:** [@Diego\_T](https://discuss.elastic.co/u/Diego_T)\ **Replies:** 5\ **Last updated:** [March 17, 2022, 4:21pm UTC](https://discuss.elastic.co/t/i-do-not-receive-the-logs-in-logstash-heroku/299895 "2022-03-17T16:21:27Z")
i have open the port 1514 (tcp) in my windows (localhost) i wanna see the logs of Heroku in the Elasticsearch remote (gcp) but I do not receive any log file .conf input { tcp { port =\> "1514" tags =\> \["h… --- ## [Logstash mutate add\_field into all logs/data](https://discuss.elastic.co/t/logstash-mutate-add-field-into-all-logs-data/299918)
**Author:** [@travlest](https://discuss.elastic.co/u/travlest)\ **Replies:** 0\ **Last updated:** [March 17, 2022, 6:28am UTC](https://discuss.elastic.co/t/logstash-mutate-add-field-into-all-logs-data/299918 "2022-03-17T06:28:43Z")
Hi, I have a question regarding logstash.. Right now, I have a few different CSV files that are being shipped into logstash. The CSV file example is below. THE example File 1 url.com ,192.xxx.xxx.xxx,product-A urlA.c… --- ## [The Network Adapter could not establish the connection](https://discuss.elastic.co/t/the-network-adapter-could-not-establish-the-connection/299885)
**Author:** [@felipeneto04](https://discuss.elastic.co/u/felipeneto04)\ **Replies:** 4\ **Last updated:** [March 16, 2022, 10:58pm UTC](https://discuss.elastic.co/t/the-network-adapter-could-not-establish-the-connection/299885 "2022-03-16T22:58:36Z")
I'm new here and also to logstash. I'm having trouble trying to connect. I have several branches spread across Brazil, I get data from each one through a VPN connection, and I use a .sh scheduled in CRON to call the sc… --- ## [Pattern not defined](https://discuss.elastic.co/t/pattern-not-defined/299893)
**Author:** [@Guif](https://discuss.elastic.co/u/Guif)\ **Replies:** 1\ **Last updated:** [March 16, 2022, 7:48pm UTC](https://discuss.elastic.co/t/pattern-not-defined/299893 "2022-03-16T19:48:30Z")
Hi, I try to ingest the logs /var/log/maillog with the following pattern via logstash: POSTFIX\_QUEUEID (\[0-9A-F\]{6,}|\[0-9a-zA-Z\]{15,}) POSTFIX\_STATUS (?\<=status=)(.\*)(?= \\() POSTFIX\_PROCESS (?=postfix\\/)(.\*?\\\[)(.\*?)(?=… --- ## [Logstash date filter plugin adds \_dateparsefailure tag on event data from JDBC](https://discuss.elastic.co/t/logstash-date-filter-plugin-adds-dateparsefailure-tag-on-event-data-from-jdbc/299855)
**Author:** [@nilei](https://discuss.elastic.co/u/nilei)\ **Replies:** 2\ **Last updated:** [March 16, 2022, 3:18pm UTC](https://discuss.elastic.co/t/logstash-date-filter-plugin-adds-dateparsefailure-tag-on-event-data-from-jdbc/299855 "2022-03-16T15:18:47Z")
Dear all, I fetch data from a MySQL database via the JDBC input plugin. Here is an excerpt from the SQL result: +---------------------------+--------------------------+ | History.Created.DateValue | History.Number.KeyV… --- ## [Unable to load beats input plugin after installing it locally](https://discuss.elastic.co/t/unable-to-load-beats-input-plugin-after-installing-it-locally/299863)
**Author:** [@Nikhitha](https://discuss.elastic.co/u/Nikhitha)\ **Replies:** 0\ **Last updated:** [March 16, 2022, 1:32pm UTC](https://discuss.elastic.co/t/unable-to-load-beats-input-plugin-after-installing-it-locally/299863 "2022-03-16T13:32:07Z")
After installing beats input plugin after dowloadong from github and generating .gem file, it appears when listed the plugins in logstash, but when ran any configuration file on beats, it says pluginLoading error. Can s… --- ## [Logstash Geoip plugin](https://discuss.elastic.co/t/logstash-geoip-plugin/299044)
**Author:** [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)\ **Replies:** 10\ **Last updated:** [March 16, 2022, 12:55pm UTC](https://discuss.elastic.co/t/logstash-geoip-plugin/299044 "2022-03-16T12:55:53Z")
Hi Im using Logstash Geoip Plugin as below geoip { source =\> "sourceIP" target =\> "geoIP" add\_field =\> { "countryCode" =\> "%{\[geoIP\]\[country\_code2\]}" } } But at… --- ## [S3 input dont delete/backup files after processing](https://discuss.elastic.co/t/s3-input-dont-delete-backup-files-after-processing/299187)
**Author:** [@froheik](https://discuss.elastic.co/u/froheik)\ **Replies:** 2\ **Last updated:** [March 16, 2022, 11:21am UTC](https://discuss.elastic.co/t/s3-input-dont-delete-backup-files-after-processing/299187 "2022-03-16T11:21:00Z")
Hello. Trouble with setting S3 input plugin with private S3 like AWS Minio. Logstash version OSS 7.16, 7.17, 8.0, 8.1. Logstash normally read object and send to output, but backup or delete is not working. Object stay… --- ## [Premature file delete when file\_completed\_action is set to "delete"](https://discuss.elastic.co/t/premature-file-delete-when-file-completed-action-is-set-to-delete/299845)
**Author:** [@pinehk](https://discuss.elastic.co/u/pinehk)\ **Replies:** 0\ **Last updated:** [March 16, 2022, 9:12am UTC](https://discuss.elastic.co/t/premature-file-delete-when-file-completed-action-is-set-to-delete/299845 "2022-03-16T09:12:01Z")
Hi, The files i needed to push to Elasticsearch for indexing are all "completed" files. To avoid duplicated document I'm trying the file\_completed\_action switch. Setting a document ID is not an option as my data files a… --- ## [How to find missing logs](https://discuss.elastic.co/t/how-to-find-missing-logs/299823)
**Author:** [@dginfi](https://discuss.elastic.co/u/dginfi)\ **Replies:** 0\ **Last updated:** [March 16, 2022, 7:01am UTC](https://discuss.elastic.co/t/how-to-find-missing-logs/299823 "2022-03-16T07:01:12Z")
I have logs being forwarding from Panorama to ELK. On ELK server I have filebeats setup with panw module enabled which receives the logs and forwards to logstash. In logstash filter I am using jdbc\_streaming filter for… --- ## [Nagios Performance data =\> Kafka Topic =\> Elastic search(Need to process Nagios performance data)](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594)
**Author:** [@ranjithkodumbu](https://discuss.elastic.co/u/ranjithkodumbu)\ **Replies:** 4\ **Last updated:** [March 16, 2022, 5:13am UTC](https://discuss.elastic.co/t/nagios-performance-data-kafka-topic-elastic-search-need-to-process-nagios-performance-data/299594 "2022-03-16T05:13:20Z")
Hello team, I am trying to process Nagios performance data to ELK using logstash, I am receiving Nagios performance data format below, need to process and update it to the Elastic index. {"check\_type": "service", "chec… --- ## [Logstash didn't send logs to Elasticsearch after upgrade](https://discuss.elastic.co/t/logstash-didnt-send-logs-to-elasticsearch-after-upgrade/299802)
**Author:** [@Othmane\_CHHAIBI](https://discuss.elastic.co/u/Othmane_CHHAIBI)\ **Replies:** 2\ **Last updated:** [March 16, 2022, 12:17am UTC](https://discuss.elastic.co/t/logstash-didnt-send-logs-to-elasticsearch-after-upgrade/299802 "2022-03-16T00:17:34Z")
hi, i upgrade logstash from version 6.8 to 7.16.2 but after this upgrade the status of logstash is running and it didn't sent logs to Elasticsearch (version 7.16.2). Here the error message from logs of logstash server : … --- ## [Getting multiple fields out of agent](https://discuss.elastic.co/t/getting-multiple-fields-out-of-agent/298228)
**Author:** [@itschobot](https://discuss.elastic.co/u/itschobot)\ **Replies:** 3\ **Last updated:** [March 15, 2022, 8:22pm UTC](https://discuss.elastic.co/t/getting-multiple-fields-out-of-agent/298228 "2022-03-15T20:22:25Z")
still new and learning filters... how would I get multiple fields out of agent? "agent" =\> { "hostname" =\> "2487b31167e2", "name" =\> "123b456c", "ephemeral\_id" =\> "123b456c", "version" =\> "7.15.1", "id" =\> "123b45… --- ## [Cisco-module (Filebeat) to Logstash - Configuration issue- Need help!](https://discuss.elastic.co/t/cisco-module-filebeat-to-logstash-configuration-issue-need-help/299765)
**Author:** [@arunhk3](https://discuss.elastic.co/u/arunhk3)\ **Replies:** 2\ **Last updated:** [March 15, 2022, 8:08pm UTC](https://discuss.elastic.co/t/cisco-module-filebeat-to-logstash-configuration-issue-need-help/299765 "2022-03-15T20:08:13Z")
Hi All, I was able to send logs to Elasticsearch using Filebeat using the below configuration successfully. # ============================== Filebeat inputs =============================== filebeat.inputs: - type: log… --- ## [Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/299747)
**Author:** [@Diego\_T](https://discuss.elastic.co/u/Diego_T)\ **Replies:** 4\ **Last updated:** [March 15, 2022, 7:22pm UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/299747 "2022-03-15T19:22:58Z")
when i run logstash on windows i get the following error C:\\logstash-8.1.0\>.\\bin\\logstash.bat -f .\\config\\syslog.conf Using LS\_JAVA\_HOME defined java: C:\\Program Files\\Java\\jdk-17.0.2 WARNING: Using LS\_JAVA\_HOME while L… --- ## [Unable to build logstash-input-tcp - compatible versions for gem "logstash-core"](https://discuss.elastic.co/t/unable-to-build-logstash-input-tcp-compatible-versions-for-gem-logstash-core/299671)
**Author:** [@vinodhreddyg](https://discuss.elastic.co/u/vinodhreddyg)\ **Replies:** 1\ **Last updated:** [March 15, 2022, 6:23pm UTC](https://discuss.elastic.co/t/unable-to-build-logstash-input-tcp-compatible-versions-for-gem-logstash-core/299671 "2022-03-15T18:23:11Z")
Hi Team, I'm trying to build logstash-input-tcp plugin. It is failing with the following error. Exception is showing as compatible versions for gem "logstash-core". ENV details: Centos Stream JAVA: javac 11.0.14 Jru… --- ## [Logstash Sleep Filter Plugin Not Working](https://discuss.elastic.co/t/logstash-sleep-filter-plugin-not-working/299753)
**Author:** [@mpride63](https://discuss.elastic.co/u/mpride63)\ **Replies:** 1\ **Last updated:** [March 15, 2022, 5:36pm UTC](https://discuss.elastic.co/t/logstash-sleep-filter-plugin-not-working/299753 "2022-03-15T17:36:29Z")
I am new to logstash and I’m trying to setup a logstash instance to see if we can use it as a syslog\\trap front end to some of our network nodes. I have it working at a very basic level but can’t seem to get the sleep f… --- ## [Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/299338)
**Author:** [@RomanKau](https://discuss.elastic.co/u/RomanKau)\ **Replies:** 12\ **Last updated:** [March 15, 2022, 8:21am UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/299338 "2022-03-15T08:21:20Z")
Running Windows 10, Logstash 8.1.0, elasticsearch, kibana and filebeat 8.0.0 all on the same machine. I also tried running version 8.0.0 of Logstash but I get the same error. Getting the data from filebeat to kibana wor… --- ## [Logstash Writing to Two Log Files](https://discuss.elastic.co/t/logstash-writing-to-two-log-files/299628)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 4\ **Last updated:** [March 15, 2022, 8:19am UTC](https://discuss.elastic.co/t/logstash-writing-to-two-log-files/299628 "2022-03-15T08:19:32Z")
Hi, Our logstash 7.16.2 instances are writing log files to /var/log/logstash/logstash-plain.log and /var/log/logstash-stdout.log. How do we prevent the creation of the second log under /var/log? Log config looks like t… --- ## [Logstash not working as a service](https://discuss.elastic.co/t/logstash-not-working-as-a-service/299600)
**Author:** [@ashiqab](https://discuss.elastic.co/u/ashiqab)\ **Replies:** 2\ **Last updated:** [March 15, 2022, 3:19am UTC](https://discuss.elastic.co/t/logstash-not-working-as-a-service/299600 "2022-03-15T03:19:26Z")
Hello, I am new to ELK and am trying to set up monitoring on few of our application jobs using ELK stack. I could run the below query successfully and the data is reflecting on Kibana: logstash -f /etc/logstash/conf.d/… --- ## [Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/299474)
**Author:** [@paulov](https://discuss.elastic.co/u/paulov)\ **Replies:** 7\ **Last updated:** [March 14, 2022, 2:25pm UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/299474 "2022-03-14T14:25:31Z")
Hi, After instaling/configuring logstash, I get this error: \[2022-03-11T09:22:27,379\]\[INFO \]\[logstash.runner \] Log4j configuration path used is: /u02/logstash\_frontend/etc/log4j2.properties \[2022-03-11T09:22:… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=152) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=154)