# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=188 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 189 --- ## [Logstash pipeline multiple if statements](https://discuss.elastic.co/t/logstash-pipeline-multiple-if-statements/286737)
**Author:** [@Robin020](https://discuss.elastic.co/u/Robin020)\ **Replies:** 1\ **Last updated:** [October 15, 2021, 5:02pm UTC](https://discuss.elastic.co/t/logstash-pipeline-multiple-if-statements/286737 "2021-10-15T17:02:58Z")
Hello, I am trying create multiple if statements in the filter section in my pipeline config but I have no success with that. This works (with one if statement) filter { json { source =\> "message" } split … --- ## [Can you reference a dynamic variable in the grok filter?](https://discuss.elastic.co/t/can-you-reference-a-dynamic-variable-in-the-grok-filter/286838)
**Author:** [@Cibot](https://discuss.elastic.co/u/Cibot)\ **Replies:** 2\ **Last updated:** [October 15, 2021, 4:57pm UTC](https://discuss.elastic.co/t/can-you-reference-a-dynamic-variable-in-the-grok-filter/286838 "2021-10-15T16:57:52Z")
I have been trying to split up the input of a script from execbeat. So far everything has been going smooth. I have input of similar fashion. rpm-2109130090009900.x86-64 Thu 2019 ... and so on. So basically an inform… --- ## [Parsing nested json to flat structure](https://discuss.elastic.co/t/parsing-nested-json-to-flat-structure/286790)
**Author:** [@IsAa](https://discuss.elastic.co/u/IsAa)\ **Replies:** 1\ **Last updated:** [October 15, 2021, 4:50pm UTC](https://discuss.elastic.co/t/parsing-nested-json-to-flat-structure/286790 "2021-10-15T16:50:36Z")
Hi Team, I was looking to parse my nested json structure using json filter. I have looked at some of the posts and tried their solution. My question, is it possible to make this structure completely flat, So if my log… --- ## [Logstash creates a 'page.xxx' file, 0KB in size, stopping all log ingestion](https://discuss.elastic.co/t/logstash-creates-a-page-xxx-file-0kb-in-size-stopping-all-log-ingestion/286849)
**Author:** [@jtocher](https://discuss.elastic.co/u/jtocher)\ **Replies:** 0\ **Last updated:** [October 15, 2021, 2:09pm UTC](https://discuss.elastic.co/t/logstash-creates-a-page-xxx-file-0kb-in-size-stopping-all-log-ingestion/286849 "2021-10-15T14:09:26Z")
We're using the Elastic Stack to centralize logs across multiple systems. We're using Windows, and are currently on version 7.10. (Upgrading to 7.15 in the near future.) Occasionally all of our log ingestion halts comp… --- ## [Elapsed time filter calculating negative time](https://discuss.elastic.co/t/elapsed-time-filter-calculating-negative-time/286786)
**Author:** [@Indigo\_Star](https://discuss.elastic.co/u/Indigo_Star)\ **Replies:** 0\ **Last updated:** [October 15, 2021, 1:58am UTC](https://discuss.elastic.co/t/elapsed-time-filter-calculating-negative-time/286786 "2021-10-15T01:58:53Z")
Hi I am loading offline logs and using elapsed plugin to calculate delay between 2 events, but the time calculating is not always correct sometimes it doesn't display the value in correct second sometimes it shows negat… --- ## [Index not creating](https://discuss.elastic.co/t/index-not-creating/285232)
**Author:** [@jubin03](https://discuss.elastic.co/u/jubin03)\ **Replies:** 16\ **Last updated:** [October 14, 2021, 6:22pm UTC](https://discuss.elastic.co/t/index-not-creating/285232 "2021-10-14T18:22:40Z")
Following with this topic Index not creating - #17 by jubin03 I have created a new ELK and fetched data and it is working fine and able to create index patter, but the existing one is not working. Could somebody have an… --- ## [Tag Doc if match to List of Text terms Keywords](https://discuss.elastic.co/t/tag-doc-if-match-to-list-of-text-terms-keywords/286662)
**Author:** [@Dallas\_Toth](https://discuss.elastic.co/u/Dallas_Toth)\ **Replies:** 2\ **Last updated:** [October 14, 2021, 5:42pm UTC](https://discuss.elastic.co/t/tag-doc-if-match-to-list-of-text-terms-keywords/286662 "2021-10-14T17:42:18Z")
I have a list of terms about 1000 words and I want to have logstash in a Filter tag the document if there is a match in any of the words in a text field. Example: text : "The big brown dog" My List of 1000 terms has D… --- ## [Log4net to logstash](https://discuss.elastic.co/t/log4net-to-logstash/286250)
**Author:** [@Vinicius\_Mylonas](https://discuss.elastic.co/u/Vinicius_Mylonas)\ **Replies:** 1\ **Last updated:** [October 14, 2021, 5:30pm UTC](https://discuss.elastic.co/t/log4net-to-logstash/286250 "2021-10-14T17:30:39Z")
I'm trying to get a log like this from log4net: \[27\]\[14:08:36.584\] \[{"name":"Vinicius","SerialNumber":"957593196"}\] \[37\]\[14:08:36.584\] \[{"name":"Mylonas","SerialNumber":"957593166"}\] ... How can I get this information… --- ## [Logstash can't finde the certificate](https://discuss.elastic.co/t/logstash-cant-finde-the-certificate/286476)
**Author:** [@smam](https://discuss.elastic.co/u/smam)\ **Replies:** 15\ **Last updated:** [October 14, 2021, 2:17pm UTC](https://discuss.elastic.co/t/logstash-cant-finde-the-certificate/286476 "2021-10-14T14:17:03Z")
Hello, I am trying to set up Logstash but I am failing for a week and now ask for your help. ( Using: /usr/share/logstash/bin/logstash ) I have a certificate chain, which contains the certificate for the webserver and t… --- ## [Save redis channel name inside document](https://discuss.elastic.co/t/save-redis-channel-name-inside-document/286633)
**Author:** [@Metehan\_Tagizade](https://discuss.elastic.co/u/Metehan_Tagizade)\ **Replies:** 2\ **Last updated:** [October 14, 2021, 1:55pm UTC](https://discuss.elastic.co/t/save-redis-channel-name-inside-document/286633 "2021-10-14T13:55:54Z")
Hi, I have logstash pipeline like: input { redis { host=\> "redis-endpoint.com" port =\> "6379" key =\> "\*" data\_type =\> "pattern\_channel" batch\_count =\> 100 codec =\> "… --- ## [Since we moved 7.15 "Pushing flush onto pipeline." and files are not parsed](https://discuss.elastic.co/t/since-we-moved-7-15-pushing-flush-onto-pipeline-and-files-are-not-parsed/286605)
**Author:** [@fabeau](https://discuss.elastic.co/u/fabeau)\ **Replies:** 2\ **Last updated:** [October 14, 2021, 9:14am UTC](https://discuss.elastic.co/t/since-we-moved-7-15-pushing-flush-onto-pipeline-and-files-are-not-parsed/286605 "2021-10-14T09:14:30Z")
We deployed logstash config files on a new server running 7.15 from a 7.11 server working perfectly . We validated .conf files in command line and everything is fine. we parse some log files locally on the server and … --- ## [Run logstash on ram](https://discuss.elastic.co/t/run-logstash-on-ram/286615)
**Author:** [@hatuli](https://discuss.elastic.co/u/hatuli)\ **Replies:** 1\ **Last updated:** [October 13, 2021, 10:09pm UTC](https://discuss.elastic.co/t/run-logstash-on-ram/286615 "2021-10-13T22:09:29Z")
Hi, im working with fluentbit to export the logs from my k8s env, then to kafka, logstash and in the end to elasticearch, the massages are indexing inside ES but after a while they stop to index, i conf heap to 8G, but s… --- ## [Grok regex pattern for symbols?](https://discuss.elastic.co/t/grok-regex-pattern-for-symbols/286624)
**Author:** [@helloworld1234](https://discuss.elastic.co/u/helloworld1234)\ **Replies:** 5\ **Last updated:** [October 13, 2021, 4:44pm UTC](https://discuss.elastic.co/t/grok-regex-pattern-for-symbols/286624 "2021-10-13T16:44:24Z")
Lets say I have a string: "petBreed":"dachshund" "petName":"rufus" "petAge":"12" The value is not constant and can vary in length and content. And i want to use grok overwrite to remove rufus from the raw string. So… --- ## [Logstash - how to remove strings within a field (whose content itself a raw string)?](https://discuss.elastic.co/t/logstash-how-to-remove-strings-within-a-field-whose-content-itself-a-raw-string/286561)
**Author:** [@helloworld1234](https://discuss.elastic.co/u/helloworld1234)\ **Replies:** 3\ **Last updated:** [October 13, 2021, 4:22pm UTC](https://discuss.elastic.co/t/logstash-how-to-remove-strings-within-a-field-whose-content-itself-a-raw-string/286561 "2021-10-13T16:22:58Z")
I have a event that consists of various fields, one of which is called "raw\_message". This field's contents is a raw string eg. Input: raw\_message: "Raw user details:: \[location:london name:kris wu id:L3j5k category:vi… --- ## [Stack Monitoring fails to load logstash pipelines](https://discuss.elastic.co/t/stack-monitoring-fails-to-load-logstash-pipelines/281209)
**Author:** [@radovan](https://discuss.elastic.co/u/radovan)\ **Replies:** 26\ **Last updated:** [October 13, 2021, 2:29pm UTC](https://discuss.elastic.co/t/stack-monitoring-fails-to-load-logstash-pipelines/281209 "2021-10-13T14:29:46Z")
Hi, I decided to create a monitoring cluster and after a few problems at the start all is running now. There are metricbeat clients on the logstash nodes with only logstash-xpack module enabled. Logs are flowing onto th… --- ## [Parsing tabular data into canvas](https://discuss.elastic.co/t/parsing-tabular-data-into-canvas/286496)
**Author:** [@Kadhem](https://discuss.elastic.co/u/Kadhem)\ **Replies:** 1\ **Last updated:** [October 12, 2021, 10:54pm UTC](https://discuss.elastic.co/t/parsing-tabular-data-into-canvas/286496 "2021-10-12T22:54:34Z")
hi, how can i parse this output of command to push into into a table in Canvas : prec: rqstd, stored, dropped, retried, rtsfail,rtrydrop, psretry, acked,utlisatn,q length,Data Mbits/s,Phy Mbits/s, %air, %effcy (v5… --- ## [Which architecture is better? Should I collect the logs on the server side using rsyslog?](https://discuss.elastic.co/t/which-architecture-is-better-should-i-collect-the-logs-on-the-server-side-using-rsyslog/286589)
**Author:** [@Sagimb](https://discuss.elastic.co/u/Sagimb)\ **Replies:** 3\ **Last updated:** [October 13, 2021, 9:34am UTC](https://discuss.elastic.co/t/which-architecture-is-better-should-i-collect-the-logs-on-the-server-side-using-rsyslog/286589 "2021-10-13T09:34:35Z")
I am trying to understand which architecture is better for me. I have a few hundreds of instances that I want to send their logs to one central server. Should I got with A (run rsyslog on the central server to collec… --- ## [Json Parsing](https://discuss.elastic.co/t/json-parsing/286277)
**Author:** [@John\_snow](https://discuss.elastic.co/u/John_snow)\ **Replies:** 7\ **Last updated:** [October 13, 2021, 9:21am UTC](https://discuss.elastic.co/t/json-parsing/286277 "2021-10-13T09:21:29Z")
i have event coming in my logstash as a json {"key1": "value1", "key2": "value2", "key3": {"key4": "value4"}} I wanted too put that event inside a key data like {"data" : { "key1": "value1", "ke… --- ## [Rsyslog, Logstash parse Error when incoming log is too big](https://discuss.elastic.co/t/rsyslog-logstash-parse-error-when-incoming-log-is-too-big/286513)
**Author:** [@Sequa](https://discuss.elastic.co/u/Sequa)\ **Replies:** 0\ **Last updated:** [October 12, 2021, 2:01pm UTC](https://discuss.elastic.co/t/rsyslog-logstash-parse-error-when-incoming-log-is-too-big/286513 "2021-10-12T14:01:30Z")
Hello, I have build a syslog server with web gui for this is use the ELK-Stack and configured Rsyslog. Everything is good and it works but when clients send log files that are more than 7999 characters long, I get the fo… --- ## [Grok pattern](https://discuss.elastic.co/t/grok-pattern/286575)
**Author:** [@Divya\_Bansal](https://discuss.elastic.co/u/Divya_Bansal)\ **Replies:** 2\ **Last updated:** [October 13, 2021, 8:54am UTC](https://discuss.elastic.co/t/grok-pattern/286575 "2021-10-13T08:54:36Z")
I have to parse this log- 05-10-21T13:17:20.457741775|lapideployment-547944bff7-4f2qr|1|iuser|20|0|25.090g|1.378g|68160|S|0.0|1.1|116:20.87|java|265 I am using pattern-%{TIMESTAMP\_ISO8601:timestamp}|%{DATA:pod}| It is… --- ## [Logstash plugin install error](https://discuss.elastic.co/t/logstash-plugin-install-error/286571)
**Author:** [@nigel.wadsworth](https://discuss.elastic.co/u/nigel.wadsworth)\ **Replies:** 0\ **Last updated:** [October 13, 2021, 6:19am UTC](https://discuss.elastic.co/t/logstash-plugin-install-error/286571 "2021-10-13T06:19:18Z")
Hi there. I have an on-prem Elastic stack implementation that needed the logstash github plugin. This isn't installed by default, and my stack is in an OT zone so is locked from the internet. To get round this, I foll… --- ## [Logstash Process](https://discuss.elastic.co/t/logstash-process/285886)
**Author:** [@adityaPsl](https://discuss.elastic.co/u/adityaPsl)\ **Replies:** 2\ **Last updated:** [October 13, 2021, 5:32am UTC](https://discuss.elastic.co/t/logstash-process/285886 "2021-10-13T05:32:26Z")
Hello Team, Just wanted to understand from logstash processes point , if I configure multiple pipelines on a single node(machine) will all the pipelines run under the single process (my understanding is it will run on s… --- ## [Getting timestamp inside timestamp](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511)
**Author:** [@Prasang-Biyani](https://discuss.elastic.co/u/Prasang-Biyani)\ **Replies:** 12\ **Last updated:** [October 13, 2021, 5:28am UTC](https://discuss.elastic.co/t/getting-timestamp-inside-timestamp/286511 "2021-10-13T05:28:05Z")
Hi Everyone, I am new to the Logstash, so please forgive me for asking silly question. I am trying to convert timestamp from the log file into @timestamp. Here is my logstash conf file. input { udp { port… --- ## [Persistent queue corruption](https://discuss.elastic.co/t/persistent-queue-corruption/286412)
**Author:** [@franck.lefebure](https://discuss.elastic.co/u/franck.lefebure)\ **Replies:** 1\ **Last updated:** [October 13, 2021, 3:59am UTC](https://discuss.elastic.co/t/persistent-queue-corruption/286412 "2021-10-13T03:59:32Z")
Hi, I had a disk saturation on a logstash 5.6.8 installation I got to remove the "checkpoint.head" (0 bytes size) file to restart the pipeline Now I have theses files in queue dir : -rw-r--r-- 1 logstash logstash 262… --- ## [Is Elastic search filter in log stash work in synchronous or asynchronous mode?](https://discuss.elastic.co/t/is-elastic-search-filter-in-log-stash-work-in-synchronous-or-asynchronous-mode/286472)
**Author:** [@Sandeep\_Thakur](https://discuss.elastic.co/u/Sandeep_Thakur)\ **Replies:** 1\ **Last updated:** [October 13, 2021, 12:49am UTC](https://discuss.elastic.co/t/is-elastic-search-filter-in-log-stash-work-in-synchronous-or-asynchronous-mode/286472 "2021-10-13T00:49:57Z")
I am facing one issue in ELK when I continuous log and tried to update some fields in log stash configuration file using Elasticsearch filter plugin its updating logs sometimes but sometimes not working, so is there and … --- ## [How to get data into the beats specific index via logstash](https://discuss.elastic.co/t/how-to-get-data-into-the-beats-specific-index-via-logstash/286328)
**Author:** [@gwvandesteeg](https://discuss.elastic.co/u/gwvandesteeg)\ **Replies:** 19\ **Last updated:** [October 13, 2021, 12:15am UTC](https://discuss.elastic.co/t/how-to-get-data-into-the-beats-specific-index-via-logstash/286328 "2021-10-13T00:15:58Z")
The use case here is that we have: \*beats -\> logstash -\> elasticsearch cloud The following requirements are in place: The hosts running the beats do not have direct internet access and can only communicate via logsta… --- ## [Finding data with logstash 7.13.4](https://discuss.elastic.co/t/finding-data-with-logstash-7-13-4/286135)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 6\ **Last updated:** [October 12, 2021, 8:44pm UTC](https://discuss.elastic.co/t/finding-data-with-logstash-7-13-4/286135 "2021-10-12T20:44:15Z")
I have my installation of the ELK stack up and running. All three aspects are running on separate servers but they all communicate and can talk to one another. I have not yet installed any Beats or Security on the inst… --- ## [Comparing two fields between two different index and output result into a third index](https://discuss.elastic.co/t/comparing-two-fields-between-two-different-index-and-output-result-into-a-third-index/286535)
**Author:** [@test\_tester](https://discuss.elastic.co/u/test_tester)\ **Replies:** 1\ **Last updated:** [October 12, 2021, 7:10pm UTC](https://discuss.elastic.co/t/comparing-two-fields-between-two-different-index-and-output-result-into-a-third-index/286535 "2021-10-12T19:10:00Z")
Hello! I got a requirement whereby I am required to match two fields from two different files and output the result into a csv file. Prior to this train of thought, i did a simple python script to do the matching logic… --- ## [Logstash Converting Base64 IP to Decimal IP](https://discuss.elastic.co/t/logstash-converting-base64-ip-to-decimal-ip/286153)
**Author:** [@pkward](https://discuss.elastic.co/u/pkward)\ **Replies:** 19\ **Last updated:** [October 12, 2021, 3:07pm UTC](https://discuss.elastic.co/t/logstash-converting-base64-ip-to-decimal-ip/286153 "2021-10-12T15:07:49Z")
Hello, I'm trying to convert a base64 IP into a decimal IP. I saw that Logstash Ruby filter plugin can assist with this by parsing the message field, but is there a way to do this for just an IP field? --- ## [Running logstash in background in linux](https://discuss.elastic.co/t/running-logstash-in-background-in-linux/286517)
**Author:** [@mangeshs](https://discuss.elastic.co/u/mangeshs)\ **Replies:** 1\ **Last updated:** [October 12, 2021, 2:25pm UTC](https://discuss.elastic.co/t/running-logstash-in-background-in-linux/286517 "2021-10-12T14:25:53Z")
I want to run logstash in background on linux server i am using command like this on bin path nohup ./logstash -f sample.conf& is it correct way to run the logstash or anything else but then I need to read nohup.out … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=187) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=189)