# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=239 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 240 --- ## [Translate filter regex capture grouping](https://discuss.elastic.co/t/translate-filter-regex-capture-grouping/269922)
**Author:** [@aap323](https://discuss.elastic.co/u/aap323)\ **Replies:** 4\ **Last updated:** [April 12, 2021, 7:55pm UTC](https://discuss.elastic.co/t/translate-filter-regex-capture-grouping/269922 "2021-04-12T19:55:42Z")
Hi, I am using Logstash to parse and enter graphite metrics into InfluxDB. However, I wanted to create a large dictionary source where it can match certain metric lines based on their pattern and reorganize the string ba… --- ## [Logstash 7.12.0 High CPU usage](https://discuss.elastic.co/t/logstash-7-12-0-high-cpu-usage/269851)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 5\ **Last updated:** [April 12, 2021, 7:41pm UTC](https://discuss.elastic.co/t/logstash-7-12-0-high-cpu-usage/269851 "2021-04-12T19:41:21Z")
Hi All, I know that there are several open discussions on the topic already but nothing in there helped me resolving my situation. #1 I installed Logstash on Ubuntu 20.04 for testing purposes and enabling it to run with… --- ## [Logstash fell under load](https://discuss.elastic.co/t/logstash-fell-under-load/269766)
**Author:** [@Andrey\_RF](https://discuss.elastic.co/u/Andrey_RF)\ **Replies:** 14\ **Last updated:** [April 12, 2021, 4:54pm UTC](https://discuss.elastic.co/t/logstash-fell-under-load/269766 "2021-04-12T16:54:31Z")
Hello. I have three clients. They send logs via filebeat to main server with logstash. A few days ago there are a lot of logs somewhere around 12 M in an amount. The logstash couldn't process all this logs and it fell. … --- ## [Logstash with elasticsearch input](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783)
**Author:** [@Kfiro](https://discuss.elastic.co/u/Kfiro)\ **Replies:** 5\ **Last updated:** [April 12, 2021, 3:50pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783 "2021-04-12T15:50:41Z")
Hi Masters, I'm trying to use Logsash with "elasticsearch" input for taking specific events from winlogbeat Index and write them to another Index, I have a problem with the "query" on the input section, when I'm tryi… --- ## [Logstash jdbc input plugin mongodb driver can not query time range](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-mongodb-driver-can-not-query-time-range/269879)
**Author:** [@john.zhang](https://discuss.elastic.co/u/john.zhang)\ **Replies:** 1\ **Last updated:** [April 12, 2021, 3:43pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-mongodb-driver-can-not-query-time-range/269879 "2021-04-12T15:43:43Z")
Hi, Please help . I want to search records in mongodb during 3 days. this is my logstash conf: input { jdbc { jdbc\_driver\_class =\> "com.dbschema.MongoJdbcDriver" jdbc\_driver\_library =\> "/usr/share/logstash/logstas… --- ## [Logstash 7.9 high CPU, RAM and hangs when a unicode message is received by syslog input](https://discuss.elastic.co/t/logstash-7-9-high-cpu-ram-and-hangs-when-a-unicode-message-is-received-by-syslog-input/269885)
**Author:** [@ivorthe3ngine](https://discuss.elastic.co/u/ivorthe3ngine)\ **Replies:** 0\ **Last updated:** [April 12, 2021, 1:36pm UTC](https://discuss.elastic.co/t/logstash-7-9-high-cpu-ram-and-hangs-when-a-unicode-message-is-received-by-syslog-input/269885 "2021-04-12T13:36:45Z")
Hi All, any assistance on this issue would be most appreciated. On receiving the following message, logstash stopped processing any further incoming messages and began consuming high CPU. Version: 7.9.0 Operating Syst… --- ## [Logstash](https://discuss.elastic.co/t/logstash/269583)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 2\ **Last updated:** [April 12, 2021, 12:03pm UTC](https://discuss.elastic.co/t/logstash/269583 "2021-04-12T12:03:12Z")
hi i hope this message finds you well , so how can configure High availibility between 2 logstash ? regards --- ## [Multiple pipelines](https://discuss.elastic.co/t/multiple-pipelines/269867)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 0\ **Last updated:** [April 12, 2021, 11:07am UTC](https://discuss.elastic.co/t/multiple-pipelines/269867 "2021-04-12T11:07:31Z")
hi, I have multiple inputs and I want to parse and enrich them by the project GitHub - Cargill/OpenSIEM-Logstash-Parsing: SIEM Logstash parsing for more than hundred technologies so how can I do that thanks --- ## [Golang logstash?](https://discuss.elastic.co/t/golang-logstash/269610)
**Author:** [@wshek](https://discuss.elastic.co/u/wshek)\ **Replies:** 0\ **Last updated:** [April 8, 2021, 2:51pm UTC](https://discuss.elastic.co/t/golang-logstash/269610 "2021-04-08T14:51:45Z")
Enable golang for your logstash pipe This is not a rewrite its an extensible idea. Has anyone come across this type idea before? I have used Elasticstack for quite some time now. I can tell you unequivocally that th… --- ## [Logstash Grok Filter Error Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \[A-Za-z0-9\_-\], '\\"', \\"'\\", \[A-Za-z\_\], \\"-\\", \[0-9\], \\"\[\\", \\"{\\"](https://discuss.elastic.co/t/logstash-grok-filter-error-expected-one-of-t-r-n-a-za-z0-9-a-za-z-0-9/269808)
**Author:** [@baklavadeseni](https://discuss.elastic.co/u/baklavadeseni)\ **Replies:** 3\ **Last updated:** [April 11, 2021, 11:42pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-error-expected-one-of-t-r-n-a-za-z0-9-a-za-z-0-9/269808 "2021-04-11T23:42:26Z")
Hello Logstash Community, I am trying to parse text below. I was able to filter it in grokdebug site. But however i can't filter in logstash. It throws this. "LogStash::ConfigurationError", :message=\>"Expected one of \[… --- ## [Get xml node name](https://discuss.elastic.co/t/get-xml-node-name/269775)
**Author:** [@Petr\_Vancl\_Hochberge](https://discuss.elastic.co/u/Petr_Vancl_Hochberge)\ **Replies:** 3\ **Last updated:** [April 11, 2021, 2:35pm UTC](https://discuss.elastic.co/t/get-xml-node-name/269775 "2021-04-11T14:35:51Z")
Hello, I am little lost with one problem when parsing xml files to ES with Logstash. Due to unfortunate xml structure which I can't change, I need to get string name of specific nodes. Example of xml structure: \… --- ## [How to print out %{host.ip} using line codec?](https://discuss.elastic.co/t/how-to-print-out-host-ip-using-line-codec/269750)
**Author:** [@hmiti](https://discuss.elastic.co/u/hmiti)\ **Replies:** 6\ **Last updated:** [April 10, 2021, 3:09pm UTC](https://discuss.elastic.co/t/how-to-print-out-host-ip-using-line-codec/269750 "2021-04-10T15:09:32Z")
I am using the line codec to output the logs into a text file. Requested to include only the IP and log entry itself. Tried different combination of %{host}.{ip} like this, nothing works so far. output { file { p… --- ## [Extract json fields from message](https://discuss.elastic.co/t/extract-json-fields-from-message/269742)
**Author:** [@adityak248](https://discuss.elastic.co/u/adityak248)\ **Replies:** 5\ **Last updated:** [April 9, 2021, 9:11pm UTC](https://discuss.elastic.co/t/extract-json-fields-from-message/269742 "2021-04-09T21:11:37Z")
Hello there, After applying grok filter to my message I get one of the fields called main\_message which looks like this main\_message Processing data {"si":"-2","a":"-54.0","r":"0","version":"1","id":"C112"} From t… --- ## [How to exclude key value pair from KV filter if Key size greater than 15 char](https://discuss.elastic.co/t/how-to-exclude-key-value-pair-from-kv-filter-if-key-size-greater-than-15-char/269566)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 6\ **Last updated:** [April 9, 2021, 6:00pm UTC](https://discuss.elastic.co/t/how-to-exclude-key-value-pair-from-kv-filter-if-key-size-greater-than-15-char/269566 "2021-04-09T18:00:42Z")
We have log message which contain key value pairs and we are using KV filter to parse these data. We need to exclude key value pair from KV filter if Key size greater than 15 char. Need to exclude https://www.google.co… --- ## [How to call mget from Logstash?](https://discuss.elastic.co/t/how-to-call-mget-from-logstash/269733)
**Author:** [@ice2021](https://discuss.elastic.co/u/ice2021)\ **Replies:** 3\ **Last updated:** [April 9, 2021, 5:12pm UTC](https://discuss.elastic.co/t/how-to-call-mget-from-logstash/269733 "2021-04-09T17:12:09Z")
The REST request below works in Kibana. Can this similarly be achieved in Logstash? I am trying to retrieve multiple documents from multiple indices(5 indices) in a single request to ElasticSearch in Logstash. GET /\_… --- ## [Split filebeat log with a variable length and create multiple events group by a field](https://discuss.elastic.co/t/split-filebeat-log-with-a-variable-length-and-create-multiple-events-group-by-a-field/269630)
**Author:** [@rmartinez.rv](https://discuss.elastic.co/u/rmartinez.rv)\ **Replies:** 13\ **Last updated:** [April 9, 2021, 4:54pm UTC](https://discuss.elastic.co/t/split-filebeat-log-with-a-variable-length-and-create-multiple-events-group-by-a-field/269630 "2021-04-09T16:54:36Z")
Hi. I'm trying to parse the information from a log line with this format. 2021 Mar 30 00:45:01;1617075901;user1,gw11,0;user1,gw22,5;user2,gw33,2;user2;gw43,3 I'm using dissect to format the first part of the log, with… --- ## [Stuck in Logstash](https://discuss.elastic.co/t/stuck-in-logstash/269704)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 15\ **Last updated:** [April 9, 2021, 4:43pm UTC](https://discuss.elastic.co/t/stuck-in-logstash/269704 "2021-04-09T16:43:54Z")
Whenever I am running file, it is stuck even I tried different file but still stuck. --- ## [Unable to parse the xml tag and create a field in Elastic Search](https://discuss.elastic.co/t/unable-to-parse-the-xml-tag-and-create-a-field-in-elastic-search/269645)
**Author:** [@sdeven](https://discuss.elastic.co/u/sdeven)\ **Replies:** 5\ **Last updated:** [April 9, 2021, 4:40pm UTC](https://discuss.elastic.co/t/unable-to-parse-the-xml-tag-and-create-a-field-in-elastic-search/269645 "2021-04-09T16:40:49Z")
Hi, I am using the http plugin in my logstash.conf file, where I am getting a XML document. I have to extract a tag from the XML document and make it as a field in elastic search. I am not getting any errors but the fil… --- ## ["\_jsonparsefailure" Getting Error while reading messages from RabbitMQ](https://discuss.elastic.co/t/jsonparsefailure-getting-error-while-reading-messages-from-rabbitmq/269690)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 6\ **Last updated:** [April 9, 2021, 4:25pm UTC](https://discuss.elastic.co/t/jsonparsefailure-getting-error-while-reading-messages-from-rabbitmq/269690 "2021-04-09T16:25:47Z")
I am reading messages from rabbitmq while running the config file getting error "jsonparsefailure". message like "You have withdrawn RS. xxxfrom ATM.", "You are welcome again", etc... My config file: input { rabbitmq… --- ## [Logstash:event api to get nested fields with attribute values in ruby filter](https://discuss.elastic.co/t/logstash-event-api-to-get-nested-fields-with-attribute-values-in-ruby-filter/269665)
**Author:** [@prathibha](https://discuss.elastic.co/u/prathibha)\ **Replies:** 1\ **Last updated:** [April 9, 2021, 4:10pm UTC](https://discuss.elastic.co/t/logstash-event-api-to-get-nested-fields-with-attribute-values-in-ruby-filter/269665 "2021-04-09T16:10:12Z")
Hello Team, I would like to use an event api inside ruby filter to get nested fields with attribute value. I have a sample here as below: "Event1" =\> { "Event2" =\> { … --- ## [Logstash docker container can not connect to the ElasticSearch (non-docker) on another host](https://discuss.elastic.co/t/logstash-docker-container-can-not-connect-to-the-elasticsearch-non-docker-on-another-host/269730)
**Author:** [@kube-elk](https://discuss.elastic.co/u/kube-elk)\ **Replies:** 0\ **Last updated:** [April 9, 2021, 4:03pm UTC](https://discuss.elastic.co/t/logstash-docker-container-can-not-connect-to-the-elasticsearch-non-docker-on-another-host/269730 "2021-04-09T16:03:52Z")
Hi I have 2 docker containers: filebeat and logstash which I run using docker-compose. I am trying to configure logstash such that it will write index to the Elasticsearch (non-docker) hosted on another node. Here is m… --- ## [Logstash-iterating through array field with specific action for every value](https://discuss.elastic.co/t/logstash-iterating-through-array-field-with-specific-action-for-every-value/269702)
**Author:** [@Jan\_Kabelka](https://discuss.elastic.co/u/Jan_Kabelka)\ **Replies:** 1\ **Last updated:** [April 9, 2021, 3:40pm UTC](https://discuss.elastic.co/t/logstash-iterating-through-array-field-with-specific-action-for-every-value/269702 "2021-04-09T15:40:00Z")
Hi,logstash wizards! I suppose ruby must be used for what I need but I tried many many ways with no success:( I parse DNS responses on Logstash and as a result I have field below with array of values: {"dns": { … --- ## [Is there any other way than rollover to create the indexes on ElasticSearch for continous 2 hours rather than per hour or per day](https://discuss.elastic.co/t/is-there-any-other-way-than-rollover-to-create-the-indexes-on-elasticsearch-for-continous-2-hours-rather-than-per-hour-or-per-day/269668)
**Author:** [@KunwarAkanksha](https://discuss.elastic.co/u/KunwarAkanksha)\ **Replies:** 1\ **Last updated:** [April 9, 2021, 1:54pm UTC](https://discuss.elastic.co/t/is-there-any-other-way-than-rollover-to-create-the-indexes-on-elasticsearch-for-continous-2-hours-rather-than-per-hour-or-per-day/269668 "2021-04-09T13:54:14Z")
I am using logstash to push the data in YYYY.MM.dd.hh format which consider AM and PM in one index means two different hour data is available in same index. I want a index of two hours but those hours need to be continou… --- ## [Parsing variable multiline text from event log](https://discuss.elastic.co/t/parsing-variable-multiline-text-from-event-log/269450)
**Author:** [@SimonR](https://discuss.elastic.co/u/SimonR)\ **Replies:** 3\ **Last updated:** [April 9, 2021, 11:35am UTC](https://discuss.elastic.co/t/parsing-variable-multiline-text-from-event-log/269450 "2021-04-09T11:35:13Z")
Hi Logstash experts. Im having big issues parsing MSSQL logs from the Windows event log, and i hope someone can assist. MSSQL logs event id 33205 to Windows event log, in a somewhat structured format, except the "State… --- ## [Formatted values in logstash are not displayed](https://discuss.elastic.co/t/formatted-values-in-logstash-are-not-displayed/268974)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 3\ **Last updated:** [April 9, 2021, 11:43am UTC](https://discuss.elastic.co/t/formatted-values-in-logstash-are-not-displayed/268974 "2021-04-09T11:43:54Z")
Formatted values in logstash are not displayed. want to I would like to split the log collected by logstash and get it by dissect, and check each in a different field in Kibana. issue However, the variable name o… --- ## [Multiple pipelines](https://discuss.elastic.co/t/multiple-pipelines/269689)
**Author:** [@dddddddddddddddd](https://discuss.elastic.co/u/dddddddddddddddd)\ **Replies:** 0\ **Last updated:** [April 9, 2021, 9:15am UTC](https://discuss.elastic.co/t/multiple-pipelines/269689 "2021-04-09T09:15:29Z")
hi everyone I have multiple sources so I want to parse them with logstash , so if that need multiple pipelines or just a pipeline is enough , in my case I have multiple source ( filebeats , winlogbeats and syslog server… --- ## [Parsing date format](https://discuss.elastic.co/t/parsing-date-format/269488)
**Author:** [@tkkchan](https://discuss.elastic.co/u/tkkchan)\ **Replies:** 2\ **Last updated:** [April 9, 2021, 8:15am UTC](https://discuss.elastic.co/t/parsing-date-format/269488 "2021-04-09T08:15:46Z")
Dear All, I am currently learning how to parse data with logstash and pass them to Logstash. Right now I am confused on how to parse date correctly, as you can see, I have a timestamp with the format yyyy/MM/dd HH:mm:s… --- ## [Traiter des sous champs comme des logs à part entier à ingérer](https://discuss.elastic.co/t/traiter-des-sous-champs-comme-des-logs-a-part-entier-a-ingerer/268771)
**Author:** [@Yahya\_Bouzoubaa](https://discuss.elastic.co/u/Yahya_Bouzoubaa)\ **Replies:** 2\ **Last updated:** [April 9, 2021, 7:17am UTC](https://discuss.elastic.co/t/traiter-des-sous-champs-comme-des-logs-a-part-entier-a-ingerer/268771 "2021-04-09T07:17:29Z")
Bonjour, Je souhaite parser un log et parser aussi un sous log (qui est un champ du log parent) et les ingérer dans deux logs différents. Est ce que c'est possible et si oui quelle méthode ou script faudrait utiliser. … --- ## [Not able to add host name to metric filter plugin output](https://discuss.elastic.co/t/not-able-to-add-host-name-to-metric-filter-plugin-output/269554)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 8\ **Last updated:** [April 9, 2021, 2:42am UTC](https://discuss.elastic.co/t/not-able-to-add-host-name-to-metric-filter-plugin-output/269554 "2021-04-09T02:42:41Z")
Hi, I wanted to see how rate at which the clients were sending logs to logstash. I wanted to use the metric filter plugin. And it is working. Except one thing. Here is the conf: input { http { … --- ## [Collect multiple line in stdin](https://discuss.elastic.co/t/collect-multiple-line-in-stdin/269647)
**Author:** [@elk\_newbie](https://discuss.elastic.co/u/elk_newbie)\ **Replies:** 4\ **Last updated:** [April 8, 2021, 11:46pm UTC](https://discuss.elastic.co/t/collect-multiple-line-in-stdin/269647 "2021-04-08T23:46:34Z")
Hi, Is it possible to read multiple lines at once? The first line indicates the error. I would like to log the statement that causes the error. \< 2021-04-07 10:19:18.883 CEST db dbuser App 37386 \> ERROR: operator… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=238) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=240)