# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=209 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 210 --- ## [Send splunk log using logstash](https://discuss.elastic.co/t/send-splunk-log-using-logstash/279376)
**Author:** [@alipujaistopo](https://discuss.elastic.co/u/alipujaistopo)\ **Replies:** 2\ **Last updated:** [July 22, 2021, 2:05pm UTC](https://discuss.elastic.co/t/send-splunk-log-using-logstash/279376 "2021-07-22T14:05:16Z")
hi eeveryone, i have a splunk server, and i want to migrate splunk's log using logstash to my elasticsearch's server. is it possible? and how it is? --- ## [Advantages of using redis](https://discuss.elastic.co/t/advantages-of-using-redis/279216)
**Author:** [@111387](https://discuss.elastic.co/u/111387)\ **Replies:** 1\ **Last updated:** [July 22, 2021, 1:58pm UTC](https://discuss.elastic.co/t/advantages-of-using-redis/279216 "2021-07-22T13:58:49Z")
Look at some documentation when building the ELK. Sometimes install Redis in front of Logstash. Even though Logstash provides a buffer, is there any advantage to installing Logstash in front? If Redis is present but n… --- ## [Logstash upsert across multiple indexes](https://discuss.elastic.co/t/logstash-upsert-across-multiple-indexes/279307)
**Author:** [@jpchev](https://discuss.elastic.co/u/jpchev)\ **Replies:** 1\ **Last updated:** [July 22, 2021, 1:39pm UTC](https://discuss.elastic.co/t/logstash-upsert-across-multiple-indexes/279307 "2021-07-22T13:39:51Z")
hi there, I need to import data in elasticsearch with an upsert logic, for this I use the following logstash plugin output { elasticsearch { hosts =\> \["localhost:9200"\] index =\> "myindex-{+YYYY.MM.dd}" doc… --- ## [\_grokparsefailure but grok debugger looks good](https://discuss.elastic.co/t/grokparsefailure-but-grok-debugger-looks-good/279269)
**Author:** [@baumi](https://discuss.elastic.co/u/baumi)\ **Replies:** 17\ **Last updated:** [July 22, 2021, 1:14pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-grok-debugger-looks-good/279269 "2021-07-22T13:14:39Z")
Hello everyone, I need some help with my grok pattern, because Logstash is not able to parse it. Kibana and other grok debuggers are able to parse the logs. Sample Log: Mar 10 00:59:50\\t1.1.1.1\\ttest@static\\t0/0/0/000… --- ## [RabbitMQ Input plugin: PKIX path building failed in Logstash 6.8.15](https://discuss.elastic.co/t/rabbitmq-input-plugin-pkix-path-building-failed-in-logstash-6-8-15/279349)
**Author:** [@Pragatheswar](https://discuss.elastic.co/u/Pragatheswar)\ **Replies:** 0\ **Last updated:** [July 22, 2021, 7:23am UTC](https://discuss.elastic.co/t/rabbitmq-input-plugin-pkix-path-building-failed-in-logstash-6-8-15/279349 "2021-07-22T07:23:45Z")
Problem statement Hi guys, I ahve upgraded Logstash from 6.2.2 to 6.5.0. Post that I receive the following error when trying to configure logstash client certificates: \[2018-12-10T22:10:32,949\]\[ERROR\]\[logstash.inputs.… --- ## [\[Logstash\] Aggregate filter - no timeout](https://discuss.elastic.co/t/logstash-aggregate-filter-no-timeout/279318)
**Author:** [@wedkarz014](https://discuss.elastic.co/u/wedkarz014)\ **Replies:** 6\ **Last updated:** [July 22, 2021, 2:24am UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-no-timeout/279318 "2021-07-22T02:24:48Z")
Hi, In my logstash 7.9.3 I would like to create filter similar to that example https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example3 I want to create index w… --- ## [JDBC input is not working when added with Kafka input](https://discuss.elastic.co/t/jdbc-input-is-not-working-when-added-with-kafka-input/279221)
**Author:** [@adityaPsl](https://discuss.elastic.co/u/adityaPsl)\ **Replies:** 3\ **Last updated:** [July 21, 2021, 9:08pm UTC](https://discuss.elastic.co/t/jdbc-input-is-not-working-when-added-with-kafka-input/279221 "2021-07-21T21:08:23Z")
Hello Team, I am trying to configure multiple inputs in a pipeline input { jdbc { } kafka { } kafka { } } but when i run the pipeline only kafka messages i am getting and jdbc input is not working. Could … --- ## [Logstash Filter incoming data](https://discuss.elastic.co/t/logstash-filter-incoming-data/279248)
**Author:** [@Hamza\_El\_Aouane](https://discuss.elastic.co/u/Hamza_El_Aouane)\ **Replies:** 7\ **Last updated:** [July 21, 2021, 5:21pm UTC](https://discuss.elastic.co/t/logstash-filter-incoming-data/279248 "2021-07-21T17:21:27Z")
Hello Everyone, I hope you are all doing well. I am sorry to come here with this question but I could really use some experts help to solve an issue I am facing with Logstash. I will explain my situation as in detail a… --- ## [Geogip data problem](https://discuss.elastic.co/t/geogip-data-problem/279280)
**Author:** [@Betorov](https://discuss.elastic.co/u/Betorov)\ **Replies:** 2\ **Last updated:** [July 21, 2021, 4:56pm UTC](https://discuss.elastic.co/t/geogip-data-problem/279280 "2021-07-21T16:56:15Z")
Hi evreyone, I'am using the geoip filter to get the map value of an ip geoip {source =\> "NATDestinationIP" } It works, it create all type geoip (latitude and longitude.....) but when I look these value on kibana the… --- ## [Logstash for finding combination of numbers](https://discuss.elastic.co/t/logstash-for-finding-combination-of-numbers/279207)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 3\ **Last updated:** [July 21, 2021, 4:50pm UTC](https://discuss.elastic.co/t/logstash-for-finding-combination-of-numbers/279207 "2021-07-21T16:50:53Z")
Hello, I'm looking for a way to write logstash code for a combination of numbers for a project , example if the input is 5 , the output needs to say 1 + 4 , where 1 means "service stopped" and 4 says "inactive" . I … --- ## [How to strip events from message field? Logstash syslog](https://discuss.elastic.co/t/how-to-strip-events-from-message-field-logstash-syslog/279110)
**Author:** [@RaonyO](https://discuss.elastic.co/u/RaonyO)\ **Replies:** 7\ **Last updated:** [July 21, 2021, 2:21pm UTC](https://discuss.elastic.co/t/how-to-strip-events-from-message-field-logstash-syslog/279110 "2021-07-21T14:21:08Z")
hello guys, i'm trying to send logs from a syslog-ng server to logstash and logstash to elasticsearch, but all events are coming in the message field. Is it possible to separate these events to create a new field for eac… --- ## [Logstash goes wrong after add filter](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730)
**Author:** [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\ **Replies:** 26\ **Last updated:** [July 21, 2021, 10:20am UTC](https://discuss.elastic.co/t/logstash-goes-wrong-after-add-filter/278730 "2021-07-21T10:20:07Z")
Hello everyone, I install Logstash in Ubuntu 18 with two conf.d file input and output. Everything run normally. But when I add 10-syslog-filter.conf something went wrong. 02-beat-input.conf input { beats { host… --- ## [How to remove backslash from weird formatted json](https://discuss.elastic.co/t/how-to-remove-backslash-from-weird-formatted-json/279084)
**Author:** [@MaxenceVacheron](https://discuss.elastic.co/u/MaxenceVacheron)\ **Replies:** 4\ **Last updated:** [July 21, 2021, 10:11am UTC](https://discuss.elastic.co/t/how-to-remove-backslash-from-weird-formatted-json/279084 "2021-07-21T10:11:19Z")
Hello :raised\_hand\_with\_fingers\_splayed:, I have the following (valid) formatted json of which I can't change the formatting : \[{ "id": "1", "sn": "00:1E:C0:8D:9A:CD", "log": "{\\"date\\":\\"16\\/04\\/2021\\",\\"t… --- ## [Dissector mapping field not found in event](https://discuss.elastic.co/t/dissector-mapping-field-not-found-in-event/279179)
**Author:** [@gongon](https://discuss.elastic.co/u/gongon)\ **Replies:** 13\ **Last updated:** [July 21, 2021, 6:59am UTC](https://discuss.elastic.co/t/dissector-mapping-field-not-found-in-event/279179 "2021-07-21T06:59:16Z")
Hi, I used the dissect filter and it worked .Now it does not. The only change is the upgrate of mac Os Catalina to mac OS Big Sur v 11.4 A message is displayed : Dissector mapping field not found in event Could you … --- ## [Logstash POS Log Transaction (TLogs) to JSON](https://discuss.elastic.co/t/logstash-pos-log-transaction-tlogs-to-json/279213)
**Author:** [@Anil\_Khiani](https://discuss.elastic.co/u/Anil_Khiani)\ **Replies:** 2\ **Last updated:** [July 21, 2021, 5:45am UTC](https://discuss.elastic.co/t/logstash-pos-log-transaction-tlogs-to-json/279213 "2021-07-21T05:45:59Z")
Hi Everyone, I have below log from IBM 4690 POS systems generating Transaction Logs (TLogs) as below "::p:�:�:�:�:U:!a" ":p::D::�::�::�::!a:U: P:�:�:#:�:�:",":�!8qg4:�:�P:�::�",":�!8qg4:�:�P:�:�F:�",":�!8qg4:�:�:�::d",… --- ## [RUBY Filter to event.set each entity in an Array](https://discuss.elastic.co/t/ruby-filter-to-event-set-each-entity-in-an-array/277749)
**Author:** [@Dallas\_Toth](https://discuss.elastic.co/u/Dallas_Toth)\ **Replies:** 2\ **Last updated:** [July 21, 2021, 12:42am UTC](https://discuss.elastic.co/t/ruby-filter-to-event-set-each-entity-in-an-array/277749 "2021-07-21T00:42:44Z")
I am stuck on how to use a Ruby script to manipulate a field with in an array. If anyone can help with this I need to take the Field "\[body\]\[entities\]\[sentiment\]\[confidence\]" and \* 100 and add that as another Field "\[bod… --- ## [Logstash KV filter time field](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183)
**Author:** [@RaonyO](https://discuss.elastic.co/u/RaonyO)\ **Replies:** 5\ **Last updated:** [July 20, 2021, 7:21pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183 "2021-07-20T19:21:12Z")
hello guys, i'm using kv filter to filter syslogs in logstash to be sent to elasticsearch. but the kv is creating a field with time and data instead of being in value, how to fix this? is reaching the field limit becaus… --- ## [All output pipelines stop when one errors (using pipeline-to-pipeline)](https://discuss.elastic.co/t/all-output-pipelines-stop-when-one-errors-using-pipeline-to-pipeline/279155)
**Author:** [@mgeldert](https://discuss.elastic.co/u/mgeldert)\ **Replies:** 4\ **Last updated:** [July 20, 2021, 5:45pm UTC](https://discuss.elastic.co/t/all-output-pipelines-stop-when-one-errors-using-pipeline-to-pipeline/279155 "2021-07-20T17:45:39Z")
I'm running logstash 7.12.1 in a container based on the standard Docker image, with the New Relic output installed on top. Despite the fact I am using pipeline-to-pipeline, I am hitting a situation where, if one output … --- ## [Logstash xml filter ruby cleanup code help](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099)
**Author:** [@grants](https://discuss.elastic.co/u/grants)\ **Replies:** 3\ **Last updated:** [July 20, 2021, 5:16pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-ruby-cleanup-code-help/279099 "2021-07-20T17:16:48Z")
I'm using the xml logstash filter to parse xml values into json, but I'm having issues indexing them into Elasticsearch due to an issue where a null string gets parsed into an empty object. Source String Snip: \ **Author:** [@London\_Kim](https://discuss.elastic.co/u/London_Kim)\ **Replies:** 1\ **Last updated:** [July 20, 2021, 3:23pm UTC](https://discuss.elastic.co/t/how-to-limit-data-print-when-i-use-es-as-input/279014 "2021-07-20T15:23:39Z") I am using ES as an input tool. I am trying to output data to stdout, but I want to see only about 5 data as a test. But until I stop logstash, logstash continues to load data. How can I control the data for as many cas… --- ## [ELK SQL query](https://discuss.elastic.co/t/elk-sql-query/278934)
**Author:** [@mrunalini](https://discuss.elastic.co/u/mrunalini)\ **Replies:** 1\ **Last updated:** [July 20, 2021, 2:17pm UTC](https://discuss.elastic.co/t/elk-sql-query/278934 "2021-07-20T14:17:12Z")
Hello Team, I want to Execute SQL query on my Index data , where i have 6 records of same transaction id in Index. In each record there is timestamp , so i need to calculate timestamp diff between 1st , 6th and 2dn ,3… --- ## [Date Time parsing problems](https://discuss.elastic.co/t/date-time-parsing-problems/279088)
**Author:** [@cibernicola](https://discuss.elastic.co/u/cibernicola)\ **Replies:** 6\ **Last updated:** [July 20, 2021, 11:38am UTC](https://discuss.elastic.co/t/date-time-parsing-problems/279088 "2021-07-20T11:38:09Z")
Hello I have a bunch of dates to sort, this is the format I have: 2018-05-12 00:00pm GMT (always 00:00pm) 2021-04-01 1:17am GMT If I try to ingest with no template, elasticsearch detects them as a string, but if I map… --- ## [Import data into ES](https://discuss.elastic.co/t/import-data-into-es/279044)
**Author:** [@kfaf](https://discuss.elastic.co/u/kfaf)\ **Replies:** 3\ **Last updated:** [July 20, 2021, 9:29am UTC](https://discuss.elastic.co/t/import-data-into-es/279044 "2021-07-20T09:29:47Z")
Please i really need help. i want to import json file to my ES using logstash. but nothing goes well on my side. this is my logstash.conf: \< input { file { start\_position =\> "beginning" path =\> "C:\\Users\\Downloa… --- ## [Error in adding yaml file for destinantion path in translate filter](https://discuss.elastic.co/t/error-in-adding-yaml-file-for-destinantion-path-in-translate-filter/278890)
**Author:** [@Sarthak\_Mishra](https://discuss.elastic.co/u/Sarthak_Mishra)\ **Replies:** 2\ **Last updated:** [July 20, 2021, 8:14am UTC](https://discuss.elastic.co/t/error-in-adding-yaml-file-for-destinantion-path-in-translate-filter/278890 "2021-07-20T08:14:49Z")
Hello community, I am trying to use translate filter where I add a yaml file with list of ips. When the ips in the csv don't match with IP in yaml file a new field is created "Added\_asset" with value as new. Please help… --- ## [2 configurations for logstash](https://discuss.elastic.co/t/2-configurations-for-logstash/279107)
**Author:** [@creativeguitar](https://discuss.elastic.co/u/creativeguitar)\ **Replies:** 1\ **Last updated:** [July 19, 2021, 8:39pm UTC](https://discuss.elastic.co/t/2-configurations-for-logstash/279107 "2021-07-19T20:39:22Z")
I have 2 configs under /etc/logstash/conf.d/ If I load the configs individually /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/config1.conf and then /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/… --- ## [Logstash to parse nested json arrays that contain different log formats](https://discuss.elastic.co/t/logstash-to-parse-nested-json-arrays-that-contain-different-log-formats/279106)
**Author:** [@JakeKos21](https://discuss.elastic.co/u/JakeKos21)\ **Replies:** 0\ **Last updated:** [July 19, 2021, 7:33pm UTC](https://discuss.elastic.co/t/logstash-to-parse-nested-json-arrays-that-contain-different-log-formats/279106 "2021-07-19T19:33:46Z")
I am seeking assistance with developing/determining the best course of action for ingesting json data from an AWS environment. Ordinarily this would not be difficult for me via Logstash, however the logs contain a neste… --- ## [Need assistance on CEF \_grokparsefailure](https://discuss.elastic.co/t/need-assistance-on-cef-grokparsefailure/278270)
**Author:** [@Xor44](https://discuss.elastic.co/u/Xor44)\ **Replies:** 7\ **Last updated:** [July 19, 2021, 6:34pm UTC](https://discuss.elastic.co/t/need-assistance-on-cef-grokparsefailure/278270 "2021-07-19T18:34:51Z")
Hi Folk , I'm new on ELK since many days I have configure a pipeline for parsing CEF logs from netscout device . When I'm checking logs I see some \_grokparsefailure tage on Kibana . I conclude that there is some issue… --- ## [Return result of a custom field using query\_template](https://discuss.elastic.co/t/return-result-of-a-custom-field-using-query-template/279051)
**Author:** [@martb](https://discuss.elastic.co/u/martb)\ **Replies:** 0\ **Last updated:** [July 19, 2021, 11:41am UTC](https://discuss.elastic.co/t/return-result-of-a-custom-field-using-query-template/279051 "2021-07-19T11:41:17Z")
Hi, I've created a query\_template that gets the average number of alerts per day, but it's not returning the avg\_count result to logstash. I done this before when the field exists but not with a custom result i.e not a … --- ## [Logstash log4j2.properties configuration for pipeline logs](https://discuss.elastic.co/t/logstash-log4j2-properties-configuration-for-pipeline-logs/278612)
**Author:** [@Kshema](https://discuss.elastic.co/u/Kshema)\ **Replies:** 3\ **Last updated:** [July 19, 2021, 11:11am UTC](https://discuss.elastic.co/t/logstash-log4j2-properties-configuration-for-pipeline-logs/278612 "2021-07-19T11:11:10Z")
In our application, logstash.yml is configured to produce separate log files for each pipeline with the help of below property pipeline.separate\_logs: true This configuration setup is creating the below mentioned logs: … --- ## [Forward azure logs with syslog?](https://discuss.elastic.co/t/forward-azure-logs-with-syslog/279022)
**Author:** [@JaVa](https://discuss.elastic.co/u/JaVa)\ **Replies:** 0\ **Last updated:** [July 19, 2021, 6:57am UTC](https://discuss.elastic.co/t/forward-azure-logs-with-syslog/279022 "2021-07-19T06:57:59Z")
Sorry for the dumb questions (newbie here) but is there any way of forwarding azure logs from Logstash to another SIEM product? I configured our ELK stack to receive Azure events from the cloud with Logstash azure module… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=208) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=210)