# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=276 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 277 --- ## [\_split\_type\_failure](https://discuss.elastic.co/t/split-type-failure/256959)
**Author:** [@salma\_widiarti](https://discuss.elastic.co/u/salma_widiarti)\ **Replies:** 6\ **Last updated:** [December 1, 2020, 4:03am UTC](https://discuss.elastic.co/t/split-type-failure/256959 "2020-12-01T04:03:44Z")
Hi all, i have config logstash like this input { http\_poller { urls =\> { users =\> { method =\> GET user =\> "user@gmail.com" password =\> "password" … --- ## [Filebeat to Logatsh, csv header d'ont send in first](https://discuss.elastic.co/t/filebeat-to-logatsh-csv-header-dont-send-in-first/257075)
**Author:** [@Alexis\_Devleeschauwe](https://discuss.elastic.co/u/Alexis_Devleeschauwe)\ **Replies:** 4\ **Last updated:** [December 1, 2020, 11:04am UTC](https://discuss.elastic.co/t/filebeat-to-logatsh-csv-header-dont-send-in-first/257075 "2020-12-01T11:04:34Z")
Hi. Description I need import csv file logs to elasticsearch, for this, have 3 stack Filebeats \> logstash \> elasticsearch. Filebeat read csv , datas is filter by logstash and elasticsearch is stores datas Problem Th… --- ## [Logstash file read , mode read, sincedb, duplicate events after restart logstash](https://discuss.elastic.co/t/logstash-file-read-mode-read-sincedb-duplicate-events-after-restart-logstash/257126)
**Author:** [@slast](https://discuss.elastic.co/u/slast)\ **Replies:** 2\ **Last updated:** [December 1, 2020, 10:15am UTC](https://discuss.elastic.co/t/logstash-file-read-mode-read-sincedb-duplicate-events-after-restart-logstash/257126 "2020-12-01T10:15:12Z")
Hi Guys Can you help me I have a problem with reading files from logstash. After restarting logstash, it rereads the files again and sends events to Elastic. This folder is mounted from another server /opt/Logstore/… --- ## [Use json and plain text beats from filebeat to output to syslog configured in logstash](https://discuss.elastic.co/t/use-json-and-plain-text-beats-from-filebeat-to-output-to-syslog-configured-in-logstash/257186)
**Author:** [@Arjun\_kochhar](https://discuss.elastic.co/u/Arjun_kochhar)\ **Replies:** 0\ **Last updated:** [December 1, 2020, 10:13am UTC](https://discuss.elastic.co/t/use-json-and-plain-text-beats-from-filebeat-to-output-to-syslog-configured-in-logstash/257186 "2020-12-01T10:13:05Z")
Hi, Very new to logstash and have a use case that I want to achieve. I have two different filebeat instances that write to same logstash instance. filebeat instance 1 sends json output and also sets field as json, while … --- ## [How to add a field if the string matches a word in message](https://discuss.elastic.co/t/how-to-add-a-field-if-the-string-matches-a-word-in-message/255804)
**Author:** [@Guhan\_S](https://discuss.elastic.co/u/Guhan_S)\ **Replies:** 2\ **Last updated:** [December 1, 2020, 6:10am UTC](https://discuss.elastic.co/t/how-to-add-a-field-if-the-string-matches-a-word-in-message/255804 "2020-12-01T06:10:40Z")
{"level":30,"time":1605686969640,"pid":1,"hostname":"ip-100-91-70-500","name":"ZA-dev", this is my sample logs, here i am trying to achieve, if level:30 in message it has to create info log-level, i tired filter { … --- ## [Mapping field as geo\_point](https://discuss.elastic.co/t/mapping-field-as-geo-point/257108)
**Author:** [@blitzcrg](https://discuss.elastic.co/u/blitzcrg)\ **Replies:** 2\ **Last updated:** [December 1, 2020, 5:26am UTC](https://discuss.elastic.co/t/mapping-field-as-geo-point/257108 "2020-12-01T05:26:23Z")
I am having some issues with the logstash geoip plugin that I think stem from a lack of understanding with respect to how the elasticsearch output plugin and index mappings interact. Specifically, I need to ensure that s… --- ## [How to rename some fields by using regex statement](https://discuss.elastic.co/t/how-to-rename-some-fields-by-using-regex-statement/257031)
**Author:** [@xrubick](https://discuss.elastic.co/u/xrubick)\ **Replies:** 2\ **Last updated:** [December 1, 2020, 2:45am UTC](https://discuss.elastic.co/t/how-to-rename-some-fields-by-using-regex-statement/257031 "2020-12-01T02:45:04Z")
this's my logstash output: "prometheus" =\> { "metrics" =\> { "container\_cpu\_load\_average\_10s" =\> 0, "container\_cpu\_user\_seconds\_total" =\> 18.15, "container\_cpu\_syst… --- ## [Grok filter does not work pipeline creation error, while its parsing with no issue on grok parser](https://discuss.elastic.co/t/grok-filter-does-not-work-pipeline-creation-error-while-its-parsing-with-no-issue-on-grok-parser/257037)
**Author:** [@amitnirmal01](https://discuss.elastic.co/u/amitnirmal01)\ **Replies:** 1\ **Last updated:** [November 30, 2020, 11:48pm UTC](https://discuss.elastic.co/t/grok-filter-does-not-work-pipeline-creation-error-while-its-parsing-with-no-issue-on-grok-parser/257037 "2020-11-30T23:48:52Z")
\`\`\`%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:loglevel}.\*\\\[%{THREADNAME:thread}.\*IdamAuthHandler.\*\\\].\* - \\\[carrier:%{GREEDYDATA:carrier}\\\]. \*\\\[product:%{ANYDATA:product}\\\].\*\\\[version:%{ANYDATA:version}\].\*\\\[call:%{ANYDATA:… --- ## [Parse KV:s beginning with a \key=value](https://discuss.elastic.co/t/parse-kv-s-beginning-with-a-xxx-key-value/257055)
**Author:** [@Elitlogik](https://discuss.elastic.co/u/Elitlogik)\ **Replies:** 2\ **Last updated:** [November 30, 2020, 10:14pm UTC](https://discuss.elastic.co/t/parse-kv-s-beginning-with-a-xxx-key-value/257055 "2020-11-30T22:14:52Z")
Is there a way to easily remove the leading in a key-value sequence? \key1=value1 key2=value2 key3=value3 Filter looks like this now: filter { kv {} } Thank you for your support! --- ## [Filter individual json fields using json filters](https://discuss.elastic.co/t/filter-individual-json-fields-using-json-filters/257028)
**Author:** [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\ **Replies:** 16\ **Last updated:** [November 30, 2020, 7:34pm UTC](https://discuss.elastic.co/t/filter-individual-json-fields-using-json-filters/257028 "2020-11-30T19:34:20Z")
I'm trying to parse my message into json fields using Json filters but running into issues Error: Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::Configurati… --- ## [How to get data from large JDBC quires?](https://discuss.elastic.co/t/how-to-get-data-from-large-jdbc-quires/254647)
**Author:** [@I\_Hathout](https://discuss.elastic.co/u/I_Hathout)\ **Replies:** 12\ **Last updated:** [November 30, 2020, 6:04pm UTC](https://discuss.elastic.co/t/how-to-get-data-from-large-jdbc-quires/254647 "2020-11-30T18:04:05Z")
Hello, I am trying to get data from a postgres database using JDBC. I managed to get data to stdout by using only 3 fields, but it fails because of the large number of fields in the query. here is the output from runni… --- ## [Dealing with "tags" =\> \[ \[0\] "multiline" \] in xml filter for logstash](https://discuss.elastic.co/t/dealing-with-tags-0-multiline-in-xml-filter-for-logstash/257094)
**Author:** [@amol\_p81](https://discuss.elastic.co/u/amol_p81)\ **Replies:** 2\ **Last updated:** [November 30, 2020, 5:19pm UTC](https://discuss.elastic.co/t/dealing-with-tags-0-multiline-in-xml-filter-for-logstash/257094 "2020-11-30T17:19:22Z")
Hi, I have setup xml filter for logstash (in config file) however how can i avoid below from getting displayed: "tags" =\> \[ \[0\] "multiline" \] Regards Amol --- ## [Logstash aggregate without unique id](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068)
**Author:** [@bbwolf](https://discuss.elastic.co/u/bbwolf)\ **Replies:** 5\ **Last updated:** [November 30, 2020, 5:06pm UTC](https://discuss.elastic.co/t/logstash-aggregate-without-unique-id/257068 "2020-11-30T17:06:17Z")
Hello, supposed il have current log INFO - 12345 - TASK\_START - start INFO - 12345 - SQL - sqlQuery1 - 12 INFO - 12345 - SQL - sqlQuery2 - 34 INFO - 12345 - TASK\_END - end INFO - 12345 - TASK\_START - start INFO - … --- ## [Parse logstash - rfc5424](https://discuss.elastic.co/t/parse-logstash-rfc5424/256992)
**Author:** [@Paulogbr](https://discuss.elastic.co/u/Paulogbr)\ **Replies:** 3\ **Last updated:** [November 30, 2020, 4:35pm UTC](https://discuss.elastic.co/t/parse-logstash-rfc5424/256992 "2020-11-30T16:35:25Z")
Hello guys, I need help to parse the syslog RFC 5424 with logstash. \<190\> 1 2020-11-28T14:18:03-02:00 workstantion1030.teste.corp WSEE - - \[message@01-\] @cee:{"type":"1","timestamp":"4698753981054","host\_name":"","user\_… --- ## [Logstash log unable to complete log read and throws error](https://discuss.elastic.co/t/logstash-log-unable-to-complete-log-read-and-throws-error/256945)
**Author:** [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Replies:** 4\ **Last updated:** [November 30, 2020, 3:12pm UTC](https://discuss.elastic.co/t/logstash-log-unable-to-complete-log-read-and-throws-error/256945 "2020-11-30T15:12:41Z")
I have setup logstash to read log files generated by another application realtime. The input config is like this. it is throwing the below error in logstash log. I am unable to track if it has been able to read the finis… --- ## [Help parse Snort IDS Alert](https://discuss.elastic.co/t/help-parse-snort-ids-alert/256991)
**Author:** [@coolkaveh](https://discuss.elastic.co/u/coolkaveh)\ **Replies:** 2\ **Last updated:** [November 30, 2020, 1:30pm UTC](https://discuss.elastic.co/t/help-parse-snort-ids-alert/256991 "2020-11-30T13:30:27Z")
Please help me to correct my Logstash conf as below: Log: 11/28-06:50:39.263833 \[\*\*\] \[1:31978:5\] OS-OTHER Bash CGI environment variable injection attempt \[\*\*\] \[Classification: Attempted Administrator Privilege Gain\] \[… --- ## [After applied the tutorial "Getting start with Elasticsearch security" Logstash receive Syslog data but Kibana can’t show it](https://discuss.elastic.co/t/after-applied-the-tutorial-getting-start-with-elasticsearch-security-logstash-receive-syslog-data-but-kibana-can-t-show-it/256666)
**Author:** [@Thelmo\_Henrique\_Sant](https://discuss.elastic.co/u/Thelmo_Henrique_Sant)\ **Replies:** 3\ **Last updated:** [November 30, 2020, 11:42am UTC](https://discuss.elastic.co/t/after-applied-the-tutorial-getting-start-with-elasticsearch-security-logstash-receive-syslog-data-but-kibana-can-t-show-it/256666 "2020-11-30T11:42:09Z")
Hi buddies! I getting a problem after applying "Getting start with Elasticsearch security" to my environment. I followed this entire procedure: And everything works fine. Elasticsearch protected with user and passwo… --- ## [Getting fatal: Not a git repository (or any of the parent directories): .git in logstash 7.4](https://discuss.elastic.co/t/getting-fatal-not-a-git-repository-or-any-of-the-parent-directories-git-in-logstash-7-4/257039)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 0\ **Last updated:** [November 30, 2020, 9:08am UTC](https://discuss.elastic.co/t/getting-fatal-not-a-git-repository-or-any-of-the-parent-directories-git-in-logstash-7-4/257039 "2020-11-30T09:08:43Z")
any idea why this comes fatal: Not a git repository (or any of the parent directories): .git sh: git: command not found Thread.exclusive is deprecated, use Thread::Mutex --- ## [Logstash Error after updating form 7.8.0 to 7.10.0, Pipeline error {:pipeline\_id=\>"main", :exception=\> javax.net.ssl.SSLException: failed to initialize the server-side SSL context](https://discuss.elastic.co/t/logstash-error-after-updating-form-7-8-0-to-7-10-0-pipeline-error-pipeline-id-main-exception-javax-net-ssl-sslexception-failed-to-initialize-the-server-side-ssl-context/255212)
**Author:** [@weeam\_Haj\_Ali](https://discuss.elastic.co/u/weeam_Haj_Ali)\ **Replies:** 15\ **Last updated:** [November 30, 2020, 8:13am UTC](https://discuss.elastic.co/t/logstash-error-after-updating-form-7-8-0-to-7-10-0-pipeline-error-pipeline-id-main-exception-javax-net-ssl-sslexception-failed-to-initialize-the-server-side-ssl-context/255212 "2020-11-30T08:13:15Z")
I update ELK stack from 7.8.0 to 7.10.0. I am facing this Error in logstash: \[ERROR\]\[logstash.javapipeline \]\[main\] Pipeline error {:pipeline\_id=\>"main", :exception=\> javax.net.ssl.SSLException: failed to initialize … --- ## [Clone plugin clone - how to use?](https://discuss.elastic.co/t/clone-plugin-clone-how-to-use/257027)
**Author:** [@vaclav1](https://discuss.elastic.co/u/vaclav1)\ **Replies:** 0\ **Last updated:** [November 30, 2020, 7:44am UTC](https://discuss.elastic.co/t/clone-plugin-clone-how-to-use/257027 "2020-11-30T07:44:35Z")
Hello, I tried to use clone plugin to get a single copy of the event. However, it did not work. Version of our elastic stack - 7.9.2. I have ids which are stored in one field - I use split on them and then I drop this… --- ## [Help parse xml from logs file](https://discuss.elastic.co/t/help-parse-xml-from-logs-file/256989)
**Author:** [@Nurlan199206](https://discuss.elastic.co/u/Nurlan199206)\ **Replies:** 2\ **Last updated:** [November 30, 2020, 5:27am UTC](https://discuss.elastic.co/t/help-parse-xml-from-logs-file/256989 "2020-11-30T05:27:45Z")
Help pls parse xml data from log files. i'm trying use xml filter plugin my conf file filter { if "xml" in \[tags\] { xml { namespaces =\> { "soapenv" =\> "http://schemas.xmlsoap.org/soap/envelope/" … --- ## [Logstash and frozen index](https://discuss.elastic.co/t/logstash-and-frozen-index/256593)
**Author:** [@mcantin](https://discuss.elastic.co/u/mcantin)\ **Replies:** 4\ **Last updated:** [November 30, 2020, 4:23am UTC](https://discuss.elastic.co/t/logstash-and-frozen-index/256593 "2020-11-30T04:23:19Z")
Hi, Last week, I froze old indexes (over a month). Everything worked and elasticsearch was faster. But since monday we have this error in loop in Logstash: \[2020-11-24T15:29:42,146\]\[INFO \]\[logstash.outputs.elasticsear… --- ## [Filter conditionals not working as expected](https://discuss.elastic.co/t/filter-conditionals-not-working-as-expected/257002)
**Author:** [@blitzcrg](https://discuss.elastic.co/u/blitzcrg)\ **Replies:** 2\ **Last updated:** [November 29, 2020, 7:44pm UTC](https://discuss.elastic.co/t/filter-conditionals-not-working-as-expected/257002 "2020-11-29T19:44:09Z")
I'm new to logstash, and writing a config to parse logs from the firewall appliance in my home office. I've debugged a few issues with this config already and now I have it to a place where it works without any errors. H… --- ## [An unexpected error occurred! SocketError: initialize: name or service not known](https://discuss.elastic.co/t/an-unexpected-error-occurred-socketerror-initialize-name-or-service-not-known/256981)
**Author:** [@Barak](https://discuss.elastic.co/u/Barak)\ **Replies:** 2\ **Last updated:** [November 29, 2020, 3:23pm UTC](https://discuss.elastic.co/t/an-unexpected-error-occurred-socketerror-initialize-name-or-service-not-known/256981 "2020-11-29T15:23:52Z")
I am using a basic logstash config that used to work: input { s3 { "bucket" =\> "logs-alb-api" "prefix" =\> "AWSLogs/934###" "type" =\> "ELB\_logs" } } filter { m… --- ## [Logstash json line by line](https://discuss.elastic.co/t/logstash-json-line-by-line/256787)
**Author:** [@Didi\_Sisi](https://discuss.elastic.co/u/Didi_Sisi)\ **Replies:** 3\ **Last updated:** [November 29, 2020, 2:44pm UTC](https://discuss.elastic.co/t/logstash-json-line-by-line/256787 "2020-11-29T14:44:45Z")
Hello; I have this logstash.conf file: input { exec { command =\> "python3 news.py" interval =\> 30 codec =\> "json" } } output { stdout { } } without the code =\> json I obtain this result: "message" =\> " {\\n … --- ## [Document\_id is value](https://discuss.elastic.co/t/document-id-is-value/256969)
**Author:** [@salma\_widiarti](https://discuss.elastic.co/u/salma_widiarti)\ **Replies:** 2\ **Last updated:** [November 29, 2020, 12:57am UTC](https://discuss.elastic.co/t/document-id-is-value/256969 "2020-11-29T00:57:12Z")
Hi all, when I run the logstash config, why is my document id its value? Any one can help me? This is my configuration logstash input { http\_poller { urls =\> { users =\> { metho… --- ## [Logstash monitoring data not showing in Stack Monitoring](https://discuss.elastic.co/t/logstash-monitoring-data-not-showing-in-stack-monitoring/256920)
**Author:** [@ffknob](https://discuss.elastic.co/u/ffknob)\ **Replies:** 1\ **Last updated:** [November 28, 2020, 11:05pm UTC](https://discuss.elastic.co/t/logstash-monitoring-data-not-showing-in-stack-monitoring/256920 "2020-11-28T23:05:28Z")
Hello, I'm not managing to get my Logstash metrics to be shown in Kibana's Stack Monitoring UI: Even though it looks like Metricbeat is sending events to the cluster: GET .monitoring-logstash-7-mb-2020.11.27/\_search… --- ## [Logstash json filter plugin values as string](https://discuss.elastic.co/t/logstash-json-filter-plugin-values-as-string/256912)
**Author:** [@nunogt](https://discuss.elastic.co/u/nunogt)\ **Replies:** 4\ **Last updated:** [November 28, 2020, 10:03pm UTC](https://discuss.elastic.co/t/logstash-json-filter-plugin-values-as-string/256912 "2020-11-28T22:03:50Z")
Hi there, Consider the following logstash json filter snippet: json { skip\_on\_invalid\_json =\> true source =\> "message\_body" target =\> "jsondoc" } which results on the following logstash… --- ## [Mutate convert - Logstash shut down](https://discuss.elastic.co/t/mutate-convert-logstash-shut-down/256237)
**Author:** [@JoAnner](https://discuss.elastic.co/u/JoAnner)\ **Replies:** 3\ **Last updated:** [November 28, 2020, 9:35pm UTC](https://discuss.elastic.co/t/mutate-convert-logstash-shut-down/256237 "2020-11-28T21:35:16Z")
This config doesn't work (mutate, convert create shut down). Could somebody explain me a way to resolve this please? # Sample Logstash configuration for creating a simple # Beats -\> Logstash -\> Elasticsearch pipeline. … --- ## [Logstash - Unknown setting 'ssl\_certificate' & 'ssl\_key' for elasticsearch](https://discuss.elastic.co/t/logstash-unknown-setting-ssl-certificate-ssl-key-for-elasticsearch/256887)
**Author:** [@atharvak](https://discuss.elastic.co/u/atharvak)\ **Replies:** 6\ **Last updated:** [November 28, 2020, 10:38am UTC](https://discuss.elastic.co/t/logstash-unknown-setting-ssl-certificate-ssl-key-for-elasticsearch/256887 "2020-11-28T10:38:12Z")
Hi guys, I am trying to connect logstash with elasticsearch that has security enabled. While elasticsearch is running well with the SSL Certificates/keys, the same certificates/keys are not working for the Logstash. Fol… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=275) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=277)