# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=245 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 246 --- ## [Parsing a syslog log](https://discuss.elastic.co/t/parsing-a-syslog-log/267651)
**Author:** [@riahc3](https://discuss.elastic.co/u/riahc3)\ **Replies:** 7\ **Last updated:** [March 22, 2021, 6:21pm UTC](https://discuss.elastic.co/t/parsing-a-syslog-log/267651 "2021-03-22T18:21:46Z")
Im receiving the following log (some things changes for privacy, althought the password is literally sent as asterisks) \<134\> 2021/03/18:13:33:41 SYSTEMNAME 0-PPE-0 : default GUI CMD\_EXECUTED 1324333 0 : User invalid… --- ## [Facing problem during Logstash(version 7.8.0 and 7.8.1) installation on ubuntu](https://discuss.elastic.co/t/facing-problem-during-logstash-version-7-8-0-and-7-8-1-installation-on-ubuntu/267841)
**Author:** [@gaurav1989](https://discuss.elastic.co/u/gaurav1989)\ **Replies:** 2\ **Last updated:** [March 22, 2021, 9:41am UTC](https://discuss.elastic.co/t/facing-problem-during-logstash-version-7-8-0-and-7-8-1-installation-on-ubuntu/267841 "2021-03-22T09:41:17Z")
root@abc:~# apt install logstash=7.8.1 Reading package lists... Done Building dependency tree Reading state information... Done E: Version '7.8.1' for 'logstash' was not found root@abc:~# apt install logstash=7.8.0 … --- ## [Logstash jdbc stuck at \[Api Webserver\] agent - Successfully started Logstash API endpoint {:port=\>9600}](https://discuss.elastic.co/t/logstash-jdbc-stuck-at-api-webserver-agent-successfully-started-logstash-api-endpoint-port-9600/267954)
**Author:** [@Burhanudin](https://discuss.elastic.co/u/Burhanudin)\ **Replies:** 0\ **Last updated:** [March 22, 2021, 8:05am UTC](https://discuss.elastic.co/t/logstash-jdbc-stuck-at-api-webserver-agent-successfully-started-logstash-api-endpoint-port-9600/267954 "2021-03-22T08:05:06Z")
hi all, i got stuck at \[Api Webserver\] agent - Successfully started Logstash API endpoint {:port=\>9600} and nothing happen after this. could anyone help what's wrong with this? and actually before it's running normaly. … --- ## [Logstash config file without input section](https://discuss.elastic.co/t/logstash-config-file-without-input-section/267931)
**Author:** [@nameisnotimportant](https://discuss.elastic.co/u/nameisnotimportant)\ **Replies:** 1\ **Last updated:** [March 22, 2021, 2:24am UTC](https://discuss.elastic.co/t/logstash-config-file-without-input-section/267931 "2021-03-22T02:24:55Z")
Hi All, I am wondering, shall we use a logstash config file without the input section? Apparently, it works without complain. I tried that using settings like below: input { stdin{} } filter { ruby { # here i … --- ## [How to stop logstash when ruby filter encountered error](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899)
**Author:** [@nameisnotimportant](https://discuss.elastic.co/u/nameisnotimportant)\ **Replies:** 4\ **Last updated:** [March 22, 2021, 2:02am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899 "2021-03-22T02:02:57Z")
Hi, May i know how to stop logstash from indexing when error found inside ruby filter? i have following config file which will call external python script to process data and insert the records into elasticsearch. It … --- ## ["reason"=\>"failed to parse field \[Time\] of type \[date\] in document with id 'xxxxxxxxxxxxxxx'](https://discuss.elastic.co/t/reason-failed-to-parse-field-time-of-type-date-in-document-with-id-xxxxxxxxxxxxxxx/267473)
**Author:** [@BlackCat](https://discuss.elastic.co/u/BlackCat)\ **Replies:** 2\ **Last updated:** [March 22, 2021, 12:26am UTC](https://discuss.elastic.co/t/reason-failed-to-parse-field-time-of-type-date-in-document-with-id-xxxxxxxxxxxxxxx/267473 "2021-03-22T00:26:35Z")
I set config of logstash, But happen warning message. Please, how to do resolve. I want to send below log to Elasticsearch. ////////////////// 2021-03-17 10:27:50.115 +0900 26240 main : INFO com.tableausoftware.acti… --- ## [Storing value and attaching it later](https://discuss.elastic.co/t/storing-value-and-attaching-it-later/267892)
**Author:** [@thaideval](https://discuss.elastic.co/u/thaideval)\ **Replies:** 2\ **Last updated:** [March 21, 2021, 4:10pm UTC](https://discuss.elastic.co/t/storing-value-and-attaching-it-later/267892 "2021-03-21T16:10:14Z")
Hello, so i have an event like this: A B С B С .. B С And i can parse it getting "A" value + making an array of "B" and "C". That's how i am handling it right now. But i got curious - if it's possible to store … --- ## [Extracting a field further after its been parsed by Grok Filter](https://discuss.elastic.co/t/extracting-a-field-further-after-its-been-parsed-by-grok-filter/267883)
**Author:** [@rmoss25](https://discuss.elastic.co/u/rmoss25)\ **Replies:** 3\ **Last updated:** [March 21, 2021, 12:44pm UTC](https://discuss.elastic.co/t/extracting-a-field-further-after-its-been-parsed-by-grok-filter/267883 "2021-03-21T12:44:48Z")
HI, Is there a way to parse a field further after its already been parsed by grok? So after I parse out DN from the main message, I now want to parse DN further as it contains a username. So I want to parse out test.T… --- ## [Unable to parse TIMESTAMP\_ISO8601 timestamp in logstash conf](https://discuss.elastic.co/t/unable-to-parse-timestamp-iso8601-timestamp-in-logstash-conf/267886)
**Author:** [@Micheal25](https://discuss.elastic.co/u/Micheal25)\ **Replies:** 2\ **Last updated:** [March 20, 2021, 7:52pm UTC](https://discuss.elastic.co/t/unable-to-parse-timestamp-iso8601-timestamp-in-logstash-conf/267886 "2021-03-20T19:52:59Z")
Hi Team, I am having parsing the "TIMESTAMP\_ISO8601" in logstash grok. The only differnce i see in the log message is extra space after the hours : in timestamp Eg: 2021-03-21T13:(\\s) 01:54.402+0000 this is occuring … --- ## [Multiline and csv filter](https://discuss.elastic.co/t/multiline-and-csv-filter/267888)
**Author:** [@vincedt09](https://discuss.elastic.co/u/vincedt09)\ **Replies:** 0\ **Last updated:** [March 20, 2021, 1:54pm UTC](https://discuss.elastic.co/t/multiline-and-csv-filter/267888 "2021-03-20T13:54:53Z")
Hi! I want to visualize in kibana map geo points with specific information(ex: latitude, longitude) csv { separator =\> "," skip\_empty\_rows =\> true columns =\> \[ "latitude","longitude" \] }… --- ## [Support matrix: luberjack plugin](https://discuss.elastic.co/t/support-matrix-luberjack-plugin/267875)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 1\ **Last updated:** [March 20, 2021, 4:25pm UTC](https://discuss.elastic.co/t/support-matrix-luberjack-plugin/267875 "2021-03-20T16:25:47Z")
Dear elasic colleagues, what I do not understand: You provide a support matrix for the logstash plugins In this matrix I do not find the lumberjack output plugin. This means for me: this plugin is not supported anym… --- ## [Logstash does not read rewritable csv file well](https://discuss.elastic.co/t/logstash-does-not-read-rewritable-csv-file-well/267880)
**Author:** [@Andrew\_Foxis](https://discuss.elastic.co/u/Andrew_Foxis)\ **Replies:** 1\ **Last updated:** [March 20, 2021, 4:20pm UTC](https://discuss.elastic.co/t/logstash-does-not-read-rewritable-csv-file-well/267880 "2021-03-20T16:20:59Z")
Hi all. I have csv file: addr;peer;port;name;datetime (headers) 172.12.10.1; 34.15.67.43; 1123; peter; 2021-03-15 00:02:34 ( value rows (100-200 rows)) This file rewritable every 5 minutes. I want to get the full con… --- ## [Index settings in logstash pipeline](https://discuss.elastic.co/t/index-settings-in-logstash-pipeline/267882)
**Author:** [@Automation\_Scripts](https://discuss.elastic.co/u/Automation_Scripts)\ **Replies:** 0\ **Last updated:** [March 20, 2021, 11:01am UTC](https://discuss.elastic.co/t/index-settings-in-logstash-pipeline/267882 "2021-03-20T11:01:13Z")
Hi guys, I have a logstash pipeline and i want to be able to set the number of replicas to 0 directly from my logstash.conf file. How can I do this? I have seen some docs with template, but is not working. The one from … --- ## [Logstash new install: response code '403' contacting Elasticsearch](https://discuss.elastic.co/t/logstash-new-install-response-code-403-contacting-elasticsearch/267862)
**Author:** [@kmp](https://discuss.elastic.co/u/kmp)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 8:45pm UTC](https://discuss.elastic.co/t/logstash-new-install-response-code-403-contacting-elasticsearch/267862 "2021-03-19T20:45:20Z")
Posting this for the record, should anyone else run into the same problem... LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: Got response code '403' contacting Elasticsearch at URL 'https://es0… --- ## [Managed index templates, composition, and ECS](https://discuss.elastic.co/t/managed-index-templates-composition-and-ecs/266734)
**Author:** [@Supermathie](https://discuss.elastic.co/u/Supermathie)\ **Replies:** 3\ **Last updated:** [March 19, 2021, 8:04pm UTC](https://discuss.elastic.co/t/managed-index-templates-composition-and-ecs/266734 "2021-03-19T20:04:40Z")
We are upgrading to Elastic 7 and at the same time attempting to move our entire pipeline to ECS. I'm wondering how template composition fits in with everything. We primarily store transient logs and I'm wondering what … --- ## [Logstash CSV adding multiple rows by splitting a column value](https://discuss.elastic.co/t/logstash-csv-adding-multiple-rows-by-splitting-a-column-value/267766)
**Author:** [@venkat\_s](https://discuss.elastic.co/u/venkat_s)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 3:22pm UTC](https://discuss.elastic.co/t/logstash-csv-adding-multiple-rows-by-splitting-a-column-value/267766 "2021-03-19T15:22:54Z")
Hello, I am importing csv file to elasticsearch using logstash. I need to duplicate a row based on a comma separated column values. This is how the data looks. I want to add multiple rows based on the number of valu… --- ## [Logstash file input/output with multiline codec](https://discuss.elastic.co/t/logstash-file-input-output-with-multiline-codec/267552)
**Author:** [@liuke](https://discuss.elastic.co/u/liuke)\ **Replies:** 8\ **Last updated:** [March 19, 2021, 3:03pm UTC](https://discuss.elastic.co/t/logstash-file-input-output-with-multiline-codec/267552 "2021-03-19T15:03:50Z")
Hello, i'm new in logstash and i'm encountering some troubles on building a config that works properly with my needings. What i want to do is: -Input files from multiples folders (application logs, so with java excepti… --- ## [\[ERROR\]\[logstash.config.sourceloader\] No configuration found in the configured sources](https://discuss.elastic.co/t/error-logstash-config-sourceloader-no-configuration-found-in-the-configured-sources/267798)
**Author:** [@sourabhjain104](https://discuss.elastic.co/u/sourabhjain104)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 2:57pm UTC](https://discuss.elastic.co/t/error-logstash-config-sourceloader-no-configuration-found-in-the-configured-sources/267798 "2021-03-19T14:57:13Z")
Hello, I am very new to ELK, I am trying to setup ELK (7.11.2) in my local windows 10 laptop. I am able to successfully run Elasticsearch and Kibana but I have issue with Logstash. I tried to figure out this issue by … --- ## [Multiple spliting in a single string using logstash](https://discuss.elastic.co/t/multiple-spliting-in-a-single-string-using-logstash/267817)
**Author:** [@Kiran\_Gupta](https://discuss.elastic.co/u/Kiran_Gupta)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 2:12pm UTC](https://discuss.elastic.co/t/multiple-spliting-in-a-single-string-using-logstash/267817 "2021-03-19T14:12:37Z")
I had a string called regions\_of\_interest which holds the following value "California - San Diego,New Zealand - Auckland,Asia - Hong Kong" I want to split this string multiple times, First with the comma(,) and then wit… --- ## [Modsecurity Json log parsing](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807)
**Author:** [@mirko.spezie](https://discuss.elastic.co/u/mirko.spezie)\ **Replies:** 4\ **Last updated:** [March 19, 2021, 1:35pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807 "2021-03-19T13:35:01Z")
Hi, I'm trying to parse with no success the modsecurity json log. This is the output: { "\_index": "modsecurity-2021.03.17", "\_type": "\_doc", "\_id": "oQRWQngB6w34lfYfAQp6", "\_version": 1, … --- ## [BEATS input: org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: \`string\[\]\`](https://discuss.elastic.co/t/beats-input-org-logstash-fieldreference-illegalsyntaxexception-invalid-fieldreference-string/267781)
**Author:** [@zarak](https://discuss.elastic.co/u/zarak)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 9:46am UTC](https://discuss.elastic.co/t/beats-input-org-logstash-fieldreference-illegalsyntaxexception-invalid-fieldreference-string/267781 "2021-03-19T09:46:57Z")
Hello, I've setup a basic filebeat to logstash infra, but I keep on getting this kind of error in logstash' logs: Mar 19 10:42:33 logstash\[5912\]: \[2021-03-19T10:42:33,892\]\[INFO \]\[org.logstash.beats.BeatsHandler\]\[http-i… --- ## [Removing unwanted data in Filebeats / Logstash](https://discuss.elastic.co/t/removing-unwanted-data-in-filebeats-logstash/267774)
**Author:** [@h\_q](https://discuss.elastic.co/u/h_q)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 9:29am UTC](https://discuss.elastic.co/t/removing-unwanted-data-in-filebeats-logstash/267774 "2021-03-19T09:29:13Z")
I started ingesting Kubernetes logs into ElasticSearch, using the Filebeat Input Container: Filebeat -\> Logstash -\> ElasticSearch My filebeat.yml looks like this: - type: container paths: - /var/lib/docker/conta… --- ## [Help please: logstash start error](https://discuss.elastic.co/t/help-please-logstash-start-error/267630)
**Author:** [@toufiq\_khan](https://discuss.elastic.co/u/toufiq_khan)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 8:08am UTC](https://discuss.elastic.co/t/help-please-logstash-start-error/267630 "2021-03-19T08:08:15Z")
logstash showing this error Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING:… --- ## [Displaying custom data into the SIEM map](https://discuss.elastic.co/t/displaying-custom-data-into-the-siem-map/267735)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 1:32am UTC](https://discuss.elastic.co/t/displaying-custom-data-into-the-siem-map/267735 "2021-03-19T01:32:50Z")
Hi im trying to display data in the SIEM map, using ECS, Im getting the data from a database using logstash JDBC this is the template for the index PUT \_template/ecs { "mappings": { "properties": { "@times… --- ## [Is this going to be efficient elasticsearch output config in logstash?](https://discuss.elastic.co/t/is-this-going-to-be-efficient-elasticsearch-output-config-in-logstash/267734)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 1:32am UTC](https://discuss.elastic.co/t/is-this-going-to-be-efficient-elasticsearch-output-config-in-logstash/267734 "2021-03-19T01:32:04Z")
I am planning to use ILM in logstash. And there is a note saying : You cannot use dynamic variable substitution when ilm\_enabled is true and when using ilm\_rollover\_alias. At the same time this is also there: In order … --- ## [Stop exporting logs after file reaches a certain size](https://discuss.elastic.co/t/stop-exporting-logs-after-file-reaches-a-certain-size/267712)
**Author:** [@agushchin](https://discuss.elastic.co/u/agushchin)\ **Replies:** 3\ **Last updated:** [March 18, 2021, 11:26pm UTC](https://discuss.elastic.co/t/stop-exporting-logs-after-file-reaches-a-certain-size/267712 "2021-03-18T23:26:28Z")
Hi, I have the following question/problem: is there a way to stop logstash sending logs to ES after the log file reaches a certain size limit (or some other limit is reached)? Asking because recently faced an issue caus… --- ## [Unable to load the xml document to elastic search](https://discuss.elastic.co/t/unable-to-load-the-xml-document-to-elastic-search/267403)
**Author:** [@sdeven](https://discuss.elastic.co/u/sdeven)\ **Replies:** 5\ **Last updated:** [March 18, 2021, 10:24pm UTC](https://discuss.elastic.co/t/unable-to-load-the-xml-document-to-elastic-search/267403 "2021-03-18T22:24:15Z")
Hi Team, I am trying to load the whole xml document to elastic search using logstash. I don't see any errors in the logstash console, and the document is not there in elastic search also. my config file below. input { … --- ## [Pipelines stop with Java::OrgJrubyExceptions::SystemCallError](https://discuss.elastic.co/t/pipelines-stop-with-java-systemcallerror/267716)
**Author:** [@manuel.ferreira](https://discuss.elastic.co/u/manuel.ferreira)\ **Replies:** 0\ **Last updated:** [March 18, 2021, 7:40pm UTC](https://discuss.elastic.co/t/pipelines-stop-with-java-systemcallerror/267716 "2021-03-18T19:40:39Z")
Hi. Some light on this issue would be very much appreciated. The scenario is the following. Logstash pipelines fail after docker restart. Logstash : 7.9.3 When logstash exits with an error, the docker configuration … --- ## [Symantec Message Gateway Multiline Logs Parsing](https://discuss.elastic.co/t/symantec-message-gateway-multiline-logs-parsing/267455)
**Author:** [@msszafar](https://discuss.elastic.co/u/msszafar)\ **Replies:** 3\ **Last updated:** [March 18, 2021, 7:02pm UTC](https://discuss.elastic.co/t/symantec-message-gateway-multiline-logs-parsing/267455 "2021-03-18T19:02:43Z")
Hi Community, I've multiline logs of Symantec message gateway (email server). {"port":49542,"host":"10.10.x.x","type":"smg","message":"\<158\>Mar 2 16:21:59 smtp01 bmserver: 1614684119|0a0a5199-534c370000006c49-29-603e… --- ## [Question on logstash removing field from json string and saving back to same field](https://discuss.elastic.co/t/question-on-logstash-removing-field-from-json-string-and-saving-back-to-same-field/267677)
**Author:** [@rgreen](https://discuss.elastic.co/u/rgreen)\ **Replies:** 2\ **Last updated:** [March 18, 2021, 6:57pm UTC](https://discuss.elastic.co/t/question-on-logstash-removing-field-from-json-string-and-saving-back-to-same-field/267677 "2021-03-18T18:57:39Z")
Is it possible to read a json string, remove specified fields then have that json string saved back into a single field with-ought parsing the json data into separate fields. So i would be reading a field from a databas… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=244) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=246)