# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=64 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 65 --- ## [Logstash unable to send network log to elastic search database but raw data successfully store in system](https://discuss.elastic.co/t/logstash-unable-to-send-network-log-to-elastic-search-database-but-raw-data-successfully-store-in-system/340243)
**Author:** [@Kiran\_K](https://discuss.elastic.co/u/Kiran_K)\ **Replies:** 1\ **Last updated:** [August 7, 2023, 5:22pm UTC](https://discuss.elastic.co/t/logstash-unable-to-send-network-log-to-elastic-search-database-but-raw-data-successfully-store-in-system/340243 "2023-08-07T17:22:10Z")
\[WARN \] 2023-08-07 08:45:44.506 \[\[main\]-pipeline-manager\] elasticsearch - Detected a 6.x and above cluster: the type event field won't be used to determine the document \_type {:es\_version=\>8} \[INFO \] 2023-08-07 08:45:44… --- ## [How to use event.set to get the values of a variable?](https://discuss.elastic.co/t/how-to-use-event-set-to-get-the-values-of-a-variable/340155)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 1\ **Last updated:** [August 4, 2023, 10:34pm UTC](https://discuss.elastic.co/t/how-to-use-event-set-to-get-the-values-of-a-variable/340155 "2023-08-04T22:34:59Z")
I'm using Kafka's input plugin within my logstash pipline and I have enabled decorated\_events =\> true If I want to get the kafka topic and partition names I can do this: mutate { add\_field =\> { "\[topic\_na… --- ## [Need to email syslog messages from alert](https://discuss.elastic.co/t/need-to-email-syslog-messages-from-alert/340146)
**Author:** [@mmercaldi](https://discuss.elastic.co/u/mmercaldi)\ **Replies:** 0\ **Last updated:** [August 4, 2023, 2:55pm UTC](https://discuss.elastic.co/t/need-to-email-syslog-messages-from-alert/340146 "2023-08-04T14:55:54Z")
I am trying to be alerted whenever a port security issue comes up, however I can only view content on {{context.hits}}. The table for the syslog message contains fields such as @timestamp, @version, host, message I am … --- ## [Getting an error while running the logstash email output plugin - Unknown Garbage collector name- "G1 -Concurrent GC"](https://discuss.elastic.co/t/getting-an-error-while-running-the-logstash-email-output-plugin-unknown-garbage-collector-name-g1-concurrent-gc/340133)
**Author:** [@AKCG23](https://discuss.elastic.co/u/AKCG23)\ **Replies:** 1\ **Last updated:** [August 4, 2023, 2:47pm UTC](https://discuss.elastic.co/t/getting-an-error-while-running-the-logstash-email-output-plugin-unknown-garbage-collector-name-g1-concurrent-gc/340133 "2023-08-04T14:47:18Z")
I Have configured Logstash 7.17.3 and Heart beats 7.17.3. I am trying to send an email alert , if the url returns a code 401 . I have configured the email output plugin. While running the logstash i get this error. \[20… --- ## [Logstash / Beats Encryption Error](https://discuss.elastic.co/t/logstash-beats-encryption-error/340140)
**Author:** [@WLhelp](https://discuss.elastic.co/u/WLhelp)\ **Replies:** 0\ **Last updated:** [August 4, 2023, 1:44pm UTC](https://discuss.elastic.co/t/logstash-beats-encryption-error/340140 "2023-08-04T13:44:09Z")
Hey folks, i have trouble setting up encryption for Beats send to logstash server. Test Config says "ok", test output on the client gives me: logstash: 10.1.7.27:5044... connection... parse host... OK dns lookup...… --- ## [Multipath in the pipeline not working](https://discuss.elastic.co/t/multipath-in-the-pipeline-not-working/339842)
**Author:** [@anupvtr](https://discuss.elastic.co/u/anupvtr)\ **Replies:** 2\ **Last updated:** [August 4, 2023, 1:09pm UTC](https://discuss.elastic.co/t/multipath-in-the-pipeline-not-working/339842 "2023-08-04T13:09:11Z")
Hello All, Thanks in advance. We have succesfully sending the data to the Logz.io console via custom application. There was a specific request to add one more path in addition to the existing path. The logs that are p… --- ## [Can Logstash be setup separately after deploying Elasticsearch using AzureRM template?](https://discuss.elastic.co/t/can-logstash-be-setup-separately-after-deploying-elasticsearch-using-azurerm-template/339152)
**Author:** [@Haralambie\_Lungu](https://discuss.elastic.co/u/Haralambie_Lungu)\ **Replies:** 1\ **Last updated:** [August 4, 2023, 11:59am UTC](https://discuss.elastic.co/t/can-logstash-be-setup-separately-after-deploying-elasticsearch-using-azurerm-template/339152 "2023-08-04T11:59:58Z")
Hi everyone, We have deployed Elasticsearch Self-Managed using the ARM template from Azure Marketplace. We haven't checked Logstash during the setup process, we only created the kibana, master-0,1 and 2 and also the da… --- ## [Logstash:grok:Create a single structure from multiple pattern](https://discuss.elastic.co/t/logstashcreate-a-single-structure-from-multiple-pattern/340098)
**Author:** [@nehag](https://discuss.elastic.co/u/nehag)\ **Replies:** 0\ **Last updated:** [August 4, 2023, 6:15am UTC](https://discuss.elastic.co/t/logstashcreate-a-single-structure-from-multiple-pattern/340098 "2023-08-04T06:15:12Z")
I have logs coming in the following pattern: ================================================================================================== CHECK 1 : Below are the missing Components in the patch =================… --- ## [Grokparse failure even grok debugger fine](https://discuss.elastic.co/t/grokparse-failure-even-grok-debugger-fine/340023)
**Author:** [@shailendra1](https://discuss.elastic.co/u/shailendra1)\ **Replies:** 8\ **Last updated:** [August 4, 2023, 4:17am UTC](https://discuss.elastic.co/t/grokparse-failure-even-grok-debugger-fine/340023 "2023-08-04T04:17:25Z")
Hi All, i am facing the grokparsefailure for my logs even the grok debugger is showing all parsed data but logstash is failing for all fields. below is my filter of logstash filter { grok { … --- ## [Rename json nested fields using mutate](https://discuss.elastic.co/t/rename-json-nested-fields-using-mutate/340063)
**Author:** [@mario\_kazela](https://discuss.elastic.co/u/mario_kazela)\ **Replies:** 4\ **Last updated:** [August 4, 2023, 3:32am UTC](https://discuss.elastic.co/t/rename-json-nested-fields-using-mutate/340063 "2023-08-04T03:32:03Z")
Hi, I have an issue with mutating a nested JSON fields using Logstash. Example of my nested JSON: "test\_results\_result\_legacy\_entities\_hashtags": \[ { "indices": \[ 34, 43 \], "text"… --- ## [Monitoring Oracle Alert log by using Logstash](https://discuss.elastic.co/t/monitoring-oracle-alert-log-by-using-logstash/339889)
**Author:** [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)\ **Replies:** 2\ **Last updated:** [August 3, 2023, 8:01am UTC](https://discuss.elastic.co/t/monitoring-oracle-alert-log-by-using-logstash/339889 "2023-08-03T08:01:45Z")
Hi Team, We had one requirement to monitor oracle alert log by using ELK stack. Could someone guide me . In my environment ELK stack running with 8.x version. Thanks, Debasis --- ## [Secure Logstash and Filebeats communication](https://discuss.elastic.co/t/secure-logstash-and-filebeats-communication/339912)
**Author:** [@Seemant\_Bind](https://discuss.elastic.co/u/Seemant_Bind)\ **Replies:** 4\ **Last updated:** [August 3, 2023, 7:51am UTC](https://discuss.elastic.co/t/secure-logstash-and-filebeats-communication/339912 "2023-08-03T07:51:17Z")
We are working on an integration where we need to take logs from Filebeat through Logstash. However, Filebeat and Logstash are hosted in different networks. In order to secure the communication, we want to implement SSL.… --- ## [How to read logs from newrelic?](https://discuss.elastic.co/t/how-to-read-logs-from-newrelic/339995)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 0\ **Last updated:** [August 3, 2023, 4:28am UTC](https://discuss.elastic.co/t/how-to-read-logs-from-newrelic/339995 "2023-08-03T04:28:46Z")
is there any logstash-input-newrelic plugins for read data from new relic enviornment? --- ## [How to query elasticsearch with array as parameter](https://discuss.elastic.co/t/how-to-query-elasticsearch-with-array-as-parameter/339991)
**Author:** [@rae93](https://discuss.elastic.co/u/rae93)\ **Replies:** 0\ **Last updated:** [August 3, 2023, 3:23am UTC](https://discuss.elastic.co/t/how-to-query-elasticsearch-with-array-as-parameter/339991 "2023-08-03T03:23:50Z")
I have a logstash config like this input { http { port =\> 8092 } } filter { ruby { code =\> ' event.set("\[@metadata\]\[leadArr\]", \[\]) c = event.get("\[@metadata\]\[leads\]") c.each { |value, index| temp = even… --- ## [Logstash failling to make connection to ElasticSearch](https://discuss.elastic.co/t/logstash-failling-to-make-connection-to-elasticsearch/339731)
**Author:** [@Ilyass\_Taybi](https://discuss.elastic.co/u/Ilyass_Taybi)\ **Replies:** 2\ **Last updated:** [July 31, 2023, 11:53pm UTC](https://discuss.elastic.co/t/logstash-failling-to-make-connection-to-elasticsearch/339731 "2023-07-31T23:53:07Z")
Hello, i am having troubles with Logstash for a week now. I do not know why does the error persists. To start Logstash, i use the following command : ./bin/logstash -f /"relative path to the file"/logstash-sample.conf . … --- ## [Using logstash to route APM data to two servers](https://discuss.elastic.co/t/using-logstash-to-route-apm-data-to-two-servers/339977)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 0\ **Last updated:** [August 2, 2023, 10:13pm UTC](https://discuss.elastic.co/t/using-logstash-to-route-apm-data-to-two-servers/339977 "2023-08-02T22:13:20Z")
Im am getting APM data on a APM server, I was wondering if its posible to place a logstash before the APM server, so I can send the same data to another server, so both receive the same data? Something like this: if … --- ## [Ingest Real time logs to elasticsearch using Logstash](https://discuss.elastic.co/t/ingest-real-time-logs-to-elasticsearch-using-logstash/339788)
**Author:** [@harshal](https://discuss.elastic.co/u/harshal)\ **Replies:** 3\ **Last updated:** [August 2, 2023, 1:17pm UTC](https://discuss.elastic.co/t/ingest-real-time-logs-to-elasticsearch-using-logstash/339788 "2023-08-02T13:17:55Z")
I want to Ingest Realtime logs of Apps into Elasticsearch using Logstash and Create Report on Kibana, So Guide me --- ## [Increase in container memory when pipelines reload in logstash](https://discuss.elastic.co/t/increase-in-container-memory-when-pipelines-reload-in-logstash/338738)
**Author:** [@Nikhitha\_Karennagari](https://discuss.elastic.co/u/Nikhitha_Karennagari)\ **Replies:** 11\ **Last updated:** [August 2, 2023, 9:24am UTC](https://discuss.elastic.co/t/increase-in-container-memory-when-pipelines-reload-in-logstash/338738 "2023-08-02T09:24:02Z")
We have a service for which certificate renewal happens for every half an hour. Whenever the certificate renewal happens , when the change is detected in the certificates, automatic reload happens in logstash and all the… --- ## [Unable to create a new Field in Logstash ElasticSearch please help](https://discuss.elastic.co/t/unable-to-create-a-new-field-in-logstash-elasticsearch-please-help/339874)
**Author:** [@Jennifer\_Coley](https://discuss.elastic.co/u/Jennifer_Coley)\ **Replies:** 0\ **Last updated:** [August 2, 2023, 12:58am UTC](https://discuss.elastic.co/t/unable-to-create-a-new-field-in-logstash-elasticsearch-please-help/339874 "2023-08-02T00:58:14Z")
hello sir, I really need an help, I'm new to elasticsearch Kibana but learnt in recent days to understand terms used. I have a Index name "logstash-\*" which receives logs constantly, my task is to filter from all logs … --- ## [Logstash pipeline getting terminated](https://discuss.elastic.co/t/logstash-pipeline-getting-terminated/339871)
**Author:** [@Arinjay\_Jain](https://discuss.elastic.co/u/Arinjay_Jain)\ **Replies:** 0\ **Last updated:** [August 2, 2023, 12:10am UTC](https://discuss.elastic.co/t/logstash-pipeline-getting-terminated/339871 "2023-08-02T00:10:06Z")
Hi Experts, I am running Logstash in a docker container and have the following pipeline configuration. input { tcp { port =\> 5000 codec =\> line } } filter { grok { match =\> {"message… --- ## [Logstash filtering](https://discuss.elastic.co/t/logstash-filtering/339864)
**Author:** [@Jennifer\_Coley](https://discuss.elastic.co/u/Jennifer_Coley)\ **Replies:** 0\ **Last updated:** [August 1, 2023, 9:45pm UTC](https://discuss.elastic.co/t/logstash-filtering/339864 "2023-08-01T21:45:00Z")
In my logstash every second logs will update, In a field name "message" consists group of data like '2023-08-01T21:11:54 \ web.site.com IncomingMax1\[123\] 2023-08-01 11:10:54,123 INFO 987654321 Message.py 12 I… --- ## [CSV and XLS import to Elastic Cloud](https://discuss.elastic.co/t/csv-and-xls-import-to-elastic-cloud/339120)
**Author:** [@Vog93](https://discuss.elastic.co/u/Vog93)\ **Replies:** 14\ **Last updated:** [August 1, 2023, 9:36pm UTC](https://discuss.elastic.co/t/csv-and-xls-import-to-elastic-cloud/339120 "2023-08-01T21:36:08Z")
Hello, I would like to automatically integrate some CSV and XLS files into Elastic Cloud. How could I do this? --- ## [retrieving a date format column in Logstash](https://discuss.elastic.co/t/retrieving-a-date-format-column-in-logstash/339821)
**Author:** [@Amal\_Krizi](https://discuss.elastic.co/u/Amal_Krizi)\ **Replies:** 0\ **Last updated:** [August 1, 2023, 3:22pm UTC](https://discuss.elastic.co/t/retrieving-a-date-format-column-in-logstash/339821 "2023-08-01T15:22:35Z")
good morning, I have a database that contains several columns, including date type columns. I was able to retrieve these date fields via logstach, but one in particular is stuck. This column takes a null date by defaul… --- ## [How to query 3 indexes in logstash](https://discuss.elastic.co/t/how-to-query-3-indexes-in-logstash/339785)
**Author:** [@willsy](https://discuss.elastic.co/u/willsy)\ **Replies:** 0\ **Last updated:** [August 1, 2023, 10:41am UTC](https://discuss.elastic.co/t/how-to-query-3-indexes-in-logstash/339785 "2023-08-01T10:41:50Z")
In logstash i am trying to forward all of the logs in elasticsearch into logstash and then to a third party. What is the correct configuration for the index query? # Sample Logstash configuration for creating a simple #… --- ## [Logstash JDBC Input Plugin Connection Pooling](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-connection-pooling/337823)
**Author:** [@alromos](https://discuss.elastic.co/u/alromos)\ **Replies:** 4\ **Last updated:** [July 31, 2023, 11:08pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-connection-pooling/337823 "2023-07-31T23:08:35Z")
Our team is actively using Logstash JDBC Input plugin with MSSQL JDBC driver for reading some data from DB for further processing. Logstash version: 8.5.1 MSSQL JDBC version: 11.2.1.jre17 At the moment we are facing s… --- ## [Aggregate filter plugin](https://discuss.elastic.co/t/aggregate-filter-plugin/339709)
**Author:** [@pero](https://discuss.elastic.co/u/pero)\ **Replies:** 2\ **Last updated:** [July 31, 2023, 8:06pm UTC](https://discuss.elastic.co/t/aggregate-filter-plugin/339709 "2023-07-31T20:06:22Z")
I have these two json documents Document 1 is { "\_index": "auditbeat-2023.07.31", "\_type": "\_doc", "\_id": "KhknrIkBBEGDHOFEynSE", "\_version": 1, "\_score": null, "\_source": { "ecs": { "version": "1… --- ## [Logstash filter mutate problem](https://discuss.elastic.co/t/logstash-filter-mutate-problem/339690)
**Author:** [@pero](https://discuss.elastic.co/u/pero)\ **Replies:** 12\ **Last updated:** [July 31, 2023, 3:01pm UTC](https://discuss.elastic.co/t/logstash-filter-mutate-problem/339690 "2023-07-31T15:01:14Z")
I have created a filter as shown below filter { if \[application\] == "today" { if field1 { mutate { add\_field =\> { mynewfield =\> "%{\[field1\]}" } … --- ## [Removing prefix from field names](https://discuss.elastic.co/t/removing-prefix-from-field-names/339674)
**Author:** [@VirusProtect](https://discuss.elastic.co/u/VirusProtect)\ **Replies:** 2\ **Last updated:** [July 31, 2023, 1:39pm UTC](https://discuss.elastic.co/t/removing-prefix-from-field-names/339674 "2023-07-31T13:39:33Z")
Hi, I have fields in Kibana such as fw.ip, fw.name, fw.test.old, and so on. I am trying to remove the "fw" prefix from all these fields using a Ruby filter in Logstash. Here's the code I'm using: ruby { code =\> " … --- ## [It is possible to set (logstash.conf )output for particular file location in logstash server](https://discuss.elastic.co/t/it-is-possible-to-set-logstash-conf-output-for-particular-file-location-in-logstash-server/339664)
**Author:** [@rkannan](https://discuss.elastic.co/u/rkannan)\ **Replies:** 2\ **Last updated:** [July 31, 2023, 12:47pm UTC](https://discuss.elastic.co/t/it-is-possible-to-set-logstash-conf-output-for-particular-file-location-in-logstash-server/339664 "2023-07-31T12:47:17Z")
It is possible to set (logstash.conf )output for particular file location in logstash server --- ## [Getting No config files found in path in the cmd](https://discuss.elastic.co/t/getting-no-config-files-found-in-path-in-the-cmd/339684)
**Author:** [@ItsGautam](https://discuss.elastic.co/u/ItsGautam)\ **Replies:** 0\ **Last updated:** [July 31, 2023, 11:23am UTC](https://discuss.elastic.co/t/getting-no-config-files-found-in-path-in-the-cmd/339684 "2023-07-31T11:23:41Z")
Logstash stopped processing because of an error: (SystemExit) exit in the cmd prompt conf file: input { file { type =\> "logs" path =\> "C:\\elk\\elk-stack" start\_position=\>"beginning" codec =\> multiline { pattern… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=63) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=65)