# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=292 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 293 --- ## [Logstash TCP Output Plugin Reconnect Interval Issue](https://discuss.elastic.co/t/logstash-tcp-output-plugin-reconnect-interval-issue/251788)
**Author:** [@Paddy\_F](https://discuss.elastic.co/u/Paddy_F)\ **Replies:** 4\ **Last updated:** [October 15, 2020, 1:18pm UTC](https://discuss.elastic.co/t/logstash-tcp-output-plugin-reconnect-interval-issue/251788 "2020-10-15T13:18:46Z")
Hi, I'm having a problem with the logstash tcp output plugin, the reconnect\_interval does not seem to have an affect on the plugin's actual reconnect attempts? I may be misunderstanding what the field is used for exactl… --- ## [Logstash not starting after enabling xpac security](https://discuss.elastic.co/t/logstash-not-starting-after-enabling-xpac-security/252183)
**Author:** [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Replies:** 1\ **Last updated:** [October 15, 2020, 12:43pm UTC](https://discuss.elastic.co/t/logstash-not-starting-after-enabling-xpac-security/252183 "2020-10-15T12:43:30Z")
Hello Team, We are using ELK with 3 node cluster. Earlier we were using elasticsearch6.4 but now we upgraded to version6.8, because we want to use role base access feature. We upgraded successfully our Elasticsaerch n… --- ## [Error Messages in Logstash Logfile io.netty.channel](https://discuss.elastic.co/t/error-messages-in-logstash-logfile-io-netty-channel/252189)
**Author:** [@bateskevin](https://discuss.elastic.co/u/bateskevin)\ **Replies:** 0\ **Last updated:** [October 15, 2020, 12:20pm UTC](https://discuss.elastic.co/t/error-messages-in-logstash-logfile-io-netty-channel/252189 "2020-10-15T12:20:17Z")
This issue was closed without answers... That's why I am reopening this here again. Here is the old issue: https://discuss.elastic.co/t/error-messages-in-logstash-logfile-io-netty-channel/248660 Hi, I am recieving the… --- ## [Logstash conf file issues (using docker container)](https://discuss.elastic.co/t/logstash-conf-file-issues-using-docker-container/252083)
**Author:** [@saj](https://discuss.elastic.co/u/saj)\ **Replies:** 8\ **Last updated:** [October 15, 2020, 10:36am UTC](https://discuss.elastic.co/t/logstash-conf-file-issues-using-docker-container/252083 "2020-10-15T10:36:07Z")
Hi, I am new to elk stuff, I am running 3 separate docker containers (i.e. elasticsearch, kibana, and logstash). I need to specify my logstash.conf but its not working out using following docker logstash command ( I am… --- ## [How to create a logtash conf file for import json file to elasticseaarch using logstash](https://discuss.elastic.co/t/how-to-create-a-logtash-conf-file-for-import-json-file-to-elasticseaarch-using-logstash/252168)
**Author:** [@Manuang](https://discuss.elastic.co/u/Manuang)\ **Replies:** 0\ **Last updated:** [October 15, 2020, 10:20am UTC](https://discuss.elastic.co/t/how-to-create-a-logtash-conf-file-for-import-json-file-to-elasticseaarch-using-logstash/252168 "2020-10-15T10:20:24Z")
I am new to logstash and elastic search. i want to import a json file to elastic search using logstash in my windows 10 system. how to create a logtash conf file for the below json. suggest me a sample config file. json… --- ## [Use a logstash plugin within iteration](https://discuss.elastic.co/t/use-a-logstash-plugin-within-iteration/252159)
**Author:** [@parosio](https://discuss.elastic.co/u/parosio)\ **Replies:** 0\ **Last updated:** [October 15, 2020, 9:21am UTC](https://discuss.elastic.co/t/use-a-logstash-plugin-within-iteration/252159 "2020-10-15T09:21:59Z")
Hello, I've got documents where a field might contain multiple values. All the values from that field have to be looked up in a different index, to get the document enriched. "countries" : "India|Congo|Italy" I shoul… --- ## [How to add Date in mutate add field](https://discuss.elastic.co/t/how-to-add-date-in-mutate-add-field/251601)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 10\ **Last updated:** [October 15, 2020, 5:46am UTC](https://discuss.elastic.co/t/how-to-add-date-in-mutate-add-field/251601 "2020-10-15T05:46:21Z")
Hi Team I am using logstash v7.8 to extract data from ES v7.8 index and inserting respective records to ES v7.8 index-2. While doing so , I want to use filter --\> mutate --\> add\_field ==\> { "doc\_date" =\> "now-1d" } Ho… --- ## [How to convert format "yyyy-MM-dd HH:mm:ss in logstash?](https://discuss.elastic.co/t/how-to-convert-format-yyyy-mm-dd-hhss-in-logstash/252005)
**Author:** [@pamiers](https://discuss.elastic.co/u/pamiers)\ **Replies:** 2\ **Last updated:** [October 15, 2020, 2:28am UTC](https://discuss.elastic.co/t/how-to-convert-format-yyyy-mm-dd-hhss-in-logstash/252005 "2020-10-15T02:28:12Z")
I fetch the data through Oracle, and the database has a field updated\_at in the format "yyyy-MM-dd HH:mm:ss". However, it has been automatically changed to ISO8601 format in logstash. So I try to change the format "yyyy-… --- ## [Remove field on logstash](https://discuss.elastic.co/t/remove-field-on-logstash/252088)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [October 14, 2020, 6:38pm UTC](https://discuss.elastic.co/t/remove-field-on-logstash/252088 "2020-10-14T18:38:49Z")
I am receiving metricbeat data on logstash and I want to remove certain part of it. how can I remove this I try mutate { remove\_field =\> \[ "\[process\]\[args\]0","\[process\]\[args\]1" \] } try mutate { remove\_field =\> \[ "\[p… --- ## [Elastic\_app\_search set engine based on document value](https://discuss.elastic.co/t/elastic-app-search-set-engine-based-on-document-value/252079)
**Author:** [@Tam2](https://discuss.elastic.co/u/Tam2)\ **Replies:** 1\ **Last updated:** [October 14, 2020, 5:24pm UTC](https://discuss.elastic.co/t/elastic-app-search-set-engine-based-on-document-value/252079 "2020-10-14T17:24:55Z")
Within my document, there is a client\_id field I would like to set the engine for my elastic\_app\_search based on this value. So each document goes into the correct engine based on the document's client\_id I tried to do… --- ## [A plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/251913)
**Author:** [@sukumar.koganti](https://discuss.elastic.co/u/sukumar.koganti)\ **Replies:** 12\ **Last updated:** [October 14, 2020, 5:20pm UTC](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/251913 "2020-10-14T17:20:08Z")
Hi Team, I am trying to read my Sql Server table from the DB and here is my .conf file, when i am trying to read the table through logstash i am getting the below error. input { jdbc { jdbc\_driver\_library =\> "C:/Pr… --- ## [Read the Logevent into exec output](https://discuss.elastic.co/t/read-the-logevent-into-exec-output/252077)
**Author:** [@praveenmak](https://discuss.elastic.co/u/praveenmak)\ **Replies:** 0\ **Last updated:** [October 14, 2020, 4:57pm UTC](https://discuss.elastic.co/t/read-the-logevent-into-exec-output/252077 "2020-10-14T16:57:09Z")
Hello folks, I want to get the whole event to a shell script, I am trying to use exec output plugin. Please note, I don't have "message" variable. This is the output in JSON. {"taskid":"vMUKrDoBSX-12349","@version":… --- ## [How to retrieve organizations IP address](https://discuss.elastic.co/t/how-to-retrieve-organizations-ip-address/251911)
**Author:** [@witcher](https://discuss.elastic.co/u/witcher)\ **Replies:** 15\ **Last updated:** [October 14, 2020, 12:56pm UTC](https://discuss.elastic.co/t/how-to-retrieve-organizations-ip-address/251911 "2020-10-14T12:56:33Z")
I am trying to create a visualization which shows the organization the IP address. --- ## [How to work on object to get 0,1,2,3 out of it](https://discuss.elastic.co/t/how-to-work-on-object-to-get-0-1-2-3-out-of-it/252040)
**Author:** [@amaleswar](https://discuss.elastic.co/u/amaleswar)\ **Replies:** 1\ **Last updated:** [October 14, 2020, 3:30pm UTC](https://discuss.elastic.co/t/how-to-work-on-object-to-get-0-1-2-3-out-of-it/252040 "2020-10-14T15:30:44Z")
"eventInfo": { "delete.user.0.firstname": "david", "delete.user.0.lastname": "flank", "delete.user.0.city": "austin", "delete.user.1.firstname": "martin", "delete.user.1.lastname": "king", "delete.user.1.city": "da… --- ## [How to change two fields in an index to a geo point?](https://discuss.elastic.co/t/how-to-change-two-fields-in-an-index-to-a-geo-point/252033)
**Author:** [@witcher](https://discuss.elastic.co/u/witcher)\ **Replies:** 5\ **Last updated:** [October 14, 2020, 3:19pm UTC](https://discuss.elastic.co/t/how-to-change-two-fields-in-an-index-to-a-geo-point/252033 "2020-10-14T15:19:47Z")
PUT \_ingest/pipeline/geoip { "description" : "Add geoip info", "processors" : \[ { "geoip" : { "field" : "source\_ip", "database\_file": "GeoLite2-City.mmdb", "target\_field": "sourceip\_geo" } } \] } … --- ## [Logstash - configuration with multiple sources](https://discuss.elastic.co/t/logstash-configuration-with-multiple-sources/252017)
**Author:** [@maria\_lopez\_perez](https://discuss.elastic.co/u/maria_lopez_perez)\ **Replies:** 2\ **Last updated:** [October 14, 2020, 3:18pm UTC](https://discuss.elastic.co/t/logstash-configuration-with-multiple-sources/252017 "2020-10-14T15:18:32Z")
Hello, I try to create a configuration in Logstash in order to have two index in elasticserach relation to two diferentes sources (two files), The configuration is: input { file { path =\> "/etc/logstash/ficheros/disc… --- ## [Logstash dash "-" (hyphen) problem with aggregation](https://discuss.elastic.co/t/logstash-dash-hyphen-problem-with-aggregation/251947)
**Author:** [@bnmtl](https://discuss.elastic.co/u/bnmtl)\ **Replies:** 8\ **Last updated:** [October 14, 2020, 2:39pm UTC](https://discuss.elastic.co/t/logstash-dash-hyphen-problem-with-aggregation/251947 "2020-10-14T14:39:44Z")
Hello, With my logstash configuration, I am reading JSON log, but in my JSON log I have fields like below; "host-name":"comp-1.example.com", "name-surname":"john-doe" My configuration in logstash; input { file {… --- ## [Getting error while starting logstash - Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAct ion::Create\, action\_result: false", :backtrace=\>nil](https://discuss.elastic.co/t/getting-error-while-starting-logstash-failed-to-execute-action-id-main-action-type-logstash-failedaction-message-could-not-execute-action-pipelineact-ion-create-main-action-result-false-backtrace-nil/252044)
**Author:** [@kanhaiyak424](https://discuss.elastic.co/u/kanhaiyak424)\ **Replies:** 0\ **Last updated:** [October 14, 2020, 12:47pm UTC](https://discuss.elastic.co/t/getting-error-while-starting-logstash-failed-to-execute-action-id-main-action-type-logstash-failedaction-message-could-not-execute-action-pipelineact-ion-create-main-action-result-false-backtrace-nil/252044 "2020-10-14T12:47:48Z")
E:\\ELK\\Software\\logstash-7.9.1\\bin\>logstash -f logstash-simple-jms.conf Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. … --- ## [Logstash Pipeline Monitoring Data Is Blank In Kibana](https://discuss.elastic.co/t/logstash-pipeline-monitoring-data-is-blank-in-kibana/251398)
**Author:** [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\ **Replies:** 0\ **Last updated:** [October 8, 2020, 8:47am UTC](https://discuss.elastic.co/t/logstash-pipeline-monitoring-data-is-blank-in-kibana/251398 "2020-10-08T08:47:18Z")
Hi All, I have 5 pipelines for logstash, out of which 4 are working fine, however for one pipeline I can see just blank in stack monitoring. I don't know where to check the error for this as well --- ## [Add field based on another](https://discuss.elastic.co/t/add-field-based-on-another/250511)
**Author:** [@sandikata](https://discuss.elastic.co/u/sandikata)\ **Replies:** 9\ **Last updated:** [October 14, 2020, 11:24am UTC](https://discuss.elastic.co/t/add-field-based-on-another/250511 "2020-10-14T11:24:07Z")
Hello. I'd like to know if there's any possibility to add new field based on request filed, but with some regex. (%{NOTSPACE:request}\[?#$\]\\S\*) i'd like to add this one as field "request\_file" and the full output to st… --- ## [Logstash grokparsefailure](https://discuss.elastic.co/t/logstash-grokparsefailure/250598)
**Author:** [@sandikata](https://discuss.elastic.co/u/sandikata)\ **Replies:** 3\ **Last updated:** [October 14, 2020, 11:22am UTC](https://discuss.elastic.co/t/logstash-grokparsefailure/250598 "2020-10-14T11:22:53Z")
Hello! Is it there any way (human readable) to debug what is causing grokparsefailure? There's my nginx filter conf filter { if "nginx\_access" in \[tags\] { grok { patterns\_dir =\> \["/etc/logstash/patterns"\] ma… --- ## [EOF in Logstash File Input](https://discuss.elastic.co/t/eof-in-logstash-file-input/251126)
**Author:** [@sfischer](https://discuss.elastic.co/u/sfischer)\ **Replies:** 4\ **Last updated:** [October 14, 2020, 8:12am UTC](https://discuss.elastic.co/t/eof-in-logstash-file-input/251126 "2020-10-14T08:12:09Z")
Hello, I am using Logstash to aggregate some data out of a log file and store it in elastic search, to get some usage data for a software. We copy the log files into a directory that logstash checks. We use the file in… --- ## [Unable to pull aws cloud data using S3 plugin/ Error : something is wrong with your configuration](https://discuss.elastic.co/t/unable-to-pull-aws-cloud-data-using-s3-plugin-error-something-is-wrong-with-your-configuration/251889)
**Author:** [@sainath](https://discuss.elastic.co/u/sainath)\ **Replies:** 2\ **Last updated:** [October 14, 2020, 7:18am UTC](https://discuss.elastic.co/t/unable-to-pull-aws-cloud-data-using-s3-plugin-error-something-is-wrong-with-your-configuration/251889 "2020-10-14T07:18:18Z")
Hi , we are trying to pull S3 bucket data from AWS cloud and below is the configuration. input { s3 { "access\_key\_id" =\> "AXXXXX" "secret\_access\_key" =\> "XXXX" "region" =\> "us-east-2" "bucket" =\> "XXX… --- ## [\_grokparsefailure and I have no idea](https://discuss.elastic.co/t/grokparsefailure-and-i-have-no-idea/251984)
**Author:** [@EunBae\_Park](https://discuss.elastic.co/u/EunBae_Park)\ **Replies:** 1\ **Last updated:** [October 14, 2020, 6:37am UTC](https://discuss.elastic.co/t/grokparsefailure-and-i-have-no-idea/251984 "2020-10-14T06:37:10Z")
Hi, I'd tried to parse NGINX Access log whose log format is '$http\_x\_forwarded\_for - $http\_referer - \[$time\_local\] "$request" $status $body\_bytes\_sent'. The logs are like below: - - - - \[08/Oct/2020:11:05:50 +0900\] "… --- ## [Can't Get Logs From Multiple Machines](https://discuss.elastic.co/t/cant-get-logs-from-multiple-machines/251991)
**Author:** [@Redwanuzzaman](https://discuss.elastic.co/u/Redwanuzzaman)\ **Replies:** 0\ **Last updated:** [October 14, 2020, 4:50am UTC](https://discuss.elastic.co/t/cant-get-logs-from-multiple-machines/251991 "2020-10-14T04:50:21Z")
Hello There, I am trying to get logs from different machines. I installed filebeat on the client machine and make this configuration. - type: log enabled: true paths: - /path/where/log/stores/in/client/machine … --- ## [Logstash Configuration for Writing to a File (that may grow in GBs)](https://discuss.elastic.co/t/logstash-configuration-for-writing-to-a-file-that-may-grow-in-gbs/251976)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 0\ **Last updated:** [October 14, 2020, 1:07am UTC](https://discuss.elastic.co/t/logstash-configuration-for-writing-to-a-file-that-may-grow-in-gbs/251976 "2020-10-14T01:07:41Z")
I have a requirement where in my logstash I have to write to Elasticsearch and as well write to a file. However the issue is, the file will be created on daily basis, and it is calculated that the file will grow in GBs. … --- ## [Java Memory Error From a Ruby Script?](https://discuss.elastic.co/t/java-memory-error-from-a-ruby-script/251611)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 3\ **Last updated:** [October 13, 2020, 9:25pm UTC](https://discuss.elastic.co/t/java-memory-error-from-a-ruby-script/251611 "2020-10-13T21:25:39Z")
Hi Logstash, I’m running the Logstash 7.7.0 Docker container, spun up on an Ubuntu machine. In my LS filter config, I “bounce” all of my records off an external C program via a TCP socket: ruby { init =\> " … --- ## [Logstash conditional test for value in a field](https://discuss.elastic.co/t/logstash-conditional-test-for-value-in-a-field/251968)
**Author:** [@Zorkmid](https://discuss.elastic.co/u/Zorkmid)\ **Replies:** 1\ **Last updated:** [October 13, 2020, 8:45pm UTC](https://discuss.elastic.co/t/logstash-conditional-test-for-value-in-a-field/251968 "2020-10-13T20:45:52Z")
Hello all, Is the follow correct for testing for the presence of a value and then applying the appropriate grok filter? The event types, urlfLog, accessLog all arrive in the same syslog input stream and I'd like to tes… --- ## [Ruby Filter to clean nulls and nils](https://discuss.elastic.co/t/ruby-filter-to-clean-nulls-and-nils/251966)
**Author:** [@mistrhanky](https://discuss.elastic.co/u/mistrhanky)\ **Replies:** 0\ **Last updated:** [October 13, 2020, 8:25pm UTC](https://discuss.elastic.co/t/ruby-filter-to-clean-nulls-and-nils/251966 "2020-10-13T20:25:58Z")
I grabbed a function from a thread somewhere that uses ruby to "clean" off nulls, nills, dashes, etc so that I dont needlessly clutter ES or import them. It has worked well up until now. Now however, I have a bit of a na… --- ## [Help with Logstash filter](https://discuss.elastic.co/t/help-with-logstash-filter/251825)
**Author:** [@earlsanchez](https://discuss.elastic.co/u/earlsanchez)\ **Replies:** 2\ **Last updated:** [October 13, 2020, 6:47pm UTC](https://discuss.elastic.co/t/help-with-logstash-filter/251825 "2020-10-13T18:47:21Z")
Hi, I'm new to ELK and Grok. I'm using ES 7.7 and logstash 7.9. Application logs are coming from Filebeat \> Logstash \> ES. My logs are being indexed into ES and viewable from Kibana. Issue is my logstash filter is not c… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=291) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=293)