# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=166 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 167 --- ## [Facing issues in logstash](https://discuss.elastic.co/t/facing-issues-in-logstash/295075)
**Author:** [@Shubham\_Singh](https://discuss.elastic.co/u/Shubham_Singh)\ **Replies:** 1\ **Last updated:** [January 21, 2022, 4:44pm UTC](https://discuss.elastic.co/t/facing-issues-in-logstash/295075 "2022-01-21T16:44:59Z")
/usr/share/logstash/bin/logstash --config.test\_and\_exit -f nginx.conf --path.settings /etc/logstash -----------------Getting this error when i run the logstash --------------------------------------------- Thread.exclu… --- ## [Logstash 8.0 @timestamp with nanoseconds precision](https://discuss.elastic.co/t/logstash-8-0-timestamp-with-nanoseconds-precision/294910)
**Author:** [@ale\_ve](https://discuss.elastic.co/u/ale_ve)\ **Replies:** 1\ **Last updated:** [January 21, 2022, 4:29pm UTC](https://discuss.elastic.co/t/logstash-8-0-timestamp-with-nanoseconds-precision/294910 "2022-01-21T16:29:01Z")
Hi, I'd need help to understand how to set the @timestamp field with the content of another field that contains date and time with nanoseconds precision. I've tried to use the date match but it's not clear to me which p… --- ## [Error Connecting kafka with logsash](https://discuss.elastic.co/t/error-connecting-kafka-with-logsash/295067)
**Author:** [@Avnish\_Singh](https://discuss.elastic.co/u/Avnish_Singh)\ **Replies:** 0\ **Last updated:** [January 21, 2022, 2:42pm UTC](https://discuss.elastic.co/t/error-connecting-kafka-with-logsash/295067 "2022-01-21T14:42:57Z")
OS: Rocky Linux 8.5 Logstash: logstash-7.16.2 Hi, I'm trying to start logstash with kafka as input, here is the config: input { kafka{ bootstrap\_servers =\> "xx.xx.xxx.xx:9092" topics =\> \["ext\_device… --- ## [Indexing flattened field with unique keys is very slow](https://discuss.elastic.co/t/indexing-flattened-field-with-unique-keys-is-very-slow/294833)
**Author:** [@caseydm](https://discuss.elastic.co/u/caseydm)\ **Replies:** 11\ **Last updated:** [January 21, 2022, 1:38pm UTC](https://discuss.elastic.co/t/indexing-flattened-field-with-unique-keys-is-very-slow/294833 "2022-01-21T13:38:20Z")
I have a logstash ingest process that ingests records at 4k events per second. Recently we added field that is an object consisting of unique keys. I save it as a flattened data type. But ingest processing time starts at… --- ## [Logstash - Json Parse Failure and Encoding Problem](https://discuss.elastic.co/t/logstash-json-parse-failure-and-encoding-problem/293447)
**Author:** [@nevincansel](https://discuss.elastic.co/u/nevincansel)\ **Replies:** 7\ **Last updated:** [January 21, 2022, 1:26pm UTC](https://discuss.elastic.co/t/logstash-json-parse-failure-and-encoding-problem/293447 "2022-01-21T13:26:32Z")
Hello, I have a pipeline with tcp input in logstash and pipeline is working without a problem but logs are broken. These are error logs in logstash. 13:34:43.868 \[nioEventLoopGroup-2-2\] WARN logstash.codecs.jsonlines … --- ## [Logstash Indices Rollover to Warm](https://discuss.elastic.co/t/logstash-indices-rollover-to-warm/295058)
**Author:** [@teamomni](https://discuss.elastic.co/u/teamomni)\ **Replies:** 0\ **Last updated:** [January 21, 2022, 1:22pm UTC](https://discuss.elastic.co/t/logstash-indices-rollover-to-warm/295058 "2022-01-21T13:22:36Z")
We have been having issues getting our Logstash data to rollover from hot to warm after 30 days on our Elastic Cloud server. Before we change any settings, we would like to make sure we understand what we are doing. Cur… --- ## [Pipeline in pipe output Logstash](https://discuss.elastic.co/t/pipeline-in-pipe-output-logstash/295032)
**Author:** [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Replies:** 0\ **Last updated:** [January 21, 2022, 8:41am UTC](https://discuss.elastic.co/t/pipeline-in-pipe-output-logstash/295032 "2022-01-21T08:41:16Z")
hello! I would like to know if, like an Elasticsearch output, we can use pipeline =\> "%{\[@metadata\]\[pipeline\]}" for parsing logs, we could use these predefined pipelines in the pipe type output to parse previously parse… --- ## [Logstash configuration to process 1M events per second](https://discuss.elastic.co/t/logstash-configuration-to-process-1m-events-per-second/288989)
**Author:** [@KunwarAkanksha](https://discuss.elastic.co/u/KunwarAkanksha)\ **Replies:** 11\ **Last updated:** [January 21, 2022, 7:23am UTC](https://discuss.elastic.co/t/logstash-configuration-to-process-1m-events-per-second/288989 "2022-01-21T07:23:56Z")
I have events of around 1M (real time), logstash is proecessing 40K at one instances and one pipeline, if I increase the logstash instance to two it goes around 30K each (60K cummulative) but increasing the logstash inst… --- ## [Logstash to cassandra JDBC Input](https://discuss.elastic.co/t/logstash-to-cassandra-jdbc-input/295019)
**Author:** [@Mandark1990](https://discuss.elastic.co/u/Mandark1990)\ **Replies:** 0\ **Last updated:** [January 21, 2022, 6:46am UTC](https://discuss.elastic.co/t/logstash-to-cassandra-jdbc-input/295019 "2022-01-21T06:46:19Z")
Dear All, Am trying to Connect to SSL enabled Cassandra 3.11 by using the Logstash 7.10.2 builds jdbc input plugin. I tried the Steps and the Cassandra Drivers given in the Below links and also copied the cassandra\_dri… --- ## [Distinguishing username from domain\\username using grok](https://discuss.elastic.co/t/distinguishing-username-from-domain-username-using-grok/294988)
**Author:** [@smithtod](https://discuss.elastic.co/u/smithtod)\ **Replies:** 1\ **Last updated:** [January 20, 2022, 10:06pm UTC](https://discuss.elastic.co/t/distinguishing-username-from-domain-username-using-grok/294988 "2022-01-20T22:06:19Z")
I am trying to create a grok statement that will pull the username out of a syslog file. When users enter their authentication information for our VPN login, they can put either domain\\username, or just username, and eit… --- ## [Remove over gsub regex](https://discuss.elastic.co/t/remove-over-gsub-regex/294875)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 2\ **Last updated:** [January 20, 2022, 9:49pm UTC](https://discuss.elastic.co/t/remove-over-gsub-regex/294875 "2022-01-20T21:49:36Z")
Hi How to fix below code for remove all item with word " term" from many of fields, one variant of fields in example line "psm\_info" =\> "7-5-term-term-term-term-term-" I have tried some like this one, but it doesn't … --- ## [How to construct JSON using filter{} and input data?](https://discuss.elastic.co/t/how-to-construct-json-using-filter-and-input-data/294867)
**Author:** [@AshwinMS](https://discuss.elastic.co/u/AshwinMS)\ **Replies:** 3\ **Last updated:** [January 20, 2022, 5:43pm UTC](https://discuss.elastic.co/t/how-to-construct-json-using-filter-and-input-data/294867 "2022-01-20T17:43:53Z")
Hi , im new to logstash and found it to be super useful to loadup data from different sources to an es cluster ! But ive run into one problem which ive been stuck with for a while now and would really love some help. Im… --- ## [Use Logstash for execute enrich policy with HTTP output](https://discuss.elastic.co/t/use-logstash-for-execute-enrich-policy-with-http-output/294975)
**Author:** [@GinkoLucas](https://discuss.elastic.co/u/GinkoLucas)\ **Replies:** 0\ **Last updated:** [January 20, 2022, 4:34pm UTC](https://discuss.elastic.co/t/use-logstash-for-execute-enrich-policy-with-http-output/294975 "2022-01-20T16:34:28Z")
Hello, I need to schedule the execution of an enrich policy. I think i can use Logstash for that, with an HTTP output. Actualy, i have a Logstash who send data to Elasticsearch from Mysql Database. These data are going… --- ## [Multiple Config Files - Duplicate data](https://discuss.elastic.co/t/multiple-config-files-duplicate-data/294898)
**Author:** [@sujeetkp](https://discuss.elastic.co/u/sujeetkp)\ **Replies:** 1\ **Last updated:** [January 20, 2022, 3:31pm UTC](https://discuss.elastic.co/t/multiple-config-files-duplicate-data/294898 "2022-01-20T15:31:01Z")
I am new to logstash and filebeat. I am trying to set up multiple config files for my logstash instance. Using filebeat to send data to logstash. Even if I have filters created for both the logstash config files, I am g… --- ## [Attempted to resurrect connection to dead ES instance, but got an error... Received fatal alert: bad\_certificate"}](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error-received-fatal-alert-bad-certificate/294708)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 2\ **Last updated:** [January 20, 2022, 3:19pm UTC](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error-received-fatal-alert-bad-certificate/294708 "2022-01-20T15:19:05Z")
Hello, I wanted to run the old logstash config that once was working. It is not working anymore I guess that because now Elasticsearch is set up with xpack.security.transport.ssl.verification\_mode: certificate Logsta… --- ## [Looking for a plugin that provides default logstash metrics](https://discuss.elastic.co/t/looking-for-a-plugin-that-provides-default-logstash-metrics/294948)
**Author:** [@Mohitj252](https://discuss.elastic.co/u/Mohitj252)\ **Replies:** 0\ **Last updated:** [January 20, 2022, 1:11pm UTC](https://discuss.elastic.co/t/looking-for-a-plugin-that-provides-default-logstash-metrics/294948 "2022-01-20T13:11:31Z")
Hi Team, we want to analyse the default metrics provided by the logstash, is there any plugin available to help us for collecting the metrics at a single place. --- ## [Logstash Filter help - Newbie question](https://discuss.elastic.co/t/logstash-filter-help-newbie-question/294580)
**Author:** [@PSFletchTheTek](https://discuss.elastic.co/u/PSFletchTheTek)\ **Replies:** 5\ **Last updated:** [January 20, 2022, 11:05am UTC](https://discuss.elastic.co/t/logstash-filter-help-newbie-question/294580 "2022-01-20T11:05:18Z")
Hi All, Sorry I'm just a bit lost on where to start with this one, I was hoping someone would be kind enough to point me in the right direction please? I have a log that looks like this {"host":"192.168.221.11","messa… --- ## [JSON Filter Errors help](https://discuss.elastic.co/t/json-filter-errors-help/294830)
**Author:** [@PSFletchTheTek](https://discuss.elastic.co/u/PSFletchTheTek)\ **Replies:** 10\ **Last updated:** [January 20, 2022, 11:03am UTC](https://discuss.elastic.co/t/json-filter-errors-help/294830 "2022-01-20T11:03:38Z")
Hi All, I'm trying to run the JSON filter on this input with continual errors. Any suggestions on how i do it please? Thanks {"host":"192.168.221.11","message":"HAL GetChassisSlot called from LACPD\_USER(timeout=3000 … --- ## [No cipher suites in common for tcp input](https://discuss.elastic.co/t/no-cipher-suites-in-common-for-tcp-input/52764)
**Author:** [@Jason\_Woodrich](https://discuss.elastic.co/u/Jason_Woodrich)\ **Replies:** 1\ **Last updated:** [January 20, 2022, 9:12am UTC](https://discuss.elastic.co/t/no-cipher-suites-in-common-for-tcp-input/52764 "2022-01-20T09:12:17Z")
I'm trying to configure a tcp syslog input with SSL support and running into some problems. I've installed logstash 2.3.2 using the RPM from the Elastic site. Below is my configuration: input { tcp { port =\> 54… --- ## [Match data from two arrays](https://discuss.elastic.co/t/match-data-from-two-arrays/294488)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 11\ **Last updated:** [January 19, 2022, 9:33pm UTC](https://discuss.elastic.co/t/match-data-from-two-arrays/294488 "2022-01-19T21:33:45Z")
Hi Can You help me for make a method for merge data between two arrays and match this into one document. pattern- \> name of fields \[a-b-c-d-e\] //we have a 3 events separated by the | value\_field = 5-7-10-12-9|5-7-9… --- ## [Error index management Filebeat to Logstash](https://discuss.elastic.co/t/error-index-management-filebeat-to-logstash/294872)
**Author:** [@Adrpan83](https://discuss.elastic.co/u/Adrpan83)\ **Replies:** 0\ **Last updated:** [January 19, 2022, 7:32pm UTC](https://discuss.elastic.co/t/error-index-management-filebeat-to-logstash/294872 "2022-01-19T19:32:16Z")
Hi, I have the following configuration: Filebeat 7.10.1 and Logstash 7.10.1. ERROR instance/beat.go:956 Exiting: Index management requested but the Elasticsearch output is not configured/enabled Exiting: Index man… --- ## [Getting timestamp field from XML attribute](https://discuss.elastic.co/t/getting-timestamp-field-from-xml-attribute/294625)
**Author:** [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\ **Replies:** 10\ **Last updated:** [January 19, 2022, 6:41pm UTC](https://discuss.elastic.co/t/getting-timestamp-field-from-xml-attribute/294625 "2022-01-19T18:41:09Z")
I'm using the xml filter plugin in Logstash to parse a XML document, and am having success with the exception of the @timestamp value, which is found as an attribute value. Here is a snippet of the XML file: \ **Author:** [@dmarshall](https://discuss.elastic.co/u/dmarshall)\ **Replies:** 2\ **Last updated:** [January 19, 2022, 6:28pm UTC](https://discuss.elastic.co/t/using-logstash-to-denomalize-data/294858 "2022-01-19T18:28:43Z") I have data coming in with field names like this: "field\_1" =\> "10" "field\_2" =\> "20" and I'm trying to figure out how to convert it to something more like this: "field" =\> \[ {"num" =\> "1", "val" =\> "10"}, {"num" =\> … --- ## [Automate Logstash Pipelines, Parsing](https://discuss.elastic.co/t/automate-logstash-pipelines-parsing/294845)
**Author:** [@harry\_cook](https://discuss.elastic.co/u/harry_cook)\ **Replies:** 0\ **Last updated:** [January 19, 2022, 3:50pm UTC](https://discuss.elastic.co/t/automate-logstash-pipelines-parsing/294845 "2022-01-19T15:50:57Z")
Hi Folks, Is there a way to automatically generate logstash pipelines based on certain inputs ? Has anyone worked on such use cases ? --- ## [Deleted logstash-plain.log - it's not coming back!](https://discuss.elastic.co/t/deleted-logstash-plain-log-its-not-coming-back/294821)
**Author:** [@PSFletchTheTek](https://discuss.elastic.co/u/PSFletchTheTek)\ **Replies:** 6\ **Last updated:** [January 19, 2022, 2:12pm UTC](https://discuss.elastic.co/t/deleted-logstash-plain-log-its-not-coming-back/294821 "2022-01-19T14:12:31Z")
Hi All, Help, in a process of trying to debug i deleted logstash-plain.log from /var/log/logstash. but its not come back! Any ideas how i get it back please? --- ## [Preserve logs original creation sequence when timestamp is identical](https://discuss.elastic.co/t/preserve-logs-original-creation-sequence-when-timestamp-is-identical/294818)
**Author:** [@alon\_carmelly](https://discuss.elastic.co/u/alon_carmelly)\ **Replies:** 0\ **Last updated:** [January 19, 2022, 1:30pm UTC](https://discuss.elastic.co/t/preserve-logs-original-creation-sequence-when-timestamp-is-identical/294818 "2022-01-19T13:30:20Z")
Hi, I have logs with similar timestamp out of order: I understand logstash does not preserve creation order. I read that using dissect filter together with grok could solve the issue but can't find any examples. This i… --- ## [Logstash not running any query except select version()](https://discuss.elastic.co/t/logstash-not-running-any-query-except-select-version/294809)
**Author:** [@Mizo10](https://discuss.elastic.co/u/Mizo10)\ **Replies:** 0\ **Last updated:** [January 19, 2022, 12:19pm UTC](https://discuss.elastic.co/t/logstash-not-running-any-query-except-select-version/294809 "2022-01-19T12:19:00Z")
Hello, I'm new to logstash, I'm running a logstash pipeline configuration in an aws instance with the following configuration : input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://my.domain:3306/db\_name" … --- ## [Inserting data in Elasticsearch 8.0.0 preview](https://discuss.elastic.co/t/inserting-data-in-elasticsearch-8-0-0-preview/294477)
**Author:** [@Abhishek\_Gautam](https://discuss.elastic.co/u/Abhishek_Gautam)\ **Replies:** 4\ **Last updated:** [January 19, 2022, 9:12am UTC](https://discuss.elastic.co/t/inserting-data-in-elasticsearch-8-0-0-preview/294477 "2022-01-19T09:12:37Z")
Hi team, is there any way to insert bulk data into Elasticsearch 8.0.0 alpha 1, as there isnt any Logstash version compatible with this version. --- ## [Import from mongodb using logstash](https://discuss.elastic.co/t/import-from-mongodb-using-logstash/294770)
**Author:** [@saeed](https://discuss.elastic.co/u/saeed)\ **Replies:** 0\ **Last updated:** [January 19, 2022, 6:29am UTC](https://discuss.elastic.co/t/import-from-mongodb-using-logstash/294770 "2022-01-19T06:29:49Z")
Hi I need to import my documents from mongodb to Elasticsearch and I used logstash. This is my output in CMD when I run this command: C:\\logstash-7.16.3\\bin\\logstash -f c:\\logstash-7.16.3\\bin\\mongo2.conf "Using bundl… --- ## [Logstash hanged, cant be even restarted, Elastic output plugin might be the rascal?](https://discuss.elastic.co/t/logstash-hanged-cant-be-even-restarted-elastic-output-plugin-might-be-the-rascal/294732)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 1\ **Last updated:** [January 18, 2022, 7:26pm UTC](https://discuss.elastic.co/t/logstash-hanged-cant-be-even-restarted-elastic-output-plugin-might-be-the-rascal/294732 "2022-01-18T19:26:22Z")
Hi, I've a configuration that reads from a file and gets its content to a data stream in Elastic. My setup is like that: I have a conf file under /etc/logstash/conf.d/ that listens on a port to get the logs and writes… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=165) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=167)