# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=137 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 138 --- ## [The indices which match this index pattern don't contain any time fields](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/306146)
**Author:** [@danconcru](https://discuss.elastic.co/u/danconcru)\ **Replies:** 0\ **Last updated:** [June 1, 2022, 4:12pm UTC](https://discuss.elastic.co/t/the-indices-which-match-this-index-pattern-dont-contain-any-time-fields/306146 "2022-06-01T16:12:23Z")
Hi everywhere, I know that this error is posted in another post, but I am tried everything without success. I have a fluentbit that create document reading differents logs from kubernet and send it's to elastcisearch. … --- ## [Generic filter for Messages containing JSON](https://discuss.elastic.co/t/generic-filter-for-messages-containing-json/304810)
**Author:** [@cRaZyT](https://discuss.elastic.co/u/cRaZyT)\ **Replies:** 4\ **Last updated:** [June 1, 2022, 12:56pm UTC](https://discuss.elastic.co/t/generic-filter-for-messages-containing-json/304810 "2022-06-01T12:56:33Z")
Hi, I have the following initial situation, we get various log messages which have JSON content, they look like this, for example: message:Log Text case1 #json: {"case1-total":26,"case-valid":8,"case-duplicates":16,"c… --- ## [Logstash empty range in char class error](https://discuss.elastic.co/t/logstash-empty-range-in-char-class-error/306103)
**Author:** [@The-morpho](https://discuss.elastic.co/u/The-morpho)\ **Replies:** 0\ **Last updated:** [June 1, 2022, 9:47am UTC](https://discuss.elastic.co/t/logstash-empty-range-in-char-class-error/306103 "2022-06-01T09:47:18Z")
Hello, I'm new to Elasticsearch I'm working with log files comming from filebeat and Filebeat and I'm trying to add a field "response\_time", and then affect the difference between timestamp to It. So I create a logsta… --- ## [Is there a way to find out on logstash pipeline status?](https://discuss.elastic.co/t/is-there-a-way-to-find-out-on-logstash-pipeline-status/306066)
**Author:** [@newelastic](https://discuss.elastic.co/u/newelastic)\ **Replies:** 1\ **Last updated:** [June 1, 2022, 8:00am UTC](https://discuss.elastic.co/t/is-there-a-way-to-find-out-on-logstash-pipeline-status/306066 "2022-06-01T08:00:23Z")
I've a logstash pipeline which extracts the data from the Elasticsearch and outputs to a file. Is there a way to check if the pipeline is successfully completed by writing to the output file? Right now, I keep checking t… --- ## [Logstash Configuration file in 8.2](https://discuss.elastic.co/t/logstash-configuration-file-in-8-2/306019)
**Author:** [@KunwarAkanksha](https://discuss.elastic.co/u/KunwarAkanksha)\ **Replies:** 3\ **Last updated:** [June 1, 2022, 7:58am UTC](https://discuss.elastic.co/t/logstash-configuration-file-in-8-2/306019 "2022-06-01T07:58:04Z")
After upgrading the ELK stack to 8.2 my logstash configuration file where i have defined the filters for logstash-filter-geoip have changed some key names , like victimASN.as\_org is now coming as victimASN.as.organizatio… --- ## [Data type is showing as text value in data view rather than decimal when uploaded using mysql config file by logstash](https://discuss.elastic.co/t/data-type-is-showing-as-text-value-in-data-view-rather-than-decimal-when-uploaded-using-mysql-config-file-by-logstash/306078)
**Author:** [@akshay\_bhardwaj](https://discuss.elastic.co/u/akshay_bhardwaj)\ **Replies:** 4\ **Last updated:** [June 1, 2022, 6:22am UTC](https://discuss.elastic.co/t/data-type-is-showing-as-text-value-in-data-view-rather-than-decimal-when-uploaded-using-mysql-config-file-by-logstash/306078 "2022-06-01T06:22:16Z")
Hi i create a sql connection using logstash and upload a data but my data is showing as text as shown in figure. --- ## [Logstash Mutate Filter](https://discuss.elastic.co/t/logstash-mutate-filter/306057)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 2\ **Last updated:** [June 1, 2022, 3:14am UTC](https://discuss.elastic.co/t/logstash-mutate-filter/306057 "2022-06-01T03:14:34Z")
Hello I am using Logstash version 7.8.0 in our environment. I am using logstash to extract data from JDBC database using input - jdbc and output == Elasticsearch v7.8.0 doing so , in between I am using filter - muta… --- ## [Add a time field in elasticsearch and calculate time between two event](https://discuss.elastic.co/t/add-a-time-field-in-elasticsearch-and-calculate-time-between-two-event/305971)
**Author:** [@The-morpho](https://discuss.elastic.co/u/The-morpho)\ **Replies:** 3\ **Last updated:** [May 31, 2022, 2:48pm UTC](https://discuss.elastic.co/t/add-a-time-field-in-elasticsearch-and-calculate-time-between-two-event/305971 "2022-05-31T14:48:57Z")
I'm new to Elasticsearch. I’m working with log files stored in Elasticsearch, that contains information about some operations on differents levels, I have a field timestamp that represents the time when each operation h… --- ## [Logstash - syslog output](https://discuss.elastic.co/t/logstash-syslog-output/305288)
**Author:** [@humartinez](https://discuss.elastic.co/u/humartinez)\ **Replies:** 8\ **Last updated:** [May 31, 2022, 12:33pm UTC](https://discuss.elastic.co/t/logstash-syslog-output/305288 "2022-05-31T12:33:53Z")
Hi, Im with some issues configuring the output to syslog. It may be some syntax that Im unaware of, but I can't use my document field values to map some of the plugin output fields. Im trying to use one of my records f… --- ## [Sincedb file is being created empty](https://discuss.elastic.co/t/sincedb-file-is-being-created-empty/305912)
**Author:** [@automation\_learner](https://discuss.elastic.co/u/automation_learner)\ **Replies:** 17\ **Last updated:** [May 31, 2022, 9:29am UTC](https://discuss.elastic.co/t/sincedb-file-is-being-created-empty/305912 "2022-05-31T09:29:19Z")
when i am trying to use file input plugin to created index it is not working as sincedb file is being created empty and in logs it says discovered files count =0 also when using disable sincedb it says open file /dev/nu… --- ## [Geoip setup failure](https://discuss.elastic.co/t/geoip-setup-failure/306011)
**Author:** [@kalev](https://discuss.elastic.co/u/kalev)\ **Replies:** 0\ **Last updated:** [May 31, 2022, 7:52am UTC](https://discuss.elastic.co/t/geoip-setup-failure/306011 "2022-05-31T07:52:32Z")
Hi, I'm trying to setup logstash (ver 7.16) with offline geoip update based on Geoip filter plugin | Logstash Reference \[7.16\] | Elastic , but it fails on step 4: ./bin/elasticsearch-geoip -s /usr/share/GeoIP A CLI to… --- ## [200 pipelines in logstash causing scalability issues](https://discuss.elastic.co/t/200-pipelines-in-logstash-causing-scalability-issues/305999)
**Author:** [@jatin\_gupta](https://discuss.elastic.co/u/jatin_gupta)\ **Replies:** 0\ **Last updated:** [May 31, 2022, 6:21am UTC](https://discuss.elastic.co/t/200-pipelines-in-logstash-causing-scalability-issues/305999 "2022-05-31T06:21:24Z")
Hey I need a design suggestion. We have 200 microservices and a 4 node logstash HA setup. Now each microservice has to send logs to a different index. Current setup I have is, for each microservice we have a different… --- ## [RESOURCE\_LOCKED messages](https://discuss.elastic.co/t/resource-locked-messages/305628)
**Author:** [@rnappert](https://discuss.elastic.co/u/rnappert)\ **Replies:** 7\ **Last updated:** [May 30, 2022, 7:02pm UTC](https://discuss.elastic.co/t/resource-locked-messages/305628 "2022-05-30T19:02:27Z")
I have a three Logstash instances running a fairly old release ( 6.2.3). Two are running in a standby mode, and one in a active mode. The node, which has the vip is considered the active node. All this said, all three … --- ## [Logstash - Parse txt with xml](https://discuss.elastic.co/t/logstash-parse-txt-with-xml/305936)
**Author:** [@Ely\_96](https://discuss.elastic.co/u/Ely_96)\ **Replies:** 4\ **Last updated:** [May 30, 2022, 5:18pm UTC](https://discuss.elastic.co/t/logstash-parse-txt-with-xml/305936 "2022-05-30T17:18:14Z")
Hi guys, I am having trouble reading a txt file that contains free text (tab separated) and xml text that I would like to read. The goal is to read only the content of the xml text, but I have no idea how to reach the g… --- ## [Ruby filter in logstash](https://discuss.elastic.co/t/ruby-filter-in-logstash/305952)
**Author:** [@Iss](https://discuss.elastic.co/u/Iss)\ **Replies:** 1\ **Last updated:** [May 30, 2022, 4:18pm UTC](https://discuss.elastic.co/t/ruby-filter-in-logstash/305952 "2022-05-30T16:18:31Z")
Hi! I have a list of geo\_points coverage: \[{"lon": \*\*\* , "lat": \*\*\*} , {"lon": \*\*\*, "lat": \*\*\*},...\] in a field (coverage) but logstash reads them as text. I put a new location (type geo\_point) tremplate PUT logsta… --- ## [Logstash password and username authentication](https://discuss.elastic.co/t/logstash-password-and-username-authentication/305786)
**Author:** [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\ **Replies:** 2\ **Last updated:** [May 30, 2022, 2:15pm UTC](https://discuss.elastic.co/t/logstash-password-and-username-authentication/305786 "2022-05-30T14:15:12Z")
Hi guys, I was searching in the documentation a way to authenticate a client, for example a metricbeat, into a logstash pipeline using an username and a password, but i couldn't find any option for this. Is there any c… --- ## [Secure between Logstash and Filebeat - File does not contain valid private key](https://discuss.elastic.co/t/secure-between-logstash-and-filebeat-file-does-not-contain-valid-private-key/305806)
**Author:** [@Bavaria](https://discuss.elastic.co/u/Bavaria)\ **Replies:** 2\ **Last updated:** [May 30, 2022, 1:20pm UTC](https://discuss.elastic.co/t/secure-between-logstash-and-filebeat-file-does-not-contain-valid-private-key/305806 "2022-05-30T13:20:59Z")
I am trying to secure between Filebeat and Logstash. However, I do encounter some difficulties. Without SSL the connection does work. First I created with the Elasticsearch-certuitil a CA. C:\\PATH\\elasticsearch-8.0.1\>b… --- ## [Logstash consuming too many resources](https://discuss.elastic.co/t/logstash-consuming-too-many-resources/305201)
**Author:** [@Simone1](https://discuss.elastic.co/u/Simone1)\ **Replies:** 2\ **Last updated:** [May 20, 2022, 6:47am UTC](https://discuss.elastic.co/t/logstash-consuming-too-many-resources/305201 "2022-05-20T06:47:13Z")
Logstash is consuming a lot of memory and there are many hanging processes redirecting output to temporary files that seem to be no longer available. Example of one process: root Ss May01 0:00 sh -c /sbin/blkid \>\>/… --- ## [How to transmitting data from logstash to pubsub lite](https://discuss.elastic.co/t/how-to-transmitting-data-from-logstash-to-pubsub-lite/305930)
**Author:** [@selflabs](https://discuss.elastic.co/u/selflabs)\ **Replies:** 0\ **Last updated:** [May 30, 2022, 9:54am UTC](https://discuss.elastic.co/t/how-to-transmitting-data-from-logstash-to-pubsub-lite/305930 "2022-05-30T09:54:34Z")
Hi Team, is there any plugin or way to transmitting data from logstash to pubsublite. i have used the ' logstash-output-google\_pubsub' but didn't work for pub/sub lite. --- ## [Collectd - Logstash not working](https://discuss.elastic.co/t/collectd-logstash-not-working/305632)
**Author:** [@John\_Murray1](https://discuss.elastic.co/u/John_Murray1)\ **Replies:** 2\ **Last updated:** [May 30, 2022, 9:06am UTC](https://discuss.elastic.co/t/collectd-logstash-not-working/305632 "2022-05-30T09:06:51Z")
Hey, Apologies if this has come up before, I am new to Elastic Stack. This was working but stopped after about 4 Hrs. I am collecting SNMP data with collectd and sending it to logstash, but its not processing any more… --- ## [Declaring Variables in Logstash Config File](https://discuss.elastic.co/t/declaring-variables-in-logstash-config-file/305826)
**Author:** [@usman1](https://discuss.elastic.co/u/usman1)\ **Replies:** 3\ **Last updated:** [May 30, 2022, 8:59am UTC](https://discuss.elastic.co/t/declaring-variables-in-logstash-config-file/305826 "2022-05-30T08:59:46Z")
I am repeating a string (let's "abc-123") in my Logstash config file multiple times. I want to declare it as a variable at the start of config file and call this variable where required. Something like: var my\_str = "ab… --- ## [Logstash file input check\_archive\_validity behaviour question](https://discuss.elastic.co/t/logstash-file-input-check-archive-validity-behaviour-question/305919)
**Author:** [@makibroshett](https://discuss.elastic.co/u/makibroshett)\ **Replies:** 0\ **Last updated:** [May 30, 2022, 8:55am UTC](https://discuss.elastic.co/t/logstash-file-input-check-archive-validity-behaviour-question/305919 "2022-05-30T08:55:01Z")
Hi, I recently migrated to logstash/logstash:6.8.23 docker image, to benefit from the archive checking setting... I see repeated logs about corrupted archives, but cannot figure out if they apply to the same filepath (… --- ## [Logstash Elasticsearch Lookup](https://discuss.elastic.co/t/logstash-elasticsearch-lookup/305841)
**Author:** [@chivas](https://discuss.elastic.co/u/chivas)\ **Replies:** 8\ **Last updated:** [May 30, 2022, 3:34am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-lookup/305841 "2022-05-30T03:34:10Z")
Hello, I am doing a lookup from logstash into Elasticsearch before loading my data, I am running into issue when the data in log file is in array format, e.g. below is my log line: {"id":1652437414971,"body":{"data":\[{… --- ## [Can't connect to Elasticsearch from logstash version8.2](https://discuss.elastic.co/t/cant-connect-to-elasticsearch-from-logstash-version8-2/305785)
**Author:** [@oga](https://discuss.elastic.co/u/oga)\ **Replies:** 7\ **Last updated:** [May 29, 2022, 5:08am UTC](https://discuss.elastic.co/t/cant-connect-to-elasticsearch-from-logstash-version8-2/305785 "2022-05-29T05:08:51Z")
I want to use Elasticsearch and Logstash(ver 8.2). After installed Elasticsearch and Kibana, I installed logstash. But I got error.It seems logstash can't connect to Elasticsearch. But Elasticsearch is runnning correc… --- ## [Error while connecting logstash to elasticsearch](https://discuss.elastic.co/t/error-while-connecting-logstash-to-elasticsearch/304693)
**Author:** [@skander\_khalfet](https://discuss.elastic.co/u/skander_khalfet)\ **Replies:** 13\ **Last updated:** [May 28, 2022, 3:05pm UTC](https://discuss.elastic.co/t/error-while-connecting-logstash-to-elasticsearch/304693 "2022-05-28T15:05:26Z")
hi guys i have installed elk stack on kubernetes using helm charts. all pods are running fine except logstash with this error message. \[2022-05-13T12:01:46,928\]\[WARN \]\[logstash.licensechecker.licensereader\] Attempted t… --- ## [Parsing timestamp](https://discuss.elastic.co/t/parsing-timestamp/305708)
**Author:** [@alon\_carmelly](https://discuss.elastic.co/u/alon_carmelly)\ **Replies:** 5\ **Last updated:** [May 28, 2022, 1:10pm UTC](https://discuss.elastic.co/t/parsing-timestamp/305708 "2022-05-28T13:10:38Z")
trying to parse with this :slight\_smile: if "one-sync" in \[tags\] and "heart-sync" in \[tags\] and "edemand" in \[tags\] and "events" in \[tags\] { date { match =\> \["timestamp", "yyyy-MM-dd'T'HH:mm:ss'.'SSS'Z'"\] … --- ## [Jdbc\_fetch\_size, what would be the ideal value?](https://discuss.elastic.co/t/jdbc-fetch-size-what-would-be-the-ideal-value/305830)
**Author:** [@Rauly\_Koto](https://discuss.elastic.co/u/Rauly_Koto)\ **Replies:** 1\ **Last updated:** [May 28, 2022, 11:36am UTC](https://discuss.elastic.co/t/jdbc-fetch-size-what-would-be-the-ideal-value/305830 "2022-05-28T11:36:31Z")
200k lines: jdbc\_fetch\_size =\> 50000 (1/4) or jdbc\_fetch\_size =\> 100000 (1/2)? --- ## [Last\_failure\_timestamp](https://discuss.elastic.co/t/last-failure-timestamp/305591)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 11\ **Last updated:** [May 27, 2022, 1:13pm UTC](https://discuss.elastic.co/t/last-failure-timestamp/305591 "2022-05-27T13:13:01Z")
Hi How I can manage reload parameter, and why this output is "null" failure\_timestamp" : null In my config I'm using log.level: info "outputs" : \[ { "id" : "1937dea243c36a25e890be2892a0097740ae92… --- ## [How do I make my LOGSTASH pipeline of input type ELASTICSEARCH to be running in certain periods automatically?](https://discuss.elastic.co/t/how-do-i-make-my-logstash-pipeline-of-input-type-elasticsearch-to-be-running-in-certain-periods-automatically/305799)
**Author:** [@Daniel\_Oliveira](https://discuss.elastic.co/u/Daniel_Oliveira)\ **Replies:** 1\ **Last updated:** [May 27, 2022, 12:56pm UTC](https://discuss.elastic.co/t/how-do-i-make-my-logstash-pipeline-of-input-type-elasticsearch-to-be-running-in-certain-periods-automatically/305799 "2022-05-27T12:56:45Z")
Hello gentlemen. I have 2 pipelines running by my LOGSTASH. the first input FILE pipeline is running normally and processing .csv data the second pipeline of input type Elasticsearch it runs only the first time, then … --- ## [Logstash cpu usage is very high, and there are a large number of thread GC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666)
**Author:** [@1065916870](https://discuss.elastic.co/u/1065916870)\ **Replies:** 6\ **Last updated:** [May 27, 2022, 12:16pm UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666 "2022-05-27T12:16:39Z")
Using 3-node logstash to consume kafka data, logstash cpu usage is very high, and there are a large number of thread GC logstash configuration: 3 node , 16C 32G jvm -Xms16g -Xmx16g two pipeline: pipeline.workers: 16 … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=136) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=138)