# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=271 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 272 --- ## [Logstash is still using old code](https://discuss.elastic.co/t/logstash-is-still-using-old-code/258494)
**Author:** [@Ferdous\_Khan](https://discuss.elastic.co/u/Ferdous_Khan)\ **Replies:** 5\ **Last updated:** [December 15, 2020, 2:27pm UTC](https://discuss.elastic.co/t/logstash-is-still-using-old-code/258494 "2020-12-15T14:27:05Z")
Hello, I was previously using elapsed filter in my logstash config to calculate time difference of start and end event. Recently I have replaced elapsed filter's code with elasticsearch output filter and painless script… --- ## [/var/log/upstart/logstash.log is too large](https://discuss.elastic.co/t/var-log-upstart-logstash-log-is-too-large/258690)
**Author:** [@jinsu](https://discuss.elastic.co/u/jinsu)\ **Replies:** 2\ **Last updated:** [December 15, 2020, 10:38am UTC](https://discuss.elastic.co/t/var-log-upstart-logstash-log-is-too-large/258690 "2020-12-15T10:38:31Z")
logstash version 7.9.3 Ubuntu when I start logstash service, the log file in /var/log/upstart/logstash.log will become too large in feu minutes, still I hava change the /path/logs to /data/logstash/. How to change t… --- ## [Logstash only reads files at startup](https://discuss.elastic.co/t/logstash-only-reads-files-at-startup/258224)
**Author:** [@BoKu](https://discuss.elastic.co/u/BoKu)\ **Replies:** 10\ **Last updated:** [December 15, 2020, 7:22am UTC](https://discuss.elastic.co/t/logstash-only-reads-files-at-startup/258224 "2020-12-15T07:22:54Z")
Hello there, if got a problem with logstash reading files. In generall my input, filter and output is working. The files are generated and copied to the correct path, every hour but logstash doesn't read them, only on (… --- ## [Null values while indexing using logstash](https://discuss.elastic.co/t/null-values-while-indexing-using-logstash/258677)
**Author:** [@Sammeta\_David\_Raju](https://discuss.elastic.co/u/Sammeta_David_Raju)\ **Replies:** 0\ **Last updated:** [December 15, 2020, 7:05am UTC](https://discuss.elastic.co/t/null-values-while-indexing-using-logstash/258677 "2020-12-15T07:05:12Z")
what happens if sql statement has record while indexing and later it changes to null value while using the same select query in logstash in a regular intervals --- ## ["\\"\\\\xF8\\" from ASCII-8BIT to UTF-8", :error\_class=\>"LogStash::Json::GeneratorError"](https://discuss.elastic.co/t/xf8-from-ascii-8bit-to-utf-8-error-class-logstash-generatorerror/258668)
**Author:** [@charan\_teja\_sana](https://discuss.elastic.co/u/charan_teja_sana)\ **Replies:** 0\ **Last updated:** [December 15, 2020, 3:54am UTC](https://discuss.elastic.co/t/xf8-from-ascii-8bit-to-utf-8-error-class-logstash-generatorerror/258668 "2020-12-15T03:54:08Z")
Hello Team, I am getting below message while running JDBC Connection against MS-SQL DB, Could you please help Error Message: \[ERROR\] 2020-12-14 23:55:12.404 \[\[main\]\>worker0\] elasticsearch - An unknown error occurred s… --- ## [Logstash comma delimited csv file with quoted strings](https://discuss.elastic.co/t/logstash-comma-delimited-csv-file-with-quoted-strings/258671)
**Author:** [@nameisnotimportant](https://discuss.elastic.co/u/nameisnotimportant)\ **Replies:** 0\ **Last updated:** [December 15, 2020, 5:13am UTC](https://discuss.elastic.co/t/logstash-comma-delimited-csv-file-with-quoted-strings/258671 "2020-12-15T05:13:05Z")
Hi, I have following comma delimited csv file, which content has a comma in some of the columns. LogTime,Level,ApplicationId,UserId,ProcessId,FormId,Action,Status,Reference,TranRef,IP,SessionId,CIF\_REF\_NO,UserAgent 11… --- ## [Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. org.jruby.exceptions.ArgumentError: (ArgumentError) key cannot be blank](https://discuss.elastic.co/t/exception-in-pipelineworker-the-pipeline-stopped-processing-new-events-please-check-your-filter-configuration-and-restart-logstash-org-jruby-exceptions-argumenterror-argumenterror-key-cannot-be-blank/258670)
**Author:** [@bhavanikallam](https://discuss.elastic.co/u/bhavanikallam)\ **Replies:** 2\ **Last updated:** [December 15, 2020, 4:42am UTC](https://discuss.elastic.co/t/exception-in-pipelineworker-the-pipeline-stopped-processing-new-events-please-check-your-filter-configuration-and-restart-logstash-org-jruby-exceptions-argumenterror-argumenterror-key-cannot-be-blank/258670 "2020-12-15T04:42:17Z")
Plz kindly help --- ## [Best option to parse some data GROK or reindex?](https://discuss.elastic.co/t/best-option-to-parse-some-data-grok-or-reindex/258663)
**Author:** [@fastxl](https://discuss.elastic.co/u/fastxl)\ **Replies:** 0\ **Last updated:** [December 15, 2020, 2:08am UTC](https://discuss.elastic.co/t/best-option-to-parse-some-data-grok-or-reindex/258663 "2020-12-15T02:08:30Z")
I am collecting machine tool data and the logs I am getting are in the JSON format that can be sent to logstash. However I need to break down one of the JSON lines into some more documents. Should I try to run this throu… --- ## [Logstash - SNS output plugin - Pipeline - MissingCredentialsError](https://discuss.elastic.co/t/logstash-sns-output-plugin-pipeline-missingcredentialserror/258649)
**Author:** [@chandra2037](https://discuss.elastic.co/u/chandra2037)\ **Replies:** 0\ **Last updated:** [December 14, 2020, 9:41pm UTC](https://discuss.elastic.co/t/logstash-sns-output-plugin-pipeline-missingcredentialserror/258649 "2020-12-14T21:41:09Z")
Currently configured Logstash V 7.8.1 with SNS output plugin . It is running okay for a while then getting the following error message, then pipeline stops and never recovers. {"level":"ERROR","loggerName":"logstash.… --- ## [Missing Converter handling for full class name=org.postgresql.util.PGobject, simple name=PGobject](https://discuss.elastic.co/t/missing-converter-handling-for-full-class-name-org-postgresql-util-pgobject-simple-name-pgobject/258645)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 0\ **Last updated:** [December 14, 2020, 8:52pm UTC](https://discuss.elastic.co/t/missing-converter-handling-for-full-class-name-org-postgresql-util-pgobject-simple-name-pgobject/258645 "2020-12-14T20:52:09Z")
Getting this error trying to use logstash jjdbc to extract complex objects from postgres. Converting to TEXT did not work as well \]\[WARN \]\[logstash.inputs.jdbc \]\[rivi-poc\_people\]\[ed94f8a65daf945d07d8fd7ebea8d75acad… --- ## [Elasticsearch error: missing authentication credentials for REST request \[/logstash-filebeat-2020.12.11/\_mapping\]](https://discuss.elastic.co/t/elasticsearch-error-missing-authentication-credentials-for-rest-request-logstash-filebeat-2020-12-11-mapping/258435)
**Author:** [@mjcammilleri](https://discuss.elastic.co/u/mjcammilleri)\ **Replies:** 1\ **Last updated:** [December 14, 2020, 8:21pm UTC](https://discuss.elastic.co/t/elasticsearch-error-missing-authentication-credentials-for-rest-request-logstash-filebeat-2020-12-11-mapping/258435 "2020-12-14T20:21:37Z")
Hi Everyone, This could be an issue that crosses with Grafana, but I'm hoping for some insight here. I am able to access the Elasticsearch API using my active token. I can test using curl in various python scripts to pu… --- ## [Push\_previous\_map\_as\_event if fields exist](https://discuss.elastic.co/t/push-previous-map-as-event-if-fields-exist/250566)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 39\ **Last updated:** [December 14, 2020, 6:31pm UTC](https://discuss.elastic.co/t/push-previous-map-as-event-if-fields-exist/250566 "2020-12-14T18:31:00Z")
Hi, I need help figuring out a way for Logstash to check if a specific filed exists for a specific task\_id and then aggregate those fields using push\_previous\_map\_as\_event. For example, my current aggregate filter looks… --- ## [Field have the same value](https://discuss.elastic.co/t/field-have-the-same-value/258604)
**Author:** [@Dea\_Agra](https://discuss.elastic.co/u/Dea_Agra)\ **Replies:** 5\ **Last updated:** [December 14, 2020, 5:08pm UTC](https://discuss.elastic.co/t/field-have-the-same-value/258604 "2020-12-14T17:08:05Z")
I don't know why logstash keep parsing the same value for the different field. Here I attach my logstash configuration input { file { start\_position =\> "beginning" sincedb\_path =\> "/dev/null" path =\> \[ "/home/elasticse… --- ## [Best practices for docker container application log shipping with Filebeat to Logstash](https://discuss.elastic.co/t/best-practices-for-docker-container-application-log-shipping-with-filebeat-to-logstash/258613)
**Author:** [@duckasylum](https://discuss.elastic.co/u/duckasylum)\ **Replies:** 1\ **Last updated:** [December 14, 2020, 4:52pm UTC](https://discuss.elastic.co/t/best-practices-for-docker-container-application-log-shipping-with-filebeat-to-logstash/258613 "2020-12-14T16:52:25Z")
Hi, I have a docker service of two containers. They both have a web server component which produces access, error and debug logs and one container has special module attached to the web server component, which produces … --- ## [Grok filter TIMESTAMP\_ISO8601 appears in Kibana as String](https://discuss.elastic.co/t/grok-filter-timestamp-iso8601-appears-in-kibana-as-string/258240)
**Author:** [@SecretSquizza](https://discuss.elastic.co/u/SecretSquizza)\ **Replies:** 3\ **Last updated:** [December 14, 2020, 2:48pm UTC](https://discuss.elastic.co/t/grok-filter-timestamp-iso8601-appears-in-kibana-as-string/258240 "2020-12-14T14:48:14Z")
Hi all, I done a bit of rooting around google but can't find any definitive answers, so i decided to humble myself and request help directly. The problem i have is that, like many incoming messages, my log entry contai… --- ## [MS SQL+ ELASTICSEARCH](https://discuss.elastic.co/t/ms-sql-elasticsearch/258563)
**Author:** [@pmate](https://discuss.elastic.co/u/pmate)\ **Replies:** 6\ **Last updated:** [December 14, 2020, 1:02pm UTC](https://discuss.elastic.co/t/ms-sql-elasticsearch/258563 "2020-12-14T13:02:21Z")
Hy everybody! I am new in elasticsearch and beginner in docker also. I need to know what is the best way to connect an ms sql data base to an elastichsearch docker instance? Thanks for Help! --- ## [Appeared \_grokparsefailure, unable to parse specific lines from log file](https://discuss.elastic.co/t/appeared-grokparsefailure-unable-to-parse-specific-lines-from-log-file/258566)
**Author:** [@Polak](https://discuss.elastic.co/u/Polak)\ **Replies:** 0\ **Last updated:** [December 14, 2020, 10:50am UTC](https://discuss.elastic.co/t/appeared-grokparsefailure-unable-to-parse-specific-lines-from-log-file/258566 "2020-12-14T10:50:57Z")
I have this kind of log: https://pastebin.com/ztXijyNV I would like to drop all events except the lines with "INFO -- :". Logs are gathered by filebeat and then they are send to logstash. Important data are in log insid… --- ## [Client requested protocol tlsv1 is not enabled or supported in server context](https://discuss.elastic.co/t/client-requested-protocol-tlsv1-is-not-enabled-or-supported-in-server-context/258114)
**Author:** [@Usman](https://discuss.elastic.co/u/Usman)\ **Replies:** 2\ **Last updated:** [December 14, 2020, 11:45am UTC](https://discuss.elastic.co/t/client-requested-protocol-tlsv1-is-not-enabled-or-supported-in-server-context/258114 "2020-12-14T11:45:24Z")
Hi, we are using ELK stack version 7.9.0 , when we try to connect logstash with elastic it give us error . 'client requested protocol tlsv1 is not enabled or supported in server context' when we specify following para… --- ## [Replace Old Data From Logstash](https://discuss.elastic.co/t/replace-old-data-from-logstash/258538)
**Author:** [@rehannali](https://discuss.elastic.co/u/rehannali)\ **Replies:** 1\ **Last updated:** [December 14, 2020, 8:36am UTC](https://discuss.elastic.co/t/replace-old-data-from-logstash/258538 "2020-12-14T08:36:30Z")
Hi, I using MySql to dumb data into elasticsearch. It runs every 5 mins and got duplicate results. I want to replace data so there will be no duplicate entries in it. How can i achieve this? P.S. I need to replace who… --- ## [Hive connection and data ingestion](https://discuss.elastic.co/t/hive-connection-and-data-ingestion/256797)
**Author:** [@irfangk1](https://discuss.elastic.co/u/irfangk1)\ **Replies:** 1\ **Last updated:** [December 14, 2020, 6:37am UTC](https://discuss.elastic.co/t/hive-connection-and-data-ingestion/256797 "2020-12-14T06:37:14Z")
Hi, I want to connect Hive with Elastic and then ingest data into it. Please share the steps/process --- ## [Forming Index name for Elasticsearch plugin](https://discuss.elastic.co/t/forming-index-name-for-elasticsearch-plugin/258398)
**Author:** [@s0umen](https://discuss.elastic.co/u/s0umen)\ **Replies:** 6\ **Last updated:** [December 14, 2020, 6:18am UTC](https://discuss.elastic.co/t/forming-index-name-for-elasticsearch-plugin/258398 "2020-12-14T06:18:20Z")
I am sending Linux system logs using filebeat to logstash. my "logstash.conf" file is something like below (within double Line). I am trying to name index dynamically using "type" variable. But when I view it in Kibana… --- ## [Change index file name from logstash command](https://discuss.elastic.co/t/change-index-file-name-from-logstash-command/258532)
**Author:** [@Aryaman\_Gupta](https://discuss.elastic.co/u/Aryaman_Gupta)\ **Replies:** 0\ **Last updated:** [December 14, 2020, 4:54am UTC](https://discuss.elastic.co/t/change-index-file-name-from-logstash-command/258532 "2020-12-14T04:54:46Z")
Is there any way to give custom name to index file in logstash config file every-time I run log-stash command? output { elasticsearch { hosts =\> \["localhost:9200"\] index =\> "global7679-%{+YYYY.MM.dd}" \<----- Need to … --- ## [How to deploy logstash with persistent volume on kubernetes?](https://discuss.elastic.co/t/how-to-deploy-logstash-with-persistent-volume-on-kubernetes/258505)
**Author:** [@iooi](https://discuss.elastic.co/u/iooi)\ **Replies:** 0\ **Last updated:** [December 13, 2020, 1:17pm UTC](https://discuss.elastic.co/t/how-to-deploy-logstash-with-persistent-volume-on-kubernetes/258505 "2020-12-13T13:17:59Z")
Using GKE to deploy logstash by statefulset kind with pvc. Also need to install an output plugin. When don't use while true; do sleep 1000; done; in container's command args, it can't deploy with pvc successfully. The … --- ## [Null and nil](https://discuss.elastic.co/t/null-and-nil/258507)
**Author:** [@rojin](https://discuss.elastic.co/u/rojin)\ **Replies:** 0\ **Last updated:** [December 13, 2020, 1:25pm UTC](https://discuss.elastic.co/t/null-and-nil/258507 "2020-12-13T13:25:44Z")
Hello everyone! I have a grok pattern which some of its fields may include "null" values. How am I supposed to show null in kibana? or logstash output? I have defined the pattern as number for those fields. input { … --- ## [Logstash UDP input buffer](https://discuss.elastic.co/t/logstash-udp-input-buffer/258480)
**Author:** [@Hendrik1](https://discuss.elastic.co/u/Hendrik1)\ **Replies:** 7\ **Last updated:** [December 12, 2020, 7:29pm UTC](https://discuss.elastic.co/t/logstash-udp-input-buffer/258480 "2020-12-12T19:29:22Z")
Hi, I am using the logstash UDP input to receive my firewall syslog messages, this seems to work fine. However, I just enabled the reverse dns lookup filter on the ip's, but now I keep wondering what happens when the U… --- ## [Deploy Logstash with persistent queue on Docker Swarm](https://discuss.elastic.co/t/deploy-logstash-with-persistent-queue-on-docker-swarm/258465)
**Author:** [@Hung\_Phan\_Huy](https://discuss.elastic.co/u/Hung_Phan_Huy)\ **Replies:** 0\ **Last updated:** [December 12, 2020, 4:30am UTC](https://discuss.elastic.co/t/deploy-logstash-with-persistent-queue-on-docker-swarm/258465 "2020-12-12T04:30:16Z")
Hi, I'm planning to deploy my Logstash on Docker Swarm. My concern is about the persistent queue feature. Should I use persistent queue? Some documents I read show that they might have a great impact on the performanc… --- ## [Logstash does not index event to elastic due to issue on host field](https://discuss.elastic.co/t/logstash-does-not-index-event-to-elastic-due-to-issue-on-host-field/258171)
**Author:** [@snolfi](https://discuss.elastic.co/u/snolfi)\ **Replies:** 5\ **Last updated:** [December 11, 2020, 9:46pm UTC](https://discuss.elastic.co/t/logstash-does-not-index-event-to-elastic-due-to-issue-on-host-field/258171 "2020-12-11T21:46:31Z")
Hi all, i'm having an issue with my logstash. I updated my infrastructure from elk 5.6 to elk 6.8 and now from elk 6.8 to elk 7.8. We are updating filebeats but logstash is reporting this error on metrics coming from th… --- ## [Logstash/ Elastic index date and time match error](https://discuss.elastic.co/t/logstash-elastic-index-date-and-time-match-error/258448)
**Author:** [@Gambit22](https://discuss.elastic.co/u/Gambit22)\ **Replies:** 2\ **Last updated:** [December 11, 2020, 8:45pm UTC](https://discuss.elastic.co/t/logstash-elastic-index-date-and-time-match-error/258448 "2020-12-11T20:45:48Z")
Hello Everyone, this is my first post here so I'll try to provide good specifics. First off, I'm trying to make an ELK stack with Filebeat all hosted on the same VM. It's just a lab environment. Kibana, Elastic, Logsta… --- ## [Kibana dont collect app logs after 0:00 in some days on 2 from 3 spaces](https://discuss.elastic.co/t/kibana-dont-collect-app-logs-after-0-00-in-some-days-on-2-from-3-spaces/258446)
**Author:** [@111419](https://discuss.elastic.co/u/111419)\ **Replies:** 0\ **Last updated:** [December 11, 2020, 7:29pm UTC](https://discuss.elastic.co/t/kibana-dont-collect-app-logs-after-0-00-in-some-days-on-2-from-3-spaces/258446 "2020-12-11T19:29:44Z")
I Have 3 spaces. in one i dont have app logs from 0:00 10th dec, in second from 0:00 11th dec, kibana recieved logs from filebeat only php-fpm access.log, all other web app logs in json format not recieved from 10 and 1… --- ## [How to create multiple pipelines with logstash helm chart so they have private data](https://discuss.elastic.co/t/how-to-create-multiple-pipelines-with-logstash-helm-chart-so-they-have-private-data/258444)
**Author:** [@jknott](https://discuss.elastic.co/u/jknott)\ **Replies:** 0\ **Last updated:** [December 11, 2020, 7:16pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-pipelines-with-logstash-helm-chart-so-they-have-private-data/258444 "2020-12-11T19:16:38Z")
I have created a helm chart with multiple pipelines but when I add a field per pipeline for example service\_name, the service name from each pipeline gets added to the field. So instead of one field being added with one … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=270) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=272)