# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=453 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 454 --- ## [Multiple http\_poller configurations](https://discuss.elastic.co/t/multiple-http-poller-configurations/206154)
**Author:** [@akhilsharma.in](https://discuss.elastic.co/u/akhilsharma.in)\ **Replies:** 0\ **Last updated:** [November 1, 2019, 11:28am UTC](https://discuss.elastic.co/t/multiple-http-poller-configurations/206154 "2019-11-01T11:28:31Z")
Hi Logstash Team, I have 5 configuration files which use http\_poller and http plug-ins for REST calls. All of the 5 conf files work perfectly if I run them one by one. But if I give the path of conf files as \* (all) mea… --- ## [Logstash Insert or Update if exists](https://discuss.elastic.co/t/logstash-insert-or-update-if-exists/206083)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 3\ **Last updated:** [November 1, 2019, 11:15am UTC](https://discuss.elastic.co/t/logstash-insert-or-update-if-exists/206083 "2019-11-01T11:15:38Z")
hello guys , so lately i've trying to insert some data from txt file into elasticsearch index , but i'm having a problem in updating my index , when i run the logstach conf file it's not updating the data or inserting th… --- ## [Logstash Filebeat incorrect SSL key](https://discuss.elastic.co/t/logstash-filebeat-incorrect-ssl-key/205849)
**Author:** [@Devopsio](https://discuss.elastic.co/u/Devopsio)\ **Replies:** 3\ **Last updated:** [November 1, 2019, 8:33am UTC](https://discuss.elastic.co/t/logstash-filebeat-incorrect-ssl-key/205849 "2019-11-01T08:33:51Z")
Hi. I can't establish an SSL connection between Logstash (containerized) and Filebeat. I'm using a self-signed certificate. Actions I performed: openssl req -newkey rsa:4096 -nodes -keyout logstash.key -subj "/CN=(hostn… --- ## [Everyone faced this issue?](https://discuss.elastic.co/t/everyone-faced-this-issue/205576)
**Author:** [@pbona](https://discuss.elastic.co/u/pbona)\ **Replies:** 34\ **Last updated:** [November 1, 2019, 2:13am UTC](https://discuss.elastic.co/t/everyone-faced-this-issue/205576 "2019-11-01T02:13:25Z")
\[2019-10-28T01:43:36,493\]\[INFO \]\[logstash.agent \] Successfully started Logstash API endpoint {:port=\>9600} \[2019-10-28T01:43:41,249\]\[INFO \]\[logstash.runner \] Logstash shut down. \[2019-10-28T01:44:13,… --- ## [Dynamic Timestamp Pattern Failure for Date Filter](https://discuss.elastic.co/t/dynamic-timestamp-pattern-failure-for-date-filter/206105)
**Author:** [@Alsheh](https://discuss.elastic.co/u/Alsheh)\ **Replies:** 2\ **Last updated:** [October 31, 2019, 9:32pm UTC](https://discuss.elastic.co/t/dynamic-timestamp-pattern-failure-for-date-filter/206105 "2019-10-31T21:32:02Z")
I'm trying to parse a date with a dynamic timestamp pattern provided by the client, but Logstash fails and won't start when accessing the timestamp\_pattern variable in the date filter. Here is an example that reproduces … --- ## [Logstash Extractnumbers plugin usage](https://discuss.elastic.co/t/logstash-extractnumbers-plugin-usage/205876)
**Author:** [@angryninja](https://discuss.elastic.co/u/angryninja)\ **Replies:** 2\ **Last updated:** [October 31, 2019, 8:50pm UTC](https://discuss.elastic.co/t/logstash-extractnumbers-plugin-usage/205876 "2019-10-31T20:50:02Z")
Hi there, I have started using Logstash just recently. How do I make use of Extractnumbers plugin with Logstash ? I can find its usage with options but don't seem to find the simplest example of a basic use case of ext… --- ## [EOFError: End of file reached](https://discuss.elastic.co/t/eoferror-end-of-file-reached/204933)
**Author:** [@Senthil\_ak](https://discuss.elastic.co/u/Senthil_ak)\ **Replies:** 3\ **Last updated:** [October 31, 2019, 5:52pm UTC](https://discuss.elastic.co/t/eoferror-end-of-file-reached/204933 "2019-10-31T17:52:33Z")
Hi, I am routing logstash logs to another logstash using TCP, there is no connection issue between them and I am able to telnet. but while sending the logs getting below error. Is that a bug ? \[2019-10-22T14:50:48,395\]… --- ## [Important logstash metrics to monitor](https://discuss.elastic.co/t/important-logstash-metrics-to-monitor/205941)
**Author:** [@funlearning321](https://discuss.elastic.co/u/funlearning321)\ **Replies:** 1\ **Last updated:** [October 31, 2019, 5:45pm UTC](https://discuss.elastic.co/t/important-logstash-metrics-to-monitor/205941 "2019-10-31T17:45:06Z")
Hello, What are the important logstash performance metrics to be monitored using watcher or zabbix ? Thanks --- ## [Logsatsh String To Date convertion](https://discuss.elastic.co/t/logsatsh-string-to-date-convertion/205705)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 2\ **Last updated:** [October 31, 2019, 4:48pm UTC](https://discuss.elastic.co/t/logsatsh-string-to-date-convertion/205705 "2019-10-31T16:48:07Z")
Hello Guys , so lately i've trying to insert some data from txt file into elasticsearch index , but i'm having a problem converting string field to date type , here is my logsatsh conf : input { file { path =\> "C:/Ela… --- ## [How to add column headings to csv](https://discuss.elastic.co/t/how-to-add-column-headings-to-csv/206082)
**Author:** [@namali](https://discuss.elastic.co/u/namali)\ **Replies:** 0\ **Last updated:** [October 31, 2019, 4:42pm UTC](https://discuss.elastic.co/t/how-to-add-column-headings-to-csv/206082 "2019-10-31T16:42:51Z")
transferred json file to csv from logstash. but there is no column headings in csv files. how can I add column headings to csv files while transfer json to csv. --- ## [Unpriviledge image](https://discuss.elastic.co/t/unpriviledge-image/206072)
**Author:** [@jof300](https://discuss.elastic.co/u/jof300)\ **Replies:** 0\ **Last updated:** [October 31, 2019, 3:43pm UTC](https://discuss.elastic.co/t/unpriviledge-image/206072 "2019-10-31T15:43:23Z")
Hi, I am trying to deploy logtash on Openshift. As you know, Openshift doesn't allow to run containers in root. Result is that it can't open /usr/share/logstash/config/logtash.yml Is there any unpriviledged logtash im… --- ## [Not able to create the filter with correct pattern](https://discuss.elastic.co/t/not-able-to-create-the-filter-with-correct-pattern/205956)
**Author:** [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\ **Replies:** 4\ **Last updated:** [October 31, 2019, 3:16pm UTC](https://discuss.elastic.co/t/not-able-to-create-the-filter-with-correct-pattern/205956 "2019-10-31T15:16:38Z")
input file - {"apps":{"app":\[{"id":"application\_1571691267503\_939654","user":"Z013J5W","name":"HIVE-d9904734e7fa7249f0","queue":"default","state":"RUNNING","finalStatus":"UNDEFINED","progress":55.259087}\],"logAggregatio… --- ## [Replace field value by part of output message](https://discuss.elastic.co/t/replace-field-value-by-part-of-output-message/206011)
**Author:** [@kavindu\_manaram](https://discuss.elastic.co/u/kavindu_manaram)\ **Replies:** 1\ **Last updated:** [October 31, 2019, 2:31pm UTC](https://discuss.elastic.co/t/replace-field-value-by-part-of-output-message/206011 "2019-10-31T14:31:09Z")
I want to replace level filed value by output message "levelName" 's value. Is it possible to do this in logstash conf? filter { if \[fields\]\[level\] == "default" { grok { match =\> {"message" =\> \["(?m)%{TIMESTAMP… --- ## [Logstash file-input doesn't pick up new files](https://discuss.elastic.co/t/logstash-file-input-doesnt-pick-up-new-files/206026)
**Author:** [@Gordon\_Thompson](https://discuss.elastic.co/u/Gordon_Thompson)\ **Replies:** 1\ **Last updated:** [October 31, 2019, 2:15pm UTC](https://discuss.elastic.co/t/logstash-file-input-doesnt-pick-up-new-files/206026 "2019-10-31T14:15:53Z")
We have a simple logstash 7.1.1 installation on a hardened Redhat 7.7 pipelines.yml - pipeline.id: synonyms-ingest pipeline.workers: 1 path.config: "/etc/logstash/CDE/synonym-logstash-csv-to-es.conf" synonym-logst… --- ## [How to configure Logstash and JDBC to upload new records based on ascending serial number](https://discuss.elastic.co/t/how-to-configure-logstash-and-jdbc-to-upload-new-records-based-on-ascending-serial-number/206032)
**Author:** [@MAvD](https://discuss.elastic.co/u/MAvD)\ **Replies:** 1\ **Last updated:** [October 31, 2019, 2:12pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-and-jdbc-to-upload-new-records-based-on-ascending-serial-number/206032 "2019-10-31T14:12:23Z")
Hello, Currently, we are updating our data in Elasticsearch by uploading a lot of data once in a while from a firebird database which uses quite some data. We want this this to be synchronized by uploading only new and … --- ## [Running tests for output file plugin](https://discuss.elastic.co/t/running-tests-for-output-file-plugin/205979)
**Author:** [@Nitin\_Gupta3](https://discuss.elastic.co/u/Nitin_Gupta3)\ **Replies:** 1\ **Last updated:** [October 31, 2019, 2:03pm UTC](https://discuss.elastic.co/t/running-tests-for-output-file-plugin/205979 "2019-10-31T14:03:00Z")
I'm not able to run tests for logstash-output-file plugin. While I'm able to execute "jruby -S bundle install" successfully. but next command: "jruby -S bundle exec rspec" gives an error: You have requested: logstas… --- ## [How to configure logstash to get only new data/updated data into elasticsearch](https://discuss.elastic.co/t/how-to-configure-logstash-to-get-only-new-data-updated-data-into-elasticsearch/203266)
**Author:** [@rameshkr1994](https://discuss.elastic.co/u/rameshkr1994)\ **Replies:** 9\ **Last updated:** [October 31, 2019, 1:41pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-to-get-only-new-data-updated-data-into-elasticsearch/203266 "2019-10-31T13:41:11Z")
Dear All. i am facing the issue from a long time.i have unique column in sql server side and i want to get only new data/updated data into elasticsearch every 15 min. how will i do ...after loading first time data int… --- ## [Different format for different output](https://discuss.elastic.co/t/different-format-for-different-output/206034)
**Author:** [@skowron-line](https://discuss.elastic.co/u/skowron-line)\ **Replies:** 3\ **Last updated:** [October 31, 2019, 1:25pm UTC](https://discuss.elastic.co/t/different-format-for-different-output/206034 "2019-10-31T13:25:51Z")
I would like to send 2 different format message to outputs so I have something like this input { beats { port =\> 5044 } } filter { clone { clones =\> \["message"\] add\_tag =\> \["siem"\] … --- ## [Using logstash plugin to change a field value](https://discuss.elastic.co/t/using-logstash-plugin-to-change-a-field-value/205907)
**Author:** [@ndg](https://discuss.elastic.co/u/ndg)\ **Replies:** 4\ **Last updated:** [October 31, 2019, 1:17pm UTC](https://discuss.elastic.co/t/using-logstash-plugin-to-change-a-field-value/205907 "2019-10-31T13:17:01Z")
I have a ELK stack deployed in docker swarm and i need to improve some data created by the filebeat. I have a field with docker metadata that contains the node id (container.labels.com\_docker\_swarm\_node\_id) with a random… --- ## [Logstash mutate replace filter fails](https://discuss.elastic.co/t/logstash-mutate-replace-filter-fails/205827)
**Author:** [@LeonZC](https://discuss.elastic.co/u/LeonZC)\ **Replies:** 4\ **Last updated:** [October 31, 2019, 11:57am UTC](https://discuss.elastic.co/t/logstash-mutate-replace-filter-fails/205827 "2019-10-31T11:57:57Z")
I have field called "zone" and i am trying to replace the field value by using mutate to "BlockA" when the field value of "zone" is "Z0". Here is the code that I am using: filter { if \[source\] =~ "file.log" { grok … --- ## [Parsing URL with Logstash (using ECS fields)](https://discuss.elastic.co/t/parsing-url-with-logstash-using-ecs-fields/205814)
**Author:** [@Vincent\_Maury](https://discuss.elastic.co/u/Vincent_Maury)\ **Replies:** 1\ **Last updated:** [October 31, 2019, 11:57am UTC](https://discuss.elastic.co/t/parsing-url-with-logstash-using-ecs-fields/205814 "2019-10-31T11:57:49Z")
Hi there i've been working on a way to parse url into ecs fields, and it seems to be working well. Hope it helps! # Parser a url filter { grok { match =\> \[ # match https://user:pwd@stuff.domain… --- ## [Login on SQL Server with domain account (using jdbc plugin)](https://discuss.elastic.co/t/login-on-sql-server-with-domain-account-using-jdbc-plugin/205742)
**Author:** [@rschirin](https://discuss.elastic.co/u/rschirin)\ **Replies:** 2\ **Last updated:** [October 31, 2019, 7:23am UTC](https://discuss.elastic.co/t/login-on-sql-server-with-domain-account-using-jdbc-plugin/205742 "2019-10-31T07:23:19Z")
Hi all, is there a way to login on MS SQL server using domain credentials? Currently I'm not able to login on it using JDBC Logstash plugin. --- ## [Logstash JMS Input version 7.4.0](https://discuss.elastic.co/t/logstash-jms-input-version-7-4-0/204776)
**Author:** [@thrishod](https://discuss.elastic.co/u/thrishod)\ **Replies:** 1\ **Last updated:** [October 31, 2019, 6:38am UTC](https://discuss.elastic.co/t/logstash-jms-input-version-7-4-0/204776 "2019-10-31T06:38:52Z")
Hi All, I am very newbie to ELK and my requirement is to read the messages from IBM WMQ. I have setup the queue manager, Queue, JMS queue, Binding etc.. as needed. In order to read messages from IBM WMQ, I believe log… --- ## [Renaming nested fields based on some condition](https://discuss.elastic.co/t/renaming-nested-fields-based-on-some-condition/205801)
**Author:** [@Thilak](https://discuss.elastic.co/u/Thilak)\ **Replies:** 2\ **Last updated:** [October 31, 2019, 6:14am UTC](https://discuss.elastic.co/t/renaming-nested-fields-based-on-some-condition/205801 "2019-10-31T06:14:29Z")
In the document, I want to change the fieldName based on conditions below 1.If the value field has JSON type. Change the value field to value-keyword and remove "value-whitespace" and \*\*"value-standard" \*\* if pres… --- ## [Logstash node not showing up in Kibana monitoring (Logstash constantly restarting)](https://discuss.elastic.co/t/logstash-node-not-showing-up-in-kibana-monitoring-logstash-constantly-restarting/205808)
**Author:** [@vdere](https://discuss.elastic.co/u/vdere)\ **Replies:** 4\ **Last updated:** [October 31, 2019, 5:29am UTC](https://discuss.elastic.co/t/logstash-node-not-showing-up-in-kibana-monitoring-logstash-constantly-restarting/205808 "2019-10-31T05:29:57Z")
One of our logstash node is not showing in the monitoring tab. So how can I log into specific logstash nodes to verify if x-pack is running? Assuming that is what is causing this problem. Or there might be any other issu… --- ## [\[ERROR\]\[logstash.agent\] Failed to execute action "Expected one of #, {, } at line 161, column 185 (byte 16646) after input {\\r\\n\\tjdbc {\\r\\n\\t\\tjdbc\_driver\_library =\> \\](https://discuss.elastic.co/t/error-logstash-agent-failed-to-execute-action-expected-one-of-at-line-161-column-185-byte-16646-after-input-r-n-tjdbc-r-n-t-tjdbc-driver-library/205566)
**Author:** [@jogabonito](https://discuss.elastic.co/u/jogabonito)\ **Replies:** 4\ **Last updated:** [October 31, 2019, 12:55am UTC](https://discuss.elastic.co/t/error-logstash-agent-failed-to-execute-action-expected-one-of-at-line-161-column-185-byte-16646-after-input-r-n-tjdbc-r-n-t-tjdbc-driver-library/205566 "2019-10-31T00:55:01Z")
Hi I get this error message while trying to indexing data to elasticsearch. Does anyone have an idea what could cause this error? \[2019-10-04T17:00:02,272\]\[ERROR\]\[logstash.agent\] Failed to execute ac tion {:action=\>L… --- ## [Logstash skips file while reading config](https://discuss.elastic.co/t/logstash-skips-file-while-reading-config/205770)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 9\ **Last updated:** [October 30, 2019, 8:28pm UTC](https://discuss.elastic.co/t/logstash-skips-file-while-reading-config/205770 "2019-10-30T20:28:08Z")
Logstash just starts and shuts down due to configpathloader error. Please explain what the issue is. Below is the error - \[DEBUG\]\[logstash.config.source.local.configpathloader\] Skipping the following files while reading… --- ## [Duplicates entries when using S3 input](https://discuss.elastic.co/t/duplicates-entries-when-using-s3-input/205932)
**Author:** [@cascoalessio](https://discuss.elastic.co/u/cascoalessio)\ **Replies:** 0\ **Last updated:** [October 30, 2019, 6:25pm UTC](https://discuss.elastic.co/t/duplicates-entries-when-using-s3-input/205932 "2019-10-30T18:25:46Z")
Hello Guys I just deployed on k8s a logstash deployment that reads logs from S3 and pushes them to Elasticsearch. I noticed that the sincedb file when using the S3 input has this content: 2019-10-30 16:14:08 UTC So t… --- ## [Logstash started successfully and within minute shuts down automatically](https://discuss.elastic.co/t/logstash-started-successfully-and-within-minute-shuts-down-automatically/205565)
**Author:** [@Mehak\_Bhargava](https://discuss.elastic.co/u/Mehak_Bhargava)\ **Replies:** 2\ **Last updated:** [October 30, 2019, 5:54pm UTC](https://discuss.elastic.co/t/logstash-started-successfully-and-within-minute-shuts-down-automatically/205565 "2019-10-30T17:54:55Z")
\[2019-10-28T16:19:39,384\]\[INFO\]\[logstash.agent \] Successfully started Logstash API endpoint {:port=\>9600} \[2019-10-28T16:19:43,944\]\[INFO\]\[logstash.runner \] Logstash shut down --- ## [Limit of total fields \[num\] in index \[logstash-XX\] has been exceeded](https://discuss.elastic.co/t/limit-of-total-fields-num-in-index-logstash-xx-has-been-exceeded/205889)
**Author:** [@wpirraglia](https://discuss.elastic.co/u/wpirraglia)\ **Replies:** 6\ **Last updated:** [October 30, 2019, 5:51pm UTC](https://discuss.elastic.co/t/limit-of-total-fields-num-in-index-logstash-xx-has-been-exceeded/205889 "2019-10-30T17:51:12Z")
Little background story first: We had an ELK 2.6 stack, which we decided to upgrade to 6.5. After creating the new stack, we decided to use the reindex API to copy our old Logstash indexes from the old stack to the new o… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=452) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=454)