# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=768 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 769 --- ## [How to change the data type values](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 18\ **Last updated:** [May 3, 2018, 10:22am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396 "2018-05-03T10:22:24Z")
i have data like this is json file {\\"dataType\\":\\"System.Single\\",\\"host\\":\\"ssss\\",\\"name\\":\\"123/xxx\\",\\"123234\\":\\"333\\",\\"time\\":\\"2018/5/3 7:49:35\\",\\"value\\":\\"116.3047\\"} question: if the dataType is System.Si… --- ## [How the logstash get my timezone](https://discuss.elastic.co/t/how-the-logstash-get-my-timezone/130410)
**Author:** [@chenzilong444](https://discuss.elastic.co/u/chenzilong444)\ **Replies:** 2\ **Last updated:** [May 3, 2018, 9:59am UTC](https://discuss.elastic.co/t/how-the-logstash-get-my-timezone/130410 "2018-05-03T09:59:57Z")
i meet a problem when i send a timestamp to logstash it can turn it to utc .but i didn't set my timezone where it get it ? from my pc? \[TimeStamp:2018-05-14 15:20:10\]ddd { "@version" =\> "1", "MSG" =\> "ddd\\r", "ho… --- ## [Help with the configuration of ELK](https://discuss.elastic.co/t/help-with-the-configuration-of-elk/130385)
**Author:** [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)\ **Replies:** 3\ **Last updated:** [May 3, 2018, 9:50am UTC](https://discuss.elastic.co/t/help-with-the-configuration-of-elk/130385 "2018-05-03T09:50:20Z")
We have one logstash server(10.89.13.27) one elasticsearch (10.89.13.26) whih space s filled to 100%. Now we have given another elastic server (10.89.13.28). so now my query is one logstash and two elastic server - ca… --- ## [Error Configuration X-pack on Logstash](https://discuss.elastic.co/t/error-configuration-x-pack-on-logstash/130411)
**Author:** [@Charaf\_Ahmed](https://discuss.elastic.co/u/Charaf_Ahmed)\ **Replies:** 1\ **Last updated:** [May 3, 2018, 9:25am UTC](https://discuss.elastic.co/t/error-configuration-x-pack-on-logstash/130411 "2018-05-03T09:25:28Z")
j'ai resou le probleme --- ## [Dead Letter Queue problem](https://discuss.elastic.co/t/dead-letter-queue-problem/130281)
**Author:** [@kernelpanic](https://discuss.elastic.co/u/kernelpanic)\ **Replies:** 2\ **Last updated:** [May 3, 2018, 9:11am UTC](https://discuss.elastic.co/t/dead-letter-queue-problem/130281 "2018-05-03T09:11:01Z")
Hello, I've attempted to add the dead letter queue to my configuration but when I restart logstash it fails to come back up, I've no doubt its something I'm doing wrong, hereis my relevant config: ###-INPUT-### dead\_let… --- ## [Monitoring open tcp connections](https://discuss.elastic.co/t/monitoring-open-tcp-connections/130408)
**Author:** [@mzominy](https://discuss.elastic.co/u/mzominy)\ **Replies:** 0\ **Last updated:** [May 3, 2018, 9:08am UTC](https://discuss.elastic.co/t/monitoring-open-tcp-connections/130408 "2018-05-03T09:08:41Z")
Hi all, Asking for a bit of help here, because we're having a bit of trouble diagnosing an issue with our logstash implementation. In an IOT-like implementation, we have several clients sending logs to our logstash ins… --- ## [Migrating multiple tables from mysql to logstash](https://discuss.elastic.co/t/migrating-multiple-tables-from-mysql-to-logstash/130387)
**Author:** [@anasov](https://discuss.elastic.co/u/anasov)\ **Replies:** 1\ **Last updated:** [May 3, 2018, 9:07am UTC](https://discuss.elastic.co/t/migrating-multiple-tables-from-mysql-to-logstash/130387 "2018-05-03T09:07:39Z")
Hi , ALL, i want to know how can i migrate mutiple tables from MySQL to logstash. I can do migration just for one table --- ## [Logstash output templating \_id defined twice](https://discuss.elastic.co/t/logstash-output-templating-id-defined-twice/130407)
**Author:** [@JensVanDeynse](https://discuss.elastic.co/u/JensVanDeynse)\ **Replies:** 0\ **Last updated:** [May 3, 2018, 9:03am UTC](https://discuss.elastic.co/t/logstash-output-templating-id-defined-twice/130407 "2018-05-03T09:03:01Z")
Hello there I've recently started using the ELK stack but ran into a problem when using templates to map the types. But when doing that I found the following errors in the logstash logs: \[2018-05-03T10:53:45,201\]\[WARN … --- ## [Load new template - Error 400](https://discuss.elastic.co/t/load-new-template-error-400/130402)
**Author:** [@giuliano](https://discuss.elastic.co/u/giuliano)\ **Replies:** 0\ **Last updated:** [May 3, 2018, 8:37am UTC](https://discuss.elastic.co/t/load-new-template-error-400/130402 "2018-05-03T08:37:21Z")
Hi all, I use ELK 6.2.4 and I want to load a new template from the output section because I have change the "target" value in the "geoip {}" plugin. a) My logstash output is: output { if \[type\] == "paloalto" { ela… --- ## [Http output plugin json\_batch format to Http input plugin](https://discuss.elastic.co/t/http-output-plugin-json-batch-format-to-http-input-plugin/127462)
**Author:** [@Aceman87](https://discuss.elastic.co/u/Aceman87)\ **Replies:** 8\ **Last updated:** [May 3, 2018, 8:25am UTC](https://discuss.elastic.co/t/http-output-plugin-json-batch-format-to-http-input-plugin/127462 "2018-05-03T08:25:00Z")
In the documentation there is an alternative to send output through the Http output plugin with the "json\_batch" format. It states: If json\_batch, each batch of events received by this output will be placed into a sing… --- ## [Kafka Input read from beginning after reboot](https://discuss.elastic.co/t/kafka-input-read-from-beginning-after-reboot/130399)
**Author:** [@JohanRask](https://discuss.elastic.co/u/JohanRask)\ **Replies:** 0\ **Last updated:** [May 3, 2018, 8:27am UTC](https://discuss.elastic.co/t/kafka-input-read-from-beginning-after-reboot/130399 "2018-05-03T08:27:09Z")
Hi, we are running logstash-6.1.3 with kafka 1.1 in a test environment. Two logstash instances and four brokers. Yesterday after all machines where suddenly restarted (yes, in test this happens), everything first looke… --- ## [Default "host" value is not fqdn in el7](https://discuss.elastic.co/t/default-host-value-is-not-fqdn-in-el7/130397)
**Author:** [@Jungnam\_Lee](https://discuss.elastic.co/u/Jungnam_Lee)\ **Replies:** 0\ **Last updated:** [May 3, 2018, 8:09am UTC](https://discuss.elastic.co/t/default-host-value-is-not-fqdn-in-el7/130397 "2018-05-03T08:09:36Z")
Hi, I am running logstash 5.6.4 on el6 and el7 hosts. I found el6 shows fqdn for "host" field but el7 shows a short one. How can I make logstash always print fqdn? I tested with the command below: /usr/share/logstash… --- ## [TCP data sending from pyton to logstash fails](https://discuss.elastic.co/t/tcp-data-sending-from-pyton-to-logstash-fails/130358)
**Author:** [@RRSR](https://discuss.elastic.co/u/RRSR)\ **Replies:** 4\ **Last updated:** [May 3, 2018, 5:44am UTC](https://discuss.elastic.co/t/tcp-data-sending-from-pyton-to-logstash-fails/130358 "2018-05-03T05:44:04Z")
Hi, I am trying to send data from python script over TCP and receive the same in the logstash. But there is no exchange of data happening. logstash.conf input { tcp { port =\> 5959 codec =\> json } } … --- ## [Logstash block](https://discuss.elastic.co/t/logstash-block/130235)
**Author:** [@piingluo](https://discuss.elastic.co/u/piingluo)\ **Replies:** 2\ **Last updated:** [May 3, 2018, 2:56am UTC](https://discuss.elastic.co/t/logstash-block/130235 "2018-05-03T02:56:01Z")
Hi all: i was trapped in a question, my logstash instance was blocked but no error. i set the config.reload.automatic and config.reload.interval the logstash-plain.log always says as follows: \[2018-05-02T18:36:56,0… --- ## [Cannot create pipeline for send bro logs to elk](https://discuss.elastic.co/t/cannot-create-pipeline-for-send-bro-logs-to-elk/130342)
**Author:** [@bz\_Os](https://discuss.elastic.co/u/bz_Os)\ **Replies:** 10\ **Last updated:** [May 3, 2018, 2:08am UTC](https://discuss.elastic.co/t/cannot-create-pipeline-for-send-bro-logs-to-elk/130342 "2018-05-03T02:08:22Z")
i attempt to send logs from bro ids to elk i am using filebeat.yml for input configuration when i specifie the path of the logs and the bro-con\_log.conf for the configuration of logstash so when i open /var/log/logstash… --- ## [Unable to start Logstash: Encountered a retryable error. Will Retry with exponential backoff](https://discuss.elastic.co/t/unable-to-start-logstash-encountered-a-retryable-error-will-retry-with-exponential-backoff/130347)
**Author:** [@betty](https://discuss.elastic.co/u/betty)\ **Replies:** 0\ **Last updated:** [May 3, 2018, 1:56am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-encountered-a-retryable-error-will-retry-with-exponential-backoff/130347 "2018-05-03T01:56:46Z")
Hi, Logstash version: 6.1.2 ES version: 6.1.2 I am using Logstash to transferring data to Elasticsearch. After starting logstash it just keeps giving logs like below: \[2018-05-02T18:48:56,892\]\[ERROR\]\[logstash.output… --- ## [Logstash FATAL error when cannot connect to elasticsearch - v6.2.3](https://discuss.elastic.co/t/logstash-fatal-error-when-cannot-connect-to-elasticsearch-v6-2-3/129377)
**Author:** [@sunny12](https://discuss.elastic.co/u/sunny12)\ **Replies:** 2\ **Last updated:** [May 2, 2018, 6:47pm UTC](https://discuss.elastic.co/t/logstash-fatal-error-when-cannot-connect-to-elasticsearch-v6-2-3/129377 "2018-05-02T18:47:32Z")
Hi there, Thanking in advance for any replies!! Symptom: If logstash cannot connect to elasticsearch, logstash has a fatal error and shutdown completely. Expected behavior: If cannot connect to elasticsearch, or ther… --- ## [I cannot scp file to the client server](https://discuss.elastic.co/t/i-cannot-scp-file-to-the-client-server/130310)
**Author:** [@Temitope1](https://discuss.elastic.co/u/Temitope1)\ **Replies:** 0\ **Last updated:** [May 2, 2018, 6:20pm UTC](https://discuss.elastic.co/t/i-cannot-scp-file-to-the-client-server/130310 "2018-05-02T18:20:39Z")
Hello everyone, I am new at setting up ELK, but I have install everything except for setting up filebeat and sharing cert with client server, so i use the below command to copy the file but i got permission denied. is t… --- ## [Logstash - extraction substring from the raw message](https://discuss.elastic.co/t/logstash-extraction-substring-from-the-raw-message/130162)
**Author:** [@jfnoteba](https://discuss.elastic.co/u/jfnoteba)\ **Replies:** 3\ **Last updated:** [May 2, 2018, 4:59pm UTC](https://discuss.elastic.co/t/logstash-extraction-substring-from-the-raw-message/130162 "2018-05-02T16:59:21Z")
Hi Everyone, I'm trying to see if there is any logstash plugin which we can use to perform a quick filtering on 2 substrings without need to parse the entire message using grok command: An example of the log is below a… --- ## [Ingest data from Postgresql DB server to logstash](https://discuss.elastic.co/t/ingest-data-from-postgresql-db-server-to-logstash/130022)
**Author:** [@kaushalshriyan](https://discuss.elastic.co/u/kaushalshriyan)\ **Replies:** 3\ **Last updated:** [May 2, 2018, 4:51pm UTC](https://discuss.elastic.co/t/ingest-data-from-postgresql-db-server-to-logstash/130022 "2018-05-02T16:51:43Z")
Hi, https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html mentions about ingest data from MySQL to logstash. Any instructions to ingest data from Postgresql DB server to logstash? Any help will be h… --- ## [Date parsed but not properly indexed](https://discuss.elastic.co/t/date-parsed-but-not-properly-indexed/130290)
**Author:** [@Jean-Charle\_Passepon](https://discuss.elastic.co/u/Jean-Charle_Passepon)\ **Replies:** 1\ **Last updated:** [May 2, 2018, 4:17pm UTC](https://discuss.elastic.co/t/date-parsed-but-not-properly-indexed/130290 "2018-05-02T16:17:30Z")
Hello, Going through logstash 6.2.3, I'm parsing some JSON from a message log and then from this json, I'm parsing the DATETIME. When the json arrived, I've a field like this: "image\_time\_stamp": "2018-05-02-15:43:30… --- ## [Invalid Frame Type, received: 69/84](https://discuss.elastic.co/t/invalid-frame-type-received-69-84/130234)
**Author:** [@lauea](https://discuss.elastic.co/u/lauea)\ **Replies:** 3\ **Last updated:** [May 2, 2018, 3:43pm UTC](https://discuss.elastic.co/t/invalid-frame-type-received-69-84/130234 "2018-05-02T15:43:52Z")
Hi Guy, I always encounter below issue from logstash when shipping log from filebeat to logstash, but if can ship to elasticsearch by filebeat directly. Handling exception: org.logstash.beats.BeatsParser$InvalidFrameP… --- ## [Simple grok filter blocking Logstash](https://discuss.elastic.co/t/simple-grok-filter-blocking-logstash/130262)
**Author:** [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Replies:** 3\ **Last updated:** [May 2, 2018, 2:39pm UTC](https://discuss.elastic.co/t/simple-grok-filter-blocking-logstash/130262 "2018-05-02T14:39:00Z")
Hi, I just created the following grok filter and it causes Logstash to go totally berserk. It produces lots of load until it ultimately stops working. Sometimes one or another logevent comes through but they keep piling… --- ## [Date Filter not working, not even Indexing with \_dateparsfailure tag](https://discuss.elastic.co/t/date-filter-not-working-not-even-indexing-with-dateparsfailure-tag/130257)
**Author:** [@Sayantan\_Dey](https://discuss.elastic.co/u/Sayantan_Dey)\ **Replies:** 3\ **Last updated:** [May 2, 2018, 2:36pm UTC](https://discuss.elastic.co/t/date-filter-not-working-not-even-indexing-with-dateparsfailure-tag/130257 "2018-05-02T14:36:46Z")
I have one field named event-reporting-time which has a string like this: "Mar 27 2018 02:32:10 AM -07:00", I have used the following filter : date { match =\> \[ "event-reporting-time", "MMM dd yyyy KK:mm:ss aa ZZ", "MM… --- ## [Will Retry with exponential backoff](https://discuss.elastic.co/t/will-retry-with-exponential-backoff/130147)
**Author:** [@Joe\_Franklin](https://discuss.elastic.co/u/Joe_Franklin)\ **Replies:** 2\ **Last updated:** [May 2, 2018, 2:25pm UTC](https://discuss.elastic.co/t/will-retry-with-exponential-backoff/130147 "2018-05-02T14:25:25Z")
I need some help on how to track down the cause of this error: \[2018-05-01T17:40:37,503\]\[ERROR\]\[logstash.outputs.elasticsearch\] Encountered a retryable error. Will Retry with exponential backoff {:code=\>400, :url=\>"http… --- ## [The logstash keep getting 'UNEXPECTED POOL ERROR', even the Elastic Search is in GREEN mode](https://discuss.elastic.co/t/the-logstash-keep-getting-unexpected-pool-error-even-the-elastic-search-is-in-green-mode/130132)
**Author:** [@michael1412](https://discuss.elastic.co/u/michael1412)\ **Replies:** 4\ **Last updated:** [May 2, 2018, 1:59pm UTC](https://discuss.elastic.co/t/the-logstash-keep-getting-unexpected-pool-error-even-the-elastic-search-is-in-green-mode/130132 "2018-05-02T13:59:29Z")
Version: Logstash 5.5, AWS Elasticsearch 5.5 Operating System: Linux CentOS 7(Logstash) Sample Error: \[2018-05-01T12:05:52,235\]\[WARN \]\[logstash.outputs.elasticsearch\] Marking url as dead. Last error: \[LogStash::Outputs… --- ## [Csv parsing through logstash](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 3\ **Last updated:** [May 2, 2018, 1:54pm UTC](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249 "2018-05-02T13:54:23Z")
Hi All, Iam trying to parse my csv file to elasticsearch through logstash my config looks like this input { file { path =\> "/home/raj/Uge 16 Spyware-malware total7.csv" type =\> "trend\_micro\_s… --- ## [Pipeline terminates in jdbc static filter plugin](https://discuss.elastic.co/t/pipeline-terminates-in-jdbc-static-filter-plugin/130204)
**Author:** [@manjur](https://discuss.elastic.co/u/manjur)\ **Replies:** 4\ **Last updated:** [May 2, 2018, 1:25pm UTC](https://discuss.elastic.co/t/pipeline-terminates-in-jdbc-static-filter-plugin/130204 "2018-05-02T13:25:02Z")
I have used this config file to perform joins on two tables. I used jdbc static filter plug in .When i run logstash it terminates. input { } filter { jdbc\_static { loaders =\> \[ { id =\> "tbl\_farmer" query =\> "sele… --- ## [No automatic logstash shutdown on Elasticsearch Input Plugin](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887)
**Author:** [@sbienert](https://discuss.elastic.co/u/sbienert)\ **Replies:** 9\ **Last updated:** [May 2, 2018, 12:21pm UTC](https://discuss.elastic.co/t/no-automatic-logstash-shutdown-on-elasticsearch-input-plugin/128887 "2018-05-02T12:21:03Z")
I have an index X which continuosly receives maschine logs. I need to reindex the data constantly to index Y which is connected to Kibana. I have a Elasticsearch input, filter and Elasticsearch output. The reindexing wor… --- ## [How grok filter use more cpu?](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 4\ **Last updated:** [May 2, 2018, 12:18pm UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194 "2018-05-02T12:18:36Z")
Hi, I have used grok filter to match the first line after the second line, When config file is loaded then it used more CPU I am not getting why it is happening here is my grok pattern (?\hd\_bus\_rxctl: rx… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=767) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=769)