# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=682 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 683 --- ## [Cannot be changed from type \[float\] to \[long\]](https://discuss.elastic.co/t/cannot-be-changed-from-type-float-to-long/147710)
**Author:** [@dmccuk](https://discuss.elastic.co/u/dmccuk)\ **Replies:** 6\ **Last updated:** [September 21, 2018, 12:18pm UTC](https://discuss.elastic.co/t/cannot-be-changed-from-type-float-to-long/147710 "2018-09-21T12:18:31Z")
Hi, I've got an issue with logstash/Elasticsearch... I'm getting the messages in my logstash log: \[2018-09-07T02:00:01,438\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400,… --- ## [Map twitter](https://discuss.elastic.co/t/map-twitter/149245)
**Author:** [@renaudpro](https://discuss.elastic.co/u/renaudpro)\ **Replies:** 4\ **Last updated:** [September 21, 2018, 12:16pm UTC](https://discuss.elastic.co/t/map-twitter/149245 "2018-09-21T12:16:59Z")
En anglais/français Hello, I want to set up a map with the number of tweets per country. I get the tweet well but impossible to set up the map. Here is what I currently have: twitter.confinput { twitter { consu… --- ## [Unable to load Configuration file into Logstash](https://discuss.elastic.co/t/unable-to-load-configuration-file-into-logstash/149403)
**Author:** [@Divya\_Boggavarapu](https://discuss.elastic.co/u/Divya_Boggavarapu)\ **Replies:** 4\ **Last updated:** [September 21, 2018, 10:24am UTC](https://discuss.elastic.co/t/unable-to-load-configuration-file-into-logstash/149403 "2018-09-21T10:24:14Z")
I'm trying to load basic configuration file into logstash but logstash doesn't seem to load the files and it's showing error as Sending Logstash logs to D:/elk/logstash-6.4.1/logs which is now configured via log4j2.prop… --- ## [DLQ Error: invalid checksum of record](https://discuss.elastic.co/t/dlq-error-invalid-checksum-of-record/149427)
**Author:** [@dorinand](https://discuss.elastic.co/u/dorinand)\ **Replies:** 0\ **Last updated:** [September 21, 2018, 10:00am UTC](https://discuss.elastic.co/t/dlq-error-invalid-checksum-of-record/149427 "2018-09-21T10:00:03Z")
I am trying to drop all messages from dlq. input { dead\_letter\_queue { path =\> "/var/lib/logstash/dead\_letter\_queue" commit\_offsets =\> true pipeline\_id =\> "default-pipeline" } } filter… --- ## [How to replace @timestamp with actual log time](https://discuss.elastic.co/t/how-to-replace-timestamp-with-actual-log-time/149276)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 18\ **Last updated:** [September 21, 2018, 9:22am UTC](https://discuss.elastic.co/t/how-to-replace-timestamp-with-actual-log-time/149276 "2018-09-21T09:22:15Z")
I am using below grok filter to parse the log , \`filter { grok { match =\> \[ "message", "%{TIMESTAMP\_ISO8601:timestamp} (\\\[%{WORD:loglevel}\\\]) %{DATA} - %{DATA:method} processing time for transactionId : … --- ## [Basic filebeats =\> logstash =\> elastic setup](https://discuss.elastic.co/t/basic-filebeats-logstash-elastic-setup/149323)
**Author:** [@nuketro0p3r](https://discuss.elastic.co/u/nuketro0p3r)\ **Replies:** 3\ **Last updated:** [September 21, 2018, 8:30am UTC](https://discuss.elastic.co/t/basic-filebeats-logstash-elastic-setup/149323 "2018-09-21T08:30:46Z")
I'm trying to debug / check if I got everything right with my logstash setup History: The logs fed from Filebeats to ES work perfectly Only downside is that the geoip information is not set I tried to add logstash in… --- ## [Unable to create dynamic index with conditional check in logstash](https://discuss.elastic.co/t/unable-to-create-dynamic-index-with-conditional-check-in-logstash/148981)
**Author:** [@anra137](https://discuss.elastic.co/u/anra137)\ **Replies:** 2\ **Last updated:** [September 21, 2018, 8:21am UTC](https://discuss.elastic.co/t/unable-to-create-dynamic-index-with-conditional-check-in-logstash/148981 "2018-09-21T08:21:03Z")
Hi, I want to dynamically create index based on the events. I have configured configured several log files in filebeat and the events are successfully stored in ElasticSearch. But when I try to differentiate the logs in… --- ## [Elk snort](https://discuss.elastic.co/t/elk-snort/149353)
**Author:** [@Pirate\_Trkmn](https://discuss.elastic.co/u/Pirate_Trkmn)\ **Replies:** 7\ **Last updated:** [September 21, 2018, 7:28am UTC](https://discuss.elastic.co/t/elk-snort/149353 "2018-09-21T07:28:18Z")
Hi i am configuring logstash for snort logs following few tutorials from web, but can not parse snort logs i have \[%{INT:ids\_gid}:%{INT:ids\_sid}:%{INT:ids\_rev}\].%{GREEDYDATA:ids\_alert}.\[Classification: %{DATA:ids\_classi… --- ## [Logstash integration with another ETL pipeline](https://discuss.elastic.co/t/logstash-integration-with-another-etl-pipeline/149408)
**Author:** [@fenster25](https://discuss.elastic.co/u/fenster25)\ **Replies:** 0\ **Last updated:** [September 21, 2018, 7:24am UTC](https://discuss.elastic.co/t/logstash-integration-with-another-etl-pipeline/149408 "2018-09-21T07:24:01Z")
Hi, I am working on a project similar to https://github.com/compose/transporter but with more support for other high-level data sources like MySQL, Redis, firestore etc I was looking for a way to integrate this with lo… --- ## [Pipeline on error: recurrent problem](https://discuss.elastic.co/t/pipeline-on-error-recurrent-problem/149406)
**Author:** [@Michel99\_7](https://discuss.elastic.co/u/Michel99_7)\ **Replies:** 0\ **Last updated:** [September 21, 2018, 7:16am UTC](https://discuss.elastic.co/t/pipeline-on-error-recurrent-problem/149406 "2018-09-21T07:16:39Z")
Hi all, We are having an error when a pipeline is starting, The log provides the following error. Any hint about how to make a diagnosis and to solve the problem (file corruption?) OS: Red Hat Enterprise Linux Server… --- ## [Convert gb mb kb to bytes](https://discuss.elastic.co/t/convert-gb-mb-kb-to-bytes/149126)
**Author:** [@Arun\_Murugappan](https://discuss.elastic.co/u/Arun_Murugappan)\ **Replies:** 5\ **Last updated:** [September 21, 2018, 6:13am UTC](https://discuss.elastic.co/t/convert-gb-mb-kb-to-bytes/149126 "2018-09-21T06:13:08Z")
The below script works fine only if there is no whitespace in the data. Firstly, example 20mb, 2gb. it works fine. What if 20 mb or 2 gb. Secondly, what if the given data itself is in bytes, how do i include that?\`\`\` sp… --- ## [How to remove data string if NULL from user-agent filtered](https://discuss.elastic.co/t/how-to-remove-data-string-if-null-from-user-agent-filtered/148400)
**Author:** [@mark.quilates](https://discuss.elastic.co/u/mark.quilates)\ **Replies:** 7\ **Last updated:** [September 21, 2018, 2:34am UTC](https://discuss.elastic.co/t/how-to-remove-data-string-if-null-from-user-agent-filtered/148400 "2018-09-21T02:34:15Z")
Hi Guys, Is there way to remove, OS field %{os\_major}.%{os\_minor} and NAME field %{major}.%{minor} of user-agent? if the value is NULL? Because, it looks like this on my Kibana, kinda irritating on my eyes. Below ar… --- ## [Kafka -\> Logstash -\> ES ..... using avro\_schema\_registry plugin & decorate\_events & mutate = Fail. I would like to get Key info into ES using Mutate](https://discuss.elastic.co/t/kafka-logstash-es-using-avro-schema-registry-plugin-decorate-events-mutate-fail-i-would-like-to-get-key-info-into-es-using-mutate/146916)
**Author:** [@Robert\_Duffy](https://discuss.elastic.co/u/Robert_Duffy)\ **Replies:** 1\ **Last updated:** [September 20, 2018, 10:27pm UTC](https://discuss.elastic.co/t/kafka-logstash-es-using-avro-schema-registry-plugin-decorate-events-mutate-fail-i-would-like-to-get-key-info-into-es-using-mutate/146916 "2018-09-20T22:27:17Z")
I am not having success with the following: Kafka Input (both key & body avro schema are in schema registry) I would like to pull fields from they Key and combine with the body and output to ES. decorate\_events=true, … --- ## [Error integrating DB postgress to Logstash](https://discuss.elastic.co/t/error-integrating-db-postgress-to-logstash/149363)
**Author:** [@nusuga](https://discuss.elastic.co/u/nusuga)\ **Replies:** 0\ **Last updated:** [September 20, 2018, 9:49pm UTC](https://discuss.elastic.co/t/error-integrating-db-postgress-to-logstash/149363 "2018-09-20T21:49:30Z")
I´m try to integrate a DB postgress query in elasticsearch/Logstash, but I'm having this problem: \[ERROR\]\[logstash.pipeline \] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"\ file-\> path…wildcard \* not working](https://discuss.elastic.co/t/logstash-input-file-path-wildcard-not-working/149133)
**Author:** [@Blake\_2112](https://discuss.elastic.co/u/Blake_2112)\ **Replies:** 5\ **Last updated:** [September 20, 2018, 9:07pm UTC](https://discuss.elastic.co/t/logstash-input-file-path-wildcard-not-working/149133 "2018-09-20T21:07:23Z")
Working on a Windows 10 system and for the life of me I cannot get this system to recognize a wildcard path statement. ELK stack is 6.4.0 I have had this work in the past on different versions and/or systems and I canno… --- ## [Logstash json filter error](https://discuss.elastic.co/t/logstash-json-filter-error/149117)
**Author:** [@chowdary](https://discuss.elastic.co/u/chowdary)\ **Replies:** 3\ **Last updated:** [September 20, 2018, 8:19pm UTC](https://discuss.elastic.co/t/logstash-json-filter-error/149117 "2018-09-20T20:19:24Z")
Hi , i am facing below error, while running the logstash . ←\[33mError parsing json {:source=\>"message", :raw=\>"}\\r", :exception=\>#\ **Author:** [@jworl](https://discuss.elastic.co/u/jworl)\ **Replies:** 1\ **Last updated:** [September 20, 2018, 7:53pm UTC](https://discuss.elastic.co/t/unix-socket-not-present-in-logs-but-exists-and-verified-with-netcat/149174 "2018-09-20T19:53:30Z") Hello, I have a Ceph storage cluster for S3 Object storage, and the radosgw service uses a Unix socket for logs. I have verified the socket works properly with netcat, and my work around as of late has been to pipe this… --- ## [Useragent filter plugin reports false IE version](https://discuss.elastic.co/t/useragent-filter-plugin-reports-false-ie-version/149344)
**Author:** [@larryhe](https://discuss.elastic.co/u/larryhe)\ **Replies:** 0\ **Last updated:** [September 20, 2018, 6:57pm UTC](https://discuss.elastic.co/t/useragent-filter-plugin-reports-false-ie-version/149344 "2018-09-20T18:57:46Z")
On our Kibana browser report, a lot of IE11 reported as IE7 which is very confusing to our whole engineering team. for example, given below user agent string Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Tri… --- ## [Cant access @metadata in DLQ](https://discuss.elastic.co/t/cant-access-metadata-in-dlq/149004)
**Author:** [@dorinand](https://discuss.elastic.co/u/dorinand)\ **Replies:** 1\ **Last updated:** [September 20, 2018, 5:34pm UTC](https://discuss.elastic.co/t/cant-access-metadata-in-dlq/149004 "2018-09-20T17:34:55Z")
I configure DLQ queue for my pipeline. It will create \-dlq index in elasticsearch. I can see my documents but I would like to see the problem that cause that the document end in DLQ. According to this article, ther… --- ## [Any plan to support graphite tag in Logstash output](https://discuss.elastic.co/t/any-plan-to-support-graphite-tag-in-logstash-output/148990)
**Author:** [@Jehutywong](https://discuss.elastic.co/u/Jehutywong)\ **Replies:** 1\ **Last updated:** [September 20, 2018, 5:34pm UTC](https://discuss.elastic.co/t/any-plan-to-support-graphite-tag-in-logstash-output/148990 "2018-09-20T17:34:00Z")
From the release of the Graphite 1.1.x, it supports storing data using tags to identify series. But the graphite output plugin, come with the latest Logstash, still has no option to support this function. A workaround … --- ## [Logstash config - JSON filter results with "\_jsonparsefailure"](https://discuss.elastic.co/t/logstash-config-json-filter-results-with-jsonparsefailure/149281)
**Author:** [@Ira](https://discuss.elastic.co/u/Ira)\ **Replies:** 5\ **Last updated:** [September 20, 2018, 5:17pm UTC](https://discuss.elastic.co/t/logstash-config-json-filter-results-with-jsonparsefailure/149281 "2018-09-20T17:17:01Z")
Hi, My log file is in JSON format. An example of a log line: {"@timestamp":"2018-09-18T10:36:27.135+03:00","@version":1,"message":{"apiMethodType": "GET","elapsedTime": 1136},"caller\_line\_number":94} I'm using Filebe… --- ## [Logstash field auto-merge & remove with a mutate filter and a strict dynamic mapping index template](https://discuss.elastic.co/t/logstash-field-auto-merge-remove-with-a-mutate-filter-and-a-strict-dynamic-mapping-index-template/149324)
**Author:** [@Joey\_Ross](https://discuss.elastic.co/u/Joey_Ross)\ **Replies:** 0\ **Last updated:** [September 20, 2018, 3:55pm UTC](https://discuss.elastic.co/t/logstash-field-auto-merge-remove-with-a-mutate-filter-and-a-strict-dynamic-mapping-index-template/149324 "2018-09-20T15:55:25Z")
Sorry for the wordy title. We have a need for Logstash to merge and drop fields dynamically. We have to enforce strict mapping on the backend Elasticsearch index template in order to keep Elasticsearch performance as we… --- ## [HTTP/2 support for Logstash http input/output plugins](https://discuss.elastic.co/t/http-2-support-for-logstash-http-input-output-plugins/149316)
**Author:** [@SnirX](https://discuss.elastic.co/u/SnirX)\ **Replies:** 0\ **Last updated:** [September 20, 2018, 2:39pm UTC](https://discuss.elastic.co/t/http-2-support-for-logstash-http-input-output-plugins/149316 "2018-09-20T14:39:18Z")
Hi, Is it planned to add HTTP/2 support for Logstash http input/output plugins? Didn't find anything about it in docs and code. Thanks. --- ## [Trying to parse the content of a json into different fields](https://discuss.elastic.co/t/trying-to-parse-the-content-of-a-json-into-different-fields/149202)
**Author:** [@ratman](https://discuss.elastic.co/u/ratman)\ **Replies:** 5\ **Last updated:** [September 20, 2018, 2:14pm UTC](https://discuss.elastic.co/t/trying-to-parse-the-content-of-a-json-into-different-fields/149202 "2018-09-20T14:14:18Z")
Hello, I am quite new in elasticsearch/logstash/kibana but I am trying to build a small dashboard with some logs I am getting from different machines. The logs are formed by statement like this : INFO - TEXT - Sat Aug … --- ## [Adding parameters from grok to syslog output](https://discuss.elastic.co/t/adding-parameters-from-grok-to-syslog-output/149269)
**Author:** [@karasmeitar](https://discuss.elastic.co/u/karasmeitar)\ **Replies:** 0\ **Last updated:** [September 20, 2018, 10:23am UTC](https://discuss.elastic.co/t/adding-parameters-from-grok-to-syslog-output/149269 "2018-09-20T10:23:07Z")
Hi All, I'm trying to add my syslog ip server from in my grok filter and consume it in my logstash syslog output. In the filter section i'm adding the following: mutate { add\_field =\> { "IP" =\> "10.10.10.1" "port"… --- ## [Time out executing grok](https://discuss.elastic.co/t/time-out-executing-grok/149232)
**Author:** [@qloun](https://discuss.elastic.co/u/qloun)\ **Replies:** 4\ **Last updated:** [September 20, 2018, 8:53am UTC](https://discuss.elastic.co/t/time-out-executing-grok/149232 "2018-09-20T08:53:40Z")
Hello, I have problem with getting timeout executing grok. what I have learn that the value exceed limit. the error " Timeout executing grok against 'message' with value 'Value too large to output (1006 bytes)! First 255… --- ## [Add field with the length of value of other field](https://discuss.elastic.co/t/add-field-with-the-length-of-value-of-other-field/149249)
**Author:** [@alfonso.viso](https://discuss.elastic.co/u/alfonso.viso)\ **Replies:** 1\ **Last updated:** [September 20, 2018, 8:48am UTC](https://discuss.elastic.co/t/add-field-with-the-length-of-value-of-other-field/149249 "2018-09-20T08:48:52Z")
Hi all, I'm trying to create a new field which value must be the length of the string of another field. I have found an advice using ruby code, like this: //ruby { // code =\> "event.set\['http\_user\_agent\_length'\] = … --- ## [Error Config send log from Windows client to ELK Server Ubuntu](https://discuss.elastic.co/t/error-config-send-log-from-windows-client-to-elk-server-ubuntu/148423)
**Author:** [@trunght2](https://discuss.elastic.co/u/trunght2)\ **Replies:** 6\ **Last updated:** [September 20, 2018, 7:38am UTC](https://discuss.elastic.co/t/error-config-send-log-from-windows-client-to-elk-server-ubuntu/148423 "2018-09-20T07:38:35Z")
Hi everypeople, I setup LAB for my room. I has been finished setup lab. but config send log from windows 10 to ELK Ubuntu has problem. 2018-09-13T15:44:20.359+0700 ERROR pipeline/output.go:91 Failed to connect: read… --- ## [429 error](https://discuss.elastic.co/t/429-error/149237)
**Author:** [@af615dbd55cac2cacf32](https://discuss.elastic.co/u/af615dbd55cac2cacf32)\ **Replies:** 1\ **Last updated:** [September 20, 2018, 7:12am UTC](https://discuss.elastic.co/t/429-error/149237 "2018-09-20T07:12:34Z")
My Logstash nodes started logging "retrying failed action with response code: 429" error lines. I attached part of my logs below. Q1. There are lines as below. "Retrying indivisual bulk ctions that failed or were rejec… --- ## [How to Monitor logstash log file filtering finish or not?](https://discuss.elastic.co/t/how-to-monitor-logstash-log-file-filtering-finish-or-not/149223)
**Author:** [@veerendranath\_lukkan](https://discuss.elastic.co/u/veerendranath_lukkan)\ **Replies:** 3\ **Last updated:** [September 20, 2018, 6:36am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-log-file-filtering-finish-or-not/149223 "2018-09-20T06:36:29Z")
logfile filter completed status apis to monitor with out elasticsearch [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=681) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=683)