I am fetching (input) logs from Azure via eventhub. The content of logstash “output” looks ok in stdout, but when sending them to blob storage it only contains a emty “message” header. I am using tuffk/Logstash-output-t…
---
## [ERROR: Starting the Logstash with Xpack](https://discuss.elastic.co/t/error-starting-the-logstash-with-xpack/186008)
I have enabled xpack in elasticsearch and kibana which is working fine. I tried to enable it in logstash and used filebeat to ingest the data. But while starting the logstash it is spitting out the following error: ( I …
---
## [\[Logstash Monitoring\] Warning about document types](https://discuss.elastic.co/t/logstash-monitoring-warning-about-document-types/186228)
Hi all, I am getting the following warning when I restarted logstash after upgrading from 6.6.1 to 6.7.2 Logstash version: 6.7.2 Elasticsearch version: 7.0.0 \[2019-06-18T10:47:50,525\]\[WARN \]\[logstash.outputs.elastics…
---
## [Question about read mode vs tail mode](https://discuss.elastic.co/t/question-about-read-mode-vs-tail-mode/186227)
Hi, I'm processing files that comes from an external server; so once copied, files are static and no new content will be added into the file. Originally I was reading the files in tail mode but I've experienced some is…
---
## [AWS sdk from logstash](https://discuss.elastic.co/t/aws-sdk-from-logstash/186186)
Hi, I have a requirement to decrypt the log line via KMS , do we have any way to call aws sdk from logstash or any KMS filter. Please suggest. Regards Ajay
---
## [Setting schedule for logstash jmx input](https://discuss.elastic.co/t/setting-schedule-for-logstash-jmx-input/185314)
hi all, I am using jmx input plugin to ship MBeans into elasticsearch. logstash get the value of each defined attribute as a separate event. in order to gather the value of same attributes in one event the aggregate fil…
---
## [Zero size file going into input directory](https://discuss.elastic.co/t/zero-size-file-going-into-input-directory/183206)
in our system getting zero size message by input plugin to output plugin, how to handle this?
---
## [Logstash not procession all events in custom output plugin](https://discuss.elastic.co/t/logstash-not-procession-all-events-in-custom-output-plugin/186179)
there is an output HTTP custom plugin with wokers\_not\_supported in logstash 2.4 which didn't implement multi\_recieve and get input from a lot of files input. I am seeing only a few file input going but not all. But after…
---
## [Logstash 2.3 CSV import Error](https://discuss.elastic.co/t/logstash-2-3-csv-import-error/186142)
Importing CSV file to index my\_index-\[0-9\] (0-9 is the last number from "owner\_id", i have 10 indices, it's just based on the last digit so..) input{ file{ path =\> "/data/logstash-2.3.1/conf/export.csv" …
---
## [JSON carriage return](https://discuss.elastic.co/t/json-carriage-return/186130)
So by adding a physical carriage return logstash will ingest data. My json data does not do a carriage return and \\n does not work either. Anyone have any suggestions?
---
## [New index not created; data pushed into the existing index](https://discuss.elastic.co/t/new-index-not-created-data-pushed-into-the-existing-index/186158)
Hi, I have setup ELK stack with 7.0 version. When I setup it created index with name logstash-2019.06.16-000001 post that its not create any indexes for 17th June. All the new data generated for 17th june is written to…
---
## [LogStash::Json::ParserError: Invalid numeric value: Leading zeroes not allowed](https://discuss.elastic.co/t/logstash-invalid-numeric-value-leading-zeroes-not-allowed/186150)
Hi, I'm trying to parse some json strings which contain an integer field with the value duration":0000. I tried to convert it to string type but didn't work. I'm still receiving the following error. #\
**Author:** [@legacyboy](https://discuss.elastic.co/u/legacyboy)\
**Replies:** 7\
**Last updated:** [June 17, 2019, 7:59pm UTC](https://discuss.elastic.co/t/grok-stopping-output-to-elasticsearch-from-logstash/185914 "2019-06-17T19:59:41Z")
I am having an issue with grok on my syslog content. I am getting logs but when I try to use grok on it everything stops. Sample data; \<85\>Jun 14 16:38:36--5:00 10.128.107.37 Action="drop" UUid="{0x0,0x0,0x0,0x0}" inz…
---
## [Logstash add\_field issue](https://discuss.elastic.co/t/logstash-add-field-issue/186121)
I am aware you can replace a field but I need to copy the content from the field agent.\_id to a new field called AgentIdentifier for testing purposes . mutate { add\_field =\> {"AgentIdentifier", "%{\[fields\]\[agent.\_id\]}"…
---
## [Set root for json codec](https://discuss.elastic.co/t/set-root-for-json-codec/185906)
Using the json\_codec on input I get all my fields parsed at the root level. I like to specify a name for the subfield for the items. So right now with { "a" : "b", "c" : "d" } to get a = b and c = d. I'd like to get myn…
---
## [Logstash 7.1 and \_geoip\_lookup\_failure](https://discuss.elastic.co/t/logstash-7-1-and-geoip-lookup-failure/186101)
Hi there, I'm somehow battling with geoip plugin in logstash in 7.1. I'm sending data to logstash and I do expect logstash will perform geoip lookup. The fileds with ip data (ie. destination.ip) are present in the sou…
---
## [Logstash Uknown error while parsing user agent data --\>how to debug further?](https://discuss.elastic.co/t/logstash-uknown-error-while-parsing-user-agent-data-how-to-debug-further/186096)
I have a agent string from apache like: " Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" and it seems to fail as I see a lot of \[2019-06-17T16:40:49,679\]\[ERROR\]\[logstash.filters.useragent\] Uknow…
---
## [Logstash csv json quotes nightmare](https://discuss.elastic.co/t/logstash-csv-json-quotes-nightmare/185982)
Hello, I'm trying to parse a Kaggle movies dataset, which are csv files, but I'm having a lot of headaches with the fields that actually hold a json object... I'm now almost done with it, but I had to do a lot of, perh…
---
## [Logstash 6.2 json parse failure](https://discuss.elastic.co/t/logstash-6-2-json-parse-failure/185995)
I have a message below that is failing due to jsonparse error. If I remove all the \\ (backslashes) from the message in text editor and check in json parser, it parsed it properly. Now, to get rid of the \\ in the json me…
---
## [Check if field is in array](https://discuss.elastic.co/t/check-if-field-is-in-array/186052)
Hi here, Thanks in advance for your time, I'm struggling here with a very basic thing but I keep finding examples on the web working for others and not for me ... I'm using logstash 7.0.0 and here is my conf : filter…
---
## [Illegal reflective access by com.headius.backport9.modules.Modules](https://discuss.elastic.co/t/illegal-reflective-access-by-com-headius-backport9-modules-modules/186035)
Hello, I am trying to run logstash 7.1 with java 11. I start the service with systemctl start logstash.servise and I get a warning when I check the service status: ● logstash.service - logstash Loaded: loaded (/etc/s…
---
## [Csv filter separator and some value in cloumn is same](https://discuss.elastic.co/t/csv-filter-separator-and-some-value-in-cloumn-is-same/186045)
Hi, "ID","FP\_TRANSACTION\_ID","REQ\_BITSTREAM","REP\_BITSTREAM","FAULT\_BITSTREAM","FWD\_BITSTREAM" 23596191,23596274,\
**Author:** [@elasticnubie](https://discuss.elastic.co/u/elasticnubie)\
**Replies:** 1\
**Last updated:** [June 17, 2019, 12:56pm UTC](https://discuss.elastic.co/t/logstash-grok-regex-bug/185981 "2019-06-17T12:56:17Z")
Logstash grok regex match result is not same with Kibana Grok Debugger. (I think Kibana Grok Debugger's result is correct.) Is there my fault? or is it grok's bug? Filebeat Config filebeat.inputs: - type: log path…
---
## [Monitor Github repository](https://discuss.elastic.co/t/monitor-github-repository/186068)
Is there a plugin already which can monitor particular git repo and create indices in elasticsearch on basis of new code commited.(i.e. monitoring git pull in specified interval )
---
## [Logstash - Use case to use syslog](https://discuss.elastic.co/t/logstash-use-case-to-use-syslog/186059)
Hi all I have a simple logastash.conf at the moment. I receive from winlogbeat and I store information in the Elasticsearch. Now I'd like to add a new output to syslog. I'd like to send only the log of the administrato…
---
## [Logstash Elasticsearch output - Recommended node types for "hosts"](https://discuss.elastic.co/t/logstash-elasticsearch-output-recommended-node-types-for-hosts/186051)
Hi, I'm currently setting up Logstash and I'm wondering which Elasticsearch nodes I should add into the "hosts" array in the elasticsearch output configuration. elasticsearch { hosts =\> ... ... In my environment I…
---
## [Logstash Monitoring API](https://discuss.elastic.co/t/logstash-monitoring-api/185601)
Hi I'm using logstash monitoring api's to get to know the performance. I executed this curl -XGET 'localhost:9600/\_node/stats/pipeline?pretty' { "host" : "0.0.0.0", "version" : "5.6.13", "http\_address" : "127.0.0.1…
---
## [Java output plugin : How to encode values using existing codec](https://discuss.elastic.co/t/java-output-plugin-how-to-encode-values-using-existing-codec/186036)
Writing java output plugin and need to convert values into the user-specified codec. Maybe JSON, AVRO. Question is can I use existing ruby written codec or need to create my own? Even if we have written java codec then…
---
## [Json object field with name "tags" get converted to array after logstash mutate](https://discuss.elastic.co/t/json-object-field-with-name-tags-get-converted-to-array-after-logstash-mutate/185888)
I have a json in the following format. I am reading this from kafka in logstash. {"id":"8eba168be707e85d","kind":"PRODUCER","name":"scheduler","timestamp":1560526848185066,"duration":23645,"tags":{"class":"TestJob","met…
[Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=525)
[Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=527)