# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=731 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 732 --- ## [Max number of Multiple Pipelines per Logstash instance](https://discuss.elastic.co/t/max-number-of-multiple-pipelines-per-logstash-instance/138074)
**Author:** [@omor](https://discuss.elastic.co/u/omor)\ **Replies:** 2\ **Last updated:** [July 1, 2018, 9:39am UTC](https://discuss.elastic.co/t/max-number-of-multiple-pipelines-per-logstash-instance/138074 "2018-07-01T09:39:02Z")
Is there a recommendation for the number of pipelines configured per LS instance? Will it be logical to configure 60 different pipelines? --- ## [Logstash Monitoring metrics receiver](https://discuss.elastic.co/t/logstash-monitoring-metrics-receiver/138070)
**Author:** [@eperry](https://discuss.elastic.co/u/eperry)\ **Replies:** 0\ **Last updated:** [July 1, 2018, 3:58am UTC](https://discuss.elastic.co/t/logstash-monitoring-metrics-receiver/138070 "2018-07-01T03:58:36Z")
I have a very complex environment spanning 8 different data centers and lots of fire walls It appears the xpack.monitoring only has a HTTP output option ( I need to use Kafka) to stay with the current architecture and a… --- ## [Date field not decoded by snmpTrap plugin](https://discuss.elastic.co/t/date-field-not-decoded-by-snmptrap-plugin/138068)
**Author:** [@JonConnor](https://discuss.elastic.co/u/JonConnor)\ **Replies:** 0\ **Last updated:** [June 30, 2018, 11:46pm UTC](https://discuss.elastic.co/t/date-field-not-decoded-by-snmptrap-plugin/138068 "2018-06-30T23:46:51Z")
Using snmpTrap plugin and all fields are decoded properly except the eventDate field. This is not decoded and appears garbled in the kibana discover page. The string in message field: @name=\[1.3.6.1.4.1.3881.2.1.8.1.4… --- ## [All loading suddenly goes to 1 server at 8AM](https://discuss.elastic.co/t/all-loading-suddenly-goes-to-1-server-at-8am/138060)
**Author:** [@macymin](https://discuss.elastic.co/u/macymin)\ **Replies:** 0\ **Last updated:** [June 30, 2018, 12:52pm UTC](https://discuss.elastic.co/t/all-loading-suddenly-goes-to-1-server-at-8am/138060 "2018-06-30T12:52:00Z")
we have 7 datanode servers, each server running 3 data nodes. it is all the way running fine. but yesterday we experienced something wired is at 8AM, all the load from the rest of servers suddenly transferred to 1 part… --- ## [Load multi mysql table to elasticsearch with logstash 6.3.0](https://discuss.elastic.co/t/load-multi-mysql-table-to-elasticsearch-with-logstash-6-3-0/137037)
**Author:** [@simmonHuong](https://discuss.elastic.co/u/simmonHuong)\ **Replies:** 2\ **Last updated:** [June 30, 2018, 11:28am UTC](https://discuss.elastic.co/t/load-multi-mysql-table-to-elasticsearch-with-logstash-6-3-0/137037 "2018-06-30T11:28:36Z")
I am using logstash to load multi table from mysql to elasticsearch.As suggested I will create index per table. Got two problems: I have to comment if \[type\] == "activity" {,then the data will successfully imported,o… --- ## [Bulk/Async push to http](https://discuss.elastic.co/t/bulk-async-push-to-http/138030)
**Author:** [@Amey\_Khedekar](https://discuss.elastic.co/u/Amey_Khedekar)\ **Replies:** 0\ **Last updated:** [June 29, 2018, 7:28pm UTC](https://discuss.elastic.co/t/bulk-async-push-to-http/138030 "2018-06-29T19:28:28Z")
Hi, We are trying to push Json data into http endpoint via logstash output. We are finding that logstash pushes each record one by one. Is there a way to push via Async or bulk push multiple records? --- ## [Split event into multiple different messages](https://discuss.elastic.co/t/split-event-into-multiple-different-messages/137659)
**Author:** [@jvolpi](https://discuss.elastic.co/u/jvolpi)\ **Replies:** 4\ **Last updated:** [June 29, 2018, 6:50pm UTC](https://discuss.elastic.co/t/split-event-into-multiple-different-messages/137659 "2018-06-29T18:50:12Z")
Hi everyone Is there a way to split an event into multiple messages, and send them to elasticsearh? The problem is the event cannot be splitted in a automatic way using plugins, because the data isn't exactly structure… --- ## [CSV import - import data + import relatioship between data](https://discuss.elastic.co/t/csv-import-import-data-import-relatioship-between-data/138000)
**Author:** [@lvboque](https://discuss.elastic.co/u/lvboque)\ **Replies:** 0\ **Last updated:** [June 29, 2018, 3:10pm UTC](https://discuss.elastic.co/t/csv-import-import-data-import-relatioship-between-data/138000 "2018-06-29T15:10:15Z")
For a project, I am importing data of person in a database (id, first name, last name, ...). I am using logstash for that using configuration file like this one .... (see below). Then, I would like to import a parent/ch… --- ## [Problem with allow\_time\_override](https://discuss.elastic.co/t/problem-with-allow-time-override/137993)
**Author:** [@Tobbis](https://discuss.elastic.co/u/Tobbis)\ **Replies:** 0\ **Last updated:** [June 29, 2018, 2:52pm UTC](https://discuss.elastic.co/t/problem-with-allow-time-override/137993 "2018-06-29T14:52:38Z")
I am using a influxdb and are using logstash to filter and send data to the influxdb but I have a problem forwaring the timestamp in my measurements to the influxdb. I have read a lot of posts that I should use the allo… --- ## [Stalled pipeline help](https://discuss.elastic.co/t/stalled-pipeline-help/137949)
**Author:** [@makibroshett](https://discuss.elastic.co/u/makibroshett)\ **Replies:** 1\ **Last updated:** [June 29, 2018, 2:32pm UTC](https://discuss.elastic.co/t/stalled-pipeline-help/137949 "2018-06-29T14:32:04Z")
Hi, I have a logstash 6.1.1 running in a docker container, and am trying to understand why one our pipeline is not sending data. I stopped the instance and noticed that in the log it mentionned the pipeline was stalled: … --- ## [Varying amount of JSON parse errors when parsing the same files](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864)
**Author:** [@Matjo](https://discuss.elastic.co/u/Matjo)\ **Replies:** 7\ **Last updated:** [June 29, 2018, 2:07pm UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864 "2018-06-29T14:07:22Z")
I have a very simple setup: input { file { path =\> "/home/elasticadmin/azure/\*\*/\*.\*" codec =\> "json" close\_older =\> 5 } } filter {} output { elasticsearch { user =\> "user" password =\> "pwd" in… --- ## [Logstash field limit?](https://discuss.elastic.co/t/logstash-field-limit/137959)
**Author:** [@adolfo.diaz](https://discuss.elastic.co/u/adolfo.diaz)\ **Replies:** 0\ **Last updated:** [June 29, 2018, 12:06pm UTC](https://discuss.elastic.co/t/logstash-field-limit/137959 "2018-06-29T12:06:52Z")
Dear All I would like to know if there is any limitation in the size of the information that can be stored in a field parsed with grok. At the moment I am trying to structure a file and I need to enter in a field more … --- ## [Run logtsash on docker with custom pattern](https://discuss.elastic.co/t/run-logtsash-on-docker-with-custom-pattern/137915)
**Author:** [@MarcusCaepio](https://discuss.elastic.co/u/MarcusCaepio)\ **Replies:** 3\ **Last updated:** [June 29, 2018, 11:09am UTC](https://discuss.elastic.co/t/run-logtsash-on-docker-with-custom-pattern/137915 "2018-06-29T11:09:01Z")
Hi all, how can I run a logstash container with a custom pattern folder? the patterns folder is in the /pipeline folder together with pipeline config --- ## [Logstash config failed](https://discuss.elastic.co/t/logstash-config-failed/137651)
**Author:** [@Abdur\_Raqeeb1](https://discuss.elastic.co/u/Abdur_Raqeeb1)\ **Replies:** 2\ **Last updated:** [June 29, 2018, 10:00am UTC](https://discuss.elastic.co/t/logstash-config-failed/137651 "2018-06-29T10:00:09Z")
HI, Please help me here as my logstash config is getting failed.Please find the error and logstash config file grok { match =\> { "message" =\> { '%{IP:remote\_address} - "remote user" - \["local time" %{HTTPDATE:time}\] "… --- ## [Error: Expected one of #, {, ,, \]](https://discuss.elastic.co/t/error-expected-one-of/137891)
**Author:** [@marcelo](https://discuss.elastic.co/u/marcelo)\ **Replies:** 2\ **Last updated:** [June 29, 2018, 7:57am UTC](https://discuss.elastic.co/t/error-expected-one-of/137891 "2018-06-29T07:57:18Z")
I noticed that over time my logstash stops receiving information from my filebeats, analyzing the log I am getting this error message: \[2018-06-28T23:55:46,344\]\[ERROR\]\[logstash.agent \] Failed to execute action… --- ## [Elastics search error triggered by logstash Grok Pattern](https://discuss.elastic.co/t/elastics-search-error-triggered-by-logstash-grok-pattern/137859)
**Author:** [@warrior](https://discuss.elastic.co/u/warrior)\ **Replies:** 1\ **Last updated:** [June 29, 2018, 6:48am UTC](https://discuss.elastic.co/t/elastics-search-error-triggered-by-logstash-grok-pattern/137859 "2018-06-29T06:48:56Z")
@dadoonet @magnusbaeck Magnus Bäck My Elasticsearch is dying with this groke pattern and the following is the error from es. Please let me know whether these groke patterns can be optimized for optimal performance of… --- ## [Lostash taking inputs from Filebeat](https://discuss.elastic.co/t/lostash-taking-inputs-from-filebeat/136740)
**Author:** [@pkshara](https://discuss.elastic.co/u/pkshara)\ **Replies:** 8\ **Last updated:** [June 29, 2018, 6:40am UTC](https://discuss.elastic.co/t/lostash-taking-inputs-from-filebeat/136740 "2018-06-29T06:40:30Z")
Hi Team, Can anyone provide me the configuration file, Where the logstash will be taking input from Filebeat port and creating the same files as output in logstash . Ex : input \> filebeat text file output \> logstash t… --- ## [Invalid\_index\_name\_exception with variable name](https://discuss.elastic.co/t/invalid-index-name-exception-with-variable-name/137435)
**Author:** [@amas](https://discuss.elastic.co/u/amas)\ **Replies:** 5\ **Last updated:** [June 28, 2018, 11:58am UTC](https://discuss.elastic.co/t/invalid-index-name-exception-with-variable-name/137435 "2018-06-28T11:58:48Z")
I'm getting these warnings with the 6.3.0 stack: \[2018-06-26T12:45:22,619\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"logstash-… --- ## [Grok failure and Failed to parse query](https://discuss.elastic.co/t/grok-failure-and-failed-to-parse-query/137742)
**Author:** [@Dave061](https://discuss.elastic.co/u/Dave061)\ **Replies:** 16\ **Last updated:** [June 29, 2018, 6:20am UTC](https://discuss.elastic.co/t/grok-failure-and-failed-to-parse-query/137742 "2018-06-29T06:20:52Z")
Hello, Guys I'am trying couple of days to setup logstash 6.3.0 on Windows Server 2012 r2.I want to parse IIS logs version on IIS is (8.5.9600) to ElasticSearch.Here is my logstash config ---\> input { file { type =\> … --- ## [Logstash system config](https://discuss.elastic.co/t/logstash-system-config/136437)
**Author:** [@MarcusCaepio](https://discuss.elastic.co/u/MarcusCaepio)\ **Replies:** 3\ **Last updated:** [June 29, 2018, 6:14am UTC](https://discuss.elastic.co/t/logstash-system-config/136437 "2018-06-29T06:14:47Z")
Hi all, I want to install Elastic + Logstash on several server I'm just wondering, if there is no "Important System Config" Part for logstash like there is for elastic: https://www.elastic.co/guide/en/elasticsearch/re… --- ## [Multiple Logstash instances running](https://discuss.elastic.co/t/multiple-logstash-instances-running/137177)
**Author:** [@Preethi](https://discuss.elastic.co/u/Preethi)\ **Replies:** 7\ **Last updated:** [June 29, 2018, 6:14am UTC](https://discuss.elastic.co/t/multiple-logstash-instances-running/137177 "2018-06-29T06:14:46Z")
Hi, I am newly started using Elasticsearch in Mac OS. The version is 6.3.0. I am amble to create an index through curl commands in terminal. But wWhen I use Logstash to import data from a .cvs file, getting below error… --- ## [How to read one specific file out of multiple files from BLOB storage](https://discuss.elastic.co/t/how-to-read-one-specific-file-out-of-multiple-files-from-blob-storage/137753)
**Author:** [@Pooja](https://discuss.elastic.co/u/Pooja)\ **Replies:** 3\ **Last updated:** [June 29, 2018, 6:12am UTC](https://discuss.elastic.co/t/how-to-read-one-specific-file-out-of-multiple-files-from-blob-storage/137753 "2018-06-29T06:12:44Z")
Is there any filter or parameter or condition that can be specified in my logstash conf file where I want to read one specific file from BLOB storage container where many different files are stored on that particular BLO… --- ## [Want to send a few data fileds to Kafka](https://discuss.elastic.co/t/want-to-send-a-few-data-fileds-to-kafka/137903)
**Author:** [@seungtae](https://discuss.elastic.co/u/seungtae)\ **Replies:** 0\ **Last updated:** [June 29, 2018, 5:26am UTC](https://discuss.elastic.co/t/want-to-send-a-few-data-fileds-to-kafka/137903 "2018-06-29T05:26:35Z")
I will connect logstash and Kafka. So now I'm trying to test. It works well this time.. But, logstash sends its all of data fields. Can I choose a few data fileds what I want to send Kafka? Is there a function in Kafka… --- ## [Dead letter queue with multiple outputs](https://discuss.elastic.co/t/dead-letter-queue-with-multiple-outputs/137881)
**Author:** [@rocknroll](https://discuss.elastic.co/u/rocknroll)\ **Replies:** 0\ **Last updated:** [June 29, 2018, 12:26am UTC](https://discuss.elastic.co/t/dead-letter-queue-with-multiple-outputs/137881 "2018-06-29T00:26:42Z")
Hi there We have a pipeline with two Elasticsearch outputs, going to separate instances of Elastic - Elastic1 and Elastic2. We would like to ensure that if Elastic2 is unavailable, events will still get sent to Elasti… --- ## ["object mapping for \[monitor\] tried to parse field \[monitor\] as object, but found a concrete value"](https://discuss.elastic.co/t/object-mapping-for-monitor-tried-to-parse-field-monitor-as-object-but-found-a-concrete-value/137806)
**Author:** [@hollowimage](https://discuss.elastic.co/u/hollowimage)\ **Replies:** 1\ **Last updated:** [June 28, 2018, 11:30pm UTC](https://discuss.elastic.co/t/object-mapping-for-monitor-tried-to-parse-field-monitor-as-object-but-found-a-concrete-value/137806 "2018-06-28T23:30:06Z")
I am using heartbeat 6.2.2, logstash 6.2.3, elasticsearch 6.2.3 When sending heartbeat tcp monitor data to logstash, i get the following error message: \[2018-06-28T00:57:22,253\]\[WARN \]\[logstash.outputs.elasticsearch\] C… --- ## [Duplicate output records](https://discuss.elastic.co/t/duplicate-output-records/137870)
**Author:** [@gt45724](https://discuss.elastic.co/u/gt45724)\ **Replies:** 0\ **Last updated:** [June 28, 2018, 10:08pm UTC](https://discuss.elastic.co/t/duplicate-output-records/137870 "2018-06-28T22:08:00Z")
Relatively new to Logstash, but I'm just trying to understand the operation of the underlying framework. I am using 5.5.0. So I am parsing an XML file using, 4000 unique records. Not that it helps, this is the gist of… --- ## [Netflow disable pipeline.yml](https://discuss.elastic.co/t/netflow-disable-pipeline-yml/137852)
**Author:** [@paulo\_bruck](https://discuss.elastic.co/u/paulo_bruck)\ **Replies:** 3\ **Last updated:** [June 28, 2018, 8:37pm UTC](https://discuss.elastic.co/t/netflow-disable-pipeline-yml/137852 "2018-06-28T20:37:21Z")
Hi Using debian 9.4 logstash 6.3.0-1 kibana 6.3.0 elasticsearch 6.3.0 When I install netflow by comnand it disable all configs via pipeline.yml and all others logstashs output that is under /etc/logstash/conf.d/\*,c… --- ## [Proxy Filebeat module events to Elasticsearch ingest pipeline](https://discuss.elastic.co/t/proxy-filebeat-module-events-to-elasticsearch-ingest-pipeline/137033)
**Author:** [@tgaudin](https://discuss.elastic.co/u/tgaudin)\ **Replies:** 1\ **Last updated:** [June 28, 2018, 2:50pm UTC](https://discuss.elastic.co/t/proxy-filebeat-module-events-to-elasticsearch-ingest-pipeline/137033 "2018-06-28T14:50:10Z")
Hi, I am using Filebeat to send custom logs to a Logstash, but I would also like to use prebuilt modules for things like system or audit logs. By default, Filebeat modules are configured to use Elasticsearch ingest pip… --- ## [Split Logfiles](https://discuss.elastic.co/t/split-logfiles/137766)
**Author:** [@A.Klos](https://discuss.elastic.co/u/A.Klos)\ **Replies:** 3\ **Last updated:** [June 28, 2018, 12:29pm UTC](https://discuss.elastic.co/t/split-logfiles/137766 "2018-06-28T12:29:02Z")
Hi, we have following logformat: 2018-03-15 11:20:46|Field1|Field2|Field3|Field4|Field5|Field6 2018-03-15 11:30:46|Field1| |Field3| |Field5|Field6 How could I split this into fields? currently my grok pattern (not a… --- ## [Mapping dynamic JSON](https://discuss.elastic.co/t/mapping-dynamic-json/137244)
**Author:** [@FrankyFire](https://discuss.elastic.co/u/FrankyFire)\ **Replies:** 1\ **Last updated:** [June 28, 2018, 7:59am UTC](https://discuss.elastic.co/t/mapping-dynamic-json/137244 "2018-06-28T07:59:35Z")
Hello there, I have an Inputfile that looks as follows: "message": { "Name1": { "color": "blue", "information": { "origin": "East" } }, "Name2" {...} } Where "Name2" has… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=730) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=732)