# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1001 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1002 --- ## [Can't parse the log](https://discuss.elastic.co/t/cant-parse-the-log/82121)
**Author:** [@amakarenkov](https://discuss.elastic.co/u/amakarenkov)\ **Replies:** 11\ **Last updated:** [April 12, 2017, 1:55pm UTC](https://discuss.elastic.co/t/cant-parse-the-log/82121 "2017-04-12T13:55:30Z")
Greetings, guys! I must say that I am a network engineer, who works with network and server environment. Sadly I am not a programmer, more to say I am really dumb at programming logic. I have a question on to how to pa… --- ## [Logstash indices mixup in Kibana](https://discuss.elastic.co/t/logstash-indices-mixup-in-kibana/81454)
**Author:** [@Vivek\_Samaga](https://discuss.elastic.co/u/Vivek_Samaga)\ **Replies:** 11\ **Last updated:** [April 12, 2017, 1:25pm UTC](https://discuss.elastic.co/t/logstash-indices-mixup-in-kibana/81454 "2017-04-12T13:25:27Z")
Am using filebeat to get logs from remote app server and Logstash jdbc plugin to get logs from DB server but at Kibana when the log is flowing the indices are mixing up i,e logs from app server is showing in DB index and… --- ## [Logstash is not loading data to elasticsearch](https://discuss.elastic.co/t/logstash-is-not-loading-data-to-elasticsearch/55772)
**Author:** [@kuntzit](https://discuss.elastic.co/u/kuntzit)\ **Replies:** 2\ **Last updated:** [April 12, 2017, 1:24pm UTC](https://discuss.elastic.co/t/logstash-is-not-loading-data-to-elasticsearch/55772 "2017-04-12T13:24:06Z")
I'm trying to upload data from redshift into elastic search. I exported some data from redshift, so now it's just a CSV file, so I thought using logstash with the CSV filter will work. It looks like it's working, the sin… --- ## [Not able to receive Logstash input fron FTP server](https://discuss.elastic.co/t/not-able-to-receive-logstash-input-fron-ftp-server/82120)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 1\ **Last updated:** [April 12, 2017, 1:21pm UTC](https://discuss.elastic.co/t/not-able-to-receive-logstash-input-fron-ftp-server/82120 "2017-04-12T13:21:06Z")
Hi, I m using Logstash 5.1.1 in Windows operating system. I am trying to receive logstash input fron FTP server with the below filter: input { exec { command =\> "curl ftp://servername/apacheLog.log" interval =\> … --- ## [Mutate is not working](https://discuss.elastic.co/t/mutate-is-not-working/82041)
**Author:** [@saad](https://discuss.elastic.co/u/saad)\ **Replies:** 2\ **Last updated:** [April 12, 2017, 1:18pm UTC](https://discuss.elastic.co/t/mutate-is-not-working/82041 "2017-04-12T13:18:40Z")
Hi, I am trying to work on mutate to convert some the source and destination IP addresses from string to ip. However, it is not working below are the results logstash.conf input { stdin { type=\>"soc" } } filte… --- ## [LS\_USER and LS\_GROUP options not working](https://discuss.elastic.co/t/ls-user-and-ls-group-options-not-working/82033)
**Author:** [@harshafrnd4u](https://discuss.elastic.co/u/harshafrnd4u)\ **Replies:** 5\ **Last updated:** [April 12, 2017, 12:48pm UTC](https://discuss.elastic.co/t/ls-user-and-ls-group-options-not-working/82033 "2017-04-12T12:48:10Z")
I configured LS\_USER="root" and LS\_GROUP="root" parameters in two places "etc/default/logstash" and "etc/logstash/startup.options". When I start with command "sudo service logstash start" it is starting with logstash use… --- ## [Eventlog input ,errors out](https://discuss.elastic.co/t/eventlog-input-errors-out/81971)
**Author:** [@debashree13](https://discuss.elastic.co/u/debashree13)\ **Replies:** 4\ **Last updated:** [April 12, 2017, 12:44pm UTC](https://discuss.elastic.co/t/eventlog-input-errors-out/81971 "2017-04-12T12:44:42Z")
Hi Team, I facing certain problem with logstash 5.3. I am trying to use eventlog as input to read eventlog and put that in elastic search but it is not able to find eventlog plug in. Plus when I tried to list down al… --- ## [How can I change the location of Logstash's logs?](https://discuss.elastic.co/t/how-can-i-change-the-location-of-logstashs-logs/81992)
**Author:** [@dolanmk](https://discuss.elastic.co/u/dolanmk)\ **Replies:** 6\ **Last updated:** [April 12, 2017, 12:30pm UTC](https://discuss.elastic.co/t/how-can-i-change-the-location-of-logstashs-logs/81992 "2017-04-12T12:30:12Z")
Hi, I am hoping to change the location of where Logstash stores the logs it creates of its own actions. As I understand it right now, this can be done with path.logs in the logstash.yml file. However, it doesn't seem a… --- ## [Define deletion frequency to delete some logstash-plain-yyyy-mm-dd.log files in logstash directly](https://discuss.elastic.co/t/define-deletion-frequency-to-delete-some-logstash-plain-yyyy-mm-dd-log-files-in-logstash-directly/81146)
**Author:** [@Diyal](https://discuss.elastic.co/u/Diyal)\ **Replies:** 0\ **Last updated:** [April 4, 2017, 1:00pm UTC](https://discuss.elastic.co/t/define-deletion-frequency-to-delete-some-logstash-plain-yyyy-mm-dd-log-files-in-logstash-directly/81146 "2017-04-04T13:00:16Z")
Hello, Is it possible (directly in logstash) to regulate deletion frequency about logstash-plain-yyyy-mm-dd.log files ? For exemple, everyday i would like to delete logstash-plain-yyyy-mm-dd.log that created more th… --- ## [Apache Error logs timestamp](https://discuss.elastic.co/t/apache-error-logs-timestamp/82010)
**Author:** [@mkorayem](https://discuss.elastic.co/u/mkorayem)\ **Replies:** 9\ **Last updated:** [April 12, 2017, 9:43am UTC](https://discuss.elastic.co/t/apache-error-logs-timestamp/82010 "2017-04-12T09:43:59Z")
Hi I tried to get the pattern for apache error timestamp but all failed The timestamp is used in elasticsearch is when it is indexed not the time of the error e.g. \[Sun Feb 05 00:32:24.992868 2017\] \[mpm\_winnt:notic… --- ## [Logstash - Comparing two fields](https://discuss.elastic.co/t/logstash-comparing-two-fields/82113)
**Author:** [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\ **Replies:** 2\ **Last updated:** [April 12, 2017, 8:24am UTC](https://discuss.elastic.co/t/logstash-comparing-two-fields/82113 "2017-04-12T08:24:53Z")
Hi All, I was trying to compare two fields of our windows events by creating a mutate filter ,but i couldn't get any results in elasticsearch, i checked the configuration ,it seems to be Ok.Could anyone help me on th… --- ## [Cannot Monitor Logstash](https://discuss.elastic.co/t/cannot-monitor-logstash/81957)
**Author:** [@arianayay](https://discuss.elastic.co/u/arianayay)\ **Replies:** 13\ **Last updated:** [April 12, 2017, 8:16am UTC](https://discuss.elastic.co/t/cannot-monitor-logstash/81957 "2017-04-12T08:16:35Z")
Hi, Logstash is not showing in the Monitor Tab of Kibana. Only Elasicsearch and Kibana is showing. I have configured mypipeline.conf input { stdin { }} output { elasticsearch { hosts =\> \["CTLSQL12WPPOC:9200"\] } … --- ## [Issues when running Logstash in Docker](https://discuss.elastic.co/t/issues-when-running-logstash-in-docker/81917)
**Author:** [@John\_06](https://discuss.elastic.co/u/John_06)\ **Replies:** 7\ **Last updated:** [April 12, 2017, 7:11am UTC](https://discuss.elastic.co/t/issues-when-running-logstash-in-docker/81917 "2017-04-12T07:11:16Z")
I am trying to run ELK in Docker and I did the following pretty simple and straightforward steps: Created directory /opt/elk/logstash and moved there sample log file and logstash.conf file: input { file { path … --- ## [Only can bind logstash http\_address to 127.0.0.1](https://discuss.elastic.co/t/only-can-bind-logstash-http-address-to-127-0-0-1/81413)
**Author:** [@batrako](https://discuss.elastic.co/u/batrako)\ **Replies:** 4\ **Last updated:** [April 12, 2017, 5:41am UTC](https://discuss.elastic.co/t/only-can-bind-logstash-http-address-to-127-0-0-1/81413 "2017-04-12T05:41:24Z")
I'm trying access to monitornig API from outside of logstash server but always bind to 127.0.0.1:9600 . In my logstash.yml config i set http.host with dns, ip,... but always bind to 127.0.0.1:9600. one of my example … --- ## [Logging at past date in logstash and elasticsearch](https://discuss.elastic.co/t/logging-at-past-date-in-logstash-and-elasticsearch/81959)
**Author:** [@Rupesh\_Ranjan](https://discuss.elastic.co/u/Rupesh_Ranjan)\ **Replies:** 3\ **Last updated:** [April 12, 2017, 4:49am UTC](https://discuss.elastic.co/t/logging-at-past-date-in-logstash-and-elasticsearch/81959 "2017-04-12T04:49:17Z")
I have some logs logged at date say Apr 9, 2017. I am starting the filebeat and ELK on say 11th Apr. Is there any way, i can log old logs at appropriate timestamp and index (i am creadting day-wise index, say test-2017… --- ## [Netflow data not being seen in elk after initial ingest](https://discuss.elastic.co/t/netflow-data-not-being-seen-in-elk-after-initial-ingest/82013)
**Author:** [@shazbot](https://discuss.elastic.co/u/shazbot)\ **Replies:** 1\ **Last updated:** [April 12, 2017, 1:19am UTC](https://discuss.elastic.co/t/netflow-data-not-being-seen-in-elk-after-initial-ingest/82013 "2017-04-12T01:19:49Z")
I configured logstash to parse netflow, opened port 9995 and restarted the service. I can see the data coming in via TCPDUMP however I do not see it in ES after the first ingest which I find kind of head scratching. … --- ## [Logstash Aggregation concepts](https://discuss.elastic.co/t/logstash-aggregation-concepts/81963)
**Author:** [@Jonesthomas](https://discuss.elastic.co/u/Jonesthomas)\ **Replies:** 1\ **Last updated:** [April 12, 2017, 1:09am UTC](https://discuss.elastic.co/t/logstash-aggregation-concepts/81963 "2017-04-12T01:09:50Z")
hi,im learning little deep in logstash filter concept. i have a sample log message,i want to aggregate them,group the particular event,and want to custom label the field.. This is my sample log message,Someone Guide me. … --- ## [Elasticsearch logs to Elasticsearch Index](https://discuss.elastic.co/t/elasticsearch-logs-to-elasticsearch-index/82067)
**Author:** [@eperry](https://discuss.elastic.co/u/eperry)\ **Replies:** 0\ **Last updated:** [April 12, 2017, 12:53am UTC](https://discuss.elastic.co/t/elasticsearch-logs-to-elasticsearch-index/82067 "2017-04-12T00:53:56Z")
Hey guys, I want to start pulling my Elasticsearch logs with kibana and put them in Elasticsearch. elasticsearch log -\> logstash -\> kafka -\> logstash -\> elasticsearch (and Repeat) :open\_mouth: Ok this is a rec… --- ## [Beats input debugging logs not working](https://discuss.elastic.co/t/beats-input-debugging-logs-not-working/82064)
**Author:** [@jbeck](https://discuss.elastic.co/u/jbeck)\ **Replies:** 0\ **Last updated:** [April 11, 2017, 11:24pm UTC](https://discuss.elastic.co/t/beats-input-debugging-logs-not-working/82064 "2017-04-11T23:24:17Z")
I'm trying to see debug logs for the beats input to logstash. I have run: curl -XPUT localhost:9600/\_node/logging -d '{ "logger.logstash.inputs.beats": "TRACE" }' which responded with {"host":"ip-10-239-197-23",… --- ## [\ When trying send data to Redis from Logstash](https://discuss.elastic.co/t/redis-timeouterror-connection-timed-out-when-trying-send-data-to-redis-from-logstash/81995)
**Author:** [@ash007](https://discuss.elastic.co/u/ash007)\ **Replies:** 1\ **Last updated:** [April 11, 2017, 9:44pm UTC](https://discuss.elastic.co/t/redis-timeouterror-connection-timed-out-when-trying-send-data-to-redis-from-logstash/81995 "2017-04-11T21:44:30Z")
Hello, We are facing the following error when trying to send data to redis. \[2017-04-11T13:36:02,010\]\[WARN \]\[logstash.outputs.redis \] Failed to flush outgoing items {:outgoing\_count=\>50, :exception=\>"Redis::TimeoutE… --- ## [Index Creation Issues](https://discuss.elastic.co/t/index-creation-issues/82015)
**Author:** [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Replies:** 11\ **Last updated:** [April 11, 2017, 9:31pm UTC](https://discuss.elastic.co/t/index-creation-issues/82015 "2017-04-11T21:31:24Z")
While looking through my ES indexes, I see these: DATANODE-03:~$ health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open .kibana … --- ## [Need to count the number of occurrence in IIS logs using aggregate function](https://discuss.elastic.co/t/need-to-count-the-number-of-occurrence-in-iis-logs-using-aggregate-function/81580)
**Author:** [@rvnrajesh](https://discuss.elastic.co/u/rvnrajesh)\ **Replies:** 1\ **Last updated:** [April 11, 2017, 9:09pm UTC](https://discuss.elastic.co/t/need-to-count-the-number-of-occurrence-in-iis-logs-using-aggregate-function/81580 "2017-04-11T21:09:31Z")
Need to count the number of occurrence in IIS logs, used logstash filter aggregate function and tagged it. There is two part on actual raw data should forward to elasticsearch and aggregate should data should be in in… --- ## [Logstash conf exception handling](https://discuss.elastic.co/t/logstash-conf-exception-handling/82045)
**Author:** [@piyush](https://discuss.elastic.co/u/piyush)\ **Replies:** 0\ **Last updated:** [April 11, 2017, 8:07pm UTC](https://discuss.elastic.co/t/logstash-conf-exception-handling/82045 "2017-04-11T20:07:56Z")
Hi Team, I am trying to read messages from 2 different rabbitmq servers and PFB my configuration sample. But, if any error on "host-a" logstash is not reading messages from "host-b" either. Can we do exception handlin… --- ## [Logstash listening on tcp6 only](https://discuss.elastic.co/t/logstash-listening-on-tcp6-only/82026)
**Author:** [@Drew\_Rogers](https://discuss.elastic.co/u/Drew_Rogers)\ **Replies:** 4\ **Last updated:** [April 11, 2017, 7:39pm UTC](https://discuss.elastic.co/t/logstash-listening-on-tcp6-only/82026 "2017-04-11T19:39:27Z")
I upgraded my stack to 5.3.0. I noticed that for all my inputs, logstash is only listening on tcp6 and not tcp4. For instance my input for beats is: and when logstash starts here are my listening ports: --- ## [Increase Size Limit of logs](https://discuss.elastic.co/t/increase-size-limit-of-logs/80496)
**Author:** [@AlecBruns](https://discuss.elastic.co/u/AlecBruns)\ **Replies:** 5\ **Last updated:** [April 11, 2017, 7:17pm UTC](https://discuss.elastic.co/t/increase-size-limit-of-logs/80496 "2017-04-11T19:17:08Z")
Hey there, Is it possible to increase the size limit of incoming logs to logstash? I am using filebeat but logstash will break up large logs which then makes the grok filter unable to parse it. I am trying to log some … --- ## [Parsing Log With grok Filter](https://discuss.elastic.co/t/parsing-log-with-grok-filter/81162)
**Author:** [@Sam67000](https://discuss.elastic.co/u/Sam67000)\ **Replies:** 10\ **Last updated:** [April 11, 2017, 4:12pm UTC](https://discuss.elastic.co/t/parsing-log-with-grok-filter/81162 "2017-04-11T16:12:24Z")
Hello EveryOne, My name is samir and i am a trainee in a company in france. I have a project to realize: Establish the ELK suite on logs of different applications of the company. I need your help because at the moment… --- ## [Logstash with S3](https://discuss.elastic.co/t/logstash-with-s3/82003)
**Author:** [@potro\_italiano](https://discuss.elastic.co/u/potro_italiano)\ **Replies:** 1\ **Last updated:** [April 11, 2017, 3:47pm UTC](https://discuss.elastic.co/t/logstash-with-s3/82003 "2017-04-11T15:47:36Z")
Hi, I have configured filebeat + logstash + S3 logstash plugin. Filebeat sends the configured log files to the logstash. Logstash receives the files and, through the output, leaves them in the defined path. The problem… --- ## [Parsing apache logs - COMBINEDAPACHELOG & COMMONAPACHELOG not working](https://discuss.elastic.co/t/parsing-apache-logs-combinedapachelog-commonapachelog-not-working/82007)
**Author:** [@Maria\_Delarosa](https://discuss.elastic.co/u/Maria_Delarosa)\ **Replies:** 0\ **Last updated:** [April 11, 2017, 3:16pm UTC](https://discuss.elastic.co/t/parsing-apache-logs-combinedapachelog-commonapachelog-not-working/82007 "2017-04-11T15:16:38Z")
I am currently trying to parse Apache access and error logs. I have used both COMBINEDAPACHELOG and COMMONAPACHELOG but no luck. For the access logs I am trying to break it down to show timestamp, source, target, userage… --- ## [Dns timeout error](https://discuss.elastic.co/t/dns-timeout-error/81293)
**Author:** [@vaishnavi](https://discuss.elastic.co/u/vaishnavi)\ **Replies:** 1\ **Last updated:** [April 11, 2017, 2:20pm UTC](https://discuss.elastic.co/t/dns-timeout-error/81293 "2017-04-11T14:20:47Z")
Hi, I am using logstash to process large files which involves many IPs and I need to convert IP to hostname in logstash using dns filter without any time delay I am already using DNS filter which convert IP to hostname… --- ## [Dynamic field names in logstash](https://discuss.elastic.co/t/dynamic-field-names-in-logstash/81981)
**Author:** [@Aditya\_Srivastava](https://discuss.elastic.co/u/Aditya_Srivastava)\ **Replies:** 5\ **Last updated:** [April 11, 2017, 3:04pm UTC](https://discuss.elastic.co/t/dynamic-field-names-in-logstash/81981 "2017-04-11T15:04:12Z")
I have a string which is dynamic .It represents resp time and ip address 60.006,0.001#10.10.24.14:8082,10.10.22.11:8082 The no of machine can grow and so will the subsequent resp time and ip address. What I am tryi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1000) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1002)