# Logstash with elasticsearch input

**URL:** <https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783>\
**Category:** Logstash\
**Created:** [April 10, 2021, 6:48pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783 "2021-04-10T18:48:24Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kfiro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kfiro/32/46872_2.png) [@Kfiro](https://discuss.elastic.co/u/Kfiro)\
**Post date:** [April 10, 2021, 6:48pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783/1 "2021-04-10T18:48:24Z")

</div>

Hi Masters,  
I'm trying to use Logsash with "elasticsearch" input for taking specific events from winlogbeat Index and write them to another Index,  
I have a problem with the "query" on the input section,  
when I'm trying to query something except this:  
query =\> '{ "query": { "query\_string": { "query": "\*" } } }'  
im getting EROORs or Logstsash failed to start.

**Examples of failures:**  
query =\> '{ "query": { "query\_string": { "event\_id" : "3" } } }'  
query =\> '{ "query": { "query\_string": { "event.id": 3 } } }'  
query =\> '{"query": {"query\_string": {"fields": ["event.id","content"],"query": "4625 OR 4624 OR 3"}}}'  
query =\> '{ "query": { "match": { "event.code": "4625" } } }'  
query =\> '{ "query": { "match": { "statuscode": 200 } }, "sort": ["\_doc"] }'  
query =\> '{ "query": { "match": { "event.id": 4625 } } }'  
query =\> '{ "query": { "match": { "event.code": 4625 } }, "sort": ["\_doc"] }'  
query =\> '{ "query": {"query\_string": {"query": "event.code:"4625"","fields": ["event.code"]}}}'  
query =\> '{"query": {"bool":{"should":[{"term":{"host.raw":"host 1"}},{"term":{"host.raw":"host 2"}}], "must\_not":{"term":{"code":"123"}}}}}'  
query =\> '{"query": {"bool":{"should":[{"term":{"event.id":"4624"}},{"term":{"event.id":"4625"}}], "must\_not":{"term":{"event.id":"123"}}}}}'

conf file:

```
input {
      elasticsearch {
        hosts=>["elk01:9200"]
		index => "winlogbeat-2021.04.09-000453"
        #WORK#query => '{ "query": { "query_string": { "query": "*" } } }'
        query => '{ "query": { "query_string": { "event_id" : "3" } } }'
		size => 500
        scroll => "5m"
        docinfo => true
		ca_file => '/etc/logstash3/config/certs/ca.crt'
		ssl => true
        user => 'user'
        password => '123456'
      }
    }
output {
      elasticsearch {
        index => "winlogbeat-2021.04.09-000453-copy"
		hosts => ["https://elk01:9200"]
		cacert => '/etc/logstash3/config/certs/ca.crt'
        user => 'user'
        password => '123456'
      }
    }

```

any help will very appreciated

---

<div class="post-metadata">

**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\
**Post date:** [April 11, 2021, 1:51pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783/2 "2021-04-11T13:51:17Z")

</div>

Hi Kfiro,

Could you share some logs when you put in "Examples of failures" to see what does the error causing your logstash fail to start?

I tried on one of the query in demo site, it works.

```
query => '{ "query": { "match": { "event.code": "4625" } } }'

```

in demo site:

```
GET .kibana-event-log-7.10.0-000002/_search/
{
  "query": {
    "match": {
      "event.code": "4625"
    }
  }
}

```

Result

```
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : []
  }
}
```

---

<div class="post-metadata">

**Author:** ![Kfiro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kfiro/32/46872_2.png) [@Kfiro](https://discuss.elastic.co/u/Kfiro)\
**Post date:** [April 11, 2021, 6:43pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783/3 "2021-04-11T18:43:55Z")

</div>

Hi Kavierkoo,  
thanks for the reply,

i deleted all logs and start logsash again with this conf file:  
i added the logs output bellow,

```
input {
      elasticsearch {
        hosts=>["elk01:9200"]
		index => "winlogbeat-2021.04.09-000453"
		query => '{ "query": { "match": { "event.code": "4625" } } }'
		ca_file => '/etc/logstash3/config/certs/ca.crt'
		ssl => true
        user => 'elastic'
        password => '123456'
      }
    }
output {
      elasticsearch {
        index => "winlogbeat-2021.04.09-000453-copy"
		hosts => ["https://elk01:9200"]
		cacert => '/etc/logstash3/config/certs/ca.crt'
        user => 'elastic'
        password => '123456'
      }
    }

```

logstash logs output after crashing:

> **[logstash-plain.log](https://drive.google.com/file/d/1jimpyuTsr4e5wnZ_fsCgpEfTojw5RGkn/view?usp=sharing)**
>
> Google Drive file.

---

<div class="post-metadata">

**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\
**Post date:** [April 11, 2021, 7:04pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783/4 "2021-04-11T19:04:39Z")

</div>

Hi Kfir,

For logstash constantly stopping after start,  
Based on the logs, it seems that this is your only pipeline, logstash will start and stop after it finishes it's job as nothing else require it to do.

Just to cross check, could you also share your pipeline.yml

Can you do a search on index "winlogbeat-2021.04.09-000453-copy" to see if "event.code 4625” successfully indexed into this index?

If no, can you do a GET search on Devtools with the same query?this is to check if there's any logs with event.code 4625 also to check if syntax of the query is correct.

---

<div class="post-metadata">

**Author:** ![Kfiro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kfiro/32/46872_2.png) [@Kfiro](https://discuss.elastic.co/u/Kfiro)\
**Post date:** [April 12, 2021, 8:08am UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783/5 "2021-04-12T08:08:33Z")

</div>

Thanks Kavierkoo,  
i had confused and i didn't see the events,  
it works like a charm,  
do you know how can i run it continuously on the last 5 minutes every time?

---

<div class="post-metadata">

**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\
**Post date:** [April 12, 2021, 3:50pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783/6 "2021-04-12T15:50:41Z")

</div>

You can take a look at "schedule" setting for elasticsearch input plugin

> **[Elasticsearch input plugin | Logstash Reference \[7.12\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#plugins-inputs-elasticsearch-schedule)**

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 10, 2021, 3:50pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input/269783/7 "2021-05-10T15:50:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.