# Unable to connect Logstash with Elasticsearch

**URL:** <https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355>\
**Category:** Logstash\
**Created:** [April 13, 2017, 8:41pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355 "2017-04-13T20:41:36Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 13, 2017, 8:41pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/1 "2017-04-13T20:41:36Z")

</div>

Hello,

I have installed Logstash, Elasticsearch and Kibana on my CentOS 7 machine. I was able to do the installations successfully. But when I try to access the apache access logs, it doesn't get the output. here is what I have done.

I've created a file sudo vi /etc/logstash/conf.d/01-webserver.conf

Added the following code in the file to access the logs:

input  
{  
file  
{  
path =\> "/var/log/httpd/access\_log"  
start\_position =\> "beginning"  
}  
}  
filter  
{  
if [type] == "apache-access"  
{  
grok  
{  
match =\> { "message" =\> "%{COMBINEDAPACHELOG}" }  
}  
}  
date  
{  
match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]  
}  
}  
output  
{  
elasticsearch  
{  
hosts =\> ["127.0.0.1:9200"]  
}  
stdout { codec =\> rubydebug }  
}

Then, I run this command to list the logstash indexes:

curl -XGET [http://127.0.0.1:9200/\_cat/indices?v](http://127.0.0.1:9200/_cat/indices?v)

It gives me this:

health status index pri rep docs.count docs.deleted store.size pri.store.size  
yellow open .kibana 1 1 1 0 3.1kb 3.1kb

It doesn't show any log related logstash indexes here.  
can you please help!!

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 13, 2017, 9:09pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/2 "2017-04-13T21:09:30Z")

</div>

This is hard to read. A little indentation goes a long way to improve readability.

You can also embed your code between two lines with triple back-ticks, like this:

````
```
YOUR CODE PASTED HERE
```

````

When I try to reformat your config, I get this:

```auto
input {
  file {
    path => "/var/log/httpd/access_log"
    start_position => "beginning"
  }
}

filter {
  if [type] == "apache-access" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
  }
  date {
    match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
  }
}

output {
  elasticsearch {
    hosts => ["127.0.0.1:9200"]
  }
  stdout { codec => rubydebug }
}

```

The first thing I notice is that you're testing for `[type]`, but you haven't assigned a `type` anywhere.

Perhaps you could add `type => "apache-access"` to your `file` plugin block:

```auto
  file {
    path => "/var/log/httpd/access_log"
    start_position => "beginning"
    type => "apache-access"
  }

```

Also, you will need to remove the sincedb for the file you're tailing. Just because it says, `start_position => "beginning"` doesn't mean it will repeatedly reinvest the file from the start. It does so the first time, but then it remembers where it left off, and resumes from there. If you're running a 5.x version of Logstash, and installed via RPM or DEB, then look in `/var/lib/logstash/input/file` for sincedb files, and delete them. Otherwise, they will be in `$LS_HOME/data/input/file`, if memory serves.

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 13, 2017, 9:47pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/3 "2017-04-13T21:47:37Z")

</div>

Hey Aaron,

Thanks for your response and sorry for the trouble in reading the code.  
I did add type =\> "apache-access", I'm not able to find the sincedb files to delete.  
After adding the type I've run the curl command and it still gives me the same output.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 14, 2017, 12:04pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/4 "2017-04-14T12:04:45Z")

</div>

What version of Logstash are you using? We need to find those sincedb files, or you will not be able to re-read the apache log files.

Also, how are you starting Logstash? You have `stdout` output plugin, but this is not useful unless you're starting Logstash manually at the command-line.

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 14, 2017, 4:06pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/5 "2017-04-14T16:06:27Z")

</div>

My Logstash version is 2.4.1. I'm starting Logstash manually from the terminal.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 14, 2017, 4:18pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/6 "2017-04-14T16:18:52Z")

</div>

If that's the case, then your sincedb entries are likely to be in `$HOME/.sincedb/` (for whomever the user is that is launching Logstash).

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 17, 2017, 5:56pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/7 "2017-04-17T17:56:19Z")

</div>

Hello Sir,

I am so sorry, but I don't find any sincedb files.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 17, 2017, 7:07pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/8 "2017-04-17T19:07:58Z")

</div>

You have the option to [specify a sincedb path](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb_path):

```auto
input {
  file {
    path => "/var/log/httpd/access_log"
    start_position => "beginning"
    sincedb_path => "/home/user/mysincedb"
  }
}

```

This provides two benefits.

1. This is a new sincedb file, so it should read from the `"beginning"` again.
2. You will know which file to erase between tests.

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 20, 2017, 1:55pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/9 "2017-04-20T13:55:17Z")

</div>

I did it but still unable to locate the sincedb files. So I've re-installed logstash, elasticsearch and kibana. It's the same issue again, unable to see the indexes. When I go check var/log/logstash, I see these errors:

{:timestamp=\>"2017-04-20T08:54:35.379000-0400", :message=\>"fetched an invalid config", :config=\>"input\n{\nfile\n{\npath =\> "/var/log/httpd/access\_log"\nstart\_position =\> "beginning"\n}\n}\nfilter \n{\nif [type] == "apache-access"\n{\ngrok\n{\nmatch =\> { "message" =\> "%{COMBINEDAPACHELOG}" }\n}\n}\ndate\n{\nmatch =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]\n}\n}\noutput\n{\nelasticsearch\n{\nhosts =\> ["localhost:9200"]\n}\nstdout { codec =\> rubydebug }\n}\n\n\ninput {\n stdin {}\n}\noutput {\n stdout {}\n}\n\ninput {\nfile {\npath =\> ["/var/log/\*.log", "/var/log/messages", "/var/log/syslog"]\ntype =\> "syslog"\n}\n}\n\noutput {\nelasticsearch { host =\> 127.0.0.1 }\nstdout { codec =\> rubydebug }\n}\n\n", :reason=\>"Expected one of #, } at line 48, column 30 (byte 526) after output {\nelasticsearch { host =\> 127.0", :level=\>:error}

{:timestamp=\>"2017-04-20T09:14:16.449000-0400", :message=\>"fetched an invalid config", :config=\>"input\n{\nfile\n{\npath =\> "/var/log/httpd/access\_log"\nstart\_position =\> "beginning"\n}\n}\nfilter \n{\nif [type] == "apache-access"\n{\ngrok\n{\nmatch =\> { "message" =\> "%{COMBINEDAPACHELOG}" }\n}\n}\ndate\n{\nmatch =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]\n}\n}\noutput\n{\nelasticsearch\n{\nhosts =\> ["localhost:9200"]\n}\nstdout { codec =\> rubydebug }\n}\n\n\ninput {\n stdin {}\n}\noutput {\n stdout {}\n}\n\ninput {\nfile {\npath =\> ["/var/log/\*.log", "/var/log/messages", "/var/log/syslog"]\ntype =\> "syslog"\n}\n}\n\noutput {\nelasticsearch { host =\> 127.0.0.1 }\nstdout { codec =\> rubydebug }\n}\n\n", :reason=\>"Expected one of #, } at line 48, column 30 (byte 526) after output {\nelasticsearch { host =\> 127.0", :level=\>:error}

{:timestamp=\>"2017-04-20T09:43:13.407000-0400", :message=\>"fetched an invalid config", :config=\>"input\n{\nfile\n{\npath =\> "/var/log/httpd/access\_log"\ntype =\> "apache-access"\nstart\_position =\> "beginning"\n}\n}\nfilter \n{\nif [type] == "apache-access"\n{\ngrok\n{\nmatch =\> { "message" =\> "%{COMBINEDAPACHELOG}" }\n}\n}\ndate\n{\nmatch =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]\n}\n}\noutput\n{\nelasticsearch\n{\nhosts =\> ["localhost:9200"]\n}\nstdout { codec =\> rubydebug }\n}\n\n\ninput {\n stdin {}\n}\noutput {\n stdout {}\n}\n\ninput {\nfile {\npath =\> ["/var/log/\*.log", "/var/log/messages", "/var/log/syslog"]\ntype =\> "syslog"\n}\n}\n\noutput {\nelasticsearch { host =\> 127.0.0.1 }\nstdout { codec =\> rubydebug }\n}\n\n", :reason=\>"Expected one of #, } at line 49, column 30 (byte 550) after output {\nelasticsearch { host =\> 127.0", :level=\>:error}

Can you please look into it.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 20, 2017, 2:16pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/10 "2017-04-20T14:16:31Z")

</div>

I can't help you if you won't do what I suggest.

The error indicates that your new configuration file is missing something, or is otherwise misconfigured. But besides this, you have ignored my suggestion to manually specify the `sincedb_path`, as is clear from the configuration I found:

```auto
:message=>"fetched an invalid config", :config=>"input\n{\nfile\n{\npath => \"/var/log/httpd/access_log\"\nstart_position => \"beginning\"\n}\n}\n

```

Which breaks down to:

```auto
input {
  file {
    path => "/var/log/httpd/access_log"
    start_position => "beginning"
  }
}

```

It also seems (though it's hard to tell for sure) you are still not indenting your configuration, which makes it extremely hard to read, especially when you are looking for a misconfiguration.

I can't help you if you don't try to do the things I recommend.

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 20, 2017, 2:32pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/11 "2017-04-20T14:32:22Z")

</div>

Hello sir, I already did manually specify the sincedb\_path and as it didn't work I had to start from scratch. I am here to seek help. And it is an error file which I have sent you, not my config file. My config file is indented. Here is my config file:

input {  
file {  
path =\> "/var/log/httpd/access\_log"  
start\_position =\> "beginning"  
sincedb\_path =\> "/home/likhita/mysincedb"  
}  
}

filter {  
if [type] == "apache-access"  
{  
grok {  
match =\> { "message" =\> "%{COMBINEDAPACHELOG}" }  
}  
}  
date {  
match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]  
}  
}

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
}  
stdout { codec =\> rubydebug }  
}

Sorry If I did something wrong!!

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 20, 2017, 2:33pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/12 "2017-04-20T14:33:03Z")

</div>

When I paste it here my indentation goes off!!!

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 20, 2017, 2:46pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/13 "2017-04-20T14:46:02Z")

</div>

![](https://us1.discourse-cdn.com/elastic/original/3X/8/7/87d66084f9e2f43d9aafb6351e66eba9475cf963.png)

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 20, 2017, 2:46pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/14 "2017-04-20T14:46:25Z")

</div>

Here it is!!

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 21, 2017, 9:40pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/15 "2017-04-21T21:40:34Z")

</div>

> [@theuntergeek](#):
>
> You can also embed your code between two lines with triple back-ticks, like this:

````
```
YOUR CODE PASTED HERE
```

````

If you paste your code between triple back-ticks, as I mentioned early in this thread, it will not lose formatting.

I need you to do this, because I cannot copy and paste a screen-shot.

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 24, 2017, 6:52pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/16 "2017-04-24T18:52:06Z")

</div>

```auto
input {
  file {
    path => "/var/log/httpd/access_log"
    start_position => "beginning"
    sincedb_path => "/home/likhita/mysincedb"
  }
}

filter {
if [type] == "apache-access"
{
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
  }
  date {
    match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
  stdout { codec => rubydebug }
}

```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 24, 2017, 7:38pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/17 "2017-04-24T19:38:51Z")

</div>

Thank you. That's much more readable, and I was able to cut/paste it and test it on my laptop. I changed these settings to fit my paths:

```auto
    path => "/Users/buh/test_access_log"
    sincedb_path => "/Users/buh/mysincedb"

```

I spun up a local elasticsearch 5.3.0 instance, so it would match what is in the config. I put a single line into the `/Users/buh/test_access_log` file:

```auto
177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] "GET / HTTP/1.0" 200 612 "-" "Wget(linux)"

```

Everything else is exactly as you have it, stored in a file I named `test.conf`. I ran:

```auto
bin/logstash -f test.conf --debug

```

and this is part of what I saw in the output:

```auto
[2017-04-24T13:29:43,513][INFO][logstash.pipeline] Pipeline main started
[2017-04-24T13:29:43,516][DEBUG][logstash.inputs.file] _globbed_files: /Users/buh/test_access_log: glob is: ["/Users/buh/test_access_log"]
[2017-04-24T13:29:43,517][DEBUG][logstash.inputs.file] _discover_file: /Users/buh/test_access_log: new: /Users/buh/test_access_log (exclude is [])
[2017-04-24T13:29:43,518][DEBUG][logstash.inputs.file] _open_file: /Users/buh/test_access_log: opening
[2017-04-24T13:29:43,518][DEBUG][logstash.inputs.file] /Users/buh/test_access_log: initial create, no sincedb, seeking to beginning of file
[2017-04-24T13:29:43,518][DEBUG][logstash.inputs.file] Received line {:path=>"/Users/buh/test_access_log", :text=>"177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\""}
[2017-04-24T13:29:43,519][DEBUG][logstash.agent] Starting puma
[2017-04-24T13:29:43,521][DEBUG][logstash.agent] Trying to start WebServer {:port=>9600}
[2017-04-24T13:29:43,522][DEBUG][logstash.api.service] [api-service] start
[2017-04-24T13:29:43,535][DEBUG][logstash.inputs.file] writing sincedb (delta since last write = 1493062183)
[2017-04-24T13:29:43,542][DEBUG][logstash.pipeline] filter received {"event"=>{"path"=>"/Users/buh/test_access_log", "@timestamp"=>2017-04-24T19:29:43.533Z, "@version"=>"1", "host"=>"localhost.local", "message"=>"177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\""}}
[2017-04-24T13:29:43,545][DEBUG][logstash.pipeline] output received {"event"=>{"path"=>"/Users/buh/test_access_log", "@timestamp"=>2017-04-24T19:29:43.533Z, "@version"=>"1", "host"=>"localhost.local", "message"=>"177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\""}}
[2017-04-24T13:29:43,556][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
{
          "path" => "/Users/buh/test_access_log",
    "@timestamp" => 2017-04-24T19:29:43.533Z,
      "@version" => "1",
          "host" => "localhost.local",
       "message" => "177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\""
}
[2017-04-24T13:29:48,519][DEBUG][logstash.pipeline] Pushing flush onto pipeline
^C[2017-04-24T13:29:53,288][WARN][logstash.runner] SIGINT received. Shutting down the agent.

```

As you can see, I hit `^C` there at the end to kill Logstash. My Elasticsearch logged this:

```auto
[2017-04-24T13:29:43,605][INFO][o.e.c.m.MetaDataCreateIndexService] [yZW_CPx] [logstash-2017.04.24] creating index, cause [auto(bulk api)], templates [logstash], shards [5]/[1], mappings [_default_]
[2017-04-24T13:29:43,793][INFO][o.e.c.m.MetaDataMappingService] [yZW_CPx] [logstash-2017.04.24/T8t6Pts3QSaVDZvoZw_JKg] create_mapping [logs]

```

When I queried Elasticsearch, I found the line I expected:

```auto
$ curl -XGET http://localhost:9200/logstash-2017.04.24/_search?pretty -d '
{
  "query": {
    "match_all": {}
  }
}
'
{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "logstash-2017.04.24",
        "_type" : "logs",
        "_id" : "AVuhcOtGokrJPubvTV9I",
        "_score" : 1.0,
        "_source" : {
          "path" : "/Users/buh/test_access_log",
          "@timestamp" : "2017-04-24T19:29:43.533Z",
          "@version" : "1",
          "host" : "localhost.local",
          "message" : "177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\""
        }
      }
    ]
  }
}

```

As you can also see in the above Logstash debug output, it updated the sincedb file:

```auto
$ cat ~/mysincedb
58694687 1 4 91

```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [April 24, 2017, 7:49pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/18 "2017-04-24T19:49:44Z")

</div>

Since it's clear you wanted your grok rule to work, I took the liberty of adding `type => "apache-access"` to your config, so the conditional `if` statement would have something to match against:

```auto
input {
  file {
    path => "/Users/buh/test_access_log"
    start_position => "beginning"
    sincedb_path => "/Users/buh/mysincedb"
    type => "apache-access"
  }
}

```

Now, I delete the `sincedb` file:

```auto
rm /Users/buh/mysincedb

```

(I also deleted the index in Elasticsearch so it would be a fresh start)

Now, when I re-run `bin/logstash -f test.conf --debug`, I see what you were probably expecting:

```auto
[2017-04-24T13:41:50,598][INFO][logstash.pipeline] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2017-04-24T13:41:50,703][INFO][logstash.pipeline] Pipeline main started
[2017-04-24T13:41:50,705][DEBUG][logstash.inputs.file] _globbed_files: /Users/buh/test_access_log: glob is: ["/Users/buh/test_access_log"]
[2017-04-24T13:41:50,706][DEBUG][logstash.inputs.file] _discover_file: /Users/buh/test_access_log: new: /Users/buh/test_access_log (exclude is [])
[2017-04-24T13:41:50,707][DEBUG][logstash.inputs.file] _open_file: /Users/buh/test_access_log: opening
[2017-04-24T13:41:50,707][DEBUG][logstash.inputs.file] /Users/buh/test_access_log: initial create, no sincedb, seeking to beginning of file
[2017-04-24T13:41:50,707][DEBUG][logstash.inputs.file] Received line {:path=>"/Users/buh/test_access_log", :text=>"177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\""}
[2017-04-24T13:41:50,714][DEBUG][logstash.agent] Starting puma
[2017-04-24T13:41:50,715][DEBUG][logstash.agent] Trying to start WebServer {:port=>9600}
[2017-04-24T13:41:50,716][DEBUG][logstash.api.service] [api-service] start
[2017-04-24T13:41:50,726][DEBUG][logstash.inputs.file] writing sincedb (delta since last write = 1493062910)
[2017-04-24T13:41:50,732][DEBUG][logstash.pipeline] filter received {"event"=>{"path"=>"/Users/buh/test_access_log", "@timestamp"=>2017-04-24T19:41:50.724Z, "@version"=>"1", "host"=>"localhost.local", "message"=>"177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\"", "type"=>"apache-access"}}
[2017-04-24T13:41:50,733][DEBUG][logstash.filters.grok] Running grok filter {:event=>2017-04-24T19:41:50.724Z localhost.local 177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] "GET / HTTP/1.0" 200 612 "-" "Wget(linux)"}
[2017-04-24T13:41:50,739][DEBUG][logstash.filters.grok] Event now: {:event=>2017-04-24T19:41:50.724Z localhost.local 177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] "GET / HTTP/1.0" 200 612 "-" "Wget(linux)"}
[2017-04-24T13:41:50,746][DEBUG][logstash.pipeline] output received {"event"=>{"request"=>"/", "agent"=>"\"Wget(linux)\"", "auth"=>"-", "ident"=>"-", "verb"=>"GET", "message"=>"177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\"", "type"=>"apache-access", "path"=>"/Users/buh/test_access_log", "referrer"=>"\"-\"", "@timestamp"=>2017-04-24T13:37:50.000Z, "response"=>"200", "bytes"=>"612", "clientip"=>"177.140.155.68", "@version"=>"1", "host"=>"localhost.local", "httpversion"=>"1.0", "timestamp"=>"24/Apr/2017:13:37:50 +0000"}}
[2017-04-24T13:41:50,747][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
{
        "request" => "/",
          "agent" => "\"Wget(linux)\"",
           "auth" => "-",
          "ident" => "-",
           "verb" => "GET",
        "message" => "177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\"",
           "type" => "apache-access",
           "path" => "/Users/buh/test_access_log",
       "referrer" => "\"-\"",
     "@timestamp" => 2017-04-24T13:37:50.000Z,
       "response" => "200",
          "bytes" => "612",
       "clientip" => "177.140.155.68",
       "@version" => "1",
           "host" => "localhost.local",
    "httpversion" => "1.0",
      "timestamp" => "24/Apr/2017:13:37:50 +0000"
}
^C[2017-04-24T13:41:53,503][WARN][logstash.runner] SIGINT received. Shutting down the agent.

```

And again, when queried from Elasticsearch:

```auto
$ curl -XGET http://localhost:9200/logstash-2017.04.24/_search?pretty -d '
{
  "query": {
    "match_all": {}
  }
}
'
{
  "took" : 29,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "logstash-2017.04.24",
        "_type" : "apache-access",
        "_id" : "AVuhfAPwRx488yJUxuVA",
        "_score" : 1.0,
        "_source" : {
          "request" : "/",
          "agent" : "\"Wget(linux)\"",
          "auth" : "-",
          "ident" : "-",
          "verb" : "GET",
          "message" : "177.140.155.68 - - [24/Apr/2017:13:37:50 +0000] \"GET / HTTP/1.0\" 200 612 \"-\" \"Wget(linux)\"",
          "type" : "apache-access",
          "path" : "/Users/buh/test_access_log",
          "referrer" : "\"-\"",
          "@timestamp" : "2017-04-24T13:37:50.000Z",
          "response" : "200",
          "bytes" : "612",
          "clientip" : "177.140.155.68",
          "@version" : "1",
          "host" : "localhost.local",
          "httpversion" : "1.0",
          "timestamp" : "24/Apr/2017:13:37:50 +0000"
        }
      }
    ]
  }
}

```

Everything works, and `/Users/buh/mysincedb` is again updated, per the log file:

```auto
$ cat /Users/buh/mysincedb
58694687 1 4 91

```

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 24, 2017, 8:01pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/19 "2017-04-24T20:01:14Z")

</div>

Thank you so much for your help!!! I will follow the same steps and see if everything works fine.

---

<div class="post-metadata">

**Author:** ![likhita](https://avatars.discourse-cdn.com/v4/letter/l/4da419/32.png) [@likhita](https://discuss.elastic.co/u/likhita)\
**Post date:** [April 25, 2017, 8:02pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355/20 "2017-04-25T20:02:25Z")

</div>

I followed the procedure that you have suggested and well it turned out to be working fine now. I am able to see the logstash indices when I run the curl command. Thank you so much for your help!!! Appreciate it.

[Next page](https://discuss.elastic.co/t/unable-to-connect-logstash-with-elasticsearch/82355.md?page=2)