# Error expecting # line

**URL:** <https://discuss.elastic.co/t/error-expecting-line/76708>\
**Category:** Logstash\
**Created:** [February 28, 2017, 12:09am UTC](https://discuss.elastic.co/t/error-expecting-line/76708 "2017-02-28T00:09:02Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![zbay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zbay/32/16735_2.png) [@zbay](https://discuss.elastic.co/u/zbay)\
**Post date:** [February 28, 2017, 12:09am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/1 "2017-02-28T00:09:02Z")

</div>

Good evening everyone,

I've been trying to track down some errors and get my first job running and collecting data from twitter, I found some examples on github and slightly modified them to fit my needs. At this point in time I don't really understand what the error is telling me although I've read up on people having similar issues I have not been able to figure this one out.

.conf file

```
input {
  twitter {
    consumer_key => "XXXXXX"
    consumer_secret => "XXXXXX"
    oauth_token => "XXXXXX"
    oauth_token_secret => "XXXXXX"
    keywords => ["thor", "spiderman", "wolverine", "ironman", "hulk"]
    full_tweet => true
  }
}

filter { }

output {
  stdout {
    codec => dots
  }
  elasticsearch {
      user => "XXXXXXX"
      password => "XXXXXX"
      hosts => "localhost:9200"
      index => "twitter_elastic_example"
      document_type => "tweets"
      template => "./twitter_template.json"
      template_name => "twitter_elastic_example"
      template_overwrite => true
  }
}

```

here is the .json

```
{
  "template": "twitter_elastic_example",
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  },
  "mappings": {
    "_default_": {
      "_all": {
        "enabled": true
      },
      "properties": {
        "@timestamp": {
          "type": "date",
          "format": "dateOptionalTime"
        },
        "text": {
          "type": "text"
        },
        "user": {
          "type": "object",
          "properties": {
            "description": {
              "type": "text"
            }
          }
        },
        "coordinates": {
          "type": "object",
          "properties": {
            "coordinates": {
              "type": "geo_point"
            }
          }
        },
        "entities": {
          "type": "object",
          "properties": {
            "hashtags": {
              "type": "object",
              "properties": {
                "text": {
                  "type": "text",
                  "fielddata": true
                }
              }
            }
          }
        },
        "retweeted_status": {
          "type": "object",
          "properties": {
            "text": {
              "type": "text"
            }
          }
        }
      },
      "dynamic_templates": [
        {
          "string_template": {
            "match": "*",
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword"
            }
          }
        }
      ]
    }
  }
}

```

So from what I've gathered is there is a problem during the concatenation between the two files, I've tried to join them but was still unable to find the error it was indicating was taking place. So currently my machine consists for 3 nodes a master, minion, and client node which should be set up as a search load balance. They are currently running on aws, and while I have a yellow status for the cluster it seem that I can monitor and communicate through out the different machines. I've also tried to start the job on the Master Node and the Ingest node, both with no luck on getting the task started.

First let me thank you in advance, This has been a really cool project although I will admit there has been a bit of a learning curve to overcome.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 28, 2017, 12:18am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/2 "2017-02-28T00:18:08Z")

</div>

Can you post the actual error?

---

<div class="post-metadata">

**Author:** ![zbay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zbay/32/16735_2.png) [@zbay](https://discuss.elastic.co/u/zbay)\
**Post date:** [February 28, 2017, 12:28am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/3 "2017-02-28T00:28:17Z")

</div>

Sorry I probably should have done that the first post, but here it is

`[2017-02-28T00:24:44,126][ERROR][logstash.agent] fetched an invalid config {:config=>"input {\n twitter {\n consumer_key => \"xxxxxxx\"\n consumer_secret => \"xxxxxxxxx\"\n oauth_token => \"xxxxxxxxxxx\"\n oauth_token_secret => \"xxxxxxxxxxx\"\n keywords => [\"thor\", \"spiderman\", \"wolverine\", \"ironman\", \"hulk\"]\n full_tweet => true\n }\n}\n\nfilter { }\n\noutput {\n stdout {\n codec => dots\n }\n elasticsearch {\n user => \"xxxxxxxxxx\"\n password => \"xxxxxxxxxxxxx\"\n hosts => \"localhost:9200\"\n index => \"twitter_elastic_example\"\n document_type => \"tweets\"\n template => \"./twitter_template.json\"\n template_name => \"twitter_elastic_example\"\n template_overwrite => true\n }\n}\n\n{ \n \"template\": \"twitter_elastic_example\",\n \"settings\": {\n \"number_of_shards\": 1,\n \"number_of_replicas\": 0\n },\n \"mappings\": {\n \"_default_\": {\n \"_all\": {\n \"enabled\": true\n },\n \"properties\": {\n \"@timestamp\": {\n \"type\": \"date\",\n \"format\": \"dateOptionalTime\"\n },\n \"text\": {\n \"type\": \"text\"\n },\n \"user\": {\n \"type\": \"object\",\n \"properties\": {\n \"description\": {\n \"type\": \"text\"\n }\n }\n },\n \"coordinates\": {\n \"type\": \"object\",\n \"properties\": {\n \"coordinates\": {\n \"type\": \"geo_point\"\n }\n }\n },\n \"entities\": {\n \"type\": \"object\",\n \"properties\": {\n \"hashtags\": {\n \"type\": \"object\",\n \"properties\": {\n \"text\": {\n \"type\": \"text\",\n \"fielddata\": true\n }\n }\n }\n }\n },\n \"retweeted_status\": {\n \"type\": \"object\",\n \"properties\": {\n \"text\": {\n \"type\": \"text\"\n }\n }\n }\n },\n \"dynamic_templates\": [\n {\n \"string_template\": {\n \"match\": \"*\",\n \"match_mapping_type\": \"string\",\n \"mapping\": {\n \"type\": \"keyword\"\n }\n }\n }\n]\n }\n }\n}\n\n", :reason=>"Expected one of #, input, filter, output at line 30, column 1 (byte 804) after "}`

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 28, 2017, 12:38am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/4 "2017-02-28T00:38:56Z")

</div>

I can't see a line 30 in your config?

---

<div class="post-metadata">

**Author:** ![zbay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zbay/32/16735_2.png) [@zbay](https://discuss.elastic.co/u/zbay)\
**Post date:** [February 28, 2017, 12:44am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/5 "2017-02-28T00:44:50Z")

</div>

So going off another post I was reading in the process of trying to get this to work it was explained that the files are joined using a command something along these lines

```
`cat /etc/logstash/conf.d/* > ./test.conf` 

```

which basically vertically concatenation the .conf file to the .json, then it's sent off to the next stop. So line 30 actually falls in either the first of 2nd line of the json file when they are linked like that.

---

<div class="post-metadata">

**Author:** ![zbay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zbay/32/16735_2.png) [@zbay](https://discuss.elastic.co/u/zbay)\
**Post date:** [February 28, 2017, 12:47am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/6 "2017-02-28T00:47:06Z")

</div>

> [@zbay](#):
>
> Expected one of #, input, filter, output at line 30, column 1

here is the link [logstash - Error: Expected one of #, input, filter, output at line 24, column 1 (byte 528) after "} - Stack Overflow](http://stackoverflow.com/questions/34164603/error-expected-one-of-input-filter-output-at-line-24-column-1-byte-528)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 28, 2017, 12:52am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/7 "2017-02-28T00:52:54Z")

</div>

> [@zbay](#):
>
> which basically vertically concatenation the .conf file to the .json, then it's sent off to the next stop. So line 30 actually falls in either the first of 2nd line of the json file when they are linked like that.

What json file, the template? If so then no, it doesn't do that, it only cat's the LS config files. Not the template.

---

<div class="post-metadata">

**Author:** ![zbay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zbay/32/16735_2.png) [@zbay](https://discuss.elastic.co/u/zbay)\
**Post date:** [February 28, 2017, 1:01am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/8 "2017-02-28T01:01:08Z")

</div>

Well that isn't good then, Not sure why it's throwing an error on line 30. I guess I'll have to do some reading and playing around with the files.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 28, 2017, 6:58am UTC](https://discuss.elastic.co/t/error-expecting-line/76708/9 "2017-02-28T06:58:36Z")

</div>

Do you have the template JSON file in the same directory as the Logstash configuration files? Don't do that.

---

<div class="post-metadata">

**Author:** ![zbay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zbay/32/16735_2.png) [@zbay](https://discuss.elastic.co/u/zbay)\
**Post date:** [February 28, 2017, 12:34pm UTC](https://discuss.elastic.co/t/error-expecting-line/76708/10 "2017-02-28T12:34:09Z")

</div>

Ok awesome I'll relocate that document, I guess it came down to a problem with the `logstash.yml`, I was able to get it to run by using the following `/usr/share/logstash/bin/logstash --path.settings=/etc/logstash/logstash.yml twitter.conf` So I'll have to dig into that this morning. However I'll also get the json moved out of there and only keep my config files there.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 28, 2017, 7:46pm UTC](https://discuss.elastic.co/t/error-expecting-line/76708/11 "2017-02-28T19:46:38Z")

</div>

If you had the template file in the same dir as the `.conf` file, that would explain the error.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 28, 2017, 7:46pm UTC](https://discuss.elastic.co/t/error-expecting-line/76708/12 "2017-03-28T19:46:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.