# Issue in parsing the file which has # character at the beginning of the line

**URL:** <https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486>\
**Category:** Logstash\
**Created:** [August 19, 2016, 9:06pm UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486 "2016-08-19T21:06:59Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![kamesh](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@kamesh](https://discuss.elastic.co/u/kamesh)\
**Post date:** [August 19, 2016, 9:06pm UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/1 "2016-08-19T21:06:59Z")

</div>

Hi All,

I am new to logstash and recently started scripiting in logstash. we have a requirement where some logs files have # in the beginning of the line. Say  
####\<Aug 11, 2016 11:40:53 AM CDT\>\<Info\> \<Security\> \<HostName\>  
####\<Aug 11, 2016 11:40:54 AM CDT\> \<Info\> \<Security\> \<HostName\>  
now, I want to parse each line separately if it has #### in the beginning, but i tried couple of scripts and all are failing with ""Expected one of #, input, filter, output at line 28, column 1 (byte 302) after ", :level=\>:error}" error:

Scripts i used:

1:  
input  
{  
file  
{  
path =\> "/tmp/testXML1.log"  
codec =\> plain { charset =\> "US-ASCII"}  
type =\> "OSBLog"  
}  
}  
output {  
elasticsearch {  
hosts =\> ["192.168.99.100:9200"]  
codec =\> plain {charset =\> "US-ASCII"}  
}  
stdout { codec =\> rubydebug }  
}  
2:  
input  
{  
file  
{  
path =\> "/tmp/testXML1.log"  
codec =\> multiline  
{  
pattern =\> "^/####"  
}  
type =\> "OSBLog"  
}  
}  
output {  
elasticsearch {  
hosts =\> ["192.168.99.100:9200"]  
}  
stdout { codec =\> rubydebug }  
}

Any help will be highly appreciated.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 21, 2016, 7:36pm UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/2 "2016-08-21T19:36:41Z")

</div>

The error message you posted indicates that your configuration doesn't pass `logstash --configtest`. You're not telling us _which_ of these configurations you had problems with, but when testing myself it seems the first one is okay but the second configuration given the following error with Logstash 2.3.4:

```plaintext
Missing a required setting for the multiline codec plugin:

  codec {
    multiline {
      what => # SETTING MISSING
      ...
    }
  } {:level=>:error}
The given configuration is invalid. Reason: Something is wrong with your configuration. {:level=>:fatal}

```

And indeed, a multiline codec with just `pattern` set doesn't make sense. I'm not sure what you're trying to do there. What does

> I want to parse each line separately if it has #### in the beginning

mean? Please give more context to the problem. Concrete examples are better than descriptions.

---

<div class="post-metadata">

**Author:** ![kamesh](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@kamesh](https://discuss.elastic.co/u/kamesh)\
**Post date:** [August 22, 2016, 1:09am UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/3 "2016-08-22T01:09:38Z")

</div>

Thanks for your reply Magnus, looks like hash characters that I included are wiped off, may be I should have been clear.

I am trying to process the log files (oracle) using logstash and sending it to Kibana dashboard. The logs are of following format:  
####\<Aug 11, 2016 11:40:53 AM CDT\> \<Info\> \<Security\> \<hostname\> \<\>  
####\<Aug 11, 2016 11:40:54 AM CDT\> \<Info\> \<Security\> \<hostname\> \<\>

The requirement is whenever logstash sees a line with #### then it should treat it like a new line, from the above case it should index two times on kibana dashboard (one for each line).

I had a similar kind of requirement where each line starts with '[', where I wrote the following script and it is working fine. I tried to use the same replacing '[' by '####'  
input  
{  
file  
{  
path =\> "/tmp/testXML.log"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
codec =\> multiline  
{  
pattern =\> "^["  
negate =\> "true"  
what =\> previous  
}  
type =\> "OSBLog"  
}  
}  
output {  
elasticsearch {  
hosts =\> ["192.168.99.100:9200"]  
}  
stdout { codec =\> rubydebug }  
}  
I was of the view that Logstash is treating lines starting with '####' as commented lines.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 22, 2016, 5:38am UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/4 "2016-08-22T05:38:44Z")

</div>

I understand what you're trying to do, and I'm telling you that the configuration you showed us is invalid and obviously doesn't pass `logstash --configtest`.

> I had a similar kind of requirement where each line starts with '[', where I wrote the following script and it is working fine. I tried to use the same replacing '[' by '####'

That should work just fine.

> I was of the view that Logstash is treating lines starting with '####' as commented lines.

In configuration files yes, in log files no.

---

<div class="post-metadata">

**Author:** ![kamesh](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@kamesh](https://discuss.elastic.co/u/kamesh)\
**Post date:** [August 22, 2016, 1:29pm UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/5 "2016-08-22T13:29:09Z")

</div>

Can you give me some suggestion, where it will work through

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 22, 2016, 1:35pm UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/6 "2016-08-22T13:35:50Z")

</div>

Suggestion of what, exactly?

---

<div class="post-metadata">

**Author:** ![kamesh](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@kamesh](https://discuss.elastic.co/u/kamesh)\
**Post date:** [August 22, 2016, 2:32pm UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/7 "2016-08-22T14:32:44Z")

</div>

Thanks for answering my questions with patience. Looks like I did something wrong from my side, now it is working fine with the following code:  
input  
{  
file  
{  
path =\> "/tmp/testXML1.log"  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
codec =\> multiline  
{  
pattern =\> "^####"  
negate =\> "true"  
what =\> previous  
}  
type =\> "OSBLog"  
}  
}  
output {  
elasticsearch {  
hosts =\> ["192.168.99.100:9200"]  
}  
stdout { codec =\> rubydebug }  
}

thanks once again.  
~  
~  
~  
~

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:42am UTC](https://discuss.elastic.co/t/issue-in-parsing-the-file-which-has-character-at-the-beginning-of-the-line/58486/8 "2017-07-06T04:42:17Z")

</div>

Closed on July 6, 2017, 4:42am UTC