Error in Logstash log . "Error: Disc quota exceeded"


#1

i am getting Error: Disc quota exceeded", :level=>:error in the logstash log. why this occurs and how can it be resolved?

Logstash is being used to Push the data to ELastic Search. In the OuptFilter of Logstash we have mentioned host name and index name


(pts0) #2

Is your disk full ?
Logstash need a little bit space to save status and queues ...

pts0


#3

We have disk space of around 40GB avaialble. And provided below how the error looks from the logstash log:
{:timestamp=>"2017-07-24T05:41:07.568000-0400", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::File type=>"Sample",
path=>["/mySampleDir/log/*"], codec=><LogStash::Codecs::Plain charset=>"UTF-8">, stat_interval=>1, discover_interval=>15, sincedb_write_inter
val=>15, start_position=>"end", delimiter=>"\n">\n Error: Disc quota exceeded", :level=>:error}

How much min/max space Logstash needs for its usage?
Do we need to specify the max size of log entries in Logstash configuration file?

Thanks in advance


(pts0) #4

Hmm,

40GB should be ok, are u sure is quota for the user set, maybe you have space but you run out of quota.
How much space for logstash, I have no idea. For the since files (store where the file pointer are keep) some kilobytes. For the queue is depending of how big are your data and how good or bad is connection to elastic.
Can you try to start with another user or root ?

Maybe you post you config ...

thanks
pts0


#5

Hi,
As you said i have started with another user but still faces the same issue . Below is my logstash config

input {
file {
type => "Audit_1"
path => "/mySampleDir/log/*"

	}
 file
    {
        type => "Audit_2"
        path => "/mySampleDir2/log/*"
       
    }

}
filter {
if [type] == "Audit_1"
{
grok {
pattern => "%{GREEDYDATA:DATE}|%{GREEDYDATA:GATEWAY}|%{GREEDYDATA:APPLICATION}|%{GREEDYDATA:REQUEST}|%{GREEDYDATA:TRACE_ID}|%{GREEDYDATA:UNIQUE_ID}|%{GREEDYDATA:RECEIVE_TIME}|"

add_field => {
"recvTime" => "%{DATE} %{RECEIVE_TIME}"
}
}
}

 if [type] == "Audit_2" 
{

grok {
pattern => "%{GREEDYDATA:DATE}|%{GREEDYDATA:REQUEST}|%{GREEDYDATA:TRACE_ID}|%{GREEDYDATA:UNIQUE_ID}|%{GREEDYDATA:ACCT_ID}|%{GREEDYDATA:RECEIVE_TIME}|"

add_field => {
"recvTime" => "%{DATE} %{RECEIVE_TIME}"
}
}
}

date{
match => ["recvTime" , "MMddyy HH:mm:ss.SSS"]
target => "my_recv_time"
}

}
output
{
elasticsearch {
hosts => ["localhost:9200"]
index => "my_audit_logs"
}

}

The error is as below:
{:timestamp=>"2017-07-27T02:45:44.407000-0400", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::File type=>"Audit_2",
path=>["/mySampleDir2/log/*"], codec=><LogStash::Codecs::Plain charset=>"UTF-8">, stat_interval=>1, discover_interval=>15, sincedb_write_inter
val=>15, start_position=>"end", delimiter=>"\n">\n Error: Disc quota exceeded", :level=>:error}

Please help as i am clue less and did not find any source for this error over the internet,

Thanks in Advance


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.