# Logstash : pipeline is running but not generating any output file

**URL:** <https://discuss.elastic.co/t/logstash-pipeline-is-running-but-not-generating-any-output-file/135871>\
**Category:** Logstash\
**Created:** [June 14, 2018, 8:16am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-running-but-not-generating-any-output-file/135871 "2018-06-14T08:16:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![priya1](https://avatars.discourse-cdn.com/v4/letter/p/9d8465/32.png) [@priya1](https://discuss.elastic.co/u/priya1)\
**Post date:** [June 14, 2018, 8:16am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-running-but-not-generating-any-output-file/135871/1 "2018-06-14T08:16:32Z")

</div>

Hi Team,  
Please find the below details :

input{  
file {  
path =\> "C:\Users\priya\_shukla\Desktop\SAPLogsERR\*.log"  
start\_position =\> "beginning"  
sincedb\_path =\> "NUL"  
}  
}  
filter {  
grok {  
match =\> ["message", "%{DATA:core\_Id} %{DATA:date} %{DATA:timestamp} %{WORD:guiid} %{WORD:book\_refId} %{DATA:lastupdate} %{WORD:type} %{WORD:class} %{WORD:status} %{NUMBER:num} %{WORD:text} %{NUMBER:source\_id} %{WORD:carrier} %{DATA:related\_booking\_ids}"]  
}

```
		if "*_SUC.log" in [path]
		{
		grok{
		#For Other_TransData
		
	    add_field => { "correlation_Id" => "%{core_Id}" }
		add_field => { "booking_ref" => "%{book_refId}" }
	    add_field => { "sap_process_status" => "Successful" }
		add_field => { "sap_process_time" => "%{date}	%{timestamp}" }
		add_field => { "sap_message" => "%{message}" }
		add_field => { "message_GUI_ID" => "%{gui_Id}" }
		add_field => { "lara_last_update_date" => "%{date} %{timestamp}" }
		add_field => { "message_type" => "%{type}" }
		add_field => { "message_class" => "%{class}" }
		add_field => { "message_status" => "%{status}" }
		add_field => { "message_number" => "%{num}" }
		add_field => { "message_text" => "%{text}" }
		add_field => { "source_port_call_id" => "%{source_id}" }
		add_field => { "carrier" => "%{carrier}" }
		add_field => { "related_booking_ids" => "%{related_id}" }
		
		remove_tag => ["logs","_grokparsefailure"]
		}
		}
		
		else
		{  
		grok{
		
		#For Other_TransData
		
        add_field => { "sap_process_status" => "Failure" }
		add_field => { "sap_process_time" => "%{date} %{timestamp}" }
		add_field => { "sap_message" => "%{message}" }
		add_field => { "message_GUI_ID" => "%{gui_Id}" }
		add_field => { "lara_last_update_date" => "%{date} %{timestamp}" }
		add_field => { "message_type" => "%{type}" }
		add_field => { "message_class" => "%{class}" }
		add_field => { "message_status" => "%{status}" }
		add_field => { "message_number" => "%{num}" }
		add_field => { "message_text" => "%{text}" }
		add_field => { "source_port_call_id" => "%{source_id}" }
		add_field => { "carrier" => "%{carrier}" }
		add_field => { "related_booking_ids" => "%{related_id}" }
		
		#For SAP_TransData
		
		add_field => { "trx_id" => "%{core_Id}_SAP_AIF" }
		#For both Other_TransData and SAP_TransData
		add_field => { "correlation_Id" => "%{core_Id}" }
		add_field => { "booking_ref" => "%{book_refId}" }
		
		remove_tag => ["logs","_grokparsefailure"]
		}
		
		}
		
	if "Correlation ID" in [correlation_id]{     
		drop { }
	}  
	if "_grokparsefailure" in [tags]{     
		drop { }
	} 
	mutate {
		gsub => ["value", "\'", ""]
		remove_field => ["timestamp", "path","host","procurve","_grokparsefailure","booking_ref","date","type","message","@version"]
	}
	date {
        match => ["sap_process_time", "dd.MM.yyyy HH:mm:ss"]
        target => "sap_process_time"
    }
	clone {
	clones => ["Other_sap"]
	remove_field => ["trx_id","@version","@timestamp"]
	}

```

}

output{

```
    if [type] != "Other_sap"{
     file {	
		path => "C:\Users\priya_shukla\Downloads\logstash-6.2.4\logstash-6.2.4\output_data.txt"
	     }
	
	 }
	
 if [type] == "Other_sap"{
	elasticsearch{
	hosts => ["localhost:9200"]
	index => "test_demo"	
	action => 'update'
	document_id => "%{booking_ref}"
	doc_as_upsert => true		
	}
   }

```

}

output in logs :

[2018-06-14T13:34:51,554][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.2.4"}  
[2018-06-14T13:34:52,559][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
[2018-06-14T13:34:57,251][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>4, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50}  
[2018-06-14T13:34:58,416][INFO][logstash.pipeline] Pipeline started successfully {:pipeline\_id=\>"main", :thread=\>"#\<Thread:0x4c557ce0 sleep\>"}  
[2018-06-14T13:34:58,541][INFO][logstash.agent] Pipelines running {:count=\>1, :pipelines=\>["main"]}

Kindly hrlp me out in this.

Priya.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 14, 2018, 9:21am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-running-but-not-generating-any-output-file/135871/2 "2018-06-14T09:21:31Z")

</div>

> path =\> "C:\Users\priya\_shukla\Desktop\SAPLogsERR\*.log"

Use forward slashes, not backslashes.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2018, 9:21am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-running-but-not-generating-any-output-file/135871/3 "2018-07-12T09:21:33Z")

</div>

Closed on July 12, 2018, 9:21am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.