# Logstash kv filter takes each line of log as separate log

**URL:** <https://discuss.elastic.co/t/logstash-kv-filter-takes-each-line-of-log-as-separate-log/198433>\
**Category:** Logstash\
**Created:** [September 6, 2019, 2:06pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-takes-each-line-of-log-as-separate-log/198433 "2019-09-06T14:06:06Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sonammarda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sonammarda/32/52241_2.png) [@sonammarda](https://discuss.elastic.co/u/sonammarda)\
**Post date:** [September 6, 2019, 2:06pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-takes-each-line-of-log-as-separate-log/198433/1 "2019-09-06T14:06:06Z")

</div>

_Logstash version_ - 7.3  
_Filebeat version_ - 7.3  
_OS_ - Windows

My log format-  
\<log4j:event logger="ESM.EasyPurchaseMarketplace.Infrastructure.Logging.Logger" timestamp="1566964971685" level="ERROR" thread="7388"\>log4j:message

```
======================================
 
 MachineName: PRODMAGWEB1 
 TimeStamp: 8/28/2019 12:02:51 AM 
 FullName: ESM.EasyPurchaseMarketplace.Infrastructure.Logging, Version=2.51.7170.6286, Culture=neutral, PublicKeyToken=null 
 AppDomainName: /LM/W3SVC/8/ROOT-1-132113700121002852 
 WindowsIdentity: ESMSOLUTIONS\EPMServiceAppIdentit 
 Message: Exception occured in PurchaseAPI token logic 

 Guid -  
 UserId -  
 EntityId -  
 ImpersonatedBy -  
 ImpersonatingUserId -  
 ImpersonatedUserName -  

</log4j:message><log4j:properties><log4j:data name="log4net:UserName" value="ESMSOLUTIONS\EPMServiceAppIdentit" /><log4j:data name="log4jmachinename" value="ProdMagWeb1" /><log4j:data name="log4japp" value="/LM/W3SVC/8/ROOT-1-132113700121002852" /><log4j:data name="log4net:HostName" value="ProdMagWeb1" /></log4j:properties><log4j:throwable><![CDATA[System.Web.HttpException (0x80070057): The remote host closed the connection. The error code is 0x80070057.
   at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
   at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()
   at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)
   at System.Web.HttpResponse.Flush()
   at System.Web.HttpWriter.WriteFromStream(Byte[] data, Int32 offset, Int32 size)
   at System.Web.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at Microsoft.Owin.Host.SystemWeb.CallStreams.OutputStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at Microsoft.AspNet.SignalR.Owin.ServerResponse.Write(ArraySegment`1 data)
   at Microsoft.AspNet.SignalR.Hosting.ResponseExtensions.End(IResponse response, String data)
   at Microsoft.AspNet.SignalR.PersistentConnection.SendJsonResponse(HostContext context, String jsonPayload)
   at Microsoft.AspNet.SignalR.PersistentConnection.ProcessNegotiationRequest(HostContext context)
   at Microsoft.AspNet.SignalR.PersistentConnection.ProcessRequest(HostContext context)
   at Microsoft.AspNet.SignalR.Hubs.HubDispatcher.ProcessRequest(HostContext context)
   at Microsoft.AspNet.SignalR.PersistentConnection.ProcessRequest(IDictionary`2 environment)
   at Microsoft.AspNet.SignalR.Owin.Middleware.HubDispatcherMiddleware.Invoke(IOwinContext context)
   at Microsoft.Owin.Infrastructure.OwinMiddlewareTransition.Invoke(IDictionary`2 environment)
   at Microsoft.Owin.Cors.CorsMiddleware.HandleCorsRequestAsync(IOwinContext context, CorsPolicy policy, CorsRequestContext corsRequestContext)
   at Microsoft.Owin.Cors.CorsMiddleware.<Invoke>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Mapping.MapMiddleware.<Invoke>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at ESM.Purchase.WebApi.Common.GlobalOwinExeptionHandler.<Invoke>d__0.MoveNext()]]></log4j:throwable><log4j:locationInfo class="ESM.EasyPurchaseMarketplace.Infrastructure.Logging.Logger" method="WriteToLog" file="" line="0" /></log4j:event>

```

Logstash conf file-

```
input { 
	beats{
		port=>5044
	}
}

filter { 
	kv {
	 value_split => ":"
	}
}

output {
if "esm-purchase-api-log" in [tags] { 
elasticsearch {
hosts => ["localhost:9200"]
index => "esm-purchase-api-log-%{+YYYY.MM.dd}"
}
}
}

```

It is splitting fields correctly based on colon( : ) but it takes each line as separate log as shown in screenshot-

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/6/86ae06859ea969cf1e650a00649ad74efdf004df.png)

It should take all values- MachineName,FullName,Timestamp,AppDomainName and WindowsIdentity into one log but it is taking each line as separate log entry.

What should I do to so that it is considered as 1 single log?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 6, 2019, 2:24pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-takes-each-line-of-log-as-separate-log/198433/2 "2019-09-06T14:24:46Z")

</div>

Have filebeat manage the [multiline messages](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 4, 2019, 2:24pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-takes-each-line-of-log-as-separate-log/198433/3 "2019-10-04T14:24:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.