# Process logs of different formats to JSON
**URL:** https://discuss.elastic.co/t/process-logs-of-different-formats-to-json/339258
**Category:** Logstash
**Created:** [July 26, 2023, 6:45am UTC](https://discuss.elastic.co/t/process-logs-of-different-formats-to-json/339258 "2023-07-26T06:45:42Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Chel\_Db](https://discuss.elastic.co/u/Chel_Db)
#### Post date: [July 26, 2023, 6:45am UTC](https://discuss.elastic.co/t/process-logs-of-different-formats-to-json/339258/1 "2023-07-26T06:45:43Z")
I'm pretty new to ELK and I'm trying to push few of our service's logs to ES.
Log funneling flow is --\> `
`Fluentd --> Logstash --> ES --> Kibana.`
`
A thing to note is that, each service has its own log format. Attached four different service's logs for reference.
How can I handle all four format in a generic way ? I'm looking to structure it or convert all the logs into a JSON format while being displayed on Kibana.
Current configuration.
```auto
input {
http {
port => 9100
}
}
filter {
json {
source => "message"
remove_field => ["message"]
}
if [kubernetes][labels][id] in ["service1", "service2", "service3", "service4"] {
json {
source => "log"
remove_field => ["log"]
}
}
mutate {
remove_field => ["_id", "headers", "_type", "character", "@version", "_index", "_score"]
}
}
output {
elasticsearch {
id => "ieo-elk-drp-log-aggregation"
hosts => ["<%= @ipaddress%>:19200"]
index => "drp-%{[kubernetes][labels][app]}-%{+YYYY.MM.dd}"
}
}
```
On the logstash logs, continuously see the below errors and it indicates the logs are of not of proper JSON. Any help here is really appreciated, thank you.
```auto
[2023-07-20T00:06:41,900][WARN][logstash.filters.json] Error parsing json {:source=>"log", :raw=>"INFO |2023-07-20 00:05:09 |1 | manageQueue.py 324 | There is nothing to process on the queue as it is empty. Returning...", :exception=>#"log", :raw=>"Thu Jul 20 00:09:07 2023 sent heartbeat", :exception=>#"log", :raw=>"No Jira servers defined at /usr/src/app/value.pm line 70.", :exception=>#}
[2023-07-20T02:59:48,393][WARN][logstash.filters.json] Error parsing json {:source=>"log", :raw=>"Thu Jul 20 02:59:23 2023 sent heartbeat", :exception=>#}
[2023-07-20T02:59:56,626][WARN][logstash.filters.json] Error parsing json {:source=>"log", :raw=>"malformed JSON string, neither array, object, number, string or atom, at character offset 11 (before \"\\n\\n\\n ...\") at (eval 250) line 1.", :exception=>#"log", :raw=>"2023/07/20 06:50:48| NETDB state saved; 1 entries, 0 msec", :exception=># **[service\_4.txt](https://www.dropbox.com/scl/fi/xopxu7qsua859pxbjxisi/service_4.txt?rlkey=xjbt58lp1tpsxw0qodlmaaayo&dl=0)**
>
> Shared with Dropbox
> **[service\_1.txt](https://www.dropbox.com/scl/fi/kncys59lnch4a30jvycyo/service_1.txt?rlkey=f026rfsr0t6xdcru6635vaxy1&dl=0)**
>
> Shared with Dropbox
> **[service\_2.txt](https://www.dropbox.com/scl/fi/e7apr5jrn4raj10frje2w/service_2.txt?rlkey=u5alj34abb9kqnlm4kflgrhwk&dl=0)**
>
> Shared with Dropbox
> **[service\_3.txt](https://www.dropbox.com/scl/fi/3bngym9egoh9903lj5gbf/service_3.txt?rlkey=xr6tft1tg56ni9dwe8acb4aj4&dl=0)**
>
> Shared with Dropbox
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 23, 2023, 6:46am UTC](https://discuss.elastic.co/t/process-logs-of-different-formats-to-json/339258/2 "2023-08-23T06:46:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.