Unable to parse pipe separated data in nifi logs using logstash(This data is being generated due to a Controller Status Reporting Task)


#1

Below is the sample log that is being generated due to Controller Status Reporting Task.

2018-12-05 10:06:38,061 INFO [Timer-Driven Process Thread-12] o.a.n.c.C.Connections Connection Statuses:

| Connection ID | Source | Connection Name | Destination | Flow Files In | Flow Files Out | FlowFiles Queued |

| 45c4XXXX-d9b2-XXXc-d168-XXXb9104816e | EvaluateJsonPath | matched | FailOverIngestionXXX | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) | 11 / 6.43 KB (+0/+0 bytes) |
| 4aab324f-b795-1c42-775b-3148a9253ef4 | putElasticsearchHttp | failure | putElasticsearchHttp | 50 / 38.23 KB (+0/+0 bytes) | 50 / 38.23 KB (+0/+0 bytes) | 5 / 3.82 KB (+0/+0 bytes) |
| 01661003-be2e-118a-b3f5-4fc0f00cda72 | FailOverIngestionPort_v0.1 | | FailOverIngestionPort_v0.1 | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) |
| dcd03f5c-c141-1dce-a426-d4a4420996ba | ingestionPort_v0.2_PE | | ingestionPort_v0.2_PE | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) |
| 5d1d2e03-0166-1000-0998-20698b061837 | ingestionPort_v0.2 | | ingestionPort_v0.2 | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) |
| 98951cbb-3ced-1cb0-6e73-2749124ee2d3 | provenanceMonitoring | | LogAttribute | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) |
| dcd04081-c141-1dce-1edd-1ea0954f92d2 | ingestionPort_v0.2_PE | | ingestionPort_v0.2_PE | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) |
| 7349e222-0167-1000-4972-8df929be36d4 | fetchDataFromHive | success | successFetchDataFromHive | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 / 0 bytes (+0/+0 bytes) |

My expected output is two have an index with attributes as per the header of the above data:
eg: {
"Connection ID":["45c4XXXX-d9b2-XXXc-d168-XXXb9104816e "],
"Source":["EvaluateJsonPath"],
"Connection Name": ["matched"].
"Destination":["FailOverIngestionXXX"],
"Flow Files In":["0 / 0 bytes (+0/+0 bytes)"],
"Flow Files Out":["0 / 0 bytes (+0/+0 bytes)"],
"FlowFiles Queued":["11 / 6.43 KB (+0/+0 bytes)"]
}

PS: I am fairly a beginner to #logstash