Importing data from MYSQL to Kibana using LOGSTASH

I am sharing my sample document

{ "_index": "publication", "_type": "_doc", "_id": "XMbVJXUBBO8i-6d1ZwTb", "_version": 1, "_score": null, "_source": { "affiliation_city": "Chennai", "description": "© 2020 Elsevier Ltd Process industries have variables that are measured at different rates, with some measurements, such as composition or quality variables, obtained after associated measurement delays. This work introduces a novel “sampled output augmentation” method for fusing delayed and infrequent primary measurements within a multirate Kalman filter (KF) framework. This is a state-augmentation-based method and is more parsimonious than other approaches. The stability of the sampled output augmentation is analyzed. We analytically show the equivalence of the proposed method with the traditional fixed-lag smoothing method for a linear system. Extension to nonlinear systems through extended Kalman filter (EKF) is presented. Finally, we assess the performance of the proposed method in comparison with other available estimators through linear and non-linear case studies.", "doi": "10.1016/j.ces.2020.115763", "author_names": "Ravi, Arvind;Narasimhan, Shankar;Kaisare, Niket S.", "issn": "00092509", "pub_id": 138191, "source_id": "16413", "citedby_count": 0, "pii": "S0009250920302955", "pagerange": "nan", "publication_year": "12 October 2020", "data_source": "update_08_10", "data_source1": "", "author_count": 3, "fund_acr": "", "title": "Sampled Ooutput augmentation method for handling measurement delays in multirate Kalman filter", "eid": "2-s2.0-85085194348", "creator": "Ravi A.", "author_afids": "60025757;60025757;60025757", "coverdate": "2020-10-11T18:30:00.000Z", "issueidentifier": "nan", "affilname": "Indian Institute of Technology Madras", "affiliation_country": "India", "publicationname": "Chemical Engineering Science", "volume": "224", "openaccess": "Closed Access", "@version": "1", "fund_no": "", "subtype": "ar", "afid": "60025757", "subtypedescription": null, "eissn": "", "authkeywords": "Extended Kalman Filter | Measurement delay | Multirate measurements | State augmentation", "fund_sponsor": "", "subject": "Applied Mathematics;Industrial and Manufacturing Engineering;General Chemical Engineering;General Chemistry", "author_ids": "57203139450;57213900501;9274526600", "aggregationtype": "Journal", "article_number": "115763", "pubmed_id": "", "@timestamp": "2020-10-14T06:38:34.586Z", "institute_id": 1011 }, "fields": { "coverdate": [ "2020-10-11T18:30:00.000Z" ], "@timestamp": [ "2020-10-14T06:38:34.586Z" ] }, "sort": [ 1602441000000 ] }

In this sample the field author_afids is having multiple values. And every Document will have multiple author_afids. I want to visualize in kibana which author ID has appeared most.

You'll need to split your author field into something analyzable because right now it looks like it's just a string. You can use our Split processor for this or roll your own solution.

And then maybe you're looking at creating a table visualization to sort all the authors.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.