# ServiceNow Incident Table data in ELK **URL:** \ **Category:** Logstash\ **Created:** [May 2, 2018, 9:41am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216 "2018-05-02T09:41:51Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 2, 2018, 9:41am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/1 "2018-05-02T09:41:52Z")
Hey Guys, I have been trying to get servicenow incident data into ELK through http\_poller method using logstash, i'm getting the data but all the incident records are sitting under a single field. Because of this i'm not able to create any visualization, as all the data are in single field i'm not able to break the data. FYI... ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/c/4cafef2e54ff4f9dffa69fdfa94ee99fa291c897.png) All the incident data is falling under this record field. Am i doing it rite, or do we need to use some other method to GET servicenow data into ELK.Please advice. Thanks Gautham ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 2, 2018, 1:25pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/2 "2018-05-02T13:25:02Z")
Make sure you have `codec => "json"` in your http\_poller configuration. See the example in the plugin's documentation. ---
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 3, 2018, 10:30am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/3 "2018-05-03T10:30:29Z")
Thanks for the response @magnusbaeck I do have codec =\> "json" Here is my config file. input { http\_poller { urls =\> { url =\> "[https://dev.service-now.com/api/208950/test\_response](https://dev.service-now.com/api/208950/test_response)" } request\_timeout =\> 60 proxy =\> { host =\> "10._._._" port =\> "__**" scheme =\> "http"} user =\> "\*\*\*\*\*" password =\> "**_\*_" schedule =\> { cron =\> " \* \* \* "} metadata\_target =\> "http\_poller\_metadata" **codec =\> "json"** } } output { elasticsearch { hosts =\> ["10..._:9200"] index =\> "servicetest" } stdout { codec =\> rubydebug } } Thanks Gautham ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 3, 2018, 11:16am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/4 "2018-05-03T11:16:25Z")
That's odd. What does the HTTP response actually look like, i.e. what do you get if you fetch the URL with e.g. curl? ---
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 3, 2018, 1:17pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/5 "2018-05-03T13:17:40Z")
@magnusbaeck Here is the output curl "[https://dev53030.service-now.com/api/now/table/incident?sysparm\_display\_value=false&sysparm\_fields=number%2Cstate%2Cpriority&sysparm\_limit=1](https://dev53030.service-now.com/api/now/table/incident?sysparm_display_value=false&sysparm_fields=number%2Cstate%2Cpriority&sysparm_limit=1)" --request GET --user '\*\*\*\*\*':'\*\*\*\*\*\*\*' **{"result":[{"number":"INC0000001","state":"7","priority":"1"}]}** Thanks Gautham ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 3, 2018, 1:33pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/6 "2018-05-03T13:33:42Z")
Okay, but what about the `records` field that's included in the screenshot you posted earlier? ---
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 3, 2018, 2:22pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/7 "2018-05-03T14:22:45Z")
That actually has a lot of data to be copied as i ran for whole Incident table. Now for quick reference have ran for only one incident with limited fields as i posted the result above. still all the data are sitting under single field. Thanks Gautham ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 3, 2018, 2:51pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/8 "2018-05-03T14:51:19Z")
I still want to see the raw contents of the `records` field. ---
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 8, 2018, 9:05am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/9 "2018-05-08T09:05:55Z")
Sorry for the delay @magnusbaeck , here is the details you asked for. records { "impact": "1", "assigned\_to": "46b87022a9fe198101a78787e40d7547", "closed\_at": "2017-11-26 23:10:06", "subcategory": "", "work\_start": "", "sys\_mod\_count": "21", "upon\_reject": "", "time\_worked": "", "additional\_assignee\_list": "", "close\_code": "Closed/Resolved by Caller", "caller\_id": "5137153cc611227c000bbd1bd8cd2005", "upon\_approval": "", "sla\_due": "", "calendar\_stc": "7937181", "active": "false", "calendar\_duration": "1970-04-02 20:46:21", "urgency": "1", "work\_notes": "", "watch\_list": "", "approval": "", "category": "network", "resolved\_by": "6816f79cc0a8016401c5a33be04be441", "short\_description": "Can't read email", "severity": "1", "escalation": "0", "parent\_incident": "", "sys\_domain\_path": "/", "company": "", "sys\_tags": "", "closed\_by": "9ee1b13dc6112271007f9d0efdb69cd0", "opened\_by": "681ccaf9c0a8016400b98a06818d57c7", "problem\_id": "9d3a266ac6112287004e37fb2ceb0133", "correlation\_display": "", "cmdb\_ci": "b0c4030ac0a800090152e7a4564ca36c", "delivery\_task": "", "comments": "", "sys\_domain": "global", "correlation\_id": "", "description": "User can't access email on [mail.company.com](http://mail.company.com).\n\t\t", "state": "7", "sys\_id": "9c573169c611228700193229fff72400", "hold\_reason": "", "sys\_updated\_on": "2018-02-25 20:16:07", "made\_sla": "false", "sys\_updated\_by": "admin", "priority": "1", "approval\_set": "", "activity\_due": "", "notify": "1", "group\_list": "", "reassignment\_count": "1", "knowledge": "false", "work\_end": "", "expected\_start": "", "delivery\_plan": "", "resolved\_at": "2018-02-25 19:56:12", "sys\_created\_on": "2016-06-26 18:24:13", "business\_stc": "1892781", "due\_date": "", "assignment\_group": "d625dccec0a8016700a222a0f7900d06", "approval\_history": "", "child\_incidents": "", "business\_service": "", "opened\_at": "2017-11-25 23:09:51", "caused\_by": "", "order": "", "\_\_status": "success", "number": "INC0000001", "contact\_type": "", "close\_notes": "Closed before close notes were made mandatory\n\t\t", "sys\_class\_name": "incident", "business\_duration": "1970-01-22 21:46:21", "incident\_state": "7", "comments\_and\_work\_notes": "", "work\_notes\_list": "", "reopen\_count": "", "location": "1083361cc611227501b682158cabf646", "sys\_created\_by": "pat", "parent": "", "rfc": "", "follow\_up": "", "user\_input": "" }, These fields are for single incident, like this i have received data for all the incidents available in my servicenow instance. As you can see, all these are falling under a single field "record" here is my configuration file. input { http\_poller { urls =\> { url =\> "[https://dev.service-now.com/incident\_list.do?JSONv2&incident\_list.do?JSONv2&display\_value=True&sysparm\_exclude\_reference\_link=True&sysparm\_fields=resolved\_by%2Cstate%2Ccmdb\_ci%2Cpriority%2Ccaller\_id%2Cshort\_description%2Cassignment\_group%2Cassigned\_to%2Cseverity%2Clocation%2Cnumber&sysparm\_limit=1sysparm\_view=json\_view](https://dev.service-now.com/incident_list.do?JSONv2&incident_list.do?JSONv2&display_value=True&sysparm_exclude_reference_link=True&sysparm_fields=resolved_by%2Cstate%2Ccmdb_ci%2Cpriority%2Ccaller_id%2Cshort_description%2Cassignment_group%2Cassigned_to%2Cseverity%2Clocation%2Cnumber&sysparm_limit=1sysparm_view=json_view)" } request\_timeout =\> 60 proxy =\> { host =\> "10.1.1.2" port =\> "8181" scheme =\> "http"} user =\> "admin" password =\> "\*\*\*\*\*\*\*\*" schedule =\> { cron =\> "\* \* \* \* \*"} codec =\> "json" metadata\_target =\> "http\_poller\_metadata" } } filter { json { source =\> "records" } } output { elasticsearch { hosts =\> ["10.1.1.4:9200"] index =\> "servicenowinc" } stdout { codec =\> rubydebug } } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 8, 2018, 10:15am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/10 "2018-05-08T10:15:26Z")
The `records` field is an array of objects. Kibana simply doesn't handle arrays of objects that well. Perhaps you should use the split filter on the `records` field and get one event per record? Then the fields will be available as I think you expect. ---
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 8, 2018, 10:21am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/11 "2018-05-08T10:21:35Z")
@magnusbaeck I have not used split filter before, is there any documentation available for this, or can you gimme me some example config file, that will help me a lot. Thanks Gautham ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 8, 2018, 11:32am UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/12 "2018-05-08T11:32:07Z")
[https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html) ---
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 8, 2018, 12:42pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/13 "2018-05-08T12:42:58Z")
Thank you very much @magnusbaeck for you guidance.....i'll go through docs. Thanks Gautham ---
**Author:** ![Gauti](https://avatars.discourse-cdn.com/v4/letter/g/cdc98d/32.png) [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Post date:** [May 9, 2018, 1:55pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/14 "2018-05-09T13:55:35Z")
Bingo......It worked as i expected @magnusbaeck .. Thank you very much.... Thanks Gautham ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 6, 2018, 1:55pm UTC](https://discuss.elastic.co/t/servicenow-incident-table-data-in-elk/130216/15 "2018-06-06T13:55:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.