# Response Headers http\_poller
**URL:** https://discuss.elastic.co/t/response-headers-http-poller/80739
**Category:** Logstash
**Created:** [March 30, 2017, 8:19pm UTC](https://discuss.elastic.co/t/response-headers-http-poller/80739 "2017-03-30T20:19:20Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@jcannell](https://discuss.elastic.co/u/jcannell)
#### Post date: [March 30, 2017, 8:19pm UTC](https://discuss.elastic.co/t/response-headers-http-poller/80739/1 "2017-03-30T20:19:20Z")
Is it possible to access the response headers from events processed by the http\_poller input plugin? I'm working with the Microsoft Table Service REST API:
[https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/querying-tables-and-entities](https://docs.microsoft.com/en-us/rest/api/storageservices/fileservices/querying-tables-and-entities)
The API returns the next row/partition in the response headers which point me where I need to go next for data.
Thank you
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 30, 2017, 8:24pm UTC](https://discuss.elastic.co/t/response-headers-http-poller/80739/2 "2017-03-30T20:24:50Z")
> Is it possible to access the response headers from events processed by the http\_poller input plugin?
Yes, they're stored in the `@metadata` field by default but you can configure the input to store it elsewhere if you like.
However, if you want this information so that the next round of http\_poller can request the next batch of data that won't be possible.
---
### Author:  [@jcannell](https://discuss.elastic.co/u/jcannell)
#### Post date: [March 30, 2017, 8:41pm UTC](https://discuss.elastic.co/t/response-headers-http-poller/80739/3 "2017-03-30T20:41:31Z")
Thank you.
So, it wouldn't be possible to add a field with an initial value, and then update it with a value in the response header?
---
### Author:  [@jordansissel](https://discuss.elastic.co/u/jordansissel)
#### Post date: [March 30, 2017, 9:48pm UTC](https://discuss.elastic.co/t/response-headers-http-poller/80739/4 "2017-03-30T21:48:08Z")
Having the http\_poller input be aware of how to paginate is something we've discussed internally. I personally feel it's not something we can achieve because of the ways that things present pagination, such as:
- Fetch the main page, get a token that you can query repeatedly in the future to get the next page of results (Elasticsearch Scroll API does this)
- Fetch the first page, and the response includes an identifier for the next page (Github API does this, as does your Microsoft service, from your description)
- Fetch pages by number (1, 2, 3, 4) and stop fetching when the current page has no results.
There are probably more pagination strategies, and I'm not sure we can prepare the http\_poller plugin to support them all.
In cases such as this, I generally recommend that a new input plugin be created that has specific knowledge of how to read data from a specific data source. In this case, it would be my recommendation to have a custom input plugin that knows how to handle the pagination strategy deployed by the Microsoft Table Service.
---
### Author:  [@jcannell](https://discuss.elastic.co/u/jcannell)
#### Post date: [March 31, 2017, 12:17pm UTC](https://discuss.elastic.co/t/response-headers-http-poller/80739/5 "2017-03-31T12:17:41Z")
Thank you for your detailed response. I understand the challenges you bring up and they are good points. I haven't looked into creating custom plugins yet but I might see if I can whip something up to handle my specific use case.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 28, 2017, 12:18pm UTC](https://discuss.elastic.co/t/response-headers-http-poller/80739/6 "2017-04-28T12:18:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.