# Custom Beats vs. Scripts

**URL:** <https://discuss.elastic.co/t/custom-beats-vs-scripts/60791>\
**Category:** Beats\
**Created:** [September 17, 2016, 6:37pm UTC](https://discuss.elastic.co/t/custom-beats-vs-scripts/60791 "2016-09-17T18:37:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![DefensiveDepth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defensivedepth/32/110034_2.png) [@DefensiveDepth](https://discuss.elastic.co/u/DefensiveDepth)\
**Post date:** [September 17, 2016, 6:37pm UTC](https://discuss.elastic.co/t/custom-beats-vs-scripts/60791/1 "2016-09-17T18:37:19Z")

</div>

Working on a project that will be querying a number of data sources and inserting the resulting data into ES. The data sources include: Bing API, Twitter, Instagram, Facebook, and more social media platforms.

Thoughts on the following options, pros/cons/etc:

Option 1: Develop custom beats for each of these data sources and output directly to ES via https

Option 2: Write a number of scripts in $language to query these data sources and output directly to ES via https

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 17, 2016, 9:02pm UTC](https://discuss.elastic.co/t/custom-beats-vs-scripts/60791/2 "2016-09-17T21:02:14Z")

</div>

> [@DefensiveDepth](#):
>
> Option 1: Develop custom beats for each of these data sources and output directly to ES via https

apibeat sounds like a good idea, with modules for each site maybe?

Otherwise Logstash has a `http_poller` input.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 18, 2016, 7:22pm UTC](https://discuss.elastic.co/t/custom-beats-vs-scripts/60791/3 "2016-09-18T19:22:46Z")

</div>

I'd consider option 3: Write a number of scripts in $language to query these data sources and dump to files that are consumed by Filebeat. It's simple, language-agnostic, and gives you a natural buffer that helps with outages on the receiving end.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [September 19, 2016, 7:44am UTC](https://discuss.elastic.co/t/custom-beats-vs-scripts/60791/4 "2016-09-19T07:44:48Z")

</div>

@DefensiveDepth I would also check what kind of plugins are already available for Logstash.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 8, 2016, 6:37pm UTC](https://discuss.elastic.co/t/custom-beats-vs-scripts/60791/5 "2016-10-08T18:37:22Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.