# OSQuery field types **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 17, 2019, 5:27am UTC](https://discuss.elastic.co/t/osquery-field-types/190875 "2019-07-17T05:27:25Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![DPattee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpattee/32/37772_2.png) [@DPattee](https://discuss.elastic.co/u/DPattee)\ **Post date:** [July 17, 2019, 5:27am UTC](https://discuss.elastic.co/t/osquery-field-types/190875/1 "2019-07-17T05:27:25Z")
I just realized all osquery fields get set to type 'string'. In osquery fields have appropriate types (for example for a temperature sensor it will have a string name and a double celsius temperature) But the json osqueryd.results.log that filebeat's osquery module reads from doesn't have any type hints. > {"name":"temperatures","calendarTime":"Wed Jul 17 05:15:37 2019 UTC","unixTime":1563340537,"epoch":0,"counter":10,"columns":{"fahrenheit":"123.1","name":"GPU Die"},"action":"added"} What's the best practices here? I need to have the right formats so I can do things like plot CPU usage and temperatures and have gauges and whatnot. The field names can change (or new ones appear at least) every time osquery gets updated (for queries that just do select \*) or one of the configured queries gets tweaked or a new osq pack enabled... Keeping ahead of those changes on the elastic index side of things seems like a losing battle ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [July 17, 2019, 12:44pm UTC](https://discuss.elastic.co/t/osquery-field-types/190875/2 "2019-07-17T12:44:28Z")
Hi @DPattee, In principle there is nothing that forces a field to be stored as string in Elasticsearch for this module, but it is true that all fields are sent as this type, so Elasticsearch automatically chooses this data type. Something you can try is to add the `convert` processor to your osquery module configuration for the fields you know that should be stored as numbers. It'd be something like: ```auto - module: osquery processors: - convert: fields: - {from: "osquery.result.fahrenheit", type: "double"} ignore_missing: true fail_on_error: false ``` Let me know if this works for you. Other option would be to add an OSQuery module to metricbeat, so it considers these fields as metrics, but this is not supported yet. ---
**Author:** ![DPattee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpattee/32/37772_2.png) [@DPattee](https://discuss.elastic.co/u/DPattee)\ **Post date:** [July 18, 2019, 1:22am UTC](https://discuss.elastic.co/t/osquery-field-types/190875/3 "2019-07-18T01:22:33Z")
Ahh, good tip, that will prevent some number of problems in the future. I'll have to reindex all my current data though. As for osquery being in metricbeat, that's actually where I looked for it originally. I'd say 75% of the things I'm planning on tracking with osquery actually match the metric style thought pattern vs the log style pattern. ---
**Author:** ![DPattee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpattee/32/37772_2.png) [@DPattee](https://discuss.elastic.co/u/DPattee)\ **Post date:** [July 18, 2019, 4:53am UTC](https://discuss.elastic.co/t/osquery-field-types/190875/4 "2019-07-18T04:53:32Z")
Hmm, I've never used the processors/convert option before so I might be configuring it wrong. > 2019-07-17T21:20:47.476-0700 ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Fileset osquery/processors is configured but doesn't exist Here's what I updated my modules.d/osquery.yml to: ``` - module: osquery result: enabled: true processors: - convert: fields: - {from: "osquery.result.columns.memory_gb", type: "double"} - {from: "osquery.result.columns.creation_time", type: "double"} - {from: "osquery.result.columns.failed_login_count", type: "integer"} - {from: "osquery.result.columns.failed_login_timestamp", type: "double"} - {from: "osquery.result.columns.fahrenheit", type: "double"} - {from: "osquery.result.columns.actual_rpm", type: "integer"} - {from: "osquery.result.columns.max_rpm", type: "integer"} - {from: "osquery.result.columns.min_rpm", type: "integer"} - {from: "osquery.result.columns.target_rpm", type: "integer"} - {from: "osquery.result.columns.executions", type: "integer"} - {from: "osquery.result.columns.interval", type: "integer"} - {from: "osquery.result.columns.output_size", type: "integer"} - {from: "osquery.result.columns.wall_time", type: "integer"} - {from: "osquery.result.columns.avg_user_time", type: "integer"} - {from: "osquery.result.columns.avg_system_time", type: "integer"} - {from: "osquery.result.columns.average_memory", type: "integer"} - {from: "osquery.result.columns.last_executed", type: "integer"} ignore_missing: true fail_on_error: false ``` Removing the entire processors section gets it back up and running. I double-checked that _ignore_ & _fail_ were lined up with _fields_, and _processors_ was lined up with _result_, and the individual field lines were one indent further in, and all the indents are double spaces... So I think I have the formatting done right at least ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [July 18, 2019, 11:57am UTC](https://discuss.elastic.co/t/osquery-field-types/190875/5 "2019-07-18T11:57:47Z")
@DPattee oh sorry, my mistake. Processors configuration must be placed at the top level configuration, in the main configuration file, and then they are applied to all events, or at the input level, so it is applied to an only input. To apply them when you are using a module, you need to override the input, something like this: ```auto - module: osquery result: enabled: true input: processors: - convert: fields: - {from: "osquery.result.columns.memory_gb", type: "double"} ... ``` You can read more about processors [here](https://www.elastic.co/guide/en/beats/filebeat/7.2/defining-processors.html). > [@DPattee](#): > > As for osquery being in metricbeat, that's actually where I looked for it originally. I'd say 75% of the things I'm planning on tracking with osquery actually match the metric style thought pattern vs the log style pattern. Feel free to [open an enhancement request](https://github.com/elastic/beats/issues/new?template=feature-request.md) for that 🙂 ---
**Author:** ![DPattee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpattee/32/37772_2.png) [@DPattee](https://discuss.elastic.co/u/DPattee)\ **Post date:** [July 18, 2019, 6:09pm UTC](https://discuss.elastic.co/t/osquery-field-types/190875/6 "2019-07-18T18:09:31Z")
Oh that was my fault... I'd "read" that page, and a couple others, several times, but only looked at the examples. Totally missed the line that says exactly what you said. > Similarly, for Filebeat modules, you can define processors under the `input` section of the module definition. Thanks for the help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 15, 2019, 6:09pm UTC](https://discuss.elastic.co/t/osquery-field-types/190875/7 "2019-08-15T18:09:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.