# Importing tweets CSV date parsing

**URL:** <https://discuss.elastic.co/t/importing-tweets-csv-date-parsing/161014>\
**Category:** Logstash\
**Created:** [December 16, 2018, 6:56am UTC](https://discuss.elastic.co/t/importing-tweets-csv-date-parsing/161014 "2018-12-16T06:56:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gid](https://avatars.discourse-cdn.com/v4/letter/g/439d5e/32.png) [@Gid](https://discuss.elastic.co/u/Gid)\
**Post date:** [December 16, 2018, 6:56am UTC](https://discuss.elastic.co/t/importing-tweets-csv-date-parsing/161014/1 "2018-12-16T06:56:39Z")

</div>

Hi all,

Trying to import tweet data from Twitter's recently released archives and running into issues with date fields. The goal is to convert the tweet\_time field from the CSV as @timestamp to be used as time filter field. The field gets loaded as text/string instead of date. The account\_creation\_date _is_ recognised correctly.

Data source:  
[https://about.twitter.com/en\_us/values/elections-integrity.html#data](https://about.twitter.com/en_us/values/elections-integrity.html#data)  
("Tweet information" link under Internet Research Agency)

Sample values & format from tweet\_date column:  
2014-07-24 19:20  
2014-11-22 15:28  
2017-03-13 22:08

For reference, account\_creation\_date field is formatted as follows (and is recognised as date, even without me explicitly defining its format):  
2012-02-25  
2014-04-29  
2013-12-07

logstash config:

```auto
input {
      file {
          path => "/Users/gid/Downloads/test.csv" # 24-row test sample
          start_position => "beginning"
          sincedb_path => "/dev/null"
      }
}

filter {
    csv {
        separator => ","

        columns => ['tweetid', 'userid', 'user_display_name', 'user_screen_name',
       'user_reported_location', 'user_profile_description',
       'user_profile_url', 'follower_count', 'following_count',
       'account_creation_date', 'account_language', 'tweet_language',
       'tweet_text', 'tweet_time', 'tweet_client_name', 'in_reply_to_tweetid',
       'in_reply_to_userid', 'quoted_tweet_tweetid', 'is_retweet',
       'retweet_userid', 'retweet_tweetid', 'latitude', 'longitude',
       'quote_count', 'reply_count', 'like_count', 'retweet_count', 'hashtags',
       'urls', 'user_mentions', 'poll_choices']

        id => "tweetid"

        convert => {
            "follower_count" => "integer"
            "following_count" => "integer"
            "is_retweet" => "boolean"
            "quote_count" => "integer"
            "reply_count" => "integer"
            "like_count" => "integer"
            "retweet_count" => "integer"
            #"account_creation_date" => "date"
            "tweet_time" => "date_time"
        }
    }

    date {
        match => ["tweet_time", "yyyy/MM/dd HH:mm"]
        target => "tweet_time"
    }
}

output {
    elasticsearch {
        hosts => "localhost"
        index => "iratweetsv1"
        document_type => "tweet"
    }
}

```

How do I configure this correctly? I've tried many different setups but have not been able to parse these dates correctly. According to the mapping, this field is text:

```auto
"tweet_time": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }

```

As opposed to account\_creation\_date:

```auto
"account_creation_date": {
          "type": "date"
        }

```

EDIT: Running ES, Kibana and logstash version 6.5.3

---

<div class="post-metadata">

**Author:** ![bryan\_stuhlsatz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bryan_stuhlsatz/32/49123_2.png) [@bryan\_stuhlsatz](https://discuss.elastic.co/u/bryan_stuhlsatz)\
**Post date:** [December 16, 2018, 8:39pm UTC](https://discuss.elastic.co/t/importing-tweets-csv-date-parsing/161014/2 "2018-12-16T20:39:57Z")

</div>

Where did u get the mapping? Did you create the mapping for your index which specifies tweet\_time to be a text field?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 13, 2019, 8:40pm UTC](https://discuss.elastic.co/t/importing-tweets-csv-date-parsing/161014/3 "2019-01-13T20:40:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.