Resolved - Need Help for the error “Invalid format: “11/19/2014” is malformed at “/19/2014”" in elasticsearch

Hi Team,

I am facing an error related with date in elasticsearch. Below is the detail information, can you please help advice what's the problem? I googled this issue and there is do similar issue, but tried many times with the possible solution, but all is failed and the error keeps occurring.

1. I have below CSV file which contains a date information "COMMIT_DATE".

GIT_ORG,GIT_REPOS,COMMIT_SHA1,COMMIT_AUTHOR,COMMIT_DATE,COMMIT_COMMENT,JIRA_KEY,JIRA_ISSUE_TYPE,JIRA_ISSUE_CATEGORY,JIRA_PRIORITY,JIRA_COMPONENTS,JIRA_DEFECT_TYPE,JIRA_TESTCASE_ID,JIRA_FIND_BY_AUTO
bizx,idl-analytics-api,1be44f52f25f6b540f284eb17e8cee5838826cb9,ssheriff,3/30/2017,RPT-7954: Cross domain reports are failing with Birt Exception,RPT-7954,Defect,Internal Issue,2-High,|Report Builder|,Regression,ENT200002079CrossDomainReport,Auto
bizx,au-recruiting,d88e03469576f30079c366850fc952864815fb65,Kaderjan Ilghar,1/24/2018,fixing SML-710\, SML-714\,SML-708\,SML-709,,,,,,,,
bizx,au-V4,f54abe7a0516407a8f6e2334ce8b1a1c229dd6aa,wwang,9/19/2016,TFT- DTCE new common component change ( Change FeatureEnum type),,,,,,,,
bizx,au-V4,da5ab4deecd598e514bc3848d1dd0204643ccc0a,I848428,3/1/2017,PLT-56858: Remove the \field-permissions\ level message key for field-permissions.,PLT-56858,Defect,Internal Issue,3-Medium,|Z. DO NOT USE Permission|,New,,No
bizx,au-V4,d08b994151f39412bea199ddcf45a3aa6d45ba1a,hkumar,11/19/2014,JIRA Defect Fixes;|PAE-1335 - Groups with same name as existing in manage employee groups can be shared by another user.|Crucible Code review Link : http://crucible.successfactors.com/cru/PAE-613|Reviewers: Vikram S,,,,,,,,
bizx,au-homepage3,a82db7f431bf5083a4bd4b7885d8c0122aa80d70,I303180,9/22/2016,DESC:UI-7047 B1611HP3:Tile Name is changing automatically while creating a tour|JIRA:http://jira.successfactors.com/browse/UI-7047|REVIEW:http://crucible.successfactors.com/cru/PLT-6638,PLT-6638,Defect,,3-Medium,|Z. DO NOT USE User Mgmt - Manage Users|,New,,
  1. Logstash log in debug mode looks like parsing as expected.
> {
>               "FILE_TYPE" => "java",
>                 "GIT_ORG" => "bizx",
>           "JIRA_PRIORITY" => "2-High",
>        "JIRA_TESTCASE_ID" => "ENT200002079CrossDomainReport",
>                    "path" => "C:/elkstack/elasticsearch-6.5.1/logs/less.csv",
>                    "host" => "PVGN50859047A",
>             "COMMIT_SHA1" => "1be44f52f25f6b540f284eb17e8cee5838826cb9",
>          "COMMIT_COMMENT" => "RPT-7954: Cross domain reports are failing with Birt Exception",
>              "@timestamp" => 2019-03-07T08:48:50.231Z,
>             "FILE_METHOD" => "PermissionMetaDataGenerator.populateMissingDataInLojMap",
>             "COMMIT_DATE" => "3/30/2017",
>         "JIRA_ISSUE_TYPE" => "Defect",
>     "JIRA_ISSUE_CATEGORY" => "Internal Issue",
>           "COMMIT_AUTHOR" => "ssheriff",
>                 "message" => "bizx,idl-analytics-api,1be44f52f25f6b540f284eb17e8cee5838826cb9,ssheriff,3/30/2017,RPT-7954: Cross domain reports are failing with Birt Exception,RPT-7954,Defect,Internal Issue,2-High,|Report Builder|,Regression,ENT200002079CrossDomainReport,Auto\r",
>               "GIT_REPOS" => "idl-analytics-api",
>                "JIRA_KEY" => "RPT-7954",
>            "FILE_CHANGED" => "idl-analytics-api-service/src/main/java/com/successfactors/analytics/permission/PermissionMetaDataGenerator.java",
>                "@version" => "1",
>         "JIRA_COMPONENTS" => "|Report Builder|",
>        "JIRA_DEFECT_TYPE" => "Regression",
>       "JIRA_FIND_BY_AUTO" => "Auto"
> }
  1. But in elasticsearch, there's an exception saying that " Caused by: java.lang.IllegalArgumentException: Invalid format: "1/24/2018" is malformed at "/24/2018" "

  2. My elasticserch index template is

     {
    
      "template": "git_template",
      "index_patterns":["logstash-git"],
      "mappings": {
        "doc": {
          "properties": {
                "GIT_ORG": { "type": "text"  },
                "GIT_REPOS": { "type": "text"  },
                "COMMIT_SHA1": { "type": "text"  },
                "COMMIT_AUTHOR": { "type": "text"  },
                "COMMIT_DATE": { "type": "date","format":"MM/dd/yyyy" },
                "COMMIT_COMMENT": { "type": "text"  },
                "JIRA_KEY": { "type": "text"  },
                "JIRA_ISSUE_TYPE": { "type": "text"  },
                "JIRA_ISSUE_CATEGORY": { "type": "text"  },
                "JIRA_PRIORITY": { "type": "text"  },
                "JIRA_COMPONENTS": { "type": "text"  },
                "JIRA_DEFECT_TYPE": { "type": "text"  },
                "JIRA_TESTCASE_ID": { "type": "text"  },
                "JIRA_FIND_BY_AUTO": { "type": "text"  }
    }
        }
      }
    }

This is resolved by setting the date format compatible with build-in date formats in elasticsearch.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.