Merge pull request #24 from milsi96/main

Add RetentionTime field

Author: agile-lab

README.md

@@ -73,6 +73,7 @@ Constraints:
 * `OutputPortType: [String]` the kind of output port: Files, SQL, Events, etc. This should be extensible with other values, like GraphQL or others.
 * `CreationDate: [Optional[String]]` when this output port has been created.
 * `StartDate: [Optional[String]]` the first business date present in the dataset, leave it empty for events, or we can use some standard semantic like: "-7D, -1Y".
+* `RetentionTime: [Optional[String]]` historical depth of data available in the Output Port
 * `ProcessDescription: [Option[String]]` what is the underlying process that contributes to generate the data exposed by this output port.
 * `DataContract: [Yaml]`: In case something is going to change in this section, it represents a breaking change because the producer is breaking the contract, this will require to create a new version of the data product to keep backward compatibility
   * `Schema: [Array[Yaml]]` when it comes to describe a schema we propose to leverage [OpenMetadata specification](https://docs.open-metadata.org/v1.0.0/main-concepts/metadata-standard/schemas/entity/data/table#definitions). Each column can have a tag array, and you can choose between simples LabelTags, ClassificationTags or DescriptiveTags. Here an example of classification Tag https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/resources/json/data/tags/piiTags.json.

data-product-specification.cue

@@ -89,6 +89,7 @@ import "strings"
   outputPortType:       string
   creationDate?:        string | null
   startDate?:           string | null
+  retentionTime?:       string | null
   processDescription?:  string | null
   dataContract:         #DataContract
   dataSharingAgreement: #DataSharingAgreement

example.yaml

@@ -77,6 +77,7 @@ components:
     outputPortType: SQL
     creationDate: 05-04-2022T17:00:00.000Z
     startDate:
+    retentionTime:
     processDescription:
     dataContract:
       schema: