RightData’s DataFactory comes with this important feature of asset approval process to support Continuous Integration and Continuous Delivery.
What does it mean and what is the value?
Every asset that has been developed within Data Factory will have to go thru proper approval process before it gets published and to get collaborated with other people in the team whoever has access to the project. This brings in proper governance aspect into the data engineering effort.
Who are the Personas?
Designer (Who composes the assets, for example… Data pipelines and JOBs to automate the data movement.
Reviewer approves or rejects the assets that are submitted for review.
The Process
- Designer submits the assets for the review upon successful completion of the development.
- The execution of the pipeline got to be successful before submitting for the review.
- Once submitted for review, the asset appears in the project's dashboard of the reviewer. Reviewer accepts the review and add comments and rejects or approves the asset.
- If the asset is rejected, the designer has to resolve all the comments by addressing the suggestions and resubmits for review.
Status of the assets during CICD cycle
| Persona | Activity | Status of the asset | Pipeline Mode |
| Designer | Development of the asset | Draft | Edit Mode <-> Monitor Mode |
| Designer | Submits for the review | In Queue | Display Mode <-> Monitor Mode |
| Reviewer | Accepts for review | In Review | Display Mode <-> Monitor Mode |
| Reviewer | Rejects with comments | Draft | |
| Reviewer | Approves the asset | Published | Published Mode <-> Monitor Mode |
| Designer | To make changes to an already published asset, designer has to “check out” the asset | Checked-out |
Edit Mode <-> Monitor Mode |