New Features and Bug fixes in Version1.3
Streaming Pipeline:
- FTP as Target: Validated FTP target with different file formats (CSV, PARQUET)
- AWS S3 Target: Validated AWS S3 target with different file formats (CSV, PARQUET)
- FTP as Source: Validated FTP source with different file formats (CSV, PARQUET)
- Azure Blob Target: Validated Azure Blob target with diff file formats (CSV, PARQUET)
- Delta lake Source and Target are implemented
- Kafka Target: Integration of Kafka is enabled target is done
- External Kafka: Validation of Kafka as Source and Target is done
- Spark Streaming job
- Snowflake RDBMS Source validation
- Redshift RDBMS Source validation
Bug Fixes:
- HIVE Integration issue is resolved
- FTP Source and Target UI changes are done
- The Spark Streaming job is running in local mode but not running on the Spark Master issue
BDP:
- Data & Session retention is implemented
Batch Pipeline:
- AWS S3 Source & Target: Validated AWS S3 with different file formats (CSV, PARQUET) using Single and Multi file load
- FTP as Source & Target is completed
- Azure Blob Source and Target with different file formats (CSV, PARQUET) is completed
- Data and Session retention is implemented
Analytics:
- The analytics module is stabilized
Projects: Bug fixes:
- Select a project to copy assets to the screen displays 15 configured connections for every project the number should be project-specific
- For the scheduled job it should display a Pause option
- Data & Session retention
- Dataset download
- When we go back to the project from the dataset details screen it should display the datasets screen instead of the project dashboard
- We can include a refresh option in the manage domain screen
- Project dashboard->datasets->click on preview ->in preview window download option is not working
- Name and Tag searches are not working for the job in the project dashboard
- Project stepper final screen alignment issue