ETL Tool Implementation

Our Extract Transform and Load (ETL) tool implemented for use in the Enterprise Data Warehouse

This project has delivered a key component of the infrastructure to support the new Enterprise Data Warehouse (EDW).  Following a successful procurement exercise, the Talend Data Management Platform has now been implemented in a Hybrid cloud architecture.  This Extract Transform and Load (ETL) tool will allow us to create and maintain robust live technical processes to extract data from our operational systems; to validate and transform the data; and then load information into the EDW (Enterprise Data Warehouse). The tool also provides facilities for monitoring these processes and correcting problems that arise.

This technically orientated project has resulted in the introduction of new technology into the University and has delivered:

• The infrastructure to support Talend Hybrid Cloud solution across all three EDW environments (Development, Test and Live), which combines infrastructure locally hosted and cloud based
• Training for fifteen members of IS Applications to provide ongoing development and support for its usage
• A working exemplar to manage the Organisational Hierarchy
• The ability to schedule and monitor multiple data transfers simultaneously
• Technical documentation associated with
  •  the Technical Design
  • a Developers Support guide outlining best practices

Although our use of the Talend Data Management Platform is still in its infancy, it is anticipated that its usage and significance will grow as the EDW matures to provide the data and information needed by decision makers across the University, giving a consistent view of data across multiple sources, adhering to consistent data definitions.