Senior Data Engineer to lead the delivery of data on data and analytics platform
Job Type: Contract
Positions to fill: 1
Start Date: May 15, 2023
Job End Date: May 20, 2024
Pay Rate: Hourly: Negotiable
Job ID: 129497
Location: Calgary, Edmonton, Halifax, London, Montreal, Ottawa, Regina, Toronto, Vancouver, Victoria, Winnipeg
Our valued client is seeking at Senior Data Engineer to lead the delivery of data on data and analytics platform
For a 1 year contract in Ottawa (100% Remote)
Responsibility:
For a 1 year contract in Ottawa (100% Remote)
Responsibility:
- Proven design, build and implementation of batch and real-time data pipelines. Driven by automated repeatable delivery of data that aligns to enterprise data governance standards.
- Experience in developing and proposing data models that conform to requirements. Responsible to ensure the proposed design, optimally addresses access and query patterns; data consumption and adheres to internal architecture standards.
- Experienced collaboration working with various stakeholders across the business, data scientists and IT. Working closely building relationships, refining data requirements to meet various data and analytics initiatives and data consumption requirements
- Increase the overall speed in which data is onboarded to the Data and Analytics platform.
- Building robust data pipelines to enable larger data consumption on the Data and Analytics Platform
- Increase the overall quality of data pipeline development through DevSecOps
- Experience in Spark using Scala, Python
- Experience working with modern data architectures like Azure Data Lake Storage, Azure, Azure Synapse and Databricks, Delta Lake, Delta Sharing, Unity catalog, DLT pipelines
- database modeling techniques: Data Vaults, Star, Snowflake, 3NF, etc.
- Experience building Rest API integration using Redis Cache, Denodo
- Experience working with streaming data architecture and technologies for real-time: Spark Streaming, Event hub, Kafka, Flink, Storm