Sr. Data Engineer for data modeling, structured streaming, quality control and monitoring in Azure
Job Type: Contract
Positions to fill: 1
Start Date: Jun 12, 2023
Job End Date: Dec 12, 2023
Pay Rate: Hourly: Negotiable
Job ID: 126760
Location: Calgary
Our large oil and gas client is seeking a Sr. Data Engineer for data modeling, structured streaming, quality control and monitoring in Azure for 6 months contract with opportunity to extend.
Must Have:
Must Have:
- 5+ years hands-on Data Engineering experience
- 3+ years experience with Azure Databricks, Data Factory, and Data Lakes
- Thorough understanding of data modeling, data lakehouse, structured streaming, quality control and monitoring.
- Experience testing CI/CD pipelines and branching strategies for environmental readiness
- Experience troubleshooting resources, pipelines and deployment within a data platform (ideally Azure)
- Experience with DeltaLoader, JobManager, or pyspark