Senior Data Engineer to Assist in designing, developing, and optimizing data pipelines, architectures, and data sets for an Insurance Client.
Contract: Contract End December 2nd, 2024 (Extensions Probable)
Environment: Hybrid 1-Day per week in Office (London or Toronto)
Responsibilities:
· Assist in designing, developing, and optimizing data pipelines, architectures, and data sets, ensuring efficient data ingestion, transformation, and reliable delivery using Microsoft Azure and Informatica/IICS to support data mesh architecture.
· Develop, automate, optimize, and fine-tune data platform provisioning, scaling, and maintenance tasks to enhance operational efficiency, performance, scalability, and cost-effectiveness.
· Ensure data security, compliance, and governance standards are consistently met across the data platform, adhering to global data engineering standards and principles.
· Creation of all infrastructure required to implement ingestion pipelines
Must-Haves:
· 6+ Years of Data and Cloud Engineering.
· Cloud-Native systems following IaC and DataOps principles (Git repo/Jenkins/terraform, CI/CD, Orchestration, Actions)
· Azure cloud following Azure Databricks, Azure Data factory and Purview
· Firewall/network rules.
· Knowledge on Syncsort.
· Protected B Enhanced Reliability Security Clearance (Required or Eligible).