Intermediate Python Data Engineer to support the automation of data engineering pipelines in python and GCP

Our client is a leading Canadian retailer. They are looking for an Intermediate Python Data Engineer to support the automation of data engineering pipelines in python and GCP

Project:

Build reusable Python AirFlow components for Data Scientist to use, as well as rewriting and automating existing ETL work in Python. Also, they will deploy and monitor Machine learning models and APIs in GCP. 

Code in Python, CICD in GitLab for GCP, provisioning resources in GCP such as Composer/Airflow, VertexAI, BigQuery, Storage, Spark Clusters, etc.

Must Haves:

• 5+ years of experience in Python, with previous experience in another OOP languages
• Designing, implementing, and monitoring data Engineering ETL cloud pipelines
• Expert in Python AirFlow, Composer, VertexAI, and Big Data tools like PySpark
• Design and build scalable cloud solutions and APIs in GCP
• CICD to automate and maintain GCP data pipelines