Senior Data Engineer to design, implement and maintain Data pipelines for a national retail client - 26928
Job Type: Contract
Positions to fill: 2
Start Date: Sep 18, 2023
Job End Date: Sep 17, 2024
Pay Rate: Hourly: Negotiable
Job ID: 132472
Location: Toronto
Our national retail client is looking for a Senior Data Engineer to design, implement and maintain Data pipelines - 26928
Location: Mississauga (on site Tuesdays + Wednesdays)
Duration: 1 year
Job ID: WECJP00026928
Must Haves:
- Designing and implementing data pipelines
- Programming languages such as Python, Java, Scala, or similar languages for data processing and scripting.
- Experience with database systems (SQL and NoSQL) and data warehousing concepts.
- Familiarity with cloud platforms and services (e.g., AWS EMR, Redshift, S3; Azure Data Factory; Google BigQuery) for data processing and storage.
- Strong understanding of data modeling, data architecture, and ETL processes.
Nice to Have:
- Knowledge of data integration tools and frameworks (e.g., Apache Spark, Apache Kafka)
Responsibilities:
- Design, develop, and maintain scalable and efficient data pipelines for ingesting, processing, and transforming large volumes of data from various sources.
- Collaborate with data scientists and analysts to understand data requirements and ensure the availability of clean, accurate, and well-structured data for analysis and modeling.
- Implement data integration solutions that connect various data sources and systems, ensuring data consistency and quality.
- Work with cloud platforms and technologies (e.g., AWS, Azure, GCP) to deploy and manage data infrastructure.
- Develop and implement data governance and security measures to protect sensitive data and ensure compliance with industry regulations.
- Optimize database performance and manage data storage solutions, including both traditional relational databases and distributed data stores.