Sr Data Engineer to assist Data engineering and implement changes to all impacted pipelines and systems - 00400

Job Type: Contract
Positions to fill: 1
Start Date: Oct 24, 2022
Job End Date: Mar 31, 2023
Pay Rate: Hourly: Negotiable
Job ID: 123615
Location: Vancouver
Our public sector client needs a Senior Data Engineer to assist Data engineering and implement changes to all impacted pipelines and systems.

  • Experience and/or understanding of Agile Methodologies
  • Advanced Database Skills (Oracle/DB2 & SQL)
  • Experience with or exposure to Big Data platforms, ideally exposure to Hadoop ecosystem (HDFS, Apache Hive, Apache Spark, SparkSQL)
  • Extensive programming experience in either Java, C++ or preferably Scala,IntelliJ, Jenkins
  • Working experience with Apache Spark Data Processing Framework, Parquet data format
  • Experience working with Jira
Duties will include:
  • Creating mapping documentation of data elements from source to target and liaising with other Data Engineers, Data Architects and SMEs.
  • Developing & testing Data Transformation pipelines by leveraging the latest Big Data tools and technologies
  • Collaborate with upstream data teams, develop data validation and test plans,
  • performing user acceptance testing, and provide feedback to development and sustainment teams
  • Conducting analysis for moderate to complex data requests, defining data fields and
  • determining data availability, developing information layout, format and interactivity. Presenting findings and providing clarification.
  • Act as a lead developer to implement code changes in various environments and platforms leveraging our DevOps & DataOps
  • Work closely with the testing team to assist with issues raised in user acceptance testing (UAT) and Business Value Testing (BVT)
  • Fully implement the new data format across all IM supported platforms including Big Data (Hadoop), Enterprise Data Warehouse (EDW) and Operational Data Store (ODS)