Etl Lead Engineer / Principle Engineer
Vancouver, BC, Canada
Job Description
Job Title: ETL Lead Engineer / Principle Engineer
Location: Remote (anywhere in Canada) but must work on PST time zone
Required Skills: Need a Cloud data engineer with strong experise in SQL, ETL and AWS Python PySPARK
Experience 10to12Yrs
Tech Stack : Pyspark, AWS, SQL, Airflow, Kafka Streams etc
Lead Engineer should have at least 10 years of hands on experience
- Design and implement distributed data processing pipelines using Spark, Hive, Python, and other tools and languages prevalent in the Hadoop ecosystem. Ability to design and implement end to end solution.
- Experience publishing RESTful API\'s to enable real-time data consumption using OpenAPI specifications
- Experience with open source NOSQL technologies such as HBase, DynamoDB, Cassandra
- Familiar with Distributed Stream Processing frameworks for Fast & Big Data like ApacheSpark, Flink, Kafka stream
- Build utilities, user defined functions, and frameworks to better enable data flow patterns.
- Work with architecture/engineering leads and other teams to ensure quality solutions are implements, and engineering best practices are defined and adhered
Associate need to perform data analysis, collaborating with database developers and enhance data collection and storage procedures, and prepare data analysis reports.
Highly capable data quality analyst to develop procedures to enhance the accuracy and integrity of organizations data.
The data quality engineer should exhibit extensive knowledge of data engineering, analysis techniques and experience
The data quality engineer should have knowledge in Data Architecture
The data quality engineer should have good knowledge in API Testing with good knowledge on AWS
Additional desired skills which are good to have is knowldge on Python
Beware of fraud agents! do not pay money to get a job
MNCJobz.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.
Related Jobs
Job Detail
-
Job IdJD2150695
-
IndustryNot mentioned
-
Total Positions1
-
Job Type:Full Time
-
Salary:Not mentioned
-
Employment StatusPermanent
-
Job LocationVancouver, BC, Canada
-
EducationNot mentioned