Job Description

Responsibilities

Design, build, and optimize scalable data pipelines using ETL (Extract, Transform, Load) processes to facilitate efficient data flow across systems. Develop and maintain data warehouses and data lakes utilizing cloud platforms such as AWS and Azure Data Lake to support large-scale analytics. Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions leveraging technologies like Hadoop, Spark, Apache Hive, and Informatica. Implement database schemas and optimize database design using Microsoft SQL Server, Oracle, and other relational databases to ensure high performance and reliability. Integrate diverse data sources including linked data, RESTful APIs, and big data repositories to create comprehensive datasets for analysis. Write complex SQL queries, Python scripts, Bash shell scripts, and VBA macros to automate workflows and enhance data processing efficiency. Support model training activities by preparing datasets and ensuring data quality for machine learning projects. Utilize analytics tools such as Looker for visualization and reporting to deliver insights that drive business strategies. Participate actively in Agile development cycles, contributing to sprint planning, stand-ups, and continuous improvement efforts. Maintain comprehensive documentation of data architecture designs, workflows, and procedures for transparency and knowledge sharing.

Qualifications

Proven experience as a Data Engineer or in a similar role with a strong understanding of database design principles and data warehousing concepts. Hands-on expertise with cloud platforms such as AWS (Amazon Web Services) and Azure Data Lake for scalable storage solutions. Proficiency in programming languages including Java, Python, Bash (Unix shell), Shell Scripting, and VBA for automation tasks. Deep knowledge of big data technologies like Hadoop ecosystem components (HDFS), Spark, Apache Hive, along with ETL tools such as Talend or Informatica. Strong understanding of SQL fundamentals with experience working on Microsoft SQL Server, Oracle databases, or similar relational systems. Familiarity with Linked Data concepts for integrating semantic web resources into enterprise datasets. Experience working with RESTful APIs for seamless data integration from external sources. Knowledge of analytics tools such as Looker or similar BI platforms for creating dashboards and reports. Ability to analyze complex datasets effectively while demonstrating excellent problem-solving skills. Experience with model training processes in machine learning projects is a plus. Excellent communication skills with the ability to collaborate effectively within Agile teams. Join us in transforming raw information into powerful insights that shape the future! This paid position offers an exciting opportunity to work at the forefront of technology innovation while developing your skills in a vibrant team environment dedicated to excellence in data management and analytics!
Job Type: Full-time

Pay: $67,908.94-$144,786.34 per year