Big Data Engineer

Job Description:

• Design, develop, and maintain robust data pipelines for ingesting, processing, and transforming large volumes of data from diverse sources.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement solutions to address business needs.
• Build and optimize data models and schemas for efficient storage and retrieval of structured and unstructured data.
• Implement data governance policies and ensure compliance with data privacy regulations.
• Develop and maintain documentation for data pipelines, data flows, and data infrastructure.
• Monitor and troubleshoot data pipeline performance issues and implement optimizations as needed.

Requirement:

• Bachelor's or Master's degree in Computer Science, Information Systems, or related field.
• Proven experience as a data engineer or similar role, with a minimum of 2 years in data engineering.
• Strong experience with distributed computing frameworks such as Apache Spark, Hadoop.
• Hands-on experience with Azure Data Lake, .
• Solid understanding of data modeling concepts and best practices.
• Excellent problem-solving skills and ability to work independently as well as part of a team.
• Strong communication skills and ability to translate technical concepts to non-technical stakeholders.