Assignment Description

Our client in the automotive industry is currently seeking a Data Engineer.

At their Software factory team, they are building a community with highly skilled personnel. They are harmonizing the workflows and developing tools to be used across R&D, with the aim to
accelerate the software development at our client.

Team Responsibilities:

• Develop continuous integration (CI) architecture and infrastructure.
• Maintain the complete software build and CI flow.
• Enable high utilization of infrastructure, test objects, and test artifacts.
• Create automated tools and applications to secure an efficient flow.
• Support traceability to fulfill all legal requirements.
• Apply manufacturing principles and techniques to software development to mimic the benefits of traditional manufacturing and create a software product line.

Main Responsibilities:

• Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
• Optimize data pipelines for peak performance and efficiency, encompassing all aspects from extraction to loading (ETL / ELT).
• Collaborate closely with data scientists, analysts, and stakeholders to decipher data requirements and deliver innovative solutions.
• Stay abreast of emerging technologies and best practices to drive continuous improvement.
• Integrate, consolidate, and cleanse data from diverse sources to ensure accuracy, completeness, and reliability.
• Initiate projects that help the organization make data-driven decisions.

Required Qualifications:

• MSc in Computer Science, Software Engineering, or similar relevant experience.
• Proficiency in the Python programming language.
• Strong understanding of SQL for data manipulation and querying.
• Experience with data orchestrators like Dagster, Airflow, etc.
• Familiarity with database management systems like Snowflake, Azure Data Lake, etc.
• Expertise in data modeling and transformation using tools like dbt.
• Ability to optimize data pipelines for performance and scalability.
• Excellent problem-solving skills and meticulous attention to detail in data management processes.
• Strong collaboration and communication skills to work effectively with data scientists, analysts, and stakeholders.
• Proactive approach to staying updated with emerging technologies and best practices in data engineering.
• Experience working with cloud platforms such as Azure.
• Experience working in an agile environment with methodologies like Scrum, Kanban, SAFe, etc.

Meritorious:

• Basic knowledge and understanding of CI/CD.
• Knowledge of Kubernetes (k8s) for container orchestration and deployment.
• Experience with AI (Artificial Intelligence) and ML (Machine Learning).

Desired Attributes:

• Structured way of working.
• Positive attitude.
• Self-driven, result-oriented, and flexible.
• Team player with good communication skills.
• Creative mindset.