As a Data Engineering Intern, you will play a crucial role in refining and updating our current Azure data pipeline and CI/CD pipeline to enhance real-time data flow. This position offers a unique opportunity to work with advanced cloud technologies and contribute to meaningful projects.

Key Responsibilities:

·       Collaborate with the data engineering team to refine and optimize Azure data pipelines for real-time data processing.

·       Assist in the development and maintenance of CI/CD pipelines, ensuring efficient deployment of data models and applications.

·       Contribute to the design and implementation of scalable and robust data solutions.

·       Participate in code reviews and adhere to best practices in data engineering.

·       Engage in troubleshooting and resolving technical issues related to data pipelines and workflows.

·       Document technical processes and contribute to the continuous improvement of our data engineering practices.

Qualifications:

·       Preferred enrollment in a Master's or PhD program in Computer Science, Data Science, Engineering, or a related field, but candidates with a Bachelor’s degree and relevant experience are also encouraged to apply.

·       Demonstrated experience in data engineering, preferably with Azure cloud services.

·       Familiarity with CI/CD tools (Azure DevOps and GitLab).

·       Previous internship experience in a similar role is highly desirable.

·       Strong programming skills in Python, SQL, or other relevant languages.

·       Ability to work in a fast-paced, team-oriented environment.

·       Excellent problem-solving and analytical skills.

·       Effective communication and interpersonal abilities.

Project Deliverables:

1. Refined and Optimized Azure Data Pipelines:

• Documentation detailing refinements made to Azure data pipelines, highlighting optimizations for real-time data processing.
• Improved pipeline efficiency metrics or performance reports.

1. CI/CD Pipeline Development and Maintenance:

• Codebase demonstrating contributions to CI/CD pipeline development.
• Maintenance records showcasing efficient deployment of data models and applications.

1. Design and Implementation of Scalable Data Solutions:

• Design documents or diagrams illustrating contributions to scalable and robust data solutions.
• Implemented solutions demonstrating scalability and robustness.

1. Code Reviews and Best Practices Adherence:

• Reports or summaries from code review participation, indicating adherence to best practices in data engineering.
• Contributions to establishing or enhancing best practices documentation.

1. Troubleshooting and Issue Resolution:

• Records or reports outlining troubleshooting processes and resolutions for technical issues related to data pipelines and workflows.

1. Technical Process Documentation:

• Documented technical processes related to data engineering, contributing to continuous improvement efforts.
• Updated or new documentation reflecting improved data engineering practices.