Data Engineer (P2/P3) (Python / SQL / Data Modelling on AWS)
Description
One of the world's largest insurance companies is seeking a a talented Data Engineer with strong skills in Python / SQL / and Data Modelling on AWS Cloud. This position is in Data Engineering department, supporting other business units.
Responsibilities
- Assist DEs with DL enhancements, including improvements in existing processes and new functionality
- Engage actively in Backlog review meetings, team discussions and discussions with users, and assist in reducing technical debt to improve existing code and to improve our development processes, with an eye towards greater Agile adoption
- Become a particular data domain expert in order to best serve business teams and manage data provisioning and optimize ETL/ETL processes when fulfilling data pipeline or data extraction/transformation/delivery requests for diverse stakeholders (data team, biz teams, other teams)
- Work and coordinate across teams and departments to accomplish tasks and overcome challenges
- Actively self-educate and work towards becoming a full member of the team, able to work with minimal direct oversight, independently and in collaboration with other DEs/SAs
- Assist other projects to the full extent of their abilities, as requested and needed
【Company Details】
Our client is one of the world's largest insurance - financial groups, trusted by over 50 million customers. The company provides various financial protections including general insurance, life insurance, retirement funds, and inheritance throughout the lifetime for individual customers, small businesses, and large companies.
【Working Hours】
9:00-18:00 (Mon-Fri)
Hybrid Remote (Office & WFH)
【Holidays】
Saturday, Sunday, and National Holidays, Year-end and New Year Holidays, Paid Holidays, Other Special Holidays
【Services / Benefits】
Social insurance, Transportation Fee, No smoking indoors allowed (Designated smoking area), etc.
Required Skills
- Practical experience working on data pipelines or analytics projects with Python and SQL
- Strong knowledge and practical experience working with at least 3 of the following AWS services: S3, EMR, ECS/EC2, Lambda, Glue, Athena, Kinesis/Spark Streaming, Step Functions, CloudWatch, DynamoDB, Aurora
- Strong experience working with data processing and ETL systems (Spark, Airflow, and/or managed cloud solutions like AWS Glue, Databricks Workflows, Cloud Composer)
- Ability to work with large volumes of both raw and processed data in a variety of formats (JSON, CSV, Parquet, ORC, etc.).
- Experience with CICD tools and services and product development, testing, and production release cycles
- Experience with DevOps tools (Jenkins, GitHub, Ansible, Docker, Kubernetes)