A global insurance company is searching for a talented data engineer within the company. Data engineers working in the datalake team carry out a wide variety of business intelligence tasks in a largely AWS based cloud computing environment.
Key Accountabilities & Specific Activities -Designing & Modeling (with minimal supervision)
- Build the first iterations of high quality and sustainable data pipelines and ETL processes to extract data from a variety of APIs and relational databases and ingest into AWS services.
- Efficiently draft complex SQL queries and data models to aggregate and transform data for reporting and analytics teams.
- Execution and Maintenance (with minimal supervision) ・Monitor existing solutions and work proactively to rapidly resolve errors and identify future problems before they occur. ・Own and keep the system design and operations documents up to date.
- Consult with a variety of stakeholders to gather new project requirements and transform these into well-defined tasks and targets.
- Relations with other departments Mainly IT teams (Enterprise Data, Data Strategy, Architecture), Finance, Marketing etc.
【会社概要 | Company Info】
Global general insurance group founded in Europe over 200 years ago. Operating in over 62 countries, with a 20+ year presence in Japan.
【就業時間 | Working Hours】
9：00 - 17：30（Mon - Fri）
Saturdays, Sundays, Year-end Holidays, Paid Holiday, Condolence Leave, etc.
Bonus (2x a year), Incentive based on performance, Social Insurance, Transportation Fee, Retirement Fund System, Savings System, Stock Options, No smoking indoors allowed (Designated smoking area), etc.
- Practical experience in data / analytics with experience working in an engineering / BI role.
- Practical experience working on data pipelines or analytics projects with Python and SQL.
- Experience with NoSQL databases (DynamoDB, MongoDB, Elasticsearch, Redis, Neo4j, etc.)
- Strong knowledge and practical experience working with at least 3 of the following AWS services: S3, EMR, ECS/EC2, Lambda, Glue, Athena, Kinesis/Spark Streaming, Step Functions, CloudWatch, DynamoDB.
- Strong experience working with data processing and ETL systems such as Oozie, Airflow, Azkaban, Luigi, SSIS.
- Experience developing solutions inside a Hadoop stack using tools such as Hive, Spark, Storm, Kafka, Ambari, Hue, etc. -Ability to work with large volumes of both raw and processed data in a variety of formats (JSON, CSV, Parquet, ORC, etc.).
- Ability to work in a Linux/Unix environment (predominantly via EMR & AWS CLI / HDFS).
- Experience with DevOps tools (Jenkins, GitHub, Ansible, Docker, Kubernetes).