Data Engineer (Data Lake)

Job Type
8,000,000 JPY - 9,000,000 JPY per year
Japanese Level
English Level
Advanced (TOEIC 860)
Start Date


A global insurance company is searching for a talented data engineer within the company. Data engineers working in the datalake team carry out a wide variety of business intelligence tasks in a largely AWS based cloud computing environment.


Key Accountabilities & Specific Activities -Designing & Modeling (with minimal supervision)

- Build the first iterations of high quality and sustainable data pipelines and ETL processes to extract data from a variety of APIs and relational databases and ingest into AWS services.

- Efficiently draft complex SQL queries and data models to aggregate and transform data for reporting and analytics teams.

- Execution and Maintenance (with minimal supervision) ・Monitor existing solutions and work proactively to rapidly resolve errors and identify future problems before they occur. ・Own and keep the system design and operations documents up to date.

- Consult with a variety of stakeholders to gather new project requirements and transform these into well-defined tasks and targets.

- Relations with other departments  Mainly IT teams (Enterprise Data, Data Strategy, Architecture), Finance, Marketing etc.


【会社概要 | Company Info】
Global general insurance group founded in Europe over 200 years ago. Operating in over 62 countries, with a 20+ year presence in Japan.

【就業時間 | Working Hours】
9:00 - 17:30(Mon - Fri)

Saturdays, Sundays, Year-end Holidays, Paid Holiday, Condolence Leave, etc.

給与評価年1回、賞与年2回、業績賞与有り、社会保険完備、屋内原則禁煙(屋外に喫煙所あり)、退職金制度、財形貯蓄制度、持株制度 など
Bonus (2x a year), Incentive based on performance, Social Insurance, Transportation Fee, Retirement Fund System, Savings System, Stock Options, No smoking indoors allowed (Designated smoking area), etc.

Required Skills

- Practical experience in data / analytics with experience working in an engineering / BI role.

- Practical experience working on data pipelines or analytics projects with Python and SQL.


Preferred Skills

- Experience with NoSQL databases (DynamoDB, MongoDB, Elasticsearch, Redis, Neo4j, etc.)

- Strong knowledge and practical experience working with at least 3 of the following AWS services: S3, EMR, ECS/EC2, Lambda, Glue, Athena, Kinesis/Spark Streaming, Step Functions, CloudWatch, DynamoDB.

- Strong experience working with data processing and ETL systems such as Oozie, Airflow, Azkaban, Luigi, SSIS.

- Experience developing solutions inside a Hadoop stack using tools such as Hive, Spark, Storm, Kafka, Ambari, Hue, etc. -Ability to work with large volumes of both raw and processed data in a variety of formats (JSON, CSV, Parquet, ORC, etc.).

- Ability to work in a Linux/Unix environment (predominantly via EMR & AWS CLI / HDFS).

- Experience with DevOps tools (Jenkins, GitHub, Ansible, Docker, Kubernetes).