Data Engineer

Job ID 17878
Job Type
Permanent
Salary
6,000,000 JPY - 9,000,000 JPY per year + Bonus
Japanese Level
None
English Level
Advanced (TOEIC 860)
Start Date
ASAP
Location
Tokyo
Job Type
Permanent
Salary
6,000,000 JPY - 9,000,000 JPY per year + Bonus
Japanese Level
None
English Level
Advanced (TOEIC 860)
Start Date
ASAP
Location
Tokyo

Description

Growing fintech company is looking for a Data Engineer to work on our payment system to deliver the best payment experience for our customers.

 

Main Responsibilities

- Work directly with Data Analysts and Platform Engineering Team to create reusable experimental and production data pipelines

- Understand, tune, and master the processing engines (like Spark, Hive, Cascading, etc) used day-to-day

- Keep the data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc)

- Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit

- Explore as many new technology options for data processing, storage, and share them with the team

- Develop tools and contribute to open source wherever possible

 

 

【会社概要 | Company Details】
Growing fintech company provides QR code and bar code based payment system. The company offers a flexible work schedule and you get your choice of tools (both hardware and software) and most importantly, the opportunity to work on tough problems with the brightest minds around you. Speaking English OR Japanese, but corporate language is the language you can program in.
QRコードとバーコードベースの支払いシステムを提供している成長中のFintech企業です。フレックス制度やツールの選択(ハードウェアとソフトウェアの両方)ができるなど、柔軟なオフィス環境です。日本語と英語が飛び交う環境で、優秀な同僚と共にチャレンジングな仕事に取り組めます。

【就業時間 | Working Hours】
10:00 - 18:45(Mon - Fri)
Super Flextime available

【休日休暇 | Holidays】
完全週休2日制(土日祝休み)、年末年始、年次有給休暇、その他特別休暇など
Saturday, Sunday, and National Holidays, Year-end and New Year Holidays, Paid Holidays, Other Special Holidays

【待遇・福利厚生 | Services / Benefits】
各種社会保険完備(厚生年金保険、健康保険、労災保険、雇用保険)、屋内原則禁煙(屋外に喫煙所あり)、通勤交通費支給等、確定拠出年金制度、各種研修制度
Social insurance, Transportation Fee, Defined contribution pension system, training programs, no smoking indoors allowed (Designated smoking area), etc.

Required Skills

Tech Stack:

- Scala, Java, Python

- PlayFramework

- Aurora, DynamoDB, TiDB, Spark, Presto, Hive, Hudi, Airflow, Kafka

- Docker, Kubernetes,AWS, GCP

- GitHub,IntelliJ

- Presto, Athena, Bigquery, Looker,Spark, Hive

- io

- Slack, Zoom, F2F

 

Qualifications:

- You have previously worked on building data pipelines ingesting and transforming large number of events per minute and terabytes of data per day

- You have worked with Spark and Kafka before and have experimented or heard about Flink/Druid/Ignite/Presto/Athena and understand when to use one over the other. Preferralbly, you have worked with cloud based big data processing platform such as AWS EMR, Google Cloud DataProc

- You are passionate about producing clean, maintainable and testable code part of real-time data pipeline

- You understand how microservices work

- You can connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues

- You understand issues with ingesting data from applications in multiple data centres across geographies, on-premise and cloud and will find a way to solve them

- Proficient in Java/Scala/Python/Spark

Preferred Skills

- Software development experience in one or more general purpose programming languages, preferably Scala

- Experience with implementing platform components such as RESTful APIs, Pub/Sub Systems, Database Clients

- Experience with AWS services

- Contribution to open source projects

- Knowledge about the Japanese payment industry