# Big Data Engineer (Advertising Business)

Job Type
7,000,000 JPY - 10,000,000 JPY per year
Japanese Level
English Level
Advanced (TOEIC 860)
Start Date


One of the largest Internet service companies in the world is hiring a Big data Engineer for their advertising business.

【Position Description】
The department is in charge of development and operation of advertising platforms in the Group. They provide products and platforms for their advertising businesses, as well as providing technology solutions for the Group's marketing activities.

They are now building scalable x stable data pipeline to consolidate various ad products' Tracking and Reporting system. You will be in charge of Technical Lead for this system, and will have a key role in feature planning, design, development, delivery and operation based on your strong Big Data knowledge / experience and Technical leadership.

You will manage unprecedented Big Data in this industry. Hence, they are looking forward to working with persons who can enjoy:
‐ Solve large-scale data processing by technology (ideas)
‐ Using modern technology to tackle ad business, issues, etc.

【Main Responsibilities】
‐ Build and maintain large-scale batch and real-time data pipelines with data processing frameworks and technologies like Spark, Hive, Flink, Spark-Streaming, Hadoop, etc.
‐ Lead and coordinate the engineering teams (or Project teams) to ensure the right architecture decisions are made, and the right development and operation processes are taken
‐ Responsible for technical decision of implementation (architecture and development process), and also the production system quality (operation and maintenance)
‐ Lead technology decisions to correctly place solutions for both functional and non-functional requirements, and to make the right decisions at the right timing for all the scaling, security, maintenance ability, etc.
‐ Provide estimates, risk info, and recommendations to PMs for creating the system roadmap and to support prioritization conversations

【Technology Stack】
‐ Language: Java, Python, HiveQL, Go, Bash, Perl
‐ Data Collectors: Fluentd, Kafka Producer API, Pub/Sub SDK
‐ Data Processing: HIVE, Spark, Kafka, Flink, Data Flow, Map Reduce, Pub/Sub
‐ Data Lake: HDFS, GCS, Big Query
‐ OLAP: CouchBase, AeroSpike, Big Query, etc.

【会社概要 | Company Details】

Our client is a large global Internet service company that has enjoyed sustained growth as they continue to expand their business in various new areas and industries. This is a great opportunity to work in a diverse and international environment in Japan. Our client actively strives to be an equal opportunity employer, and they have many female and foreign nationals in upper management positions. Their brand has also gained global recognition as they sponsor some of the world's most famous sports teams. Our client prides themselves in providing a comfortable working environment for their employees. Engineers are welcome to choose their own setup (Windows/Mac, etc.); whatever makes them comfortable! Free meals are also provided at the company cafeteria. Their chefs work to create exciting new menus and dishes, so employees never get tired of the food!
【就業時間 | Working Hours】
9:00 - 17:30(Mon - Fri)
【休日休暇 | Holidays】
Saturday, Sunday, and National Holidays, Year-end and New Year Holidays, Paid Holidays, Other Special Holidays
【待遇・福利厚生 | Services / Benefits】

各種社会保険完備(厚生年金保険、健康保険、労災保険、雇用保険)、 屋内原則禁煙(屋外に喫煙所あり)、 通勤交通費支給等

Social insurance, Transportation Fee, No smoking indoors allowed (Designated smoking area), etc.

Required Skills

‐ Experience in large-scale data processing experience using Big Data tools such as Hadoop, Spark, Kafka, Hive, etc.
‐ Experience in making both functional and non-functional requirements, design, building and operating Data pipelines
‐ Software development experience using the following languages (Java, Python, Go, Bash, Perl, etc.)
‐ Software development experience (and consistent development experience in requirements definition, specification formulation, implementation, testing, release, and maintenance operation)
‐ Experience in database design and performance tuning

Preferred Skills

‐ Experience in operation for online advertising (Ad product)
‐ Experience in middleware selection and integration
‐ Experience in scrum development