Big Data Engineer (Java)

RESPONSIBILITIES:

This is a development project. As a Hadoop Big Data Engineer, you will operate and monitor scalable and resilient data platform based on Hadoop ecosystem to address the business requirements:

• Engineer reliable data pipelines for sourcing, processing, transforming, enriching and storing data in different ways, using data platform infrastructure effectively


• Ingest and transform data sets from a variety of data sources


• Focus on ingesting, storing, processing, and analyzing large datasets


• Create scalable, high-performance web services for tracking data


• Using Java (Spring Boot framework) for development tasks

REQUIREMENTS:

• Must have 4+ years of experience at a similar role


• Having hands on experience in Hadoop ecosystem (on-prem) including Spark, HDFS,


• MapReduce, YARN, …


• Good in programming language Python.


• Experience in monitoring large-scale data processing job (batch-processing, stream processing)


• Having a background of software developing on Java Spring Boot


• Experience with Hadoop distributions such as Cloudera, HortonWorks, comparison and feasibility


• Experience with Data warehouse and Data management: Data Quality, Data integration


• Experience in ETL, SQL and NoSQL Database


• Experience with SRE, Patching & Automation: Kubernetes or Docker & Containerization


• Experience working with Big Data in a Cloud environment


• Experience in Data API


• Good to have Architecture knowledge or experience

Cập nhật gần nhất lúc: 2023-03-19 10:45:19