JOB DESCRIPTION
Design and deliver the next- gen data lifecycle management suite of tools/frameworks, including ingestion and consumption on the top of the data lake to support real- time as well as batch use cases
Help the team in integrating various data sources across GFGs group vertical
Build and expose metadata catalog for the Data Lake for easy exploration, profiling as well as lineage requirements
Build and manage the data asset using some of the most scalable and resilient open source big data technologies like Airflow, Spark, Kafka, etc
Build and manage a highly scalable, efficient Data and ML Infrastructure by adopting microservices driven design and architecture with proper DevOps principles and practices
JOB REQUIREMENTAble to drive devops best practices like CI/CD, containerization, blue- green deployments, secrets management etc in the Data ecosystem.
Able to maintain and monitor the ecosystem with high availability.
At least 3+ years of relevant experience in developing scalable, secured, fault tolerant, resilient & mission- critical Big Data platform.
Strong understanding of big data and related technologies like HDFS, Spark, Presto, Airflow, Kafka, Apache Atlas etc.
Good knowledge of Complex Event Processing systems like Spark Streaming, Kafka, Apache Flink, Beam etc.
Must have sound understanding for all Big Data components & Administration Fundamentals. Hands- on in building a complete data platform using various open source technologies.
Must have good fundamental hands- on knowledge of Linux and building big data stack on top of AWS/GCP using Kubernetes.
Able to develop an observability and monitoring ecosystem for all the components in use in the data ecosystem.
Proficiency in at least one of the programming languages Java, Scala, Python or Go.
Able to develop an agile platform with auto scale capability up & down as well vertically and horizontally.
BENEFIT
Home Workspace support: up to 7mil/person.
Learning budget up to EUR 500/person.
Sick leave/Mental Health Days: 30 days.
Macbook is provided.
Support Gym membership.
Annual Leave: 15 days
Support for English class.
13th month salary.
Additionally, we offer a flat hierarchy, a fun and flexible work environment.
Medical benefits: Social insurance, medical insurance & AON insurance.
Annual bonus: up to 3 months based salary (depends on company performance).
Occasion leave: 1 day.
