Senior Data Engineer (Remote)

- Create and maintain optimal data architectures
- Assemble & unify large, complex datasets that meet functional/non- functional business requirements.
- Build the infrastructure necessary to extract, transform, and load optimal data from a variety of data sources using SQL, AWS, and other relevant open source technologies.
- Current IOT Project: American Customer Product Project (Temperature Sensor System in Food Factories) 2- 4 years
Take over following things:
Grafana- Dashboard shows different information of the whole system
Python, node.js, Mongo, data engineering , SQL (Postgresql)
Batch data collection processes in AWS Lambda written in Python.- Scripts that aggregate data from multiple sources
Grafana/data collection/Freshdesk integration.- Integrate services to automate some processes (e.g. create Jira/Freshdesk tickets yourself when there is something wrong with the data)
Slack tool- Integrate with other services to display data via Slack

Understanding of distributed computing principles and scalable architectures.
Familiarity with data governance, data security, and compliance frameworks.
Proficiency in the latest Big Data tools/technologies like Hive, Hadoop, Yarn, Kafka, Spark Stream.
Experience with infrastructure- as- code tools like Terraform
Proficiency in data processing frameworks such as Apache Spark, Apache Hadoop, or cloud- native data processing services (Azure Data Lake, Azure Data factory, Azure Databricks, Azure Synapse, Snowflake,CosmosDB)
Experience with data integration and ETL (Extract, Transform, Load) processes, including tools like Apache Airflow or cloud- native orchestration services.
Strong experience in cloud computing platforms such as Microsoft Azure
Proven experience as a Data Engineer, Software Engineer, or similar role, with a focus on building cloud- based data platforms (5- 7 YOE+)
Knowledge of database systems (relational and NoSQL) and data modeling principles.

Salary up to 28,000,000 VND
Working time: Monday to Friday.
Flexible working hours.
Contract 3- 6 months
Youthful and dynamic environment.

Cập nhật gần nhất lúc: 2024-12-28 17:20:02