Data Engineer

Role Description:
Data engineers also need to understand how to optimize data retrieval and how to develop dashboards, reports and other visualizations for stakeholders.
Data engineers are often responsible for building approaches, pipelines and algorithms to help give easier access to raw data, but to do this, they need to understand company’s or client’s objectives.
Data engineers are mainly tasked with transforming data into a format that can be easily analyzed. They do this by developing, maintaining, and testing infrastructures for data generation. Data engineers work closely with data analysts and are largely in charge of technical solutions to enable more efficient and timeline data analysis
Reporting Line: Team lead
Responsibilities
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Deploy sophisticated analytics programs, machine learning and statistical methods
- Develop, construct, test and maintain architectures.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and ‘big data’ technologies.
- Align architecture with business requirements.
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Conduct research for industry and business questions.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
- Prepare data for predictive and prescriptive modeling.
- Data collection: Collecting, measuring and analyzing.

Key skills:
Know SQL, ER model, normalization, ACID Transactions.
Experience and deep understanding with relational SQL, NoSQL, Document data, Wide Column, Graph and Key value
- Relational SQL : MySQL, PostgreSQL, MSSQL, Oracle...
- Non-Relational:
Document: MongoDB, Elasticsearch..
Key-Value: Redis, Memcached, Amazon DynamoDB..
Wide Column: Hbase, Cassandra, GG Bigtable..
Data Warehouse: Experience with concepts data modeling like snowflake, Golden Gate, Google BigQuery, Dremio, Presto..
Know most modern data processing frameworks like Arrow, Spark, Flink, Kafka... beyond traditional data processing and workflow scheduling: NIFI, Airflow, ODI, SSIS...
Know most monitoring: Prometheus, Grafana, Splunk ...
Nice to have:
- Visualize data.
- Machine learning OPS: TensorFlow Extended, Kubeflow, Seldon, Mlflow, GG AI Platform.
- Infrastructure as Code: Container, Infa provisioning
- Machine learning: Terminology, TensorFlow, Pytorch...
- Python and/or Java programming
- CICD

13th salary, rewards for achievements, initiatives and good deeds

Annual leave: 15 – 20 working days/year and Other leaves/public holidays

Providing customized training courses according to business needs and upon your request

Cập nhật gần nhất lúc: 2021-02-03 15:00:01