Data Engineer

课程简介

数据工程师目前的需求量巨大, 甚至企业的对数据工程师的需求量远大于数据科学分析师。 “figures from job posting websites show much higher demand for data engineers than for data scientists.” Faced with a tsunami of big data, companies are eager for experts who can “ensure that data pipelines are scalable, repeatable, and secure, and can serve multiple constituents in the enterprise.” 基于这样的工作需求的缺口, 我们特设了这个4~5月零基础强化培训, 把一个新手打造成一位合格的数据工程师。

课程大纲

(Total hours: Est. 80 H)

 

更多信息

(Big)Data Engineer(更偏计算机应用)是一个最近比较流行的职位, 其有部分类似于但又不完全类同于Data Scientist (更偏统计模型)也不同于以往BI Engineer/developer。

一般市场对Data Engineer的要求是,

1, Good work experience in using SQL and databases in a business environment.

熟知数据库技术

2, Strong experience in Data Warehouse and Business Intelligence application development

深刻理解数据仓库/商务智能应用开发

3, Experience with scripting languages, i.e. Perl, Python etc. preferred

掌握一门语言, 一般推荐学习Python (一般Java/Python二选一, Python学习更快)

4, Evaluate and implement various big-data technologies and solutions (Hive/EMR, Tez, Spark) to optimize processing of extremely large datasets in an accurate and timely fashion.

大数据的知识(Spark, HIVE, PIG…)

5, Other soft skills

简单归纳, 从以上要求来看, Data Engineer其实就是一个传统的数据库/BI开发人员+Python+Big Data