Systematic Learning Matters (for Engineers)
During the first 20 years of my life, I was a passive learner. Schools and college had been my major knowledge source: Every semester, faculty members provided a syllabus with studying plans for each week, and all I did is strictly following the study guide without questioning. I went to all lectures and labs and completed all homework on time in order to eventually have those nice A’s on my transcripts. I thought I was a excellent student.
I never doubted my learning methods and habits until my first internship. I was assigned to develop major performance and scale reports using Microsoft SSRS. Back then, SQL was a completely new technology to me, so I started “learning” SQL by Google and Stack Overflow. Luckily, I got all of my work done, so I thought I was proficient at SQL. Since I really enjoyed SQL development, I took a course about relational database management system (RDBMS) in college, thanks to which I realized I have been writing sub-optimal SQL code, for example:
- I used
GROUP BYbecause I was familiar with looping concept but I didn’t know RDBMS is based on set theory.
- I used to filter rows using whichever column that can work as I didn’t know how indexes or keys work, which could have improved the run-time of some queries by more than 1,000 times.
After the course ended, I volunteered to rewrite working queries to improve performance. Most importantly, I realized how vital it is to acquire solid foundational knowledge when learning a new skill. Therefore, I started reflecting on my learning methods and finally settled down with a systematic approach.
How to Systematically Learn
To learn a new Technology X, such as a programming language or a framework, systematically, your core learning objectives are being able to answer three questions:
- What X is [foundation]
- Why/when (not) to use X [foundation]
- How to use X [hands-on]
For example, to apply the systematic approach on Hive, you should:
- (to answer “What Hive is”) learn about the definition of Hive, high-level architecture of Hive, motivation behind creating Hive and also technologies that Hive is closely related to (e.g., Hadoop and NoSQL concepts).
- (to answer “Why/when to use X”) learn about the pros and cons of Hive compared to alternative technologies (e.g., RDBMS, Druid), and use cases of Hive.
- (to answer “How to use Hive”) set up and configure Hive, write HQLs, tune HQLs and practice with real-world data.
Benefits of Systematic Learning
Solid foundational knowledge: Completing objectives No. 1 and 2 will help you build strong basics, which means deeper and better understanding of the technology.
High-performance code: A systematic learning approach starts with building a solid foundational knowledge, so you will more likely write high-performance code by making connections between concepts and implementations.
Identify optimal solution: The ultimate goal of learning technologies is to solve technical problems. Systematic learning can help you identify the best technology to use because you know why and when (not) to use those in your skillset.
Beyond-and-above: With solid foundational knowledge that includes aptitude with high-performance code and an ability to identify the best tool for a particular problem, you can exceed expectations rather than just meet them.
Basics, Hands-On, Which First
When people attempt to acquire a new technology, most of them like to start from (and probably only learn) how to use the new technology. Especially when you are given limited time to get a task done using a new technology, it is impractical to spend substantial amount of time on learning basics first. Remember, the key of systematic learning is about mastering both foundation and application to achieve true proficiency and the order of learning them depends on your timing.
Systematic Learning at Target
As one of the instructors for Target’s internal training sessions on Hadoop, I integrated the systematic learning approach in the training curriculum, aiming to cover both the foundation of Hadoop ecosystem and programming Hadoop applications. During the training sessions, trainees will learn the key components of Hadoop ecosystem, architecture and core concepts of Hadoop, and use cases of Hadoop applications. In addition, we designed a step-by-step, hands-on Hadoop lab in which participants will solve a real-world data problem using HDFS, Yarn, Hive and Oozie and also learn to optimize their data solutions. The systematic training approach will help attendees build a comprehensive understanding of Hadoop: what Hadoop is, when to use Hadoop, and how to program Hadoop, which is a small but important step of improving technical agility and technical practices at Target.
Xinran (Julina) Waibel is a Target software engineer with a specialization in data engineering. She is also a developer of the open-source programming language ForEST.