This project is designed to minimize hiring errors at a tech company. Using the Design Thinking method, I researched the problem from multiple perspectives to identify the needs and their root causes. I then proposed a multi-component solution and alternative strategies to enhance hiring managers' performance and foster cross-team collaboration when interviewing internship candidates. In collaboration with the stakeholders, I designed, tested, and developed what was considered the most suitable solution for the audience, a microlearning experience integrated into the existing training initiative, following a self-assessment instructional approach. The project represents the culmination of my learning in the ITEC program at SFSU.
THE PROBLEM
In a kick-off meeting with a manager of the Geographic Information Systems Data group, I was informed that a significant number of the data-research interns underperformed in the last seven years, costing the company over $200,000.
40 K
per intern per year
50%
had no outcome
33%
received a job offer
17%
joined the company
NEEDS ASSESSMENT
NA PROPOSAL & PLAN
After the kick-off meeting, I investigated the problem to unpack areas of need even more. The main research goals were to
-
Gain insight into the company's approach to training in interviewing.
-
Identify the gap between the interviewers' current and target performance.
-
Understand the interviewers' experiences in the interview process to identify their pain points.
-
Gain insight into the interviewers' insights about the roots of the problem.​
​
Read the Needs Assessment Plan to learn more about the process.
AGGREGATED EMPATHY MAP, PERSONA, AND JOURNEY MAP
.jpg)
Following the Design Thinking model, I summarized the data on an aggregated empathy map and developed one persona that highlighted the interviewers' main pain points and goals. Additionally, I created a journey map to identify opportunities for improvement in their interviewing process and a learner story to amplify their target actions and obstacles.​
CAUSAL ANALYSIS
.jpg)
Three need were identified during the needs assessment research, which I prioritized the following need statements based on the magnitude of the discrepancies and the risk in other need areas if the need was not met:
-
All the Geographic Information Systems Data scientists will be able to prepare effectively for the interviews with internship candidates by reading publications and papers in the candidates' research areas and developing questions to evaluate the candidates' knowledge of general and focused concepts based on at least three papers of general and focused interest.
-
All the Geographic Information Systems Data scientists will be able to write informed feedback about the candidates' abilities and competencies by including their responses to the questions as accurately as possible, scoring them, and justifying their judgment with references to books and articles.
-
All the Geographic Information Systems Data scientists will be able to establish a cross-team communication and collaboration strategy to develop resources that the interviewers can utilize, such as a standards-based system of creating interview questions and a question bank.
-
The interns will be able to demonstrate professional skills, enthusiasm, and independence in the project.
​
Limitations: Due to access to resources, the validity of my interpretations would be higher.
NA REPORT & ACTION PLAN
Problem Statement
In the post-assessment phase, I synthesized what I learned from the NA research and developed the problem statement: The Geographic Information Systems Data scientists need a flexible and easy way to learn how to prepare for interviews with candidates outside of their research areas, create standard-based interview questions, and write feedback, because there is an interview performance gap resulting in hiring errors for internship positions.
Goal
Through flexible learning experiences on better interview preparation and feedback provision and by collaborating with the other teams, the Geographic Information Systems Data scientists will be better prepared to interview new internship candidates, develop effective questions, and provide the hiring managers with better feedback about the candidates' competencies.
​Read the Action Plan to learn more about the suggested solution.
IDEATE
HOW-MIGHT-WE STATEMENTS, COMPETITIVE AUDIT, CRAZY 8s, AND STORYBOARD
With the instructional goal in mind, I ideated instructional solutions for microlearning experiences following different strategies. I developed how-might-we statements to frame my brainstorming process and "crazy eights" to brainstorm as many solutions as possible. I then conducted a competitive audit to get inspiration for existing solutions, and finally, I developed a big picture storyboard to see what the experience of the persona would look like. Of all the instructional solutions I ideated (video, infographic, document, email, peer check-ins, short instructor-led session, and gamification), the e-learning with elements of self-assessment and gamification seemed to be the most appropriate microlearning experience to meet the Geographic Information Systems Data scientists' learning needs.
ANALYSIS
TASK ANALYSIS AND INSTRUCTIONAL ANALYSIS
At this stage of the project, I conducted the job-task analysis and identified the tasks that the interviewers must perform before, during, and after the interview. Then, I analyzed the tasks into the steps that the interviewers must perform, and the subordinate skills they must demonstrate in each step, and developed the following instructional goals:
After the microlearning experience, the Geographic Information Systems Data scientists will be able to:
-
Prepare for the interviews with internship candidates, and more specifically, review the candidates' resumes, read papers and publications on the candidates' research areas, develop appropriate interview questions about the fundamentals in the field and in the candidates' areas of expertise, and insert them in the portal.
-
Provide detailed feedback about the candidates' knowledge and skills, write the candidates' answers as accurately as possible, rate them, and justify their feedback.
​​
Then, I converted the instructional goals into the terminal goals
After the Geographic Information Systems Data scientists take the e-learning, they will be able to:
-
Describe the interview preparation and feedback provision process.
-
Explain why it is important to read literature and publications on the candidates' research areas.
DESIGN
LEARNING ASSESSMENT & INSTRUCTIONAL STRATEGIES
To assess the employees' performance I developed the interview performance evaluation rubric that measures three levels of performance (basic, proficient, advanced) on the following criteria: accept the interview invitation, formulate appropriate interview questions to collect evidence for the candidates' knowledge and skills in their areas of expertise, and write thorough feedback.
For learning to occur, I developed the self-assessment instructional strategy that draws its rationale from the cognitive and constructivist theories of learning and motivation, the metacognition theory, and the self-efficacy theory. The process of self-assessment consists of four stages: introduce the assessment criteria, demonstrate how to apply the criteria, provide feedback on the application of the criteria, and implement the learning goal and strategies. In the microlearning experience, the criteria are introduced through self-reflection questions that the employees must answer based on three levels of competence. After they give their answers, feedback about the right performance and a justification are provided. At the end of the self-assessment, the participants are provided with a description of their performance based on their answers and guidance to set learning goals and strategies.
Read the design document to learn more about the learning objectives, the assessment plan, and the instructional strategies.
​
PROTOTYPE & TEST
PAPER PROTOTYPE, FOCUS GROUP RESEARCH, AND USABILITY TEST
After I developed the instructional strategies, I created two versions of the low-fidelity wireframes on paper. In the first version, the users would answer four questions to evaluate their interview preparation, read descriptions of the accepted performance, and receive feedback, personalized goals, and learning strategies. In the second version, the users would read information about preparing effectively for an interview, then answer questions to evaluate their performance, and finally identify their learning goals and strategies.
To determine which version was more effective, suitable, and enjoyable for the learners, I tested them with a representative group of four data scientists. In this informal exploratory test, I walked them through the preliminary designs and received feedback. Then, I summarized the data and identified the following patterns:
-
A design that focuses on applying instruction in the work setting is more conducive to learning than a conventional one that focuses on the presentation of information.
-
Game elements (like the "contribution bar" in the first version) foster engagement and motivation.
-
Self-assessment is a suitable instructional approach for the audience.
​
Read the Research Plan and the Report to learn more about the testing process.
The insights I gained from the first test helped me create an improved version of the low-fidelity prototype. Then, I tested it again. Read the High-Fidelity Prototype Test Plan and Report to learn more. Here is how I improved the high-fidelity prototype:
1. I added a slide with the terminal objectives.
HIGH-FIDELITY PROTOTYPE
2. I separated the demonstration of the preferred performance and the self-assessment questions to draw the learners' attention to the critical information and applied the multimedia principles to improve understanding.
Before
.png)
After
3. I improved the way critical information is presented to draw the learners' attention to the personalized learning objectives and strategies to improve their performance.
Before
After
The insights from the focus-group interview helped me create the low-fidelity prototype. Another round of testing revealed new patterns and insights about what the microlearning experience should look like. The most important changes in the design related to the method of presenting the stimulus to improve attention and cognition and the interactivity.​
​
The effectiveness of the solution would be measured by comparing the interview questions and the written feedback about the candidates generated by the interviewers before and after the training. Specific criteria for the evaluation are delineated on the Interview Performance Rubric. Additionally, the effectiveness of the solution would be measured by tracking the interns' interview and performance scores and job offer rates.​