Ethical Design of Location-Data-Driven Applications
Timeline: 3 months
Model: Dick & Carey Systems Approach Model
Responsibilities: Needs Assessment, Instructional Design, Prototyping, Interaction Design
Tools: Figma, Keynote, Microsoft Forms
Ethical Design of Location-Data-Driven Applications is a 2-hour instructor-led training designed to improve the performance of entry-level Geographic Information Systems Data (GISD) scientists at a tech company. Participants learn strategies for identifying and prioritizing design solutions, as well as for gathering and processing user data in compliance with the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). Opportunities to assess their understanding are incorporated throughout the training, including quizzes and hands-on activities. The final assessment includes scenario-based questions that simulate real-world job tasks.​This project is the culmination of my learning in the ITEC program at SFSU. It is based on a real performance problem, yet there are limitations in the collection of data about the needs.
THE PROBLEM
To identify the problem, I interviewed a senior data science manager of the GISD group. In the kick-off meeting, I gathered information about the business goal and the target performance. In alignment with the business goal to restore and maintain public trust and acceptance of location technologies, all employees must demonstrate competency in the ethical collection and use of location data as mandated by the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). Yet, 83% of the employees failed the user privacy checks. Moreover, the employees have not received training in the lawful collection and use of personal data, contributing to the problem. The outcome of the front-end analysis was the following performance goal: The GISD scientists will demonstrate ethical use of location data in their code files, resulting in a 60% or more (at the adequacy level) and 83% (at the excellence level) decrease in user privacy violations.
NEEDS ASSESSMENT
Before deciding that training was needed, I investigated the areas of concern identified in the kick-off meeting. In collaboration with the GISD manager, I became aware of what was already known about the employees' performance and what else needed to be investigated (read the Needs Assessment Proposal and the Needs Assessment Plan). I identified the discrepancies in the GISD scientists' knowledge and skills in user-privacy protection and unpacked other causes of the problem.
From the data I gathered, I identified the learning and performance gaps in the following areas:
-
Lawful collection and use of personal data
-
Principles for identifying design solutions that comply with the user-privacy protection regulations
-
Strategies to edit data to protect user privacy
-
Data-analysis techniques to protect user privacy​​​
Read the Needs Assessment Report & Action Plan to learn more.​
Limitations: Due to the limited access to resources and the low participation (20%) in the survey, no valid inferences about the magnitude of the discrepancies and their causes could be made safely. If more data were gathered and the discrepancies in the need areas were clear, the conclusions of the causes and the association of the employees' current knowledge and skills would lead to safer decisions regarding the training needs.
IDEATE
COMPETITIVE AUDIT
Competitive Audit Spreadsheet.
Before designing the training, I evaluated four direct competitors' strengths and weaknesses and compared the training type, duration, content, and learner experience. For more information about the research, you may refer to the competitive audit report. The most appropriate intervention seemed to be an instructor-led and highly interactive training session with simulations of tasks that research scientists need to complete.
ANALYSIS
JOB ANALYSIS
To focus the training on tasks that offer a high probability of gains in job efficiency, effectiveness, and satisfaction, I interviewed the GISD manager to identify the job tasks and tools the employees use in the various product design phases (see the diagram above). The outcome of this process was the following instructional goal: The scientists in the Geographic Information Systems Data group will be able to identify the risks that come with data-driven applications and the effects of the use of location data, and apply methods of ethical use of data according to the Locus Charter, the General Data Protection Regulation, and the California Consumer Privacy Act.
INSTRUCTIONAL ANALYSIS
Learning Goal Analysis into the Subordinate Sills. Full diagrams in Miro
After the job analysis, I conducted the instructional analysis. I reviewed the GDPR, CCPA, and relevant articles, listened to webinars, and interviewed a senior data scientist to identify the important knowledge that the participants needed to develop. Then, I analyzed the instructional goal into steps, and each step into the subordinate and entry skills (see the image in the carousel above). Finally, I converted the instructional goal and subordinate skills into the terminal and subordinate objectives.
LEARNER ANALYSIS
Learner Persona
In parallel with the instructional analysis, I gathered information about the context and the audience. I interviewed the data science manager to learn about the demographics, characteristics, prior knowledge, and motivation of the learners. Then, I created a learner persona.
DESIGN
After I developed the terminal and subordinate objectives, I designed the summative and formative assessments in alignment with the terminal objectives: the entry skilscenario-based questions and tasks that the trainees must complete on a computer, a quiz to check the learners' understanding of the risks of harm that come with the data-driven applications and the main principles of data use, group activities with scenario-based questions on the strategies to protect user privacy.
​Then, I developed the content outline and the strategies and media to catch the learners' attention, recall prior knowledge, present the content, and provide opportunities for practice before the final assessment.
​
Read the Design Document to learn more about the learning outcomes, the assessments, and the instructional strategies.
DEVELOPMENT & TEST
The development of the learning materials and assessment method was an iterative process that involved the stakeholders who provided critical feedback. First, I created a text-based storyboard for the presentation and received feedback from the senior data scientist. Then, I developed the mockups in Keynote, which provided the stakeholders with a clear picture of the presentation.
Additionally, I created a low-fi prototype of the summative assessment in Figma, and the senior data scientist provided feedback. Finally, I developed the presentation in Keynote, the quiz in Microsoft Forms (formative assessment), and the group-discussion handout in Pages (formative assessment).
Evaluation of the training session's usability was incorporated in the various prototyping phases as described above. The research manager provided feedback for each learning material at each stage. A full evaluation plan to measure the usability and the perceived impact from the learners’ perspective in the different design and development phases is included in the design document.
EVALUATION
TAKEAWAYS
This project was an excellent exercise in working with subject matter experts to identify the target performance and develop the content of the learning experience and engaging a representative sample of the end users in the design process to create a learner-centered learning experience.