Blame
|
1 | <!-- Generated from applied-computing-au vic/unit3-4/sat/C09-2026 by port-reference-godot-to-wiki.py — do not hand-edit; re-run the port. --> |
||||||
| 2 | # 9-2 Beta Testing Data Collection Methods |
|||||||
| 3 | ||||||||
| 4 | ## Understanding Data Types for Beta Testing |
|||||||
| 5 | ||||||||
| 6 | Before selecting collection methods, understand what type of data you need: |
|||||||
| 7 | ||||||||
| 8 | **Quantitative Data (Measurable)** |
|||||||
| 9 | ||||||||
| 10 | - Likert scale ratings (1-5 satisfaction scores) |
|||||||
| 11 | - Yes/No responses |
|||||||
| 12 | - Task completion times |
|||||||
| 13 | - Error counts |
|||||||
| 14 | - Success/failure rates |
|||||||
| 15 | ||||||||
| 16 | **Qualitative Data (Descriptive)** |
|||||||
| 17 | ||||||||
| 18 | - User opinions and experiences |
|||||||
| 19 | - Explanations of problems encountered |
|||||||
| 20 | - Suggestions for improvements |
|||||||
| 21 | - Behavioral observations |
|||||||
| 22 | ||||||||
| 23 | ## Four Main Data Collection Methods |
|||||||
| 24 | ||||||||
| 25 | ### 1. **Surveys/Questionnaires** |
|||||||
| 26 | ||||||||
| 27 | **Purpose**: Collect standardised, quantifiable feedback from multiple testers simultaneously |
|||||||
| 28 | ||||||||
| 29 | **When to Use**: When you need measurable data to compare across users and identify trends |
|||||||
| 30 | ||||||||
| 31 | **Types of Questions**: |
|||||||
| 32 | ||||||||
| 33 | **Closed Questions (Quantitative)** |
|||||||
| 34 | ||||||||
| 35 | - _Rating Scale_: "Rate the user interface design: Very Poor (1) to Excellent (5)" |
|||||||
| 36 | - _Multiple Choice_: "Which feature was most difficult to use? A) Login process B) Main navigation C) Search function D) Settings menu" |
|||||||
| 37 | - _Yes/No_: "Were you able to complete the task without assistance?" |
|||||||
| 38 | ||||||||
| 39 | **Open Questions (Qualitative)** |
|||||||
| 40 | ||||||||
| 41 | - _Experience-based_: "Describe any moment when the interface caused you to pause or feel uncertain" |
|||||||
| 42 | - _Problem-focused_: "What was the most challenging aspect of using this software?" |
|||||||
| 43 | - _Improvement-focused_: "What changes would make this software easier to use?" |
|||||||
| 44 | ||||||||
| 45 | **Advantages**: Efficient for large groups, standardized responses, easy to analyze statistically |
|||||||
| 46 | **Best For**: Measuring satisfaction, ease of use, feature preferences |
|||||||
| 47 | ||||||||
| 48 | --- |
|||||||
| 49 | ||||||||
| 50 | ### 2. **Observation** |
|||||||
| 51 | ||||||||
| 52 | **Purpose**: Record actual user behavior without relying on self-reported data |
|||||||
| 53 | ||||||||
| 54 | **When to Use**: When you need objective evidence of how users actually interact with your software |
|||||||
| 55 | ||||||||
| 56 | **Observation Techniques**: |
|||||||
| 57 | ||||||||
| 58 | - **Screen recording** during testing sessions |
|||||||
| 59 | - **Task completion monitoring** (time-on-task measurements) |
|||||||
| 60 | - **Error frequency tracking** and navigation patterns |
|||||||
| 61 | - **Behavioral notes** (hesitation, confusion, workarounds) |
|||||||
| 62 | ||||||||
| 63 | **Example Observation Tasks**: |
|||||||
| 64 | ||||||||
| 65 | - Record how long it takes users to complete core workflows |
|||||||
| 66 | - Observe navigation patterns and identify common user pathways |
|||||||
| 67 | - Note frequency and types of errors made during typical tasks |
|||||||
| 68 | - Document where users pause or show confusion |
|||||||
| 69 | ||||||||
| 70 | **Advantages**: Objective data, reveals actual vs. reported behavior, identifies usability issues |
|||||||
| 71 | **Best For**: Workflow efficiency, interface design problems, actual vs. perceived performance |
|||||||
| 72 | ||||||||
| 73 | --- |
|||||||
| 74 | ||||||||
| 75 | ### 3. **Interviews** |
|||||||
| 76 | ||||||||
| 77 | **Purpose**: Explore the "why" behind user behaviors and gather detailed feedback |
|||||||
| 78 | ||||||||
| 79 | **When to Use**: When you need to understand user motivations, clarify survey responses, or explore unexpected findings |
|||||||
| 80 | ||||||||
| 81 | **Interview Structure**: |
|||||||
| 82 | ||||||||
| 83 | **Structured Questions** |
|||||||
| 84 | ||||||||
| 85 | - "Walk me through how you typically use this software" |
|||||||
| 86 | - "What was your initial reaction to the main interface?" |
|||||||
| 87 | - "Describe your experience with the most important features" |
|||||||
| 88 | ||||||||
| 89 | **Follow-up Probes** |
|||||||
| 90 | ||||||||
| 91 | - "Can you tell me more about that challenge?" |
|||||||
| 92 | - "How did that make you feel as a user?" |
|||||||
| 93 | - "What would have made that process easier?" |
|||||||
| 94 | - "Why do you think that happened?" |
|||||||
| 95 | ||||||||
| 96 | **Advantages**: Rich detailed feedback, clarifies confusing survey responses, uncovers unexpected issues |
|||||||
| 97 | **Best For**: Understanding user emotions, complex workflows, training and support needs |
|||||||
| 98 | ||||||||
| 99 | --- |
|||||||
| 100 | ||||||||
| 101 | ### 4. **Reports/Documentation** |
|||||||
| 102 | ||||||||
| 103 | **Purpose**: Systematically compile and analyse all collected data into actionable insights |
|||||||
| 104 | ||||||||
| 105 | **When to Use**: To synthesise findings from multiple methods and present recommendations |
|||||||
| 106 | ||||||||
| 107 | **Key Components**: Combine quantitative data (scores, statistics) with qualitative insights (themes, quotes) to create comprehensive findings and prioritized recommendations. |
|||||||
| 108 | ||||||||
| 109 | **Advantages**: Provides comprehensive overview, supports decision-making, documents test outcomes |
|||||||
| 110 | **Best For**: Final assessment, stakeholder communication, future development planning |
|||||||
| 111 | ||||||||
| 112 | ## Data Analysis Strategy |
|||||||
| 113 | ||||||||
| 114 | **For Closed Survey Questions**: |
|||||||
| 115 | ||||||||
| 116 | - Calculate average scores and percentages |
|||||||
| 117 | - Create charts showing satisfaction trends |
|||||||
| 118 | - Identify lowest-scoring areas for improvement focus |
|||||||
| 119 | ||||||||
| 120 | **For Open Survey Responses and Interview Data**: |
|||||||
| 121 | ||||||||
| 122 | - Use thematic analysis to identify common issues |
|||||||
| 123 | - Code responses into categories (usability, functionality, performance) |
|||||||
| 124 | - Extract representative quotes for reporting |
|||||||
| 125 | ||||||||
| 126 | **For Observation Data**: |
|||||||
| 127 | ||||||||
| 128 | - Compile task completion statistics |
|||||||
| 129 | - Map common error patterns |
|||||||
| 130 | - Document workflow efficiency metrics |
|||||||
| 131 | ||||||||
| 132 | ## Mixing Methods for Comprehensive Results |
|||||||
| 133 | ||||||||
| 134 | **Best Practice**: Use multiple methods to triangulate findings |
|||||||
| 135 | ||||||||
| 136 | **Recommended Combination Approach**: |
|||||||
| 137 | ||||||||
| 138 | 1. **Survey** to measure overall satisfaction and identify problem areas |
|||||||
| 139 | 2. **Observation** to objectively measure task performance in those problem areas |
|||||||
| 140 | 3. **Interview** to understand why those problems occur and how users work around them |
|||||||
| 141 | 4. **Report** to synthesize quantitative metrics with qualitative insights |
|||||||
| 142 | ||||||||
| 143 | This approach ensures you capture both the "what" (quantitative metrics) and the "why" (qualitative insights) of user experience. |
|||||||
| 144 | ||||||||
| 145 | ||||||||
| 146 | --- |
|||||||
| 147 | ||||||||
| 148 | _Remember: Your beta testing plan should specify which methods you'll use for each test scenario, ensuring alignment between your testing objectives and your data collection approach._ |
|||||||
| 149 | ||||||||
| 150 | ||||||||
| 151 |  |
|||||||
