Creating a Training Feedback Loop

Remember that bad survey you wrote? The one that resulted in responses filled with blatant bias and caused you to doubt whether your respondents even understood the questions? Creating a survey may seem like a simple task, but even minor errors can result in biased results and unreliable data. If this has happened to you before, it's likely due to one or more of these common mistakes in your survey design: 1. Ambiguous Questions: Vague wording like “often” or “regularly” leads to varied interpretations among respondents. Be specific—use clear options like “daily,” “weekly,” or “monthly” to ensure consistent and accurate responses. 2. Double-Barreled Questions: Combining two questions into one, such as “Do you find our website attractive and easy to navigate?” can confuse respondents and lead to unclear answers. Break these into separate questions to get precise, actionable feedback. 3. Leading/Loaded Questions: Questions that push respondents toward a specific answer, like “Do you agree that responsible citizens should support local businesses?” can introduce bias. Keep your questions neutral to gather unbiased, genuine opinions. 4. Assumptions: Assuming respondents have certain knowledge or opinions can skew results. For example, “Are you in favor of a balanced budget?” assumes understanding of its implications. Provide necessary context to ensure respondents fully grasp the question. 5. Burdensome Questions: Asking complex or detail-heavy questions, such as “How many times have you dined out in the last six months?” can overwhelm respondents and lead to inaccurate answers. Simplify these questions or offer multiple-choice options to make them easier to answer. 6. Handling Sensitive Topics: Sensitive questions, like those about personal habits or finances, need to be phrased carefully to avoid discomfort. Use neutral language, provide options to skip or anonymize answers, or employ tactics like Randomized Response Survey (RRS) to encourage honest, accurate responses. By being aware of and avoiding these potential mistakes, you can create surveys that produce precise, dependable, and useful information. Art+Science Analytics Institute | University of Notre Dame | University of Notre Dame - Mendoza College of Business | University of Illinois Urbana-Champaign | University of Chicago | D'Amore-McKim School of Business at Northeastern University | ELVTR | Grow with Google - Data Analytics #Analytics #DataStorytelling

Ran a training session a few days back. Was asked for tips on how to ask for feedback, so here goes: 1️⃣ Confirm that you will do something (positive) with the feedback. If not, don’t bother. 2️⃣ Know yourself: if you’re a sensitive soul, overly defensive, a praise-junkie, factor that in. Especially if you want accurate data. 3️⃣ Know your client: if you’re asking a dream-crusher, hard-nosed b&stard, over-enthusiast, mute manager, people-pleaser: adapt your approach before and after. 4️⃣ Self-assess: feedback on you first so you get a lay of the land. 5️⃣ Give people time to prep the feedback, but not too much time - once a year, as in Xmas, is insufficent. 6️⃣ Specify: know why you’re asking, exactly what you’re asking for, and how you’d like people to deliver it. 7️⃣ Questions to use: - what do you see me doing well? - what don’t I know about myself? - where should I invest my improvement efforts? - what do you trust me with? - what not? - three words to describe? - who can I best learn from? 8️⃣…and breathe. In the build-up, in the room, in the moment, in the aftermath. 9️⃣ see FB as data you’ve collected, nothing more, nothing less. 🔟 thank ‘em. Through gritted teeth, if you have to.😬 Brucey Bonus: Look for patterns, not opinions. One person says it, <meh>. Three people say it? *That’s* your reputation. (FD: I am the *worst* when it comes to feedback, some of the above has made me better.) What to add?

A lot of trainers run a great exercise… and then waste the learning moment that follows. The debrief is where performance improvement actually happens. But too often we get generic reflections: “Yeah, that was good” or “Interesting exercise.” None of that helps anyone perform better back on the job. A simple tool I use in almost every session, face-to-face or virtual, is the Feedback Grid. It structures the debrief so delegates can evaluate the outcomes of an exercise, not just how it felt. Here’s exactly how to use it straight after an activity: 1. Set up the 4 quadrants before the exercise Worked Well (+) Needs Change (Δ) Questions (?) New Ideas (💡) By having it visible from the start, delegates know there will be a structured review, not a free-for-all discussion. 2. Immediately after the exercise, ask individuals to add notes Give everyone 2–3 minutes to jot down their thoughts in each category. This stops dominant voices from setting the tone and gives you a broader view of what actually happened. In a virtual room, this is as simple as shared online sticky notes. Face-to-face, use flipcharts or a whiteboard. 3. Analyse the activity, not the activity’s “vibe” This is where most trainers go wrong. We’re not asking whether they “liked” the exercise. We’re capturing what the exercise showed about their skills, behaviours, and decision-making. Examples might include: Worked Well: “Clearer roles helped us move faster.” Needs Change: “We didn’t communicate early enough.” Questions: “How do we apply this under time pressure?” New Ideas: “Create a decision checklist before starting.” These are performance insights, not opinions. 4. Turn the grid into next-step actions Once patterns emerge, summarise 2–3 practical actions they can take into the workplace. This is where the ROI sits. The exercise becomes a rehearsal, and the grid becomes the bridge to real work. 5. Keep the pace tight A structured debrief shouldn’t drag. Five to eight minutes is enough to turn a simple exercise into a meaningful learning moment. When used properly, the Feedback Grid transforms exercises from “fun activities” into performance diagnostics. That’s the whole point of training, to improve what people do, not what they think about the training. What do you use for this? -------------------- Follow me at Sean McPheat for more L&D content and then hit the 🔔 button to stay updated on my future posts. ♻️ Save for later and repost to help others. 📄 Download a high-res PDF of this & 250 other infographics at: https://lnkd.in/eWPjAjV7

The document “Qualitative Data Collection Techniques” by Dr. Khalifa Elmusharaf offers a detailed exploration of various methods for gathering qualitative data, including interviews, focus group discussions (FGDs), observations, and document reviews. It emphasizes the importance of systematic and contextually sensitive data collection for generating robust and meaningful insights, particularly in fields like health research and social sciences. The content addresses practical skills, such as preparing for interviews, developing discussion guides for FGDs, and observing human behavior and phenomena. Techniques for enhancing communication, probing responses, and documenting non-verbal cues are extensively discussed, providing readers with tools to capture in-depth information. Furthermore, the document outlines strategies for selecting participants, arranging physical settings, and ensuring ethical practices such as informed consent. This resource is particularly valuable for professionals and researchers aiming to deepen their understanding of qualitative methods. It combines theoretical frameworks with actionable guidance, making it an essential reference for those involved in social research, program evaluations, and evidence-based decision-making. Let me know if you need a summary of specific sections or further insights.

“What’s the ROI of this training?”, asked the organization that: • Didn’t brief the manager on what the program actually covers • Didn’t align learning to real, on-the-job challenges • Didn’t follow up meaningfully beyond Day 1 • Didn’t change supporting systems, KPIs, or everyday behaviors • Relied on generic 30-60-90 journeys with limited ownership or reinforcement • Still expects transformation in 2 days Let’s get something straight. Training is not a vending machine. You don’t insert a trainer and expect “Productivity +15%” to pop out. Training is an enabler. A catalyst. A spark. Not the fire. Not the fuel. Not the oxygen. 70% of learning happens on the job. And yet, most managers: • Don’t know what was taught • Don’t reinforce it • Don’t coach for application • Don’t ask reflective questions Then we ask: “Why didn’t behavior change?” Because you sent people to the gym… and expected muscles without lifting weights. Here’s the uncomfortable part. Most post-training follow-ups rely on: • Happy sheets • LMS completion ticks • TMS attendance reports Which raises a simple question: If your Level-1 feedback is superficial, how are you expecting Level-3 results to be meaningful? Smiles, stars, and “great session” comments don’t measure: • Behavior shifts • Manager reinforcement • Real workplace application • Obstacles participants are facing You can’t build business impact on feel-good feedback. Real ROI happens when: • Learning captures real challenges, not just reactions • Reflection continues beyond the classroom • Managers see, coach, and reinforce micro-behaviors • Follow-up is designed, not assumed Otherwise, don’t ask for ROI. Ask instead: “Did we measure learning deeply enough to deserve results?” #SaHRcasm #LearningAndDevelopment #TrainingROI #BehaviorChange #ManagersMatter #BeyondHappySheets

𝗧𝗵𝗲 𝗜𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝗰𝗲 𝗼𝗳 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 𝗶𝗻 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁 🗣️ Ever feel like your Learning and Development (L&D) programs are missing the mark? You're not alone. One of the biggest pitfalls in L&D is the lack of mechanisms for collecting and acting on employee feedback. Without this crucial component, your initiatives may fail to address the real needs and preferences of your team, leaving them disengaged and underprepared. 📌 And here's the kicker—if you ignore this, your L&D efforts risk becoming irrelevant, wasting valuable resources, and ultimately failing to develop the skills your workforce truly needs. But don't worry—there’s a straightforward fix: integrate feedback loops into your L&D programs. Here’s a clear plan to get started: 📝 Surveys and Questionnaires: Regularly distribute surveys and questionnaires to gather insights on what’s working and what isn’t. Keep them short and focused to maximize response rates and actionable feedback. 📝 Focus Groups: Organize small focus groups to dive deeper into specific issues. This setting allows for more detailed discussions and nuanced understanding of employee needs and preferences. 📝 Real-Time Polling: Use real-time polling tools during training sessions to gauge immediate reactions and make on-the-fly adjustments. This keeps the learning experience dynamic and responsive. 📝 One-on-One Interviews: Conduct one-on-one interviews with a diverse cross-section of employees to get a more personal and detailed perspective. This can uncover insights that broader surveys might miss. 📝 Anonymous Feedback Channels: Ensure there are anonymous ways for employees to provide feedback. This encourages honesty and helps identify issues that employees might be hesitant to discuss openly. 📝 Feedback Integration: Don’t just collect feedback—act on it. Regularly review the feedback and make necessary adjustments to your L&D programs. Communicate these changes to employees to show that their input is valued and acted upon. 📝 Continuous Monitoring: Use analytics tools to continuously monitor engagement and performance metrics. This provides ongoing data to help refine and improve your L&D initiatives. Integrating these feedback mechanisms will not only enhance the effectiveness of your L&D programs but also boost employee engagement and satisfaction. When employees see that their feedback leads to tangible changes, they are more likely to be invested in the learning process. Have any innovative ways to incorporate feedback into L&D? Drop your tips in the comments! ⬇️ #LearningAndDevelopment #EmployeeEngagement #ContinuousImprovement #FeedbackLoop #ProfessionalDevelopment #TrainingInnovation

A while back, I observed a well-intentioned practitioner facilitating focus group discussions (FGD) with rural farmers about their experience accessing local markets. He stood in front of the group, holding his discussion guide like a clipboard, and went around the circle calling on participants to respond to each question one by one. This was a group interview, not an FGD.  🛑 The point of qualitative work is “thick description”—rich, contextual insight that captures meaning, nuance, power dynamics, contradiction, and context. You don’t get thick description from: 🚫 Reinforcing power hierarchies and formality 🚫 Processing responses rather than inviting organic conversation 🚫 Following the questionnaire guide like a clinical checklist You get it through standards-based quality control. People are often skeptical of qualitative work because so often it’s done badly. And the problem isn’t bad facilitators — it’s the absence of systems that ensure quality. Based on my experience across dozens of contexts here’s what works: Standards-Based Training ✔️ Teams learn qualitative theory and key OBSERVABLE facilitation standards. ✔️ They practice the FGD guide until fluent. Appropriate Staffing and Supervisory Structure ✔️ One facilitator + (ideally) two note-takers per group. One supervisor per two facilitator teams.  ✔️ Hire and compensate facilitators, notetakers, and supervisors based on qualitative expertise. Structured Observation + Supportive Supervision  ✔️ On day one, supervisors observe each facilitator team at least once using a Quality Improvement Verification Checklist (QIVC) based on the standards introduced during training.  ✔️ Following each observation, supervisors reinforce strengths, highlight gaps, and identify needed adjustments. ✔️ If needed, repeat on Day 2. Data quality is made or lost in the first 48 hours. Observation + supervision are keys to success. Daily Debriefs + Iteration ✔️ Each facilitation team debriefs after every FGD to identify opportunities for improvement. ✔️ Every evening: full team debrief (all facilitator teams + supervisors + leadership) to further refine questions and probes, analyze trends and track theme saturation. Daily Note Review ✔️ When full transcription isn’t feasible notes ARE the data. Review notes for content, tone, emotion, dynamics, and non-verbal cues. ✔️ Provide feedback to ensure iterative improvement. Quality control is not an “extra.” If you’re not using consistent standards across your qualitative data collection function, you’re simply not producing (or using) credible evidence. These steps can help you get there. 👊 #FitForPurposeEvidence #Evaluation #QualitativeMethods #EvidenceBasedPractice #MethodologicalIntegrity #QualitativeStandards #SocialImpact

From raw feedback to actionable insights: My AI-powered workflow. I'm running an AI-Native PM training and for each cohort I like to close the feedback loop in a more dynamic, engaging, and collaborative way. Here’s the 3-step, AI-powered, collaborative process I use. Step 1: Capturing the raw feedback with Google Forms. It starts with a simple Google Form to gather candid feedback on the training. Step 2: Transforming raw feedback into an engaging video with Notebook LM. This is where the magic happens. Instead of manually combing through the feedback and creating slides, I took a different approach. I uploaded all the raw, anonymized feedback directly into Notebook LM and then prompted it to act as a product manager synthesizing user research, asking it to identify the core positive themes, the most critical areas for improvement, and to structure these findings into a concise video. Step 3: Uploading the video to Loom for sharing and collaboration. Numbers are great, but a video is more personal and engaging. This final step is key because Loom transforms a one-way summary into a two-way conversation. By sharing a Loom link with my stakeholders, they can: • Watch the summary on their own time. • Leave comments and reactions tied to specific moments in the video. • Engage in threaded discussions right on the video timeline. This workflow didn't just save me time but created a richer, more collaborative way to understand and act on valuable feedback. It’s a simple and fun example of how we can use AI tools not just to build products, but to improve how we communicate and share learnings.

[EN] Ways to measure the success of your training and prove impact: In years of experience in the L&D market and being an L&D mentor at the L&D SHAKERS community, I’ve faced different situations where mentees come to me for help in thinking about how to “prove that their training programs really work.” Today, I decided to share a little about it and maybe help more L&D fellows. ♥️ The truth is that there isn't a simple answer (like everything in life haha), but there are some ways to do it better. Below are some insights (hope they help you): 👉 Planning the measurement method is part of training design Often forgotten but always necessary. Your L&D solution isn’t ready to launch if you don’t know which indicators, success metrics, and measurement methods you’ll use. Believe me: if you only think about it in the middle of the training program or session, it will be harder to prove success and results! 👉Tracking the experience: NPS/CSAT/Feedback survey The top method and easiest way to measure. There are no tricks here. Make sure all participants receive an NPS/CSAT/Feedback survey at the end of the session. The recommendation is: be simple! Ask 1 to 3 closed questions plus an optional comment box. The number of responses is highly dependent on the size of the form. Fewer questions = More answers = More data reliability 👉Tracking behavior change: Assessments My fav, especially in the case of training programs (not too good for workshops or one-session training). The idea is to track some participants’ behaviors before the training and sometime after the training (weeks to months) using a survey where you can identify how learners act and think about certain issues. In general, “Likert-type” questions are the best. The difference between the first assessment and the final one will help you understand where your training program helps. An extra option is to use a 180º or 360º assessment, where coworkers, stakeholders, and/or direct reports receive the same assessment to answer about the participant’s behaviors. 👉Tracking incidental results: Business changes Let’s face it: proving that a training program changes a business metric is a dream but hard to achieve. But with time and attention, in some cases (especially in more technical training), this can be possible. To make it happen, try to map all the impacts that your training program can cause. Which aspects of leadership can change? How has the client's NPS improved after the CSM team completed the training program? After mapping the possible impacts, call on some coworkers from the business/growth/data areas to help you track these results during the training period. So, there are countless ways to explore this topic, and this really deserves an exploratory article (soon). But until then, I’d recommend you check out some experts in HR data and training measurement: 📌 Dr. Alaina Szlachta 📌 David Green 🇺🇦 📌 Kirkpatrick Partners, LLC

When I first started teaching online back in 2017, the course evaluation process bothered me. Initially, I was excited to get feedback from my students about their learning experience. Then I saw the survey questions. Even though there were about 15 of them, none actually helped me improve the course. They were all extremely generic and left me scratching my head, unsure of what to do with the information. It’s not like I could ask follow-up questions or suggest improvements to the survey itself. Understandably, the institution used these evaluations for its own data points, and there wasn’t much chance of me influencing that process. So, I decided to take a different approach. What if I created my own informal course evaluations that were completely optional? In this survey, I could ask course-specific and teaching-style questions to figure out how to improve the course before the next run started. After several revisions, I came up with these questions: - Overall course rating (1–5 stars) - What was your favorite part (if any) of this course? - What did you find the least helpful (if any) during this course? - Please rate the relevancy of the learning materials (readings and videos) to your academic journey, career, or instructional design journey. (1 = not relevant at all, 10 = extremely relevant) - Please rate the relevancy of the learning activities and assessments to your academic journey, career, or instructional design journey. (1 = not relevant at all, 10 = extremely relevant) - Did you find my teaching style and feedback helpful for your assignments? - What suggestions do you have for improving the course (if any)? - Are there any other comments you'd like to share with me? I was—and still am—pleasantly surprised at how many students complete both the optional course survey and the official one. If you're looking for more meaningful feedback about your courses, I recommend giving this a try! This process has really helped me improve my learning experiences over time.