User Task Analysis Techniques

Readers responded with both surprise and agreement last week when I wrote that the single biggest predictor of how rapidly a team makes progress building an AI agent lay in their ability to drive a disciplined process for evals (measuring the system’s performance) and error analysis (identifying the causes of errors). It’s tempting to shortcut these processes and to quickly attempt fixes to mistakes rather than slowing down to identify the root causes. But evals and error analysis can lead to much faster progress. In this first of a two-part letter, I’ll share some best practices for finding and addressing issues in agentic systems. Even though error analysis has long been an important part of building supervised learning systems, it is still underappreciated compared to, say, using the latest and buzziest tools. Identifying the root causes of particular kinds of errors might seem “boring,” but it pays off! If you are not yet persuaded that error analysis is important, permit me to point out:  - To master a composition on a musical instrument, you don’t only play the same piece from start to end. Instead, you identify where you’re stumbling and practice those parts more. - To be healthy, you don’t just build your diet around the latest nutrition fads. You also ask your doctor about your bloodwork to see if anything is amiss. (I did this last month and am happy to report I’m in good health! 😃) - To improve your sports team’s performance, you don’t just practice trick shots. Instead, you review game films to spot gaps and then address them. To improve your agentic AI system, don’t just stack up the latest buzzy techniques that just went viral on social media (though I find it fun to experiment with buzzy AI techniques as much as the next person!). Instead, use error analysis to figure out where it’s falling short, and focus on that. Before analyzing errors, we first have to decide what is an error. So the first step is to put in evals. I’ll focus on that for the remainder of this letter and discuss error analysis next week. If you are using supervised learning to train a binary classifier, the number of ways the algorithm could make a mistake is limited. It could output 0 instead of 1, or vice versa. There is also a handful of standard metrics like accuracy, precision, recall, F1, ROC, etc. that apply to many problems. So as long as you know the test distribution, evals are relatively straightforward, and much of the work of error analysis lies in identifying what types of input an algorithm fails on, which also leads to data-centric AI techniques for acquiring more data to augment the algorithm in areas where it’s weak. With generative AI, a lot of intuitions from evals and error analysis of supervised learning carry over — history doesn’t repeat itself, but it rhymes. [Truncated due to length limit. Full text: https://lnkd.in/gjqv6VeA ]

Every product manager is racing to ship AI features. But here's what nobody talks about: most ship broken, get fixed quietly, or die slowly. The difference between shipping and shipping something that works? Evals. An eval = systematic way to measure if your AI output is actually good. If you want an AI feature that actually works for real users (not just in demos), evals are the most important thing you need to learn. These insight comes from Hamza Husein (ex-OpenAI, ex-Airbnb) and Shrea Shanker (ex-Atlassian, ex-GitHub), two of the sharpest minds in AI product management. Here’s a simple 5-step framework to get started 👇 1️/ Start with Error Analysis Generate 50 diverse outputs For each answer this: "Would I ship this? Yes or No?" For every "No," write why in 1-2 sentences Output: A list of 5 -10 recurring failure patterns. 2️/ Find Your Failure Modes Group similar errors together. Give each a clear name and note how often it appears. Example: Hallucination (12), Wrong Tone (18), Missing Context (8) Stop when you’ve reviewed around 20 more outputs without discovering any new failure types. Output: 3-5 named failure modes with counts 3/ Build Binary Rubrics Turn your top 3 failure modes into clear rubrics For each, define: → A pass/fail rule (no 1–5 ratings) → 3 examples of PASS → 3 examples of FAIL Example - Hallucination: PASS: Every fact is verifiable or clearly marked as inference. FAIL: Any unverifiable or made-up fact. Output: 3 rubrics with examples that define your quality bar. 4/ Test for Alignment Take 20 new outputs. You and a teammate score them independently using your pass/fail rules. Then calculate → (number of agreements) / 20. Target: 80 % + agreement. Below that? Your rubric is unclear. Refine the definitions or examples and test again. Output: Rubrics you can trust across the team. 5/ Diagnose & Fix with the Three Gulfs Now that you know your failure modes, it’s time to diagnose why they’re happening. There are only three reasons your AI feature isn’t working and each needs a completely different fix: Gulf #1 — Specification Problem → Fix with better prompting (days to fix) Gulf #2 — Knowledge Problem → Fix with RAG or retrieval (weeks to fix) Gulf #3 — Capability Problem → Fix with better models or fine-tuning (months to fix) Most teams reach for the wrong solution. In reality, 80% of problems are Gulf #1 (specification) but teams jump straight to Gulf #3 (fine-tuning) way too early. I’ll break down the complete Three Gulfs Framework with detailed examples and fixes in my upcomig posts. It’s dense enough to deserve its own deep dive. Liked this breakdown? Follow + Save for more no-fluff posts on how to build AI features that actually work.

Evaluating LLMs is hard. Evaluating agents is even harder. This is one of the most common challenges I see when teams move from using LLMs in isolation to deploying agents that act over time, use tools, interact with APIs, and coordinate across roles. These systems make a series of decisions, not just a single prediction. As a result, success or failure depends on more than whether the final answer is correct. Despite this, many teams still rely on basic task success metrics or manual reviews. Some build internal evaluation dashboards, but most of these efforts are narrowly scoped and miss the bigger picture. Observability tools exist, but they are not enough on their own. Google’s ADK telemetry provides traces of tool use and reasoning chains. LangSmith gives structured logging for LangChain-based workflows. Frameworks like CrewAI, AutoGen, and OpenAgents expose role-specific actions and memory updates. These are helpful for debugging, but they do not tell you how well the agent performed across dimensions like coordination, learning, or adaptability. Two recent research directions offer much-needed structure. One proposes breaking down agent evaluation into behavioral components like plan quality, adaptability, and inter-agent coordination. Another argues for longitudinal tracking, focusing on how agents evolve over time, whether they drift or stabilize, and whether they generalize or forget. If you are evaluating agents today, here are the most important criteria to measure: • 𝗧𝗮𝘀𝗸 𝘀𝘂𝗰𝗰𝗲𝘀𝘀: Did the agent complete the task, and was the outcome verifiable? • 𝗣𝗹𝗮𝗻 𝗾𝘂𝗮𝗹𝗶𝘁𝘆: Was the initial strategy reasonable and efficient? • 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻: Did the agent handle tool failures, retry intelligently, or escalate when needed? • 𝗠𝗲𝗺𝗼𝗿𝘆 𝘂𝘀𝗮𝗴𝗲: Was memory referenced meaningfully, or ignored? • 𝗖𝗼𝗼𝗿𝗱𝗶𝗻𝗮𝘁𝗶𝗼𝗻 (𝗳𝗼𝗿 𝗺𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝘀𝘆𝘀𝘁𝗲𝗺𝘀): Did agents delegate, share information, and avoid redundancy? • 𝗦𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗼𝘃𝗲𝗿 𝘁𝗶𝗺𝗲: Did behavior remain consistent across runs or drift unpredictably? For adaptive agents or those in production, this becomes even more critical. Evaluation systems should be time-aware, tracking changes in behavior, error rates, and success patterns over time. Static accuracy alone will not explain why an agent performs well one day and fails the next. Structured evaluation is not just about dashboards. It is the foundation for improving agent design. Without clear signals, you cannot diagnose whether failure came from the LLM, the plan, the tool, or the orchestration logic. If your agents are planning, adapting, or coordinating across steps or roles, now is the time to move past simple correctness checks and build a robust, multi-dimensional evaluation framework. It is the only way to scale intelligent behavior with confidence.

A big mistake teams make with AI is trying to automate the one step that requires your unique expertise: **looking at your data.** It’s tempting to plug in an off-the-shelf tool to tell you what’s wrong. But this is where teams get led astray. Generic evaluators often lack key business context and won't spot the domain-specific issues that kill trust. Even worse, they'll have you chasing the wrong metrics and waste precious engineering time. Building reliable AI isn't luck; it's a science. And the scientific method for evals is as follows: 1. Sample, Don't Boil the Ocean. You don't need to read every trace. Start with a representative sample of ~100 to find initial patterns. 2. Build Intuition. Read the traces yourself. Your goal isn't just to find errors, but to understand *how* your system is failing in your specific context. This is the highest ROI activity in all of AI engineering. 3. Define the Failures. Systematically categorize the errors you find. This process, called error analysis, turns a messy problem into a structured taxonomy of failures that are specific to your application. 4. Automate with Purpose. *Now* you can build evaluators. Once you know the specific failure modes you're looking for, you can use LLMs or code to find more instances at scale. This process ensures your evaluations are grounded in real application behaviors, not counter-productive generic metrics. The September cohort of AI Evals for Engineers & PMs is now open: https://lnkd.in/g8ApB2AF

My obsession with Human factors (HF)in safety continues... An example of HF in a root cause. Task: Drain solvent into a drum. A completely manual operation. The task is done by opening and keeping the valve open by an operator. If the operator leaves the handle, valve goes to a close mode. A well established and risk assessed procedure. Incident: The drum overflowed leading to a big spill. Root cause: The valve was kept open by the operator. HUMAN ERROR. A natural selection and the easiest way to close the RCA. Now let us apply Human factor on this incident. It takes 20 mins to fill the drum. He has to fill 20 drums. During the fill he has to oversee the filling standing there, to avoid spill. A stupid monotonous job! If the job is monotonous and not value adding, people will find alternate means to ease out the job. This operator tied the handle of the valve with a rope and started doing another activity thinking that he can be back in 20 mins. This did not happen and solvent started overflowing. Now step into his shoes and think. Would you have done it differently? All of us wants to add value to our job. Monotonous task sucks. On the hindsight it is always easy to say that guy did not follow the procedure. It is the soft end of the incident. The blunt end is the organisation. Did organisation think of a better way to do this by redesigning the process? Monotonous task, inclement weather, inadequate workspace, work pressure, bad design etc etc are all factors that can lead to Human errors. Unless we reach there (the blunt end) root cause is incomplete. This is Human factor simplified !

Most AI agents fail in production not because of poor models, but because of poor debugging. I was listening to Andrew Ng's episode on the No Priors podcast, and he raised a critical point: the biggest barrier to successful agentic workflows isn't technical complexity, it's the lack of systematic error analysis skills. While many can whip up a basic agent, the ability to effectively debug and improve complex agentic workflows through a structured process is a different ballgame. Thinking about it, the approach closely mirrors what Site Reliability Engineers (SREs) do when tackling incidents. A simplified systematic error analysis process for AI agents could look something like this: ⚙️ Observe the Failure: Identify what went wrong (users getting incorrect output from your ecommerce agent? Maybe there's a task failure with booking a flight? Or perhaps people really hate your sales funnel agent and show it in the feedback popups?) ⚙️ Isolate the Component: Pinpoint which part of the agent's workflow is likely responsible: planner, tools, retrieval, generation? (is your flight booking agent failing because the planner can't handle multi-city trips, or because the booking tool keeps timing out, or because retrieval is pulling outdated airline data?) ⚙️Root Cause Analysis: Dig deep to understand why it failed (your ecommerce agent recommending winter coats in July because the prompt doesn't include seasonal context, or your booking agent failing because the API schema changed and wasn't updated in the tool description) ⚙️Assess Impact: Understand the severity and frequency of the error (is this affecting 2% of users or 40%? Are these edge cases or core functionality failures? Is it losing you customers or just causing minor frustration?) Implement & Evaluate: Develop a fix and rigorously test its effectiveness (adding seasonal awareness to your product recommendation prompts, updating API documentation, or implementing proper error handling with graceful fallbacks) This kind of systematic approach isn't often a core skill in initial AI development but is crucial for building robust and reliable AI agents. #AI #AIAgents #MachineLearning #SystematicErrorAnalysis #SRE #AIEngineer #AndrewNg #CareerDevelopment #Tech

Why Do 50% of Autonomous Agent Tasks Fail? A Study Reveals Systemic Weaknesses 👉 Why This Matters Autonomous agents built on LLMs promise to automate complex workflows, but what happens when they silently fail? Current evaluations focus on success rates, leaving developers in the dark about "why" agents struggle. This gap limits progress: Without understanding failure modes, we can’t build reliable systems. 👉 What the Study Uncovered Researchers evaluated 3 popular frameworks (TaskWeaver, MetaGPT, AutoGen) with GPT-4o and GPT-4o mini on 34 real-world tasks like web scraping and data analysis. Key findings: - 50% average failure rate across tasks - Web crawling had the lowest success (16.7% with GPT-4o) due to complex reasoning requirements - Smaller models (GPT-4o mini) sometimes outperformed GPT-4o by avoiding “overthinking” traps Failure Taxonomy 1. Planning Errors   - Unnecessary confirmation loops (“Should I use linear regression?”)   - Impossible sub-task sequences   - Infinite replanning without progress 2. Execution Failures   - Code with syntax errors or incorrect API calls   - Environment issues (missing dependencies, file paths) 3. Response Breakdowns   - Formatting mismatches (e.g., returning text instead of numbers)   - Context window overflow in large tasks 👉 How to Fix It The authors propose actionable solutions: 1. Feedback-Driven Planning: Agents that dynamically adjust strategies based on execution results, avoiding rigid workflows. 2. Early-Stop Mechanisms: Detect repetitive errors and halt tasks before resource waste. 3. Tool-Aware Code Generation: Better integration of API/documentation context to reduce incorrect tool usage. Practical Takeaways - Structured tasks (data analysis, file ops) work best—agents achieved 75% success here. - Avoid “max iterations” as a crutch: Performance plateaus after ~10 attempts. - Smaller models can outperform larger ones in constrained scenarios. The team open-sourced their benchmark for reproducibility. For developers, this work provides a roadmap to build agents that fail less and adapt faster. Question to the community: How are you addressing reliability gaps in your AI agent systems?