The Home Depot: Building an AI Knowledge Retrieval Agent
Role: Analyst · Timeline: 3 months · Outcome: Adopted by two teams; one place to search research, test results, and processes
Project Context
The Home Services content team made decisions based on years of research: behavioral analysis of how customers used our service pages and forms, results and findings from past A/B tests, performance benchmarks, UX research, and documented team processes. But it lived in PowerPoint decks and documents scattered across shared folders. Finding a single result meant opening 3–5 decks and scanning slides, and every teammate had to repeat that search on their own. I wanted to remove that step entirely.
My Role
I identified the need, scoped the solution, secured buy-in, wrote the agent’s requirements and instructions, built it in Microsoft Copilot Studio, tested it, and led the rollout.
Timeline
About 3 months, from scoping through launch
Tools Leveraged
Microsoft Copilot Studio, Microsoft Teams, OneDrive, Microsoft Loop
The Problem
Finding one result took 3–5 decks. Behavioral findings, test results, benchmarks, research, and process documentation were spread across many files, with no way to search across them.
Everyone repeated the same search. Each teammate had to dig through the same decks on their own, or track down someone who knew where to find the information.
Past learnings were underused. When findings are hard to find, teams risk repeating tests and research, or missing what already worked. When someone left the team, their knowledge of where things lived left with them.
The Approach
Choosing The Platform
The team’s files already lived in SharePoint and OneDrive, and the team worked in Microsoft Teams every day. I built the agent in Microsoft Copilot Studio so it could connect to those files where they already lived, without moving anything, and so the team could use it inside Teams instead of learning a new tool.
Scoping a Phased Roadmap
I scoped the hub as the first phase of a multi-agent system. Later phases would add specialist agents, like a content brief generator, an onboarding agent, and a test planning agent, with the hub routing each question to the right one through A2A. The system would also connect to live data and tools like Mixpanel, Adobe Target, and Jira through MCP, so answers wouldn’t rely only on archived files. Each specialist agent was designed to support the team, by drafting briefs, suggesting tests, and give teammates the ability to create or modify Jira tickets in one place. I started with a single retrieval agent to prove the value and earn adoption before adding complexity.
Securing Buy-In
I presented the pilot to my manager, who approved it. After I presented it to the team, my manager and the director suggested connecting UX research as well. I gave the UX research lead a live demo, he approved it, and I connected the research repository as a knowledge source, so the team could find qualitative research alongside performance data.
Writing The Requirements
I wrote the agent’s instructions like a requirements spec, defining:
Sources: only the team’s approved files, covering five kinds of knowledge:
Behavioral analysis: form and webpage performance findings
A/B test results: outcomes and findings from tests we’d run in the past
Benchmarks: performance baselines to compare against
UX research: findings from the UX research repository
Content documentation: team processes and how work gets done
Reporting rules: A/B results always in a table with Control, Variant, Lift, and Winner, plus the season and content version
Explainability: every answer cites its source file, link, and slide, so people can verify it
Guardrails: never estimate missing metrics, never claim statistical significance unless the source states it, and never blend data from different files without labeling each source
I wrote the first version myself, then used AI to help strengthen the failure handling.
Edge Cases and Guardrails
The most important part of the design was deciding what the agent does when it can’t give a clean answer.
Testing & Launch
Testing
Before launch, I tested the agent in three ways:
Accuracy: asked questions with known answers and checked responses against the source files
Guardrails: asked questions where the data didn’t exist, to confirm it said so instead of guessing
Usability: had my manager use it on her own. She found what she needed without my help.
Launch and Adoption
I published the agent to Microsoft Teams, so it lived where the team already worked. I also set up a Loop workspace as the pilot hub, with a library of example prompts and weekly updates based on teammate feedback.
Example Questions The Team Could Ask
I built a prompt library in the pilot workspace so teammates knew where to start. A few examples:
A/B test results: “Has our team ever tested a shorter headline on the HVAC page? If so, what were the results?”
Comparing tests: “Show me all A/B tests for Window Installation in 2025. Summarize the winning and losing tests.”
Post-launch performance: “How did the single-step lead form perform after launch compared to the previous version?”
Seasonal context: “What was the visit-to-lead rate for HVAC pages last summer?”
UX research: “What did research find about how homeowners compare HVAC providers?”
Team processes: “What’s the process for requesting a new A/B test?”
Outcome & Business Impact
The hub was adopted by approximately 20 users across the content and UX research teams. Instead of opening 3–5 decks, teammates could ask one question in Teams and get the answer with its source, whether they needed behavioral findings, past A/B test results, benchmarks, UX research, or process documentation. Every answer pointed back to its source, so the team could trust and verify what it told them.
Connecting the research repository also brought qualitative and quantitative findings together for the first time, so the team could see why customers behaved a certain way alongside what the numbers showed.
What I’d Do Next
Connect live data through MCP: Connect the hub to live analytics and testing tools like Mixpanel, Adobe Target, and Jira through MCP, so answers reflect current performance, in addition to archived findings.
Expand into a multi-agent system with A2A: Build the specialist agents from the original roadmap, like a content brief generator and a test planning agent, and connect them using A2A, so the hub can route each question to the right specialist.
Formalize evaluation as it grows: Build a standard set of test questions and rerun it whenever agents, sources, or instructions change, so quality keeps up with the system.