← All AI UX work

02 · Lumen

Checking AI-generated partner research against its sources.

Lumen’s innovation team had 106 potential partners to evaluate. I built the research workflow, and when an early AI-generated report cited sources I couldn’t verify, I changed it so every finding needed a passage from its source. I reorganized the partner workbook around a Core Strategy director’s questions about customer need, product fit and the value to each side. Later, I built an evidence agent in Copilot Studio for other teams to use.

My role
Lead Innovation Service Designer. I initiated and owned the research workflow, built the evidence agent and restructured the partner-analysis workbook.
Working with
Core Strategy, innovation leaders, partner teams and a principal architect.
Stage
Working research workflow and an evidence agent published to my immediate team.

Stakeholder feedback

Why I reorganized the partner workbook.

Leadership supplied 106 potential partners. I initiated the research workflow to help Core Strategy examine which partnerships could address a customer problem, through what combination of products and with what business value.

A Senior Director on the Core Strategy team asked for more than a partner match: why would the partnership matter, what would each side gain, and which products could work together? I reorganized the workbook around those questions, using Copilot to help expand it.

How the analysis changed

Customer
Connect the proposed partnership to a documented customer pain.
Solution
Make the possible product combination and its feasibility explicit.
Business
Explain the value to Lumen and to the partner, alongside the business levers.

Summary of the revised workbook structure. The product combinations remained hypotheses.

I kept the owner’s original priorities beside the analysis but left them out of the scoring, so Core Strategy could compare those priorities with what the research showed.

Where the workbook records these changes

The partner workbook’s June 24–29 changelog records the matching additions to solutioning, partner value and business fields, followed by the Customer / Solution / Business structure, made with Copilot working on my behalf.

Source locations: partner hypothesis workbook, Changelog rows 2, 4–5 and 7–8; Cover cells C25 and C32 for the owner-priority overlay. The explanation above reformats those records for this case.

Who it was for

How the different teams would use the research.

The goal was an AI agent built on the final workbook, so three groups could draw on the same evidence instead of starting from scratch.

Intended users

Core Strategy
Analyze further and build an evidence-backed rationale for prioritizing the 106 potential partners.
Innovation executives
Ask what to explore with a partner: how Lumen and the partner could work together and which products to focus on. They would still make the final decision, but the research would give them a more specific place to start.
Go-to-market
The team already used Copilot research to build marketing personas. The agent would be one more source in that routine, adding customer pain from more than 600 sources, including IDC, Frost & Sullivan, Forrester and Gartner reports that general research tools often miss behind paywalls.

These were the intended uses. The agent reached my immediate team before my role ended; see Results and adoption.

Failure changed the design

I couldn’t find the AI’s quotes in the reports it cited.

After presenting an early AI-generated report, I checked its quotes and page references the next day and could not verify them. I rejected the report, told stakeholders it had hallucinated and sent a source-checked replacement.

I moved the first checkpoint ahead of extraction: could the AI actually read the source? A candidate pain point then needed an inspectable passage and location before entering the library. Later review checked support, scope, classification, attribution and duplication.

I kept the courtroom method in a separate Markdown file and instructed the AI to use it as part of the research process.

One recorded review

Seven rejected candidates. One disputed finding.

The preserved logs show how one report was reviewed for a specific question: did it contain customer pain relevant to traffic between services or connectivity across clouds? General networking problems could sound relevant without meeting that scope.

Preserved evidence · src_006a358b3b

From source to decision

Selected records from the saved research logs, reformatted for this case study.

Source and passage
IDC PlanScape: Managed Connectivity Services, September 2023, page 3, Figure 2. The library records the chart label “Aligning network infrastructure with cloud connectivity requirements.”
Candidate finding
Enterprises had difficulty aligning network infrastructure with cloud connectivity. This was an analyst survey finding.
Scope check
Seven other candidates from the report were rejected. For example, a general hybrid-network management problem did not specifically establish traffic-between-services or cross-cloud connectivity pain.
AI review decision
UPHELD 4–1 · DISSENT RECORDED

Four AI review roles accepted the cloud-connectivity finding within the multicloud-gateway domain. One objected that it could describe a generic single-cloud problem. The entry was retained with that objection.

Locate the original records

PainPoints_MASTER_Matrix: “WAN Pains,” row 13, contains the finding, source passage and page. “Rejection Log,” rows 6–12, records seven rejected candidates and their reasons. “Decision Log & Scope,” row 3, preserves the majority decision and minority objection.

The passage check and the scope judgment answered different questions. A passage could be traceable and still be a weak fit for the strategy question. Keeping the objection beside the retained finding gave the next reviewer a specific issue to resolve.

Interaction model

Where the AI hands off to a person.

01

Capture

AI extracts a candidate pain point with source text and location.

02

Challenge

Separate passes check support, scope, attribution and duplicates.

03

Resolve

Admit, reject, contest or defer. Keep disagreement visible.

04

Decide

A person resolves ambiguity and owns the strategic call.

Reconstructed workflow overview. The workbook preserves AI review decisions; the method document specifies human escalation.

A useful evidence record

What each record contained.

Person or role · situation · friction · exact source passage · location · confidence · review reason.

A stopping condition

When the evidence didn’t support a partner idea.

The workflow could return no result or defer to a person. It did not have to manufacture a recommendation.

What the preserved workflow rules specify

The pain-point method defines three extraction lenses and five review roles. The roles check different questions.

  1. Keep the reason. Admitted findings retain a source quote and classification. Rejected findings retain the reason for rejection.
  2. Escalate ambiguity. Unverified attribution, unclear classification or an editorial scope question goes to a human review queue.
  3. Resume from a checkpoint. The method calls for batches of up to 20 sources, a saved resume record and one writer merging into the master library.

Source: Courtroom Method: Pain-Point Matrix, headings Role map, End states, Workflow, Guardrails and Status.

Working with engineering

How my evidence agent worked with the architect’s agent.

The principal architect and I discussed how the agents could work together. His agent assessed how a company fit a Lumen product; mine supplied customer-pain context. I built the evidence agent in Copilot Studio, grounded it in eight domain-specific Word documents through SharePoint, tested questions against that material and configured the connection.

For companies in one of my eight researched domains, his agent called mine for end-user pain context. He owned the technical-matching logic; I owned the evidence agent and its connection. The separate research workflow ran in Copilot Cowork.

Results and adoption

Delivered to my team, with a wider presentation planned.

The work produced a source-linked research library, review and rejection logs, a repeatable workflow method and a 106-partner hypothesis workbook. I published the Studio agent to my immediate team.

My manager supported presenting the workflow more broadly; my role ended before that presentation. Before a rollout like that, I would start from the team’s recurring tasks and incentives, so the agent supports work people already need to finish.

Proposed next step

Test an internal task before expanding to clients.

For a future pilot, I would begin with one recurring task the intended team needs to complete now. The employee doing the work, the accountable decision-maker and a technical partner would shape the test together.

  1. Understand the existing work. Observe the task, its exceptions and the handoffs between roles. Agree where AI can assist, who can accept or challenge an output, and where a person must take over. Check whether review adds work the team has time and authority to do.
  2. Follow a task through its unresolved questions. Test whether people can inspect support, correct an interpretation and return to a deferred decision. Gather feedback in an existing team routine, revise what blocks them, and look for repeat use and useful task completion alongside errors and extra review work.
  3. Recheck the context for each client. Confirm the client’s task, language, incentives, decision authority, source access and support owner. Reuse only the parts that still fit, with a way to stop or return to a person when they do not.

A proposed approach based on what I learned.

Separate project · Sales adoption

A solution menu for existing sales meetings.

In Lumen’s Connected Ecosystem work, one-to-one sales pitches spent too much time explaining the initiative. After attending a sales call, I convened Business Development leads, service designers and principal architects from two pods. I created a menu of technically validated solutions organized by customer problem and fit, then proposed taking it to existing sales-team meetings.

The architects owned technical validation. The Business Development Lead presented the menu and managed rep follow-up. Partnership sales still did not count toward rep quotas, so the incentive problem remained.

Next: search experiments at Costco Travel →

Ariel Koh · Seattle, Washington

Interested in working together?

© 2026 Ariel KohAI experience design · Research · Service design