Lumen · Human-centered AI strategy
Designing a human-centered AI workflow for evidence-grounded strategy
I built a source-grounded research system from a corpus of approximately 680 analyst PDFs, then added critique gates, human override, partner prioritization, and a SharePoint-connected Copilot agent.
The challenge
Innovation and strategy teams needed to connect customer pain to product and partner decisions, but the evidence was scattered across analyst reports, internal documents, spreadsheets, and individual interpretation.
The immediate decision involved 106 potential partners. Leadership needed more than a ranked list. They needed to understand which customer problems a partnership could address, whether the solution was feasible, and where the business value came from.
The trust problem became clear when Copilot generated convincing quotations and citations that were not in the source material. I rejected the report, corrected it, and redesigned the workflow so that source access and claim support had to be proven before synthesis could continue.
The challenge was not getting AI to produce an answer. It was designing a workflow in which a person could inspect the evidence, challenge the reasoning, and decide whether the answer deserved attention.
The working system
Turn a large research corpus into reviewable evidence
I designed and iterated a multi-stage workflow in Copilot Cowork. Because each run could only handle a limited set of files, I tiered the corpus, checked that every PDF was readable, and used parallel extraction passes to create candidate pain-point records. Later stages developed segments, jobs to be done, confidence assessments, and partner matches.
- 01 Prove source access Tier files and stop when a source is unreadable
- 02 Extract candidates Create structured records with traceable evidence
- 03 Challenge claims Test grounding, scope, attribution, and duplicates
- 04 Support decisions Compare customer, solution, and business evidence
The reframing
A pain point became a record, not a sentence
The first version organized findings by analyst-defined domains. That made the material easier to sort, but it risked allowing the source taxonomy to define the strategy.
I shifted to a first-principles structure. Domains remained useful metadata, but no longer determined what counted as the problem.
Who is experiencing the problem?
In what context does it occur?
What are they trying to accomplish?
What prevents progress?
What happens when the problem persists?
What source supports the observation?
How strong and complete is the support?
Verified, contested, incomplete, or deferred?
Human-centered AI
Automation with an explicit responsibility boundary
I acted as the workflow designer and critic. AI handled repetitive extraction and comparison inside defined boundaries. I reviewed its behavior, identified failure modes, turned those failures into new gates, and retained responsibility for what entered the evidence base.
AI-assisted system
- Retrieve relevant evidence
- Extract candidate pain points
- Normalize and cluster records
- Identify uncertainty or conflict
- Surface capability and partner candidates
Human responsibility
- Define the question and scope
- Judge whether an extraction is meaningful
- Resolve ambiguous merges
- Approve strategic relationships
- Decide, act, and override
Validation
Challenging plausible answers before they influence strategy
I developed a courtroom-style method for the working Cowork loops. Three independent passes extracted candidate claims. Contested records then moved through five review lenses covering grounding, scope, taxonomy, attribution, and duplication.
- Extract
Run independent passes against the same supplied evidence.
- Challenge
Test whether each source supports the candidate claim.
- Adjudicate
Record an admit, reject, or defer-to-human state.
- Review
Inspect the rationale, resolve ambiguity, and override when needed.
Example: In one contested case, the workflow rejected seven plausible network pain points because the source covered an adjacent domain. One narrower claim was admitted only after a divided review. The disagreement was preserved instead of being hidden behind a confidence score.
The Cowork loops were runnable working workflows. The simplified diagram shown here is a public reconstruction of their behavior, not a production architecture diagram.
Decision framework
Connect customer evidence to partner strategy without hiding the tradeoffs
I translated the evidence base into a prioritization model for the 106-partner universe. I kept value and feasibility separate so a high revenue estimate could not erase a weak customer case or an unresolved implementation constraint.
The model grouped candidates into build waves instead of forcing a false ranking from one to 106. I also tested it against a simpler revenue-and-buildability baseline. When a stronger customer lens changed the first wave, that sensitivity became part of the recommendation instead of something to smooth over.
Example: One proposed fit looked strong until I checked the technical requirements. The partner supported public APIs but not the private connectivity the concept required. I rejected the original match, reframed the customer value, and moved it to a lower-feasibility strategic bet.
AI experience
Put the agent downstream of evidence and inside a larger workflow
I configured a Copilot Studio agent using eight domain-specific knowledge documents in SharePoint. I tested it with grounded questions, published it to my immediate team, and connected it to a Principal Architect's technical-compatibility agent.
The architect's agent assessed whether a company was technically compatible with Lumen products. When the question fell within one of my covered domains, my agent added evidence about the end-user problem. This separated technical fit from customer need while allowing the two agents to support the same decision.
What evidence supports this customer problem, and is this partner a credible fit?
- Technical compatibility from the architect's agent
- Relevant customer-pain evidence from my agent
- Source references and covered domain
- Uncertainty and missing evidence
- Questions that require human judgment
I configured the connection and owned the customer-evidence agent. The Principal Architect owned the technical-compatibility logic. A planned migration from the eight SharePoint documents to the more mature evidence workbook was not completed.
Outcome
A working foundation, with clear limits on what I can claim
The work produced a reusable research workflow, adjudication records, a three-lens partner model, and a published team agent. The completed prioritization was intended to support a Senior Director of Core Strategy's review of the 106-partner universe.
I was laid off before presenting the final recommendation, so I do not claim that it changed leadership priorities. The research used secondary analyst and company sources, not customer interviews, surveys, sales data, support records, or product telemetry. No measured reduction in hallucinations, time savings, revenue impact, or production adoption is claimed.
What I would test next
Evaluate decision quality and design the path to adoption
I would create a representative evaluation set covering well-supported questions, conflicting evidence, questions with no answer, cross-domain synthesis, and likely false-positive matches. I would pair that with a staged rollout, reviewer guidance, feedback capture, and a named operational owner.
- Measure citation correctness and unsupported-claim rate.
- Evaluate retrieval relevance, duplicates, and contradictions.
- Track reviewer corrections and appropriate defer-to-human behavior.
- Run governance, security, accessibility, and failure-recovery reviews.
- Test whether the workflow improves a real strategy decision.
Takeaway
The core design problem was deciding where AI should stop.
This project brought together the work I want to keep doing: finding the right use for AI, translating evidence into a usable workflow, designing critique and escalation paths, and helping people make better decisions without giving up responsibility for them.