01 — Work
Emerging Technology Vendor Evaluation
Built the scoring, governance, and stakeholder-feedback system behind a board-level platform evaluation — and used AI for vendor diligence with a verification step that caught what it got wrong.
Challenge
Sunward was evaluating whether to introduce a new member-facing financial capability through a third-party platform. The opportunity was strategically significant but unfamiliar, the timeline was compressed, and new requirements kept surfacing as stakeholders learned more. Senior stakeholders didn’t agree on what made one platform stronger than another — some prioritized simplicity for everyday members, others valued advanced capability and long-term flexibility. The challenge wasn’t comparing features. It was building a decision process that could preserve disagreement, surface non-negotiable risks, and produce a recommendation that would withstand executive and board scrutiny.
My role
I led the product-management structure for the evaluation: defining what a successful platform needed to accomplish, translating strategic priorities into measurable criteria, designing the scorecard and its weights and advancement rules, keeping quantitative scoring separate from qualitative feedback, coordinating input across Product, Technology, Compliance, Risk, Digital, Operations, and executive leadership, and validating internal conclusions against outside technical and market information.
What I discovered
The same demonstration produced different, equally valid conclusions
One group saw an advanced platform as a strategic differentiator; another saw the same platform as too complex for everyday members and hard for frontline teams to support. Both were right.
Aggregate scores could hide the risks that mattered most
A strong total could conceal an unacceptable weakness in a high-priority area, and presentation quality quietly influenced perception. Numeric scoring alone would have flattened legitimate strategic disagreement.
The process needed two tools, not one
A quantitative scorecard for consistent comparison, and a qualitative instrument for judgment and concern. Neither could substitute for the other.
What I created
- A weighted vendor scorecard
- Clearly defined evaluation criteria
- Minimum advancement requirements and disqualifying rules for high-priority criteria
- A standalone qualitative stakeholder-feedback framework
- A tracker for unresolved technical and operational questions
- Governance and participation rules
- External validation of integration and implementation assumptions
- An evaluation cadence for demonstrations, deep dives, feedback, and decision checkpoints
Artifacts from this work
Feedback workflow — representative
Vendor demonstrationStakeholder observationsHard concerns flaggedFollow-up questionsScorecard & diligence updatesQualitative evaluation framework and intake formRepresentative reconstruction Stakeholder Vendor Feedback Framework
The qualitative instrument that ran alongside the scorecard — framework, and the intake form stakeholders actually filled in.
View artifactAI-assisted diligence — representative
Evidence gaps identified
- Regulatory readiness — unclear from materials provided
- Implementation support — timeline not substantiated
Follow-up questions developed
- Request documentation on regulatory review status
- Confirm implementation support model and timeline
Reviewed by Alicia — refined before use in diligence
AI-assisted diligence briefRepresentative reconstruction AI-Assisted Vendor Diligence Brief
A structured research brief used to identify evidence gaps and develop deeper follow-up questions for finalist vendors.
View artifact
How the evaluation worked
The scorecard assessed member experience, platform capability, technical integration, data and member protection, regulatory readiness, security, vendor health, implementation support, commercial model, and strategic differentiation. Advancement required both a minimum total and passing every high-priority criterion, so a strong aggregate score couldn’t carry a platform past a critical failure. Qualitative feedback stayed deliberately separate, so stakeholder judgment was never compressed into an average and disagreement stayed visible. Diligence didn’t rely on vendor demonstrations alone: integration paths, dependencies, data handling, and commercial assumptions were checked against an independent external reference. AI supported synthesis and question development, never selection — it drafted the diligence brief, I verified it, cut what the source material didn’t support, and turned the gaps into vendor follow-up questions.
Outcome
The evaluation turned a collection of demonstrations, documents, and opinions into a structured decision system — one that stayed usable as requirements changed, technical questions emerged, and new stakeholders joined. It gave leadership a defensible basis for comparing platforms in an unfamiliar category, with transparent treatment of disagreement and a governance model suited to executive and board review.
Capabilities demonstrated
- Emerging product strategy
- Vendor evaluation
- Decision framework design
- Product governance
- Executive decision support
- Technical diligence
- AI-assisted research
- Cross-functional leadership
Confidentiality Vendor names, internal discussions, and procedural detail are generalized; no internal identifiers, names, or vendor-specific information are disclosed.