Summary
Critical issues
C1 — Case study depth is near-zero
All three case study pages (AI Portfolio, Synthetic Users, Autodesk Planning System) are stubs — 3–5 sentences each. They have titles, taglines, and one short paragraph, then end. No process, no artifacts, no decisions, no tradeoffs, no what-failed. This is the most consequential structural failure in the portfolio.
C2 — Role and scope are ambiguous at the hero level
"I lead CX, product, and service transformation" is a positioning statement, not a role statement. It's unclear whether Matt is a Principal IC, a Director, a Head of, or a VP-equivalent. This creates friction for recruiters and hiring managers in the first 10 seconds, especially without a title, team size, or reporting level anywhere on the homepage or About page.
C3 — $50M metric has no denominator, no attribution, and no timeframe context
The homepage states "$50M+ YOY AOV driven through service design leadership." The Impact page frames it as "Monetized AOV engine · 12 mo." But Matt's specific contribution vs. org-wide motion is never established. No baseline, no comparison year, no indication whether Matt led, contributed to, or influenced this number. Skeptics will dismiss it without that frame.
Major issues
M1 — No evidence of backstage/frontstage architecture in Autodesk work
The Autodesk work is framed in outcomes (service tiers, revenue lift) but the case study stub doesn't show the system architecture — who does what backstage to make the three-tier model real. Service blueprinting, org behavior change, internal capability gaps: none visible.
M2 — Synthetic Users case study has no methodology shown
The page acknowledges the right framing ("complement to research, not a replacement") but shows nothing. No persona construction logic, no hypothesis format, no sample outputs, no edge cases, no failure modes. It's a description of the concept, not the method.
M3 — AI Portfolio case study is opaque about the human/AI split
"AI supports drafting, variation, and refinement; the narrative and IA remain intentionally human-led." This is the correct framing but it's a claim, not a demonstration. Which sections were AI-drafted? What did Matt override? Where did the AI get it wrong? Without that, the case study reads as defensive rather than transparent.
M4 — "15 services shipped" is an output metric, not an outcome metric
The Impact page lists "15+ Services shipped" under Autodesk. Shipping services is an activity. The outcome is whether customers adopted them, which tiers had retention, and which drove NRR. This metric needs to be replaced or contextualized against an adoption or retention signal.
M5 — No failures, pivots, or constraints mentioned anywhere
The portfolio reads as a sequence of wins. Every metric is positive. No project failed, stalled, or changed course. This is the single biggest "polished but shallow" trigger for senior design reviewers and hiring leaders who have seen a lot of portfolios.
Minor issues
mn1 — "Signal → Story" section has no clear purpose for hiring audiences
The short films are distinctive and show craft range. But a recruiter or VP Product doesn't immediately understand why it's here or what it signals about Matt as a design leader. One line of intentional framing would resolve this.
mn2 — EY vaccination metric scale mismatch
4.57M engagements and 715 vaccinations sit side by side without explanation of the conversion gap. At a glance this looks like a very low conversion rate. The program logic (engagements were awareness, vaccinations were a separate downstream activation) needs a one-sentence bridge.
mn3 — About page aerospace metaphor runs long
The launch pad / mission / telemetry framing is original and visually interesting. But it's applied to every section, including the career arc, which makes "Operator Log" a strained metaphor for a work history. The framework earns its place in the process section; it doesn't need to govern the entire About page.
mn4 — No LinkedIn or contact method visible on the homepage
Contact lives at the bottom footer only. For someone who closes tabs quickly, there's no low-friction next step that doesn't require scrolling to the end.
Positive signals
What's working
1. The three-tier service model visualization on Impact is the strongest systems-thinking artifact on the site — it shows the architecture, not just the outcome.
2. "Outcomes that compound" in the homepage subhead is excellent. It's a distinct, defensible positioning phrase that doesn't read as AI-generated.
3. The Impact console UX — framed as a "live briefing" — is genuinely surprising and memorable. It's the one moment most reviewers will pause at.
4. The Wipro before/after handoff table (6.8 → 4.7 days, structured payload, tier ownership) is the clearest cause-and-effect design story on the site.
5. The "even over" statements on About (clarity even over comfort, progress even over perfection) read as genuine values, not LinkedIn platitudes.
6. Stanford AI certification prominently placed signals seriousness about the tools without overclaiming.
Prioritized change list
1
Expand all three case study stubs into real studies. Each needs: problem framing, constraints, what Matt did (not the team), key decision moments, one thing that didn't work, and outcome with attribution. Target: 600–900 words each minimum.
2
Add role clarity to the homepage hero. One line: something like "Product & Service Design Leader — IC to director-level scope, enterprise clients." This removes the recruiter's first friction point.
3
Give $50M a denominator. Add a brief attribution frame: "as the design and strategy lead for..." + the baseline year and what shifted. One sentence suffices.
4
Replace "15 services shipped" with an adoption or retention metric. E.g., "% of eligible customers onboarded to a paid tier within 90 days" or "tier-1 attach rate in first 6 months."
5
Add one honest failure or pivot to at least one case study. It doesn't need to be dramatic — a hypothesis that was wrong, a stakeholder who pushed back hard, a feature that shipped but didn't move the metric. This single addition changes the trust register.
6
Show the AI/human split in the AI Portfolio case study. One annotated artifact — a before/after draft, a decision log, or a version comparison — would demonstrate rather than assert the method.
7
Add one page of synthetic user methodology to the Synthetic Users case study. Show the persona construction framework, a sample synthetic user card, and what finding surfaced that human research hadn't yet reached.
8
Add a one-line framing sentence to Signal → Story explaining its relationship to design practice.
9
Add a conversion bridge to the EY metrics (engagements vs. vaccinations are separate program stages).
10
Surface a contact/LinkedIn link in or near the homepage hero area.
Sam R. — Senior recruiter
Partial pass
Sam loads the homepage. The logo carousel and enterprise client logos (Autodesk, Wipro, EY) land well — credibility signals in the first viewport. The hero reads as a positioning statement, not a role label. Sam is scanning for: title, level, company, scope. She finds "CX · Product · Service transformation" as a category tag but no seniority signal. She scrolls to the Autodesk card, sees $50M+ and 27% RR. The numbers are credible-looking but she doesn't know if Matt was the lead or one of twenty people. Navigation makes sense — five clear tabs, no mystery. She doesn't close the tab. But she's not sure if this is a Director or a senior IC.
Role label and level signal missing from hero. She'd proceed to request a resume, but the portfolio hasn't answered the seniority question independently.
Morgan V. — VP Product
Pass
Morgan moves faster. He reads "I rebuild it as one operating model that actually ships" and pauses — that's the language of someone who's been in the political reality of enterprise transformation. He goes straight to the Autodesk card. The three-tier service ladder on Impact is exactly the kind of artifact that earns respect from a product leader: it shows sequencing logic, tier economics, and a monetization strategy. He doesn't close the tab. He clicks to the Impact console and finds it genuinely well-structured. He'd book a call.
Morgan passes, but would be even more convinced if the case study depth matched the Impact console quality. Right now Impact overdelivers relative to the case studies, which feels asymmetric.
Marc S. — Service systems purist
Fail
Marc opens the Autodesk Planning System case study. He sees: "Enterprise planning spans tools, teams, and time horizons." Then one paragraph. Then a copyright notice. He is immediately skeptical. The Impact console shows the three-tier model with revenue attribution, which is genuinely promising — he can see service architecture implied in the tier structure (Included retains, Professional attaches, Business expands). But the backstage is missing entirely. Who handles delivery of the Business tier? What organizational capability did Autodesk have to build or borrow? Was there a service blueprint? What changed in how teams were incentivized? None of this appears. The work reads as a product redesign with a service label on it, not a true service system design.
Add a service blueprint excerpt or organizational behavior change narrative to the Autodesk case study. Even one diagram showing what changed internally — not just the customer-facing model — would flip Marc's read.
Indi Y. — Thinking styles researcher
Partial pass
Indi reads the Synthetic Users page carefully. She notices the framing is correct — "complement to research, not a replacement" — and that Matt has clearly thought about the epistemological limits of the method. But she wants to see the actual persona construction. How were the synthetic users built? What inputs went in? Were they constructed from interview transcripts, behavioral patterns, or from the designer's assumptions about role-based behavior? The page doesn't show a single user card, a single hypothesis, or a single moment where the synthetic user said something the team didn't expect. Without that, Indi can't tell whether this is a sophisticated research scaffold or a renamed assumption dump.
Show one complete synthetic user card and one finding it surfaced. The methodology earns credibility through demonstration, not description.
Ethan M. — AI pragmatist
Partial pass
Ethan reads the AI Portfolio case study and finds the right instincts: "structure first, then tooling, then craft." He agrees with the framing. But he immediately asks: what would this portfolio look like without AI? If the answer is "basically the same structure with more manual writing effort," that's a Centaur workflow — human in the saddle, AI extending reach. If the answer is "it wouldn't exist in this form," that's Cyborg territory, more integrated and harder to separate. The case study doesn't declare which it is, and that ambiguity reads as evasion to Ethan. He also wants to see a specific decision where AI was wrong and Matt corrected it — that's the honesty signal he's looking for in AI-augmented work claims.
Declare the workflow model explicitly (Centaur is the defensible claim based on the framing). Add one annotated "AI said X, I changed it to Y because Z" moment. That single decision trace changes the credibility of the whole case study.
John M. — Metrics skeptic
Partial pass
John's first stop is the $50M number. His question: is that AOV growth total, incremental, or Matt's attributable slice? The "Incremental AOV" label helps, but "driven through service design leadership" is doing too much work. He'd cut "15+ Services shipped" immediately — it's a vanity count. He also notes the 4.57M engagements at EY next to 715 vaccinations and raises an eyebrow. The Wipro metrics are the strongest in the portfolio — MTTR reduction is a hard operational number with a clear before/after. He'd actually simplify the Autodesk Impact console — there are too many metrics presented as equal when 27% RR and 106% NRR path are clearly the story; the others dilute them.
Reduce Autodesk to 3 headline metrics maximum. Give $50M a denominator or attribution sentence. Cut or reframe "15 services shipped." Add a conversion-rate or context line to the EY engagements figure.
Teresa H. — Outcome enforcer
Fail
Teresa is direct: "15 services shipped" is an output. Shipping is an activity. The outcome is what happened after they shipped. Did customers use them? Did account managers sell them? Did they drive the 27% RR or was RR already trending? She challenges the $50M AOV figure similarly — AOV increasing could be pricing inflation, not experience-driven behavior change. The metrics she trusts most are MTTR reduction (−31%) because it's a process behavior change with a time baseline, and 16k annual lockout reduction because it implies a behavioral shift in how users self-serve. Her verdict: the Wipro metrics pass; the Autodesk metrics need behavior change anchors, not just outcome totals.
Replace "15 services shipped" with a behavior metric: tier adoption rate, first-value time within tier, or upsell conversion from Included to Professional. Add a line to the Autodesk section framing what customer behavior changed — not just what the revenue did.
Avery K. — Staff product designer peer
Partial pass
Avery's first read: the Impact console is genuinely impressive — the "live briefing" UX is the kind of thing a real systems thinker would build. She appreciates it. She clicks the Autodesk case study expecting depth and finds three sentences. This is where "polished but shallow" crystallizes. The homepage promises "I work at the system level" — but the case studies don't show the system. There's no journey map, no blueprint, no org diagram, no decision tree, no constraint acknowledgment. The even-over statements on About feel real. The Signal → Story films are a genuine surprise — she wasn't expecting documentary filmmaking in a design portfolio, and it reframes Matt's range in an interesting way. But the absence of tradeoffs is the tell. Real work has tradeoffs. Where are they?
Near-pass. The craft signals are there — the Impact console and the films prevent a "shallow" verdict. But case study depth is the barrier to full trust. One real case study with constraints and a failure moment would shift Avery's read from "impressive presentation" to "this person actually did this work."
Skeptical design leader — 300 portfolios this month
Partial pass
She opens the homepage. The hero line is competent but she's heard "I work at the system level" before. She scrolls fast. Then the Impact console stops her — it looks different. The "live briefing" frame, the tier ladder with revenue attribution per tier, the Wipro handoff delta visualization — these don't look generated. They look decided. She clicks the AI Portfolio case study expecting a defensive essay about AI ethics and finds… three sentences and a methodology tagline. She closes it. Then she goes back to the Impact page and spends 90 seconds on the Autodesk tier structure. She's moderately impressed by the tier economics logic. She checks the Synthetic Users page. Empty. Her verdict: "There's a real designer in here, but they're hiding behind the frame. Show me the work, not the frame."
Survives initial hostility — the Impact console and tier architecture are the genuine surprise moments that prevent immediate dismissal. But the case study stubs actively undermine trust for the skeptical reader. The ratio of presentation quality to work depth is too high. That gap is the AI portfolio red flag, not the AI use itself.
Opening frame — what this document is
This case study uses a real QA session — nine synthetic personas stress-testing this portfolio before it shipped to live audiences — as the primary research artifact. The methodology is documented exactly as practiced: persona construction, hypothesis formation, environmental staging, and findings under adversarial conditions.
01 — The research problem
Design artifacts fail in predictable ways: they're evaluated by the wrong audience at the wrong moment, with no structured way to stress-test assumptions before live exposure. Traditional usability testing requires access to real participants, which is often impossible for portfolios, internal tools, or pre-launch strategy work. The hypothesis: structured synthetic personas — modeled on real practitioners with documented worldviews — can surface category-level blind spots that self-review misses.
02 — Persona construction logic
Each persona was built from a known practitioner worldview (Mager, Young, Morville, etc.) and assigned a specific critical frame: systems architecture, research epistemology, AI workflow transparency, metric rigor, outcome vs. output distinction. Personas were not role-based stereotypes — they were hypothesis engines. The question for each: what would this person look for first, and what would make them stop reading?
03 — Environmental staging
Four QA environments mapped to research phases: DEV (structural/navigational validation), QA (content and methodology critique), UAT (peer-level authenticity check), PROD (adversarial stress test). Each environment had defined pass criteria before the session ran — not post-hoc. This staging is borrowed from software release management and applied to experience validation.
04 — What the method found that self-review missed
Three findings that didn't surface in solo review: (1) The ratio of presentation quality to case study depth reads as an AI-generation signal to experienced reviewers — not the AI use itself. (2) "$50M YOY AOV" and "15 services shipped" belong to different metric categories (outcome vs. output) and their coexistence undermines the more defensible numbers. (3) The Synthetic Users case study — the very methodology being documented here — had no methodology shown. The meta-irony was surfaced only by running the session.
05 — Limitations of the method
Synthetic users cannot replace real ones. They don't surface unknown unknowns — only hypotheses the designer was already able to form. The quality of a synthetic user session is bounded by the quality of the persona model. In this session, adversarial personas (PROD environment) were the most productive; consensus-seeking personas surfaced less. The method is a pre-research sharpening tool, not a substitute for observation.
06 — Recommended practice format
Persona card template: name, worldview anchor, first question asked, pass criteria, failure mode. Hypothesis format: "If this artifact works as intended, [persona] will [specific behavior] within [timeframe or scroll depth]." Finding format: persona · observation · hypothesis confirmed or disconfirmed · change recommended. Research log format: structured table (as produced in this session) — usable directly as documentation, not just a workshop artifact.
07 — Closing argument
The most honest thing this case study can say: this QA session improved this portfolio. The findings documented here became the change list. That traceability — from synthetic persona to specific design decision — is what separates a research methodology from an AI experiment. The method has inputs, outputs, and audit trail. That makes it research.