Subject Matter Expert

A Subject Matter Expert (SME) is a domain-specialist human reviewer used in agent evaluation workflows to assess the quality and correctness of outputs that require specialized knowledge.

Role in Agent Trace Review

Rustem Feyzkhanov (Snorkel AI) identifies SMEs as one mechanism for reviewing agent traces — the recorded logs of an agent's intermediate reasoning steps and actions. Specifically, SMEs can be brought in to inspect traces and evaluate whether an agent's behavior was appropriate, catching domain-specific errors that general reviewers might miss. 11:34

This positions SME review as a complement to automated evals, filling the gap where human judgment over specialized content is necessary before trusting agent outputs at scale.