Human feedback
person concept tool org talk claim — click a node to jump to its page; hover an arrow for the relation
Human feedback is one of five distinct flavors of evals signal used to assess AI system quality, representing direct input from end users or other human stakeholders about the outputs they receive.
Role in evaluation pipelines
Dat Ngo of Arize positions human feedback as an irreplaceable signal within a broader evaluation framework that also includes LLM as a judge, golden datasets, deterministic checks, and business metrics. He emphasizes that practitioners must not overlook this source: "you can't forget about your humans. Whether it's the end users using your product, it's extremely valuable signal." (Dat Ngo, Arize, 12:06)
The framing as one of five flavors of eval signal suggests human feedback is neither sufficient on its own nor redundant alongside automated methods — each signal type captures something the others may miss. End-user feedback in particular surfaces real-world quality judgments that synthetic benchmarks or automated judges cannot fully replicate, making it especially valuable for understanding how a system performs in production.