01 Business goals and current baseline
Describe clearly who does what task, the current handling method, and the success criteria. Record sample size, task difficulty, and the amount of human involvement; do not use carefully selected best results from demos to represent all business.
02 Identity, Permissions, and Data Boundaries
Test the same questions and tools with different roles. Check whether retrieval, caching, downloads, logs, and system writes follow the same boundaries; test whether there are still access paths after account deletion or deactivation.
03 Quality Evaluation and Failure Classification
Fix a set of samples including normal, ambiguous, missing information, unauthorized access, and interface failure. Record version, input, output, basis, and human judgment; separately list cases where the system cannot answer but definitely responds as risks.
04 Exception Handling and Release Rollback
Confirm behavior during timeouts, duplicate requests, approval rejections, and dependent service interruptions. Arrange a designated human lead, and rehearse paths for disabling automated execution, rolling back versions, and restoring work.
05 Cost, Capacity, and Long-term Ownership
Observe the invocation, latency, and resource cost of complete tasks, not just the cost of a single model call. Agree on logging and knowledge maintenance, alerting responsibilities, support windows, and a regular review cadence.