--- title: "How to test your AI Employee correctly" description: "After an important change, test the key customer journeys, find the source of any problem, fix that source, and retest before relying on the change with customers." language: en canonical: https://mrailabs.com/docs/en/learn/test-ai-employee-before-real-customers/ published: 2026-09-29T19:19:11.235236 last_verified: 2026-09-29T19:19:11.235236 --- # How to test your AI Employee correctly ## I changed my AI Employee. What should I do before relying on the change? Run a short test on the most important customer questions, then focus on the area you changed. If a reply is wrong or weak, fix the source that owns that behavior and start a fresh test. The practical loop is: change → test → fix the cause → retest → then rely on it with customers. - Test after every important change, even when saving succeeded. - Write questions like a real customer instead of feeding the expected answer into the prompt. - Evaluate the reply actually returned by Mr. AI Testing. ## A quick test after any important change - Ask about one core Product or Service, then an important price, offer, schedule, or policy. - Ask for the same meaning in different words, then include a wrong assumption and check whether the AI corrects it. - Continue for several turns and verify that context is preserved without inventing missing customer data. - Give a realistic objection and see whether the conversation continues naturally. - Ask for information that is not available and verify that the AI does not fabricate it. - Ask for a human agent while checking tone, language, business name, and identity. - If you use Conversation Stages, run a full journey and make sure the stage does not advance too early. - If you use an Action, test when the AI should and should not try to trigger it. > **Start with the big picture** > This is a fast regression check. Go deeper only in the area you changed instead of repeating a huge checklist every time. ## If you changed identity or speaking behavior - Business name and AI Employee role. - Tone, language, and speaking style. - Things the AI must not say or do. - How it handles a question whose answer is unknown. - When it should suggest or request human help. ## If you changed Catalog - Ask for a product or service by name, then verify price, currency, and Options / Variants. - Ask a natural discovery question without repeating the item name. - Test Semantic Classification and Semantic Tags with questions that match the meaning behind the structure. - Test AI Search Attributes with a precise question about Instructor, Branch, Level, or another exact value. - When you use Aliases or Search Name, ask for the same value using different wording or language and confirm Mr. AI still reaches the same correct item. - Test two similar items so similarity alone does not return the wrong product. ## If you changed Deep Knowledge - Ask for one direct fact. - Ask for the same meaning using different wording. - When information is distributed, ask a question that requires combining it. - Ask for something that is not present and check for non-fabrication. - After updating the Resource, start a fresh test and ask again. ## If you changed Conversation Stages - Start the journey from the beginning and continue for multiple turns. - Check that stage transitions happen at the right time, not too early. - Take a small detour and then return to the main journey. - Confirm the conversation reaches the correct final state. ## If you changed Actions - Test the case that should trigger the Action. - Test the same journey before required information is complete and confirm it does not trigger too early. - Try missing or invalid data. - Review what the AI would attempt if the conversation were Live. > **Testing an Action is not executing it** > Test Assistant evaluates the AI decision and behavior. It is not proof that a real Booking, Order, Payment, Refund, or other Business Action happened in the production system. ## If you changed Follow-up or Stage Follow-up Test whether the conversation reaches the right stage and context for follow-up. A Test Chat itself does not prove that a real Follow-up or Task was scheduled or executed in Production. ## When you find a problem: fix the right source 1. Keep the customer question and the reply that exposed the problem. 2. Identify the owning source: Catalog, Deep Knowledge, identity and speaking behavior, Conversation Stages, or Action. 3. Change the smallest responsible source instead of adding one broad instruction to hide the symptom. 4. Start a fresh Testing Chat or Scenario after the important change. 5. Repeat the same question, then try nearby wording to make sure the fix is not overfitted to one sentence. 6. After the core tests pass, watch the first real conversations to verify quality in use. > **Use a fresh test after changes** > After an important change, start a fresh test so old conversation context does not blur the behavior of the new version. [Open Test Assistant](/client/settings/test-assistant) [Test manually with Test Assistant](/docs/en/getting-started/test-assistant-before-live/) [Test with ChatGPT or a connected AI assistant](/docs/en/settings-and-usage/test-ai-employee-with-connected-assistant/) [Understand the AI Employee quality loop](/docs/en/knowledge-and-behavior/continuous-ai-employee-quality-loop/) [Understand grounded AI replies](/docs/en/knowledge-and-behavior/grounded-ai-replies-from-business-knowledge/) [Review the knowledge base when you find a gap](/docs/en/knowledge-and-behavior/knowledge-base-overview/) [Review channel setup after testing passes](/docs/en/channels-and-launch/connect-channels/)