How to test your AI Employee correctly

✓

I changed my AI Employee. What should I do before relying on the change?

Run a short test on the most important customer questions, then focus on the area you changed. If a reply is wrong or weak, fix the source that owns that behavior and start a fresh test. The practical loop is: change → test → fix the cause → retest → then rely on it with customers.

• Test after every important change, even when saving succeeded.
• Write questions like a real customer instead of feeding the expected answer into the prompt.
• Evaluate the reply actually returned by Mr. AI Testing.

A quick test after any important change

• Ask about one core Product or Service, then an important price, offer, schedule, or policy.
• Ask for the same meaning in different words, then include a wrong assumption and check whether the AI corrects it.
• Continue for several turns and verify that context is preserved without inventing missing customer data.
• Give a realistic objection and see whether the conversation continues naturally.
• Ask for information that is not available and verify that the AI does not fabricate it.
• Ask for a human agent while checking tone, language, business name, and identity.
• If you use Conversation Stages, run a full journey and make sure the stage does not advance too early.
• If you use an Action, test when the AI should and should not try to trigger it.

If you changed identity or speaking behavior

• Business name and AI Employee role.
• Tone, language, and speaking style.
• Things the AI must not say or do.
• How it handles a question whose answer is unknown.
• When it should suggest or request human help.

If you changed Catalog

• Ask for a product or service by name, then verify price, currency, and Options / Variants.
• Ask a natural discovery question without repeating the item name.
• Test Semantic Classification and Semantic Tags with questions that match the meaning behind the structure.
• Test AI Search Attributes with a precise question about Instructor, Branch, Level, or another exact value.
• When you use Aliases or Search Name, ask for the same value using different wording or language and confirm Mr. AI still reaches the same correct item.
• Test two similar items so similarity alone does not return the wrong product.

If you changed Deep Knowledge

• Ask for one direct fact.
• Ask for the same meaning using different wording.
• When information is distributed, ask a question that requires combining it.
• Ask for something that is not present and check for non-fabrication.
• After updating the Resource, start a fresh test and ask again.

If you changed Conversation Stages

• Start the journey from the beginning and continue for multiple turns.
• Check that stage transitions happen at the right time, not too early.
• Take a small detour and then return to the main journey.
• Confirm the conversation reaches the correct final state.

If you changed Actions

• Test the case that should trigger the Action.
• Test the same journey before required information is complete and confirm it does not trigger too early.
• Try missing or invalid data.
• Review what the AI would attempt if the conversation were Live.

If you changed Follow-up or Stage Follow-up

Test whether the conversation reaches the right stage and context for follow-up. A Test Chat itself does not prove that a real Follow-up or Task was scheduled or executed in Production.

When you find a problem: fix the right source

1. 1

   Keep the customer question and the reply that exposed the problem.

2. 2

   Identify the owning source: Catalog, Deep Knowledge, identity and speaking behavior, Conversation Stages, or Action.

3. 3

   Change the smallest responsible source instead of adding one broad instruction to hide the symptom.

4. 4

   Start a fresh Testing Chat or Scenario after the important change.

5. 5

   Repeat the same question, then try nearby wording to make sure the fix is not overfitted to one sentence.

6. 6

   After the core tests pass, watch the first real conversations to verify quality in use.

Open Test Assistant

Test manually with Test Assistant

Test with ChatGPT or a connected AI assistant

Understand the AI Employee quality loop

Understand grounded AI replies

Review the knowledge base when you find a gap

Review channel setup after testing passes