I’m creating a chatbot to chat with an historical figure – in this case the 16th century Anabaptist leader, Menno Simons. I’ve loaded many of his writings into the knowledge base.
I discovered what seems like a pretty good way to add a quiz generation feature so people can easily test their knowledge of what the chatbot knows.
In the Prompt - Introductions section, I have two icebreaker questions, but I added a third one as a request for a quiz: “Quiz me about aspects of your life and theology.” (That’s the user telling Menno Simons to give them a quiz.)
In the “Configure - Training Dialogue” section, I created a User/Bot Response pair where the User Input was “Quiz me about aspects of your life and theology” and the “Bot Response” was as follows:
Response:
Step 1: Ask the user whether they want questions about events in your life, or about your theology? Wait for a response.
Step 2: Based on the user's response, present a question. Number each question sequentially.
-- Example, if the user wants events: [#]. In what year did I become a priest?
-- Example, if the user wants theology: [#}. What did I believe regarding Holy Communion?
Step 3: Wait for the user to respond.
Step 4: Comment briefly and concisely on how accurate the response was. Cite the knowledge file, but do not include further details that may give away the answer. Immediately present the next question.
[RETURN TO STEP 2 and REPEAT]
These instructions would probably work in the main prompt as well, but in my case, my main prompt is loaded with other instructions, and my bot seems to do a nice job following these instructions and examples in the Training Dialogue. It could of course be adapted to suit your use case.
One adaptation is to stipulate that the questions should be presented in multiple-choice format (e.g., A, B, C). I implemented a multiple-choice quiz today, but initially discovered that the correct answer was nearly always the same letter. I added: “After generating the answers, shuffle or reorder the answer choices randomly. Check the location of the correct answer of your previous question, and place the correct answer of the current question at a different location.”
Even that didn’t achieve much randomness. Finally, this further addition achieved a good level of randomness:
Example:
-- If the previous correct answer was A, place this correct answer at B or C.
-- If the previous correct answer was B, place this correct answer at A or C.
-- If the previous correct answer was C, place this correct answer at A or B.
This isn’t truly random, of course, because following it strictly means that the same letter would never be used for the correct answer twice in a row. Anyone have a truly random way to do this?
Best regards Andreas