Send user question to slack based on confidence level

Actions
1

I’m looking for help triggering the action correctly.

The chatbot is evaluating the responce as a confidence level, and if it is low enough I want the user input to be sent to the slack action so I can become aware of it and add to the known answers PDF file and update the chat bot knowledge.

I’m not able to get it to send to slack - not able to trigger the action.

I have a rule that tells it to use the action - and when the confidence level is below 80% the chat tells the user it is sending it to slack to improve knowledge (will probably remove that later as the user doesn’t need to know).

RULE

  • If the confidence of your response is below 80% use the action to send a message to slack including the user question.

Also, in the action the trigger prompt is:
When a response is provided with a confidence level below 80% use this action to send the user question to Slack.

I have tested with the default trigger prompt and I am getting input to slack, so the connection is working, I’m just not getting it triggered when the confidence level is low enough.

Thank you!

2

This needs to be done entirely through prompting. The confidence level is a good idea, but just recognize the AI will be making up a confidence level. These are language models, so there is not internal confidence level calculator. When I write prompts for actions like these I use phrasing like “If you aren’t sure about the answer”. Stuff like that.

That being said, the confidence level idea can work. Just recognize it’s made-up and the AI is making up its own rules!

3

Thank you @admin_mike – I’m starting to get the hang of this better and have it working now… thank you!!

Rule:

  • Do NOT MAKE UP answers. If you truly don’t know the answer to a question, tell the user you don’t know, use the action to send a message to slack with the question, and encourage them to reach out to our support team.

Actions trigger prompt:
When you don’t know the answer, use this action to send the most recent user input to slack.

1 Like
4

Hi @cfdynamics, there is an action in Pickaxe that integrates with the Pinecone assistant. You can use one of the functions in Pinecone to evaluate the answers: