My AI chatbot forgets things in the conversation (Pickaxe.co)

[clojobobo]

Hi there. I am having issues with user memory on pickaxe.co. I have a pickaxe that creates a case assessment for a client. If they return and want to update the case assessment unless they go to their historical chat, it will not retain all the information in the user memory. Only somethings seem to go across. I want all the user memory to be available even in a new chat. I also would like all the user memory to be available when using other pickaxes in the studio.

Essentially, we have one free tool which we then use to upsell the other tools within the studio. But we don’t want them to have to keep entering all there data everytime. It seems random what goes across to other pickaxes or new chats in the Studio. Is this a bug?

Thanks in advance. Clojo.

[am3]

Probably a bug.

I’m experiencing close to the same thing as well.

For example:

If I asked, “Who is the president of United States today?”

It will give an answer.

If I asked again, “How old is he?”

The answer will be somewhere like: “How old is who? You should give us more context about your question.”

Normally, it will give the age of the person being asked from the previous question.

[admin_mike]

If I’m understanding correctly, you’re saying that it won’t overwrite existing user memories?

Or are you saying that the User Memory won’t contain the full case assessment?

[admin_mike]

If I’m understanding correctly you’re talking to Tool #1 about the US President. Then you want to talk to Tool #2 about the US President without any extra context?

If you’re asking two different tools, they won’t know each other’s chats in entirety.

User Memory does not remember everything. It only remembers the specific thing you want it to remember. For example, you might want to remember the user’s location or job. If that comes up it will remember it. But it won’t remember the user’s lunch routine unless that is relevant to the Memory you’re asking it to collect.

[am3]

No. It’s just a simple conversation in a single thread. A normal use of AI chatbots. One chat, not separate.

(1) Here’s an example using my other software (not Pickaxe)

User input #1: Joe is a male. His 5th birthday is this coming April 3, 2025. How old is he today March 15, 2025?

Chatbot: Joe is 4 years old today.

Since his 5th birthday is coming up on April 3, 2025, and today is March 15, 2025, he hasn’t reached his 5th birthday yet. He was born on April 3, 2020, making him currently 4 years old.

User input #2: When was he born?

Chatbot: Joe was born on April 3, 2020.

Since his 5th birthday will be on April 3, 2025, we can determine he was born exactly 5 years before that date.

(2) Here’s an example using Pickaxe.

User input #1: Joe is a male. His 5th birthday is this coming April 3, 2025. How old is he today March 15, 2025?

Pickaxe: Let me help you calculate Joe’s age.

Given:

Joe will turn 5 on April 3, 2025
Today’s date is March 15, 2025
Since Joe hasn’t reached his birthday yet this year (April 3, 2025), he is currently 4 years old.

This is a simple calculation that doesn’t require any external tools - Joe will be 5 years old in about 19 days from the given current date, so he is currently 4 years old.

User input #2: When was he born?

Pickaxe: I apologize, but I need to know who “he” refers to in your question. Could you please specify which person you’d like to know the birth date for? Once you provide the name of the specific person, I can search for that information.

The problem is after asking a follow-up question, it doesn’t have the connection on what we are talking about. I am talking about Joe. My other software knows exactly who I’m talking about with the follow up question. But with PA, it doesn’t.

AI Chats in my experience normally have a conversation to a user based on the entire conversation thread, which Pickaxe right now don’t have.

Is this a memory problem? I thought this is a memory issue. Meaning, it cannot remember the previous conversation we had as a context to the new input I just gave.

[admin_mike]

This sounds like an issue with the overall context window of the chatbot, not the User Memory feature.

I would suggest you go into Configure>Token Lengths and then increase the Memory Buffer

if you can’t figure it out, you can share the tool link here and I can take a look at it.

[clojobobo]

My issue is that it is not remembering saved memories in new chats. So if you leave a chat and the come back as a logged in user then it doesn’t seem to remember everything in a new chat. If you go to an old chat it will remember but new chats in remember certain things. I’ll try and do a video to explain. I have not tested extensively with user memory across other tools in the studio but if it can’t pass user memory between one chat and another in the same tool it doesn’t bode well I am sure it must be something that can easily be fixed.

[am3]

I see. I was really thinking this has to be something about memory stuff.

But I’ve been trying to find the token length for memory but I cannot find it unlike with V1.

Here’s what I see on V2.

[stephenasuncion]

Ahh “memory buffer” wasn’t displayed on Form Pickaxes, should be fixed now.

[am3]

Now it’s working.

But there’s a new problem I noticed. When I ask a new question, it removes the previous output.

It’s not like a conversation thread where I can see all input and output.

What’s happening is;

Input #1 > Output #1 > Input #2 then Output #1 will be lost in the screen and I can only see Output #2