User Document uploads: RAG vs "deep learning" (token consumption)

General
2

You’ve written a wonderful question here. And I will answer it below.

So, how do end-user file uploads function? Well, end-user file uploads are handled in one of two ways depending on the size of the file and the maximum input length you’ve set.

THE TWO WAYS END-USER FILES ARE HANDLED

  • If the file is smaller than the maximum input length, it will be dropped into the conversation in totality.

  • If the file is larger than the maximum input size, the file will be turned in vector embeddings and queried through RAG for the duration of the conversation.

I hope that elucidates the process a bit. In the Builder, you can adjust the maximum input size in by going to configure>Tokens Lengths. The observant user will notice that as soon as you toggle on end-user upload, we automatically bump up maximum input length to 3000 tokens.

1 Like