I recently watched a very informative video on the Pickaxe YouTube Channel about using “RAG mode” to process files used for training our pickaxes (knowledge base) before making them available to users, and how this optimizes token consumption.
Based on this information, I have a specific question: when a user is authorized to add their own files to the chat, does the processing of these files also occur through RAG, or does it work as a “deep learning” approach where the file is sent entirely as a prompt, resulting in full token accounting?
(Note.: I understand that user files, by definition, don’t modify our pickaxe’s knowledge base.)
I haven’t found any content related to this specific topic in the YouTube videos or community topics, and this information would be very relevant to our understanding of the process.
You’ve written a wonderful question here. And I will answer it below.
So, how do end-user file uploads function? Well, end-user file uploads are handled in one of two ways depending on the size of the file and the maximum input length you’ve set.
THE TWO WAYS END-USER FILES ARE HANDLED
If the file is smaller than the maximum input length, it will be dropped into the conversation in totality.
If the file is larger than the maximum input size, the file will be turned in vector embeddings and queried through RAG for the duration of the conversation.
I hope that elucidates the process a bit. In the Builder, you can adjust the maximum input size in by going to configure>Tokens Lengths. The observant user will notice that as soon as you toggle on end-user upload, we automatically bump up maximum input length to 3000 tokens.