Deepseek + Pickaxe RAG: Good Results but a Few Questions

General
1

Hello everyone,
I’m working on a RAG (Chatbot) project and so far I’ve gotten the best results with Deepseek. (In my opinion, it’s the most faithful model to the documents I uploaded.)

The most important thing for me is that the model can educate the user based on the documents and transfer the doctrine or perspective reflected in the documents to the user effectively.

I uploaded my documents as PDFs, but when I tried to test and see which parts of the documents were being used, I noticed that the retrieved chunks were not in PDF format. I’m also not sure whether the document chunking is being done by the AI itself? (If so, I think that would allow the system to access more specific sources based on user input.)

Due to the current instability of Deepseek, I sometimes get more or less detailed responses to the same questions. However, I still feel like I have to stick with it for now, because other models were far less loyal to my knowledge base using the same configuration.

I had read that smaller parameter models might perform better in RAG structures, but for now, I don’t plan to switch to a different framework than Pickaxe, since it already gives me very good results thanks to its strong configuration capabilities.

If there are other users like me who need a minimal-hallucination RAG setup, I believe there would be interest in seeing different or smaller Deepseek models available within Pickaxe.

Thanks in advance, friends!
You’re doing a great job — even for someone like me who doesn’t have a lot of technical knowledge, it’s been a great experience. :folded_hands:

2

@user30, well done on building your RAG.

A couple of comments:

  1. When you upload a file, the system (Pickaxe) creates chunks, vectorises them (i.e. converts the chunk into a mathematical element called vector), and adds the vector to a vector database.
  2. When you perform a search, the system decides which vector to use based on semantic relevance and passes the vector together with the user queries to the LLM to add additional context.
  3. The LLM sends the response back to Pickaxe.

This means that when the chunck is extracted and passed to the LLM, it is not in a PDF format and the quality of the response doesn’t necessarily improve based on a smaller model if you have a well structured prompt and context.

3

Thank you for the information.
It’s clear that LLMs with fewer parameters are less capable of accurately interpreting fragments retrieved from a vector database. However, it’s also important to note that different Deepseek models may be better at interpreting both the retrieved chunks and the prompts.

If this is how the system works, then for a project where the details in the information source truly matter, I believe structuring the source content in TXT format(organized by titles and parameters into clearly defined assertions)before feeding it into the vector store would allow the LLM to better grasp and interpret the information.

1 Like