Many users who upload websites into the knowledge base ask about how often the system will scan the website and update its information. I wanted to explain how this system works.
Short Answer
The knowledge base does not auto-update. But you can toggle a setting under “Advanced Options” to make it auto-update daily, weekly, monthly, or never.
The Longer Explanation
Whenever you upload a website, Pickaxe will scrape the website’s contents and add them to the knowledge base. This upload will reflect the website’s contents at the time of scraping. By default Pickaxe will not regularly visit to re-scrape it.
In order to change this setting, in the builder click on “Advanced Options” at the bottom of the page. This will open up some extra toggles. Scroll down to a section labeled Knowledge Base. Here you can set how often the web-based knowledge will be refreshed.
Please feel free to ask questions below.
1 Like
Quick question… this is awesome. How many links can we include that are set to auto update? Have an idea - not in the 1000s of sites but a few that would update weekly let’s say.
I believe you can add up to 50 links. However, you may be able to add more.
You can tell me how many fit in there 
Hi there, would it be possible to indicate the sitemap URL (using the index URL), so that the KM base could be updated (according to the frequency already available) with new Articles or Products… Many thanks for your response. Denys.
1 Like
Hi @Denys_LEVASSORT,
Currently, if you connect a custom domain with your Studio, our system will generate a custom sitemap for your website at /sitemap.xml directly, without a sitemap index. Owners don’t have the ability to manually add links to a sitemap index at the moment.
1 Like
Thanks Stéphane, do you mean that this automatism is valid for deployment in Studio mode, and not embedded? In the latter case, if I enter www.mysite/blog/, or /products in my KM Base, will all the old content be indexed and the next ones too (with the frequency indicated)? Or do I have to enter each URL separately? Thank you very much. Denys
1 Like
Denys, did you find out the answer? I have the same question.
This is a setting at the individual Pickaxe level.
Once you add at least on website into the knowledge base, an option pops up in the Advanced Settings that lets you set the frequency with which the websites are re-scraped.
When it re-scrapes the webpage, it replaces the old version of the webpage rather than adds to it.
Yes, I actually saw that option in the editor after posting here. Thanks!
1 Like
