Hi there, I am trying to build an FAQ webbot for my website. I have been following along on a youtube video from Pickaxe but when it comes to adding my website it only returns the first page and no other page from the site. I tried adding other pages but it only returned the social media links and not the content. Please help as this is a fundamental part of the bot.
If it’s not returning all the pages, then you should enter the webpages individually. Sometimes it’s not able to pull all the relevant pages from the sitemap.
when entering the individual pages it doesnt scape the content at all!
What does it show? can you share one of the webpages that is failing to scrape with me? I will take a look at it.
Hi Mike,
Thanks for your reply. Here is one of the pages Expert Research Impact Consulting | Research Impact Academy
I’m also having very similar issues. When I try to scrape a url, either the scrape just hangs and never returns anything, or it just returns the primary url. I’ve never had it return more than the primary url, even with visiting multiple public sites.
For example:
These are just examples of urls that I have tried that either hang or just return the primary url.