A recent patent from Yahoo describes in detail how people or users can help in the crawling of web pages. This patent is known as the automatically fetching patent and will come up with details of how a user or site interaction could be used to help the search engine crawling program.
The activities which are involved and addressed include the webmaster training the crawler of how to find out the pages which are indexed in their website. A human reviewer is teaching the crawler of how to find out the pages. The content subscriber will show the crawling program, the information that would be subscribed to and updates will be sending about it.
As per this patent, the search engine will allow people to teach the web crawling program. This will help in the navigation of pages of the website. This will be particularly more useful in accessing web pages which have java script links and those pages which are accessible only through various selections. Pages which are accessible through inputting text into forms can also be navigated with the help of this patent implementation.
The patent find application when search engines find difficulties in crawling pages which could be due to java script links.
Subscribe to our blog to receive new posts and updates by Email