The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

CrawlWeb - Crawl only certain parts of a website

ojuarezojuarez Member Posts: 5 Contributor II
edited June 2019 in Help
Hello.

I am crawling a website with "CrawlWeb" operator and it seems to be doing ok.  The problem is when I want to crawl only one area of the site:

I am using the following values.

url:                          http://www.autopredios.com
crawling rules:      .+vans.+


I only want to crawl:    http://www.autopredios.com/vehicles


Do I have to do it by  changin the URL  or by  changing the crawling rules.  I tried changing  the URL and doesn't seems to exclude other directories.


Tagged:

Answers

  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    You definitely have to change to URL. If it goes to other parts of the website, you have to tune your crawling rules.

    Best regards,
    Marius
Sign In or Register to comment.