The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

"Web mining a list of URLs"

adommnichadommnich Member Posts: 2 Contributor I
edited May 2019 in Help
Hello,

how can I loop over a list of different URLs and extract the content? The operator Crawl Web has no input but the parameter 'URL'. Is there a way to fill it dynamically using a Loop operator?

Thanks for your attention.

Regards
adommnich
Tagged:

Answers

  • colocolo Member Posts: 236 Maven
    Hi adommnich,

    I would use a "Loop Examples" operator for the URL list (if you read it into an example set before). Inside the loop you can easily extract the current URL with "Extract Macro" and use the value for the "Get Page" operator. If you want to crawl and not just retrieve one website you can replace "Get Page" by "Crawl Web".

    Regards,
    Matthias
  • adommnichadommnich Member Posts: 2 Contributor I
    Hi Matthias,

    thanks a lot for your quick reply. I am afraid I wasn' t aware of the Macro concept.  :-[

    It works fine now.

    Regards
    Alex
Sign In or Register to comment.