The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
"Can I use a macro value to crawl sites ?"
Unicorn
Hi there,
Can I use macro values one way or another with the Crawl web operator to set the url value ? I've tried the obvious %{my_macro_value} but the system didn't like that.
I would like to use this option (or any alternative as my process is as follows :
I have an example set containing urls I want to crawl, for example
www.somedomain.com/somepageformat?id=12345
www.somedomain.com/somepageformat?id=54321
www.somedomain.com/somepageformat?id=...
...
However, every url above can have several pages also, like
www.somedomain.com/somepageformat?id=12345&;page=1
www.somedomain.com/somepageformat?id=12345&;page=2
which I all need to crawl strting from the 'master' page. Works perfectly if I use the crawl web parameter with static data, but since the list is variable I need to find a way to provide the id, or the full url, as a variable to the operator. Using the get page operator is not a real option since I do not know upfront how many pages are available as query string.
Any suggestion ?
Can I use macro values one way or another with the Crawl web operator to set the url value ? I've tried the obvious %{my_macro_value} but the system didn't like that.
I would like to use this option (or any alternative as my process is as follows :
I have an example set containing urls I want to crawl, for example
www.somedomain.com/somepageformat?id=12345
www.somedomain.com/somepageformat?id=54321
www.somedomain.com/somepageformat?id=...
...
However, every url above can have several pages also, like
www.somedomain.com/somepageformat?id=12345&;page=1
www.somedomain.com/somepageformat?id=12345&;page=2
which I all need to crawl strting from the 'master' page. Works perfectly if I use the crawl web parameter with static data, but since the list is variable I need to find a way to provide the id, or the full url, as a variable to the operator. Using the get page operator is not a real option since I do not know upfront how many pages are available as query string.
Any suggestion ?
Tagged:
0
