The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

random user agent for web mining needs an update

kaymankayman Member Posts: 662 Unicorn
edited December 2018 in Product Feedback - Resolved

Hi guys, not a real bug in itself, but it appears there are some fairly old random agents in the scripts used, and this causes problems time by time when using the crawler. The behaviour is noticeable by sites using a browser version validator, and then forward you to a page mentioning 'hey buddy, looks like your browser could use an update' or similar.

 

I start to notice this more and more lately, and while unchecking the 'random user agent' solves the problem it was a nice feature to have.

 

Any plans to bring a new version, including some new browser /agent logic and cleaning out the old ones?

Tagged:
3
3 votes

Declined · Last Updated

No active voting since Dec 2018 - please cc @sgenzer if you want to reopen this idea. PROD-863

Comments

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    Hi @kayman - unfortunately the whole Web Mining extension needs a complete overhaul and I do not see it on the immediate future roadmap. I'm going to push this to Product Ideas so people can upvote if they feel this is a priority.

     

    Scott

     

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    There are definitely several other more critial bugs in the web mining extension, but this should get added to the list.  I do hope the developers are able to look at improving it all sometime soon!

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
Sign In or Register to comment.