The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Rapidminer with Spark

d_walden18863d_walden18863 Member Posts: 1 Learner II
edited November 2018 in Help

I realize RapidMiner can be implemented with Spark in the Hadoop environment on a cluster but my question is can RadipMiner be implemented with Spark on a Standalone? If so, can you direct me to some information to help me with the implementation?

Tagged:

Answers

  • zprekopcsakzprekopcsak RapidMiner Certified Expert, Member Posts: 47 Guru

    Hi,

     

    RapidMiner Radoop always requires Hadoop (or more specifically Hive, HDFS and YARN) underneath so it cannot work with pure Spark.

     

    Best, Zoltan

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    I don't if that's been done yet but it might be feasible.

     

    You could, in theory write your spark code and then have it kick off in RapidMiner using an Execute Program operator. You could pass commands, environment variables, and even some sort of throughput file into the operator that could in theory feed your pure spark program. 

     

Sign In or Register to comment.