The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Arithmetic Manipulation of Performance Values

lsevellsevel Member Posts: 18 Contributor II
edited August 2019 in Help

Hi all,

 

 

I am wondering if there are ways in rapidminer to manipulate performance values and calculate new indices/variables. For example, we are looking to use Harrell's algorithm for calculating optimism to obtain the performance of each of 1000 permutations of a data set. We have processes (attached) to generate the performance metrics for each of the two models described on page 2 of http://support.sas.com/resources/papers/proceedings13/504-2013.pdf, but I am unsure of how to use rapidminer substact and average these values such that we can obtain an optimism score for the performance of each of the permuted samples (process attached second). 

Bootstrap process:
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.6.001" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="retrieve" compatibility="7.6.001" expanded="true" height="68" name="Retrieve (2)" width="90" x="45" y="901">
        <parameter key="repository_entry" value="//Local Repository/data/classifier_structural_self_report_cf_HC_2.5.16"/>
      </operator>
      <operator activated="true" class="filter_examples" compatibility="7.6.001" expanded="true" height="103" name="Filter Examples (2)" width="90" x="45" y="1003">
        <list key="filters_list">
          <parameter key="filters_entry_key" value="ID.does_not_contain.RH"/>
        </list>
      </operator>
      <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="68" name="Collect (4)" width="90" x="648" y="748">
        <parameter key="unfold" value="true"/>
      </operator>
      <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (4)" width="90" x="849" y="748">
        <parameter key="name" value="performance_fSMRI_bs"/>
        <parameter key="io_object" value="Collection"/>
      </operator>
      <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="68" name="Collect (2)" width="90" x="715" y="1054">
        <parameter key="unfold" value="true"/>
      </operator>
      <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (2)" width="90" x="849" y="1054">
        <parameter key="name" value="performance_fSMRI_bs_o"/>
        <parameter key="io_object" value="Collection"/>
      </operator>
      <operator activated="true" class="set_role" compatibility="7.6.001" expanded="true" height="82" name="Set Role (4)" width="90" x="179" y="1156">
        <parameter key="attribute_name" value="Label"/>
        <parameter key="target_role" value="label"/>
        <list key="set_additional_roles"/>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes (4)" width="90" x="246" y="1003">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attributes" value="Label|CC_Anterior|CC_Central|CC_Mid_Anterior|CC_Mid_Posterior|CC_Posterior|Fifth_Ventricle|Fourth_Ventricle|Left_Accumbens_area|Left_Amygdala|Left_Caudate|Left_Cerebellum_Cortex|Left_Cerebellum_White_Matter|Left_Hippocampus|Left_Inf_Lat_Vent|Left_Lateral_Ventricle|Left_Pallidum|Left_Putamen|Left_Thalamus_Proper|Left_VentralDC|Right_Accumbens_area|Right_Amygdala|Right_Caudate|Right_Cerebellum_Cortex|Right_Cerebellum_White_Matter|Right_Hippocampus|Right_Inf_Lat_Vent|Right_Lateral_Ventricle|Right_Pallidum|Right_Putamen|Right_Thalamus_Proper|Right_VentralDC|lh_bankssts_area|lh_bankssts_thickness|lh_bankssts_volume|lh_caudalanteriorcingulate_area|lh_caudalanteriorcingulate_thickness|lh_caudalanteriorcingulate_volume|lh_caudalmiddlefrontal_area|lh_caudalmiddlefrontal_thickness|lh_caudalmiddlefrontal_volume|lh_cuneus_area|lh_cuneus_thickness|lh_cuneus_volume|lh_entorhinal_area|lh_entorhinal_thickness|lh_entorhinal_volume|lh_frontalpole_area|lh_frontalpole_thickness|lh_frontalpole_volume|lh_fusiform_area|lh_fusiform_thickness|lh_fusiform_volume|lh_inferiorparietal_area|lh_inferiorparietal_thickness|lh_inferiorparietal_volume|lh_inferiortemporal_area|lh_inferiortemporal_thickness|lh_inferiortemporal_volume|lh_insula_area|lh_insula_thickness|lh_insula_volume|lh_isthmuscingulate_area|lh_isthmuscingulate_thickness|lh_isthmuscingulate_volume|lh_lateraloccipital_area|lh_lateraloccipital_thickness|lh_lateraloccipital_volume|lh_lateralorbitofrontal_area|lh_lateralorbitofrontal_thickness|lh_lateralorbitofrontal_volume|lh_lingual_area|lh_lingual_thickness|lh_lingual_volume|lh_medialorbitofrontal_area|lh_medialorbitofrontal_thickness|lh_medialorbitofrontal_volume|lh_middletemporal_area|lh_middletemporal_thickness|lh_middletemporal_volume|lh_paracentral_area|lh_paracentral_thickness|lh_paracentral_volume|lh_parahippocampal_area|lh_parahippocampal_thickness|lh_parahippocampal_volume|lh_parsopercularis_area|lh_parsopercularis_thickness|lh_parsopercularis_volume|lh_parsorbitalis_area|lh_parsorbitalis_thickness|lh_parsorbitalis_volume|lh_parstriangularis_area|lh_parstriangularis_thickness|lh_parstriangularis_volume|lh_pericalcarine_area|lh_pericalcarine_thickness|lh_pericalcarine_volume|lh_postcentral_area|lh_postcentral_thickness|lh_postcentral_volume|lh_posteriorcingulate_area|lh_posteriorcingulate_thickness|lh_posteriorcingulate_volume|lh_precentral_area|lh_precentral_thickness|lh_precentral_volume|lh_precuneus_area|lh_precuneus_thickness|lh_precuneus_volume|lh_rostralanteriorcingulate_area|lh_rostralanteriorcingulate_thickness|lh_rostralanteriorcingulate_volume|lh_rostralmiddlefrontal_area|lh_rostralmiddlefrontal_thickness|lh_rostralmiddlefrontal_volume|lh_superiorfrontal_area|lh_superiorfrontal_thickness|lh_superiorfrontal_volume|lh_superiorparietal_area|lh_superiorparietal_thickness|lh_superiorparietal_volume|lh_superiortemporal_area|lh_superiortemporal_thickness|lh_superiortemporal_volume|lh_supramarginal_area|lh_supramarginal_thickness|lh_supramarginal_volume|lh_temporalpole_area|lh_temporalpole_thickness|lh_temporalpole_volume|lh_transversetemporal_area|lh_transversetemporal_thickness|lh_transversetemporal_volume|rh_bankssts_area|rh_bankssts_thickness|rh_bankssts_volume|rh_caudalanteriorcingulate_area|rh_caudalanteriorcingulate_thickness|rh_caudalanteriorcingulate_volume|rh_caudalmiddlefrontal_area|rh_caudalmiddlefrontal_thickness|rh_caudalmiddlefrontal_volume|rh_cuneus_area|rh_cuneus_thickness|rh_cuneus_volume|rh_entorhinal_area|rh_entorhinal_thickness|rh_entorhinal_volume|rh_frontalpole_area|rh_frontalpole_thickness|rh_frontalpole_volume|rh_fusiform_area|rh_fusiform_thickness|rh_fusiform_volume|rh_inferiorparietal_area|rh_inferiorparietal_thickness|rh_inferiorparietal_volume|rh_inferiortemporal_area|rh_inferiortemporal_thickness|rh_inferiortemporal_volume|rh_insula_area|rh_insula_thickness|rh_insula_volume|rh_isthmuscingulate_area|rh_isthmuscingulate_thickness|rh_isthmuscingulate_volume|rh_lateraloccipital_area|rh_lateraloccipital_thickness|rh_lateraloccipital_volume|rh_lateralorbitofrontal_area|rh_lateralorbitofrontal_thickness|rh_lateralorbitofrontal_volume|rh_lingual_area|rh_lingual_thickness|rh_lingual_volume|rh_medialorbitofrontal_area|rh_medialorbitofrontal_thickness|rh_medialorbitofrontal_volume|rh_middletemporal_area|rh_middletemporal_thickness|rh_middletemporal_volume|rh_paracentral_area|rh_paracentral_thickness|rh_paracentral_volume|rh_parahippocampal_area|rh_parahippocampal_thickness|rh_parahippocampal_volume|rh_parsopercularis_area|rh_parsopercularis_thickness|rh_parsopercularis_volume|rh_parsorbitalis_area|rh_parsorbitalis_thickness|rh_parsorbitalis_volume|rh_parstriangularis_area|rh_parstriangularis_thickness|rh_parstriangularis_volume|rh_pericalcarine_area|rh_pericalcarine_thickness|rh_pericalcarine_volume|rh_postcentral_area|rh_postcentral_thickness|rh_postcentral_volume|rh_posteriorcingulate_area|rh_posteriorcingulate_thickness|rh_posteriorcingulate_volume|rh_precentral_area|rh_precentral_thickness|rh_precentral_volume|rh_precuneus_area|rh_precuneus_thickness|rh_precuneus_volume|rh_rostralanteriorcingulate_area|rh_rostralanteriorcingulate_thickness|rh_rostralanteriorcingulate_volume|rh_rostralmiddlefrontal_area|rh_rostralmiddlefrontal_thickness|rh_rostralmiddlefrontal_volume|rh_superiorfrontal_area|rh_superiorfrontal_thickness|rh_superiorfrontal_volume|rh_superiorparietal_area|rh_superiorparietal_thickness|rh_superiorparietal_volume|rh_superiortemporal_area|rh_superiortemporal_thickness|rh_superiortemporal_volume|rh_supramarginal_area|rh_supramarginal_thickness|rh_supramarginal_volume|rh_temporalpole_area|rh_temporalpole_thickness|rh_temporalpole_volume|rh_transversetemporal_area|rh_transversetemporal_thickness|rh_transversetemporal_volume"/>
      </operator>
      <operator activated="true" class="sample_bootstrapping" compatibility="7.6.001" expanded="true" height="82" name="Sample (Bootstrapping)" width="90" x="380" y="901">
        <parameter key="sample" value="absolute"/>
        <parameter key="sample_size" value="6600"/>
      </operator>
      <operator activated="true" class="shuffle" compatibility="7.6.001" expanded="true" height="82" name="Shuffle" width="90" x="514" y="1003"/>
      <operator activated="true" class="loop_batches" compatibility="7.6.001" expanded="true" height="68" name="Loop Batches" width="90" x="648" y="850">
        <parameter key="batch_size" value="33"/>
        <process expanded="true">
          <operator activated="true" class="handle_exception" compatibility="7.6.001" expanded="true" height="82" name="Handle Exception" width="90" x="246" y="136">
            <process expanded="true">
              <operator activated="true" class="optimize_parameters_grid" compatibility="7.6.001" expanded="true" height="124" name="Optimize Parameters (Grid)" width="90" x="112" y="136">
                <list key="parameters">
                  <parameter key="SVM_full_SVM.C" value="[.03125;32768;20;logarithmic]"/>
                </list>
                <process expanded="true">
                  <operator activated="true" class="normalize" compatibility="7.6.001" expanded="true" height="103" name="Normalize" width="90" x="112" y="85"/>
                  <operator activated="true" class="support_vector_machine_libsvm" compatibility="7.6.001" expanded="true" height="82" name="SVM_full_SVM" width="90" x="246" y="85">
                    <parameter key="kernel_type" value="linear"/>
                    <parameter key="C" value="32768.0"/>
                    <list key="class_weights"/>
                  </operator>
                  <operator activated="true" class="apply_model" compatibility="7.1.001" expanded="true" height="82" name="Apply Model (6)" width="90" x="447" y="136">
                    <list key="application_parameters"/>
                  </operator>
                  <operator activated="true" class="performance_binominal_classification" compatibility="7.6.001" expanded="true" height="82" name="performance_imaging_bs" width="90" x="715" y="289">
                    <parameter key="AUC (optimistic)" value="true"/>
                    <parameter key="AUC" value="true"/>
                    <parameter key="AUC (pessimistic)" value="true"/>
                    <parameter key="precision" value="true"/>
                    <parameter key="recall" value="true"/>
                    <parameter key="sensitivity" value="true"/>
                    <parameter key="specificity" value="true"/>
                    <parameter key="positive_predictive_value" value="true"/>
                    <parameter key="negative_predictive_value" value="true"/>
                  </operator>
                  <connect from_port="input 1" to_op="Normalize" to_port="example set input"/>
                  <connect from_op="Normalize" from_port="example set output" to_op="SVM_full_SVM" to_port="training set"/>
                  <connect from_op="SVM_full_SVM" from_port="model" to_op="Apply Model (6)" to_port="model"/>
                  <connect from_op="SVM_full_SVM" from_port="exampleSet" to_op="Apply Model (6)" to_port="unlabelled data"/>
                  <connect from_op="Apply Model (6)" from_port="labelled data" to_op="performance_imaging_bs" to_port="labelled data"/>
                  <connect from_op="Apply Model (6)" from_port="model" to_port="result 1"/>
                  <connect from_op="performance_imaging_bs" from_port="performance" to_port="performance"/>
                  <portSpacing port="source_input 1" spacing="0"/>
                  <portSpacing port="source_input 2" spacing="0"/>
                  <portSpacing port="sink_performance" spacing="0"/>
                  <portSpacing port="sink_result 1" spacing="0"/>
                  <portSpacing port="sink_result 2" spacing="0"/>
                </process>
              </operator>
              <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (3)" width="90" x="380" y="187">
                <parameter key="name" value="performance_fSMRI_bs"/>
                <parameter key="io_object" value="Collection"/>
              </operator>
              <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="103" name="Collect (3)" width="90" x="514" y="34">
                <parameter key="unfold" value="true"/>
              </operator>
              <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (3)" width="90" x="648" y="136">
                <parameter key="name" value="performance_fSMRI_bs"/>
                <parameter key="io_object" value="Collection"/>
              </operator>
              <operator activated="true" class="retrieve" compatibility="7.6.001" expanded="true" height="68" name="Retrieve (3)" width="90" x="45" y="442">
                <parameter key="repository_entry" value="//Local Repository/data/classifier_structural_self_report_cf_HC_2.5.16"/>
              </operator>
              <operator activated="true" class="filter_examples" compatibility="7.6.001" expanded="true" height="103" name="Filter Examples (3)" width="90" x="45" y="595">
                <list key="filters_list">
                  <parameter key="filters_entry_key" value="ID.does_not_contain.RH"/>
                </list>
              </operator>
              <operator activated="true" class="set_role" compatibility="7.6.001" expanded="true" height="82" name="Set Role (5)" width="90" x="112" y="748">
                <parameter key="attribute_name" value="Label"/>
                <parameter key="target_role" value="label"/>
                <list key="set_additional_roles"/>
              </operator>
              <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (5)" width="90" x="179" y="901">
                <parameter key="name" value="performance_fSMRI_bs_o"/>
                <parameter key="io_object" value="Collection"/>
              </operator>
              <operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes (2)" width="90" x="246" y="595">
                <parameter key="attribute_filter_type" value="subset"/>
                <parameter key="attributes" value="Label|CC_Anterior|CC_Central|CC_Mid_Anterior|CC_Mid_Posterior|CC_Posterior|Fifth_Ventricle|Fourth_Ventricle|Left_Accumbens_area|Left_Amygdala|Left_Caudate|Left_Cerebellum_Cortex|Left_Cerebellum_White_Matter|Left_Hippocampus|Left_Inf_Lat_Vent|Left_Lateral_Ventricle|Left_Pallidum|Left_Putamen|Left_Thalamus_Proper|Left_VentralDC|Right_Accumbens_area|Right_Amygdala|Right_Caudate|Right_Cerebellum_Cortex|Right_Cerebellum_White_Matter|Right_Hippocampus|Right_Inf_Lat_Vent|Right_Lateral_Ventricle|Right_Pallidum|Right_Putamen|Right_Thalamus_Proper|Right_VentralDC|lh_bankssts_area|lh_bankssts_thickness|lh_bankssts_volume|lh_caudalanteriorcingulate_area|lh_caudalanteriorcingulate_thickness|lh_caudalanteriorcingulate_volume|lh_caudalmiddlefrontal_area|lh_caudalmiddlefrontal_thickness|lh_caudalmiddlefrontal_volume|lh_cuneus_area|lh_cuneus_thickness|lh_cuneus_volume|lh_entorhinal_area|lh_entorhinal_thickness|lh_entorhinal_volume|lh_frontalpole_area|lh_frontalpole_thickness|lh_frontalpole_volume|lh_fusiform_area|lh_fusiform_thickness|lh_fusiform_volume|lh_inferiorparietal_area|lh_inferiorparietal_thickness|lh_inferiorparietal_volume|lh_inferiortemporal_area|lh_inferiortemporal_thickness|lh_inferiortemporal_volume|lh_insula_area|lh_insula_thickness|lh_insula_volume|lh_isthmuscingulate_area|lh_isthmuscingulate_thickness|lh_isthmuscingulate_volume|lh_lateraloccipital_area|lh_lateraloccipital_thickness|lh_lateraloccipital_volume|lh_lateralorbitofrontal_area|lh_lateralorbitofrontal_thickness|lh_lateralorbitofrontal_volume|lh_lingual_area|lh_lingual_thickness|lh_lingual_volume|lh_medialorbitofrontal_area|lh_medialorbitofrontal_thickness|lh_medialorbitofrontal_volume|lh_middletemporal_area|lh_middletemporal_thickness|lh_middletemporal_volume|lh_paracentral_area|lh_paracentral_thickness|lh_paracentral_volume|lh_parahippocampal_area|lh_parahippocampal_thickness|lh_parahippocampal_volume|lh_parsopercularis_area|lh_parsopercularis_thickness|lh_parsopercularis_volume|lh_parsorbitalis_area|lh_parsorbitalis_thickness|lh_parsorbitalis_volume|lh_parstriangularis_area|lh_parstriangularis_thickness|lh_parstriangularis_volume|lh_pericalcarine_area|lh_pericalcarine_thickness|lh_pericalcarine_volume|lh_postcentral_area|lh_postcentral_thickness|lh_postcentral_volume|lh_posteriorcingulate_area|lh_posteriorcingulate_thickness|lh_posteriorcingulate_volume|lh_precentral_area|lh_precentral_thickness|lh_precentral_volume|lh_precuneus_area|lh_precuneus_thickness|lh_precuneus_volume|lh_rostralanteriorcingulate_area|lh_rostralanteriorcingulate_thickness|lh_rostralanteriorcingulate_volume|lh_rostralmiddlefrontal_area|lh_rostralmiddlefrontal_thickness|lh_rostralmiddlefrontal_volume|lh_superiorfrontal_area|lh_superiorfrontal_thickness|lh_superiorfrontal_volume|lh_superiorparietal_area|lh_superiorparietal_thickness|lh_superiorparietal_volume|lh_superiortemporal_area|lh_superiortemporal_thickness|lh_superiortemporal_volume|lh_supramarginal_area|lh_supramarginal_thickness|lh_supramarginal_volume|lh_temporalpole_area|lh_temporalpole_thickness|lh_temporalpole_volume|lh_transversetemporal_area|lh_transversetemporal_thickness|lh_transversetemporal_volume|rh_bankssts_area|rh_bankssts_thickness|rh_bankssts_volume|rh_caudalanteriorcingulate_area|rh_caudalanteriorcingulate_thickness|rh_caudalanteriorcingulate_volume|rh_caudalmiddlefrontal_area|rh_caudalmiddlefrontal_thickness|rh_caudalmiddlefrontal_volume|rh_cuneus_area|rh_cuneus_thickness|rh_cuneus_volume|rh_entorhinal_area|rh_entorhinal_thickness|rh_entorhinal_volume|rh_frontalpole_area|rh_frontalpole_thickness|rh_frontalpole_volume|rh_fusiform_area|rh_fusiform_thickness|rh_fusiform_volume|rh_inferiorparietal_area|rh_inferiorparietal_thickness|rh_inferiorparietal_volume|rh_inferiortemporal_area|rh_inferiortemporal_thickness|rh_inferiortemporal_volume|rh_insula_area|rh_insula_thickness|rh_insula_volume|rh_isthmuscingulate_area|rh_isthmuscingulate_thickness|rh_isthmuscingulate_volume|rh_lateraloccipital_area|rh_lateraloccipital_thickness|rh_lateraloccipital_volume|rh_lateralorbitofrontal_area|rh_lateralorbitofrontal_thickness|rh_lateralorbitofrontal_volume|rh_lingual_area|rh_lingual_thickness|rh_lingual_volume|rh_medialorbitofrontal_area|rh_medialorbitofrontal_thickness|rh_medialorbitofrontal_volume|rh_middletemporal_area|rh_middletemporal_thickness|rh_middletemporal_volume|rh_paracentral_area|rh_paracentral_thickness|rh_paracentral_volume|rh_parahippocampal_area|rh_parahippocampal_thickness|rh_parahippocampal_volume|rh_parsopercularis_area|rh_parsopercularis_thickness|rh_parsopercularis_volume|rh_parsorbitalis_area|rh_parsorbitalis_thickness|rh_parsorbitalis_volume|rh_parstriangularis_area|rh_parstriangularis_thickness|rh_parstriangularis_volume|rh_pericalcarine_area|rh_pericalcarine_thickness|rh_pericalcarine_volume|rh_postcentral_area|rh_postcentral_thickness|rh_postcentral_volume|rh_posteriorcingulate_area|rh_posteriorcingulate_thickness|rh_posteriorcingulate_volume|rh_precentral_area|rh_precentral_thickness|rh_precentral_volume|rh_precuneus_area|rh_precuneus_thickness|rh_precuneus_volume|rh_rostralanteriorcingulate_area|rh_rostralanteriorcingulate_thickness|rh_rostralanteriorcingulate_volume|rh_rostralmiddlefrontal_area|rh_rostralmiddlefrontal_thickness|rh_rostralmiddlefrontal_volume|rh_superiorfrontal_area|rh_superiorfrontal_thickness|rh_superiorfrontal_volume|rh_superiorparietal_area|rh_superiorparietal_thickness|rh_superiorparietal_volume|rh_superiortemporal_area|rh_superiortemporal_thickness|rh_superiortemporal_volume|rh_supramarginal_area|rh_supramarginal_thickness|rh_supramarginal_volume|rh_temporalpole_area|rh_temporalpole_thickness|rh_temporalpole_volume|rh_transversetemporal_area|rh_transversetemporal_thickness|rh_transversetemporal_volume"/>
              </operator>
              <operator activated="true" class="normalize" compatibility="7.6.001" expanded="true" height="103" name="Normalize (6)" width="90" x="380" y="646"/>
              <operator activated="true" class="apply_model" compatibility="7.1.001" expanded="true" height="82" name="Apply Model (5)" width="90" x="447" y="493">
                <list key="application_parameters"/>
              </operator>
              <operator activated="true" class="performance_binominal_classification" compatibility="7.6.001" expanded="true" height="82" name="performance_imaging_bs_o" width="90" x="581" y="442">
                <parameter key="AUC (optimistic)" value="true"/>
                <parameter key="AUC" value="true"/>
                <parameter key="AUC (pessimistic)" value="true"/>
                <parameter key="precision" value="true"/>
                <parameter key="recall" value="true"/>
                <parameter key="sensitivity" value="true"/>
                <parameter key="specificity" value="true"/>
                <parameter key="positive_predictive_value" value="true"/>
                <parameter key="negative_predictive_value" value="true"/>
              </operator>
              <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="103" name="Collect (5)" width="90" x="514" y="850">
                <parameter key="unfold" value="true"/>
              </operator>
              <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (5)" width="90" x="648" y="748">
                <parameter key="name" value="performance_fSMRI_bs_o"/>
                <parameter key="io_object" value="Collection"/>
              </operator>
              <connect from_port="in 1" to_op="Optimize Parameters (Grid)" to_port="input 1"/>
              <connect from_op="Optimize Parameters (Grid)" from_port="performance" to_op="Collect (3)" to_port="input 1"/>
              <connect from_op="Optimize Parameters (Grid)" from_port="result 1" to_op="Apply Model (5)" to_port="model"/>
              <connect from_op="Recall (3)" from_port="result" to_op="Collect (3)" to_port="input 2"/>
              <connect from_op="Collect (3)" from_port="collection" to_op="Remember (3)" to_port="store"/>
              <connect from_op="Retrieve (3)" from_port="output" to_op="Filter Examples (3)" to_port="example set input"/>
              <connect from_op="Filter Examples (3)" from_port="example set output" to_op="Set Role (5)" to_port="example set input"/>
              <connect from_op="Set Role (5)" from_port="example set output" to_op="Select Attributes (2)" to_port="example set input"/>
              <connect from_op="Recall (5)" from_port="result" to_op="Collect (5)" to_port="input 2"/>
              <connect from_op="Select Attributes (2)" from_port="example set output" to_op="Normalize (6)" to_port="example set input"/>
              <connect from_op="Normalize (6)" from_port="example set output" to_op="Apply Model (5)" to_port="unlabelled data"/>
              <connect from_op="Apply Model (5)" from_port="labelled data" to_op="performance_imaging_bs_o" to_port="labelled data"/>
              <connect from_op="performance_imaging_bs_o" from_port="performance" to_op="Collect (5)" to_port="input 1"/>
              <connect from_op="Collect (5)" from_port="collection" to_op="Remember (5)" to_port="store"/>
              <portSpacing port="source_in 1" spacing="0"/>
              <portSpacing port="source_in 2" spacing="0"/>
              <portSpacing port="sink_out 1" spacing="0"/>
              <portSpacing port="sink_out 2" spacing="0"/>
            </process>
            <process expanded="true">
              <operator activated="true" class="print_to_console" compatibility="7.6.001" expanded="true" height="82" name="Print to Console" width="90" x="112" y="136">
                <parameter key="log_value" value="ERROR Imaging"/>
              </operator>
              <connect from_port="in 1" to_op="Print to Console" to_port="through 1"/>
              <connect from_op="Print to Console" from_port="through 1" to_port="out 1"/>
              <portSpacing port="source_in 1" spacing="0"/>
              <portSpacing port="source_in 2" spacing="0"/>
              <portSpacing port="sink_out 1" spacing="0"/>
              <portSpacing port="sink_out 2" spacing="0"/>
            </process>
          </operator>
          <connect from_port="exampleSet" to_op="Handle Exception" to_port="in 1"/>
          <portSpacing port="source_exampleSet" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (4)" width="90" x="983" y="850">
        <parameter key="name" value="performance_fSMRI_bs"/>
        <parameter key="io_object" value="Collection"/>
      </operator>
      <operator activated="true" class="write_as_text" compatibility="7.6.001" expanded="true" height="82" name="Write as Text" width="90" x="1117" y="850">
        <parameter key="result_file" value="/Users/lsevel/Desktop/fSMRI_performance_bs.txt"/>
      </operator>
      <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (2)" width="90" x="983" y="1156">
        <parameter key="name" value="performance_fSMRI_bs_o"/>
        <parameter key="io_object" value="Collection"/>
      </operator>
      <operator activated="true" class="write_as_text" compatibility="7.6.001" expanded="true" height="82" name="Write as Text (2)" width="90" x="1117" y="1207">
        <parameter key="result_file" value="/Users/lsevel/Desktop/fSMRI_performance_bs_o.txt"/>
      </operator>
      <connect from_op="Retrieve (2)" from_port="output" to_op="Filter Examples (2)" to_port="example set input"/>
      <connect from_op="Filter Examples (2)" from_port="example set output" to_op="Set Role (4)" to_port="example set input"/>
      <connect from_op="Collect (4)" from_port="collection" to_op="Remember (4)" to_port="store"/>
      <connect from_op="Collect (2)" from_port="collection" to_op="Remember (2)" to_port="store"/>
      <connect from_op="Set Role (4)" from_port="example set output" to_op="Select Attributes (4)" to_port="example set input"/>
      <connect from_op="Select Attributes (4)" from_port="example set output" to_op="Sample (Bootstrapping)" to_port="example set input"/>
      <connect from_op="Sample (Bootstrapping)" from_port="example set output" to_op="Shuffle" to_port="example set input"/>
      <connect from_op="Shuffle" from_port="example set output" to_op="Loop Batches" to_port="example set"/>
      <connect from_op="Recall (4)" from_port="result" to_op="Write as Text" to_port="input 1"/>
      <connect from_op="Recall (2)" from_port="result" to_op="Write as Text (2)" to_port="input 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <description align="center" color="yellow" colored="false" height="182" resized="false" width="180" x="1085" y="177">aSMRI--a priori SMRI features&lt;br&gt;fSMRI--full set of SMRI features&lt;br&gt;SR--self-report&lt;br/&gt;bs=model fit on bootstrapped samples&lt;br/&gt;bs_o--bootstrap generated model fit on original sample</description>
    </process>
  </operator>
</process>

Permutation process:
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
 <context>
 <input/>
 <output/>
 <macros/>
 </context>
 <operator activated="true" class="process" compatibility="7.6.001" expanded="true" name="Process">
 <process expanded="true">
 <operator activated="true" class="retrieve" compatibility="7.6.001" expanded="true" height="68" name="Retrieve (2)" width="90" x="45" y="901">
 <parameter key="repository_entry" value="../../data/_plos_one_permuted_cfs_hc"/>
 </operator>
 <operator activated="true" class="filter_examples" compatibility="7.6.001" expanded="true" height="103" name="Filter Examples (2)" width="90" x="45" y="1003">
 <list key="filters_list">
 <parameter key="filters_entry_key" value="ID.does_not_contain.RH"/>
 </list>
 </operator>
 <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="68" name="Collect (4)" width="90" x="648" y="748">
 <parameter key="unfold" value="true"/>
 </operator>
 <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (4)" width="90" x="849" y="748">
 <parameter key="name" value="performance_fSMRI_perm"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="68" name="Collect (2)" width="90" x="715" y="1054">
 <parameter key="unfold" value="true"/>
 </operator>
 <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (2)" width="90" x="849" y="1054">
 <parameter key="name" value="fSMRI_perm_param"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <operator activated="true" class="set_role" compatibility="7.6.001" expanded="true" height="82" name="Set Role (4)" width="90" x="179" y="1156">
 <parameter key="attribute_name" value="Label"/>
 <parameter key="target_role" value="label"/>
 <list key="set_additional_roles"/>
 </operator>
 <operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes (4)" width="90" x="313" y="901">
 <parameter key="attribute_filter_type" value="subset"/>
 <parameter key="attributes" value="Label|CC_Anterior|CC_Central|CC_Mid_Anterior|CC_Mid_Posterior|CC_Posterior|Fifth_Ventricle|Fourth_Ventricle|Left_Accumbens_area|Left_Amygdala|Left_Caudate|Left_Cerebellum_Cortex|Left_Cerebellum_White_Matter|Left_Hippocampus|Left_Inf_Lat_Vent|Left_Lateral_Ventricle|Left_Pallidum|Left_Putamen|Left_Thalamus_Proper|Left_VentralDC|Right_Accumbens_area|Right_Amygdala|Right_Caudate|Right_Cerebellum_Cortex|Right_Cerebellum_White_Matter|Right_Hippocampus|Right_Inf_Lat_Vent|Right_Lateral_Ventricle|Right_Pallidum|Right_Putamen|Right_Thalamus_Proper|Right_VentralDC|lh_bankssts_area|lh_bankssts_thickness|lh_bankssts_volume|lh_caudalanteriorcingulate_area|lh_caudalanteriorcingulate_thickness|lh_caudalanteriorcingulate_volume|lh_caudalmiddlefrontal_area|lh_caudalmiddlefrontal_thickness|lh_caudalmiddlefrontal_volume|lh_cuneus_area|lh_cuneus_thickness|lh_cuneus_volume|lh_entorhinal_area|lh_entorhinal_thickness|lh_entorhinal_volume|lh_frontalpole_area|lh_frontalpole_thickness|lh_frontalpole_volume|lh_fusiform_area|lh_fusiform_thickness|lh_fusiform_volume|lh_inferiorparietal_area|lh_inferiorparietal_thickness|lh_inferiorparietal_volume|lh_inferiortemporal_area|lh_inferiortemporal_thickness|lh_inferiortemporal_volume|lh_insula_area|lh_insula_thickness|lh_insula_volume|lh_isthmuscingulate_area|lh_isthmuscingulate_thickness|lh_isthmuscingulate_volume|lh_lateraloccipital_area|lh_lateraloccipital_thickness|lh_lateraloccipital_volume|lh_lateralorbitofrontal_area|lh_lateralorbitofrontal_thickness|lh_lateralorbitofrontal_volume|lh_lingual_area|lh_lingual_thickness|lh_lingual_volume|lh_medialorbitofrontal_area|lh_medialorbitofrontal_thickness|lh_medialorbitofrontal_volume|lh_middletemporal_area|lh_middletemporal_thickness|lh_middletemporal_volume|lh_paracentral_area|lh_paracentral_thickness|lh_paracentral_volume|lh_parahippocampal_area|lh_parahippocampal_thickness|lh_parahippocampal_volume|lh_parsopercularis_area|lh_parsopercularis_thickness|lh_parsopercularis_volume|lh_parsorbitalis_area|lh_parsorbitalis_thickness|lh_parsorbitalis_volume|lh_parstriangularis_area|lh_parstriangularis_thickness|lh_parstriangularis_volume|lh_pericalcarine_area|lh_pericalcarine_thickness|lh_pericalcarine_volume|lh_postcentral_area|lh_postcentral_thickness|lh_postcentral_volume|lh_posteriorcingulate_area|lh_posteriorcingulate_thickness|lh_posteriorcingulate_volume|lh_precentral_area|lh_precentral_thickness|lh_precentral_volume|lh_precuneus_area|lh_precuneus_thickness|lh_precuneus_volume|lh_rostralanteriorcingulate_area|lh_rostralanteriorcingulate_thickness|lh_rostralanteriorcingulate_volume|lh_rostralmiddlefrontal_area|lh_rostralmiddlefrontal_thickness|lh_rostralmiddlefrontal_volume|lh_superiorfrontal_area|lh_superiorfrontal_thickness|lh_superiorfrontal_volume|lh_superiorparietal_area|lh_superiorparietal_thickness|lh_superiorparietal_volume|lh_superiortemporal_area|lh_superiortemporal_thickness|lh_superiortemporal_volume|lh_supramarginal_area|lh_supramarginal_thickness|lh_supramarginal_volume|lh_temporalpole_area|lh_temporalpole_thickness|lh_temporalpole_volume|lh_transversetemporal_area|lh_transversetemporal_thickness|lh_transversetemporal_volume|rh_bankssts_area|rh_bankssts_thickness|rh_bankssts_volume|rh_caudalanteriorcingulate_area|rh_caudalanteriorcingulate_thickness|rh_caudalanteriorcingulate_volume|rh_caudalmiddlefrontal_area|rh_caudalmiddlefrontal_thickness|rh_caudalmiddlefrontal_volume|rh_cuneus_area|rh_cuneus_thickness|rh_cuneus_volume|rh_entorhinal_area|rh_entorhinal_thickness|rh_entorhinal_volume|rh_frontalpole_area|rh_frontalpole_thickness|rh_frontalpole_volume|rh_fusiform_area|rh_fusiform_thickness|rh_fusiform_volume|rh_inferiorparietal_area|rh_inferiorparietal_thickness|rh_inferiorparietal_volume|rh_inferiortemporal_area|rh_inferiortemporal_thickness|rh_inferiortemporal_volume|rh_insula_area|rh_insula_thickness|rh_insula_volume|rh_isthmuscingulate_area|rh_isthmuscingulate_thickness|rh_isthmuscingulate_volume|rh_lateraloccipital_area|rh_lateraloccipital_thickness|rh_lateraloccipital_volume|rh_lateralorbitofrontal_area|rh_lateralorbitofrontal_thickness|rh_lateralorbitofrontal_volume|rh_lingual_area|rh_lingual_thickness|rh_lingual_volume|rh_medialorbitofrontal_area|rh_medialorbitofrontal_thickness|rh_medialorbitofrontal_volume|rh_middletemporal_area|rh_middletemporal_thickness|rh_middletemporal_volume|rh_paracentral_area|rh_paracentral_thickness|rh_paracentral_volume|rh_parahippocampal_area|rh_parahippocampal_thickness|rh_parahippocampal_volume|rh_parsopercularis_area|rh_parsopercularis_thickness|rh_parsopercularis_volume|rh_parsorbitalis_area|rh_parsorbitalis_thickness|rh_parsorbitalis_volume|rh_parstriangularis_area|rh_parstriangularis_thickness|rh_parstriangularis_volume|rh_pericalcarine_area|rh_pericalcarine_thickness|rh_pericalcarine_volume|rh_postcentral_area|rh_postcentral_thickness|rh_postcentral_volume|rh_posteriorcingulate_area|rh_posteriorcingulate_thickness|rh_posteriorcingulate_volume|rh_precentral_area|rh_precentral_thickness|rh_precentral_volume|rh_precuneus_area|rh_precuneus_thickness|rh_precuneus_volume|rh_rostralanteriorcingulate_area|rh_rostralanteriorcingulate_thickness|rh_rostralanteriorcingulate_volume|rh_rostralmiddlefrontal_area|rh_rostralmiddlefrontal_thickness|rh_rostralmiddlefrontal_volume|rh_superiorfrontal_area|rh_superiorfrontal_thickness|rh_superiorfrontal_volume|rh_superiorparietal_area|rh_superiorparietal_thickness|rh_superiorparietal_volume|rh_superiortemporal_area|rh_superiortemporal_thickness|rh_superiortemporal_volume|rh_supramarginal_area|rh_supramarginal_thickness|rh_supramarginal_volume|rh_temporalpole_area|rh_temporalpole_thickness|rh_temporalpole_volume|rh_transversetemporal_area|rh_transversetemporal_thickness|rh_transversetemporal_volume"/>
 </operator>
 <operator activated="true" class="loop_clusters" compatibility="7.6.001" expanded="true" height="82" name="Loop Clusters" width="90" x="581" y="901">
 <process expanded="true">
 <operator activated="true" class="handle_exception" compatibility="7.6.001" expanded="true" height="82" name="Handle Exception" width="90" x="246" y="136">
 <process expanded="true">
 <operator activated="true" class="normalize" compatibility="7.6.001" expanded="true" height="103" name="Normalize" width="90" x="45" y="340"/>
 <operator activated="true" class="support_vector_machine_libsvm" compatibility="7.6.001" expanded="true" height="82" name="SVM_apriori_SVM" width="90" x="179" y="340">
 <parameter key="kernel_type" value="linear"/>
 <parameter key="C" value="0.03125"/>
 <list key="class_weights"/>
 </operator>
 <operator activated="true" class="apply_model" compatibility="7.1.001" expanded="true" height="82" name="Apply Model (6)" width="90" x="313" y="340">
 <list key="application_parameters"/>
 </operator>
 <operator activated="true" class="performance_binominal_classification" compatibility="7.6.001" expanded="true" height="82" name="performance_imaging_bs" width="90" x="514" y="289">
 <parameter key="AUC (optimistic)" value="true"/>
 <parameter key="AUC" value="true"/>
 <parameter key="AUC (pessimistic)" value="true"/>
 <parameter key="precision" value="true"/>
 <parameter key="recall" value="true"/>
 <parameter key="sensitivity" value="true"/>
 <parameter key="specificity" value="true"/>
 <parameter key="positive_predictive_value" value="true"/>
 <parameter key="negative_predictive_value" value="true"/>
 </operator>
 <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (3)" width="90" x="380" y="187">
 <parameter key="name" value="performance_fSMRI_perm"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="103" name="Collect (3)" width="90" x="514" y="34">
 <parameter key="unfold" value="true"/>
 </operator>
 <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (3)" width="90" x="648" y="136">
 <parameter key="name" value="performance_fSMRI_perm"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (5)" width="90" x="380" y="901">
 <parameter key="name" value="fSMRI_perm_param"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <operator activated="true" class="collect" compatibility="7.6.001" expanded="true" height="103" name="Collect (5)" width="90" x="514" y="646">
 <parameter key="unfold" value="true"/>
 </operator>
 <operator activated="true" class="remember" compatibility="7.6.001" expanded="true" height="68" name="Remember (5)" width="90" x="648" y="646">
 <parameter key="name" value="fSMRI_perm_param"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <connect from_port="in 1" to_op="Normalize" to_port="example set input"/>
 <connect from_op="Normalize" from_port="example set output" to_op="SVM_apriori_SVM" to_port="training set"/>
 <connect from_op="SVM_apriori_SVM" from_port="model" to_op="Apply Model (6)" to_port="model"/>
 <connect from_op="SVM_apriori_SVM" from_port="exampleSet" to_op="Apply Model (6)" to_port="unlabelled data"/>
 <connect from_op="Apply Model (6)" from_port="labelled data" to_op="performance_imaging_bs" to_port="labelled data"/>
 <connect from_op="Apply Model (6)" from_port="model" to_op="Collect (5)" to_port="input 1"/>
 <connect from_op="performance_imaging_bs" from_port="performance" to_op="Collect (3)" to_port="input 1"/>
 <connect from_op="Recall (3)" from_port="result" to_op="Collect (3)" to_port="input 2"/>
 <connect from_op="Collect (3)" from_port="collection" to_op="Remember (3)" to_port="store"/>
 <connect from_op="Recall (5)" from_port="result" to_op="Collect (5)" to_port="input 2"/>
 <connect from_op="Collect (5)" from_port="collection" to_op="Remember (5)" to_port="store"/>
 <portSpacing port="source_in 1" spacing="0"/>
 <portSpacing port="source_in 2" spacing="0"/>
 <portSpacing port="sink_out 1" spacing="0"/>
 <portSpacing port="sink_out 2" spacing="0"/>
 </process>
 <process expanded="true">
 <operator activated="true" class="print_to_console" compatibility="7.6.001" expanded="true" height="82" name="Print to Console" width="90" x="112" y="136">
 <parameter key="log_value" value="ERROR Imaging"/>
 </operator>
 <connect from_port="in 1" to_op="Print to Console" to_port="through 1"/>
 <connect from_op="Print to Console" from_port="through 1" to_port="out 1"/>
 <portSpacing port="source_in 1" spacing="0"/>
 <portSpacing port="source_in 2" spacing="0"/>
 <portSpacing port="sink_out 1" spacing="0"/>
 <portSpacing port="sink_out 2" spacing="0"/>
 </process>
 </operator>
 <connect from_port="cluster subset" to_op="Handle Exception" to_port="in 1"/>
 <portSpacing port="source_cluster subset" spacing="0"/>
 <portSpacing port="source_in 1" spacing="0"/>
 <portSpacing port="sink_out 1" spacing="0"/>
 </process>
 </operator>
 <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (4)" width="90" x="983" y="850">
 <parameter key="name" value="performance_fSMRI_perm"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <operator activated="true" class="write_as_text" compatibility="7.6.001" expanded="true" height="82" name="Write as Text" width="90" x="1117" y="850">
 <parameter key="result_file" value="/Users/lsevel/Desktop/fSMRI_performance_perm.txt"/>
 </operator>
 <operator activated="true" class="recall" compatibility="7.6.001" expanded="true" height="68" name="Recall (2)" width="90" x="983" y="1156">
 <parameter key="name" value="fSMRI_perm_param"/>
 <parameter key="io_object" value="Collection"/>
 </operator>
 <operator activated="true" class="write_as_text" compatibility="7.6.001" expanded="true" height="82" name="Write as Text (2)" width="90" x="1117" y="1207">
 <parameter key="result_file" value="/Users/lsevel/Desktop/fSMRI_perm_param.txt"/>
 </operator>
 <connect from_op="Retrieve (2)" from_port="output" to_op="Filter Examples (2)" to_port="example set input"/>
 <connect from_op="Filter Examples (2)" from_port="example set output" to_op="Set Role (4)" to_port="example set input"/>
 <connect from_op="Collect (4)" from_port="collection" to_op="Remember (4)" to_port="store"/>
 <connect from_op="Collect (2)" from_port="collection" to_op="Remember (2)" to_port="store"/>
 <connect from_op="Set Role (4)" from_port="example set output" to_op="Select Attributes (4)" to_port="example set input"/>
 <connect from_op="Select Attributes (4)" from_port="example set output" to_op="Loop Clusters" to_port="example set"/>
 <connect from_op="Recall (4)" from_port="result" to_op="Write as Text" to_port="input 1"/>
 <connect from_op="Recall (2)" from_port="result" to_op="Write as Text (2)" to_port="input 1"/>
 <portSpacing port="source_input 1" spacing="0"/>
 <portSpacing port="sink_result 1" spacing="0"/>
 <description align="center" color="yellow" colored="false" height="182" resized="false" width="180" x="1085" y="177">aSMRI--a priori SMRI features&lt;br&gt;fSMRI--full set of SMRI features&lt;br&gt;SR--self-report&lt;br/&gt;bs=model fit on bootstrapped samples&lt;br/&gt;bs_o--bootstrap generated model fit on original sample</description>
 </process>
 </operator>
</process>

Is this possible in rapidminer? 

 

Thank you all,

Tagged:

Answers

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    Hello @lsevel - that's an interesting question.  Generally I use one of the following operators depending on what is needed:

       Aggregate

       Generate Aggregation

       Generate Attributes (and using the function generator)

     

    Does this help?  I will leave it to @IngoRM or @TobiasMalbrecht who wrote the Logistic Regression Optimization code to chime in on the specifics of implementing Harrell's Optimism.  :)

     

    Scott

     

     

    Scott

Sign In or Register to comment.