how to generate a neural net weight-matrix?

eldenoso

Hello altogether,

as I am currently trying to get more insight into what neural nets are doing, I wanted to know, whether there is a way of exporting the weight-matrices of a neural net so that you can visualize it (e.g. heat map). By then changing the training set a couple of times, you may see differences in the "heat-map-weight-matrix" and can conclude which attributes are the ones that matter.

I know that there are real scientists out there, searching for answers to the black-box-problem of neural nets. But my curiosity just drives me towards this way and I would be glad to try this out :-).

Thank you for your answers

 

Philipp

MartinLiebig

Hi Philipp,

 

in fact you can use Groovy Script to do this. I've tried this, but it wasn't as easy as i thought. Attached is a process with the script. It always extracts the first Layer. i've not tested it in various layouts though

 

Best,

Martin

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.3.001">
 <context>
 <input/>
 <output/>
 <macros/>
 </context>
 <operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
 <process expanded="true">
 <operator activated="true" class="retrieve" compatibility="7.3.001" expanded="true" height="68" name="Retrieve Sonar" width="90" x="112" y="34">
 <parameter key="repository_entry" value="//Samples/data/Sonar"/>
 </operator>
 <operator activated="true" class="neural_net" compatibility="7.3.001" expanded="true" height="82" name="Neural Net" width="90" x="313" y="34">
 <list key="hidden_layers">
 <parameter key="1" value="10"/>
 </list>
 </operator>
 <operator activated="true" class="execute_script" compatibility="7.3.001" expanded="true" height="82" name="Execute Script" width="90" x="581" y="34">
 <parameter key="script" value=" import java.util.logging.Level&#10; import com.rapidminer.tools.LogService;&#10; import com.rapidminer.tools.Ontology;&#10; import com.rapidminer.example.utils.ExampleSetBuilder;&#10; import com.rapidminer.example.utils.ExampleSets;&#10;// we assume only one layer&#10;model = input[0]&#10;&#10;int NumberOfNodes = model.innerNodes.size()&#10;int weightLength = model.innerNodes[10].getWeights().size()&#10;&#10;for(int k = 0; k &lt; NumberOfNodes; ++k){&#10;&#9;if(model.innerNodes[k].layerIndex == 0)&#10;&#9;&#9;weightLength = model.innerNodes[k].getWeights().size();&#10;&#10;}&#10;attributes= new Attribute[weightLength];&#10;for(int n = 0; n &lt; weightLength; n++){&#10; &#9;attributes[n] = AttributeFactory.createAttribute(&quot;Weight_&quot;+Integer.toString(n), Ontology.REAL);&#10;}&#10;ExampleSetBuilder builder = ExampleSets.from(attributes)&#10;double[] row = new double[weightLength]&#10;&#10;for(int n = 0; n &lt; NumberOfNodes; n++){&#10;&#9;innerNodes = model.innerNodes[n]&#10;&#9;&#10;&#9;weights = innerNodes.getWeights()&#10;&#9;LogService.root.log(Level.SEVERE,Integer.toString(weights.size()))&#10;&#9;LogService.root.log(Level.SEVERE,Integer.toString(weightLength))&#10;&#9;for(int i = 0; i &lt; weights.size(); ++i){&#10;&#9;&#9;if(innerNodes.layerIndex == 0){&#10;&#9;&#9;&#9;row[i] = weights[i]&#10;&#9;&#9;}&#10;&#9;&#9;//LogService.root.log(Level.INFO,Double.toString(weights[i]))&#10;&#9;}&#10;&#9;if(innerNodes.layerIndex==0)&#10;&#9;&#9;builder.addRow(row)&#10;}&#10;&#10;return builder.build()"/>
 </operator>
 <connect from_op="Retrieve Sonar" from_port="output" to_op="Neural Net" to_port="training set"/>
 <connect from_op="Neural Net" from_port="model" to_op="Execute Script" to_port="input 1"/>
 <connect from_op="Execute Script" from_port="output 1" to_port="result 1"/>
 <portSpacing port="source_input 1" spacing="0"/>
 <portSpacing port="sink_result 1" spacing="0"/>
 <portSpacing port="sink_result 2" spacing="0"/>
 </process>
 </operator>
</process>

 

eldenoso

Perfect Martin! Thank you. 

In your case the data means that there were ten neurons in the first layer with 60 edges (weights), right?

Best, 

Philipp

MartinLiebig

Hi,

 

yes. And two output nodes which are interestingly part of the innerNodes object and just identfiable by their layerId...

 

Best,

Martin