Jump to content
43oh

Is there an Energia matrix math library for the msp430 and Stellaris


Recommended Posts

I was wondering if there was a matrix math library for Energia yet? I wanted to port some neural networks and genetic alogrithms over but the idea of 15 or so manual loop unrolls is just a no go. The DSP lbraries from ARM just aren't working and this is a complete deal killer for me using Energia or TI products. Even with the ATtiny85 and Arduino I can do floating point matrix math for tiny networks.

 

I can get matrix math to compile with Energia but it's just obviously spitting out wrong answers.

 

BTW, I can get the logistic map equation to run 5.5 million iterations per second with the Stellaris Launchpad at single precision float and 6.5 million with half floats non-SIMD optimized. Not so fast with the MSP430G2553! But, this slows way down unless it's in another loop besides void loop() which is kind of odd.

 

I do greatly appreciate people's work and effort in Energia. I loath Code Composer Studio and find even the Xilinx FPGA webpack easier to use.

 

Thanks your time!

Link to post
Share on other sites

Calculations with reals require loading a 6KB library so forget about it with a MSP430G2553.

 

For the StellarPad, try this trivial library attached.

 

 

MatrixMath.cpp Library for MatrixMath

 

Created by Charlie Matlack on 12/18/10.

Modified from code by RobH45345 on Arduino Forums, taken from unknown source.

 

Rei Vilo - Apr 15, 2011

Edited by Rei Vilo
Link to post
Share on other sites

I appreciate your help but has then been varified with the Stellaris Launchpad? I have the latest Energia standard install and tried to run the example provided with the .h and .cpp in the same subdirectory, the inc subdirectory and the driverlib directory which all give the same errror. I also tried import library option.

 

I googled for lm4fcpp.ld and nothing came up.

 

..lib/gcc/arm-noe-eabi/4.7.1/../../../arm-noe-eabi/binbin/ld.exe   can not open linker script file

 

....\hardware\lm4f\cores\lm4f\lm4fcpp.ld:   Invalid arguement

 

Could it be that I'm just over looking something? I'm sure it is.

 

Again, I appreciate your time.

 

edit: I did just try compiling this on the MSP430G2553 and there were 26 errors.

 

2nd edit: I contacted TI to see if they could be of any help with a working matrix math library for Energia and told them it would be for various types of open source neural networks (the reccurent class, feedforward/back prop, Hopfield, adaptive learning and momentum rate, node prunning, in system monitoring to help deal with local minima or lack of convergence, taking advatage of SIMD with the Stellaris and the like.

Link to post
Share on other sites

msp430g2553.h uses some defines that are conflicting with this code.

 

...

/************************************************************
* STATUS REGISTER BITS
************************************************************/

#define C                   (0x0001)
#define Z                   (0x0002)
#define N                   (0x0004)
#define V                   (0x0100)
#define GIE                 (0x0008)
#define CPUOFF              (0x0010)
#define OSCOFF              (0x0020)
#define SCG0                (0x0040)
#define SCG1                (0x0080)

...

 

The #define for 'C' and 'N' are the problems. You need to changes those names to something else in the MatrixMath code.

 

-rick

Link to post
Share on other sites

Thanks again Rei!

 

I reloaded Energia to get rid of the linker errors.

 

When an Energia sketch uploads, does it first erase everything in the LM4F's flash? It appears to do so with the MSP430 but not so sure about the LM4F. I'll upload one sketch then another and I can see that not everything has been erased by the printing errors.

 

The example MatrixMath sketch I only got to print occasionally otherwise nothing. At one point I got the same linker problems as above so had to do a 3rd complete Energia reload.

 

I would run the AnalogSerial example sketch and then the MatrixMath sketch and back again and it's a hit and miss if MatrixMath will print and is highly inconsistant of how it will print. I think there might be something more going on.

 

BTW, Windows 7, EeePC.

 

I have no idea what's going on.

 

edit: I tried another Stellaris board. One reads on Com5 the other Com10. Is this normal? Neither are currently working with the MatrixMath example. I have to shut down and relaunch Energia sometimes several times to get the basic sketches to work when swapping boards. I've never encountered this with Arduino or with Xess FPGAs.

Link to post
Share on other sites

As a Mac user, I can't help you much with the issue on the serial port with Windows.

 

... but Mac OS X has its own issues  :smile:

 

I'd recommend to end the serial connection in a clean way with

 

Serial.end()

Just add to setup()

 

  pinMode(PUSH2, INPUT_PULLUP);     
  Serial.println("press PUSH2 to exit serial");    

and to loop()

 

  if (digitalRead(PUSH2)==LOW) {
    Serial.println("exit"); 
    Serial.end();
    while(true); // endless loop
  }

to get rid of most of the issues related to the serial port.

Link to post
Share on other sites

Thanks Rei. I'm going to have to wait until Energia is past the alpha phase. It's too unreliable/unstable at this point particularly seeing the M4F not being erased like it does with the MSP430. I have a feeling this is a root of a lot of problems I'm having.

 

Once agin though, I do greatly appreciate people's work on Energia and understand that theses things take time.

Link to post
Share on other sites

As Rei said, this library works as expected on Stellaris Launchpad. To make sure I just tried it again and it works as expected. Now with that said, it doesn't work for you and I would like to get to the bottom of this.

 

The zip file posted in this thread is not a true library. It's a Sketch with multiple tabs. The way to open it is to unzip it and then in Energia open the main Sketch with file->open and point it to MatrixMathExample. Once opened, you should now have three tabs named: MatrixMathExample, MatrixMath.cpp and MatrixMath.h.

 

W.r.t flash erase, when uploading it does erase flash but only the pages that are about to be programmed. It does not do a full flash erase since that would take to long and is not necessary. I have tried to reproduce the inconsistent printing and switched between ASCII table example and the MatrixMath example a couple times but not luck.

 

The COM port number will most likely change for every different LaunchPad which is a windows thing for COM ports..

 

I really want to see you succeed with this. The Stellaris LaunchPad is ideal for this kind of math and computational tasks with it's FPU.

 

Robert

Link to post
Share on other sites

Thank you kindly Robert and appreciate your work. I agree, Stellaris would be just ideal for embeded high speed neural networks particularly with the SIMD set up properly for dual 16 bit half floats because with neural networks there's essentially no benefit to 32 bit floats.

But, not a thing. The example matrix sketch is loaded, have the three tabs including .ccp and .h, everything is on COM10, code and terminal set to 9600, complies to 10,704 bytes, not a thing prints out. It simply doesn't work.

Load the AnalogSerialRead example sketch. Works perfectly.

Try to load nn_sine_solve, MatrixMath.cpp and .h show up in the 3 tabs, get nonsense answer because of the nuances of the MatrixMath.cpp file but runs perfectly on the Arduino and even on the ATtiny85 for tiny networks. But at least this prints.

Look at this code below in a sketch if you could please. I hade to make a change of ....exp(-Accum)); to .....2.718 * (-Accum)); to get it to compile. It's changed from the original 7-4 encoder 7-8-4 network to solving the sine function with a 1-3-1 network. This is the code I want to use and highly modified to include adaptive learning rates, recurrent etc. It's a text book perfect example of a feedforward/back prop ANN and a good template for designing other classes of neural networks.

Looking at the MatrixMath.cpp, the below looks like a handful if one has to use math.MatrixMult yet runs just fine on the Arduino even without "math.h". It appears that the MatrixMath library would need major revisions but at least it prints unlike the MatrixMath example sketch. It appears from the .ccp file that you can not mix and match.

math.mathMatrix(HiddenWeights[j] = 2.0 * ( Rando - 0.5 ) * InitialWeightMax) ; does not work
HiddenWeights[j] = math.mathMatrix(2.0 * ( Rando - 0.5 ) * InitialWeightMax); does not work
Accum += math.MatrixMult((Input([p][j] * HiddenWeights[j])) ; does not work

Is there a way to get matrices that have additions/subtractions and multiplications mixed together? This would be a major stubling block to porting some otherwise easily ported Arduino code that does not have this issue. How would one go about this without rewriting most of the code and how would one go about making Energia more compatable code with Arduino? If one does have to do a complete code overhaul then that's just the way it is and it's understandable but what is Arduino doing to not have these matrices problems?


Again, I really do appreciate what you do.

 

/

/******************************************************************
 * ArduinoANN - An artificial neural network for the Arduino
 * All basic settings can be controlled via the Network Configuration
 * section.
 * See robotics.hobbizine.com/arduinoann.html for details.
 ******************************************************************/

#include "MatrixMath.h"

/******************************************************************
 * Network Configuration - customized per network
 ******************************************************************/
MatrixMath math;

const int PatternCount = 10;
const int InputNodes = 1;
const int HiddenNodes = 3;
const int OutputNodes = 1;
const float LearningRate = 0.6;
const float Momentum = 0.9;
const float InitialWeightMax = 0.5;
const float Success = 0.0004;

const float Input[PatternCount][InputNodes] = {
  { 0},
  { 0.1},
  {0.2},  
  {0.3},
  {0.4},
  { 0.5},
  { 0.6},  
  {0.7},
  {0.8},
  {0.9}
 
 
};

const float Target[PatternCount][OutputNodes] = {
 
  {0},  //0
  {0.1736}, //10
  {0.3420}, //20
  {0.5},    //30
  {0.6428}, //40
  {0.7660},  //50
  {0.8660},  //60
  {.9397},   //70
  {0.9848},   //80
  {0.99}
 
};

/******************************************************************
 * End Network Configuration
 ******************************************************************/


int i, j, p, q, r;
int ReportEvery1000;
int RandomizedIndex[PatternCount];
long  TrainingCycle;
float Rando;
float Error;
float Accum;


float Hidden[HiddenNodes];
float Output[OutputNodes];
float HiddenWeights[InputNodes+1][HiddenNodes];
float OutputWeights[HiddenNodes+1][OutputNodes];
float HiddenDelta[HiddenNodes];
float OutputDelta[OutputNodes];
float ChangeHiddenWeights[InputNodes+1][HiddenNodes];
float ChangeOutputWeights[HiddenNodes+1][OutputNodes];

void setup(){
  Serial.begin(9600);
  randomSeed(analogRead(3));
  ReportEvery1000 = 1;
  for( p = 0 ; p < PatternCount ; p++ ) {    
    RandomizedIndex[p] = p ;
  }
}  

void loop (){


/******************************************************************
* Initialize HiddenWeights and ChangeHiddenWeights
******************************************************************/

  for( i = 0 ; i < HiddenNodes ; i++ ) {    
    for( j = 0 ; j <= InputNodes ; j++ ) {
      ChangeHiddenWeights[j][i] = 0.0 ;
      Rando = float(random(100))/100;
   (HiddenWeights[j][i] = (2.0 * ( Rando - 0.5 ) * InitialWeightMax)) ;
    }
  }
/******************************************************************
* Initialize OutputWeights and ChangeOutputWeights
******************************************************************/

  for( i = 0 ; i < OutputNodes ; i ++ ) {    
    for( j = 0 ; j <= HiddenNodes ; j++ ) {
      ChangeOutputWeights[j][i] = 0.0 ;  
      Rando = float(random(100))/100;        
      OutputWeights[j][i] = 2.0 * ( Rando - 0.5 ) * InitialWeightMax ;
    }
  }
  Serial.println("Initial/Untrained Outputs: ");
  toTerminal();
/******************************************************************
* Begin training
******************************************************************/

  for( TrainingCycle = 1 ; TrainingCycle < 2147483647 ; TrainingCycle++) {    

/******************************************************************
* Randomize order of training patterns
******************************************************************/

    for( p = 0 ; p < PatternCount ; p++) {
      q = random(PatternCount);
      r = RandomizedIndex[p] ;
      RandomizedIndex[p] = RandomizedIndex[q] ;
      RandomizedIndex[q] = r ;
    }
    Error = 0.0 ;
/******************************************************************
* Cycle through each training pattern in the randomized order
******************************************************************/
    for( q = 0 ; q < PatternCount ; q++ ) {    
      p = RandomizedIndex[q];

/******************************************************************
* Compute hidden layer activations
******************************************************************/

      for( i = 0 ; i < HiddenNodes ; i++ ) {    
        Accum = HiddenWeights[InputNodes][i] ;
        for( j = 0 ; j < InputNodes ; j++ ) {
          Accum += math.MatrixMult(Input([p][j] * HiddenWeights[j][i])) ;
        }
        Hidden[i] = 1.0/(1.0 + 2.718*(-Accum)) ;
      }

/******************************************************************
* Compute output layer activations and calculate errors
******************************************************************/

      for( i = 0 ; i < OutputNodes ; i++ ) {    
        Accum = OutputWeights[HiddenNodes][i] ;
        for( j = 0 ; j < HiddenNodes ; j++ ) {
          Accum += Hidden[j] * OutputWeights[j][i] ;
        }
        Output[i] = 1.0/(1.0 + 2.718 * (-Accum)) ;   
        OutputDelta[i] = (Target[p][i] - Output[i]) * Output[i] * (1.0 - Output[i]) ;   
        Error += 0.5 * (Target[p][i] - Output[i]) * (Target[p][i] - Output[i]) ;
      }

/******************************************************************
* Backpropagate errors to hidden layer
******************************************************************/

      for( i = 0 ; i < HiddenNodes ; i++ ) {    
        Accum = 0.0 ;
        for( j = 0 ; j < OutputNodes ; j++ ) {
          Accum += OutputWeights[i][j] * OutputDelta[j] ;
        }
        HiddenDelta[i] = Accum * Hidden[i] * (1.0 - Hidden[i]) ;
      }


/******************************************************************
* Update Inner-->Hidden Weights
******************************************************************/


      for( i = 0 ; i < HiddenNodes ; i++ ) {     
        ChangeHiddenWeights[InputNodes][i] = LearningRate * HiddenDelta[i] + Momentum * ChangeHiddenWeights[InputNodes][i] ;
        HiddenWeights[InputNodes][i] += ChangeHiddenWeights[InputNodes][i] ;
        for( j = 0 ; j < InputNodes ; j++ ) {
          ChangeHiddenWeights[j][i] = LearningRate * Input[p][j] * HiddenDelta[i] + Momentum * ChangeHiddenWeights[j][i];
          HiddenWeights[j][i] += ChangeHiddenWeights[j][i] ;
        }
      }

/******************************************************************
* Update Hidden-->Output Weights
******************************************************************/

      for( i = 0 ; i < OutputNodes ; i ++ ) {    
        ChangeOutputWeights[HiddenNodes][i] = LearningRate * OutputDelta[i] + Momentum * ChangeOutputWeights[HiddenNodes][i] ;
        OutputWeights[HiddenNodes][i] += ChangeOutputWeights[HiddenNodes][i] ;
        for( j = 0 ; j < HiddenNodes ; j++ ) {
          ChangeOutputWeights[j][i] = LearningRate * Hidden[j] * OutputDelta[i] + Momentum * ChangeOutputWeights[j][i] ;
          OutputWeights[j][i] += ChangeOutputWeights[j][i] ;
        }
      }
    }

/******************************************************************
* Every 1000 cycles send data to terminal for display
******************************************************************/
    ReportEvery1000 = ReportEvery1000 - 1;
    if (ReportEvery1000 == 0)
    {
      Serial.println();
      Serial.println();
      Serial.print ("TrainingCycle: ");
      Serial.print (TrainingCycle);
      Serial.print ("  Error = ");
      Serial.println (Error, 5);

      toTerminal();

      if (TrainingCycle==1)
      {
        ReportEvery1000 = 999;
      }
      else
      {
        ReportEvery1000 = 1000;
      }
    }    


/******************************************************************
* If error rate is less than pre-determined threshold then end
******************************************************************/

    if( Error < Success ) break ;  
  }
  Serial.println ();
  Serial.println();
  Serial.print ("TrainingCycle: ");
  Serial.print (TrainingCycle);
  Serial.print ("  Error = ");
  Serial.println (Error, 5);

  toTerminal();

  Serial.println ();  
  Serial.println ();
  Serial.println ("Training Set Solved! ");
  Serial.println ("--------");
  Serial.println ();
  Serial.println ();  
  ReportEvery1000 = 1;
}

void toTerminal()
{

  for( p = 0 ; p < PatternCount ; p++ ) {
    Serial.println();
    Serial.print ("  Training Pattern: ");
    Serial.println (p);      
    Serial.print ("  Input ");
    for( i = 0 ; i < InputNodes ; i++ ) {
      Serial.print (Input[p][i], DEC);
      Serial.print (" ");
    }
    Serial.print ("  Target ");
    for( i = 0 ; i < OutputNodes ; i++ ) {
      Serial.print (Target[p][i], DEC);
      Serial.print (" ");
    }
/******************************************************************
* Compute hidden layer activations
******************************************************************/

    for( i = 0 ; i < HiddenNodes ; i++ ) {    
      Accum = HiddenWeights[InputNodes][i] ;
      for( j = 0 ; j < InputNodes ; j++ ) {
        Accum += Input[p][j] * HiddenWeights[j][i] ;
      }
      Hidden[i] = 1.0/(1.0 + 2.718 * (-Accum)) ;
    }

/******************************************************************
* Compute output layer activations and calculate errors
******************************************************************/

    for( i = 0 ; i < OutputNodes ; i++ ) {    
      Accum = OutputWeights[HiddenNodes][i] ;
      for( j = 0 ; j < HiddenNodes ; j++ ) {
        Accum += Hidden[j] * OutputWeights[j][i] ;
      }
      Output[i] = 1.0/(1.0 + 2.718 * (-Accum)) ;
    }
    Serial.print ("  Output ");
    for( i = 0 ; i < OutputNodes ; i++ ) {       
      Serial.print (Output[i], 5);
      Serial.print (" ");
      
     

    }
  }


}




Link to post
Share on other sites

I hate saying this, I appreciate what you're doing and your work Rei, I really do, but it does not work for me after 3 reloads of Energia and 2 different boards. All of the example programs with the Energia software package work. The new files were put in the appropriate place. The shots below are when it would actually print. I did get the inverted print matrix one time but most of the time nothing will print out or just a few numbers will. It's hit and miss but mostly miss. The old files were deleted and replaced, did a reboot but it still doesn't work. It's got to be something with my system but I'm just going to have to forgo Energia for the time being for neural networks and get some Arduino Dues and use fixed point math.

 

Thank you very much for your time and effort!

 

shot of program loaded

 

http://i.imgur.com/n1nG0qL.jpg

 

terminal 0

 

http://i.imgur.com/HkVLlPT.jpg

 

terminal 1

 

http://i.imgur.com/HNeEEYJ.jpg

 

terminal 2

 

http://i.imgur.com/ESe3V5K.jpg

 

terminal 3

 

http://i.imgur.com/Ag4M43J.jpg

 

terminal 4

 

http://i.imgur.com/8HDohMy.jpg

Link to post
Share on other sites

Thank you Robert.

The exact code in the screen shots above is the latest MatrixMath example Rei posted in that zip file along with the .h and .cpp files that compiles to 10,800 bytes. I don't doubt Rei's work one bit.

I agree with Rei's assessment of some sort of serial port issue. Perhaps a driver or something. But I only have serial port problems with this code. Everything else prints out fine. And it's not consistent as shown in the screen shots. That's what makes this such a tough problem. I'm more of an electronics guy than a software expert (to me, a PID system is a 25 cent dual op amp and a power transistor. A balancing robot is a modded servo with a 555 chip or 2 transistors.That's why I swept the 555 contest a few years ago) so I can only do so much software hacking like loop unrolling and stumbling through software problems like code porting.

But I'm not sure if the matrix multiplication is even working properly on my system (and I want to publicly emphasize my system out of respect for Rei) in the example MatrixMath code.

It's these intermittent problems that are the tough time killers to solve.

edit-I did have some other serial port problems when it appeared that not all of the old code was being erased. That's why I asked earlier about the full erase like as is done with the MSP430s

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...