Speed up MATLAB code on Linux Cluster - repost

IN PROGRESS
Bids
6
Avg Bid (USD)
$784
Project Budget (USD)
$30 - $250

Project Description:
Hi there,
For a customer I've got a matlab project:

I've got a quite large matlab code that takes a a long time to run. Toe code solve an economic model numerically and consists of two main part: The first part solves an analytic equiation of on a large multidimensional grid (similar to the finite elements method). The second part simulates the model using random numbers and the solutions found in the first part. Part 1 is about 90 lines including emtpy lines (see EGM_parfor.m), and part 2 is about 160 lines (see Simulate.m). Both files use similar functions.

The code runs fine, but it doesn't scale well and the speed up is not so large when i run on multiple cores. What I'd like help with is:

EGM_parfor.m:
Optimize the code so that it runs faster in parallel. Please leave inputs and outputs unchanged so I don't need to alter the code in any other place in order to use the changes.
* There is probably a lot of performance to gain by correctly accessing the memmory ex by accessing data in colunmns and avoid creating unneccesary variables as outlined in [url removed, login to view])
* There might be some help on this blog?: http://www.walkingrandomly.com/?p=4912 (But the code don't have many matrix multiplications)

Simulate.m:
This code is not yet parallelized. It is much quicker than the EGM_parfor, but still a little slow. Please leave inputs and outputs unchanged so I don't need to alter the code in any other place in order to use the changes.
* Please optimize the code to run faster, maybe some of the tricks from above?
* Further, please paralellize the code by adding an "outer" loop around it that divides the code into N "chunks" that each run on a separate matlab worker. For example, if there is 100k simulations and N=4, then create a parfor-loop that runs the first 25k simulations in the first iteration, simulations 25001-50k on the second iteration, and so on.

I attach the files you need to see in order to make a bid. Once I've decided on a bidder I will send all of the files as a .zip-file.

In order for me to take your bid seriously please write a few sentences about your experience of using PCT and MDCS in general and your experience with optimizing such code on a 64bit linux cluster specifically.

I will put all the money on escrow, but not release the payment until I'm 100% satisfied that the code is working as before, but substantially faster. I expect a code improvement of around 10 times (EG_parfor + Simulate). If you can speed it up as much as 50 times I will double your bid as payment, 100 times tripple it.

Additional Project Description:
02/27/2014 at 3:09 IST
Hi, I'm sorry I was late in replying to this thread and had to re-open it.. My client had some trouble with the software and didn't want to send the broken version. I uploaded the full code as a zip-file.

I also create milestones for each part. The priorities and descriptions (in addition to those above) milestones are given below. Please note that I want ONE deliverable per milestone so that I can verify them one-by-one. After you finished one milestone, send me the files before starting on the next one.

1) plotSimulationGraphs.m and functions therein.
This is basically a function that iterates through the simulation results and does prctile() a bunch of times, and stores the result in a dataset. It takes forever. I have no clear idea about how to speed this up. I want the input and output to be the same. I also want the "configurability" of the code to remain (the part where I choose which variables and levels to compute the prctile for). But please change the structure of the code to make it faster.
Milestones:
A) if you can speed it up by a factor of 20 you get $20, then
B) by a factor of 50 (you get $20)

2) EGM.m and functions therein.
The milestones will be:
A) Correctly access the memory as described in http://www.mathworks.com/company/newsletters/articles/programming-patterns-maximizing-code-performance-by-optimizing-memory-access.html. Remember that the input and output shall be unchanged, so if you reshuffle the dimensions of the matricies (which I assume you will do) then you have to reverse this when returning the data. This will be $20.
B) An additional speed up by a factor of 15 will be another $20.

3) Simulate.m and functions therein:
The milestones will be:
A) Further, please paralellize the code by adding an "outer" loop around it that divides the code into N "chunks" that each run on a separate matlab worker. For example, if there is 100k simulations and N=4, then create a parfor-loop that runs the first 25k simulations in the first iteration, simulations 25001-50k on the second iteration, and so on. This will be $20.
B) An additional speedup of a factor of 15 will be another $20.

4) Restructure code.
The code is quite complex and hard to follow. If you can provide me with a code that is a lot simpler to read and understand (and either fully adopts or fully skips the OOP) that will be another $50.

Skills required:
Algorithm, Engineering, Mathematics, Matlab & Mathematica, Physics
Additional Files: 217-Freelancer.zip
About the employer:
Verified
Public Clarification Board
Bids are hidden by the project creator. Log in as the employer to view bids or to bid on this project.
You will not be able to bid on this project if you are not qualified in one of the job categories. To see your qualifications click here.


Hire enggworks
$ 222
in 15 days
$ 250
in 30 days
$ 500
in 10 days
Hire stephanembinky
$ 211
in 15 days
Hire erasiterajiv
$ 188
in 5 days
Hire CJentzsch
$ 3333
in 30 days