Archive For November 16, 2015
The GNU parallel allows you take advantage of multiple cores on your machine Nice (bioinformatics heavy) explanation here
This follows from the previous post where i tried out Hadoop. This uses spark on the same comet cluster. To run interactively, the best thing to do first off is add this to my .bashrc
|
1 |
module load python scipy |
Then in the folder with the spark code (get it here), run this
|
1 2 3 4 5 |
# this didn't work for me #qsub spark.cmd # this only worked, after commenting out # the line that says res=ucla2015 sbatch spark.cmd |
Basically, it sleeps for 4…