Day 32

Day 32

Oops, it's 2010 already...

Day 30

Day 30

After spending a hell of a lot of time trying to fix one annoying issue caused by inadequacy of some online tutorials (i.e. by my own stupidity) spotted a very good online guide on SSH: http://kimmo.suominen.com/docs/ssh/, which came up for quite a hopeless/desperate search request "ssh-keygen no id_rsa wtf?!!!" This guide not only solved my problem this time but also was immediately added to important bookmarks since most probably it will save my arse a few times more. Recommend.

Day 28

Day 28

Probably, SVN was designed by Hitler, implemented by Nazis during the WW2 and later supported by Soviet KGB. I'm pissed off and switching to GIT.



[update] Hmm... git's nice! Should've switched to it much earlier....

Day 24

Day 24

A quick note.

Haven't updated my blog for a while, since only recently have I had a chance to resume working on the Project, after all the plains and trains. Currently I am 6097 miles away from home, in a hotel room in Siem Reap, Camodia, and going to Malaysia in 2 days. Warm, sunny, awesome food - ... I know I'm a lucky bastard=)

Anyway, for the last few days I was trying to build a small network of university machines to run my experiments on, but haven't succeeded yet and now thinking of dealing with it after I get back to Edinburgh: ssh to my uni account here works soooo incredibly slow. For now I'll stick to working locally on my machine and concentrate on development, leaving later for all configs and such.

Day 12

Day 12

I MADE IT!!! No more concrete vibrators, no more contractors!!! Screw you, Belbin!!!!!!!

Day 11

Day 11

Still doing revision. Most probably I'll resume my work only on Tuesday, after the exam. Yup.... I'm almost done with slides and notes, just need to read a few more pages and a book or two if still have some time left - shouldn't be too hard. Contract Management doesn't make me puke and concrete layers with vibrators don't seem that ridiculous anymore, I just want to be done with that stuff.....

Possibly no more posting till Monday/Tuesday

Day 10

Day 10

Two words that make me want to puke, sleep and bang my head against a wall - all at the same time: Contract Management.

Day 9

Day 9

Hi. As promised, I'm going to post some code here today. I'll give a relatively simple example of MPI Dynamic Process Management which is good for two reasons:

a) It shows in a very basic way how to spawn processes and how mpi4py communicators work
b) It reproduces the CPU issue I described earlier

[update: blogger editor removes empty spaces and tabs from the beginning of a line and it screwed up my Python indentation entirely, so be careful]

master_test.py :

"""The following directives are not compulsory. However, my Python interpreter was complaining about encoding so I included these"""
#!/usr/bin/python
# -*- coding: iso-8859-15 -*-


from mpi4py import MPI
import sys

"""The number of processes we want to spawn"""
numprocs = 5

"""Objects comm1 and comm2 will serve for communicating with two independent of each other groups of processes. The following commands spawn these groups with 5 processes in the first one and 6 in the second one."""
comm1 = MPI.COMM_SELF.Spawn(sys.executable,args=['slave_test.py'], maxprocs=numprocs)
comm2 = MPI.COMM_SELF.Spawn(sys.executable,args=['slave_test.py'], maxprocs=numprocs+1)

print "Type some text and press Return please: "

"""Receiving user input"""
uin = raw_input()
uin1 = "FROM MASTER TO G1: " + uin
uin2 = "FROM MASTER TO G2: " + uin

"""Sending data to all processes in both groups. First parameter of the send() function is data that we want to send, second - rank of a target process, third - tag. Tags can be used for managing sequential messanges, although this can be totally ignored in the example"""
for i in range(0,numprocs):
comm1.send(uin1, i,0)
comm2.send(uin2, i,0)
#endfor

"""Since there's one more process in the second group, we need to send data there as well"""
comm2.send(uin2,numprocs,0)

"""Receiving input from the first group process with rank 4"""
data = comm1.recv(source=4,tag=1)

print "Master: ", data

slave_test.py :

#!/usr/bin/python
# -*- coding: iso-8859-15 -*-

from mpi4py import MPI

"""This object will serve for communicating with Master process"""
comm = MPI.Comm.Get_parent()

"""Getting the rank of the process (a number between 0 and the number of processes spawned in this group)"""
rank = comm.Get_rank()


data = ''

"""Receiving data from process with rank 0. Here it is very important to understand that comm object (initialised as MPI.Comm.Get_parent) links to the parent group where there is only one process - Master, and it has rank 0. In other words, there are 3 separate groups of processes in our system - Master (with 1 node - Master itself), first group of slaves (with 5 nodes and ranks from 0 to 4) and second group of slaves (with 6 nodes and ranks from 0 to 5)"""
data = comm.recv(source=0,tag=0)

print "Slave", comm.rank, "/", comm.size-1, ": ","["+ data + "]"

"""The following bit of code illustrates how different processes can communicate within one group and with the Master process. A message will be sent from node 0 of group 1 to node 4 of group 1 and then from node 4 - to Master"""

"""Getting a communication object for inter-group communicating"""
group_comm = MPI.COMM_WORLD

if group_comm.Get_rank()==0 and comm.size==5:
data = "FROM SLAVE 0 TO SLAVE 4: " + "[" + data + "]"
"""Sending data to process 4"""
group_comm.send(data,4,1)
#endif

if group_comm.Get_rank()==4 and comm.size==5:
data = group_comm.recv(source=0,tag=1)
print "Slave", group_comm.rank, "/", group_comm.size-1, ": ","[" + data + "]"
data = "FROM SLAVE 4 TO MASTER: " + "[" + data + "]"
"""Notice: using comm, not group_comm, since now sending data to Master. Calling the same send command on group_comm would send data to slave process 0, back to where it came from."""
comm.send(data,0,1)
#endif

Now, if you run master_test.py, but before typing in text, go to a different terminal window (that's for UNIX-like OS users. For Windows users - get yourself a Linux distro, install it, rejoice and go to a different terminal window), type "top", and see what I've been talking about earlier - 11 python processes sitting on your CPU and pretending to do a lot of work! However, you might not have this problem if you're using something different from OpenMPI (or, perhaps a newer version of OpenMPI?).

Day8

Day 8

Hi.

I won't post any code here today, sorry. I spent entire day revising for EPM exam and covered one of the biggest topics of that course: Cost Estimation. To illustrate how much I am excited about that I will give an example of what I've learned today (taken from notes):









Doesn't it look like the most important thing I've learned over the last 3.5 years of studying informatics?

Day 7

Day 7
07/12/2009 04:03

Sorry for a very short note, but it's 4am and I really want to go sleep. Another day full of revision. However, I managed to find time to write some code for topographica, and I finally implemented CFProxy. It works now! The entire amount of code I wrote today would not exceed 10 lines, but that's not the point - I know how it all works now and ....... now I need to find a different way to make it work since blocking receive still eats up a hell of a lot of CPU, bastard. I'll show what I mean exactly by posting my code tomorrow, but now I need a rest.

I'm meeting Jim (my supervisor) tomorrow at noon and don't really want to be in a zombie state then, so ... yeah, bed time.


[by the way, by "tomorrow" I actually mean "today". I usually refer to "tomorrow" as to the time after I wake up, even if it's on the same day. Just in case, wanted to clarify]