Day 65

Day 65

I just thought I haven't posted for a while, so this is a quick update on what's currently going on.

First, this Monday my laptop has refused to switch on and now seems to be pretty much dead, but hopefully this is just coma and anyways my warranty hasn't expired yet, so this Wednesday it's going to be picked up and hopefully returned in a week, fully functional. Not a big issue, just a bit annoying.

Second, some time ago Chris suggested I have a look at PMI - "a pure python module that allows libraries to provide functions that are parallelized using MPI but that can nonetheless be called from serial Python scripts." I'll write about it if I find the way to utilize this thing, but at the moment it seems to be quite a handy module. However, here's what I want to get rid of in my implementation by using PMI:

'''
Created on 20 Jan 2010

@author: megatelevizor
'''
from mpi4py import MPI
import time

from MPI_client import client


""" Vocabulary

a   = assign flatcfs
c   = compute activity
i   = Initialize
i_a = Get a copy of input_activity matrix
k   = Kill
p   = Print
r   = Return activity
s   = Set strength


"""

class communicator(object):

   def listen(self):
       self.comm = MPI.COMM_WORLD
       self.rank = self.comm.Get_rank()
      
       self.client_node = client()
      
       while True:
           while not self.comm.Iprobe(source=0, tag=0):
               time.sleep(0.1)
           command_prefix = self.comm.recv(source=0, tag=0)
          
           """ Print """
           if command_prefix == "p":
               i_command_body = self.comm.recv(source=0,tag=1)
               self.client_node.print_command(command = i_command_body)
          
           """ Kill """
           if command_prefix == "k":
               exit()
          
           """ Initialize MPI_CFProjection_slave"""
           if command_prefix == "i":
               self.client_node.init_mpi_cfprojection_slave(node_id=self.rank)
          
           """ Assign flatcfs"""
           if command_prefix == "a":
               new_flatcfs = self.comm.recv(source=0,tag=1)
               print "Node",self.rank,": received", len(new_flatcfs),"flatcfs"
               self.client_node.assign_cfs(new_flatcfs)
          
           """ Set a copy of input activity matrix """
           if command_prefix == "i_a":
               input_activity = self.comm.recv(source=0,tag=1)
               self.client_node.set_input_activity(input_activity)
                      
           """ Compute activity """
           if command_prefix == "c":
               self.client_node.compute_activity()
              
           """ Return activity """
           if command_prefix == "r":
               activity = self.client_node.get_activity()
               print "Node",self.rank,"activity:",activity
               self.comm.send(obj=activity, dest=0, tag=2)
          
           """ Set strength"""
           if command_prefix == "s":
               strength = self.comm.recv(source=0,tag=1)
               self.client_node.set_strength(strength)
              

              
              

""" TAGS:
0 - command prefix
1 - input data
2 - output data
"""

This class allows master node to communicate with slave nodes. Potentially, PMI will eliminate all need to have such class.



[update] No, it will not. What PMI allows you to do is hide the parallel implementation, so that your programme looks like it is a serial (single-processed) code. However, it does not enhance the underlying MPI implementation in any way, so it still needs to be implicit. What does that mean in my case? Well, if I was to introduce PMI into my code as it is now, then there would have been a class - something like CFProjection_proxy, linked to the real (i.e. parallel) implementation of CFProjection_slave using PMI mechanism. CFPRojection_slave's would still have to communicate with the master node CFProjection using MPI_io (the code listed above) because PMI does not influence or change the way different nodes of an MPI application communicate. In simple words, it just makes your code look like it's not using MPI, when it actually does.

Well, so what's so bad about it? Why not use PMI anyway, since from all that it looks like it's actually doing a good job? Actually, there's nothing wrong with it and it might be a good idea to use it. No, it won't add any new functionality, but it will definitelly make things look a lot nicer. However, I prefer dealing with things one at a time, and also finishing what I started, so I think I'll concentrate on looking for the solution to my initial problem.

To summarise what I'm looking for, have a look at this: Java RMI . Without going into too much detail, RMI is a mechanism that allows Java objects calling each other's methods accross several different JVM's (which can be run on different hosts!) and it stands for Remote Method Invocation. Combined with serialization (something similar to Python's pickling), it allows to hide all explicit communication-level code away from the user, which is great! Thus, my assumption is that if something exists in Java, it has a chance to exist in Python. As simple as that.

Searching goes on.