I am working on a Parallelization of iDMRG following https://arxiv.org/abs/1606.06790 which could be interesting for TeNPy running on a Cluster and, if it runs, I would like to add to the package. So far I implemented, quick and dirty, a running version of it using the multiprocessing module and partial from the functools module. Here a minimal code snippet (without tenpy) as basis for further discussions on the parallelization scheme:
Python: Select all
import multiprocessing as mp
from functools import partial
# define a worker function for doing some calculations
def worker_function(data, iterable):
# do something
data = 1+iterable
#print(mp.current_process())
return(data)
# prepare pool with nbp=4 processes
pool = mp.Pool(4)
# prepare list of iterables, e.g. sites in the system
iterables = [0,1,2,3]
# some data here, tensor network data like mps, mpo, etc. in the final code
data = None
# prepare worker and do calculations
worker = partial(worker_function, data)
results = pool.map(worker, iterables)
# close and join
pool.close()
pool.join()
# print the content of results
for result in results:
print(result)
.Best, mm
), I think it's easier to just set the number of cores beforehand in the bash script which starts the job, using the suitable environment variables like 
. I took a look at the wrapper functions in