## Write MPS/MPO to disk or increase RAM?

How do I use this algorithm? What does that parameter do?
Sergi
Posts: 1
Joined: 01 Apr 2020, 10:11

### Write MPS/MPO to disk or increase RAM?

I am simulating a fermionic model on a cylinder. Everything is working well, except that I need to simulate a quiet large system (20x4 unit cells, and each unit cell composed of two sites).

In order to achieve convergence I need a bond dimension of ~3000-5000, which is not achievable with the resources that I currently have (64GB of RAM divided in several cores).

Is it possible to store the MPS/MPO to disk in order to save memory? I have seen the related topic for the case of the MPO (viewtopic.php?f=7&t=55), but would it be much more work for the MPS also?

Alternatively, any hints or flavour whether incrementing the RAM memory would suffice, or it is not feasible to simulate such a 2D system without a write-to-disk functionality?

Thanks a lot!
kevln
Posts: 17
Joined: 12 May 2022, 07:02

### Re: Write MPS/MPO to disk or increase RAM?

Hi,Sergi
DO you know how to solve this problem now?Or someother ways to store the MPS/MPO to disk in order to prevent out of memory?
Thanks a lot!
Johannes
Posts: 399
Joined: 21 Jul 2018, 12:52
Location: TU Munich

### Re: Write MPS/MPO to disk or increase RAM?

In terms of memory usage of DMRG, the limiting factore are usually the MPO environments rather than the MPS itself:
the number of parameters for the MPS scales as $$O(L\chi^2 d)$$, where L is MPS unit cell length, chi is bond dimension and d is local dimension.
In contrast, MPO environments go as $$O(L\chi^2 D)$$ with MPO bond dimension $$D \propto L_y \approx 20-40$$ scaling with cylinder circumference (and number of terms in H).

We fully support caching all the MPO environments to disk, using the simulation class with cache_params.
It can be as easy as adding those lines in your yaml simulation config:

Code: Select all

cache_params:
storage_class: PickleStorage
# further specify directory or tmpdir on the cluster node's local file system
tmpdir: /scratch/local/tmp
cache_threshold_chi: 2000   # use cache for chi larger than that
kevln
Posts: 17
Joined: 12 May 2022, 07:02

### Re: Write MPS/MPO to disk or increase RAM?

Johannes wrote: 24 Apr 2023, 16:46 In terms of memory usage of DMRG, the limiting factore are usually the MPO environments rather than the MPS itself:
the number of parameters for the MPS scales as $$O(L\chi^2 d)$$, where L is MPS unit cell length, chi is bond dimension and d is local dimension.
In contrast, MPO environments go as $$O(L\chi^2 D)$$ with MPO bond dimension $$D \propto L_y \approx 20-40$$ scaling with cylinder circumference (and number of terms in H).

We fully support caching all the MPO environments to disk, using the simulation class with cache_params.
It can be as easy as adding those lines in your yaml simulation config:

Code: Select all

cache_params:
storage_class: PickleStorage
# further specify directory or tmpdir on the cluster node's local file system
tmpdir: /scratch/local/tmp
cache_threshold_chi: 2000   # use cache for chi larger than that
Hi,Johannes
Your suggestion above is correct and helpful. Following the instructions provided, I successfully managed to write some env to the disk.

However, based on my testing, it seems that tenpy unable to write the majority of the computations to the disk . For example, in a complex calculation I am currently attempting, it requires 120GB of RAM, but only 81GB is being written to the disk. While my disk has sufficient capacity, the RAM using is nearing its limit.This makes it challenging for me to perform more accurate computations with larger bond sizes and obtain the desired results.

Therefore, is there a way for Tenpy to write more computational processes to the disk and reduce the consumption of RAM? The parameters I previously used were as follows:

Code: Select all

'cache_params':{
'storage_class': 'PickleStorage',
# further specify directory or tmpdir on the cluster node's local file system