print msg into file

How do I use this algorithm? What does that parameter do?
Post Reply
QichengTang
Posts: 32
Joined: 08 Jan 2019, 03:03

print msg into file

Post by QichengTang »

I'm wondering if the msg can be printed into a log file?
It's not hard to modify the code, and I think this is necessary.
User avatar
Johannes
Site Admin
Posts: 198
Joined: 21 Jul 2018, 12:52
Location: UC Berkeley

Re: print msg into file

Post by Johannes »

There are different standard ways to do this (On all computing clusters I know, there is at least an option to do it automatically).
Let me point out a few which should suffice your needs:

If you start your program from a Linux console, it's very easy:
you can use the > at the and of any command to redirect all standard output (which usually get's printed on your terminal otherwise.
If you use &>, both the standarad output and standard error get redirected, such that you also get tracebacks and error messages.
Examples:

Code: Select all

python my_script.py > file_where_stdout_appears.txt   # here, error messages still get printed to your terminal
python my_script.py &> file_for stdout_and_stderr.txt  # here, both output and error messages get redirected to the file
python my_script.py > file_for_stdout.txt 2> file_for_stderr.txt # redirect error message to different file
If you've never learned this, take for example a look at this (old) website. This cheatsheet is also very nice.

Of course, you might be bound (or just more used) to Windows...
In that case, I'd suggest to simply use the build-in ways provided by python itself ;)
Take a look at this page of the python documentation.
The relevant example is this one:

Code: Select all

with open('help.txt', 'w') as f:
    with redirect_stdout(f):
        help(pow)
Slightly adjusted to our needs:

Code: Select all

from contextlib import redirect_stdout

def my_function():
    print("here, I could call tenpy models...")

print("this is still usual output")
with open('myoutput.txt', 'w') as f:
    with redirect_stdout(f):
        my_function()
print("and this is again usual output")
A drawback of this approach is that it seems to be more difficult to also capture error messages. (Although there is a redirect_stderr, it does not seem to redirect any python error messages...)
QichengTang
Posts: 32
Joined: 08 Jan 2019, 03:03

Re: print msg into file

Post by QichengTang »

maybe i didnot explain my point clearly enough.

The point is to save the printing msg in terminal at time.

If simply use python3 abc.py > log
the printing msg will be saved after the task is done, i.e. during running of the task, you can not read the msg in this log file.

In the case of using a python print to file, you can not obtain the printing msg in log file immediately.
You need close this file in python after you print something, if not, these msg are just saved in the memory.

Therefore, in the case of, e.g. tebd with large bond dimension, the running process of the total task will speed efficient large time, and you want to read these printing msg after every update. So you need to open, close, open, close, open, close ... this log file in the loop.
User avatar
Johannes
Site Admin
Posts: 198
Joined: 21 Jul 2018, 12:52
Location: UC Berkeley

Re: print msg into file

Post by Johannes »

Still, there are ways around this. You're describing the problem that the file is "buffered", which Linux/Python does automatically to avoid unnecessary hard disk usage slowing down the execution of the program.
However, this can be switched off. For python, try to use the -u option for the executable:

Code: Select all

python3 -u abc.py > log.txt
Checking on a very simple example like

Code: Select all

import time
for i in range(100):
    print(i)
    time.sleep(1)
I find that the output appears immediately in the log file - no need to write new files all over again.

Does this help? ;)
User avatar
Johannes
Site Admin
Posts: 198
Joined: 21 Jul 2018, 12:52
Location: UC Berkeley

Re: print msg into file

Post by Johannes »

There's also a

Code: Select all

flush=True
option to print().
User avatar
Johannes
Site Admin
Posts: 198
Joined: 21 Jul 2018, 12:52
Location: UC Berkeley

Re: print msg into file

Post by Johannes »

I've added the `flush=True` to the print statements during the loops of TEBD and DMRG in ce32fd1fc30073918371717beea110083bab8cd1
On my workstation, the flush causes the output to appear immediately.
If you redirect to a file on a network-shared file system, there might be a delay of the change being transported through the network. Here on our cluster, I get a delay of just a few seconds, which I think is good enough for the purpose needed.
QichengTang
Posts: 32
Joined: 08 Jan 2019, 03:03

Re: print msg into file

Post by QichengTang »

Thanks for your help Johannes, I actually do not know how to deal with this problem for a long time (not just for tenpy, also other code written by myself or others).
Post Reply