Skip to main content

Linux File Descriptor Explained


Basics
A Linux file descriptor is an integer for accessing a file like object(file, socket, FIFO, ... ). File descriptors is a per process concept, we may get a list of file descriptor of a process at
/proc/PID/fd/    
or use command lsof: 
lsof -p pid

Standard input, output and error
Every process have 3 file descriptors in default: that is 0 for stdin, 1 for stdout, and 2 for stderr. For example, a shell command "cmd > log.txt" only pipe the stdout of cmd to log.txt in default. Use "cmd > log.txt 2>&1" will pipe both the stdout and the stderr to the log.txt file. "2>&1" means redirecting stderr to stdout.

For example:
python -c "import sys;sys.stdout.write('abc')" > abc.txt
abc.txt will be 'abc'.

python -c "import sys;sys.stderr.write('abc\n')" > abc.txt 2>&1
abc.txt will be a blank file.
 
python -c "import sys;sys.stderr.write('abc\n')" > abc.txt 2>&1
abc.txt will be 'abc'.

Duplicate File Descriptor
We may duplicate file descriptor in a process. For example, in process(PID=28833), if we have 
f = open('test')
We may get the following from the lsof -p 28833:
python  28833 ubuntu    3r   REG  252,0      581 134489 /home/ubuntu/test

And we can use os.dup to duplicate the file descriptor of f by
fd2 = os.dup(f.fileno())
See what we may get the from lsof:
python  28833 ubuntu    3r   REG  252,0      581 134489 /home/ubuntu/test
python  28833 ubuntu    4r   REG  252,0      581 134489 /home/ubuntu/test

And we can use f2 = os.fdopen(f2), and operate f2 as f.
Now let's create f3=open('/dev/null'), so we have the following in lsof:
python  28833 ubuntu    5r   CHR    1,3      0t0   1331 /dev/null

If we use os.dup2 as below:
os.dup2(f.fileno(), f3.fileno())
We may get:
python  28833 ubuntu    3r   REG  252,0      581 134489 /home/ubuntu/test
python  28833 ubuntu    4r   REG  252,0      581 134489 /home/ubuntu/test
python  28833 ubuntu    5r   REG  252,0      581 134489 /home/ubuntu/test














Labels:

Comments

Post a Comment

Popular posts from this blog

A simple implementation of DTW(Dynamic Time Warping) in C#/python

DTW(Dynamic Time Warping) is a very useful tools for time series analysis. This is a very simple (but not very efficient) c# implementation of DTW, the source code is available at  https://gist.github.com/1966342  . Use the program as below: double[] x = {9,3,1,5,1,2,0,1,0,2,2,8,1,7,0,6,4,4,5}; double[] y = {1,0,5,5,0,1,0,1,0,3,3,2,8,1,0,6,4,4,5}; SimpleDTW dtw = new SimpleDTW(x,y); dtw.calculateDTW(); The python implementation is available at  https://gist.github.com/3265694  . from python-dtw import Dtw import math dtw = Dtw([1, 2, 3, 4, 6], [1, 2, 3, 5],           distance_func=lambda x, y: math.fabs(x - y)) print dtw.calculate() #calculate the distance print dtw.get_path() #calculate the mapping path
14 comments
Read more

Change the default user when start a docker container

When run(start) a docker container from an image, we can specify the default user by passing -u option in command line(In https://docs.docker.com/engine/reference/run/#user ). For example docker run -i -t -u ubuntu ubuntu:latest /bin/bash We can also use the USER instruction in DOCKERFILE to do the same thing(In https://docs.docker.com/engine/reference/builder/#user), note that the option in command line will override the one in the DOCKERFILE. And there is actually another way to start a container with neither DOCKERFILE nor -u option, just by a command like: docker run -i -t ubuntu:latest /bin/bash # with ubuntu as the default user This happens when your start the container from an image committed by a container with ubuntu as the default user. Or in detail: Run a container from some basic images, create ubuntu user inside it, commit the container to CUSTOM_IMAGE:1 . Run a container from CUSTOM_IMAGE:1 with "-u ubuntu" option, and commit the container to CUSTOM...
Post a Comment
Read more

Install mysql-python with mariadb

mysql-python requires libmysqlclient-dev in ubuntu, but the installation of mariadb will have the lib with unmet dependenccies, so the error of "mysql_config not found" may occurred if you install mysql-python via pip. The case is that mariadb has a compatible package, if you have the ppa setup as in  http://downloads.mariadb.org/ . Just "sudo apt-get install libmariadbclient-dev".
2 comments
Read more