Skip to main content

Posts

Showing posts from 2013

Install packages from git repo using pip

Packages in git repo (or hg, svn, bzr) can be installed with " pip install -e " directly. Base on the help message of " pip install ", we have -e VCS+REPOS_URL[@REV]#egg=PACKAGE, --editable=VCS+REPOS_URL[@REV]#egg=PACKAGE Install a package directly from a checkout. Source will be checked out into src/PACKAGE (lower-case) and installed in-place (using setup.py develop). You can run this on an existing directory/checkout (like pip install -e src/mycheckout). This option may be provided multiple times. Possible values for VCS are: svn, git, hg and bzr. To use it, first make sure you have git installed. Such as " sudo apt-get install git-core " on ubuntu. And then you may follow the below examples(install redis-py): # Install from master pip install -e git+https://github.com/andymccurdy

The default CREATE TABLE options for Aria Engine in mariadb

The official document of mariadb does not mention the default CREATE TABLE options for tables using Aria Engine.  The default options are list as below: TRANSACTIONAL,  the default value is TRANSACTIONAL=0, i.e., non-transactional. ROW_FORMAT, the default value is ROW_FORMAT=PAGE, which may suits both transactional and non-transactional tables. PAGE_CHECKSUM,  the default value will follow aria_page_checksum system variable, which has default value ON. For the TRANSACTIONAL option, you may consider create a table as below(and ALTER the TRANSACTIONAL=1): CREATE TABLE `test_table` ( `id` int(11) NOT NULL AUTO_INCREMENT, PRIMARY KEY (`id`) ) ENGINE=Aria; If you change the ROW_FORMAT to DYNAMIC or FIXED, everything just goes fine. But if you have ALTER the table with TRANSACTION=1 and change the ROW_FORMAT to DYNAMIC or FIXED, you may got a warning: SHOW WARNINGS; +-------+------+----------------------------------------------------------+ | Level | Code | Message | +-------+--

prompt less installing mariadb-server-5.5 on ubuntu

You need to previously set the root password echo mariadb-server-5.5 mysql-server/root_password password xxxxx | sudo debconf-set-selections echo mariadb-server-5.5 mysql-server/root_password_again password xxxxx | sudo debconf-set-selections And then sudo apt-get -y install mariadb-server If you are going to uninstall mariadb totally, you may sudo apt-get purge mariadb* && sudo apt-get autoremove

Install mysql-python with mariadb

mysql-python requires libmysqlclient-dev in ubuntu, but the installation of mariadb will have the lib with unmet dependenccies, so the error of "mysql_config not found" may occurred if you install mysql-python via pip. The case is that mariadb has a compatible package, if you have the ppa setup as in  http://downloads.mariadb.org/ . Just "sudo apt-get install libmariadbclient-dev".

About install from Ubuntu PPA

The PPA is short for Personal Package Archives for Ubuntu , which is a place for publishing/installing personal organized packages from  launchpad.net . Usually, the url of a package archive is organized as https://launchpad.net/~PUBLISHER_NAME/+archive/PACKAGE_ARCHIVE_NAME. At the url, you may see a full list of packages included. You can also search PPA packages in  https://launchpad.net/ubuntu/+ppas  . To install some packages using a ppa, you may use sudo add-apt-repository ppa:PPA_PUBLISHER_NAME/PPA_ARCHIVE_NAME sudo apt-get update sudo apt-get install PACKAGE_NAME If you want to remove a ppa sudo add-apt-repository --remove ppa:PPA_PUBLISHER_NAME/PPA_ARCHIVE_NAME For the command add-apt-repository, if not found, you may need to install python-software-properties for Ubuntu version <= 12.04, or software-properties-common for Ubuntu version >= 12.10. For earlier versions of Ubuntu or just you want manually add the ppa into the source.list, you may follow  https:/

A possible [Errno 32] Broken pipe when using socket._fileobject

In Python 2.x, socket object have an makefile() method to provide you a file like object. Please do use it carefully, if the socket itself closed unexpectedly, the fileobject may not know the connection is actually closed. So some buffered write is not going to be performed correctly. For example, considering the following code: import socket sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM) sock.connect(('google.com', 80)) wfile = sock.makefile('w') wfile.write('a') sock.shutdown(socket.SHUT_WR) wfile.close() It will cause exceptions as below: Traceback (most recent call last): File "socket_shut.py", line 9, in wfile.close() File "/usr/lib/python2.7/socket.py", line 279, in close self.flush() File "/usr/lib/python2.7/socket.py", line 303, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) socket.error: [Errno 32] Broken pipe The reason is that wfile has some buffered write,

A workaround for UnicodeDecodeError for kid template

Like previous post , the UnicodeDecodeError may occur in the following code: import kid t_mod = kid.load_template(template_file t_class = t_mod.Template t_class.serializer = kid.HTMLSerializer() t = t_class(**data) return t.serialize(output='html') The problem is caused by data contains some non-ascii data, and the default encoding kid choose for decoding is ascii. Actually kid has an attributing assme_encoding, are used to converting data to unicode. import kid t_mod = kid.load_template(template_file t_class = t_mod.Template t_class.serializer = kid.HTMLSerializer() t = t_class(**data) t.assume_encoding = 'utf-8' return t.serialize(output='html')

A workaround for UnicodeDecodeError when using mako template

Consider following code to rendering data with mako: from mako.template import Template tmpl = Template(filename=some_template_flle, module_directory=some_template_module_path) print tmpl.render(**some_data) The code may have following error, if the some_data contains non-ascii characters, because the implementation use directly use unicode method to convert some_data: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0: ordinal not in range(128) The error can be fixed by decode the some_data. Or there is another workaround, which is injecting a partial unicode function into the module generated: from mako.template import Template import functools tmpl = Template(filename=some_template_flle, module_directory=some_template_module_path) tmpl.module.unicode = functools.partial(unicode, encoding='utf-8') print tmpl.render(**some_data)

Several Cases of Bit Torrent Sync

Here we are going to test several cases when using BitTorrent Sync. Assuming we have two devices, A and B. Below are the cases: Case 1 1) Create and share folder test1 on Device A, and use the readonly key to create the same folder on Device B. 2) Create test.txt in folder test1 on Device A, the file is synced correctly on Device B. 3) Change test.txt in on Device A, the change is synced correctly on Device B. 4) Create other files in folder test1 on Device B, nothing synced back to Device A. 5) Change the test.txt in Device B, nothing synced back to Device A. 6) Change the test.txt in Device A, nothing synced to Device B. 7) Create test2.txt in folder test1 on Device A, the file is synced correctly on Device B, but test.txt remains modified version on Device B. 8) Remove test.txt in Device B, nothing happend. 9) Change the test.txt in Device A, nothing happened. 10) Remove the folder test1 on Device B from btsync, and readd it with the readonly key, all files in Device A synced t

A simple script for get birth time of Linux process

This is a simple script to get the Linux birth time of the system or a process using /proc/stat and /proc/pid/stat. For example, you may get local datetime when process 12345 started: import datetime import btime datetime.datetime.fromtimestamp(btime.process_time(12456)[1]) View the source code in github gists:

winpath.py -- A simple command line tool for managing windows PATH

I have write a simple command line tool for managing windows PATH env variable with in Python. The program mainly uses _winreg lib to access the Windows Registry. Several Example: # List all the current PATH entries python winpath.py list # Find if C:\Temp in PATH python winpath.py find C:\Temp # Add C:\Temp to System PATH python winpath.py add system C:\Temp # Remove C:\Temp from PATH(both system and user) python winpath.py remove C:\Temp The code the tool is available at this github gists .

The Awesome BitTorrent Sync

The BitTorrent Sync is really an awesome tool for syncing files between your different devices. Currently, it supports Windows/Mac/Linux. The software is not like dropbox(cloud services stored you files on their servers),  it syncs folders among your own devices(the security is protected by a secret key and encrypted file transfer). And actually BitTorrent Sync is not Dropbox killer, the two services are both just awesome in their own way. Dropbox is still consider to be my own online safe storage, but BitTorrent Sync can be used as much faster sharing and syncing. For example, you may setup it on Linux via follwing steps #Downloading and unpacking wget http://btsync.s3-website-us-east-1.amazonaws.com/btsync_x64.tar.gz tar vzxf btsync_x64.tar.gz #Copy to /usr/bin/ sudo cp btsync /usr/bin/btsync #Generate the config from sample config btsync --dump-sample-config > ~/btsync.conf # Edit the ~/btsync.conf based on the embedded comments # Detail config in linux, please refer http://

Change emacs code page when shell starts in Windows

Use "chcp 65001" can change the current code page of windows console to UTF-8. To change this in emacs when "M-x shell" starts, you need to add a ~/.emacs_shellname(in the same directory of your .emacs file) containing the "chcp 65001" command.  But note that, the shellname in Windows is cmdproxy.exe, so your ~/.emacs-shellname should be ~/.emacs-cmdproxy.exe (if this is not working, try "echo %SHELL%" in emacs shell).