Skip to main content

Posts

Showing posts from 2011

Memcached stat items explained

This is a really useful post, which gathers the explanation of every items in memcached stat result. The full post is available at  Memcached statistics (stats command) The explanation is as followed: Field Sample value Description accepting_conns 1 The Memcached server is currently accepting new connections. auth_cmds 0 Number of authentication commands processed by the server – if you use authentication within your installation. The default is IP (routing) level security which speeds up the actual Memcached usage by removing the authentication requirement. auth_errors 0 Number of failed authentication tries of clients. bytes 6775829 Number of bytes currently used for caching items, this server currently uses ~6 MB of it’s maximum allowed (limit_maxbytes) 1 GB cache size. bytes_read 880545081 Total number of bytes received from the network by this server. bytes_written 3607442137 Total number of bytes send to the network by this server. ca

Linux File Descriptor Explained

Basics A Linux file descriptor is an integer for accessing a file like object(file, socket, FIFO, ... ). File descriptors is a per process concept, we may get a list of file descriptor of a process at /proc/PID/fd/     or use command lsof:  lsof -p pid Standard input, output and error Every process have 3 file descriptors in default: that is 0 for stdin, 1 for stdout, and 2 for stderr. For example, a shell command "cmd > log.txt" only pipe the stdout of cmd to log.txt in default. Use "cmd > log.txt 2>&1" will pipe both the stdout and the stderr to the log.txt file. "2>&1" means redirecting stderr to stdout. For example: python -c "import sys;sys.stdout.write('abc')" > abc.txt abc.txt will be 'abc'. python -c "import sys;sys.stderr.write('abc\n')" > abc.txt 2>&1 abc.txt will be a blank file.   python -c "import sys;sys.stderr.write('abc\n&#

A "Node is not runnning!" error on Riak

Today I got an "Node is not runnning!" error when test with "riak-admin status" after starting riak 0.14.2 server is about the ~/.erlang.cookie. Yet when I "ps xua |grep riak", all the riak processes were running well. I tried "riak ping", and it lefe me following error: Error when reading /home/username/.erlang.cookie: eaccesescript: exception error: no match of right hand side value I checked .erlang.cookie and found it with owner root. So I removed /home/username/.erlang.cookie, and start riak without root permission(chown should also work). Now, everything works fine.

Several solutions for a few mysql errors

1062 Duplicate entry 'xxxx' for key xxxx. The key is already in current mysql. In mysql slave server, try "STOP SLAVE;SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;START SLAVE;". You may also set slave_skip_errors = 1062 in my.cnf file to ignore all the 1062 errors. 1236  Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position' The binlog position in master mysql server is not available. Some problem of master mysql server may cause this issue. You may want change the mysql replication position to next binary log file on the master mysql server(using "change master to ..."). 126 Error 'Incorrect key file for table './db/xxx.MYI'; try to repair it' on query. The error means that index file is crashed, and you may repair it by "repair table xxx". Or you want to change mysql storage to another hard disk since the error my be induced by an disk

One thing about python default parameter list

Python interpreter stores the default parameter list in a dict named func_defaults meeting a "def func(a=0, b=[], c={})" statement. So there is func.func_defaults = [0, [], {}]. When we specify an object in default parameter list of a function, python interpreter will keep its reference in the list. Therefore, an object as a default parameter of a function may change if you change it. For example: def func(a = []):     a.append(0)     print a When call the function in a python IDLE, we may have: >>> func()   [0]   >>> func()   [0, 0]   >>> func()  [0, 0, 0]   >>> func([1])   [1, 0]   >>> func()  [0, 0, 0, 0] You may find some detail example at http://effbot.org/zone/default-values.htm

Install mysql-server silently on Ubuntu

First, you need debconf installed by sudo apt-get -y install debconf. Then, you may install mysql-server silenly by: echo mysql-server-5.1 mysql-server/root_password password PASSWORD-STRING | sudo debconf-set-selections echo mysql-server-5.1 mysql-server/root_password_again password PASSWORD-STRING | sudo debconf-set-selections sudo apt-get -y install mysql-server All pretty simple!

boto api on amazon ec2

Here are several examples of boto api on amazon ec2: Connect to a region: conn=boto.ec2.connect_to_region('us-west-1a') Get an instance by id: instance = conn.get_all_instances(['i-xxxxxxxx'])[0].instances[0] Run(create) an instance: conn.run_instances(image_id=u'ami-xxxxxxxx', key_name='default', security_groups=['default-group'],                   instance_type='m2.4xlarge', placement='us-west-1a') Add/remove a tag for an instance: instance.add_tag('Name','default') instance.remove_tag('Name') Reboot an instance: instance.reboot() Terminate an instance: instance.terminate() Get an ebs volume by id: volume = conn.get_all_volumes(['vol-xxxxxxxx'])[0] Attach an ebs volume to an instance: volume.attach(instance.id, '/etc/sdh') conn.attach_volume(volume.id, instance.id, '/dev/sdh')

Using HTMLParser to extract links from html files

We can use python htmllib.HTMLParser to extract link and corresponding text from html files: import urllib from formatter import * from htmllib import * formatter = AbstractFormatter(NullWriter()) class LinkParser(HTMLParser):     def __init__(self, *sub, **kw):         HTMLParser.__init__(self, *sub, **kw)         self.current_link = self.current_text = None     def handle_starttag(self, tag, method, attrs):         if tag == 'a':             for attr in attrs:                 if attr[0]=='href':                     self.current_link = attr[1]         return HTMLParser.handle_starttag(self, tag, method, attrs)     def handle_endtag(self, tag, method):         if tag == 'a':             if self.current_link and self.current_text:                 print self.current_link, self.current_text             self.current_link = self.current_text = None         return HTMLParser.handle_endtag(self, tag, method)     def handle_data(self, data):   

Use user-defined functions in with sqlalchemy

You may use user-defined functions with sqlalchemy. First how to create a function with sqlalchemy: cmd1 = 'DROP FUNCTION IF EXISTS `dummyfunc`' cmd2 = '''create function dummyfunc  ()   RETURNS INT   DETERMINISTIC    BEGIN     DECLARE ret INT;     SET ret = 0;     RETURN ret;    END''' metadata.bind.execute(cmd1) metadata.bind.execute(cmd2) Second, use the function you just created with func.function_name like: session.query(TestTable,func.dummyfunc()).first()   Very simple!

Solve KeyboardRandomPool not found in Crypto.Util.randpool

Errors like Crypto.Util.randpool.KeyboardRandomPool not found may occur if you update your pycrypto to a newer version(Like 2.3). Due to this article , randpool.KeyboardRandomPool has been marked deprecated in previous version. We should use Crypto.Random instead. Just change things like: Crypto.Util.randpool.KeyboardRandomPool().get_bytes(1) to: Crypto.Random.get_random_bytes(1)

One trick for building mysql master-slave chains

There are many tutorials for master-slave replication in Mysql( link1 , link2 ). But there is one trick if you want to build a slave for a slave. It is log-slave-updates, you may add it under [mysqld] in /etc/mysql/my.cnf or attach --log-slave-updates in mysqld. By default, the slave server doesn't write bin log for relication data, change log-slave-updates to yes will let slave replicate the bin log for master.