Expat-IT Tech Bits




Search this site:


/ (289)
  Admin/ (123)
    Apache/ (10)
      HTTPS-SSL/ (4)
      PHP/ (3)
      performance/ (2)
    Cherokee/ (1)
    LAN/ (4)
    LVM/ (6)
    Monitoring/ (2)
      munin/ (2)
    SSH/ (6)
    SSL/ (1)
    Samba/ (1)
    VPN-options/ (7)
      OpenVPN/ (1)
      SSH-Proxy/ (3)
      Tinc/ (1)
      sshuttle/ (1)
    backups/ (17)
      SpiderOak/ (1)
      backuppc/ (5)
      dirvish/ (1)
      misc/ (6)
      rdiff-backup/ (1)
      rsync/ (1)
      unison/ (2)
    commandLine/ (24)
      files/ (8)
      misc/ (10)
      network/ (6)
    crontab/ (1)
    databases/ (15)
      MSSQL/ (2)
      MySQL/ (8)
      Oracle/ (3)
      PostgreSQL/ (1)
    dynamicDNS/ (2)
    email/ (11)
      Dovecot/ (1)
      deliverability/ (1)
      misc/ (1)
      postfix/ (7)
      puppet/ (1)
    iptables/ (3)
    tripwire/ (1)
    virtualization/ (9)
      VMware/ (1)
      virtualBox/ (8)
  Coding/ (14)
    bash/ (1)
    gdb/ (1)
    git/ (3)
    php/ (5)
    python/ (4)
      Django/ (2)
  Education/ (1)
  Hosting/ (27)
    Amazon/ (18)
      EBS/ (3)
      EC2/ (10)
      S3/ (1)
      commandline/ (4)
    Godaddy/ (2)
    NearlyFreeSpeech/ (3)
    Rackspace/ (1)
    vpslink/ (3)
  Linux/ (31)
    Android/ (1)
    Awesome/ (3)
    CPUfreq/ (1)
    China/ (2)
    Debian/ (8)
      APT/ (3)
      WPA/ (1)
    audio/ (1)
    encryption/ (3)
    fonts/ (1)
    misc/ (6)
    remoteDesktop/ (1)
    router-bridge/ (3)
  SW/ (45)
    Micro$soft/ (1)
    browser/ (2)
      Chrome/ (1)
      Firefox/ (1)
    business/ (28)
      Drupal/ (9)
      KnowledgeTree/ (6)
      Redmine/ (2)
      SugarCRM/ (7)
      WebERP/ (2)
      WordPress/ (1)
      eGroupware/ (1)
    chat/ (1)
    email/ (1)
    fileSharing/ (2)
      btsync/ (1)
      mldonkey/ (1)
    graphics/ (2)
    research/ (2)
    website/ (6)
      blog/ (6)
        blosxom/ (3)
        rss2email/ (1)
        webgen/ (1)
  Security/ (15)
    IMchat/ (2)
    circumvention/ (2)
    cryptoCurrency/ (1)
    e-mail/ (4)
    greatFirewall/ (1)
    hacking/ (1)
    password/ (1)
    privacy/ (2)
    skype/ (1)
  Services/ (1)
    fileSharing/ (1)
  TechWriting/ (1)
  xHW/ (14)
    Lenovo/ (1)
    Motorola_A1200/ (2)
    Thinkpad_600e/ (1)
    Thinkpad_a21m/ (3)
    Thinkpad_i1300/ (1)
    Thinkpad_x24/ (1)
    USB_audio/ (1)
    scanner/ (1)
    wirelessCards/ (2)
  xLife/ (17)
    China/ (9)
      Beijing/ (5)
        OpenSource/ (3)
    Expatriation/ (1)
    Vietnam/ (7)


  • 2019/06
  • 2016/07
  • 2016/05
  • 2016/02
  • 2016/01
  • 2015/12
  • 2015/11
  • 2015/06
  • 2015/01
  • 2014/12
  • 2014/11
  • 2014/10
  • 2014/09
  • 2014/07
  • 2014/04
  • 2014/02
  • 2014/01
  • 2013/12
  • 2013/10
  • 2013/08
  • 2013/07
  • 2013/06
  • 2013/05
  • 2013/04
  • 2013/02
  • 2013/01
  • 2012/12
  • 2012/10
  • 2012/09
  • 2012/08
  • 2012/07
  • 2012/06
  • 2012/05
  • 2012/04
  • 2012/03
  • 2012/01
  • 2011/12
  • 2011/11
  • 2011/10
  • 2011/09
  • 2011/08
  • 2011/07
  • 2011/06
  • 2011/05
  • 2011/04
  • 2011/02
  • 2010/12
  • 2010/11
  • 2010/10
  • 2010/09
  • 2010/08
  • 2010/07
  • 2010/06
  • 2010/05
  • 2010/04
  • 2010/03
  • 2010/02
  • 2010/01
  • 2009/12
  • 2009/11
  • 2009/10
  • 2009/09
  • 2009/08
  • 2009/07
  • 2009/06
  • 2009/05
  • 2009/04
  • 2009/03
  • 2009/02
  • 2009/01
  • 2008/12
  • 2008/11
  • 2008/10
  • 2008/09
  • Subscribe XML RSS Feed

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

    This site has no ads. To help with hosting, crypto donations are accepted:
    Bitcoin: 1JErV8ga9UY7wE8Bbf1KYsA5bkdh8n1Bxc
    Zcash: zcLYqtXYFEWHFtEfM6wg5eCV8frxWtZYkT8WyxvevzNC6SBgmqPS3tkg6nBarmzRzWYAurgs4ThkpkD5QgiSwxqoB7xrCxs

    Thu, 26 Feb 2009

    /Coding/python: Using Python to Connect to MySQL Over SSL

    Assuming you have already setup your MySQL server to accept external SSL connections[1], with the MySQLdb module and the help of a little documentation[2][3] getting Python to talk to the MySQL server is fairly straightforward:

    #!/usr/bin/python import MySQLdb ssl_settings = {'ca': 'db/cacert.pem', 'cert': 'db/cert.pem', 'key': 'db/key.pem' } try: db=MySQLdb.connect(host="www.yoursite.com", user="SSLremote", passwd="yourpassword", db="databasename", ssl=ssl_settings ) except: print "<p>ERROR: could not connect to database." c=db.cursor() c.execute("""SELECT * FROM thistablename""") while True: row = c.fetchone() if row == None: break else: print "<p>" print row

    Point your web browser at this piece of code (you probably have to name it "something.cgi" and put it a folder where your web server is enabled for cgi). This code will connect to the MySQL server and dump the contents of table thistablename into your browser window. If you get "ERROR: could not connect to database." then execute the same code in a terminal on the Python interpreter command line to see the actual MySQL error returned.

    Note that the path specification and permissions of the SSL files specified in "ssl_settings" are quite sensitive, as the web server must have access to them. I have stored the files with my code to allow a relative path. For security, these files should be "chmod 600" with the owner set to www-data (or whatever user name your webserver runs under....)

    [1] http://blog.langex.net/index.cgi/Admin/MySQL/
    [2] http://riskable.com/2009/02/12/how-to-use-ssl-with-the-python-mysqldb-module
    [3] http://mysql-python.sourceforge.net/MySQLdb.html

    posted at: 00:32 | path: /Coding/python | permanent link to this entry

    Sun, 22 Feb 2009

    /Admin/databases/MySQL: Enabling Remote Connections to a MySQL Database

    This is not straight-forward, and thanks to this post[1] for getting me pointed in the right direction.

    Firewall: open port 3306.

    Configuration: ensure both of

    are commented out in /etc/mysql/my.cnf. These are security options meant to confine MySQL server access to the local machine only.

    Grant Access: MySQL has one more layer of security: access to a specific database can be explicitly granted to a specific user at a specific IP (host). There are, of course, wild cards that permit making the access wide-open, but why not be secure:

    GRANT ALL privileges ON databasename.* TO username@'' IDENTIFIED BY 'password';

    This command appears to create username, or modify an existing username.

    [1] http://ubuntuforums.org/showthread.php?t=608435

    posted at: 03:14 | path: /Admin/databases/MySQL | permanent link to this entry

    Wed, 18 Feb 2009

    /Security/greatFirewall: All Google HTTP Services Blocked

    From where I am sitting near Beijing, for the second day all things Google http have been inaccessible: gmail webmail, the search engine, and Google Adsense ads, to name a few. Web pages containing Google Adsense ads display a banner and maybe a little bit more, and then just sit there and spin. gmail can thankfully still be accessed via an e-mail client using POP and SMTP.

    Since Google Adsense ads are just about everywhere, that means a vast array of websites (including many hosted within China) are effectively blocked. Even my trusty Tor[1] is very, very, slow, no doubt because there are a lot of other people like myself using Tor to try to get some work done.

    This is one of a very short list of things that sometimes makes me wonder why I continue to live in China.

    [1] http://www.torproject.org/

    posted at: 03:01 | path: /Security/greatFirewall | permanent link to this entry

    /Hosting/Amazon/EBS: Keep Your Amazon Server's Data on EBS

    Amazon's EC2 servers do not provide persistent storage. If the server is turned off or even perhaps halted, then all data since the server was initialized is lost. (I am pretty sure one can reboot an Amazon server without losing data, but have not tried....)

    Therefore, at the least, all volatile data should be stored on an Amazon "Elastic Block Storage" (EBS) volume. As a further incentive, Amazon says[1] that EBS is faster then the storage associated directly with an Amazon server, and that it is replicated across multiple servers in the data center. The downside, of course, is that you pay for EBS, both by gigabyte per month and by I/O per month.

    Here[2] is an excellent tutorial on how to use EBS with a MySQL database, and here[3] is a nice general introduction.

    I am going to move my Amazon Server's /var/www/ onto EBS. First use ec2-describe-instances to find out which Amazon zone the server is located in[4]: us-east-1c. Now create a 5G EBS volume:

    ec2-create-volume -z us-east-1c -s 5

    From what I can see so far, changing the volume size probably involves snapshotting the volume, and then restoring the snapshot to a bigger volume. This small headache must be traded-off against the EBS cost of 10 cents per gigabyte per month.

    ec2-describe-volumes will tell you when the new volume is available. Now attach the volume to the server instance at /dev/sdz:

    ec2-attach-volume -d /dev/sdz -i i-c1a320a8 vol-3159bd58

    Now login to the server and format the volume:

    mkfs.ext3 /dev/sdz

    (Note that xfs has apparently been a popular format choice in the past because of its explicit freezability for snapshots, but I have seen in more then one place now that Amazon EC2 has problems with xfs.)

    Setup /etc/fstab by adding the following entry:

    /dev/sdz /vol ext3 noatime 0 0
    And mount the volume:
    cd /
    mkdir vol
    mount vol
    Now move the contents of /var/www to the volume:
    /etc/init.d/apache2 stop
    cd /var
    mv www /vol/
    ln -s /vol/www/ www
    /etc/init.d/apache2 start

    [1] http://aws.amazon.com/ebs/
    [2] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1663
    [3] http://blog.rightscale.com/2008/08/20/amazon-ebs-explained/
    [4] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1347

    posted at: 02:35 | path: /Hosting/Amazon/EBS | permanent link to this entry

    Tue, 17 Feb 2009

    /Security/IMchat: Chat Monitoring is a Standard Firewall Feature

    And I quote[1]:

    "The IM proxy is the best I’ve seen. Once it’s enabled, every incoming and outgoing IM conversation is logged. After opening up a few channels in IRC - in real-time - it’s possible to view any conversation going through the firewall. MSN, AIM, and other protocols are supported as well. It’s a big-brother feature, but if you want to monitor who you children are talking to, or for whatever reason, I can see it being an invaluable resource to monitor what is going on in a network you control. It would almost be easier to keep track of conversations using the logging tool in Smoothwall instead of multiple instant messenger clients."

    The above quote is from a review of the free version of the Smoothwall[2] firewall -- you do not even have to pay money for this feature in Smoothwall! Meaning this feature is probably simple and common among commercial firewalls. Anyone who thinks that their employer is not listening in on their Yahoo Messenger / MSN / AIM / etc. chat sessions is being extremely naive. Check out this[3]:

    "New IM Reports - Generate new reports on IM including time spent messaging and number of chat friends per user."

    The solution: use a chat client that does encryption:

    [1] http://www.fsckin.com/tag/smoothwall/
    [2] http://www.smoothwall.net/products/
    [3] http://www.smoothwall.net/products/corporatefirewall2008/?whatsnew
    [4] http://www.pidgin.im/
    [5] http://www.cypherpunks.ca/otr/
    [6] http://www.adiumx.com/
    [7] http://www.skype.com/
    [8] http://www.cypherpunks.ca/otr/software.php
    [9] http://www.google.com/talk/
    [10] http://blog.langex.net/index.cgi/Security/skype/

    posted at: 22:45 | path: /Security/IMchat | permanent link to this entry

    Fri, 06 Feb 2009

    /Hosting/Amazon/EC2: Managing Amazon EC2 Instances

    The Amazon EC2 Compute Cloud provides some very powerful capabilities, but a fair bit of complexity comes with that power. This article is a brief description of the process of setting up an Amazon account, getting an instance running, making changes to that instance, and then storing the changed instance / server as a new instance. (This new instance could be a backup for an existing server, or the starting point for one or more new servers based upon it....) Here are some good references[1][2][3].

    First register for an Amazon account here[4]. Then install Amazon's (Java-based) EC2 command line tools[5]. (I will write a separate article on this at another time.)

    Choose the instance that you would like to run:

    This is surprisingly not straight-forward. Amazon seems to have a number of more or less official instance images. If you run the command

    ec2-describe-images -o self -o amazon | grep machine

    you will see a list of image candidates which seem to be almost exclusively Fedora- or Windows-based. What about Debian users like myself? Here[6] there seems to be a large collection of user-submitted images. I resorted to Google, and settled upon this[7] Debian Lenny-based image that seems to have an ongoing community around it, complete with a mailing list.

    Start Your Instance / Server:

    By default the root account of a new instance is accessed via SSH using a pre-defined "key pair" rather then the usual root password. First generate this key pair:

    ec2-add-keypair {key_pair_name}

    and copy the output lines into a private key file, which will later be referenced by the -k option in EC2 commands. Run "chmod 600" on this file to make it readable only by root, otherwise the EC2 commands will reject it.

    Now start your server:

    ec2-run-instances ami-115db978 -k {path_to_above_key_file}

    where ami-115db978 is the instance ID of my previously chosen Debian image[7]. Note the instance ID of your new running instance: i-c1a320a8. Check on the status of the server:

    ec2-describe-instances --or--
    ec2-describe-instances {above_noted_instance_id}

    Once the server's status is "running", you can login. First open the SSH port:

    ec2-authorize default -p 22
    Now login to the server:
    ssh -i {path_to_above_key_file} root@{hostname}

    where the hostname is the long URL output by ec2-describe-instances, of the form ec2-{IP_address}.*.amazonaws.com. (There should be no password prompt....)

    Save a snapshot of your changed server:

    After making some changes to the server, it is natural to want to save a copy. Note that Amazon EC2 instances by default are not persistent. If you shut down the server, or turn it off with ec2-terminate-instances, all changes you made since it was started from the original instance image will be lost. (This is one striking difference from the behavior of a typical Virtual Private Server.) So one simple way to backup is to create a new instance image based upon your running server[8].

    Since the following operations will be performed on the running server, and not your desktop, the Amazon account keys need to be uploaded to the server, ie.:

    scp -i ~/.ec2/keys/id_key pk-XXX.pem cert-xxx.pem root@ec2-45-105-235-35.compute-1.amazonaws.com:/mnt/

    Now on the server, create an image of the running server (a new instance):

    ec2-bundle-vol -k /mnt/pk-xxx.pem -c /mnt/cert-xxx.pem -u {Amazon_user_id} -d /mnt

    This will take a while. Note that ec2-bundle-vol will by default exclude a number of sensible directories like /sys, /proc, /mnt, etc. Now, using the Debian package s3cmd (more detail forthcoming in another post on the subject of Amazon S3) create a "bucket" in the Amazon S3 service in which to store the generated instance:

    s3cmd mb s3://xxxxx-snapshots

    Now upload the image to the created bucket:

    ec2-upload-bundle -b {bucket_name} -m /mnt/image.manifest.xml -a {aws-access-key-id} -s {aws-secret-access-key-id}

    where aws-access-key-id and aws-secret-access-key-id are your Amazon accounts 20- and 40-character access key and secret access key, respectively. This will also take a little while. Now on your desktop machine (ec2-register is not on my server, at least) register the image:

    ec2-register {bucket_name}/image.manifest.xml

    and record the instance ID generated: ami-20ef0849. Now look at bucket contents:

    s3cmd ls s3://xxxxx-snapshots

    and see that all the image.* files in the server's /mnt directory are now in the S3 bucket. Ie. logically there will be one instance snapshot per bucket.

    Test the Newly Created Instance:

    This would seem extremely prudent, especially the first time through the process:

    ec2-run-instances ami-20ef0849 -k {key_pair_name}

    Note that, if reusing key_pair_name from the original server that we have just made a backup copy of, the name specified has to be the same as originally passed to the ec2-add-keypair command. This may or may not be the same name as the file it is saved in. On my desktop, for instance, the keypair name is ktdebian, and it is saved in a file called id-rsa-ktdebian.

    Login to the test server and verify that it is working, then stop it:

    ec2-terminate-instances {test_server_instance_id}

    Note that our newly created image should still be private, no one else should be able to use and start it. To make it publicly available requires explicit action of the form:

    ec2-modify-image-attribute --launch-permission -a all

    (Note that I have not tested ec2-modify-image-attribute....)

    [1] http://docs.amazonwebservices.com/AWSEC2/2008-02-01/GettingStartedGuide/
    [2] http://wiki.rpath.com/wiki/rBuilder_Online:Amazon_Elastic_Compute_Cloud
    [3] http://paulstamatiou.com/2008/04/05/how-to-getting-started-with-amazon-ec2
    [4] http://aws.amazon.com/
    [5] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88
    [6] http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=101
    [7] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1615&categoryID=101
    [8] http://afkham.org/2008/10/how-to-create-ec2-ami.html

    posted at: 17:23 | path: /Hosting/Amazon/EC2 | permanent link to this entry

    /Hosting/Amazon/EC2: The Amazon Elastic Compute Cloud (Amazon EC2) web service

    I have been hearing a lot of buzz about "cloud computing"[1] and particularly about Amazon's version[2][6]. Finally, I have been hired by a client to build a server using the Amazon service, and get to try it out.

    I am still in the early stages, but I am not seeing yet how Amazon's "Compute Cloud" is fundamentally different from your average "Virtual Private Server" (VPS) service. They do provide a capability of saving spanshots of a machine, and then redeploying copies of that snapshot to multiple other machines. I guess that is kind of special....

    The "Compute Cloud" is also not as user-friendly as your average VPS service. With the latter, setup is a matter of button pushing. With the former, you have to download and install a command line application, and jump through quite a few command line hoops to get things running. Not particularly difficult for an experience Linux sysadmin, but GUI addicts definitely will find themselves challenged. This article[3] should provide a good appreciation, and most or all of the details, of getting an Amazon server up and running.

    Note that Amazon seems to really like Fedora, and most of their ready-made images seem to be Fedora-based. As a Debian user, I had to go a little further afield[4] to find a suitable Debian image[5] to install.

    So far so good, it is running as I write, and was really not that hard to figure out.

    [1] http://en.wikipedia.org/wiki/Cloud_computing
    [2] http://docs.amazonwebservices.com/AWSEC2/2008-02-01/GettingStartedGuide/index.html?introduction.html
    [3] http://paulstamatiou.com/2008/04/05/how-to-getting-started-with-amazon-ec2
    [4] http://alestic.com/
    [5] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1615&categoryID=101
    [6] http://www.geekzone.co.nz/foobar/5654

    posted at: 14:08 | path: /Hosting/Amazon/EC2 | permanent link to this entry

    Tue, 03 Feb 2009

    /SW/website/blog/webgen: Getting Started with Webgen

    Webgen is a static website generator. Easiest is to start with an existing webgen example site, or a default site generated by "webgen create", then:

    1. First take a few minutes to choose, install, and tweak a template.
    2. Create your website's pages in the src directory (optionally containing a bit of HTML markup).
    3. Run the webgen command, and voila, the static HTML for your new site appears in the output directory, ready for upload to your server.

    There are a few useful details and maybe a couple of gotchas along the way, but that is basically it.

    First gotcha: as of this moment, there are two webgen packages in the Debian repository: webgen and webgen0.4. If you start with a webgen0.4-generated site from somewhere else, and then install just webgen, which is currently version 0.3.8, webgen may fail with some rather obtuse errors. Be careful which version you are installing, where your example site is coming from, and which version was used to generate it.

    The list of templates is actually a little hard to find on the website, so here it is[1]. To install a new template named "andreas07", in the site root (one directory up from the src and output directories):

    webgen use website_style andreas07

    Note that webgen will not overwrite existing template files, so you will probably first have to do a "rm default*" and "rm *.css" in the src directory.

    Then use your favorite text editor to tweak src/default.template and src/default.css to make the site look exactly how you want it. Graphics used in the layout are found in the images directory, and of course may also be customized.

    Check your layout by running "webgen" in the site root (one directory up from the src and output directories) and point your web browser at the output directory to see the result. Note that webgen runs very fast, so it is a good idea to frequently check your layout changes so that the cause of an error is immediately obvious.

    Once you are satisfied with the layout, start adding content to the src directory. Each *.page file corresponds to a page on your site, and after running "webgen" will appear as a *.html file in the output directory. Each page will also appear in the site template's output HTML menu.

    [1] http://webgen.rubyforge.org/documentation/0.4.x/examples/website_styles/

    posted at: 17:13 | path: /SW/website/blog/webgen | permanent link to this entry

    Mon, 02 Feb 2009

    /Admin/SSH: Controlling Multiple Machines with Cluster SSH

    If you find yourself doing exactly the same set of steps on more then one machine, then Cluster SSH[1] is an option. Its very simple to get started. In ~/.csshrc define the set(s) of workstations that you would like to control:

    clusters = <tag1> <tag2> <tag3> <tag1> = host1 host2 host3 <tag2> = user@host4 user@host5 host6 <tag3> = <tag1> <tag2>

    Then execute "cssh tag1" and three terminals, one each for tag1, tag2, and tag3 will appear, plus the cssh control window. Any text typed into the control window will be echoed in all terminals.

    If passwordless authentication has not been configured for the remote boxes, then you will have to enter a password in each terminal at the start.

    [1] http://sourceforge.net/projects/clusterssh/

    posted at: 04:11 | path: /Admin/SSH | permanent link to this entry