Expat-IT Tech Bits




Search this site:


/ (289)
  Admin/ (123)
    Apache/ (10)
      HTTPS-SSL/ (4)
      PHP/ (3)
      performance/ (2)
    Cherokee/ (1)
    LAN/ (4)
    LVM/ (6)
    Monitoring/ (2)
      munin/ (2)
    SSH/ (6)
    SSL/ (1)
    Samba/ (1)
    VPN-options/ (7)
      OpenVPN/ (1)
      SSH-Proxy/ (3)
      Tinc/ (1)
      sshuttle/ (1)
    backups/ (17)
      SpiderOak/ (1)
      backuppc/ (5)
      dirvish/ (1)
      misc/ (6)
      rdiff-backup/ (1)
      rsync/ (1)
      unison/ (2)
    commandLine/ (24)
      files/ (8)
      misc/ (10)
      network/ (6)
    crontab/ (1)
    databases/ (15)
      MSSQL/ (2)
      MySQL/ (8)
      Oracle/ (3)
      PostgreSQL/ (1)
    dynamicDNS/ (2)
    email/ (11)
      Dovecot/ (1)
      deliverability/ (1)
      misc/ (1)
      postfix/ (7)
      puppet/ (1)
    iptables/ (3)
    tripwire/ (1)
    virtualization/ (9)
      VMware/ (1)
      virtualBox/ (8)
  Coding/ (14)
    bash/ (1)
    gdb/ (1)
    git/ (3)
    php/ (5)
    python/ (4)
      Django/ (2)
  Education/ (1)
  Hosting/ (27)
    Amazon/ (18)
      EBS/ (3)
      EC2/ (10)
      S3/ (1)
      commandline/ (4)
    Godaddy/ (2)
    NearlyFreeSpeech/ (3)
    Rackspace/ (1)
    vpslink/ (3)
  Linux/ (31)
    Android/ (1)
    Awesome/ (3)
    CPUfreq/ (1)
    China/ (2)
    Debian/ (8)
      APT/ (3)
      WPA/ (1)
    audio/ (1)
    encryption/ (3)
    fonts/ (1)
    misc/ (6)
    remoteDesktop/ (1)
    router-bridge/ (3)
  SW/ (45)
    Micro$soft/ (1)
    browser/ (2)
      Chrome/ (1)
      Firefox/ (1)
    business/ (28)
      Drupal/ (9)
      KnowledgeTree/ (6)
      Redmine/ (2)
      SugarCRM/ (7)
      WebERP/ (2)
      WordPress/ (1)
      eGroupware/ (1)
    chat/ (1)
    email/ (1)
    fileSharing/ (2)
      btsync/ (1)
      mldonkey/ (1)
    graphics/ (2)
    research/ (2)
    website/ (6)
      blog/ (6)
        blosxom/ (3)
        rss2email/ (1)
        webgen/ (1)
  Security/ (15)
    IMchat/ (2)
    circumvention/ (2)
    cryptoCurrency/ (1)
    e-mail/ (4)
    greatFirewall/ (1)
    hacking/ (1)
    password/ (1)
    privacy/ (2)
    skype/ (1)
  Services/ (1)
    fileSharing/ (1)
  TechWriting/ (1)
  xHW/ (14)
    Lenovo/ (1)
    Motorola_A1200/ (2)
    Thinkpad_600e/ (1)
    Thinkpad_a21m/ (3)
    Thinkpad_i1300/ (1)
    Thinkpad_x24/ (1)
    USB_audio/ (1)
    scanner/ (1)
    wirelessCards/ (2)
  xLife/ (17)
    China/ (9)
      Beijing/ (5)
        OpenSource/ (3)
    Expatriation/ (1)
    Vietnam/ (7)


  • 2019/06
  • 2016/07
  • 2016/05
  • 2016/02
  • 2016/01
  • 2015/12
  • 2015/11
  • 2015/06
  • 2015/01
  • 2014/12
  • 2014/11
  • 2014/10
  • 2014/09
  • 2014/07
  • 2014/04
  • 2014/02
  • 2014/01
  • 2013/12
  • 2013/10
  • 2013/08
  • 2013/07
  • 2013/06
  • 2013/05
  • 2013/04
  • 2013/02
  • 2013/01
  • 2012/12
  • 2012/10
  • 2012/09
  • 2012/08
  • 2012/07
  • 2012/06
  • 2012/05
  • 2012/04
  • 2012/03
  • 2012/01
  • 2011/12
  • 2011/11
  • 2011/10
  • 2011/09
  • 2011/08
  • 2011/07
  • 2011/06
  • 2011/05
  • 2011/04
  • 2011/02
  • 2010/12
  • 2010/11
  • 2010/10
  • 2010/09
  • 2010/08
  • 2010/07
  • 2010/06
  • 2010/05
  • 2010/04
  • 2010/03
  • 2010/02
  • 2010/01
  • 2009/12
  • 2009/11
  • 2009/10
  • 2009/09
  • 2009/08
  • 2009/07
  • 2009/06
  • 2009/05
  • 2009/04
  • 2009/03
  • 2009/02
  • 2009/01
  • 2008/12
  • 2008/11
  • 2008/10
  • 2008/09
  • Subscribe XML RSS Feed

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

    This site has no ads. To help with hosting, crypto donations are accepted:
    Bitcoin: 1JErV8ga9UY7wE8Bbf1KYsA5bkdh8n1Bxc
    Zcash: zcLYqtXYFEWHFtEfM6wg5eCV8frxWtZYkT8WyxvevzNC6SBgmqPS3tkg6nBarmzRzWYAurgs4ThkpkD5QgiSwxqoB7xrCxs

    Tue, 26 May 2009

    /Admin/email/Dovecot: Imposing Per User E-mail Quotas with Dovecot

    Information seems to be scattered around a bit, and not crystal-clear-specific, so I will be very specific about my setup here.

    For a postfix / dovecot imap setup, it looks like dovecot does all the quota control.

    To enable quotas per http://wiki.dovecot.org/Quota, add the following to /etc/dovecot/dovecot.conf:

    protocol imap {
      mail_plugins = quota imap_quota
    protocol pop3 {
      mail_plugins = quota
    # In case you're using deliver:
    protocol lda {
      mail_plugins = quota

    Now per http://wiki.dovecot.org/Quota/Maildir if Dovecot quotas have been turned on per above, but no limits are specified within Dovecot, then Dovecot will look for the maildirsize file in the user's mail directory. If this file does not exist, that user has an unlimited quota. If the file exists, the first line should contain the quota in the form

    <storage limit in bytes>S,<messages limit>C

    ie. "3072000S" would indicate a 3M quota, and Dovecot will enforce that quota. This then would be a manual method for implementing user-specific quotas, by hand- (or script-) editing the maildirsize file in each user's mail directory.

    As of Dovecot 1.1 and later, http://wiki.dovecot.org/Quota/1.1 tells us how to enforce quotas from within Dovecot. To apply a uniform system-wide quota to every user, add this to /etc/dovecot/dovecot.conf:

    plugin {
       # Dovecot 1.1 config
       quota = maildir:User quota
       quota_rule = *:storage=5M
       # 10% of 5M = 500k: Trash gets 10% above quotas to enable moves into Trash
       quota_rule2 = Trash:storage=10%%

    (quota_rule2 for Trash is an IMAP-specific fudge to avoid deletion problems with many e-mail clients when quota is exceeded....)

    For user-specific quotas stored in the SQL user database, http://wiki.dovecot.org/Quota/1.1 is again our guide, with a little help from http://oli.lugh.ch/quota-workaround.org.html. First add a quota field to the database and try it out with a test user:

    mysql> alter table virtual_users add column quota_kb varchar (11);
    mysql> update virtual_users set quota_kb=6000 where user="testing";

    Note that the following rules apply:

    Then tell dovecot to go to SQL for the quota by inserting this into /etc/dovecot/dovecot.conf

    userdb sql {
    args = /etc/dovecot/dovecot-sql.conf

    in front of the "userdb static" block so preference is given to SQL. And define the SQL query in /etc/dovecot/dovecot-sql.conf:

    user_query = SELECT 5000 AS uid, 5000 AS gid,'/home/vmail/%d/%n' as home, concat('*:storage= ', quota_kb) AS quota_rule FROM virtual_users WHERE user = '%n';

    E-mails that exceed the quota will receive this error response:

    Automatically rejected mail: Quota exceeded

    posted at: 04:58 | path: /Admin/email/Dovecot | permanent link to this entry

    Sun, 24 May 2009

    /Security/hacking: Did I Just Experience an Attempted "Man-in-the-Middle" Attack?

    Normally, of course, one expects that an encrypted connection between two computers will be private and free of eavesdroppers. But of course, no defence is perfect....

    Just now I tried to SSH from China into one of my servers in the USA. The SSH command failed, with a big banner saying that the key that the server just answered with was not the same as the last time I logged in, and that I might be the subject of a Man-in-the-Middle Attack.

    In a Man-in-the-Middle Attack, someone on a network between me and my destination (those "people" running the Great Firewall of China come to mind, for instance) intercepts my communication with my server, and pretends to be my server. They relay the connection to the actual server, so if I were to ignore the warning about the bad key and log in anyway, I would actually succeed in logging into my server. But whoever intercepted and forwarded the connection would now be able to eavesdrop on the communication, and I bet (do not know right off hand) that they might get my server password as well.

    My response? I tried an SSH into a second server in the USA, and from there SSH'ed into the first server. No problem with both of those. Then I tried a direct connection straight from home to the first server again. This time it worked. No hacker in the middle.

    Do not take security warnings (from software you trust, for which SSH definitely qualifies) lightly.

    posted at: 01:59 | path: /Security/hacking | permanent link to this entry

    Sat, 23 May 2009

    /Linux/Debian/APT: Software Package Management in Debian (and Ubuntu)

    I particularly dislike graphical package managers, as they are all seem slow and, I think, not much easier to use then the command line. Here is a little quick start guide to using the command line to install and remove software in Debian or Ubuntu:

    Search for software:

    apt-cache search {space separated list of key words}
    Display information about a particular package:
    apt-cache show {name of package}
    Upgrade: update your software list and then upgrade all installed software to the latest version:
    apt-get update
    apt-get upgrade
    Install a defined list of packages:
    apt-get install {space separated list of package names}
    wajig, a useful utility: install wajig and less:
    apt-get update
    agt-get install wajig less
    Then "wajig list-commands", which will produce output that is more then one page long, so even better is:
    "wajig list-commands | less"

    (Exit less by entering "ZZ")

    "|" is the "pipe" operator, found on the same key as "\" on my keyboard, and is one of those whiz-bang UNIX features that, after you accumulate a short list, will leave you quite disinterested in ever going back to Micro$oft Windows.

    Hold a package and exclude it from automatice upgrades:

    'echo -e "python-pysqlite2 hold" | dpkg --set-selections'

    where python-pysqlite2 is the package to be held. Thereafter, when one performs an "apt-get upgrade" the held package appears in the "following packages have been kept back" section, and requires a manual "apt-get install python-pysqlite2" to re-instate the latest version.

    Revert a package to an older version:

    For instance, if you track "testing", but want to use the occasional unbroken package in "stable", your "/etc/apt/sources.list" should contain the following two lines:

    deb http://ftp.us.debian.org/debian/ testing main contrib non-free
    deb http://ftp.us.debian.org/debian/ stable main contrib non-free

    "testing" will take precedence over "stable" as it appears first in the list. Now suppose a just updated piece of "testing" software is broken. First find out what versions are available in your current archive:

    apt-cache policy python-pysqlite2

    To install an older version:

    apt-get install python-pysqlite2=2.4.1-1

    where the version identifier comes after the "=". And now, to prevent your just downgraded package from being clobbered by automatic upgrades, put the package on "hold", as described in the previous section.

    posted at: 04:35 | path: /Linux/Debian/APT | permanent link to this entry

    Fri, 22 May 2009

    /Admin/email/misc: Securing Your E-mail Server

    I used this excellent tutorial[1] for most of the basic server setup (which is still in progress....) What follows are some supplementary security-related notes.

    My goal is not so much a big server with a lot of users, but rather to provide a highly secure, private e-mail service. I want to set it up so that it is not easy to avoid using encryption in order to read, download, or send e-mail through my server. Therefore my users can have confidence that their communications with my server will be private, and they will not needlessly expose themselves to eavesdropping. (It would be preferred to always insist on encryption, but this is not possible. SMTP, for instance, must accept unencrypted connections from other servers, and there is no way to prevent an e-mail client from behaving like a server and making an unencrypted connection to deliver e-mail to an account that is local to the server, as this does not constitute relaying and does not require authentication.)

    This server uses a postfix[2] e-mail server, a dovecot[3] imap server, prayer[6] for webmail, and a MySQL[4] database backend containing user account data.

    The security steps I have taken so far:

    Here[5] is a good resource for e-mail client configuration. For my sylpheed e-mail client, I use the following settings:

    [1] http://workaround.org/articles/ispmail-etch/ (Note: I keep a local copy of this howto (hidden) which can be resurrected if the source disappears.)
    [2] http://www.postfix.org/
    [3] http://www.dovecot.org/
    [4] http://dev.mysql.com/
    [5] http://ist.uwaterloo.ca/cs/emailsecure/
    [6] http://www-uxsup.csx.cam.ac.uk/~dpc22/prayer/

    posted at: 03:35 | path: /Admin/email/misc | permanent link to this entry

    /SW/business/KnowledgeTree: Metadata and the KnowledgeTree REST API

    The only way I have personally been able to get this working is by patching KnowledgeTree. The problem is that there is a function get_packed_metadata in ktapi/KTAPIDocument.inc.php that expects to receive metadata in the form of an array. There is no way to pass an array through a URL as a POST parameter using PHP (at least, none that I have yet found....)

    What I have found is that the PHP serialize() / unserialize() functions are meant to get this job done: serialize() the array before passing, unserialize() it after passing, and then process the array on the receiving end. So I have added this chunk of code to the top of function get_packed_metadata in ktapi/KTAPIDocument.inc.php:

    if (!is_array($metadata)) { $metadata = unserialize($metadata); }

    And then if I serialize my metadata array before sending it into the API, everything works. The exact structure of the metadata array was also a bit of a hard-won mystery, so I will reproduce below an actual working metadata update snippet of code:

    <?php $url = 'https://apps.vancouversolidcomputing.com/kt-dms-oss-3.6.0/ktwebservice/KTWebService.php'; require_once('funCurlPost.php'); // ************************** // Login // ************************** $postfields = "method=login&password=123456&username=admin"; $response = curlPost($url, $postfields); $xml = new SimpleXMLElement($response); if( $xml->status_code != 0 ){ echo 'Error - authentication failed: ' . $xml->message; } else { $session_id = $xml->results; echo "Login successful, session ID = " . $session_id; } // *********************************** // Update document metadata // *********************************** $metadata = array( array( "fieldset" => "Tag Cloud", "fields" => array( array( "name" => "Tag", "value" => "Tag name from API call" ) ) ) ); $metadata = serialize($metadata); $postfields = "method=update_document_metadata&session_id=$session_id&document_id=36&metadata=$metadata"; $response = curlPost($url, $postfields); $xml = new SimpleXMLElement($response); echo "<p>Update document metadata:<br>"; if( $xml->status_code != 0 ){ echo 'Error - get_document_metadata failed: ' . $xml->message; } else { echo 'Metadata updated!!'; }

    I have posted on this subject to both the KnowledgeTree forum[1] and the issue tracker[2], and received just about zero response.

    I have a working example of a file upload followed by an add-document-with-metadata API operation, if anyone is interested in seeing that.

    [1] http://forums.knowledgetree.com/viewtopic.php?f=10&t=14291&p=31392#p31392
    [2] http://issues.knowledgetree.com/browse/WSA-169

    posted at: 00:03 | path: /SW/business/KnowledgeTree | permanent link to this entry

    Thu, 07 May 2009

    /Coding/python: Python: How to Put Some of Your Code in a Subdirectory

    Splitting one's code up into logical sub-directories is a fairly normal practice. This seems to be a little more complicated then usual with Python. One uses the "import" command to access functions in other files / directories. Of course Python automatically searches the system path and your execution directory to satisfy an import command. But to pull in a subdirectory one must place an (empty? Mine is....) __init__.py file in any sub-directory containing Python code that one wants to access. Then the directory name is added to the Python name space and is treated the same as files and functions defined within files.

    A concrete example is most definitely in order. Suppose I want to place all my database connect code into a subdirectory called "db":

    db$ ll
    total 16
    -rw------- 1 1489 2009-02-23 15:18 armor-mail-cert.pem
    -rw------- 1 3243 2009-02-23 15:18 armor-mail-key.pem
    -rw------- 1 1009 2009-02-23 14:48 armorMailSSL-cacert.pem
    -rwxr-xr-x 1  436 2009-05-07 15:55 connectArmorMail.py
    -rw-r--r-- 1    0 2009-05-07 16:33 __init__.py

    My main code is executing one directory up. Any code there that wants to access the connect function must contain:

    import db.connectArmorMail
    db = db.connectArmorMail.connect_armor_mail()

    Note that we are importing db.connectArmorMail, *not* db/connectArmorMail, and then the reference in the next line also uses a ".", not a "/". "db" is a directory, "connectArmorMail" is a file name, and "connect_armor_mail" is a function within a file, but all are treated equally in Python name space, separated by periods. db.connectArmorMail contains:

    def connect_armor_mail():
       import MySQLdb
       ssl_settings = {'ca': 'db/armorMailSSL-cacert.pem',
           'cert': 'db/armor-mail-cert.pem',
           'key': 'db/armor-mail-key.pem'
          print "

    ERROR: could not connect to database." return db

    Note that the files referenced in ssl_settings are in the same directory as db.connectArmorMail, but they are referenced relative to the execution directory (db/*).

    posted at: 05:01 | path: /Coding/python | permanent link to this entry