Expat-IT Tech Bits




Search this site:


/ (289)
  Admin/ (123)
    Apache/ (10)
      HTTPS-SSL/ (4)
      PHP/ (3)
      performance/ (2)
    Cherokee/ (1)
    LAN/ (4)
    LVM/ (6)
    Monitoring/ (2)
      munin/ (2)
    SSH/ (6)
    SSL/ (1)
    Samba/ (1)
    VPN-options/ (7)
      OpenVPN/ (1)
      SSH-Proxy/ (3)
      Tinc/ (1)
      sshuttle/ (1)
    backups/ (17)
      SpiderOak/ (1)
      backuppc/ (5)
      dirvish/ (1)
      misc/ (6)
      rdiff-backup/ (1)
      rsync/ (1)
      unison/ (2)
    commandLine/ (24)
      files/ (8)
      misc/ (10)
      network/ (6)
    crontab/ (1)
    databases/ (15)
      MSSQL/ (2)
      MySQL/ (8)
      Oracle/ (3)
      PostgreSQL/ (1)
    dynamicDNS/ (2)
    email/ (11)
      Dovecot/ (1)
      deliverability/ (1)
      misc/ (1)
      postfix/ (7)
      puppet/ (1)
    iptables/ (3)
    tripwire/ (1)
    virtualization/ (9)
      VMware/ (1)
      virtualBox/ (8)
  Coding/ (14)
    bash/ (1)
    gdb/ (1)
    git/ (3)
    php/ (5)
    python/ (4)
      Django/ (2)
  Education/ (1)
  Hosting/ (27)
    Amazon/ (18)
      EBS/ (3)
      EC2/ (10)
      S3/ (1)
      commandline/ (4)
    Godaddy/ (2)
    NearlyFreeSpeech/ (3)
    Rackspace/ (1)
    vpslink/ (3)
  Linux/ (31)
    Android/ (1)
    Awesome/ (3)
    CPUfreq/ (1)
    China/ (2)
    Debian/ (8)
      APT/ (3)
      WPA/ (1)
    audio/ (1)
    encryption/ (3)
    fonts/ (1)
    misc/ (6)
    remoteDesktop/ (1)
    router-bridge/ (3)
  SW/ (45)
    Micro$soft/ (1)
    browser/ (2)
      Chrome/ (1)
      Firefox/ (1)
    business/ (28)
      Drupal/ (9)
      KnowledgeTree/ (6)
      Redmine/ (2)
      SugarCRM/ (7)
      WebERP/ (2)
      WordPress/ (1)
      eGroupware/ (1)
    chat/ (1)
    email/ (1)
    fileSharing/ (2)
      btsync/ (1)
      mldonkey/ (1)
    graphics/ (2)
    research/ (2)
    website/ (6)
      blog/ (6)
        blosxom/ (3)
        rss2email/ (1)
        webgen/ (1)
  Security/ (15)
    IMchat/ (2)
    circumvention/ (2)
    cryptoCurrency/ (1)
    e-mail/ (4)
    greatFirewall/ (1)
    hacking/ (1)
    password/ (1)
    privacy/ (2)
    skype/ (1)
  Services/ (1)
    fileSharing/ (1)
  TechWriting/ (1)
  xHW/ (14)
    Lenovo/ (1)
    Motorola_A1200/ (2)
    Thinkpad_600e/ (1)
    Thinkpad_a21m/ (3)
    Thinkpad_i1300/ (1)
    Thinkpad_x24/ (1)
    USB_audio/ (1)
    scanner/ (1)
    wirelessCards/ (2)
  xLife/ (17)
    China/ (9)
      Beijing/ (5)
        OpenSource/ (3)
    Expatriation/ (1)
    Vietnam/ (7)


  • 2019/06
  • 2016/07
  • 2016/05
  • 2016/02
  • 2016/01
  • 2015/12
  • 2015/11
  • 2015/06
  • 2015/01
  • 2014/12
  • 2014/11
  • 2014/10
  • 2014/09
  • 2014/07
  • 2014/04
  • 2014/02
  • 2014/01
  • 2013/12
  • 2013/10
  • 2013/08
  • 2013/07
  • 2013/06
  • 2013/05
  • 2013/04
  • 2013/02
  • 2013/01
  • 2012/12
  • 2012/10
  • 2012/09
  • 2012/08
  • 2012/07
  • 2012/06
  • 2012/05
  • 2012/04
  • 2012/03
  • 2012/01
  • 2011/12
  • 2011/11
  • 2011/10
  • 2011/09
  • 2011/08
  • 2011/07
  • 2011/06
  • 2011/05
  • 2011/04
  • 2011/02
  • 2010/12
  • 2010/11
  • 2010/10
  • 2010/09
  • 2010/08
  • 2010/07
  • 2010/06
  • 2010/05
  • 2010/04
  • 2010/03
  • 2010/02
  • 2010/01
  • 2009/12
  • 2009/11
  • 2009/10
  • 2009/09
  • 2009/08
  • 2009/07
  • 2009/06
  • 2009/05
  • 2009/04
  • 2009/03
  • 2009/02
  • 2009/01
  • 2008/12
  • 2008/11
  • 2008/10
  • 2008/09
  • Subscribe XML RSS Feed

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

    This site has no ads. To help with hosting, crypto donations are accepted:
    Bitcoin: 1JErV8ga9UY7wE8Bbf1KYsA5bkdh8n1Bxc
    Zcash: zcLYqtXYFEWHFtEfM6wg5eCV8frxWtZYkT8WyxvevzNC6SBgmqPS3tkg6nBarmzRzWYAurgs4ThkpkD5QgiSwxqoB7xrCxs

    Mon, 02 Dec 2013

    /Admin/email/postfix: Postfix and Opportunistic TLS Between Mail Servers

    Thanks to this thread[1] I was alerted to the feasability of having my server use TLS when sending and receiving e-mail over SMTP with other like-minded servers[2]. Which normally should include all of the big name free e-mail services. For Postfix, if this is not already the default (for me it was NOT the default for outgoing mail!) adding this:

    smtpd_tls_auth_only = yes
    smtpd_tls_security_level = may
    smtpd_tls_loglevel = 1
    smtp_tls_security_level = may
    smtp_tls_loglevel = 1

    to /etc/postfix/main.cf makes it happen. (And the first line instructs Postfix that mail clients authenticating with the server MUST use TLS.)

    I now see something like this in my mail log when connecting to gmail servers:

    postfix/smtp[32148]: Untrusted TLS connection established to gmail-smtp-in.l.google.com[]:25: TLSv1.1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)

    [1] https://overbenny.wordpress.com/2013/11/14/wanted-most-secure-unencrypted-email-solution/
    [2] https://www.d7031.de/content/postfix-and-tls

    posted at: 03:20 | path: /Admin/email/postfix | permanent link to this entry

    Sat, 04 Oct 2008

    /Admin/email/postfix: Receiving Mail for Multiple Domains

    Since I have customized my postfix setup somewhat at this point, I am no longer sure what default behavior is. But in my current state, just adding domains to the mydestination line in /etc/postfix/main.cf was not sufficient. I kept getting the following error for any mail received for my just added domains:

    mail for ckintl.biz loops back to myself
    The solution lies in my /etc/postfix/transport file (see Multi Relay). Domains for which I am handling e-mail must me directed to "local", ie:
        langex.net              local
        ckintl.biz              local
        doesharleysuck.com      local
    otherwise the final
    * smtp

    line forces incoming e-mail for these domains to be sent out of the machine via smtp, where DNS just resolves back to the very same machine. Thus the "mail loops back to myself" wording of the error message.

    Don't forget to add all local domain userids that don't already exist as an account on your mail server, and that you would like to receive mail for, to /etc/aliases. Then run

    /etc/init.d/postfix reload

    posted at: 04:44 | path: /Admin/email/postfix | permanent link to this entry

    /Admin/email/postfix: Relay Your E-mail Through Multiple Servers with Postfix

    Why would you want to do this? As mentioned in Simple Relay, if you have a lot of e-mail going out and you are trying to relay all of it through one poor free relay server, that relay server is probably going to reach a threshold at which it labels you a "spammer" and starts deferring or even bouncing your e-mails. So what I do is use my own server to send e-mails directly to their destination as much as possible. And for those domains that reject my direct connections (theoretically because my server is running on a dynamic ip) I relay through another server. Preferably I have several such "other" relay servers so as to spread the load around and not have to deal with bounced e-mail.

    Postfix is not really designed to do this, so what I will present here is a limited solution that will work under specific circumstances.

    Basically the problem is that any free SMTP server I have been able to find insists on authentication (logging in with a valid userid and password) and further insists that the envelope address of the e-mail you are sending must agree with the userid you are logging in with. For instance, if you have a sina.com account with userid=mysinaname and password=mysinapassword, if the envelope address is anything but mysinaname@sina.com, the sina SMTP server will reject the e-mail. Note that the "envelope address" is not necessarily the same as the "From:" address in the e-mail's header, although they are usually the same.

    If you are sending e-mail with an e-mail client, the client will of course take care of all of this. But if you are sending e-mail through your own local e-mail server, and then onwards to its destination (optionally through another relay server) things get a little more complicated as there may be e-mails being queued to the server from multiple sources, and Postfix appears to have no way of associating a given envelope address with a given destination address/domain.

    Postfix is capable of routing e-mails to different relay servers based upon their destination address. But it is left up to any software queuing e-mail to Postfix for external delivery to take care of making sure that the envelope address is correct for the relay server that Postfix will later relay to. That is what makes this a limited solution.

    Set Up Postfix

    The solution lies in /etc/postfix/transport. Here is mine:

        langex.net               local
        gmail.com               smtp:smtp.sohu.com
        hotmail.com             smtp:smtp.sohu.com
        sina.com                smtp:smtp.sina.com
        boltblue.com            smtp:smtp.sina.com
        msn.com                 smtp:smtp.sina.com
        onlineworkshop.net      smtp:smtp.sina.com
        *                       smtp

    Mail destined for langex.net never leaves my server and is delivered locally. gmail.com and hotmail.com are both relayed via SMTP through smtp.sohu.com. sina.com, msn.com, etc. are relayed via SMTP through smtp.sina.com. Everything else ("*") goes out via SMTP directly to the destination domain (no relay).

    Ensure that /etc/postfix/sasl_passwd contains userid:password for both smtp.sohu.com and smtp.sina.com, then run the following commands:

        postmap /etc/postfix/sasl_passwd
        postmap /etc/postfix/transport
        /etc/init.d/postfix restart

    Set Up PHP

    My main source of e-mail on this server is a PHP-driven website. In this website I have the following function:

            function emailWrap($thisEmailAddress, $thisSubject, $thisMessage, $thisReplyTo)
              $toDomain = stristr( $thisEmailAddress, "@" ); // extract domain from e-mail address
              $toDomain = strtolower ( $toDomain ); // make all alphabetic characters lower case
                  $headMe = 'webmaster@langex.net';
                case "@gmail.com":
                  $headMe = 'bjlangex@sohu.com';
                case "@hotmail.com":
                  $headMe = 'bjlangex@sohu.com';
                case "@sina.com":
                  $headMe = 'langexnet@sina.com';
                case "@boltblue.com":
                  $headMe = 'langexnet@sina.com';
                case "@msn.com":
                  $headMe = 'langexnet@sina.com';
                case "@onlineworkshop.net":
                  $headMe = 'langexnet@sina.com';
              $headFrom = "From: Language Exchange Webmaster <" . $headMe . ">";
              $headReplyTo = "Reply-To: " . $thisReplyTo;
              $headContent = "Content-Type: text/plain; charset=GB2312";
              $addheader = $headFrom . "\n" . $headReplyTo . "\n" . $headContent;
              $addParam = "-f " . $headMe; // the PHP magic for setting the envelope address
              mail($thisEmailAddress, $thisSubject, $thisMessage, $addheader, $addParam);

    There are two main things to take note of in this function:

    1. The contents of the switch must be kept in alignment with /etc/postfix/transport
    2. $addParam = "-f " . $headMe; is the statement that specifies/sets the envelope address within PHP.

    Re. item 2, I chased my tail for a long time before figuring out that the "additional parameter" to the PHP mail()[1] function was the Linux server way to do this. There is one particularly misleading PHP function called ini_set()[2] which DOES NOT WORK on Linux, and is apparently meant for Windows servers. In Summary

    For those e-mail destination domains (hopefully the vast majority) that do not give your server any grief, everything works just like a default Postfix server, ie. Postfix connects directly to the destination SMTP server and delivers the e-mail.

    For those domains that regularly give you delivery problems for whatever reason, if you can find a relay server that it will accept mail from, then use that relay server.

    But remember: any application on your server that queues an e-mail to Postfix for one of these problem destination domains, where you have not correctly configured the application to set the envelope address to correspond to the relay server, will result in bounced e-mail. This is a seriously non-trivial problem if you have other people using the server and/or e-mail being sent from multiple different sources. If you can limit the number of applications sending external e-mail to one, this method works really quite well.

    [1] http://www.php.net/mail
    [2] http://www.php.net/ini_set

    posted at: 04:38 | path: /Admin/email/postfix | permanent link to this entry

    /Admin/email/postfix: Simple E-mail Server Relay

    The obvious first thing to try is an unauthenticated relay through your ISP's e-mail server: since I am a paying ADSL customer within their network, hopefully authentication will not be necessary. My ISP is China Netcom (中国网通 - “Wang Tong”). From China Netcom's e-mail web page I deduce that one of their servers is smtp.bbn.cn. If I add

    relayhost = smtp.bbn.cn
    to /etc/postfix/main.cf, my e-mails bounce with the error “530 Authentication required”, ie. for this SMTP server I need to set up a China Netcom/网通 e-mail account and use authenticated relaying.

    I then set up a bbn.cn account on http://mail.bbn.com.cn/. Note that if you are not using a China Netcom/网通 internet connection, you will get the following error message when you attempt to register: “提示:请您使用北京网通宽带接入才能购买此产品”. Also note that if you cannot read Chinese then the translation function of www.xuezhongwen.net/chindict/chindict.php is your friend.

    I then tested my new bbn.cn account's SMTP with my regular e-mail client, and it seemed to work fine. I then configured my postfix server to relay ALL e-mails through my bbn.cn account:

    First edit /etc/postfix/sasl_passwd to contain:

    smtp.bbn.cn username:password
    Restrict access to this password file:
    chown root:root /etc/postfix/sasl_passwd && chmod 600 /etc/postfix/sasl_passwd
    Create a database version of the file (sasl_passwd.db) for postfix:
    postmap hash:/etc/postfix/sasl_passwd
    Now edit /etc/postfix/main.cf to add the following lines:
    relayhost = smtp.bbn.cn
    smtp_sasl_auth_enable = yes
    smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
    smtp_sasl_type = cyrus
    smtp_sasl_security_options = noanonymous
    and restart postfix:
    /etc/init.d/postfix restart
    You can then verify that your server’s outgoing e-mails are passing through smtp.bbn.cn by examining the header of a sent e-mail. Also, in /var/log/mail.log you should also see something like:
    postfix/smtp[7052]: 34CEA238D8: to=user@email.com, relay=smtp.bbn.cn[]:25, delay=1388, delays=1382/0.11/5.6/0.14, dsn=2.0.0, status=sent (250 ok: Message 135909587 accepted)

    Sadly, there seems to be a problem with smtp.bbn.cn, as some of my e-mail disappeared without a trace or a bounce. Bad server. Stay away from smtp.bbn.cn.

    But the process is very easy to replicate. Next I registered for a sohu.com e-mail account at mail.sohu.com and substituted:

    relayhost = smtp.sohu.com

    in /etc/postfix/main.cf. (And don't forget the correct userid/password for your sohu.com account in /etc/postfix/sasl_passwd...)

    smtp.sohu.com seems to be reliable in the sense that they do not lose my e-mail, but if there is a lot of outgoing mail in a short period, they start refusing service and finally bounce everything for a period of time until things quiet down. In other words, a simple relay through sohu will probably work for personal e-mail, but if you have other people using your server this will probably not be the final solution.

    I am running now on a somewhat more complicated configuration: a multiple relay.

    posted at: 04:21 | path: /Admin/email/postfix | permanent link to this entry

    /Admin/email/postfix: Filtering E-mail Content

    This is remarkably simple in Postfix.

    Create two files, /etc/postfix/header_checks and /etc/postfix/body_checks, with content like the following:

    /Anatrim/ REJECT
    /Viagra/ REJECT
    Then add the following lines to /etc/postfix/main.cf:
    body_checks = regexp:/etc/postfix/body_checks
    header_checks = regexp:/etc/postfix/header_checks
    and as usual restart postfix:
    /etc/init.d/postfix restart

    "header_checks" will reject any e-mail which contains any of the specified strings in the header, and "body_checks" will reject any e-mail which contains any of the specified strings in the body. Note that the header of an e-mail is generally much smaller then the body, so there is much less overhead with header checking. Also, these checks are case insensitive.

    There is the added bonus that the offending e-mails will be rejected by your e-mail server in a way that might very well leave a spammer with the impression that they are sending to an invalid e-mail address.

    An excellent resource: http://www.securityfocus.com/infocus/1598

    posted at: 04:10 | path: /Admin/email/postfix | permanent link to this entry

    /Admin/email/postfix: Blocking SPAM

    For the simple case of blocking/bouncing a particular e-mail address, create a file /etc/postfix/spammer containing something like the following:

    spammer@domain.com REJECT
    spammer2@spamcentral.net 554 Die Spammer, die!

    Note that a full list of possible return codes can be seen in RFC-821[1].

    Then run

    postmap spammer

    to generate a db file for postfix. Add the following line to /etc/postfix/main.cf:

    smtpd_sender_restrictions = reject_non_fqdn_sender, check_sender_access hash:/etc/postfix/spammer, permit

    Note that "reject_non_fqdn_sender" will bounce any e-mail coming from an address that does not have a fully-qualified domain name.

    Restart postfix:

    /etc/init.d/postfix restart

    That simple.

    Some useful resources: http://www.securityfocus.com/infocus/1593

    [1] http://www.ietf.org/rfc/rfc0821.txt

    posted at: 04:04 | path: /Admin/email/postfix | permanent link to this entry

    Sat, 06 Sep 2008

    /Admin/email/postfix: Impeding Spammers: Cap the SMTP E-mail Send Rate

    One name: postfix-policyd[1]. Awesome. And I am only using just one of its features to-date.

    I am not sure what the performance hit will be, because policyd is quite data intensive, and therefore is in fairly constant communication with MySQL. But this is an unavoidable nature of the beast.

    The feature I am using is "Sender-based Throttling", wherein I can restrict *anyone* (not just my own users) who connects to my SMTP server to sending no more then a specified number of messages into my server over a specified period of time (message rate - say max 10 msgs/hr, for instance) addressed to no more then a specific number of addressees per period (say max 100 addressees per hour, for instance). The same feature is also supposed to restrict message size, and bandwidth used (ie. MB per hour, for instance, per user) but I have not yet gotten this working.

    One caveat: webmail clients running on the same server do not use SMTP to send mail, they just use sendmail. So this method of restriction does not apply to webmail users (who are a *much* smaller spam problem...)

    One gotcha, that wasted several of my hours: I finally figured out that when I used StartTLS / SASL to send a message through SMTP, it was not being counted. This was caused by an order-of-parameters problem in /etc/postfix/main.cf. The postfix website[2] says main.cf should be configured as follows:

     5 /etc/postfix/main.cf:
     6     smtpd_recipient_restrictions =
     7         ... 
     8         reject_unauth_destination 
     9         check_policy_service unix:private/policy 
    which implied that check_policy_service should be the last item on the list. My current smtpd_recipient_restrictions list looks like this:
    smtpd_recipient_restrictions = 
       check_policy_service inet:
    My problem was that before, permit_sasl_authenticated was at the top of the list, which meant that SASL authenticated e-mails were immediately accepted and NEVER PASSED to policyd. Move permit_sasl_authenticated below the check_policy_service and policyd gets to make its decision first.

    [1] http://www.policyd.org/
    [2] http://www.postfix.org/SMTPD_POLICY_README.html

    posted at: 03:08 | path: /Admin/email/postfix | permanent link to this entry