Expat-IT Tech Bits

Home

Contact

Links

Search this site:

Categories:

/ (287)
  Admin/ (122)
    Apache/ (10)
      HTTPS-SSL/ (4)
      PHP/ (3)
      performance/ (2)
    Cherokee/ (1)
    LAN/ (4)
    LVM/ (6)
    Monitoring/ (2)
      munin/ (2)
    SSH/ (6)
    SSL/ (1)
    Samba/ (1)
    VPN-options/ (6)
      OpenVPN/ (1)
      SSH-Proxy/ (3)
      Tinc/ (1)
      sshuttle/ (1)
    backups/ (17)
      SpiderOak/ (1)
      backuppc/ (5)
      dirvish/ (1)
      misc/ (6)
      rdiff-backup/ (1)
      rsync/ (1)
      unison/ (2)
    commandLine/ (24)
      files/ (8)
      misc/ (10)
      network/ (6)
    crontab/ (1)
    databases/ (15)
      MSSQL/ (2)
      MySQL/ (8)
      Oracle/ (3)
      PostgreSQL/ (1)
    dynamicDNS/ (2)
    email/ (11)
      Dovecot/ (1)
      deliverability/ (1)
      misc/ (1)
      postfix/ (7)
      puppet/ (1)
    iptables/ (3)
    tripwire/ (1)
    virtualization/ (9)
      VMware/ (1)
      virtualBox/ (8)
  Coding/ (14)
    bash/ (1)
    gdb/ (1)
    git/ (3)
    php/ (5)
    python/ (4)
      Django/ (2)
  Education/ (1)
  Hosting/ (27)
    Amazon/ (18)
      EBS/ (3)
      EC2/ (10)
      S3/ (1)
      commandline/ (4)
    Godaddy/ (2)
    NearlyFreeSpeech/ (3)
    Rackspace/ (1)
    vpslink/ (3)
  Linux/ (30)
    Android/ (1)
    Awesome/ (3)
    CPUfreq/ (1)
    China/ (2)
    Debian/ (8)
      APT/ (3)
      WPA/ (1)
    audio/ (1)
    encryption/ (3)
    fonts/ (1)
    misc/ (6)
    remoteDesktop/ (1)
    router-bridge/ (3)
  SW/ (45)
    Micro$soft/ (1)
    browser/ (2)
      Chrome/ (1)
      Firefox/ (1)
    business/ (28)
      Drupal/ (9)
      KnowledgeTree/ (6)
      Redmine/ (2)
      SugarCRM/ (7)
      WebERP/ (2)
      WordPress/ (1)
      eGroupware/ (1)
    chat/ (1)
    email/ (1)
    fileSharing/ (2)
      btsync/ (1)
      mldonkey/ (1)
    graphics/ (2)
    research/ (2)
    website/ (6)
      blog/ (6)
        blosxom/ (3)
        rss2email/ (1)
        webgen/ (1)
  Security/ (15)
    IMchat/ (2)
    circumvention/ (2)
    cryptoCurrency/ (1)
    e-mail/ (4)
    greatFirewall/ (1)
    hacking/ (1)
    password/ (1)
    privacy/ (2)
    skype/ (1)
  Services/ (1)
    fileSharing/ (1)
  TechWriting/ (1)
  xHW/ (14)
    Lenovo/ (1)
    Motorola_A1200/ (2)
    Thinkpad_600e/ (1)
    Thinkpad_a21m/ (3)
    Thinkpad_i1300/ (1)
    Thinkpad_x24/ (1)
    USB_audio/ (1)
    scanner/ (1)
    wirelessCards/ (2)
  xLife/ (17)
    China/ (9)
      Beijing/ (5)
        OpenSource/ (3)
    Expatriation/ (1)
    Vietnam/ (7)

Archives:

  • 2016/07
  • 2016/05
  • 2016/02
  • 2016/01
  • 2015/12
  • 2015/11
  • 2015/06
  • 2015/01
  • 2014/12
  • 2014/11
  • 2014/10
  • 2014/09
  • 2014/07
  • 2014/04
  • 2014/02
  • 2014/01
  • 2013/12
  • 2013/10
  • 2013/08
  • 2013/07
  • 2013/06
  • 2013/05
  • 2013/04
  • 2013/02
  • 2013/01
  • 2012/12
  • 2012/10
  • 2012/09
  • 2012/08
  • 2012/07
  • 2012/06
  • 2012/05
  • 2012/04
  • 2012/03
  • 2012/01
  • 2011/12
  • 2011/11
  • 2011/10
  • 2011/09
  • 2011/08
  • 2011/07
  • 2011/06
  • 2011/05
  • 2011/04
  • 2011/02
  • 2010/12
  • 2010/11
  • 2010/10
  • 2010/09
  • 2010/08
  • 2010/07
  • 2010/06
  • 2010/05
  • 2010/04
  • 2010/03
  • 2010/02
  • 2010/01
  • 2009/12
  • 2009/11
  • 2009/10
  • 2009/09
  • 2009/08
  • 2009/07
  • 2009/06
  • 2009/05
  • 2009/04
  • 2009/03
  • 2009/02
  • 2009/01
  • 2008/12
  • 2008/11
  • 2008/10
  • 2008/09
  • Subscribe XML RSS Feed

    Creative Commons License
    This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
    PyBlosxom

    This site has no ads. To help with hosting, crypto donations are accepted:
    Bitcoin: 1JErV8ga9UY7wE8Bbf1KYsA5bkdh8n1Bxc
    Zcash: zcLYqtXYFEWHFtEfM6wg5eCV8frxWtZYkT8WyxvevzNC6SBgmqPS3tkg6nBarmzRzWYAurgs4ThkpkD5QgiSwxqoB7xrCxs

    Wed, 17 Feb 2016


    /Admin/VPN-options/Tinc: Simple Tinc VPN

    Tinc[1] is a rare animal, an actual peer-to-peer VPN that (for *NIX users) is easy to setup, not widely used and so (as far as I am aware) not blocked by anyone, including the GFW (Great Firewall of China).

    My main OS is Debian, so this example of a very simple tinc configuration will follow Debian standards in getting two Tinc VPN nodes talking to one another -- typically, one would be your Desktop, and the other would be a server with a public IP address running Squid.

    apt-get install tinc
    This is all that /etc/tinc contains after install:
    /etc/tinc/nets.boot

    Tinc can run multiple daemons, each handling a separate Tinc network on a separate subnet. To have each tinc network started automatically, simply add the network name to a list of same in nets.boot

    Each tinc network is represented by a separate directory under /etc/tinc/. Each Tinc network also requires a hosts subdirectory where the public keys for other peers in this network are held. For the simplest possible configuration, here are the main decisions to make:

    Let's first configure your desktop:

    Create the requisite directory structure:

    mkdir -p /etc/tinc/myvpn/hosts
    And create the two configuration files for this network:
    vi /etc/tinc/myvpn/tinc-up
    containing something like this
    modprobe tun
    ifconfig myvpn 10.99.3.1 netmask 255.255.0.0
    10.99.3.1 is the private tinc IP address of the node you are currently configuring. And
    vi /etc/tinc/myvpn/tinc.conf
    containing this:
    Name = mydesk
    Device = /dev/net/tun
    Port = 19001
    ConnectTo = myremote

    The Port line is optional, if omitted tinc will listen on the default port 655.

    Create your keys for the myvpn network (each separate tinc network/subnet has different keys) for the desktop node by running this on it:

    tincd -K -n myvpn
    (If things are correctly configured you should be able to just accept the defaults.) This is the pair of keys you just created:
    /etc/tinc/myvpn/rsa_key.priv
    /etc/tinc/myvpn/hosts/mydesk
    The former is your private key, the latter is your public key. Now edit the public key:
    vi /etc/tinc/myvpn/hosts/mydesk
    DO NOT modify the key, but add this config block ABOVE the key:
    Subnet = 10.9.3.1/32
    Address = x.x.x.x 19001

    The top line is the VPN private IP of the node, the bottom line is the real world (public, but not necessarily) IP and port where OTHER peers in this tinc network will find the machine. You will be sharing this file with all other peers in this network, and this config block tells them where to find this node.

    IMPORTANT, EASILY OVERLOOKED STEP: fix permissions:

    chown -R root: /etc/tinc/
    chmod a+rx /etc/tinc/myvpn/tinc-*
    tinc-* are scripts that must be executable, otherwise your configuration will subtly break. Now start tinc:
    systemctl start tinc.service

    If all goes well, ifconfig should show a myvpn device with IP 10.9.3.1.

    Configure the remote machine:

    It is the same as the above desktop config, with these exceptions:

    Putting it together:

    Once tinc is running on the server, copy the public tinc key of each machine into the tinc hosts directory of the other machine.

    Make sure port 19001 is open in the firewall on the myremote end.

    Restart tinc on both ends:

    systemctl restart tinc.service

    and you should now be able to ping the tinc IP of the other machine from both ends.

    This delivers to you a secure connection between desktop and a remote machine. If you would like to proxy browser traffic from mydesk through myremote, just install squid on myremote and enable connections from the tinc subnet. In your browser, set the proxy IP to the myremote tinc IP, port 3128 (default squid port), and select a proxy type of socks v5.

    [1] http://www.tinc-vpn.org/

    posted at: 08:17 | path: /Admin/VPN-options/Tinc | permanent link to this entry

    Sun, 07 Feb 2016


    /SW/business/WordPress: Drupal 6 to WordPress 4 Migration

    With many thanks to this[1] post for doing most of the work. Note that I have a simpler situation with only one user and no URL aliases.

    Prepare your freshly install WordPress 4.4 database by truncating default posts / pages / comments after installation.

    TRUNCATE TABLE wordpress.wp_comments;
    TRUNCATE TABLE wordpress.wp_links;
    TRUNCATE TABLE wordpress.wp_postmeta;
    TRUNCATE TABLE wordpress.wp_posts;
    TRUNCATE TABLE wordpress.wp_term_relationships;
    TRUNCATE TABLE wordpress.wp_term_taxonomy;
    TRUNCATE TABLE wordpress.wp_terms;

    Migrate tags:

    INSERT INTO wordpress.wp_terms (term_id, name, slug, term_group) SELECT d.tid, d.name, REPLACE(LOWER(d.name), ' ', '-'), 0 FROM drupal.term_data d INNER JOIN drupal.term_hierarchy h USING(tid);
    INSERT INTO wordpress.wp_term_taxonomy (term_id, taxonomy, description, parent) SELECT d.tid `term_id`, 'category' `taxonomy`, d.description `description`, h.parent `parent` FROM drupal.term_data d INNER JOIN drupal.term_hierarchy h USING(tid);

    Copy over Drupal posts to your WordPress database.

    INSERT INTO wordpress.wp_posts (id, post_author, post_date, post_content, post_title, post_excerpt, post_modified, post_type, post_status, to_ping, pinged, post_content_filtered) SELECT DISTINCT n.nid `id`, n.uid `post_author`, FROM_UNIXTIME(n.created) `post_date`, r.body `post_content`, n.title `post_title`, r.teaser `post_excerpt`, FROM_UNIXTIME(n.changed) `post_modified`, n.type `post_type`, IF(n.status = 1, 'publish', 'private') `post_status`, '', '', '' FROM drupal.node n, drupal.node_revisions r WHERE n.vid = r.vid;
    UPDATE wordpress.wp_posts SET post_type = 'post' WHERE post_type <> 'page' OR post_type <> 'post';

    Update post to tag / category relationship:

    INSERT INTO wordpress.wp_term_relationships (object_id, term_taxonomy_id) SELECT nid, tid FROM drupal.term_node;

    Update tags / category post count:

    UPDATE wordpress.wp_term_taxonomy tt SET `count` = (SELECT COUNT(tr.object_id) FROM wordpress.wp_term_relationships tr WHERE tr.term_taxonomy_id = tt.term_taxonomy_id);

    The following code is supposed to help fix taxonomy:

    UPDATE IGNORE wordpress.wp_term_relationships, wordpress.wp_term_taxonomy SET wordpress.wp_term_relationships.term_taxonomy_id = wordpress.wp_term_taxonomy.term_taxonomy_id WHERE wordpress.wp_term_relationships.term_taxonomy_id = wordpress.wp_term_taxonomy.term_id;

    Insert comments to posts:

    INSERT INTO wordpress.wp_comments (comment_post_ID, comment_date, comment_content, comment_parent, comment_author, comment_author_email, comment_author_url, comment_approved) SELECT DISTINCT nid, FROM_UNIXTIME(timestamp), comment, thread, name, mail, homepage, ((status + 1) % 2) FROM drupal.comments;

    Update post comments count:

    UPDATE wordpress.wp_posts SET `comment_count` = (SELECT COUNT(`comment_post_id`) FROM wordpress.wp_comments WHERE wordpress.wp_posts.`id` = wordpress.wp_comments.`comment_post_id`);

    You can avoid this step if you omit copying the teaser / post_excerpt post field. Or, after the fact:

    UPDATE wp_posts SET post_excerpt = NULL WHERE post_excerpt is not null;

    My tag category links did not display out of the box, and this fixed that:

    In backend --> Settings --> Permalinks set a custom value ("topics" for instance) for category base.

    And finally, the comments that I imported are not displaying properly yet. At this point I do not care enough to try to fix it, so caveat emptor.

    [1] http://www.jamediasolutions.com/blog/migrating-drupal-to-wordpress.html

    posted at: 03:49 | path: /SW/business/WordPress | permanent link to this entry

    Thu, 04 Feb 2016


    /Admin/databases/MySQL: Master-Slave Replication: First Steps

    MASTER CONFIGURATION:

    There are a lot of search engine hits for this subject, I liked Rackspace's contribution[1] the best personally. I am going to improvise off of that document. And I am going to assume there is an existing MySQL server that will be the master that we will replicate. First of all, the new master will need a 'slave' user:

    grant replication slave on *.* TO slave_user@'ip-address' identified by 'password';
    flush privileges;

    And on a Debian / Ubuntu server, make these changes to /etc/mysql/my.cnf:

    # bind-address = 127.0.0.1
    server-id = 1
    log_bin = /var/log/mysql/mysql-bin.log
    expire_logs_days = 15
    max_binlog_size = 200M
    binlog_ignore_db = mysql

    Then restart MySQL. We comment out bind-address to permit non-localhost connections to the MySQL master from the slave. Both master and slave need a defined server-id, and they need to be different. log_bin is where the master records transactions that the slave will later pickup. The rest should be self-explanatory, except to say that in my setup binlog_ignore_db seems to be ignored. I wish it was not, but so far no major consequences.

    COPY DATABASES:

    Before replication can be started, the databases on both ends need to be exactly the same. On the master:

    FLUSH TABLES WITH READ LOCK;
    SHOW MASTER STATUS;

    The first line puts all master databases into read-only mode, and the second line will print out the file name and position (an integer) at which the binlog's record of writes to the database stopped. It is very important to record these two values, as they will be needed later on the slave. Now dump all the databases except (optionally) mysql and (not optionally) information_schema and performance_schema (the latter two are internal MySQL things that do not replicate). First get a list of all databases:

    mysql -uroot -p -e 'show databases' --skip-column-names | tr '\n' ' '

    Edit the above list to remove mysql, information_schema and performance_schema, and then dump all databases:

    mysqldump -uroot -p --databases list-of-databases | gzip > alldbs.sql.gz

    Don't forget to release the read lock on the master and resume normal operation!!!:

    UNLOCK TABLES;

    Copy alldbs.sql.gz to the slave server.

    SLAVE SETUP:

    Install mysql-server on the slave, and make these changes to /etc/mysql/my.cnf:

    # bind-address = 127.0.0.1
    tmpdir = /var/tmp
    server-id = 2

    and restart MySQL. Allowing non-localhost connections on the slave is actually not necessary for replication, but will be necessary later for the tools we will use for integrity checking and repairs. The tmpdir must be preserved through reboots, so we have moved it from /tmp to /var/tmp (and installed tmpreaper to keep it clean). Now import the dump of databases from the master:

    zcat alldbs.sql.gz | mysql -uroot -p

    And start replication:

    CHANGE MASTER TO MASTER_HOST='master-ip-address', MASTER_USER='slave_user', MASTER_PASSWORD='password', MASTER_LOG_FILE='filename', MASTER_LOG_POS=123456;

    start slave;

    where MASTER_LOG_FILE and MASTER_LOG_POS are the values that you recorded on the master when you locked the databases and issued a "SHOW MASTER STATUS" command, and slave_user is the user you created earlier on the master. Now check replication status:

    show slave status\G

    The field of particular interest is Seconds_Behind_Master. If things are working properly that integer should become smaller quite rapidly as the slave catches up with the master. Eventually that integer should get down very close to zero, if not zero. I am almost always seeing zero with my setup.

    Something you will want to verify after master and slave are synced is a slave reboot. You should find that after a reboot Seconds_Behind_Master quickly returns to zero and replication continues uninterrupted.

    [1] http://www.rackspace.com/knowledge_center/article/set-up-mysql-master-slave-replication

    posted at: 00:48 | path: /Admin/databases/MySQL | permanent link to this entry