Home Automation DevOps

Home automation is coming into the mainstream a with recent offerings from the big tech companies, and I'm interested in it too. I've had a number of home automation controllers, most recently an H3 Pro SEL from HomeSeer.

I've also become quite interested recently in DevOps monitoring, using InfluxDB, and GrafanaSensu, Telegraf and Prometheus are great too, but not for this project.

So, I decided to import home status information into InfluxDB and graph it. The first step was getting the information from HomeSeer.   There is a JSON-over-HTTP API, but no existing Java client.  So, a simple HomeSeer H3 API client in Java was written: hsclient

The next step was getting the data into influx.  The best way to do this would be to write a Telegraf plugin.  However, I wanted a chance to learn a little more about Java closures and lambas, so I wrote hsinflux a threaded command-line app which connects to HomeSeer on a schedule, downloads the device status data and drops it into InfluxDB. hsinflux is here.

The net result, with Grafana, is the ability to see my house temperatures, humidity, door and window status, and thermostats.  Here are some examples.

House temperatures this week on 5 minute intervals

At the time of writing, its -17C outside, and my server room temperature is showing a steady decrease, down to -6C. The increase on the far right is the result of opening the server room door to get some warm air from the house.

Dimmer Status

And since I have a TSDB, why not record the time it's taking to collect data from HomeSeer?

Since I can, I'm also querying battery charge on the devices that have batteries. It's not an interesting graph, but down the road I can use Grafana to alert me when the batteries need to be changed.

Overall, a great opportunity to become familiar with the technologies. Once the data is in Influx, building arbitrary graphs to visualize data is both simple and powerful. The graphs I've built are simple, however for IT professionals wanting insight into complex infrastructure, tools such as Influx and Grafana represent a step change in visibility and analysis.

From an operational perspective, TSDB's give DevOps engineers an opportunity to answer a question we've been trying to answer for decades:

What changed the hour before it crashed? The week before it crashed? The month before it crashed?

FreeBSD on AWS with Ansible

I often have a need for a fresh FreeBSD install, to host a server, or perhaps just experiment with. My usual solution is FreeBSD jails. However, I've recently gotten interested in Ansible and AWS.

There are quite a few advantages to installing infra with AWS, however the one I'm most interested in is automating the simple stuff and getting it right the first time. For example, remembering to:

  • Add the service accounts
  • Ensure that all files are added by the right accounts
  • Add startup and shutdown scripts
  • Configure pf
  • Ensure logging is "just right"

AWS also has the notable advantage that with VPC, I can configure not only the OS and applications, but build infra.

With the help of these two pages (1,2) I've been able to provision a simple FreeBSD-12 host, in a VPC. Code is here.

K-REPL

A couple projects I've been working on have a need for a REPL style console.  I had written a simple framework for creating REPL's, which is open-sourced here.

 

Quadrigacx API

I've recently become very interested in blockchain, and that, naturally led me to Ethereum and Bitcoin.  From there, I got a little interested in online trading, but since I prefer not to have to think about trading, I started thinking about a trading bot.  Part of the work I needed to do was write a Java API for Qaudrigacx.  The API is here and is free to use under the GPL v3 terms.

 

FreeBSD Jail filesystems

I've been using FreeBSD jails for a while, and was looking for a simple way to create jail filesystems.   After some reading, I came up with this simple script which creates full FreeBSD filesystems.   The filesystem can then be used via jail.conf.

#!/bin/bash
JAILNAME=$1
echo "kjail "$1
JAIL_HOME=/tank/kjail/
FTP=http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/11.1-RELEASE/
# 
# get packages
#
if [ ! -f $JAIL_HOME/packages/doc.txz ]; then
        wget $FTP/doc.txz -O $JAIL_HOME/packages/doc.txz
        wget $FTP/base.txz -O $JAIL_HOME/packages/base.txz
        wget $FTP/kernel.txz -O $JAIL_HOME/packages/kernel.txz
        wget $FTP/lib32.txz -O $JAIL_HOME/packages/lib32.txz
fi
#
# make jail
#
mkdir -p $JAIL_HOME/$JAILNAME
tar zxvf $JAIL_HOME/packages/doc.txz -C $JAIL_HOME/$JAILNAME
tar zxvf $JAIL_HOME/packages/base.txz -C $JAIL_HOME/$JAILNAME
tar zxvf $JAIL_HOME/packages/kernel.txz -C $JAIL_HOME/$JAILNAME
tar zxvf $JAIL_HOME/packages/lib32.txz -C $JAIL_HOME/$JAILNAME

 

Cross-Compiler fun

I've been interested in OS development for a while, and now have a prototypical ARM OS on my private source tree.  For that I used gcc-arm-embedded, which worked quite well.  However, as time went on, i became interested in building my own tool chains.  I started with this list of requirements

  • An up-to-date C/C++ compiler, such as gcc or clang
  • I prefer gmake
  • On i386 I like nasm, but I'll use gas if necessary
  • I like ld, objcopy, etc from binutils
  • The targets I prefer to cross compile from are OS X and FreeBSD.  I prefer to cross-compile to i386, arm, sparc and mips targets

I ended up with two shell scripts, one for GCC/Binutils/Make and one for Clang/Binutils/Make, which download and install the latest GCC/Clang/Binutils and create a fully operational toolchain for the target of my choice.  You can find them here:

 

IPSec VPNs with pfsense

I've been trying forever to get a mobile IPSec connection up between my OS X laptop and pfsense.   Finally, thanks to this outstanding blog post, it works.  I'm especially excited that it works with the default OS X and Android VPN clients.

My pfsense config closely mirrors the one specified by Mike Murray

pfsense

Phase 1

Key Exchange version: Auto
Internet Protocol: V4
Interface: <my ISP>

Authentication Method: Mutual PSK + Xauth
Negotiation Mode: Aggressive
My identifier: My IP Address
Peer identifier Distinguished Name: <my vpn name>
Pre-Shared Key: <my key>

Encryption Algorithm: AES 256
Hash Algorithm: SHA1
DH Group: 2
Lifetime (Seconds): 28800

Disable rekey: unchecked
Responder Only: checked
NAT Traversal: Forced
Dead Peer Detection: checked
Delay: 10
Max failures: 5

Phase 2

Mode Tunnel: IPv4
Local Network Network: 0.0.0.0/0
NAT/BINAT translation: None

Protocol: ESP
Encryption Algorithms: AES Auto
Hash Algorithms: SHA1
PFS key: group off
Lifetime: 3600

Mobile Cilents

IKE Extensions: checked
User Authentication: Local Database
Group Authentication: system

Virtual Address Pool: checked.  192.168.76.0 / 27

Virtual IPv6 Address Pool: unchecked
Network List: unchecked
Save Xauth Password checked
DNS Default Domain: checked  khubla.local

Split DNS unchecked
DNS Servers checked
192.168.75.1
8.8.8.8.

WINS Servers: unchecked
Phase2 PFS Group: unchecked
Login Banner unchecked

OS X

Add VPN and choose VPN Type "Cisco IPSEC".  Use the Group Name specified in Phase 1 "Peer identifier Distinguished Name"

Android

Add VPN and choose "IPSec VPN with pre-shared keys and XAuth authentication".     Use the IPSEC Identifier specified in Phase 1 "Peer identifier Distinguished Name"

Configuring the FreeBSD automounter

I have a number of disks in my FreeBSD box which exist only as backups of data.  I prefer that they not be mounted all the time, but instead mounted on use.  They are 2x internal SATA disks and an external USB3 disk.   This blog post explains how they are set up to auto-mount.

Firstly, my /etc/fstab entry for these disks looks like this:

/dev/ada5p1     /mnt/backup1    ufs     rw,noauto,noexec,nosuid,late            0       0
/dev/ada6p1     /mnt/backup2    ufs     rw,noauto,noexec,nosuid,late            0       0
/dev/da0p1      /mnt/usb1       ufs     rw,noauto,noexec,nosuid,late            0       0

All three disks are ufs, and noauto.

Next, my /etc/amd.conf

[ global ]
restart_mounts =   yes
unmount_on_exit=  yes

and my amd.map

# $FreeBSD: releng/10.3/etc/amd.map 164015 2006-11-06 01:42:11Z obrien $
#
*               opts:=rw,grpid,resvport,vers=3,proto=tcp,nosuid,nodev
localhost            type:=auto;fs:=${map};pref:=${key}/
localhost/backup1      type:=program;fs:=/mnt/backup1;\
                        mount:="/sbin/mount mount /mnt/backup1";\
                        unmount:="/sbin/umount umount /mnt/backup1"
localhost/backup2      type:=program;fs:=/mnt/backup2;\
                        mount:="/sbin/mount mount /mnt/backup2";\
                        unmount:="/sbin/umount umount /mnt/backup2"
localhost/usb1      type:=program;fs:=/mnt/usb1;\
                        mount:="/sbin/mount mount /mnt/usb1";\
                        unmount:="/sbin/umount umount /mnt/usb1"

In /etc/rc.conf, we need to start amd, and the portmapper:

portmap_enable="YES"
portmap_flags="-h 127.0.0.1"
amd_enable="YES"
amd_flags="-a /.amd_mnt -l /var/log/amd /host /etc/amd.map"