The devices used for smart home implementation usually have limited hardware resources, computing power and memory. This makes it very difficult or impossible to implement complex algorithms that ensure security and management.
From my point of view the main design assumptions of centralized smart home management system includes:
All the requirements mentioned above make the construction of hardware for smart home management exciting. But let's make it more exciting! I genuinely believe that the measure of technical improvements is related to the level how an engineer is respecting own free time. The improvement of home to be smart are bring about a lot of devices. All these devices are requiring support. Thinking over I came up with the idea to consolidate the hardware for centralized smart home management with home media center. Maybe it sounds strange but it isn't!
The main requirements for both hardware’s are:
We live in era of a data-driven culture. When we need data, we need them right away, whether it is last month’s electricity consumption or photos from last year's vacation. At both case the easy accessibility is desirable.
Writing to two or more disks at the same time. Having the same data stored on separate disks enables the data to be recovered in the event of a disk failure without resorting to expensive data recovery techniques.
User management describes the ability for us to manage access to various data such as media content like movies, family albums or smart home applications and more. User management is a core part to any service and is a basic security essential for any home (home as organization).
Looking ahead there are some additional, technical, requirements:
So, the task is given let's see how I solved it!
Considering the above presented requirements such as easy maintenance and reproducibility of hardware also taking price and compute power into account, I decided to buy a used HPE MicroServer. The compatibility, power and possibilities to expand the devices are impressed me.
As first step I removed the physical RAID card because any special hardware is not acceptable. Also, the requirement for file system are forcing solution without physical controller. I decided to use ZFS.
ZFS protects data by enabling volume management on filesystem level. This feature makes “Copy on Write” technology possible. When a block of data is altered, it will change its current location on the disk before the new write is finished. If your system crashes or loses power in the process, that data would be lost or damaged. ZFS does not change the location of the data until the write is completed and verified, thus keeping your data safe in case of a system crash. To verify data integrity, ZFS uses checksums to ensure that the data remains original from write to write.
ZFS also allows to send writes to individual physical disks, instead of just the RAID volume. Because of this, ZFS can stripe writes across RAID volumes, which is speeding up write performance. In the case you need to sync mirrors with only a little bit of information, you do not have to wait for it to sync any of the empty disk space, which can take a good amount of time. ZFS incorporates algorithms to ensure that your most recently used and most frequently used data are kept in the fastest system storage media. Spinning disks are known to be slow and SSD drives come at a very high price compared to regular disks. By using these algorithms in combination with flash-based ZFS write cache and L2ARC read cache devices, you can speed up your performance. Other great feature of ZFS are the intelligently designed snapshot, clone, and replication functions. ZFS snapshots only update based on what has changed since the last snapshot. This means that clone and replication tasks are less time consuming compared to traditional replication technology.
A configured ZFS pool can easily be changed in its size to accommodate a growing need for more storage. The pool can be upgraded step by step with larger disks, without compromising the filesystem or complicated procedures. Hard disks can even be added on different physical ports or in a changed order in a new computer system, as long as the ZFS version on the target system is the same or higher. The current rule of thumb when making a ZFS RAID is:
The HPE MicroServer contains four slots for SATA drives. But there are some more possibilities such as place for optical drive and a hidden USB 3.0 port on motherboard. It means there are six places for drives. The main details of hardware in my case are:
Considering the rule of thumb for ZFS RAID and the possibility to speed up the read, I made following decision:
The core system is installed on SSD in 1st slot and it is mirrored with SSD which is connected over USB (yellow drives). I know, over USB the boot is slower, but my opinion is that the slower boot is better than lose time when the main SSD breaks.
Using whole physical disks is the recommended way to create ZFS storage pools. But I think it is expensive to waste an entire SSD for logs and cache. Before building of NAS I made some tests and ZFS worked fine in case when I created partitions. So, on both 'blue' SSDs I created the below described three partitions:
The sizing of ZIL and L2ARC partitions are an exciting tech challenge. After studying the documentations of ZFS I made following decision:
The partitions for ZIL are mirrored. It’s given more security. The partitions for L2ARC are striped. So, it’s meant the full size of cache in my case is 80GB. The remaining part of volume is mirrored (60GB). Later this part will be dedicated for services such as Plex and OpenHUB. In ZFS, you can enable compression at the filesystem level. This will store the data in compressed format, which will save lot of disk space. LZ4 provides the best compression with the lowest CPU overhead. In my case the partitions of both “blue” drives are:
Creation of ZFS pool for storage is similar than it was for compute. The only difference is that ZIL and L2ARC are added to improve the performance. This is the storage of NAS it is named as ”storage”:
Verifying the installation, you will see a similar output with following ZFS pools:
The above presented architecture has a redundant storage with improved read performance. In case of emergency both “red” data disks could be removed and easily attached to any other hardware while keeping your data completely secure.
The storage pool named as “compute” has no improved read performance. It does not need it because it used for containers such as Jail and Docker. This is the place where the services for smart home and media center are live.
The main OS is FreeNAS. It is FreeBSD based system. FreeNAS is a free and open source network attached storage (NAS) software appliance. This means that you can share data over file-based sharing protocols, including CIFS, NFS and AFP. FreeNAS uses the ZFS file system to store, manage, and protect data.
FreeNAS has capability to install additional components for other purposes, such as media streaming servers (Plex) or platform to run containers (Rancher) and cloud backup services. Also, it has capability to create virtual machines. So, it can fulfil many roles of a home server as well. I take advantage of these roles for implementation of centralized management solution for smart home.
Rancher implements a portable layer of infrastructure services designed specifically to power containerized applications. Rancher infrastructure services include networking, storage, load balancer, DNS, and security. Rancher infrastructure services are typically deployed as containers themselves.
The Jail mechanism is an implementation of FreeBSD's OS-level virtualisation that allows partitioning a FreeBSD-derived computer system into several independent mini-systems called jails, all sharing the same kernel, with very little overhead.
I prefer containers instead of virtual machines. For home management platform I choose Docker based solutions. OpenHUB is implemented as centralized management solution of smart home devices. OpenHUB is an open source home automation platform. It is vendor and technology agnostic. It is developed in Java, which gives you endless devices where you can run the system. Its architecture is based on bindings that bring support for different smart home devices.
Also, inside of Rancher I deployed Traefik and Shinobi. Traefik is an open-source Edge Router that makes publishing your services on Internet easy experience. It receives requests on behalf of your system and finds out which components are responsible for handling them. Further advantage of Traefik is that use ACME provider such as Let's Encrypt for automatic certificate generation. Shinobi is open source, written in Node.js, and really easy to use. It is a powerful solution for CCTV and NVR. The security recordings are saved on the storage pool.
Regarding the media server I solved it with FreeNAS plugin for Plex. It runs inside a Jail container on compute pool and over NFS are reaching the media content on storage pool. The reason why media server is in Jail is that my goal was to totally separate the two functionalities of each other. It made improvement of solutions for smart home more convenient.
Keep Calm and Enjoy
The first benefit of using a NAS is that all your important files can be stored in a central location, allowing you to access them from multiple devices simultaneously, while also keeping your backup and redundancy resources in one place.
The possibilities of HPE MicroServer made the maintenance of physical hardware easy and effective. The management port and remote console access are huge advantage. It means that hardware maintenance is possible over network from any place of the World. So, the upgrade of core system could be performed from beach as well.
ZFS are doing a great job regarding data security. ZFS natively manages both disks and volumes, with solutions similar to those provided by hardware RAID technologies. Integrity and reliability are the pillars of the ZFS. Snapshots, which are natively available, are a very useful feature as well.
The upgrade of FreeNAS is also work out of the box. The whole process is well thought out and work as a charm. The architecture design, the separated pool for system, the improved read performance and the mirrored data disks are further improving system reliability. So, the time required for maintenance can be minimized and planned. Thanks to the architecture the continuous availability of services is guaranteed. The management with endpoints and users is also handled on efficient way. There are bunch of possibilities to grant access to different data and services on the secure way.
And finally, the whole system is built with open source solutions without any special device. In case that you need to rebuild the system, you can solve it on cost efficient way without any dependency.