Data Synchronization over an Unreliable Connection
I Designed a system to keep terabytes worth of multi-gigabyte files (a large HD movie collection) synchronized between three homes. One of the homes has an unreliable and slow internet connection. Slow internet connection made streaming video quality over the internet unacceptable, requiring a local copy of the entire library. I Originally used NAS storage and synchronized remote drives by mounting them over internet. This proved unreliable and tedious. Mounting NAS drives over a slow connection often encountered timeouts when transferring large files. I Eventually settled on recycling three old laptops (one in each home) and attaching large HDDs. The laptops use Bittorrent Sync to keep each local library of files in sync with the other two. Bittorrent Sync can reliably copy large files even over a slow connection. Laptops have scheduled bootup and shutdown to keep energy costs down. Each computer also runs a Plex server which automatically updates its library whenever file changes are detected. Plex also adds relevant metadata to the video files. Plex can serve video files to a wide variety of devices over LAN, and can transcoding the video files to compatible formats. The home with the best internet connection also has an external Plex server, which allows devices outside of LAN (such as a cell phone) to stream video files. In total, this setup allows for instant playback of HD videos from a large, constantly up-to-date terabyte-size library on any of 8 Laptops (5 Mac, 3 Windows), 4 Android Phones, 3 Chromecasts, 2 Google TVs, 2 Apple TVs, 2 iPads, and an Xbox 360 spread across three homes (one of which with poor connection speed) and with access to videos anywhere via a cell network.