I have a few monthly digital magazine subscriptions that I have been getting for years. I have saved and aggregated on my hard disk. Recently, I lost a chunk of these due to my mistake at the terminal. As a precaution, I had these sync to my cloud storage space on hubic (Offer 25GB of storage space + 50 with 5 referrals). I thought I hand a second copy secured. However, as I logged in to the hubic web interface, I saw files vanishing right before my eyes. The hubic client program syncing between my computer and the cloud was sending instructions to get rid of files that were no longer on the computer. By the time I could log in via SSH and terminate the client program, it was too late. Unlike Dropbox, hubic does not keep deleted files to undelete. Years of aggregated magazine subscriptions lost in minutes.
This made me see the necessity to have offline backups.
One easy way was to set
crontasks to periodically copy certain folders to an external drive via
rsync. However, leaving an external drive connected does turns this "offline" approach into "online". I felt, it would be ideal if I could plug in my drive into my NAS whenever, I wanted and this would cause a backup job to start running. I looked around and discovered that I could use
udev, but reading more, I decided I did not want to dive into
I finally settled on an idea, where I plugin my hard disk into the NAS whenever, I wanted to back up and that night, the selected folders would back up to the hard drive. The next morning, I could unplug the disk and voila, I have my little offline backup system.
I set about implementing this using a shell script that would run every night and check, if my external USB backup, is plugged in or not. If found, it would initiate a backup sequence. Here is the shell script that I used -
#! /bin/bash PART_ID="926fc38e-bc7e-467a-953e-25c6555939a0" now=$(date) echo "===== Running at $now =====" if [ -b /dev/disk/by-uuid/"$PART_ID" ]; then echo "Partition Found!" mkdir ~/$PART_ID mount $PART_ID if [ $? -eq 0 ]; then echo "Mounted" # Run backups rsync -r "/home/bharath/hubic/Datasheets" ~/$PART_ID/. rsync -r "/home/bharath/hubic/Harvard Business Review" ~/$PART_ID/. rsync -r "/home/bharath/hubic/IEEE Computer Magazine" ~/$PART_ID/. rsync -r "/home/bharath/hubic/Linux Journal" ~/$PART_ID/. rsync -r "/home/bharath/hubic/MagPi" ~/$PART_ID/. rsync -r "/home/bharath/hubic/Make Magazine" ~/$PART_ID/. rsync -r "/home/bharath/Photos" ~/$PART_ID/. rsync -rl "/home/bharath/Datasets" ~/$PART_ID/. sync umount $PART_ID pushb "Offline backup done." if [ $? -eq 0 ]; then echo "Unmounted" rm -Rf ~/$PART_ID now=$(date) echo "===== Done at $now =====" fi fi else echo ":-(" fi
Here is the gist containing the code.
To have this script work, I had to add a line allowing users to mount the disk in the
UUID=926fc38e-bc7e-467a-953e-25c6555939a0 /home/bharath/926fe38e-bc7e-467a-959e-25c6535939f0 ext4 noauto,noatime,nodiratime,relatime,user 0 1
Apart from this, I had to set the user
rwxpermissions on the entire disk. A big problem with doing this is that If, I move the disk to a different computer - one that does not have the user
bharath(me!) created with the user id of 1000, someone else, who has
UID=1000would have the
rwxpermissions. This is something, I would have to live with till I figure a way around this or write my own portable journaling file system.
To perform a nightly check, I would have to add something like the following line to my crontab -
0 4 * * * /home/bharath/do_offline_backup.sh >> offline_backup.log
On a side note, comparing hubic to Dropbox, I see hubic having a huge price advantage -
hubic Dropbox AWS S3 (Oregon) €10/year (100 GB) $100/year (1TB) $370/year (1TB) €50/year (10 TB) + Data transfer cost
* 2016 October Prices
And hubic servers are hosted in France, we can reasonably hope that this in out of reach of the various 3 letter government agencies of the US.
The downside of hubic are -
- Speed - it is slightly slower than dropbox.
- UI - not as great a UI as dropbox.
- Versioning - no versioning or undelete.
- Dropbox paper - no on website editors with hubic.
If all you want to do is have an online cloud backup, this is the service for you.
Recently I was working on building myself a script to download my securities transactions from Robinhood and save it as a CSV. I found an unofficial python library that was reverse engineered from the app. As I started building the script, I realized that I could not host it on git. I had hardcoded my credentials in the code.
Yahoo Open Data Tables is probably the last free source of financial information available after Google shut down it's finance API a few years back. In a recent project to analyze my stocks trades, I decided to cache 10 year historical prices form the Yahoo Open Data Tables.
I have been using cygwin for a while. To say the least, I get to do X11 forwarding on windows. Windows 10 now has an experimental feature called Bash for Windows. I think this was released along with the Windows 10 Anniversary update. However, I have not succeeded in getting X11 to run on it.
Occasionally, something I do at the terminal causes the terminal to misbehave at the end of the output. This usually happens when I display a binary file. For example -
subscribe via RSS