pi – Echoic Dev

Organizing A Photo Library With Rclone And Exiftool

I recently found out Google Photos makes it difficult to get metadata from photos. Also, they were using up all my cloud storage. So I decided to self-host my photo library. To get the original photos with metadata intact, I had to export them using Google Takeout. I had about 1 TB of Google Photos data which I had to download in 50 GB chunks. If you’re downloading to a flash drive, there’s a good chance it’s formatted using FAT which means you’ll need to download them in 4GB chunks. 4GB * 250 downloads is quite a bit and Google makes you download each archive manually. So yeah, don’t use Google Photos as a photo backup solution. It’s free and the search tools are pretty good, though.

Once I downloaded all the archives, I deleted any duplicates using rclone. I chose to dedupe by hash and automatically delete every file except for the first result. The first result may not have the name I want, but I’m going to use exiftool to rename everything later. I redirect output to a log file in case I need to analyze things later.

rclone dedupe --by-hash --dedupe-mode first --no-console --log-file ../rclone.log .

Next, I organized everything into directories by date using exiftool. If the picture had metadata containing the date the picture was taken, I used that. If not, I used the time the file was modified. I redirected output to a log file so I can address things like duplicate files later.

exiftool -r -d %Y/%m/%d "-directory<filemodifydate" "-directory<createdate" "-directory<datetimeoriginal" -overwrite_original . 1> ../exiftool.log 2>&1

I’m trying to think of some other identifying information to add to the directory names, like the location or activity or event. Location is pretty easy to get in a parseable format.

exiftool -gpslatitude -gpslongitude -n -json . | jq

Then I just gotta reverse geocode it.

I’d also like to do some deduplication using an image similarity detector. Facebook open-sourced a tool that looks like it might work.

Troubleshooting `error creating vxlan interface: operation not supported` on Docker on Ubuntu on Raspberry Pi

I recently did a big update on my home server and noticed my containerized services weren’t starting up. First, I looked into the Portainer service:

docker stack ps portainer --no-trunc

After searching around the internet, I found this advice to install a package.

sudo apt-get update && sudo apt-get install linux-modules-extra-raspi

After that, I got a scary warning about how the new kernel wouldn’t be loaded automatically so I should reboot.

sudo reboot now

Things still weren’t working after I rebooted, but restarting Docker seemed to kick Portainer into motion.

sudo systemctl restart docker

How to set up VNC on Ubuntu on Raspberry Pi

Credit: This is more or less this DigitalOcean post with my own notes thrown in.

I don’t like using VNC on my Pi because of the hefty resource requirements for a desktop environment, but I wanted to get a G Photos token for rclone so I could sync off a bunch of 360 videos eating up all my G Drive space, and I needed a graphical web browser to do that.

1. Install XFCE

This is only required if you don’t already have a desktop environment. You can also use another desktop environment, but XFCE is pretty lightweight.

sudo apt update sudo apt install xfce4 xfce4-goodies

When you’re prompted to do so, pick gdm3 or lightdm display manager. I picked lightdm because it uses less resources. Here’s an article comparing the display managers.

Side note:

During installation, I got some errors.

Errors were encountered while processing: avahi-daemon libnss-mdns:arm64 avahi-utils

I found this article, but it didn’t help me. So I uninstalled and reinstalled everything thinking I could select gdm3 maybe not encounter the same issues, but it didn’t prompt me, and since it worked anyway, I didn’t bother figuring it out. If you know why those errors occurred, please leave a comment.

2. Install and configure TightVNC Server

You could use another VNC server, but TightVNC works for me. Also, there’s a PortableApps version of the client which I can install on a USB stick and use on any Windows machine. Anyways, install the server.

sudo apt update sudo apt install tightvncserver

Run the server for the first time. This will initialize its configuration.

vncserver

You should be prompted to enter a password and get some output like:

You will require a password to access your desktops.

Password: 
Verify:
Would you like to enter a view-only password (y/n)? n
xauth:  file /home/ubuntu/.Xauthority does not exist

New 'X' desktop is localhost:1

Creating default startup script /home/ubuntu/.vnc/xstartup
Starting applications specified in /home/ubuntu/.vnc/xstartup
Log file is /home/ubuntu/.vnc/localhost:1.log

Notice that it created a startup script ~/.vnc/xstartup. You’ll need to edit that file to get XFCE to work, but shut down the server first.

vncserver -kill :1

Back up the original ~/.vnc/xstartup.

mv ~/.vnc/xstartup ~/.vnc/xstartup.bak

Create a new ~/.vnc/xstartup with the following contents:

#!/bin/bash xrdb $HOME/.Xresources startxfce4 &

Make it executable.

chmod +x ~/.vnc/xstartup

The VNC server should work now, but VNC traffic is unencrypted, so if you care about people being able to snoop on what you’re typing (like passwords!) and whatever else you’re doing on your VNC connection, you should set up a SSH tunnel to encrypt your traffic.

3. Run the server securely

Run the VNC server such that it only accepts local connections.

vncserver -localhost

4. Securely connect to the server

Set up a SSH tunnel from your local machine to the remote server. This is how you’ll make the VNC connection “locally”.

On your local machine (the VNC client), run the following command, replacing <remote user> with your username on the server and <remote host> with the server’s hostname or IP address. You can also change 59001 to a different port if you want, but make the port isn’t already being used by another service and it’s above 1024.

ssh -L 59001:localhost:5901 -C -N -l <remote user> <remote host>

On your VNC client, connect to localhost:59001 (or whatever port you used) and you should see your Pi desktop.

That’s it, but if it doesn’t work for you, I’d be interested to hear about it in the comments.