diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/backups.org | 187 | ||||
| -rw-r--r-- | docs/desktop.org | 19 | ||||
| -rw-r--r-- | docs/gcloud.org | 21 | ||||
| -rw-r--r-- | docs/install.org | 153 | ||||
| -rw-r--r-- | docs/secrets.org | 29 | ||||
| -rw-r--r-- | docs/tools.org | 167 | ||||
| -rw-r--r-- | docs/wireguard.org | 21 |
7 files changed, 0 insertions, 597 deletions
diff --git a/docs/backups.org b/docs/backups.org deleted file mode 100644 index a1db502..0000000 --- a/docs/backups.org +++ /dev/null @@ -1,187 +0,0 @@ -#+TITLE: Backups - -Each host can be configured to store a backup on the NAS using restic. The backups are synchronized once a day to rsync.net. - -* restic -For backups I'm using [[https://restic.readthedocs.io/][restic]]. - -On the NAS itself, we backup the git repositories to =/data/backups=. - -The password is stored in =/etc/restic/password= (this is not managed by puppet for now, and the password is stored within 1password). -** List the snapshots -To get a list of snapshots: -#+BEGIN_SRC sh :dir /ssh:nas: :results verbatim -sudo restic -r /data/backups/ -p /etc/restic/password snapshots -#+END_SRC - -#+RESULTS: -#+begin_example -repository a37cfab5 opened successfully, password is correct -ID Time Host Tags Paths ---------------------------------------------------------------------------------- -e36e9100 2020-02-29 08:43:37 nas /home/git/repositories -603a46a7 2020-03-31 08:39:03 nas /home/git/repositories -e890453b 2020-04-30 08:22:37 nas /home/git/repositories -0affa4d9 2020-05-10 08:47:18 nas /home/git/repositories -a01d8be4 2020-07-31 08:41:25 nas /home/git/repositories -78afb27a 2020-08-31 08:23:52 nas /home/git/repositories -68a417b1 2020-09-30 08:44:49 nas /home/git/repositories -ac6701b4 2020-10-18 06:00:00 nas git /home/git/repositories -4f183431 2020-10-25 06:00:00 nas git /home/git/repositories -aec0b472 2020-10-25 07:24:10 aptos home /home/fcuny -3e98a872 2020-10-30 06:00:00 nas git /home/git/repositories -0268f733 2020-10-31 06:00:00 nas git /home/git/repositories -1b840de3 2020-11-01 06:00:00 nas git /home/git/repositories -2d224944 2020-11-02 06:00:00 nas git /home/git/repositories -fa0107dd 2020-11-03 06:00:00 nas git /home/git/repositories -1165032b 2020-11-04 06:00:00 nas git /home/git/repositories -612b66e3 2020-11-05 06:00:00 nas git /home/git/repositories -2de6fb79 2020-12-31 06:01:19 nas gitea /data/containers/gitea -ece08207 2020-12-31 06:01:41 nas traefik /data/containers/traefik -d59bd75a 2020-12-31 06:06:19 nas grafana /data/containers/grafana -168c0ddf 2020-12-31 06:07:24 nas unifi /data/containers/unifi -5882ffe4 2021-01-27 18:58:06 aptos home /home/fcuny -3565b23b 2021-01-31 06:05:18 nas traefik /data/containers/traefik -653d4411 2021-01-31 06:14:12 nas gitea /data/containers/gitea -38a3e50e 2021-01-31 06:15:13 nas unifi /data/containers/unifi -542e2c80 2021-01-31 06:15:13 nas grafana /data/containers/grafana -8c804805 2021-02-06 19:13:24 aptos home /home/fcuny -3f38d369 2021-02-28 06:03:28 nas grafana /data/containers/grafana -ef2042e2 2021-02-28 06:11:50 nas unifi /data/containers/unifi -b429ef99 2021-02-28 06:18:02 nas gitea /data/containers/gitea -b73f5128 2021-02-28 06:18:04 nas traefik /data/containers/traefik -7a7e3e06 2021-03-28 09:05:35 aptos home /home/fcuny -3a0c790f 2021-03-30 06:12:20 nas grafana /data/containers/grafana -58179a2f 2021-03-31 06:05:04 nas gitea /data/containers/gitea -fc4ede5d 2021-03-31 06:08:18 nas unifi /data/containers/unifi -5eaa5148 2021-03-31 06:17:13 nas traefik /data/containers/traefik -d7c95e53 2021-04-27 18:10:36 aptos home /home/fcuny -4c702501 2021-04-30 06:02:11 nas gitea /data/containers/gitea -8de29c3c 2021-04-30 06:04:42 nas unifi /data/containers/unifi -66664254 2021-04-30 06:08:25 nas traefik /data/containers/traefik -9a3ad896 2021-04-30 06:15:15 nas grafana /data/containers/grafana -344ef4c3 2021-05-15 14:22:05 aptos home /home/fcuny -6141b888 2021-05-30 06:14:37 nas traefik /data/containers/traefik -106c4819 2021-05-31 06:04:56 nas grafana /data/containers/grafana -8e0ba4c3 2021-05-31 06:12:37 nas gitea /data/containers/gitea -8cba7fbf 2021-05-31 06:17:26 nas unifi /data/containers/unifi -2cc04ad6 2021-06-28 17:08:25 aptos home /home/fcuny -8b04e195 2021-06-30 06:03:56 nas grafana /data/containers/grafana -d21a464f 2021-06-30 06:09:56 nas unifi /data/containers/unifi -f180e1a0 2021-06-30 06:10:20 nas gitea /data/containers/gitea -b9e0ce43 2021-06-30 06:11:50 nas traefik /data/containers/traefik -512e80fb 2021-07-23 17:25:45 aptos home /home/fcuny -28b32d1f 2021-07-31 06:03:50 nas gitea /data/containers/gitea -884574c8 2021-07-31 06:11:13 nas unifi /data/containers/unifi -a61cd90f 2021-07-31 06:16:50 nas grafana /data/containers/grafana -614f9123 2021-07-31 06:19:38 nas traefik /data/containers/traefik -17698a8a 2021-08-14 06:05:34 nas git /data/containers/git -b5674e76 2021-08-16 13:47:52 aptos home /home/fcuny -d7c251f6 2021-08-31 06:16:07 nas gitea /data/containers/gitea -ef20f101 2021-08-31 06:16:11 nas unifi /data/containers/unifi -b7cd0d5c 2021-08-31 06:16:16 nas grafana /data/containers/grafana -facffc9a 2021-08-31 06:16:19 nas traefik /data/containers/traefik -b2d31938 2021-08-31 06:16:22 nas syncthing /data/containers/syncthing -8ab3bee2 2021-09-27 10:35:27 aptos home /home/fcuny -1559f48c 2021-09-30 04:11:21 nas gitea /data/containers/gitea -353d202d 2021-09-30 04:11:25 nas unifi /data/containers/unifi -b567fec1 2021-09-30 04:11:30 nas grafana /data/containers/grafana -d7b239c1 2021-09-30 04:11:33 nas traefik /data/containers/traefik -4890d748 2021-09-30 04:11:35 nas syncthing /data/containers/syncthing -4d6b6646 2021-10-31 04:11:55 nas gitea /data/containers/gitea -b2820465 2021-10-31 04:12:01 nas unifi /data/containers/unifi -cd2230ff 2021-10-31 04:12:07 nas grafana /data/containers/grafana -807f1bb3 2021-10-31 04:12:12 nas traefik /data/containers/traefik -5d9c2314 2021-10-31 04:12:15 nas syncthing /data/containers/syncthing -5f1a2de0 2021-10-31 12:38:40 carmel home /home/fcuny -89f6bbec 2021-10-31 14:53:27 aptos home /home/fcuny -5bb120c9 2021-11-05 15:54:28 aptos home /home/fcuny -5fb31f63 2021-11-06 16:05:30 aptos home /home/fcuny -9bfd32e2 2021-11-07 18:02:06 aptos home /home/fcuny -d4dd252f 2021-11-17 13:40:16 aptos home /home/fcuny -b072a3a1 2021-11-21 04:18:17 nas gitea /data/containers/gitea -6ba6bff3 2021-11-21 04:18:32 nas unifi /data/containers/unifi -bb697aae 2021-11-21 04:18:38 nas grafana /data/containers/grafana -33ba0e83 2021-11-21 04:18:41 nas traefik /data/containers/traefik -e2cae3b5 2021-11-21 04:18:43 nas syncthing /data/containers/syncthing -1caaca88 2021-11-21 13:35:29 carmel home /home/fcuny -97d034ce 2021-11-27 19:16:12 aptos home /home/fcuny -5fa6b510 2021-11-28 04:11:27 nas gitea /data/containers/gitea -6670d391 2021-11-28 04:11:32 nas unifi /data/containers/unifi -77d11ce4 2021-11-28 04:11:38 nas grafana /data/containers/grafana -04ee74c6 2021-11-28 04:11:40 nas traefik /data/containers/traefik -1371d8d2 2021-11-28 04:11:43 nas syncthing /data/containers/syncthing -3b2a45ee 2021-11-28 09:19:13 aptos home /home/fcuny -b19902e6 2021-11-28 15:25:29 carmel home /home/fcuny -02fb34d8 2021-11-30 04:05:15 nas gitea /data/containers/gitea -1ac8f79f 2021-11-30 04:05:21 nas unifi /data/containers/unifi -848505be 2021-11-30 04:05:26 nas grafana /data/containers/grafana -2e48e232 2021-11-30 04:05:29 nas traefik /data/containers/traefik -47732732 2021-11-30 04:05:34 nas syncthing /data/containers/syncthing -dd141856 2021-11-30 12:06:56 carmel home /home/fcuny -00e5429b 2021-12-03 18:31:51 aptos home /home/fcuny -31b849ad 2021-12-05 04:06:10 nas gitea /data/containers/gitea -8cc78932 2021-12-05 04:06:26 nas unifi /data/containers/unifi -b7364a55 2021-12-05 04:06:38 nas grafana /data/containers/grafana -043c4b36 2021-12-05 04:06:43 nas traefik /data/containers/traefik -2e415963 2021-12-05 04:06:48 nas syncthing /data/containers/syncthing -1ef944db 2021-12-05 11:14:51 carmel home /home/fcuny -e58a2421 2021-12-06 04:02:44 nas gitea /data/containers/gitea -907bb839 2021-12-06 04:02:50 nas unifi /data/containers/unifi -050dcff3 2021-12-06 04:02:55 nas grafana /data/containers/grafana -72092444 2021-12-06 04:03:00 nas traefik /data/containers/traefik -d04b79bb 2021-12-06 04:03:03 nas syncthing /data/containers/syncthing -2ef060ec 2021-12-06 11:36:51 carmel home /home/fcuny -a3036320 2021-12-07 04:19:42 nas gitea /data/containers/gitea -18af7ba5 2021-12-07 04:19:48 nas unifi /data/containers/unifi -ba7adae4 2021-12-07 04:19:53 nas grafana /data/containers/grafana -b71283de 2021-12-07 04:19:57 nas traefik /data/containers/traefik -d1918837 2021-12-07 04:19:59 nas syncthing /data/containers/syncthing -ec06c179 2021-12-07 17:24:07 carmel home /home/fcuny -49722319 2021-12-08 04:11:10 nas gitea /data/containers/gitea -b7cfa0d8 2021-12-08 04:11:18 nas unifi /data/containers/unifi -64e98ec2 2021-12-08 04:11:25 nas grafana /data/containers/grafana -d5f848fd 2021-12-08 04:11:30 nas traefik /data/containers/traefik -ce58becc 2021-12-08 04:11:33 nas syncthing /data/containers/syncthing -8342e5b7 2021-12-08 17:45:07 carmel home /home/fcuny -93584f9e 2021-12-09 04:06:27 nas gitea /data/containers/gitea -fb0e6073 2021-12-09 04:06:33 nas unifi /data/containers/unifi -68d354c2 2021-12-09 04:06:39 nas grafana /data/containers/grafana -73e199bd 2021-12-09 04:06:46 nas traefik /data/containers/traefik -47e0e0a6 2021-12-09 04:06:49 nas syncthing /data/containers/syncthing -9d7bcb97 2021-12-09 11:53:49 carmel home /home/fcuny -c2130706 2021-12-10 04:00:56 nas gitea /data/containers/gitea -29af7e4f 2021-12-10 04:01:03 nas unifi /data/containers/unifi -393b006b 2021-12-10 04:01:08 nas grafana /data/containers/grafana -433a00d1 2021-12-10 04:01:13 nas traefik /data/containers/traefik -d4949919 2021-12-10 04:01:18 nas syncthing /data/containers/syncthing -ce2a8a73 2021-12-10 12:10:49 carmel home /home/fcuny -c8d56977 2021-12-11 04:11:20 nas gitea /data/containers/gitea -40f3c6d8 2021-12-11 04:11:25 nas unifi /data/containers/unifi -f24178f5 2021-12-11 04:11:30 nas grafana /data/containers/grafana -3ca4553f 2021-12-11 04:11:33 nas traefik /data/containers/traefik -ca41fe42 2021-12-11 04:11:35 nas syncthing /data/containers/syncthing -b2643ef9 2021-12-11 12:40:49 carmel home /home/fcuny -50cb9254 2021-12-12 04:10:34 nas gitea /data/containers/gitea -85de9005 2021-12-12 04:10:40 nas unifi /data/containers/unifi -0fd36196 2021-12-12 04:10:46 nas grafana /data/containers/grafana -bd8f14dd 2021-12-12 04:10:50 nas traefik /data/containers/traefik -ee0735e3 2021-12-12 04:10:53 nas syncthing /data/containers/syncthing ---------------------------------------------------------------------------------- -148 snapshots -#+end_example - -** How to configure a backup -All daily backups are added to the [[file:~/workspace/infrastructure/puppet/site-modules/backup/files/etc/systemd/system/backups.service][unit file]]. Each backup needs a tag (to make it easier to filter/search). - -This will run once a day. The backups will be stored in =/data/backups= and then be exported to GCS. -** How to restore the backup -First, this is the [[https://restic.readthedocs.io/en/latest/050_restore.html][documentation]] to read. Here's an example: -#+begin_src sh -$ sudo restic -r /data/backups/ -p /etc/restic/password restore 8dbaaf98 --target /tmp/this-is-a-test -repository a37cfab5 opened successfully, password is correct -restoring <Snapshot 8dbaaf98 of [/data/containers/traefik] at 2021-08-14 06:05:49.547829076 -0700 PDT by restic@nas> to /tmp/this-is-a-test -$ sudo ls -l /tmp/this-is-a-test/data/containers/traefik -total 4 -drwxrwxr-x 2 root root 4096 Nov 6 2020 config -#+end_src diff --git a/docs/desktop.org b/docs/desktop.org deleted file mode 100644 index a52fc53..0000000 --- a/docs/desktop.org +++ /dev/null @@ -1,19 +0,0 @@ -* Next build -** Requirements -- Future proof (PCIe 5, DDR5) -- Re-use the nr200p case -- 2 NVMe drive would be nice -- not have to use a GPU would be nice -** Hardware selection - -| component | model | price | note | -|-------------+-----------------------------------------------+-------+------| -| CPU | Intel Core i7-12700K | 380 | | -| CPU cooler | Noctua NH-U9S chromax.black | 0 | | -| motherboard | Asus ROG STRIX B660-I GAMING | 220 | | -| memory | Corsair Vengeance 32 GB (2 x 16 GB) DDR5-5200 | 309 | | -| boot drive | Western Digital Black SN850 | 160 | | -| case | nr200p | 0 | | -|-------------+-----------------------------------------------+-------+------| -| | | 1069 | | -#+TBLFM: @8$3=vsum(@2..@-1) diff --git a/docs/gcloud.org b/docs/gcloud.org deleted file mode 100644 index 95e7531..0000000 --- a/docs/gcloud.org +++ /dev/null @@ -1,21 +0,0 @@ -#+TITLE: Gcloud - -* Initial setup -First we need to create a service account, with: -#+begin_src sh -gcloud --project fcuny-homelab iam service-accounts create world-nix -#+end_src - -Next we need to bind the new policy: -#+begin_src sh -gcloud projects add-iam-policy-binding fcuny-homelab --member="serviceAccount:world-nix@fcuny-homelab.iam.gserviceaccount.com" --role="roles/accessapproval.configEditor" -#+end_src - -Note: I had to add DNS administrator in the console, I don't know what I need to add to this command. - -Finally we need the key: -#+begin_src sh -gcloud iam service-accounts keys create world-nix.json --iam-account=world-nix@fcuny-homelab.iam.gserviceaccount.com -#+end_src - -This will create a file name =world-nix.json=. It's best to encrypt it with =age= and move it under the =secrets= directory for a host. diff --git a/docs/install.org b/docs/install.org deleted file mode 100644 index 40ba5a8..0000000 --- a/docs/install.org +++ /dev/null @@ -1,153 +0,0 @@ -#+TITLE: Installation -#+AUTHOR: Franck Cuny -#+EMAIL: franck@fcuny.net - -* Prepare the USB stick -Download the most recent image from https://nixos.org/download.html then put it on a stick: -#+begin_src sh -sudo cp ~/downloads/nixos-minimal-21.11.336020.2128d0aa28e-x86_64-linux.iso /dev/sda -#+end_src -* Partitioning -** For the workstation (desktop/laptop) -All hosts have the same partitioning for the boot drive: -- /boot partition for UEFI -- / encrypted with btrfs -- a 8GB swap - -If we assume the boot drive to be =nvme0n1=, we will do the following: -#+begin_src sh -parted /dev/nvme0n1 -- mklabel gpt -parted /dev/nvme0n1 -- mkpart primary 512MiB -8GiB -parted /dev/nvme0n1 -- mkpart primary linux-swap -8GiB 100% -parted /dev/nvme0n1 -- mkpart ESP fat32 1MiB 512MiB -parted /dev/nvme0n1 -- set 3 esp on -#+end_src - -Running =lsbkl= should give the following output: -#+begin_src sh -[root@nixos:~]# lsblk -NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS -loop0 7:0 0 709M 1 loop /nix/.ro-store -sda 8:0 1 29.9G 0 disk -├─sda1 8:1 1 784M 0 part /iso -└─sda2 8:2 1 37M 0 part -nvme0n1 259:0 0 465.8G 0 disk -├─nvme0n1p1 259:1 0 457.3G 0 part -├─nvme0n1p2 259:2 0 8G 0 part -└─nvme0n1p3 259:3 0 511M 0 part -#+end_src - -Then we create the LUKS device: -#+begin_src sh -cryptsetup --verify-passphrase -v luksFormat /dev/nvme0n1p1 -cryptsetup open /dev/nvme0n1p1 system -#+end_src - -We can create the partition for the boot drive and activate the swap: -#+begin_src sh -mkswap -L swap /dev/nvme0n1p2 -swapon /dev/nvme0n1p2 -mkfs.fat -F 32 -n nixos-boot /dev/nvme0n1p3 -#+end_src -#+begin_src sh -mkfs.btrfs /dev/mapper/system - -mount -t btrfs /dev/mapper/system /mnt - -btrfs subvolume create /mnt/nixos -btrfs subvolume create /mnt/home -btrfs subvolume create /mnt/snapshots - -umount /mnt -#+end_src - -Now we can re-mount the partitions with the proper options: -#+begin_src sh -mount -o subvol=nixos,compress=zstd,noatime,autodefrag /dev/mapper/system /mnt - -mkdir /mnt/{home,boot,.snapshots} - -mount -o subvol=home,compress=zstd,noatime,autodefrag /dev/mapper/system /mnt/home -mount -o subvol=snapshots,compress=zstd,noatime /dev/mapper/system /mnt/.snapshots -mount /dev/nvme0n1p3 /mnt/boot -#+end_src - -Once the installation is completed: -#+begin_src sh -CUSTOMIZE_TIMESTAMP=$(date -u +%Y%m%dT%H%M%S) -btrfs subvolume snapshot /mnt /mnt/.snapshots/$CUSTOMIZE_TIMESTAMP -#+end_src -** Partitions for the NAS -Create the RAIDs: -#+begin_src sh -mdadm --create /dev/md/fast --level=mirror --raid-devices=2 /dev/sda /dev/sdb -mdadm --create /dev/md/slow --level=mirror --raid-devices=2 /dev/sdc /dev/sde -#+end_src - -Encrypt the RAIDs: -#+begin_src sh -cryptsetup --verify-passphrase -v luksFormat /dev/md/slow -cryptsetup --verify-passphrase -v luksFormat /dev/md/fast -#+end_src - -Then open them: -#+begin_src sh -cryptsetup open /dev/md/fast raid-fast -cryptsetup open /dev/md/slow raid-slow -#+end_src - -Create the filesystem: -#+begin_src sh -mkfs.btrfs /dev/mapper/raid-fast -mkfs.btrfs /dev/mapper/raid-slow -#+end_src - -Then we can mount them to generate the host configuration -#+begin_src sh -btrfs subvolume create /mnt/media -btrfs subvolume create /mnt/containers -umount /mnt - -mount -t btrfs /dev/mapper/raid-slow /mnt/ -btrfs subvolume create /mnt/backups -mkdir /mnt/data/{backups,containers,media} -mount -o subvol=media,compress=zstd,noatime,autodefrag /dev/mapper/raid-fast /mnt/data/media -mount -o subvol=media,compress=zstd,noatime,autodefrag /dev/mapper/raid-fast /mnt/data/media -mount -o subvol=containers,compress=zstd,noatime,autodefrag /dev/mapper/raid-fast /mnt/data/containers -mount -o subvol=backups,compress=zstd,noatime,autodefrag /dev/mapper/raid-slow /mnt/data/backups -#+end_src -* Installing the system -Let's add git and nixFlakes: -#+begin_src sh -nix-shell -p git nixFlakes -#+end_src - -#+begin_src sh -nixos-generate-config --root /mnt -mkdir /mnt/root -git clone https://git.fcuny.net/fcuny/world.git /mnt/root/world -mkdir /mnt/root/world/hosts/<host name> -cp /mnt/etc/nixos/hardware-configuration.nix /mnt/root/world/hosts/<host name>/ -cp /mnt/root/world/hosts/aptos/default.nix /mnt/root/world/hosts/<host name>/ -vim /mnt/root/world/hosts/<host name>/default.nix -cd /mnt/root/world -git add hosts/tahoe -cd / -nixos-install --root /mnt --flake /mnt/root/world#<host name> -#+end_src - -Create another snapshot -#+begin_src sh -CUSTOMIZE_TIMESTAMP=$(date -u +%Y%m%dT%H%M%S) -btrfs subvolume snapshot /mnt /mnt/.snapshots/$CUSTOMIZE_TIMESTAMP -#+end_src - -And a =reboot= should be enough. -* home-manager initial install -After a reboot, as root: -#+begin_src sh -nix-channel --add https://github.com/nix-community/home-manager/archive/release-21.11.tar.gz home-manager -nix-channel --update -nix-shell '<home-manager>' -A install -home-manager build --flake .#fcuny@<host name> -#+end_src diff --git a/docs/secrets.org b/docs/secrets.org deleted file mode 100644 index 5c350e0..0000000 --- a/docs/secrets.org +++ /dev/null @@ -1,29 +0,0 @@ -#+TITLE: Secrets - -* secrets -** system -Secrets at the system level are managed by [[https://github.com/ryantm/agenix][agenix]]. The secrets are encrypted with a couple of =age= keys. I do not use ssh keys to encrypt the secrets. Instead, I do the following: -- each system has a key for the user root, and the secrets for that host are encoded with it as a recipient -- on each workstation, my user (=fcuny=) has a key and the secrets for all the hosts are encrypted with it as a recipient -- in addition, I've a backup key stored on a USB device, and I used its public key to encrypt all the secrets with it - -These keys are backed up on an external USB device and in passage. When re-provisioning a host, the keys are restored from the USB device or from passage itself. - -When provisioning a new host, a key for root (and my user if it's a workstation) is created and stored on the USB device and in passage. -*** add a new secret -#+begin_src sh -nix run github:ryantm/agenix -- -i ~/.age/key.txt -e sendsms/sendsms.age -#+end_src -*** re-key secrets -#+begin_src sh -nix run github:ryantm/agenix -- -i ~/.age/key.txt -r -#+end_src -** home-manager -Nothing for now. -** passage -I use [[https://github.com/FiloSottile/passage][passage]] to store passwords locally. The content of the store is pushed to a remote git repository, and I synchronized the store regularly to the USB device. -* misc -** GPG -nop nop nop nop nop -** keyring -I don't need one anymore. diff --git a/docs/tools.org b/docs/tools.org deleted file mode 100644 index e093f5c..0000000 --- a/docs/tools.org +++ /dev/null @@ -1,167 +0,0 @@ -#+TITLE: Collection of recipes for various tools - -* syncthing -** connection to the remote UI -The web UI for syncthing is binded to localhost. To access the UI of a remote host, create a SSH tunnel: -#+begin_src sh -ssh -L 1235:localhost:8384 -N -f 192.168.0.106 -#+end_src -* yt-dlp -- use =--merge-output-format=mkv= -- check what's the best audio and video for a video -- prefer =mp4= for the audio over =webm= - -** List of supported formats -#+begin_src sh :results verbatim -yt-dlp --list-formats https://www.youtube.com/watch?v=igH-NgcuW2M -#+end_src - -#+RESULTS: -#+begin_example -[youtube] igH-NgcuW2M: Downloading webpage -[youtube] igH-NgcuW2M: Downloading android player API JSON -[info] Available formats for igH-NgcuW2M: -ID EXT RESOLUTION FPS | FILESIZE TBR PROTO | VCODEC VBR ACODEC ABR ASR MORE INFO ---- ---- ---------- --- - ---------- ----- ----- - ----------- ----- --------- ---- ------- ----------------- -139 m4a audio only | 15.00MiB 47k https | mp4a.40.5 47k 22050Hz low, m4a_dash -249 webm audio only | 15.28MiB 48k https | opus 48k 48000Hz low, webm_dash -250 webm audio only | 19.58MiB 62k https | opus 62k 48000Hz low, webm_dash -140 m4a audio only | 40.06MiB 127k https | mp4a.40.2 127k 44100Hz medium, m4a_dash -251 webm audio only | 39.20MiB 124k https | opus 124k 48000Hz medium, webm_dash -17 3gp 176x144 12 | 24.81MiB 78k https | mp4v.20.3 78k mp4a.40.2 0k 22050Hz 144p -160 mp4 256x144 12 | 34.44MiB 109k https | avc1.4d400c 109k 144p, mp4_dash -278 webm 256x144 12 | 28.61MiB 90k https | vp9 90k 144p, webm_dash -133 mp4 426x240 24 | 77.23MiB 244k https | avc1.4d4015 244k 240p, mp4_dash -242 webm 426x240 24 | 72.41MiB 229k https | vp9 229k 240p, webm_dash -134 mp4 640x360 24 | 178.23MiB 565k https | avc1.4d401e 565k 360p, mp4_dash -18 mp4 640x360 24 | 231.71MiB 734k https | avc1.42001E 734k mp4a.40.2 0k 44100Hz 360p -243 webm 640x360 24 | 137.73MiB 436k https | vp9 436k 360p, webm_dash -135 mp4 854x480 24 | 329.98MiB 1046k https | avc1.4d401e 1046k 480p, mp4_dash -244 webm 854x480 24 | 244.94MiB 776k https | vp9 776k 480p, webm_dash -136 mp4 1280x720 24 | 638.05MiB 2023k https | avc1.4d401f 2023k 720p, mp4_dash -22 mp4 1280x720 24 | 2150k https | avc1.64001F 2150k mp4a.40.2 0k 44100Hz 720p -247 webm 1280x720 24 | 490.14MiB 1554k https | vp9 1554k 720p, webm_dash -137 mp4 1920x1080 24 | 1.13GiB 3685k https | avc1.640028 3685k 1080p, mp4_dash -248 webm 1920x1080 24 | 893.45MiB 2833k https | vp9 2833k 1080p, webm_dash -#+end_example -** Best audio and video -#+begin_src sh -yt-dlp -f 'bv*+ba' https://www.youtube.com/watch?v=igH-NgcuW2M -o '%(id)s.%(ext)s' -#+end_src -** Download a playlist -Save into =channel_id/playlist_id= directory with the video added to an archive text file: -#+begin_src sh -yt-dlp -f 'bv*[height=1080]+ba' --download-archive videos.txt https://www.youtube.com/playlist?list=PLlVlyGVtvuVnUjA4d6gHKCSrLAAm2n1e6 -o '%(channel_id)s/%(playlist_id)s/%(id)s.%(ext)s' -#+end_src -** Download a channel -#+begin_src sh -yt-dlp -f 'bv*[height=720]+ba' --download-archive videos.txt https://www.youtube.com/c/FootheFlowerhorn/videos -o '%(channel)s/%(title)s.%(ext)s' -#+end_src -* exiftool -** Copy media based on the creation date -#+begin_src sh -exiftool -v -o . '-Directory<CreateDate' -d /data/photos/%Y/%Y-%m-%d/ . -#+end_src -** Move media based on the creation date -#+begin_src sh -exiftool -v '-Directory<CreateDate' -d /data/photos/%Y/%Y-%m-%d/ . -#+end_src - -Alternatively, in case the creation date is incorrect: -#+begin_src sh -exiftool -v '-Directory<DateTimeOriginal' -d /data/photos/%Y/%Y-%m-%d/ -#+end_src -** Move pdf to a directory -To move papers (for example) using the title and date of creation to a specific destination: -#+begin_src sh -exiftool '-filename<${Title;}.%e' '-directory<CreateDate' -d ~/documents/papers/%Y/ . -#+end_src -** Edit metadata from a google takeout -This [[https://github.com/kaytat/exiftool-scripts-for-takeout][repository]] as a few scripts for =exiftools= that are interesting. In case this repository were to disappear in the future, here is the script to update the metadata from the JSON files: -#+begin_src sh :filename use_json.args -# Fill in from Google's JSON - -# Look at all media files and ignore JSON ---ext -json - -# Recursive --r - -# Show processed filenames --v0 - -# Check if the corresponding JSON exists --if -(-e "${Directory}/${Filename}".".json") - -# Attempt to modify media only if the info doesn't already exist --if -($Filetype eq "MP4" and not $quicktime:TrackCreateDate) or ($Filetype eq "MP4" and $quicktime:TrackCreateDate eq "0000:00:00 00:00:00") or ($Filetype eq "JPEG" and not $exif:DateTimeOriginal) or ($Filetype eq "PNG" and not $PNG:CreationTime) - -# Attempt to read in the JSON --tagsfromfile -%d%F.json - -# -# Write out the tags. Use ConvertUnixTime to try and convert the UTC timestamp -# to a reasonable local EXIF string. -# - -# EXIF for regular JPG photos --AllDates<${PhotoTakenTimeTimestamp;$_=ConvertUnixTime($_,1)} - -# PNG-specific --XMP-Exif:DateTimeOriginal<${PhotoTakenTimeTimestamp;$_=ConvertUnixTime($_,1)} --PNG:CreationTime<${PhotoTakenTimeTimestamp;$_=ConvertUnixTime($_,1)} - -# Quicktime / MP4. Assume that timestamp is in UTC. --QuickTime:TrackCreateDate<${PhotoTakenTimeTimestamp;$_=ConvertUnixTime($_,0)} --QuickTime:TrackModifyDate<${PhotoTakenTimeTimestamp;$_=ConvertUnixTime($_,0)} --QuickTime:MediaCreateDate<${PhotoTakenTimeTimestamp;$_=ConvertUnixTime($_,0)} --QuickTime:MediaModifyDate<${PhotoTakenTimeTimestamp;$_=ConvertUnixTime($_,0)} - -# Clobber everything --overwrite_original -#+end_src - -and to run it: =exiftool -@ use_json.args <takeout_dir>= -* beet -=beet= is a media library management system for music. The main documentation is [[https://beets.readthedocs.io/en/latest/index.html][here]]. -** search -By album -#+begin_src shell -tahoe:~ beet ls album:henry -Nick Cave & the Bad Seeds - Henry’s Dream - Papa Won’t Leave You, Henry -Nick Cave & the Bad Seeds - Henry’s Dream - I Had a Dream, Joe -Nick Cave & the Bad Seeds - Henry’s Dream - Straight to You -Nick Cave & the Bad Seeds - Henry’s Dream - Brother, My Cup Is Empty -Nick Cave & the Bad Seeds - Henry’s Dream - Christina the Astonishing -Nick Cave & the Bad Seeds - Henry’s Dream - When I First Came to Town -Nick Cave & the Bad Seeds - Henry’s Dream - John Finn’s Wife -Nick Cave & the Bad Seeds - Henry’s Dream - Loom of the Land -Nick Cave & the Bad Seeds - Henry’s Dream - Jack the Ripper -#+end_src - -All the albums from 2023 -#+begin_src shell -tahoe:~ beet ls year:2023 -a -ALL HANDS_MAKE LIGHT - "Darling the Dawn" -Big ‡ Brave - Nature Morte -boygenius - the record -Ky - Power Is The Pharmacy -OM - Gebel Barkal / Version -Joni Void - Everyday Is The Song -#+end_src -** Update -Modify the year for an album: -#+begin_src shell -tahoe:~ beet modify path:/data/fast/music/Nick\ Cave\ \&\ the\ Bad\ Seeds/B-Sides\ \&\ Rarities,\ Part\ I year=2005 -Modifying 56 items. -Nick Cave & Dirty Three - B-Sides & Rarities, Part I - Time Jesum Transeuntum et Non Riverentum - year: 2021 -> 2005 -Nick Cave & Shane MacGowan - B-Sides & Rarities, Part I - What a Wonderful World - year: 2021 -> 2005 -... -Really modify, move and write tags? (Yes/no/select) yes -#+end_src diff --git a/docs/wireguard.org b/docs/wireguard.org deleted file mode 100644 index 154c159..0000000 --- a/docs/wireguard.org +++ /dev/null @@ -1,21 +0,0 @@ -#+TITLE: Configuration for wireguard - -* Creating the keys -We need a key for the host: -#+begin_src sh -(umask 0077; wg genkey > peer_A.key) -#+end_src - -Next we create the public key: -#+begin_src sh -wg pubkey < peer_A.key > peer_A.pub -#+end_src - -Now we need to add the private key to the list of secrets: -#+begin_src sh -nix run github:ryantm/agenix -- -e secrets/network/<host name>/wireguard_privatekey.age -#+end_src - -Once this is done, update [[file:~/workspace/world/configs/wireguard.toml][wireguard.toml]] to add the new peer with the public key. - -Once this is completed, we can delete the files =peer_A.key= and =peer_A.pub=. |