Automating Obsidian With Python Organize Tool
When self-hosting it's easy to accumulate self-hosted apps that look pretty but you don't use much. These often store data that could've been a text file.
A few years ago I started using Obsidian to store Markdown notes. Obsidian's value comes from plugins rather than the core app itself. The Remotely Save plugin combined with the amazing CopyParty syncs my notes across mobile and desktop. I could use Syncthing but on iOS it's a second-class citizen.
The downside of all of this is Obsidian is closed-source and electron based. I want my markdown notes to be platform agnostic, accessible on any device I have and on any operating systems without needing Obsidian. Obsidian's plugins are fantastic, but each plugin ties me further into them. I want Obsidian as a notes client, not the client. When it comes to creating and auto-populating them, I'd rather keep that outside of Obsidian.
As I have a server, I decided I'd use that to automatically generate files that can then be sync'd to any device. Surprisingly there's a (unfortunately named) tool for that: Python Organize.
Organize is very similar to tools like Hazel on MacOS. You create rules which contain locations, filters (conditions) and actions. The documentation is great. Possibly due to the name I struggled to find real-world walkthroughs, hence this post. It's nothing particularly exciting, I just started with automating the creation of a daily note.
On the desktop this is done with a template note and maybe a couple of plugins. Moving automatic note creation to my server reduces my reliance on these plugins. It also lets me do things I can't do from within Obsidian alone.
I installed python-organize via a python venv. I ran organize new and organize edit to create a config file. This is stored in ~/.config/organize/config.yaml:
rules:
- name: Daily note from template
locations:
- '/my/nas/obsidian/data/vault/00 Meta/01 Templates'
filters:
- name: 'Daily Journal'
- extension: md
actions:
- copy:
dest: "/my/nas/obsidian/data/vault/10 Periodic/Daily/{ now().strftime('%Y') }/{ now().strftime('%m') }/{ now().strftime('%Y-%m-%d') }.md"
on_conflict: skip
tags:
- daily
I structure my Obsidian vault using folders prefixed with numbers.
00 Meta/01 Templatescontains my daily note (journal) template, while my completed daily notes are stored under the structure10 Periodic/Daily/YYYY/MM/YYYY-MM-DD.md.
The rule copies my Daily note (journal) template to my daily note folder. on_conflict: skip ensures I don't overwrite a daily note if I run organize more than once in a day. I use tags to allow me to run specific rules at specific times via cron. I have something like the following shell script in ~/bin/organize-daily.sh:
#!/bin/bash
set -e
export SERVICE=https://my.healthchecks.server
curl -fsS --retry 3 "${SERVICE}/ping/xxx-f120-4c9c-bf67-xxx/start" > /dev/null
/home/username/venv/bin/organize run --tags=daily ~/.config/organize/config.yaml 1>/dev/null
export EXIT_CODE=$?
curl -fsS --retry 3 "${SERVICE}/ping/xxx-f120-4c9c-bf67-xxx/${EXIT_CODE}" > /dev/null
exit ${EXIT_CODE}
This script contacts my self-hosted healthchecks server. The flow is pretty simple:
- Tell healthchecks I'm starting
- Run organize only on rules with the
dailytag set - Tell healthchecks I've finished, with success or failure values
The online version of healthchecks is free to try if you want to try before possibly self-hosting.
The funny bit with EXIT_CODE is so Healthchecks can warn me if organize fails. My crontab contains the following line:
5 0 * * * /home/username/bin/organize-daily.sh
This process works surprisingly well. I can use the shell action to generate activity-based notes via scripts. If I just want quick tweaks I can use the python action to modify files in-situ.
I see this working particularly well with some of my other organizational tools. I can automate ingestion and info gathering on Paperless-ngx. I could get images of the week from Immich. I can also replace the Karakeep sync plugin I use for synchronizing bookmarks to notes.
Although Obsidian is my UI of choice right now, by using plain text I can access my notes with any tool I like at any time. This approach brings my data under my total control without reliance upon external services. I plan to extend this to my weekly and monthly notes before looking at how I can replace Karakeep's plugin, and then start integrating paperless-ngx. But that's for another post.