More Tools For Blogging Tool

Spent the last week working on Blogging Tool. I want to get as much done as a I can before motivation begins to wain, and it begins languishing like every other project I've worked on. Not sure I can stop that, but I think I can get the big ticket items in there so it'll be useful to me while I start work on something else.

I do have plans for some new tools for Blogging Tool: making it easier to make Lightbox Gallery was just the start. This last week I managed to get two of them done, along with some cross-functional features which should help with any other tools I make down the road.

Move To Sqlite

First, a bit of infrastructure. I moved away from Rainstorm as the data store and replaced it with Sqlite 3. I'm using a version of Sqlite 3 that doesn't use CGO as the Docker container this app runs in doesn't have libc. It doesn't have as much support out there as the more popular Sqlite 3 client, but I've found it to work just as well.

One could argue that it would've been fine sticking with Rainstorm for this. But as good as Rainstorm's API is, the fact that it takes out a lock on the database file is annoying. I'm running this app using Dokku, which takes a zero-downtime approach to deployments. This basically means that the old and new app container are running at the same time.  The old container doesn't get shut down for about a minute, and because it's still holding the lock, I can't use the new version during that time as the new container cannot access the Rainstorm database file. Fortunately, this is not an issue with Sqlite 3.

It took me a couple of evenings to port the logic over, but fortunately I did this early, while there was no production data to migrate. I'm using Sqlc for generating Go bindings from SQL statements, and a home grown library for dealing with the schema migrations. It's not as easy to use as the Rainstorm API but it'll do. I'm finding working with raw SQL again to be quite refreshing so it may end up being better in the long run.

Imaging Processing

Once that's done, I focused on adding those tools I wanted. The first one to sit alongside the gallery tool, is something for preparing images for publishing. This will be particularly useful for screenshots. If you look carefully, you'd noticed that the screenshots on this site have a slightly different shadow than the MacOS default. It's because I actually take a screenshot without the shadow, then use a CLI tool to add one prior to upload. I do this because the image margins MacOS includes with the shadow are pretty wide, which makes the actual screenshot part smaller than I like. Using the CLI tool is fine, but it's not always available to me. So it seemed like a natural thing to add to this blogging tool.

So I added an image processing "app" (I'm calling these tools "apps" to distinguish them from features that work across all of them) which would take an image, and allows you to apply a processor on it. You can then download the processed image and use it in whatever you need.

This is all done within the browser, using the Go code from the CLI tool compiled to WASM. The reason for this is performance. These images can be quite large, and I'd rather avoid the network round-trip. I'm betting that it'll be faster running it in the browser anyway, even if you consider the amount of time it takes to download the WASM binary (which is probably around a second or so).

One addition I did add was to allow processors to define parameters which are shown to the user as input fields. There's little need for this now — it's just being used in a simple meme-text processor right now — but it's one of those features I'd like to at least get basic support for before my interest wains. It wouldn't be the first time I stopped short of finishing something, thinking to my self that I'd add what I'll need later, then never going back to do so. That said, I do have some ideas of processors which could use this feature for real, which I haven't implemented yet. More on that in the future, maybe.

Audio Transcoding And Files

The other one I added deals with audio transcoding. I've gotten into the habit of narrating the long form posts I write. I usually use Quicktime Player to record these, but it only exports M4A audio files and I want to publish them as MP3s.

So after recording them, I need to do a transcode. There's an ffmpeg command line invocation I use to do this:

ffmpeg -i in.m4a -c:v copy -c:a libmp3lame -q:a 4 out.mp3

But I have to bring up a terminal, retrieve it from the history (while it's still in there), pick a filename, etc. It's not hard to do, but it's a fair bit of busy work

I guess now that I've written it here, it'll be less work to remember. But it's a bit late now since I've added the feature to do this for me. I've included a statically linked version of ffmpeg in the Docker container (it needs to be statically linked for the same reason why I can't use CGO: there's no libc or any other shared objects) and wrapped it around a small form where I upload my M4A.

The transcoding is done on the server (seemed a bit much asking for this to be done in the browser) but I'm hoping that most M4A files will be small enough that it wouldn't slow things down too much. The whole process is synchronous right now, and I could've make the file available then and there, but it wouldn't be the only feature I'm thinking of that would produced files that I'd like to do things with later. Plus, I'd like to eventually make it asynchronous so that I don't have to wait for long transcodes, should there be any.

So along with this feature, I added a simple file manager in which these working files will go.

They're backed by a directory running in the container with metadata managed by Sqlite 3. It's not a full file system — you can't do things like create directories, for example. Nor is it designed to be long term storage for these files. It's just a central place where any app can write files out as a result of their processing. The user can download the files, or potentially upload them to a site, then delete them. This would be useful for processors which could take a little while to run, or run on a regular schedule.

I don't have many uses for this yet, apart from the audio transcoder, but having this cross-functional facility opens it up to features that need something like this. It means I don't have to hand-roll it for each app.

Anyway, that's the current state of affairs. I have one, maybe two, large features I'd like to work on next. I'll write about them once they're done.