Infra: Nextcloud via MASH

October 9, 2024

Another upgrade in my self-hosted IT infrastructure.

Nextcloud is a self-hostable platform centred around file storage, providing a plug-in framework expandable to a broad range of services. It is one of the older hosting platforms, and unfortunately showing its age in several respects. I consider few of its default and plug-in apps to be of high quality. However, due to its popularity, stability and support it seems to make a good base for file storage services and connecting other services.

I have been running Nextcloud for years but so far only using it for one core stable service (contacts and calendar) and some experiments. This is partly because my deployment is an old, stale configuration, stuck on an old version. I need to update to a maintainable deployment.

There are a few ways to deploy Nextcloud. For this I will use MASH. It is not one of the “official” methods; it's a self-hosting framework from the author of the popular Matrix self-hosting framework.

My goal with Nextcloud is:

on my own domain, under my own control
self-hosted on my own infra for now (long term goal: easy migration between self-hosted and hosted elsewhere)
stable for the long-ish term
integrates as a component of my infrastructure alongside other own services; it's not the portal for my whole system

My required and desired Nextcloud services:

now: family contacts and calendar storage (CardDAV, CalDAV) (already been running for years on my old nextcloud installation)
now: WebDAV storage (initial use: for Seedvault backups for our android-compatibles; other uses later)
later: DAV-Push calendar/contacts push notification service (works over UnifiedPush through Bitfire's DAVx5)
later: use Single Sign-On (SSO) (or perhaps be my SSO provider)
later: migrate or attach our documents storage (personal and family collections)
later: possibly migrate or attach our photos storage (personal and family collections), with sync, gallery (likely with Nextcloud Gallery + Memories) and/or integration with Immich
later: document editing (with Collabora Online, because open formats first)

Integrating as a component of my infrastructure

A note on integrating as a component of my infrastructure. Nextcloud does not play well in this respect. It rather wants to be one's hub, one's portal to all services. As an example particularly relevant to my service migration today, Nextcloud's calendar and contacts support is designed around its built-in calendar and contacts storage service. Various people have asked in the public discussion forum if they could connect it to a separate calendar and contacts service. The answer has been that's not within the goals and so unlikely ever to be supported.

Others take the same attitude. I have written how my current email provider Fastmail assumes they also host my contacts and calendar. Their web-mail interface has no way of connecting to an externally hosted calendar and contacts server.

The problem with this attitude is that software services are so numerous and diverse that one is unlikely to be using just one system. As soon as we have two systems each assuming they own my contacts and calendar, we have a problem. External sync tools could be deployed, but would not make a complete solution.

I am seriously considering moving my contacts and calendar out of Nextcloud, to be hosted by a separate service such as Baïkal or Davis (both based on Sabre/DAV, as is Nexcloud's implementation). I would lose some Nextcloud integration niceties (such as showing contacts on its map) but decoupling this service feels like a good move.

So, I don't hold out hope that Nextcloud would integrate nicely with everything else. My main expectation is that it will be effective as a file storage provider.

MASH Installation

I started by creating a new staging/testing VM for MASH services, configuring the MASH Traefik and Postgresql services, and forwarding connections from my upstream Traefik reverse-proxy to it.

The MASH Nextcloud doc suggests to me a configuration like this, for the basics. Enable it, give it a base domain name, and some place-holder comments for further work:

# ansible inventory section for: MASH nextcloud

nextcloud_enabled: true

nextcloud_hostname: "nextcloud.{{ base_domain }}"
#nextcloud_path_prefix: /

#nextcloud_config_additional_parameters: {key,value,type:string}...

# exim-relay...
# (configure outbound email here)

# (no KeyDB: we'll keep it simple)

# SSO...

# Collabora Online...

# Preview Generator...

Initial installation:

just run --tags=install-all,start

(When I run the ansible MASH playbook, I invoke it through just run but from a different working directory with --justfile=.../justfile, which in turn runs ansible-playbook .../setup.yml. I point to my staging inventory with -i .../inventories/staging/, and add --vault-id= and -D and maybe -v. Here I will write just run.)

Finish Nextcloud Installation: Interactive Step?

At this point the doc says we need to intervene:

After installation, follow Nextcloud's setup wizard... choose any username/password for your account... choose PostgreSQL with the credentials you see after running just run-tags print-nextcloud-db-credentials

Once you've fully installed Nextcloud, adjust its default configuration (URL paths, trusted reverse-proxies, etc.) by running: just run-tags adjust-nextcloud-config

That's a second ansible run to print some credentials, then an interactive cut-and-paste session, also creating an admin password at this time, then a third ansible run to finish the installation.

But we want to do this non-interactively: Infrastructure as Code. To be able to set up a testing/staging server, tear it down, set it up again the same way or with controlled changes. Side project: let's improve the MASH playbook: MASH Nextcloud Non-interactive Installation.

For now, I proceed with just run-tags print-nextcloud-db-credentials, that occ command with the credentials filled in, and just run-tags adjust-nextcloud-config.

On trying to access the URL of the completed installation in a web browser, we see a message that the domain is not configured as trusted. We need to configure it:

nextcloud_config_additional_parameters:
  - { key: "trusted_domains 1", value: "{{ nextcloud_hostname }}", type: string }

Default Apps, Files, Settings

Nextcloud tries to be “friendly” to new users by adding a selection of apps such as a “photos” app, and default/skeleton files such as examples of different file formats and sample photos. We want to choose the initial apps as part of installation. We don't want sample files cluttering our space.

nextcloud_config_additional_parameters:
# skeletondirectory: blank to disable the default 'skeleton files'
  - { key: skeletondirectory, value: "", type: string }

However, by the time this runs, the “admin” user has already been created, getting the default skeleton files. This doesn't really matter, but it would be nice to get the settings in place earlier. (TODO: Perhaps we could do so by creating a “something-config.php” file before running the “maintenance:install” step? Or by passing this setting in another way such as through an environment variable?)

Remove and lock down some settings to simplify the experience. See: Configuration Parameters documentation.

  - { key: default_language, value: "en", type: string }
  - { key: force_language, value: "en", type: string }
  - { key: default_locale, value: "en_GB", type: string }
  - { key: force_locale, value: "en_GB", type: string }
  - { key: default_phone_region, value: "GB", type: string }
  - { key: allow_user_to_change_display_name, value: "false", type: "boolean" }
  - { key: customclient_android, value: "null", type: "null" }
  - { key: customclient_ios, value: "null", type: "null" }
  - { key: upgrade.disable-web, value: "true", type: "boolean" }
  - { key: simpleSignUpLink.shown, value: "false", type: "boolean" }

(Apps: TODO.)

(Theming: TODO.)

Email Sending

For system purposes such as password resets.

Via MASH exim-relay.

exim_relay_enabled: true
exim_relay_hostname: "{{ base_domain }}"
exim_relay_sender_address: "<my name>@{{ exim_relay_hostname }}"

Run the playbook again with tags 'install-all,start', followed by running again the 'adjust-nextcloud-config' (or, with my improvements, post-install-nextcloud) tag.

The “Test and verify email settings” button in the Nextcloud admin “Basic settings” section then succeeds in sending a test email to me.

(Well, it was a bit trickier to run the email test. It takes its destination email address from the logged in user's profile. The admin UI blocked me from adding an email address to my 'admin' user account. Instead I set up another test user account, with an email address, made that an admin too, and logged in with that. Then I could push the email test button.)

Contacts and Calendar (CardDAV, CalDAV)

In Nextcloud, export and import of address books and calendars seems to be a manual process, requiring access to each user account involved. We may have both personal and shared address books and calendars. It's fiddly.

On old system: manual export. From the Nextcloud Contacts and Calendar GUIs, export each of our address books (as .vcf files) and calendars (as .ics files).
On this new system: manual import. For each address book, we have to first create a new address book, then import the corresponding file into it. For calendars, the import GUI allows to select the file first, then select “New calendar” as the destination.

Is there a more automatable way? Surely? Probably with external sync tools such as vdirsyncer. Or perhaps database export/import, as hinted by Davis's Migrating from Baïkal instructions.

WebDAV storage

TODO. (Nothing is needed to get going initially, but later I will want to connect external storage.)

WebDAV-Push

install Nextcloud app WebDAV-Push
install Nextcloud app UnifiedPush Provider
install on android-compatibles, “NextPush”
install on android-compatibles, “DAVx5” ...

Well-Known

The MASH playbook installs /.well-known redirects for CardDAV and CalDAV on the base domain of the Nextcloud server, allowing simpler configuration of clients such as DAVx5.

The Nextcloud server domain is a subdomain of our home domain. Ideally we will redirect also from our home domain, so the required address is shorter.

Probably we can add these redirects to our main reverse-proxy, redirecting to the Nextcloud server's /.well-known, and let it take it from there. From previous experience we likely need to modify the Host header to match the Nextcloud subdomain, in order to match the MASH Traefik routing rule.

Follow/Feedback/Contact: RSS feed · Fedi follow this blog: @julian@wrily.foad.me.uk · use the Cactus Comments box above · matrix me · Fedi follow me · email me · julian.foad.me.uk Donate: via Liberapay All posts © Julian Foad and licensed CC-BY-ND except quotes, translations, or where stated otherwise