awesomeFOSS — julian

UnifiedPush -- Wider Developments

Wed, 27 Nov 2024 14:48:00 +0000

An article about Building the Self-Agency Mobile Ecosystem: Push Messaging

Push messaging is the system that enables incoming messages to wake up and reach our apps, instantly and efficiently. But Who Cares Who Delivers Our Notifications? Our answer is: we care, and we do not like being locked in to depending on one mega-corp's system.

Therefore, in our libre mobile computing devices, we require a push-messaging infrastructure built from open standard technology that gives us freedom to choose our service providers and authority over them.

How We Get There

How can we build this? What is needed to get widespread UnifiedPush support in /e/ and other mobile OS?

The big picture involves several areas of work.

OS integration of the distributor, including how it's configured, auto-detected, optimised, etc.
Server Software: Making sure a suitable server-side component is available, supported, optimised for this use case.
Server hosting: making sure a server component is easily deployable by others, including making generic deployment methods (Debian, Docker, Nix, K8s, ...) and integrating it in some hosting systems.
Apps: Working with other apps to make sure they connect and work with UP out-of-the-box.
Sharing this work among other mobile OS projects (microG, CalyxOS, GrapheneOS, PostmarketOS, ...) .
Consider standardising a UP client-server protocol so that a built-in distributor can work with several different server implementations and vice-versa.

(Links in this and the following sections lead to relevant issues or discussions, where I could find any.)

OS Integration of the UP Distributor

Including how the UP distributor is configured, auto-detected, optimised, Android permissions, etc.

I recently analysed how /e/OS 2.5 has started the ball rolling

Server Software

Making sure a suitable server-side component is available, supported, optimised for this use case

Nextcloud: UnifiedPush Provider — exists
It could alternatively begin with a fork of ntfy-server for example.

Server Hosting: Technical

Making sure a server component is easily deployable by others, including making generic deployment methods (Debian, Docker, Nix, K8s, ...) and integrating it in some hosting systems, ranging from hobbyist to serious commercial systems.

Docker ntfy — exists
K8s ntfy helm-chart — exists
Nextcloud: UnifiedPush Provider — exists
Matrix-Docker-Ansible-Deploy: ntfy-server — exists
Yunohost: ntfy-server — exists
SelfHostBlocks — not yet
CoopCloud — not yet
Fediversity (an upcoming hosting project) — not yet

Work out and write up (shareable) policies about who hosts, terms and conditions, privacy, rate limiting and other special measures, etc.?

Apps

Working with other apps to make sure they connect and work with UP out-of-the-box.

Category 1: default apps in /e/OS etc., like...

Mail: K9/Thunderbird-andriod #5165 and UnifiedPush for IMAP (NGI proposal 2024-10-535)
Nextcloud -uppush #17 -notifications #1225 -android #8684
Accounts/Calendar/Contacts (DAVx5 #36 + DAV-push)

Category 2: other popular apps like...

HomeAssistant #446344
Fluffychat — exists
Element — exists
Signal forks
Telegram-FOSS #577
Conversations.im — exists
Pixelfed #86

Standardising UP client-server protocol?

Should we consider standardising the UP client-server protocol? At present, by design, one must choose a UP distributor and UP server together as a pair. If we standardised the protocol between them, so any distributor would work with any server, this would certainly have some benefits, but also down-sides, and is not necessarily the right thing to do. UP was designed this way on purpose and achieves its goals without needing to standardise there. No ordinary person needs to be able to change their UP implementation. Only techies and service providers might like to do that.

Therefore this is an exploration rather than a development. I discuss this further in a separate article.

#unifiedPush #degoogled #awesomeFOSS #eOS #Murena #ntfy #mobiFree

Follow/Feedback/Contact: RSS feed · Fedi follow this blog: @julian@wrily.foad.me.uk · matrix me · Fedi follow me · email me · julian.foad.me.uk Donate: via Liberapay All posts © Julian Foad and licensed CC-BY-ND except quotes, translations, or where stated otherwise

UnifiedPush: Standardise UP Client-Server Protocol?

Tue, 26 Nov 2024 20:00:00 +0000

This is a technical article. For a more general introduction to UnifiedPush you might read the first half of Going Google-Free with UnifiedPush in /e/OS or my other articles about UnifiedPush .

Should we consider standardising a UP client-server protocol so that a built-in distributor can work with several different server implementations and vice-versa?

Not necessarily, and I will attempt to explain why.

UnifiedPush High Level Architecture

Look at the diagrams on the unifiedpush.org home page. There are four connections involved, shown arranged in a square. UP standardises the protocol for one of the connections, the bottom one (UP distributor <–> app). It also standardises something about how the whole system works.

The protocol on the top connection (App server –> UP server) has recently been upgraded to be compatible with WebPush.

The other two (App <–> app-server and UP <–> UP-server) are two other, separate, independent protocols. The protocol the app developer uses for app <–> app-server is not relevant to UnifiedPush. At present, UnifiedPush standards also do not specify a protocol to be used for the connection between the UP server and the UP distributor (shown on the left).

The ntfy distributor requires a ntfy server, whereas the nextpush distributor requires a nextpush server, and so on. They use different protocols for that link in the chain.

Multiple Protocols or Standardised Protocol

It would be possible to make a UP server that supports (for example) both the NextPush and ntfy protocol(s). In fact ntfy supports two different protocols (the “json” one and the “websocket” one) and you choose one in the settings. The ntfy server supports both of those.

The point is, UP achieves its goals without needing to standardise there. In some respects it's beneficial that the user and/or service provider can choose those components and protocol that best suit their needs, while the end-user apps and app servers all still remain completely compatible (standardised) no matter what UP components are used.

Additionally standardising the UP <–> UP-server protocol could make different UP servers and distributors interoperable. It could potentially help with users bringing their own choice of device (assuming it has a UP distributor built in) from one service provider to another, for example.

But that's more of a deployment-level standard: something that service providers might want to standardise. Android-compatible mobile OS's (for example) might get together and declare “we” (as a loose group) like to standardise on (let's say) ntfy-websocket protocol here, because (suppose) it works best with android's OS network layer and sleep optimisations. (Just examples; I have not researched what protocol actually works best in each scenario.) Then, Linux-based mobile OS's might choose a different protocol that works better with Linux's networking and sleep optimisations. Then, if a certain libre mobile service provider wanted to support both android-compatible and linux-based devices, they might choose to support two protocols. Or would standardising on one protocol be more helpful in such a scenario? It's not clear.

No ordinary person needs to be able to change their UP implementation. Only techies and service providers might like to do that.

So, at this stage this is perhaps the least important of all the areas to work on.

First explained in #unifiedpush:matrix.org.

#unifiedPush #degoogled #awesomeFOSS #eOS #Murena #ntfy #mobiFree

Going Google-Free with UnifiedPush in /e/OS

Tue, 19 Nov 2024 12:00:00 +0000

Congratulations UnifiedPush! Congratulations Murena!

Murena's /e/OS 2.5 ships with UnifiedPush support included as announced by a small note in the 2.5-t release notes .

This exciting development brings Google-free push messaging to the regular users of an important player in the freedom mobile OS space.

So What?

Push messaging is the system that enables incoming messages to wake up and reach our apps, instantly and efficiently.

To achieve this, the phone operating system keeps a single network connection open to a push messaging server. The apps don't need to keep running in the background: they can go to sleep. When a push message arrives, the OS wakes up the relevant app and hands it the message.

All our apps share this push service — even the “private” ones — if they want push messaging.

That's great, except... Who Cares Who Delivers Our Notifications?

Until now, most people's phones run Google or Apple software. The Big Techs operate and control those phones. Along with everything else, they provide the push messaging service. And so they, Big Tech, control it.

The UnifiedPush public open standard changes this. With UnifiedPush, we are no longer merely “users” of our devices but are in charge of them. With UnifiedPush, we can choose which service provider will deliver our push messaging. Thereby we are in control of who has the ability to monitor or block our push messaging.

UnifiedPush has been available to those interested, for a while now, but until now it has required installing extra software.

Putting the support for push messaging into the operating system means ~~users~~ people who choose not to be operated by Big Tech will now be able to simply install the apps they care about and immediately have the alternative system working for them.

Technical

I'm going to delve into the technical side of this initial release, because it's a big interest area of mine. If you are a regular user, I suggest at this stage you might concentrate on switching to /e/OS or another freedom mobile OS and expect this functionality will be maturing quickly from now on and soon be seamless. If you're an early adopter or interested in its development, read on.

I am trying it out. I already run /e/OS on my main phone and upgrading to version 2.5 was simple with the built-in upgrader.

The OS embeds a custom fork of ntfy, calling itself “foundation.e.ntfy 1.17.0”. So, congratulations also to ntfy! At this stage the developer of ntfy, Philip Heckel, told me in #ntfy room he was “not yet involved” in this development and this is the “First I'm hearing about it.”

ntfy is a notification system that can also provide UnifiedPush service. It is so good at this job that it has for some time been the UnifiedPush implementation recommended for most people. One can install the ntfy app and it provides UnifiedPush service to our other apps.

When we install the ntfy app ourself, we may either let it use the creator's default server (at ntfy.sh) or configure it to use our choice of ntfy server, if we know of one or if we run our own. As with anything, using someone else's server comes with conditions and restrictions, especially if we don't pay for the service, so we need to think about that.

Let's see what /e/OS does.

If not Google's then Whose Server?

In this push service built in to /e/OS, there is no configuration UI. There is a tiny settings UI where we have to “enable” the service — see below. Technically, the full ntfy UI is also available, but not accessible through the normal system UI — neither the app launcher nor the settings. But as curious techies we can hack in to it if we like. See below.

What we can do is install and use the UP-example app. Click the “REGISTER” button, and we discover it's preconfigured to connect to... . o O (their own ntfy server?) No! It connects to ntfy.sh, the server run by the author of ntfy. Hmm.

I should say that I found this while not signed in to Murena's “e cloud” services. It's possible and would make some sense if it would point to a different server if signed in. I will be interested to find out.

The first thing to comment on, then, is Murena will surely need to run their own push server, for privacy and scaling and economic reasons, or else make an agreement with ntfy. In the short term, to get things started, it may be acceptable for everyone to be subject to the free (gratis) ntfy service's rate limits and other conditions. That's something that can be changed later, seamlessly, because UnifiedPush doesn't require app (server) developers to know or care which service provider they will eventually be connecting to.

Secondly, for privacy and self-determination reasons, there should be a way to use our own choice of server, but not necessarily as a dedicated configuration option. A better way might be to tie it to the main Nextcloud account through a single-sign-on, so that when we set up the phone we have only to make that one choice of service provider and expect it to cover all services. That would give people the advantage of simplicity which is how the Big Techs play that game, but now with open standard technologies enabling us to have a choice among service providers.

When an app registers for UnifiedPush service, if more than one distributor is installed, it asks which to use. A normal user would have only one, and would not see this. Here as an experimentor and developer I already had both ntfy and NextPush installed:

It's not ideal that it identifies as “ntfy”, exactly the same as the official ntfy app. I expect this will soon be changed to /e/OS branding.

The tiny settings UI is found a the bottom of the system settings menu.

It's disabled by default. In this state, the UnifiedPush service is still advertised to apps, and they can register, but push messages are not delivered. (I suspect this isn't how we want the disabled state to be. Probably it should not advertise as available.)

That's all the configuration we get in this version of /e/OS.

(Aside: I couldn't find this in settings, to begin with. Searching in settings for “ntfy” or “push” or “unifiedpush” doesn't return any results, for me.)

Let's Hack In

As a developer we can access the ntfy UI through the system terminal, for example through an ADB shell (running this command on a USB-connected computer with “adb” tool installed, with ADB USB debugging enabled in the phone's developer settings):

adb shell am start -n foundation.e.ntfy/io.heckel.ntfy.ui.MainActivity

Just the same as in the user-installed ntfy app. (I can compare them side by side because I also have it installed, because I was already using it before this upgrade.)

We first notice this screen prominently shows some warnings about battery optimisation and choice of connection protocol. Are these warnings relevant to this version?, I wonder. (At first it showed only the first, and later both.)

And it lets us open its settings. In the ntfy settings we can choose our own server, and I can confirm this takes effect and works.

This screen also shows the currently subscribed topics — here, the “example” one — and after configuring it to use my own server, the displayed URL confirms that. (Generally one “topic” corresponds to one app.)

Ntfy Not NextPush?

I'm a little surprised Murena chose ntfy, seeing as their system is closely coupled to Nextcloud: there is also Nextpush (source on Codeberg, yay!), a Nextcloud-hosted UnifiedPush server module and its client app, that could be used instead. Perhaps they found ntfy is more stable or refined than Nextpush.

UnifiedPush developer S1m says the “Next version of NextPush will be a lot more stable, and I've rewritten the UI.”

The Future

While UnifiedPush is pretty good standard already, it is still maturing. Some important, recent improvements are making their way into the standard and into implementations like ntfy right now. These include full compatibility with the WebPush standard, and a better way to wake an app without it needing permission for unrestricted battery usage.

An adjacent recent development is DAV Push, an open standard for pushing instant updates to our shared files, calendars and contacts through CalDAV, CardDAV and WebDAV, which can also work through UnifiedPush.

To ensure people can exercise their right NOT to depend on Google, we have wider work to do. Integrating a UnifiedPush distributor is one useful step towards it. Beyond this, we need to be working with the developers of popular apps to get them supported, sharing this development across the freedom mobile OS ecosystem, deploying servers, ensuring reliability, and more. I write about this in a follow-up post: UnifiedPush — Wider Developments .

Conclusion

The details of this initial implementation seem to me to indicate an early preview release, with significant changes still needed. But that's OK: these technical implementation details can be changed.

Leaving that aside, the social side of this is amazing: it's making UP available to main-stream users. That's a big deal for the libre mobile ecosystem. Hurray!

Add UnifiedPush to /e/OS to make it possible for developers to avoid FCM and better support F-Droid applications — an /e/ community forum thread requesting UP support.
Source code of /e/ fork of ntfy: https://gitlab.e.foundation/e/os/ntfy-android
“Integrate ntfy in /e/OS” —merge request, including a simple settings UI (just enable/disable).
Romain Hunault of Murena is presenting it at Capitole du Libre 2024 this afternoon.

#unifiedPush #degoogled #awesomeFOSS #eOS #Murena #ntfy #mobiFree

Immich -- Awesome Open Source

Thu, 12 Sep 2024 10:00:00 +0000

In our family we look after our own photos — we don't want Google or any other company deciding what we can and can't do with them.

In the past we used various desktop/laptop based open source viewer software, with storage on local disks. More recently we have been running the awesome open source PhotoPrism, with its smartphone-compatible web interface and photo library management features. Although PhotoPrism is impressive judged on its own merits, and has indeed allowed us to manage our photos ourselves, it's just not quite as usable as we'd wish.

Next up is Immich.

Immich promises “partner sharing”, where for example husband and wife might choose to share their whole photo libraries with each other. Immich offers stronger photo library management such as tagging, metadata manipulation, and deduplication of identical photos. Immich performs face recognition. Immich viewer apps seem to be user friendly and slick.

What's Good, What's Not So Good

For viewer apps we use the official Immich app from F-Droid on our degoogled phones and tablet, and the Immich web interface which works well on both mobile and desktop. The general impression is slick. It's fun to see a selection from “N years ago”. It's nice that various features of the viewer, such as the side-bar options to view by folder and view by people, can be turned on or off in the settings, depending on how we prefer to use the system. The main “time line” view works very well. Other views such as the view by people, by tags, on a map, and so on are functional but much less developed.

The user interface could be improved a lot by allowing us to use the different selection criteria together: having begun to explore a region on the map, we then want to narrow the results by people or by tags. As it stands, the interface is an example of the all-too-common anti-pattern in which every time we select a new option the computer “forgets” what we just told it a moment ago. (Nearly every shopping web site is a dire example of that anti-pattern. We might filter our results by “category=hardware” and by “material=metal”, and then adjust our search terms, and watch the computer immediately throw away our carefully chosen filters.)

Face Recognition

Face recognition is impressive. I set Immich to scanning my existing library of thousands or tens of thousands of photos dating back twenty years, which took it a few hours on my few-years-old server. Immich created groups of faces that the algorithm considered to be the same person. These groupings were amazingly accurate. The few false matches were predominantly small blurry faces picked out of the background (but still the majority of these appeared accurate), or baby faces (where to be fair I can't tell the difference even among members of my own family), or where only a small part of a face could be seen (and even many of these were accurate).

For naming the people, Immich presented a list of faces, one from each identified “person” group. The faces it chose were unfortunately often a small blurry image. It would be helpful if it could choose a “clear” image as the representative of each “person” group. On clicking this image we get to see all the photos in which that same person was identified, and there we can set the name of that person, and we can also choose a different representative image. This view of all the photos containing that person doesn't indicate where in the photo the person was detected, which in some cases puzzles me as they don't seem to be there at all, but perhaps knowing what the algorithm “saw” would just give me somewhere to direct my scorn and derision at its folly and might not really add anything helpful to the overall purpose.

An enthusiast has created a better user interface for naming and managing the faces more quickly, all on one screen. It's the main feature so far of Varun Raj Manoharan's Immich Power Tools, and the demo video there shows it in action.

The partner sharing works by including both my photos and my wife's photos in my time line view. The same for her view, if it is set up reciprocally. That is a good start. It would be more useful if we could see which photos are “ours” and which are “theirs”. I would like to see them tagged with our initials, or perhaps distinguished by a colour or styling. I would like to be able to quickly show or hide each set, and to select by partner in the search.

I hoped the partners would be able to share the same categorisation, tagging, face recognition and so on. At present unfortunately this is not possible. It is discussed as a potential future improvement in Show Face Recognition for Partner #5089.

Because the current implementation of Partner Sharing isn't close enough, I am thinking about using just one “family” account in Immich, with both (or all) partners' photo libraries in it. That might be the best way for our family to use the existing capability. Whether this would suit us depends on whether we use the upload/sync feature from our smartphones. If we do, then Immich's sync feature would place all of our photos into the same library. (I haven't found a feature to separate or tag them on upload based on which camera-phone they came from, although there might be a way.) Currently we are not using Immich's upload: we are using 'syncthing' for that, copying our photos directly to our respective photo libraries on the server. Then in Immich we define these as “external” libraries (meaning they are outside Immich's built-in upload storage area).

Folders, Albums, Tags

I have a collection of (older) photos that I already organised into folders with descriptive names. I would like these to be shown as “Albums” or “Tags”. Immich imports tags if found in embedded metadata, and a few of my photos had these. Due to the sad state of standardisation of photo metadata, however, naming the files and folders is a more universal solution than embedding descriptions and tags in the photo metadata.

The way I would expect or wish to get albums and tags from my collection would be to specify a set of patterns that match my folder and file names, and to specify rules to derive album names and tags from these patterns.

There are open issues about potential future development of folder imports in Immich. In the meantime I might try the extension Salvoxia/immich-folder-album-creator noted below.

Extensions

Extensions I have noted to investigate further:

– immich-folder-album-creator — “Automatically create and populate albums in Immich from a folder structure in external libraries” – Immich Power Tools — alternative UI for face naming and management – Immich-Kiosk — “A web slideshow for Immich”

Freedom Software Ethics

In terms of open source “freedom software” ethics, Immich sits somewhere in the middle of the spectrum.

On the positive side, Immich is open source under the GNU AGPL v3 license: that's very good for maintaining the freedoms granted to its users. Immich has its own website at its own domain, immich.app, where it hosts its blog and documentation.

On the other hand, like too many other good projects, the development is hosted in Microsoft Github rather than in a freedom-respecting forge such as Codeberg or at its own address. Use of MS Github is a real and present danger to the open source ecosystem: freedom software lives in freedom-respecting software forges. And development discussion is invited inside the proprietary Discord instead of through a freedom-respecting method such as Matrix: see Discord vs community values. Immich in its website footer also promotes other proprietary silos (Reddit, Youtube) and does not even mention any open/freedom tech channels such as the Fediverse.

Immich advertise that their client app is available through the big silo stores (Google, Apple). They also also distribute it through the freedom-respecting app store F-Droid, for Android and android-compatible degoogled phones. They mention this in the docs but not on the main page.

#awesomeFOSS #selfHosted #degoogled

WhoBIRD -- Awesome Open Source

Thu, 27 Jun 2024 08:19:16 +0000

I recently learned something new about those tweety birds we hear in the tall trees all along our school run, cycling or walking past the golf course. Near the school end, lots of coal tits, a name with which I was barely familiar. Near our end, lots of wrens. I had an idea that wrens were rare and secretive and tiny, so it had not occurred to me they were responsible for that noisy cheerful chatter. Perhaps it's hard to spot one but they're loudly singing.

These I learnt last week helped by an app named “whoBIRD” which, as you may guess, listens for bird calls and displays the most likely matches. It's delightful to identify real birds, and surprisingly successful if they're not too far away.

It's also fun trying to imitate bird calls, and satisfyingly hard to get past's WhoBIRD's correct identification of “human whistle”.

My eye was drawn to WhoBIRD when I stumbled across it while browsing for another open-source app in F-Droid. It is a little volunteer/hobbyist app which matches the sound from the microphone against Cornell University's bird call data, displaying a name and a picture of the most likely match. That's all it does.

Another family member has been using an app named “Merlin”, I soon found out, so I tried it too. Merlin is Cornell's own app, using the same data source. By contrast with WhoBird, Merlin seems aimed at serious amateur birders for research data collection, professionally produced, multi-featured and slick — and on the other hand being a huge download and demanding sign-up into their club. (For a quick try-out I found that entering the fake email address “a@b.c” on every start suffices to access the sound identification feature with just names but no pictures or additional information.)

It's nice to have the choice, which Cornell University enabled by publishing their research results as Open Data in their BirdNET project so that others can re-use it in other ways. Birders and ornithologists interested in the sightings and soundings side of this should consider Merlin or the other BirdNET apps. Technologists and researchers on the data analysis side might like to study and modify whoBIRD. As a builder and tinkerer myself, I value open source software on principle. Already I am imagining how I or anyone could modify whoBIRD to do things like send a picture of each identification to my smart watch. Or send the identifications through my Matrix network to co-ordinate the listening among multiple devices left near open windows at home, with a separate display standing on the kitchen table.

#awesomeFOSS #freedomSoftware #degoogled

Troubleshooting UnifiedPush

Fri, 21 Jun 2024 05:39:46 +0000

Let's take charge of another part of our digital life! Must we be beholden to Google for delivering our push notifications? No! No longer! With apps that support UnifiedPush neither Google nor the app makers get to determine where our data goes and who gets to monitor it. We get the control, the freedom to choose.

To me this feels like a breath of fresh air!

You might find your way here while installing an app that uses UnifiedPush. Let's see how to make sure the notifications work properly.

The official UnifiedPush Troubleshooting page is, at the time of this writing, rather bare. (Perhaps, as an open source project, I can contribute to it. EDIT: Here's a preview I have submitted on Codeberg, that's becoming better than this article.) Meanwhile, read on.

Following my earlier articles such as Who Cares Who Delivers Our Notifications? and how to self-host UnifiedPush notifications for your Matrix server with ntfy, here are some tips for using a UnifiedPush-compatible app on your Android phone or your freedom-supporting (degoogled) android-compatible.

Install a Distributor to Enable UnifiedPush

Some U-P Distributors: ntfy, NextPush, Conversations, ...

The first step to using UnifiedPush is to get a UnifiedPush “distributor” app running and configured. A distributor is the one app that will be listening all the time on your device, on behalf of all the apps that want to receive UnifiedPush notifications. When notifications arrive, it will distribute them to the correct target apps (usually messaging apps).

Ideally, in the near future, freedom phones would include a UnifiedPush distributor built-in. For now, you have to install one.

You choose your distributor app once, the first time you want to start using UnifiedPush. The distributor works in concert with one UnifiedPush server (again, your choice, depending on what the distributor supports), so consider which server or service provider you want to use. Install the distributor app, and configure it according to its documentation. Some are preconfigured to use one particular server, while others allow or require you to choose a server.

The distributor must be working before any U-P-compatible app will be able to use UnifiedPush. If no distributor is detected, most apps will choose an alternative method, usually either polling or the proprietary Google system. However, if a distributor is installed but not working properly, notifications may not arrive promptly and consistently or at all.

Follow the instructions for your chosen combination of server and distributor, or start from the UnifiedPush Distributors page.

Troubleshooting UnifiedPush

Now we assume you have installed a U-P-compatible client app, perhaps a messaging app, and are expecting it to use UnifiedPush.

The general expectation is the app should “just work”, without even telling you, if it detects a functioning UnifiedPush distributor as described above.

How does this magic work? Three steps. First, your app asks the distributor on the same device for the details of your chosen UnifiedPush service. Second, your app passes those details on to its own server (wherever that is). Third, even after your app has disconnected from its own server or “gone to sleep”, your app's server continues to use those details to push notifications to your UnifiedPush server, which delivers them to your distributor, which delivers them to your app, waking it up.

Let's see what we can check if it does not seem to “just work”.

Check your app is supported and the right variant

First check that the client app you are using supports UnifiedPush. The UnifiedPush web site lists some of the first U-P-compatible apps. Some apps come in different variants, with for instance the variant available through the Google Play store using Google's notification system while the variant available though the F-Droid freedom-software store uses UnifiedPush. Some apps give you a choice if they detect both methods are available, and some choose one method without asking you.

Use the simple UP-Example app

Install the UnifiedPush troubleshooting app UP-Example from F-Droid. Use it to manually test UnifiedPush registration and operation: simply press the “register” button and then the “test” button, and you should immediately receive an example notification named “Test”.

If this simple test fails, you need to debug your UnifiedPush distributor app and its configuration.

(TODO: see troubleshooting for specific distributors 'ntfy', 'Conversations', 'NextPush', etc.)

Use advanced notification troubleshooting tools

You can find more advanced notification troubleshooting tools in some apps.

Element-Android and Schildichat-Android have a useful, active, troubleshooting tool in “Settings –> Notifications –> Troubleshoot Notifications”. In particular the “Gateway” and “Endpoint” steps of that troubleshooter should display your ntfy server's URL that it has discovered from the ntfy client app.

Check also “Settings –> Notifications –> Notification Targets” in Element or SchildiChat, or “Settings –> Notifications –> Devices” in FluffyChat. There should be one entry for each matrix client app that has enabled push notifications, and when that client is using UnifiedPush you should see a URL that begins with your ntfy server's URL.

In the “Notification Targets” screen in Element or SchildiChat, two relevant URLs are shown, “push_key” and “Url”, and both should begin with your ntfy server's URL. If “push_key” shows your server but “Url” shows an external server such as up.schildi.chat then push notifications are being routed through that external server before they reach your ntfy server, which might or might not be what you want.

Using your own UnifiedPush server

If you are running your own UnifiedPush server, of course you should also look at its logs and debugging capabilities. That is out of scope for this article.

These tips should help you resolve any issues

For further help, contact the most relevant or specific of the projects involved, such as your client app provider or your U-P distributor provider, or for more general help contact the UnifiedPush project, through the links provided here. (Note: The blog comments are for discussing the blog post, not for getting help.)

Once it is working, enjoy one more Google-free, self-determined aspect of your digital life!

Matrix: Your Hi-Fi Separates

Wed, 24 Apr 2024 11:00:00 +0000

An idea for explaining Matrix: it's hi-fi separates vs. a mini-hi-fi.

WhatsApp is a mini-hi-fi.

It's packaged just the way it is. Not working the way you want? Throw it away or put up with it. But worse: it plays only the music subscription it came with. You can't choose another radio station. Want to buy independent music? Nope. Lend albums to your friends? Only if your friends subscribe to the same manufacturer's club.

Look at those buttons on the all-in-one remote... That's all the control you get.

Matrix is Separates.

Choose your loudspeakers. Choose your cd/radio/tape/record players: they all play the standard music formats. Buy or borrow or create whatever music, audio books or news you like, from wherever you wish. It plays them all. Something not working the way you want? Replace or upgrade any part, or ask an independent repair shop or friend to adjust it. Lend it to your local club, hook it up to their big stage amp. No problem. It's standardised, it's a tool, it's yours.

Of course your friends are free to choose their own separates. Maybe neat and tidy like yours. Maybe extreme.

#matrix #awesomeFOSS

PubHubs Moderation update 2024-03: A Web Site

Thu, 28 Mar 2024 18:43:57 +0000

I put up a web site with details of my PubHubs work:

https://www.ph.trax.im

Under Moderation, there are subsections for the main work areas of the project: Draupnir+Pantalaimon, Disclosure of Identity Attributes, and Civilised Discourse.

That last area is the one I am currently working on.

#PubHubs #matrix #awesomeFOSS

Transfer Photos Between PC and Phone, Locally

Wed, 28 Feb 2024 12:40:06 +0000

“How can I transfer photos between my PC and my phone?”

perhaps to get a digital camera photo from the PC into a phone app
perhaps to get a few phone photos onto the PC for editing
perhaps any other kinds of files, not only photos
perhaps Internet access is slow

Here we are not looking at ways to manage our whole photo collection, we are looking at simple ways to transfer files between our devices.

What we're looking for is a local transfer method, the wi-fi equivalent of a USB cable connection. Actually we can use a USB cable and the phone storage may show up in our desktop file manager and we can copy files to/from the other device that way. (Probably/depending.) But let's look for a more convenient wireless equivalent.

It doesn't make sense to email our files to Microsoft's mail server or Google's or our ISP's mail server and then download them back into our home onto the other device, neither for privacy (they get to see our photos) nor for practicality if we have a slow internet connection. And it doesn't make any more sense to use any other server based method, such as iTunes or Dropbox, where the server is run by someone else outside our home.

What we need is either a serverless (peer-to-peer) solution, where the two devices talk directly over the local network connection, like they would over a wire; or else we could happily use a server-based solution if the server were inside our home and under our control. The latter is not crazy — a server doesn't have to be a new or powerful box, it could be just a tiny app running on our PC, providing this service. We'll come to that later. First, let's look at peer-to-peer solutions.

I would expect of computing devices in the 21st century that one should be able to transfer files in a standard way, over whatever connection method the hardware supports — USB, Bluetooth, WiFi, and more. I can't believe this isn't standard functionality on every device nowadays, but such is the power of Apple and Microsoft and Google who want us to use their services for everything instead of looking after our own data ourselves. (Thanks for tolerating me getting that off my chest once again.)

PairDrop: Peer-to-Peer Transfer as a Handy Web App

Here's a peer-to-peer solution. That is, a way to send from one device to the other without going through any middle-man server.

“PairDrop” is a nice Freedom-software web-app to do this. (I always look for free-as-in-freedom software, constructed to values that align with mine, such as not stealing or mining my data.) We can load it here:

https://pairdrop.foad.me.uk/

It's a serverless web app, which means the application code is all in the web page. There is no attached server behind this, no database or anything on my side. When we open this web app, it just finds our other device(s) on our local network and transfers the files directly between them. It doesn't send our files or anything about us outside our local network. (I didn't create this software, I just deployed a copy of it. It's also available at 'pairdrop.net' and there are other similar ones with similar names.)

To get started, open that web page, in a web browser, on both devices. Each one should show an icon at the bottom, with a made-up name for itself, and an icon in the middle of the screen representing the other device once it sees it in the local network. If they don't both show the other device, refresh/reload the page. Click that other device icon to choose files, and it sends them to that device. On the receiving side, it presents the files ready for you to save to a folder.

Getting a little more advanced, you can transfer a folder or arbitrary selection of files in one go. You can use multi-select in the file picker, or drag-and-drop on the desktop, or “Share” function from a phone's file manager. PairDrop puts multiple files in a zip file and assumes (I think) you already have a zip file extractor installed at the other end, which you probably already do on a PC, and you can easily install a zip extractor app on your phone if there isn't one yet.

A (Little) File Server Inside Our Home

Now for another way.

As we said earlier, a “server” need not be a big new box, it could be just a tiny app running on our own PC, providing a service. The server software won't care if one of the clients requesting a file transfer is another application running on the same PC, while the other client is a phone or another PC.

There is a simple built-in file server in Windows and most desktop operating systems, which can be enabled by the “file sharing” feature. Method 3 in a Guiding Tech article has step-by-step instructions with screenshots, for setting up a Windows file share and accessing it from an iPhone's Files app. Once we get that connection set up, we can copy files in and out of the shared folder on either end. I just found this article from a quick search and am not endorsing it but at a quick read through it looks generally sound.

(If you are interested in running a more long term and dedicated home file server, you could look at Nextcloud or other options. However, that's another topic.)

Continuous Sync: SyncThing

What I do for most of my file synchronisation needs is install Syncthing on both devices, and configure it to sync my phone's “DCIM” folder with a same-named folder on my laptop, and similarly for my “Documents” folder on both.

This kind of synchronisation runs locally (within the local network), continuously, in the background, whenever the devices are on and connected to WiFi.

For now I refer you to Syncthing's documentation at https://syncthing.net/ to learn more about this method.

#awesomeFOSS

Matrix Needs: Homeserver Local Discovery

Tue, 27 Feb 2024 21:16:31 +0000

Towards Matrix, Properly Decentralised

One of matrix's key properties is decentralised communication.

No single party controls the whole system, there is not a single service provider of whom everybody is a customer, anyone can choose who provides their service or run it themself.

When I install a Matrix client app, I expect it to connect to my Matrix server. What happens currently: most matrix client apps invite me to connect to one particular de-facto “central” server called “matrix.org”. As a user, connecting to my own server is a chore. In Element's flagship “Element X” preview, for example, currently I have to choose the less obvious buttons to skip past screens inviting me to choose the “matrix.org” server, twice, and input my server's domain on a “search” screen separate from where I enter my “username”.

What matrix client apps should do: discover my local server automatically, make a reasonable suggestion for my username (e.g. from my computer's or phone's user account details, address-book, etc.), and move on to asking me for my password.

It would be nice if we could standardise a way for a local matrix homeserver to advertise itself on the local network, and for a client to detect the local homeserver(s) automatically. Home media servers do this, and some other kinds of home servers. Technically there are a few different ways to achieve Zero-configuration networking, like UPnP, mDNS, DNS-SD.

A different and complementary approach is to integrate with the accounts already configured in the phone's settings (on Android for example). I should be able to configure one matrix account in one place, and then any matrix client app I install should pick up the settings from there.

Which technique(s) we use is not as important as that it should Just Work, for ordinary low-tech people who have not set up their own infrastructure but are perhaps subscribing to a matrix server provider.

(Traditional “enterprise” deployment techniques are still valid in an enterprise setting, for example where the enterprise IT department configures the employees' devices. But that is something they have the resources to develop and deploy. I am interested in what we can do for ordinary individuals at home.)

It seems to me that's the way things should Just Work, everywhere from homes to enterprises, if we care about decentralisation.

#matrix #awesomeFOSS

awesomeFOSS — julian

UnifiedPush -- Wider Developments

How We Get There

OS Integration of the UP Distributor

Server Software

Server Hosting: Technical

Server Hosting: Social/Business Aspects

Apps

Sharing with other mobile OS projects

Standardising UP client-server protocol?

UnifiedPush: Standardise UP Client-Server Protocol?

UnifiedPush High Level Architecture

Multiple Protocols or Standardised Protocol

Going Google-Free with UnifiedPush in /e/OS

So What?

Technical

If not Google's then Whose Server?

Let's Hack In

Ntfy Not NextPush?

The Future

Conclusion

Immich -- Awesome Open Source

What's Good, What's Not So Good

Face Recognition

Partner Sharing

Folders, Albums, Tags

Extensions

Freedom Software Ethics

WhoBIRD -- Awesome Open Source

Troubleshooting UnifiedPush

Install a Distributor to Enable UnifiedPush

Troubleshooting UnifiedPush

Check your app is supported and the right variant

Use the simple UP-Example app

Use advanced notification troubleshooting tools

Using your own UnifiedPush server

These tips should help you resolve any issues

Read More about UnifiedPush

Matrix: Your Hi-Fi Separates

WhatsApp is a mini-hi-fi.

Matrix is Separates.

PubHubs Moderation update 2024-03: A Web Site

Transfer Photos Between PC and Phone, Locally

PairDrop: Peer-to-Peer Transfer as a Handy Web App

A (Little) File Server Inside Our Home

Continuous Sync: SyncThing

Matrix Needs: Homeserver Local Discovery

Towards Matrix, Properly Decentralised