Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
Get ready for another Hookshot release! The flagship new feature is the ability for Hookshot to respond to webhooks with custom response data, once Hookshot has handled the webhook message. This feature exists because while some services expect an immediate response to webhooks, others wait for the request to complete. With this release, Hookshot can now handle both cases. For more details, refer to the documentation.
The release is available at https://github.com/matrix-org/matrix-hookshot/releases/tag/4.6.0, or by doing docker pull halfshot/matrix-hookshot:4.6.0. And as usual, feel free to direct any questions about Hookshot in our Matrix room: #hookshot:half-shot.uk
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
Cross-posting from TravisR's announcement in the Office of the Spec Core Team room...
Matrix 1.9
The next release, Matrix 1.9, is scheduled for release on Wednesday, November 29, 2023. This will largely be a maintenance release, but if there's anything you think we might have missed, please let us know.
Matrix 1.10
The Spec Core Team is now accepting MSC and feature suggestions towards the Matrix 1.10 release, due sometime around FOSDEM in February (unsure if it'll be before or after). So far we've got the following features up for consideration:
Extensible Events (at least proposing FCP on the majority of the core system; plan to merge for Matrix 1.11)
Encrypted appservices (like MSC2409)
Custom emoji/stickers (merge)
If there's specific MSCs, or even vague ideas you'd like the SCT to focus on, let us know in the Office of the Spec Core Team room. The SCT will generally find it harder to actually write the MSCs ourselves, but if you're planning on implementing a feature that will need spec review, let us know so we can ensure bandwidth is made available for that review to happen.
This MSC is paving the way towards the use case of a room where, for instance, announcements can be made by admins, while general users can start or participate in a thread branched off from an announcement. But they otherwise would not be able to make "top-level" posts.
This feature is common in services such as Telegram, for announcement or support rooms. And it would be nice to be able to support the same kind of use case in Matrix.
It looks like I dropped the ball on keeping up with the discussion on this one. So excuse me while I go do that!
Hello! today we are announcing the latest minor update, v0.1.3! This contains mostly small bug fixes and UI Improvements, as well as a few quality of life features!
What's New?
Rooms and spaces shared between multiple accounts are visually distinguished
Improved HTML Rendering
Implemented lazy loading of room members
Zoom and pan on images and video
Background task indicator for downloads
Direct messages in spaces are now visually distinct from rooms
Bug Fixes
Fixed an issue where a room shared between multiple logged in accounts wouldn't open correctly
Fixed notifications being displayed for old messages
FluffyChat v1.15.1 has been released with a lot of bugfixes and updated translations.This release also finally makes it possible to select text in the chat on web, desktop but also on mobile. Please be aware that the default long press behavior to select events has changed to normal taps because of this.
For the next release you can also expect the return of presences. While this feature is disabled
on matrix.org, it seems to work fine on other servers. The app tries to filter out users from servers where presences are disabled to
not display them as always-offline. I hope this works fine. As far as I can see, the difference is that on server with presences
enabled there is always a „last_active_ago“ field in the presence content. So the app assumes that, if the user is offline and
last_active_ago is NOT set, then it is better to just not display the presence status at all.
Element X Android 0.3.1 is available on the PlayStore.
We spent the week fixing bugs and improving our codebase. We are also adding the read marker and read receipt rendering to the timeline and are also progressing on the mention epic.
Jorge also made a big improvement on the release time for the Rust SDK. It now takes around 20 minutes, previously it was up to 5 hours!
We’ve tightened scope and started reviewing feedback and iterating on the new room header and right panel designs. Keep your feedback coming, we’d love to hear it!
If you don’t have the new header already, go ahead and try it out on Element Develop by using the labs flag: “Under active development, new room header & details interface”
The team’s been working on streamlining the process we use to release Element Web. We’ve simplified it and are working through different testing tools to ensure that we’ve got our automated tests running smoothly.
Our week has also been focussed on reducing the number of failing notification tests by fixing bugs and working through the test suite.
Another full release 0.23.0 of is out as of a few minutes ago (right as I'm typing this), with some nice bugfixes, nicer features, and important progress for a whole lot more (I'll give you a hint: 🧵)
MatrixJoinLink is bot that allows the creation of invite links to non-public rooms in matrix.
Several months have passed since I initially introduced the bot in TWIM.
In the interim, I've seamlessly incorporated the bot into various work related contexts as well as in private contexts.
The bot addresses the challenge of being unable to invite individuals via a link to a non-public room or space.
Since my last announcement, I further improved UX. The bot is now able to deal with various room identifiers (i.e., matrix.to links, internal room ids, and public room aliases).
How to use JoinLink
MatrixJoinLink simplifies the process of inviting users to non-public spaces.
Here's a guide on how to make the most of it:
Invite the bot into your room.
Once the bot joins, use the command !join help to explore its features. Please note that the bot responds exclusively to users or servers configured by the bot owner.
To create a Join Link, kindly type !join link Demonstration_Link, and the bot will promptly generate a join link for you. Ensure that the bot has the necessary permissions to invite users.
Help
Creation of Join Links (with Developer Mode Details)
Entering a Join Link Room
Feel free to test the bot and raise any issue using GitHub or using the development room that is mentioned in the project's README. Also consult the README for technical details.
a heads-up that the Matrix.org Foundation & Community Devroom has been accepted at FOSDEM24
You can find the CfP here, and submit your talk propositions before December 4
Centralize the Matrix Community at the Chaos Congress
As you may know, the 37th Chaos Congress takes place in Hamburg, 27.-30.12.2023. If you don't know, the Chaos Congress is a collection of cool people mostly from the different Chaos communities to talk about interesting topics ranging from technology all the way to politics. You will however need a ticket, which will only be available in very limited quantities.
Similar to the what we already did at the Chaos camp, we have applied for an assembly at the congress. An assembly basically is a few tables with people centered around a topic, where you can join to chat, participate in workshops or even get someone to fix or setup your Matrix homeserver!
Currently the assembly is not yet approved. However we are announcing it now, so that you can give it some thought if you want to participate and maybe help shape the Matrix Community assembly!
Since the proposal isn't public yet, there is no link to share to find more information about it. But if you are helping to organize the assemblies, you can find it in the backoffice.
So for now (and for later), if you want to discuss it, suggest things to do or have other comments, feel free to join us in #chaosevents:matrix.org!
So far we don't have a program yet, but maybe you have a cool workshop you want to do in our small round? Or maybe you just want to hang out and maybe answer questions people have about Matrix? Or maybe you just want to meet some Matrix developers, that you have always looked up to, to complain about how this one button is slightly of center when running their client on your PDP11?
In the end YOU can decide what you want this assembly to be, but now at least you know about it (if it gets approved)! Further updates will follow as we make progress and when we think you may have forgotten about us!
As of today, 8251 Matrix federateable servers have been discovered by matrixrooms.info, 2068 (25.1%) of them are publishing their rooms directory over federation.
The published directories contain 274767 rooms. Do you know what room isn't one of them? There are no rooms about backpacking among them, so if you're wondering "what an interesting room I can create to find great people to socialize with" - a room about backpacking would be a great choice!
Building on last year's success, the Matrix.org Foundation is excited to host a Matrix.org Foundation and Community devroom in person this year again at FOSDEM. Half a day of talks, demos and workshops around Matrix itself and projects built on top of Matrix.
We encourage people working on the Matrix protocol or building on it in an open source project to submit a proposal! Note that companies are welcome to talk about the Matrix details of their open source projects, but marketing talks are not welcome.
The Matrix.org Foundation is proud to join with Mozilla, the Electronic Frontier Foundation, and over 400 security experts, researchers, and NGOs in sounding the alarm about the EU's proposed eIDAS reform.
As written, the proposal radically expands the ability of governments to surveil their citizens and residents across the EU.
We encourage citizens and residents of the EU to pay close attention and look for ways to get involved – urgently.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
IETF 118 is happening next week, and as per usual we'll be there participating as the Matrix.org foundation! We're working away at drafts in the hopes of converging with the MIMI design team and working group on a new MIMI protocol. Said protocol will hopefully allow the various gatekeepers and other players to have interoperable instant messaging, under the requirements laid out by the EU's Digital Markets Act.
Travis, Matthew and others have been spearheading this work in various areas, with myself helping out on the content format side of things. More to come post-IETF 118!
This is an MSC written by yours truly! I have no influence over what the script chooses, the script is it's own master.
This (quite old now) proposal aims to argue that the client_secret parameter on various third-party ID-related endpoints in the spec is not necessary. As we already have a server-generated session_id which can be used to identify a particular client's session and validate that the client is allowed to interact with the session (by way of knowing its ID).
Essentially, we have two values in play where we only practically need one. This spec aims to simplify things, as well as reduce the amount of client-controlled parameters are involved in third-party ID processes (which can be a bit finicky as they often interact with external systems).
I have yet to update this or really do anything with it for the past few years. But it exists, and now you know!
This week released a patch release (v1.95.1) that fixes a moderate security issue in Synapse. See the advisory for more details.
We also released an RC (v1.96.0rc1) with some new module APIs, experimental support for running multiple receipt workers, and the usual tranche of bugfixes.
The team has also been looking at performance improvements, and some low-hanging fruits have already landed on develop.
This week was mostly polishing trying to make the Qt6 version of Nheko stable enough for a release. However I also got to close some ancient issues like search stopping pagination randomly.
One fun issue was, that you currently can't sign in with any mxid longer than 128 characters... When we first started to persist data in Nheko (before I joined the project), we ensured the database was unique by storing the database in a file named after the hex encoded mxid. However most filesystems have a file name limit of 255 characters or so... However, we are not in 2017 anymore, so we switched to hashing the mxid, which ensures the database name will always have a constant length, that is legal on most filesystems (sorry, no DOS support for you!).
Another fun change is that Nheko now reduces how often it syncs, when the window is unfocused for a while. This decreases how much power Nheko uses on my system by quite a bit, since it used to sync 5-10 times a second before just to process typing notifications in some unimportant room... Now it will at most sync once a second (if the window is unfocused for 30s or more). This means you will still receive important messages immediately unless someone was typing immediately before it, in which case it might be delayed by up to one second. I guess most people won't even notice this change.
We also tried to get a working msix package of Nheko. The biggest blocker currently is signing it with a useful certificate, but there are also a few minor issues like notifications not showing the pretty app name and the protocol handler not forwarding the url. If we can resolve those, we might provide an MSIX in the future instead of the traditional installer, which should allow for automatic updates on Windows, might make Nheko on the Windows Store possible and is also a smaller package with more security options. Half of the time was spent on reducing our compile times though, since with this new build step we hit the 1 hour build time limit quite a lot (on a clean cache run).
Nheko now also uses im.nheko.Nheko as the appid on all platforms and there are also some fun, non-code things happening behind the scenes, which might show up in TWIM in the future!
I wish you all a wonderful weekend and maybe give the nightlies a go and report regressions you want fixed before a potential release!
This week we’ve been focused on releasing version 1.4, which is now available on TestFlight and should be making its way to the App Store next week. The following features have been enabled:
User Mention Pills and Suggestions.
Voice Messages.
Starting a Call from inside a room.
Screen Lock with a PIN code, Touch ID or Face ID.
This version also lays the groundwork ready for us to enable key backup support and you may find that after cross signing a fresh session, some parts of the backup start to work in the background (no promises though).
This week we're back on track with notifications! Next week's release will contain fixes for problems that happen when messages are deleted, and the following release will contain fixes for editing messages.
On the new room header and right panel we’re nearly ready for a ‘proper’ launch. Releasing the new version to all users, not just those on Develop or Nightly. We’ll continue to iterate once the first version has landed but we’re really proud of the new design direction.
And, work progresses on our project to automate some of our CI processes!
We are pleased to announce that NeoBoard had it's first major release, v1.0.0!
NeoBoard is an open-source collaborative whiteboard widget for Matrix! It's an ideal tool for creating and sharing content, conducting presentations, and collaborating with colleagues or friends. All data is stored as Matrix events within the Matrix room, leveraging Matrix's native end-to-end encryption, decentralization and data privacy features.
This update is our first stable release and includes some new features:
Buttons to easily navigate to the next and previous slide in a presentation
The JavaScript package for libolm is now available at npmjs.com. This means that your package.json files no longer have to include the full URL for the package, and can just include the version specifier. This was the last package type that was relying on the gitlab.matrix.org package registry.
This week, we did a lot of bug fixing and refactoring, notably compile times and binary sizes should be down due to the trimming of our dependency tree. We also made a bunch of progress on key backup support, with the crypto-crate level support being moved out of its feature flag soon (stabilization of the functionality in the higher-level crate will follow).
v0.7.0 is here! The bot-sdk is generally fairly stable as it's a very low-level SDK over top of the Matrix Client-Server and Appservice APIs, meaning it gets few releases. This might change in the future, but for now its twice-yearly release has happened 🎉
This release's highlight is a PostgreSQL storage provider to accompany the existing filesystem storage provider. Like SimpleFsStorageProvider, it's not great, but it does work. Other changes and details are available in the changelog: https://github.com/turt2live/matrix-bot-sdk/releases/tag/v0.7.0
As of today, 8679 Matrix federateable servers have been discovered by matrixrooms.info, 2096 (24.2%) of them are publishing their rooms directory over federation.
The published directories contain 272740 rooms.
A couple months in, it’s a joy and a privilege to introduce myself as the first Managing Director of The Matrix.org Foundation! Read my intro post to learn more about me, the work ahead, what you can do to help, and how the leaders of other open source foundations are reacting to my appointment.
I’m thrilled to take this opportunity to also announce the Office of the Matrix.org Foundation, a new room that offers a direct line to me and the rest of the small but mighty team behind the Foundation. Come say hi! 👋🏻
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
MSC Status
New MSCs:
There were no new MSCs this week.
MSCs in Final Comment Period:
No MSCs are in FCP.
Accepted MSCs:
No MSCs were accepted this week.
Closed MSCs:
No MSCs were closed/rejected this week.
Spec Updates
Pretty quiet in spec land this week, while IETF/MIMI continues to whir away in the background.
In other news, the SCT expects to start focusing on the custom emote MSCs in the very near future! ✨
This MSC is fairly straight-forward, proposing the addition of two new endpoints to allow you to immediately cancel a 3PID verification session on an identity server (instead of waiting for the automatic 24-hour timeout). This is particularly helpful if you put in the wrong email/phone number on your first attempt.
I don't think Identity Servers get much love today, but this sounds like a no-brainer to add. Go forth and comment on if it sounds interesting to you!
Let me start right away with no less than a sneak peek at Matrix 2.0. Matthew walks us through the most exciting MSCs currently in flight with working implementations, that will be drastically changing the experience on Matrix.
One of those MSCs makes Matrix support OIDC. But Synapse already supports OIDC, right? If you want to learn more about the difference between OIDC and OIDC (sic) and how it's going to impact you, the auth team and I wrote a post for you.
Before we explain all about our new authentication system - don't panic! This change is part of Matrix 2.0, the next step for Matrix that will be introduced in a blog post later today.
Nothing breaks
Matrix 2.0 is a reference to Matthew’s FOSDEM talk Matrix 2.0, itself a reference to the name he coined in the Matrix Summer Special 2022: evolutions of the spec that are making the Matrix experience so much better it almost feels like a different protocol. One of the major changes in this bundle is the introduction of OpenID Connect-native Matrix and its implementation matrix-authentication-service (MAS).
Don’t panic: your current setup is not at risk, you don’t urgently need to deploy the matrix-authentication-service to keep using Matrix.
Let’s walk together through what OIDC-native Matrix, what changes as compared to the Matrix we know and love, and what’s the impact for your deployment.
From OIDC-ish to OIDC
One of the most popular Matrix server implementations, Synapse, supports authentication via OIDC. This is the only mainstream server implementation to support authentication via an OpenID Connect Provider. OpenID Connect, SAML, and more generally Single Sign On (SSO) are a requirement for most companies past a certain size, as well as a security best practice.
As it is today, Synapse is more OIDC-compatible than OIDC-native. This means that it’s possible to use SSO (typically OpenID Connect) to connect to Synapse, but that’s about where the OpenID Connect adventure stops. Once you’re authenticated, Synapse generates a Matrix Access Token for your client, but that is not an OAuth2/OIDC Access Token. From your client point of view, it is doing the “Matrix SSO dance” defined by the spec, but the fact that Synapse does a login via OIDC, SAML or CAS is irrelevant to the client. Whenever your client asks Synapse for anything that requires being logged in, it sends that Matrix Access Token in the Authorization header of the http requests.
While this approach has served Matrix well, the Matrix Access Token method is reimplementing some concepts of OpenID Connect without all the benefits of thousands of developers battle testing and fixing every edge case they meet.
MSC3861, which proposes the adoption of OIDC in Matrix, is all about embracing the best of OIDC to make Matrix even better. By adopting the standard OIDC flows we allow Matrix to stand on the shoulders of another battle-tested industry standard. Not only does it improve security overall, it also unlocks new use cases for Matrix.
Unlocking use cases
The Matrix Access Tokens Matrix currently relies on are secure, but not very flexible. A unique token is granted to each and every client during user login. This Matrix Access Token is only revoked when the user manages their sessions and logs a device out. A Matrix Access Token gives full access to a Matrix Account. Session management based on Matrix Access Tokens is quite nuclear.
Using OpenID Connect for Matrix comes with three major areas of improvements:
Standard authentication process on every client, including on devices without a keyboard
Improved security with token rotation
Better granularity of permissions (which improves security too!)
Authenticate like you want
When relying on OIDC, the client delegates authentication to the OpenID Provider. This means that the client redirects the user to a web page on the OpenID Provider, where the user needs to authenticate. This authentication can happen through a username and password, WebAuthn (passwordless login), additional steps if you want to add MFA… and much more.
Since the authentication happens on the OpenID Provider, the client doesn’t need to support any authentication method other than “redirect the user to the OpenID Provider, and handle the result”. Once the user has authenticated against the OpenID Provider, it is going to be redirected back to its client, to allow it to retrieve an Access Token on their behalf.
During the whole process, no password was given to the client, and the client doesn’t even know how the user authenticated. All that matters to the client is that it now has an Access Token it can use to perform the regular Matrix API calls.
From the user perspective, the authentication is a very familiar process very well integrated in their password manager regardless of the client they use. They can use any client that supports OIDC, without having to worry about whether it supports every particular step of their authentication process. It’s also worth noting that matrix-authentication-service has a compatibility layer to support the m.login.password flow. This means compatibility with older clients will not break!
From the administrator perspective, it’s possible to force authentication flows (e.g. MFA), making sure the login is secure and matches their organisation’s policies. It also makes it possible to have a central management of all users' devices, including the ability to enforce policies upon them as provided by the OpenID Provider. For example, it becomes possible to the re-authentication of the user if they leave a trusted network.
From the developer perspective, neither clients nor servers have to add support for every new authentication method, existing or to come.
Element has been spearheading the effort to implement MSC3861 and matrix-authentication-service, and Element X is the first client to support them. Here is how the authentication flow and account management look like as seen from Element X iOS.
Token rotation
While current Matrix Access Token can expire thanks to MSC2918, this best practice didn’t make it to most of the clients of the ecosystem. If the access token of a user was leaked for one reason or another, they wouldn’t notice since this wouldn’t create a new session, and the leaked token would be valid forever. Please note that thanks to E2EE, attackers who gain access to a Matrix Access Token would not be able to read the encrypted content in encrypted rooms.
OpenID Connect relies on two tokens:
an Access Token, that is used to perform the authenticated API calls, and that expires regularly,
and a Refresh Token that is used in conjunction with the Access Token to rotate both.
This ensures that if the Access Token was leaked during an API call, its validity would be very limited in time. Leaking a Refresh Token will also have limited impact, since the server would notice that two different clients are trying to consume the same Refresh Token and would log the user out from devices using this token.
Principle of least privilege
Matrix is not just about apps and servers. It’s also a vibrant ecosystem of widgets, bots and various CLI to toy with. One of the strengths of Matrix has been its hackability: log into a client, dump the matrix access token, and you can hand it to a CLI or a bot for it to do things on your behalf.
While this approach is particularly convenient for development, it comes with its own lot of issues. Most of the time you don’t want the CLI or bot to be able to do everything on your behalf.
Widgets are another case where OIDC can help. While widgets are not yet part of the spec, in practice they are already commonly used. In essence, a widget is a tiny app that has access to an (unspecified) widgets API that allows it to do a limited amount of things on behalf of the user, such as sending a message in the rooms it’s in. While this has served several clients well, this can be improved. Indeed, it’s up to the client to enforce the restrictions mandated by the widgets API. This means you can’t be assured that all the clients will consistently enforce them.
With OpenID Connect, widgets can eventually become tiny embedded Matrix clients on their own, and they can get their own Access Token with explicit and granular permissions of their own. Those restrictions will be enforced by the server, who will not even be aware it’s a widget calling the API, making the experience consistent across clients. This behaviour still needs to be added to the specification, but a more OIDC-native Matrix paves the way for it.
Upgrading to use matrix-authentication-service
MSC3861 (and friends) define how the OIDC flows and concepts (e.g. token exchange, dynamic client registration) should be used in Matrix, as well as the requirements for things which need to be coordinated with the homeserver (e.g. session lifecycle management).
Because MSC3861 is changing the technical mechanism by which your client gets access to your account it is not something that can be done by pushing a single, big button.Instead we need a transition period where both OIDC and the existing legacy Matrix authentication methods coexist so that different clients and homeservers can migrate at their own pace.
One way in which MAS facilitates this transition is by allowing deployments without an external Identity Provider (IdP) to use OIDC out-of-the-box. This means that MAS will support some of the current flows that Synapse supports, primarily in the form of password-based login. Please note those flows are not supported yet: MAS requires an external IdP for now.
Another way is that MAS allows for existing sessions and access tokens to be imported (just from Synapse so far) so that there is no need for clients to re-authenticate when a homeserver is migrated.
Because of these migration capabilities we expect MAS to be widely used, however, in keeping with the open philosophy of Matrix, we expect other implementations to exist either standalone or integrated with other OIDC providers like Keycloak or Dex.
MAS is not intended to be a fully fledged IdP solution: we suggest using a dedicated software for this if you want to do more than what is included.
Currently MAS is deployable as a sidecar service alongside the homeserver. While larger setups shouldn’t see this as a problem, it might be seen as cumbersome in smaller self-hosted setups. For the sake of convenience, the Synapse and MAS team are investigating how to bundle MAS along with Synapse natively using PyO3. We anticipate such a bundle to exist by the end of Q2 2024.
