Josh Simmons announces
Have you seen the Stack Overflow Developer Survey results for 2023?!
Of the 83,830 folks surveyed, Matrix was the #1 chat tool in terms of current users' satisfaction. It was also rated as the most desirable among the open source tools with open governance, but there is a lot of room for improvement in awareness. We’re excited to build on this 2024! 🚀
No Matrix Live this week, but a YouTube playlist of the Matrix talks given at FrOSCon!
Andrew Morgan (anoa) announces
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
MSC Status
New MSCs:
- There were no new MSCs this week.
MSCs in Final Comment Period:
- MSC3981:
/relationsrecursion (merge)Accepted MSCs:
- No MSCs were accepted this week.
Closed MSCs:
- No MSCs were closed/rejected this week.
Spec Update
For those familiar with Travis' weekly task lists of MSCs for the Spec Core Team to review in the Office of the Matrix Spec Core Team room, a new weekly list is now being posted in the Matrix Spec & Docs Authoring room. This list is aimed at technical writers who can help by converting MSC authors' words into PRs against the spec text itself.
This is the final step for getting an MSC integrated into a new release of the Matrix spec, and anyone can try their hand at it! It would also very much help the Spec Core Team by freeing up more bandwidth for review of the MSC backlog, as well as push forward the protocol itself. Thank you!
If you have any questions, feel free to ask them in the relevant Matrix rooms.
Random MSC of the Week
The random MSC of the week is... MSC4003: Semantic table attributes!
This MSC proposes expanding the set of suggested, interpreted HTML tags in Matrix clients to include additional tags related to tables. With them, more control over table rendering is possible. The proposal itself includes one such (albeit fairly arbitrary) example
The proposal is well-written and straight-forward, so do feel free to have a look if the subject interests you!
This year again the Matrix Foundation and Community will have plenty of opportunities to meet at FOSDEM! Together with our awesome community, we’re organising a FOSDEM Fringe Event before FOSDEM itself, we will have a booth to meet everyone and spread the word about Matrix, and a devroom to go more in depth on Matrix topics.
Josh Simmons reports
We’re excited to announce the timeline for our first ever elections as we take the next big step in open governance for Matrix. We’re also introducing two new membership tiers to increase community representation by including open source projects and foundations on the Governing Board.
Read about this and more in our latest blog post.
Matthew says
I dusted off MSC4016 (Streaming and resumable E2EE file transfers) over the holidays and turned it from a WIP into a full MSC, complete with draft proof-of-concept clientside implementations for the streaming attachment fileformat (hypothetical 'v3 attachments') at https://github.com/matrix-org/matrix-encrypt-attachment/pull/26 and https://github.com/matrix-org/streaming-files-test. Some of the benefits this would bring include:
- zero-latency file transfer (receiver can start decrypting & playing the download as soon as the upload begins)
- halving file-transfer duration (as you don't need to wait for the upload to finish before you start to download it)
- resumble uploads via tus
- unlimited file transfer size, given it's streamed - no longer limited to ~1GB to fit inside v8's 1.4GB heap on Web.
- O(1) memory usage for sending files, given it's streamed (great for share extensions)
It works by switching from AES-CTR to splitting the file into blocks, and encrypting each one as AES-GCM, muxing into a very simple file format which provides basic headers to allow framing for incremental decryption and random access. Given this changes the security model of attachments significantly, and given the fileformat is homemade, this MSC is now blocked on needing some serious cryptographic review before it proceeds any further :)
Matthew says
I just came across https://benstokman.me/blog/my-thoughts-on-msc-4080/ and https://benstokman.me/blog/my-new-thoughts-on-msc-4080 - an interesting discussion on the design of CryptoIDs (Client-Owned Identities - MSC4080); the cornerstone of mainstream account portability in Matrix. (Account Portability work is currently paused for a bit, but it shall return...)
Andrew Morgan (anoa) announces
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
MSC Status
New MSCs:
- There were no new MSCs this week.
MSCs in Final Comment Period:
- No MSCs are in FCP.
Accepted MSCs:
- No MSCs were accepted this week.
Closed MSCs:
- No MSCs were closed/rejected this week.
Spec Updates
It's been a quite week from the perspective of the MSC Status list, but there has definitely been activity across various MSCs (at least according to my inbox). MSC3981 has seen discussion, as has MSC4041 and MSC2448 (we're considering switching the proposal to Thumbhashes).
More to come next week I suspect!
Synapse is a Matrix homeserver implementation developed by the matrix.org core team
Andrew Morgan (anoa) reports
Happy New Year!
Following on from the release of v1.98.0 (the final release of 2023), as well as seasonal holidays, we expect to return to our usual 2-week release schedule. Starting with a release candidate on Tuesday, January 9th.
As a reminder, Element is switching its development on Synapse to be AGPLv3 at https://github.com/element-hq/synapse, and development from Element's side now continues there (see the blog post if you haven't already). We have done our best to migrate issues from the now-archived matrix-org/synapse repo over, as well as preserving issue and PR numbers. PRs were not migrated, and their PR number has been reserved by a stub issue instead. Huge thanks to Erik and WillL for figuring out the plan there and implementing it (and working with GitHub's heavy rate limits on issue creation!). As well as fixing up CI, packaging, etc. etc.
As things have been a bit topsy-turvy with the migration (and of course the holidays), there's not a huge list of changes to report from the development side. But we did have a few excellent community contributions.
Thanks to @dklimpel, @loelkes, @lanker, @FadhlanR, and @adamjedrzejewski for their PRs!
Expect an increased pace of developer pick right back up from next week, as folks return from holidays.
A multi purpose multi platform bridge, formerly known as matrix-github
Half-Shot says
matrix-hookshot 5.1.1!
Hello wonderful people. I hope you all are having/had a good break. Over in bridge towers we've been working on a few holiday treats in the shape of hookshot improvements. Let me lay them down for you:
- dark mode 🌗 is now a thing! We've moved the majority of the components used in the widget to use Compound, which means a more coherent style and light/dark support. It's still early days so please log bugs as you find them. Also, the dark mode is triggered by whatever the browser believes is your operating systems preference, so currently we don't match any overrides your Matrix client might be setting.
- Lots of bug fixes around bits of widget interface not updating properly, particularly around feeds.
- Node 18 support has been dropped, as it's not long for this world and we would like to focus on Node 20/21 support.
- We have added a
listandremovecommand for webhooks.- We now tell you if your bot doesn't have enough PL to speak, meaning you don't need me to confirm it for you :)
- The GoNEB migrator has been removed, though you can still migrate in a prior release and upgrade.
You can grab the release from here
A Matrix client; forked from Mirage.
Maze announces
Moment release 0.7.4
I bet no one was expecting this! After 2 years, finally another release of Moment (fork of the old client Mirage)!
Not much has changed. Mostly we're just keeping up to date with dependencies. But you will notice a couple differences:
- 👍 Emoji reactions to messages are now displayed (previously not visible)
- 🖉 Edited messages are displayed properly (previously only fallback text)
- 🚀 Almost all UI animations were removed, to fit the "snappy powerclient" idea
Matrix messaging app for GNOME written in Rust.
Kévin Commaille announces
It is a new year 🎆️, and what better way to celebrate this than to release Fractal 6.rc? It has been only 2 weeks since our latest beta release, but we have been hard at work during the holidays.
Here is an excerpt:
- Matrix URIs can be opened with Fractal, it is even registered as a handler for the
matrixscheme- Our Join Room dialog now shows some room details as a preview upon entering an identifier or URI
- The verification flow was rewritten to rely more on the Matrix Rust SDK, hopefully solving most issues that occurred before
- Room members now have a profile page that allows, among other things, to kick, ban or ignore them
- Speaking of ignoring users, the list can be managed from the account settings
- The dialog to view an event’s source was reworked to show more details about the event
… and a lot of other improvements, fixes and new translations thanks to all our contributors, and our upstream projects.
It is available to install via Flathub Beta, see the instructions in our README.
As the version implies, it should be mostly stable and we expect to only include minor improvements until the release of Fractal 6.
If one of your New Year’s resolutions is to contribute to an open source project, take your pick amongst our issues. Any help is very welcome!
Android Matrix messenger application using the Matrix Rust Sdk and Jetpack Compose
benoit announces
Version 0.4.0 of Element X Android is live on the PlayStore. It includes new features like Read Receipt, Chat backup and Mentions, and lots of other features and bugfixes. Full details here: https://github.com/element-hq/element-x-android/releases/tag/v0.4.0. We are now focusing on performance, testing and stabilizing the application, in particular around the Rich Text Editor.
Lots of new features are planned for 2024, stay tuned!
Secure and independent communication, connected via Matrix. Come talk with us in #element-web:matrix.org!
Florian Heese says
Video rooms Jitsi backend deprecation
Hello from the VoIP team,
In the past months we've had a lot of progress on Element Call our scalable video and voice conferencing solution with calling built natively on Matrix and so benefitting from Matrix’s end-to-end encryption. Now is the time to slowly convert the existing video solutions to default to Element Call. We are starting with video rooms, which will be using the Element Call stack by default in the near future. If you (still) have a Jitsi video room, you will need to re-create it to make use of Element Call.
Stay tuned
Emma [it/its] ⚡️ reports
Happy 2024! It's been a while since I last posted a TWIM update... Any help I can get is super appreciated!
Changes
- Lots and lots of code cleanup
- Sync deserialisation uses a compiled model now (still slow, but turns out this only affects debug builds)
- Floating point numbers are now sent as strings (since matrix doesnt like floats)
- Some internal fields were renamed to be more clear
- Join rules are now mapped
- Added utility function to get a room's avatar url as a resolved URI
- Added support for setting identity assertion on client instances (for appservices)
- Added overload to get member list as a list instead of an async enumerable
- Added GetAsObject and GetAsJson to MatrixException, to allow serialization
- Hot paths no longer use typed event content, due to the overhead of deserializing it
- Rewrote initialization of a client instance to be cleaner
And, as always:
- The code is available at cgit.rory.gay
- All contributions are more than welcome, be it documentation, code, anything! Perhaps, example usecases, bots, ...?
- Discussion, suggestions and ideas are welcome in #libmatrix:rory.gay (Space: #mru-space:rory.gay)
Bram says
This week, beta version 2.1.0 of the Elm SDK has been published!
The minor update is similar to version 2.0.0, where it introduces a new module
Matrix.Eventbut offers no major functionality at the time. The API interactions are planned to come after implementing an exposedMatrix.Roommodule for interactions with Matrix rooms.This week, I would love it if you take a look at the
Matrix.Eventmodule, and let me know if the documentation is sufficient and meets your requirements. I'd love to hear your feedback in the Matrix room #elm-sdk:matrix.org.
The 37th Chaos Communication Congress, sort of a school reunion of hackers, recently ended. From the 27th to the 30th of December 2023, a bunch of nerdy people met up in Hamburg (northern Germany) to discuss all the cool stuff they had been up to. Topics ranged as wide as technology, society, politics and even food hacking as well as a whole lot of other interesting things (talks can by the way be found at https://media.ccc.de/ and most are already uploaded, wow!). We had a Matrix Community Assembly there. An assembly is basically an area for like-minded people to meet up and talk about their topics, in this case Matrix.
We answered questions, sold merch, talked about Matrix-related projects, had workshops about topics like setting up Hookshot, took group photos, tapped beer via Matrix, handed out a bunch of cool new Matrix Community stickers and even fixed some Matrix servers. And Nico bridged the whole Eventphone network to his Matrix account while tinkering and got very confused.
If you have said hello at the event, then also say hello at #chaosevents:matrix.org!
urp announces
An example minimal website, dynamically sourcing its content from a public matrix room that anyone can read and publish to.
It uses a custom event
io.gitlab.ugrp.post(with two keys,titleandtext) and provide an interface for guest and registered users to create a newpost(cannot edit yet, tbd).The site is built in vanilla HTML, CSS and Javascript, using the @sctlib/mwc library of web-components (the new version
v0.2.1allows to register aformTemplateanddisplayTemplateto be able to display and create and custom event in a room).This is all prototypes (and examples), experimenting using public matrix data outside of traditional chat clients (see also matrix-static). The objectives are to allow users to manage and display their data in environments they control, and subscribe to event feeds from sources they chose.
Josh Simmons says
New year, new homeserver! I completed a migration from managed Matrix with EMS to self-hosting using the matrix-docker-ansible-deploy project. It was a bit tricky as someone new to self-hosting Matrix, but a lot of folks helped along the way and the matrix-docker-ansible-deploy project greatly simplified the process.
I wrote up my notes in the Foundation blog in hopes that they'll be helpful, especially for other folks like me who were on small EMS plans that are being discontinued.
Aine announces
collected by MatrixRooms.info - an MRS instance by etke.cc
As of today,
8389Matrix federateable servers have been discovered by matrixrooms.info,2269(27.0%) of them are publishing their rooms directory over federation. The published directories contain168449rooms.
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
| Rank | Hostname | Median MS |
|---|---|---|
| 1 | rom4nik.pl | 263 |
| 2 | utzutzutz.net | 265.5 |
| 3 | nerdhouse.io | 297 |
| 4 | maunium.net | 299 |
| 5 | fostered.uk | 373 |
| 6 | matrix.org | 454.5 |
| 7 | matrix.its-tps.fr | 503 |
| 8 | mailstation.de | 957 |
| 9 | mtrx.fail | 1544.5 |
| 10 | littlevortex.net | 2006 |
Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
| Rank | Hostname | Median MS |
|---|---|---|
| 1 | inu.is | 144 |
| 2 | nerdhouse.io | 158 |
| 3 | matrix.its-tps.fr | 160 |
| 4 | spqr2gang.com | 163 |
| 5 | frei.chat | 170 |
| 6 | matrix.org | 316 |
| 7 | matrix.maymundere.org | 897 |
| 8 | littlevortex.net | 1479.5 |
| 9 | shiftsystems.net | 24439.5 |
| 10 | chat.eutampieri.eu | 88738 |
See you next week, and be sure to stop by #twim:matrix.org with your updates!
I confess I'm awfully chuffed with myself as we return from the holiday break. I just completed a successful migration of my main Matrix account from managed hosting to a homeserver that I run for myself on a virtual private server (VPS).
The whole experience has been illuminating, and there are some specific details that are timely for people like me who needed to migrate off of Element Matrix Services (EMS) as they pivot to focus on enterprise.
Thus, this blog post. I'm going to share my experience in hopes that it'll help some folks with that migration!
Hi all,
2023 has been a pivotal year for Matrix, with huge changes landing both organisationally and technically to prepare the protocol for future generations. The ecosystem has once again gone from strength to strength, with active users (based on Synapse opt-in phone-home reporting) doubling across the public network, and more projects building on Matrix than we can count (look out for the “This Year in Matrix” community wrap-up blog post) - and more organisations than we can track joining Matrix for all their secure decentralised communication needs.
On the governance side, we are in an incredibly exciting new era with Josh joining the Matrix.org Foundation as its first ever Managing Director (and employee!), with a mandate to cement sustainable funding for Matrix as an independent foundation, governed by the forthcoming elected open Governance Board. Now, Matrix needs funding more than ever - but rather than turning the entirety of this post into a plea for donations, I’m going to let Josh fly the flag in the coming weeks. Meanwhile, if you want Matrix to keep existing (especially if you’re an organisation who builds on Matrix) please join the Foundation and donate.
On the technical side: the theme of the year has been one of focus: extreme, overdue, focus.
Over the years, it’s fair to say that the core team has tried to strike a balance between building the core foundational technology of Matrix (the spec, a stable server implementation, client SDKs, end-to-end encryption, VoIP, etc)... and long-term forward-looking projects designed to futureproof Matrix (e.g. Account Portability, P2P Matrix, Dendrite, Hydrogen) and/or inspire developers to build on Matrix for more than just chat (e.g. Third Room, Applications Beyond Chat). In retrospect, this was wildly optimistic: we underestimated the amount of remaining work needed to polish the foundational tech to mainstream quality - and despite Matrix uptake going through the roof, this hasn’t translated into sufficient funding to have the luxury to support folks to proactively work on next-gen projects (or foundational projects, for that matter).
So, this year, we’ve ended up focused on one thing: getting the foundational Matrix featureset to better-than-mainstream quality, performance and stability. We’ve dubbed the overall initiative Matrix 2.0, and kicked it off at FOSDEM 2023 with our Matrix 2.0: How we’re making Matrix go vooooom main-stage talk.
Matrix 2.0 isn’t (yet) an actual versioned release of the Matrix specification - instead, it describes the various foundational projects needed to get quality, performance and stability up to and beyond that of today’s mainstream messaging apps. These projects are:
Over the course of the year Matrix 2.0 has gone from the initial demo on stage at FOSDEM to concrete implementations which users can play with today as announced in our Matrix 2.0: The Future of Matrix post in September. Since then, we’ve been busy polishing away. On Sliding Sync, the proxy has pretty much stabilised - although the protocol itself can and should be simplified before we think seriously about native implementations (in practice, having the server track room list ordering gets very fiddly when only clients can really determine the final ordering, due to E2EE). Element X and matrix-rust-sdk has been the main implementation driving forwards Sliding Sync and much of the other Matrix 2.0 work, for those itching to play with it.
On Native Group VoIP: we’ve gone through many iterations over the year - starting off with Full Mesh calling (good for ~7 users per call); then switching to the experimental waterfall Selective Forwarding Unit (SFU) to provide scalable but not-E2EE conferencing; and then switching to a hybrid solution using LiveKit to provide an E2EE-capable scalable SFU, but with the signalling and encryption all handled by Matrix. Element Call is the main implementation driving forwards the underlying Matrix work here, and Element Call Beta 3 showed off the new LiveKit based implementation in July - which was then integrated with Element X complete with end-to-end encryption in November. There’s still some polishing remaining here, with a new layout engine in the wings for Element Call, and enabling full encrypted-per-sender conferencing by default in both Element Web and Element X, but it really feels like the hardest work is behind us now: the core team has been successfully doing all of its collaboration on Element Call for months now, like so:
On Open ID Connect: things are also shaping up well. This will be the first time that we’ve replaced a large chunk of the Matrix spec with something else, and in order to manage your account in Matrix 2.0-native clients like Element X homeserver admins will need to migrate their authentication to the new OIDC World using matrix-authentication-service (MAS). There’s a great blog post from September which explains what this will entail - and since then, we even have the beginnings of syn2mas: a migration script to migrate from Synapse-managed accounts to MAS-managed accounts (warning: still experimental). The Matrix.org homeserver hasn’t been migrated yet (as we need to support social login first), but an increasing number of standalone Matrix servers are going OIDC-native, so arguably the migration has already begun! We’ll keep https://areweoidcyet.com updated as the project progresses.
Finally, the core of Faster Remote Room Joins (FRRJ) shipped in Synapse back in February. There’s still some major speedups that FRRJ could unlock, but the other tracks of Matrix 2.0 have been taking priority.
So: Matrix 2.0 is palpably on the horizon - all that remains is polish on the example clients (Element X & Element Call), full support for migrating to OIDC, and landing the MSCs into the spec. For instance, Element X just added read receipts and (early) E2EE backup support in the last few days - the gap is closing! It’s worth noting that significant amounts of this work has been funded by BWI for BwMessenger and BundesMessenger: huge thanks to BWI for supporting core Matrix development by contracting Element.
Encryption stability received a huge amount of attention this year. It turns out that reliable end-to-end encryption is surprisingly tricky in a decentralised environment, and historically we’ve been playing on hard mode by implementing three entirely separate implementations of the Matrix layer of encryption between matrix-js-sdk, matrix-ios-sdk and matrix-android-sdk2, each with their own bugs - more than tripling the costs of development, audits, and maintenance by the Foundation (quite ignoring the independent implementations from the community in mtxclient, libquotient, matrix-dart-sdk, trixnity etc).
So a huge project has been underway to converge on a single auditable codebase for the core team’s E2EE implementation so that any bugs or future features can be resolved in a single place. That codebase is matrix-rust-sdk’s matrix-sdk-crypto crate (and our underlying vodozemac double ratchet implementation) - and we’re proud to say that we are using it for encryption in matrix-rust-sdk itself (as showcased by Element X and Fractal 5), matrix-ios-sdk and matrix-android-sdk2 (as used in the old Element iOS & Android apps), and have now merged it in matrix-js-sdk too (available for new logins on develop.element.io). The process of rustifying the encryption in Element Web and the old iOS & Android apps has been nicknamed “Element R”.
The process of switching matrix-js-sdk to use Rust encryption has been particularly gruelling, requiring compiling matrix-sdk-crypto down to WASM as matrix-rust-sdk-crypto-wasm and then doing heart surgery to replace the old JS crypto implementation… while also needing to extensively loop from WASM back into the browser to use IndexedDB for storage, all while outperforming the old implementation. It’s tantalisingly close now: while develop.element.io has it turned on by default for new logins, there are still a few remaining performance edge cases to be chased down related to online backup before we migrate everyone to it. The remaining blocking issues can be found on GitHub for those interested in tracking progress.
matrix-crypto-sdk is already manifestly more reliable than the old implementations (in terms of the chances of hitting infamous Unable To Decrypt errors) - and now that we are so close to converging on it everywhere, the race is on to ensure that any remaining defects get flushed out for once and for all. One of the new initiatives here is called complement-crypto - a full end-to-end torture testing suite specifically for matrix-crypto-sdk. You can read all about it in the announcement post a few weeks ago, but suffice it to say it’s a super exciting project which stress-tests both matrix-rust-sdk and matrix-js-sdk (with the new rust crypto implementation) against federated Synapse containers in order to test E2EE under the most horrible failure modes imaginable. It’s already picked up some elusive bugs which have plagued us for literally years, and it looks set to be the main framework by which we will hunt down and kill any remaining issues. See the Test hitlist for the full scope we’re targeting.
Now that everyone’s (almost) converged on matrix-sdk-crypto, the next big project for the Crypto Team is going to be improving the E2EE usability (at last!). The big news here is that we’re shifting to Trust On First Use (TOFU) for user trust. Specifically: this means that we will only encrypt messages to devices whose owner has explicitly cross-signed them (essentially trusting the owner by default). You will still be able to explicitly verify that other users are not being impersonated (via QR scan or emoji comparison), but this should improve the default behaviour to be much more secure. Alongside TOFU will come other radical simplifications of the E2EE UX (both around login, self-verification, cross-verification and backup), so watch this space: the game is afoot to finally fix Matrix’s E2EE usability, now we can make all the changes in one place!
Finally, work continues to progress at matrix-dmls on supporting a decentralised dialect of Messaging Layer Security (MLS, RFC9420) on top of Matrix as an alternative to our normal Olm/Megolm encryption, with recent work focused on making it play nice with matrix-sdk-crypto. https://arewemlsyet.com is the place to track updates (although it’s a bit overdue for an update).
Faced with limited funding and the decision to focus exclusively on stability, reliability and performance, there have inevitably been some major changes impacting the core team.
One of the biggest changes is that Element (the company formed by the core Matrix team back in 2017 to try to fund our work on Matrix) can no longer financially afford to donate its work on Synapse and other server components to the Matrix Foundation under the permissive Apache licence. Instead, Element is continuing development under the copyleft AGPLv3 licence at github.com/element-hq/synapse going forwards. This is to let Element sell AGPL exceptions to commercial Matrix vendors in order to fund their underlying Matrix development: you can read more about it at Element’s announcement - or you can listen to this week’s Matrix Live for a firsthand explanation:
The other major change is that we’ve had no choice but pause development on the majority of the core team’s next-generation Matrix projects. We had high hopes of being able to secure dedicated funding for Third Room (especially after the awesome Tech Preview 2: Creator Update in June), but the interested parties did not come through, and the team has now disbanded. Meanwhile, P2P Matrix and Low Bandwidth Matrix is on hiatus until there’s dedicated funding - and Account Portability work is also temporarily paused in favour of commercial Element work, despite the fantastic progress made recently with Pseudo IDs (MSC4014) and Cryptographic identifiers (MSC4080). Given P2P Matrix and Account Portability were the main projects driving Dendrite development recently, this may also cause a slow-down in Dendrite development, although Dendrite itself will still be maintained.
Needless to say, this is far from an ideal situation: we sent up distress flares loud and clear at the beginning of last year’s holiday update; and we’ve now had to shrink to focus exclusively on the core projects. However, we’re optimistic that the tighter focus in the medium term will help us get back to the point where we can resume the longer-term projects - assuming that organisations (and individuals) dependent on Matrix sign up to support the project.
Despite the downsides of 2023, right now we’re feeling distinctly optimistic: Matrix 2.0 clients like Element X already outperform the best proprietary mainstream options by many metrics - and focusing purely on improving the foundations is only going to improve that. We may not have taken the most direct route to get to where we are today, but it genuinely feels like 2024 will be the year where Matrix overtakes the incumbents.
Talking of which, there’s just one last thing to mention: amidst the economic challenges of 2023, one future-facing core team Matrix project has survived: our work around the EU Digital Markets Act (DMA). The DMA is leading antitrust legislation from the European Commission, which aims to stop big centralised tech companies (so called ‘gatekeepers’) from carelessly suppressing innovation, competition and consumer choice by trapping users inside their walled gardens.
Needless to say, we’ve been tracking the DMA closely throughout its gestation, and we’re now in the final sprint: in March 2024, messaging services which have been identified as gatekeepers will have to open their networks to allow interoperability with requesting messaging services (while preserving E2EE, if they’re encrypted). So far, only WhatsApp and Facebook Messenger have been identified as gatekeeper services (Apple is doing everything they can to wriggle out of it). However, it looks like WhatsApp is taking it seriously, which could prove very interesting indeed.
As Matrix, we’ve taken a two-pronged approach: on one side, showcasing how Matrix as it stands today can already bridge existing chat systems together as a highest-common-denominator protocol (including preserving E2EE, if they happen to already use the Double Ratchet). On the other side, we’ve also been contributing significantly to MIMI, the IETF Working Group dedicated to standardising a lowest-common-denominator protocol specifically for DMA interoperability.
2023 has involved a lot of work on MIMI, participating in the Design Team alongside Phoenix, Wire, Cisco, Google and Wickr - and at IETF 118 in Prague in November we collectively proposed the first draft of the protocol (see also the architecture presentation for context). MIMI has ended up taking some inspiration from Matrix (and Linearized Matrix, a simplified dialect we proposed which does away with full-mesh decentralisation), while focusing very tightly on the specific problems of server-to-server interop between existing communication services - leaning on MLS (where available) for synchronising state across the services, while also leaving the door open for using the Double Ratchet to provide an on-ramp for pragmatic bridging to today’s services (including Matrix!).
MIMI’s tight focus means that it doesn’t currently provide conversation history, arbitrary state events, or decentralised conversations - and is focused purely on server-server communication. However, there’s certainly a world where Matrix could evolve to be compatible with MIMI - such a hypothetical Matrix 3.0 would effectively layer Matrix’s richer existing semantics (decentralised conversations, extensible state events, state resolution, group VoIP etc) on top of MIMI’s subset of functionality. It’ll be interesting to see how this plays out. Meanwhile, we’ll continue to provide both Matrix for pragmatic DMA interoperability for today - and participate in MIMI for IETF-track interoperability.
Anyhow: hopefully it’s clear that 2024 is going to be a super interesting year for Matrix - whether that’s simply by nailing Matrix 2.0, or whatever excitements DMA interoperability brings. And if you’re relying on Matrix: please donate.
Meanwhile - have a fantastic end of the year; and thanks once again for flying Matrix.
- Matthew, Amandine, Josh & the whole team.
Nico announces
(Please ignore, that I never know, what year number to attach to such a "Year In")
You may remember I organized a small blog post last year to collect stories from the different community projects and what they did in the year and maybe some sneak peaks at the next year. If not, you can find it here or on the Matrix.org blog.
Anyway, enough about 2022, I now encourage you to talk about 2023 and beyond! If you have interesting stuff to report about your projects or projects you have been involved in in 2023, feel free to join #year-in-2023:neko.dev and talk about it! The usual TWIM rules apply there, just that we talk about a whole year and it may involve lots of manual editing on my side, so don't try to break it. Also please be positive in your news and lets try to end 2023 with a bang!
And please share this with projects you want to hear about. :)
As 2023 winds down and I find myself in the thick of planning for 2024, I’d like to start preparing all of us in the Matrix ecosystem for what is to come.
Next year will mark a number of important milestones in the history and evolution of Matrix: the protocol will mark its 10th birthday, we’ll see key initiatives in the spec cross the finish line, and we’ll seat the first ever community-elected Governing Board.
The election of our first Governing Board is what I’d like to focus on today, because it is a huge milestone on the path to an independent, self-sustaining, and self-governing ecosystem. When we celebrate Matrix’s 20th birthday, we’ll look back and our history will be divided much the same way it is in other ecosystems: before and after incorporating a foundation, and before and after introducing community governance.
Let’s talk about what the Governing Board is, why it matters, and how to get involved!
Thib says
I'll start with a heads-up that the Foundation is going to clean-up all the
libera.chataliases on Matrix. You might want to check the rooms under your control to update the alias and the matrix.to links to it.You can find all the details on our blog
Andrew Morgan (anoa) announces
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
MSC Status
New MSCs:
- There were no new MSCs this week.
MSCs in Final Comment Period:
- No MSCs are in FCP.
Accepted MSCs:
Closed MSCs:
- No MSCs were closed/rejected this week.
Spec Updates
As you can make out from the stats, the spec process has been mostly quiet over the past couple weeks. In fact, the stats above are from the last couple weeks, as someone (cough, me) forgot to post a spec TWIM last Friday. All to be expected with the holidays looming however, including my own!
Random MSC of the Week
The random MSC of the week is... MSC3871: Gappy timelines!
This MSC actually gives background and summarises itself pretty well, so I'm just going to copy/paste from it here as an explainer!
/messagesreturns a linearized version of the event DAG. From any given homeservers perspective of the room, the DAG can have gaps where they're missing events. This could be because the homeserver hasn't fetched them yet or because it failed to fetch the events because those homeservers are unreachable and no one else knows about the event.Currently, there is an unwritten rule between the server and client that the server will always return all contiguous events in that part of the timeline. But the server has to break this rule sometimes when it doesn't have the event and is unable to get the event from anyone else. This MSC aims to change the dynamic so the server can give the client feedback and an indication of where the gaps are.
This way, clients know where they are missing events and can even retry fetching by perhaps adding some UI to the timeline like "We failed to get some messages in this gap, try again."
It does this by adding a
gapsproperty to theGET /_matrix/client/v3/rooms/{roomId}/messagesendpoint, which tells the client which events its missing, instead of the client just assuming incorrectly!Do check out the MSC if it sounds like something that would be useful to your client!
Second generation Matrix homeserver
Devon Dmytro reports
This week we released v0.13.5. Upgrading to this version is highly recommended, as it fixes several long-standing bugs in our CanonicalJSON implementation. Here are a few of the highlights:
Fixes
- Convert unicode escapes to lowercase (gomatrixserverlib)
- Fix canonical json utf-16 surrogate pair detection logic (gomatrixserverlib)
- Handle negative zero and exponential numbers in Canonical JSON verification (gomatrixserverlib)
- Issues around the device list updater have been fixed, which should ensure that there are always workers available to process incoming device list updates.
- Fixes around the way we handle database transactions (including a potential connection leak)
- ACLs are now updated when received as outliers
Features
- Appservice login is now supported!
- Users can now kick themselves (used by some bridges)
...and a whole lot more. Check out the release notes for the full set of changes! As always, feel free to stop by #dendrite:matrix.org to join in on the discussion and if you encounter a bug make sure to report it here.
A Qt5-based IM client for Matrix
kitsune reports
The long-awaited Release Candidate for version 0.0.96 is finally out - not many changes as the last beta has been reasonably good. 0.0.96 release will be the last one supporting Qt 5.15; packagers are recommended to build it with the most recent Qt 6 at their disposal. Binaries for macOS and Windows are available from the GitHub Releases page linked to above, and a flatpak is already available from the beta channel at Flathub. Totally not promising the release before Christmas ;)
Desktop client for Matrix using Qt and C++17.
Nico reports
I didn't post many updates recently, but that was mostly because I was working on boring and tedious stuff, that isn't interesting to hear about. Nevertheless, I'll tell you about some of the things now.
On Wayland, if you want to activate your window (as in, bring it to the foreground), you are not allowed to do that. This is for security reasons, since when Wayland was designed, many people were dealing with bad web popups that always stole your focus. In itself that design decisions does make sense, but it does cause some problems. Want to open a notification? App will blink, but not come to the top. Want to open a url in your browser like matrix:r/nheko:nheko.im ? Well, the app will blink and open the url, but stays in the background! Want to run the app from the command line or an icon? Well, the existing window will blink, but not come to the foreground!
However, we fixed that. Usually you should get an activation token, but not all apps provide you with one (like X11 applications, KDEs Konsole, etc). Nheko will now try to use an activation token, when one is passed from your notification manager, but that didn't work reliably for the other ways Nheko could be activated. In those cases Nheko now creates a small window with a loading spinner to get an activation token (because launching a new window of course will bring you to the top...) and then uses that window after 100ms to create an activation token for the main app, sends it to the main app and now the main app can raise itself. Do you think this sounds like a terrible hack? Well, it is, but it works and that's better than nothing! Hopefully all the wayland users won't complain about the useless loading indicator in the future...
Also, on Windows clicking a notification didn't put you in the room. This is now fixed! And we have a shiny new Windows CI runner.
Speaking about Windows, we now also provide an msix for Windows users, that even has a valid signature (that costs quite a lot of money....)! This should make the installer experience quite a bit better for Windows users and soon even should provide auto-updates, but we still need to figure out if we want to enable the "ping nheko.im to check for an update every 15 days" or if we don't want to have our app call home... In theory that should be optional depending on how you install Nheko though. It will only be enabled when you use the appinstaller link, not the msix directly. Apart from that there is a small issue still in the notifications, where the App name now has a weird suffix... and possibly some issues with the protocol handler. If anybody has experience with Windows development, I would appreciate some help. I don't use the platform, so I am mostly flying blind there!
We also now use a different package to manage the single instance behaviour, which should work more reliably on Qt6. However that means you really shouldn't run Qt5 and Qt6 Nheko side by side!
There have been plenty of updates to the spanish translation by CM0use and leaving a space should now properly unhide its rooms again. Begasus also improved the Haiku support by a lot and plenty of smaller fixes!
At the current rate, see you next year, I guess? Thank you for reading our update!
Bram says
The Elm SDK is an SDK in Elm that compiles to JavaScript and hence aims to be a competitor to the established matrix-js-sdk. This week, the SDK has released its beta 1.0.0 release on the public Elm registry!
The project has received funding from NLnet 💸 and it is now time to start releasing beta versions in an accessible way. The beta versions are experimental and unstable, but feedback is already going to be very much welcome as new versions are going to be released!
You can follow the progress in the Matrix room #elm-sdk:matrix.org and the project will also give you timely release updates on the Fediverse. 🦣
Your matrix server on your conditions
Aine announces
Attention: Changes to the Pricing Model at etke.cc
We're excited to announce a significant update to our pricing model at etke.cc, your trusted managed Matrix servers provider.
Transition to Pay-By-Complexity
In a move towards fairness, we're shifting from a flat-fee pricing model to a more dynamic, pay-by-complexity approach. This means you'll now pay based on the services you choose, allowing for a more tailored and equitable pricing structure.
Fair Pricing for Everyone
The previous flat-fee model sometimes led to discrepancies, where customers with different service needs paid the same amount. With pay-by-complexity, you'll only pay for the services you select, ensuring fair pricing for every user.
Introducing Base Matrix Server at $5/month
As part of this change, we're thrilled to offer a base Matrix server (just a Matrix homeserver and some core services, without bridges/bots, etc.) on your hardware for as low as $5/month—a 50% reduction compared to our previous Maintenance tier. This enables you to enjoy Matrix services at a more affordable rate.
New Services Available
Expanding beyond Matrix, we're reintroducing additional services, such as Miniflux, Radicale, Uptime Kuma, GoToSocial, Linkding, and Vaultwarden, available as paid add-ons. Your feedback will guide us in expanding this service portfolio gradually.
Hello, America and AMD-powered Servers
We're now supporting Hetzner Cloud's new AMD-powered
CPXserver line, offering improved performance-per-dollar and double the disk space. These servers are available not only in Europe but also in the US, opening up new possibilities for our customers.FAQs: Your Questions Answered
We understand you may have questions about these changes. We've covered topics ranging from pricing adjustments to server upgrades and service additions in the full announcement, and in the FAQ entry. For existing customers, there's no rush—current pricing remains in place until at least July 1, 2024.
Explore the new offerings and get started with the enhanced etke.cc experience. To stay in touch, join the #news:etke.cc room and keep an eye on etke.cc/news.
HarHarLinks reports
There is a large interest in FOSDEM from the Matrix community, and we are very excited to see not just a stand, not just an online track, but an in-person Matrix devroom happening again this year! If you have been following the CfP, you know that the officially allotted time for the Matrix devroom at FOSDEM however is only half a day.
You may be wondering "wait, haven't I read this before?" and you would be correct. Last year, at this point a couple friends from the Community involved with organising several other Matrix events got together and started the "FOSDEM 2023 Matrix Community Meetup", and I dare say, to great success! So, this year again, we got together and sat down with the Matrix Foundation to plan yet another Matrix FOSDEM Fringe event, which is why this time around...
We are happy to announce the FOSDEM 2024 Matrix Foundation & Community Meetup at Hackerspace Brussels (HSBXL)! The meetup is set to begin on Friday February 02 around noon local time (CET) and will go into the evening. See https://hsbxl.be/enter/ for multiple well documented ways how to find the way there. Thank you HSBXL for providing a location for this! Further, we are also happy to announce that we have found some sponsors which again will enable us to provide some drinks and food for everyone! If you or your company are planning to attend and are able to contribute to the sponsoring, please contact Thib on Matrix or using legacy comms at mailto:thib@matrix.org!
More details will be announced soon in a blog post, so if you're not subscribed to the feed (https://matrix.org/atom.xml), do so using your favorite feed reader and watch this space!
In the meantime, you can already join #fosdem2024-foundation-community-meetup:matrix.org for all upcoming info, program suggestions, to follow the process & help out, general discussion and everything else related to the event.
See you in Brussels!
ben says
v1.23.1214 is now on the AppStore for Mac
After what seemed like forever we finally found and fixed the bug that prevented our submissions to be accepted by Apple. With the latest release from thursday Acter is finally available for Mac to be installed (and kept up-to-date) conveniently via the the preinstalled Apple Mac AppStore - 💥 whopwhop, pew-pew 💥.
message edit and improved multimedia previews
Additionally this release contains important improvements and fixes in the chat: for once, you can edit chat messages now from within the Acter app (before it would display them correctly but you couldn't edit yourself) on all platforms. Secondly, we have reworked the entire multimedia handling for chat messages allowing for inline preview and viewing as well as in full-screen now, as well as several fixes around posting media. Learn more about all that in our latest blog post
becoming an Acter supporter
Some of you might already are aware that Acter isn't your average "startup" but build by a non-profit association. We have recently taking some time to explain better what that means, why that matters (to us) and how you can become a supporting member of the association (or gift it someone else for the holidays) and what's in it for you.
uhoreg says
I've added another section to my Matrix client tutorial, explaining how to encrypt and send to-device events with Olm. I had previously written sections on encrypting and sending to-device events. This new section deals with some of the additional considerations when there are multiple recipients, and how to manage the network requests. Notably missing from this section, so far, is how to deal with errors such as failing to get one-time keys; that will come later. Next up will be encrypting room events (which is already mostly written, but is lacking tests).
Aine announces
collected by MatrixRooms.info - an MRS instance by etke.cc
As of today,
8330Matrix federateable servers have been discovered by matrixrooms.info,2186(26.2%) of them are publishing their rooms directory over federation. The published directories contain272674rooms.
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
| Rank | Hostname | Median MS |
|---|---|---|
| 1 | matrix.sp-codes.de | 293 |
| 2 | fostered.uk | 331 |
| 3 | nerdhouse.io | 335 |
| 4 | maunium.net | 425 |
| 5 | ilmari.org | 460 |
| 6 | catboyindustries.co | 516 |
| 7 | littlevortex.net | 583 |
| 8 | mailstation.de | 958 |
| 9 | plocki.org | 1629 |
| 10 | jauriarts.org | 2475 |
Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
| Rank | Hostname | Median MS |
|---|---|---|
| 1 | spqr2gang.com | 167 |
| 2 | dendrite.s3cr3t.me | 169 |
| 3 | fostered.uk | 192.5 |
| 4 | nerdhouse.io | 194 |
| 5 | kumma.juttu.asia | 282.5 |
| 6 | matrix.org | 289 |
| 7 | matrix.its-tps.fr | 333 |
| 8 | inu.is | 397.5 |
| 9 | kanp.ai | 481 |
| 10 | herkinf.de | 592 |
See you next week, and be sure to stop by #twim:matrix.org with your updates!
The Matrix.org Foundation has taken down the bridge with the Libera Chat network. This only prevented messages from making it across the bridges, for Matrix users to appear on the IRC side, and for new IRC users to appear on the Matrix side.
As part of our work to remove the bridge leftovers, we have removed the ghosts in Matrix rooms and demoted the Libera Chat appservice user. We will now remove the aliases from the rooms, and strongly encourage you to make sure you update the links to your Matrix room if they relied on a matrix.to link that contains :libera.chat
