Hiding room metadata from servers

Every year, Element takes on a small number of summer interns to work on various research and development projects. Internships are a significant part of our culture, and ~20% of the current Element engineering team has completed some form of internship with us before joining permanently.

This year, our interns will publish a report outlining their project, their progress, and opportunities for further work. To start with, Skye will take you through their encrypted state event project.

— Neil Johnson, Chief Engineering Officer, Element

Greetings! I’m Skye, one of Element’s software engineering interns, and I recently graduated from Cambridge. I’d like to take a few minutes to discuss a crucial aspect of Matrix privacy that I have been working on: room state encryption. This blog post is all about my work on implementing Encrypted State Events, a new feature that ensures sensitive room info stays hidden from homeservers, giving users more privacy.

When you create an encrypted room in any Matrix-powered application, such as Element Web/X, Cinny, or Fluffychat, its name, topic, and other metadata are stored on the homeserver in plaintext as “state events.” This differs from regular messages, which are fully end-to-end encrypted.

In an ideal world, this information would be hidden from the server, preventing any malicious actor or homeserver implementation from snooping on it, ensuring we maintain the highest possible level of privacy. This is especially important for Matrix-based software that uses state events to store sensitive information persistently, which can create compliance and privacy concerns.

Understanding types of events

To understand why encrypting state events is both important and complex, let’s first examine how Matrix organises information within rooms. In the specification, there is a section on types of room events:

Room events are split into two categories:

• State events: These are events which update the metadata state of the room (e.g. room topic, room membership etc). State is keyed by a tuple of event type and a state_key. State in the room with the same key-tuple will be overwritten.
• Message events: These are events that describe transient “once-off” activity in a room: typically communication such as sending an instant message or setting up a VoIP call.

Let’s look at some simple examples:

• If Alice sends a message to Bob in an encrypted room, this is sent as an end-to-end message event, which will be completely invisible to the homeserver.
• If Alice renames the room to “Super Secret Title”, this will be visible to the homeserver.

The crucial difference is that each state event attempts to update a (event_type, state_key) to event key-value mapping in a given room, allowing for key-value data to be stored per room. This means there can only be one "current" value for each (type, state_key) combination, unlike message events, which accumulate indefinitely.

Encrypting what servers don't need to see

MSC3414 (Encrypted State Events) proposes a solution: encrypt room state events that aren't critical for server operation. Room names, topics, avatars, and other metadata would be encrypted end-to-end, just like messages, while keeping essential events server-visible for proper server functionality, such as power levels and join rules, which describe what permissions users have in rooms, as enforced by the server.

The core idea is simple: we funnel state events through the existing encryption architecture used for message events. This allows us to reuse code and simplify the problem, all while using the cryptographically secure foundation already in place for message events.

However, a little more work is also needed:

• Encrypted message events always have the type m.room.encrypted.
• As described earlier, for the server to consider an event a state event, it must have a state key.

Of course, we could use the existing, unencrypted state key, but this could reveal information we want to keep secret, and we would encounter issues where state events with different types but identical state keys overwrite each other in the room state. 

Instead, the proposal chooses to mask the original event type and state key by computing an obfuscated version of the state key. This requires a distributed system that allows all members of the room to derive the correct state key for any state event they wish to create, without the homeserver also being capable of this derivation. Many state events use empty state keys, including the aforementioned room topic and name events, which makes this a bit tricky.

The proposal has been in development since 2021, with extensive community discussion around implementation challenges. 

A naive, but workable solution

As a proof of concept, I opted for a more straightforward approach: state key packing. This was very simple to implement, requiring only a few hundred lines of code.

As I described earlier, state events have state keys, and since we need to preserve the server’s ability to resolve state events correctly, we can simply “pack” them together in a single string, which we can then use as the state key. Let’s look at some examples:

• A room topic event, m.room.topic with a blank state key becomes m.room.topic:.
• A location share initiation event, m.beacon_info with the state key @user:example.com becomes m.beacon_info:@user:example.com.

This approach allows us to avoid the need for a distributed state key derivation system, as outlined in the proposal, as it is incredibly simple - it’s just string concatenation!

However, there is an obvious downside: it exposes both the event type and state key to the server, meaning any malicious actor can see the type and state key of each event. The content of the event, however, remains obscured, hiding the most important information, which is a significant step forward for privacy.

Using a colon as a delimiter might also not be the best idea - state keys can contain colons themselves, e.g. when they store user IDs, which have the form @user:example.com. Scanning up to the first colon during unpacking can be slow, especially when processing events with long type names at a rapid rate.

To maintain message integrity (i.e. to ensure the server has not tampered with the event), we include the same information inside the encrypted payload and then cross-reference it with the packed state key to ensure that a malicious actor cannot sneak a state event into an encrypted message event and vice versa.

Skye shared a demo of their work on encrypted state events during Matrix Live S11E13

Building across the stack

My implementation spanned Element's entire client stack, requiring changes across four major repositories:

• Rust SDK - The core encryption logic lives in the matrix-sdk-crypto crate. I implemented state event encryption by extending the existing OlmMachine infrastructure - the same system that handles message event encryption. This reuse made the implementation remarkably straightforward, leveraging Matrix’s proven cryptographic primitives and key management.
• WASM SDK - This acts as a bridge, exposing the functionality of the cryptography crate to JavaScript environments. I extended this layer with a few new methods dedicated to encrypting state events.
• JS SDK - For its cryptography functionality, the JS SDK bundles the WASM SDK, so modifications to this were limited. I made some changes to MatrixEvent that allow the SDK to consider state keys as encrypted, and added helper methods to call into the code I introduced in the WASM SDK.
• Element Web - To test experimental features, Element Web has “labs” - on/off switches that interested users can toggle to opt into new functionality. I implemented encrypted state events as a labs flag, utilising the updated JS SDK.

Difficulties and annoyances

Overall, the development of encrypted state events went very smoothly, and there are only two minor difficulties I ran into:

• The biggest use of my time was waiting for feedback on my code, but this was well worth the wait, as all the feedback I received was always productive and helpful. 
• I found the JS SDK and its primary dependent Element Web a little harder to navigate than the Rust SDK, but this is what one should expect when working with a codebase that recently celebrated its 10th birthday.

What's next?

My implementation focuses on core functionality, specifically encrypting all state events not declared critical in MSC3414 by using string-packed state keys. The full MSC3414 specification includes additional complexities related to restricted rooms and advanced key sharing scenarios that are reserved for future development. Some of this functionality is already being implemented through history sharing.

The next immediate step is to merge pending pull requests in the JS SDK and Element Web, making encrypted state events available to users via labs flags. This will enable real-world testing and feedback to guide further development.

Much discussion is underway on how best to resolve the issues with state key packing. Suggestions include using a keyed hash function, such as HMAC, to derive state keys or performing some degree of state resolution client-side. Encrypted state events also have some undesirable consequences: public room lists provided by homeservers will be reduced to garbled noise, since the server no longer has access to the room name, with similar behaviour for spaces.

Why this matters

Encrypted state events represent more than a technical improvement; they are another step towards comprehensive privacy in an increasingly surveilled world. When healthcare providers, journalists, activists, and everyday people choose Matrix for communication, they shouldn't have to worry about metadata leakage undermining their privacy.

My internship enabled me to contribute to this process, working across Element's entire technical stack to bring a years-in-development specification proposal to production, albeit in experimental form. While challenges remain, I am optimistic that the foundation is now in place for truly metadata-private Matrix rooms.

Acknowledgements

There are a few people I would like to thank for their contributions and support during my work:

• Andy Balaam and Richard van der Hoff for mentoring me and providing guidance throughout my internship, without whom I doubt I would have been able to contribute my changes.
• Kévin Commaille for making changes to the macro generation in Ruma that allow my datatypes to work correctly.
• The entirety of the Matrix SDK Rust and Cryptography teams, who graciously granted me their patience and expertise while reviewing my contributions.
• Andy and Hubert Chathi, who spared some time to read and comment on the many drafts of this post.