Plurality Support

artemis_everfree commented

2023-08-01 01:32:03 +02:00

Let's get the ball rolling on discussing plurality support. This is basically an infodump of everything I've thought about it as it relates to matrix, and we can go from there.

Background

There are many situations where someone might want to use different names and profile pictures in different social contexts. This is already supported to a degree in the protocol with per-room avatar/name events, though the UX leaves something to be desired if you want to change in multiple rooms.

However, there are also situations where we want a single account to be able to use multiple display names/profile pictures within a single conversation, in a way that is semantically important to the conversation. The two most well-known cases are plurality and role-playing.

This has to date not been terribly well supported in Matrix, and it is one of the common things stopping plural systems from being willing to leave Discord. We wrote a thread some time ago on mastodon about various ways to hack around this, under the assumption that we could not get support for it into the protocol. But here we are, so let's talk about it.

Plurality

In overly simplistic terms, plural people are in a situation where multiple people co-exist within the same body. Often we use "system" to refer to the entire set of people within the one body. This is not a comprehensive definition of the term, but we'll use this to discuss the issue as it relates to matrix.

People within a system often would like to speak as themselves, and that typically means having their own profile picture and name displayed alongside their messages rather than a system-wide picture/name. This caries semantic information to other parties, as the context of the author may imply the tone in which words are meant to be read, what information that author has (people within a system may have disjoint memories), or anything else that arises from the relationship between the reader and the author.

At present, the protocol has no good way to support this. It would in theory be possible to send an avatar/name change event prior to messages when one account changes author, but this is a loose association, and does not allow chat history to reflect that author changing their picture/name in the future.

It is also worth noting that, on Discord, systems may still make multiple accounts for a variety of reasons. Some of these are things we could avoid the need for multiple accounts for on matrix. For example, one main reason is so that the different accounts have access to different Discord guilds, or different channels within those guilds. In this way, some members of the system can opt-out of participation in certain spaces without an everpresent risk of stumbling into those spaces.

Role-playing

Anyone using a chat program for role-playing (TTRPGs, unstructured role-playing, etc.) also benefits from UX and protocol improvements made to support plurality. This has been proven out by the well-known Discord bot, PluralKit, which has become popular amongst both plural systems and roleplayers simultaneously.

PluralKit, the current standard for UX.

PluralKit is a Discord bot that hacks in support to allow multiple identities to speak from one or more accounts. To send a message with a specific identity attached to it, a user sends the message with a particular prefix and/or suffix ("proxy tags"). That prefix/suffix is optionally stripped from the message, and the remaining message is re-sent with the appropriate picture and name. This is known as "proxying".

Click to view animated GIF demonstrating proxying with PluralKit

There are some well-known issues with this approach:

Messages have to make its way through Discord, into the channel, and into PluralKit, before they can be reproxied. This can lead to messages flashing in/out of existence causing jarring shifts in the chat timeline, and can lead to message re-ordering
PluralKit uses Webhooks with a custom name/profile picture set per message. There is no way to retroactively change these.
Moderation load is increased, as server admins need to use pluralkit-specific tools to query sender information, etc.
Proxied messages evade per-user blocks.
Editing messages is cumbersome, integration with replies is bespoke, etc.

There are also some VERY GOOD things we should attempt to replicate:

Proxy tags provides a very low effort way for multiple people within a system to participate in a conversation in real time, without having to switch between accounts, click through menus, or anything like that.
Multiple Discord accounts can be linked to the same PluralKit system, allowing those accounts to share the same set of proxy tags and identities.
- We may not want to support this, but we should consider the context in which is is useful. As mentioned above, one reason to have multiple discord accounts is so that one account can participate in some spaces while another account does not have access. If this functionality was implemented within a single account, then the need to have a system linked to multiple accounts is decreased.
It is extremely simple to export and import the list of identities.
- This is particularly important in the case of Matrix to support migration between servers.
- If this is supported, then some aspects of linking a system to multiple accounts can be implemented after the fact.

How Many

Any ideal solution should account for some users having hundreds or thousands of identities. Most users will only have a handful, somewhere in the 2-30 range, but I know multiple systems that are hovering around 100 or so.

PluralKit has a default limit of 1000 identities for an account. I went and asked some communities I'm in and learned that some systems do hit that limit, and ask for that limit to be increased for their account.

There's a variety of reasons for that, but the end consequence is the same: UX and protocol decisions should accommodate a large number of identities linked to an account. Friction for creating and modifying identities should be low. A large quantity of identities should not cause performance problems. Any hard-cap on the number of identities an account can have should be generous. Most people will not approach those numbers, but some will.

Obviously there are computational limits we need to work within, but we should avoid getting ourselves stuck with inefficient decisions here.

Protocol Considerations

There are a number of ways to implement this, but one of the most promising I'm aware of is to introduce an additional UUID or similar identifier to messages that provides the ability to differentiate between authors. Then, we can also associate display names, pictures, or perhaps other metadata (pronouns?) with authors.

This also allows us to avoid the moderation/blocking woes that PluralKit suffers from.

Client-Server

Identity resolution should happen on the client, before messages are sent to the server. Clients can choose to attach a author ID to a message, or omit it, in which case the account's primary name/avatar will be displayed alongside a message.

To support proxy tags, it would be beneficial for the homeserver to retain a
copy of the proxy tags, that way clients can retrieve the list as necessary. Otherwise, tags would need to be configured per-device and per-client. We should just specify this.

Proxy tags, following pluralkit style, are something like

enum Tag {
	Prefix(String),
	Suffix(String),
	PrefixAndSuffix(String, String)
}

Identities are not required to have a tag, it is optional. If we're specifying this we should probably also specify the simple logic for matching a message against a list of tags.

Proxy tag resolution MUST happen client-side for them to work with end to end encryption.

Clients may implement other methods of proxying and tracking state. They may implement latching, (where-in the last used identity is maintained until a new proxy tag is sent), heck they could even integrate with the PluralKit API if the client author wanted to make them. The server does not need to care.

A Semblance of Backwards Compatibility

Old clients will not support these multiple identifiers, and perhaps some never will. What can we do to have some degree of graceful degradation? There are some things we could do:

Include proxy tags in the normal message body, add a new message body that contains a proxy-stripped message

With this method, old clients would not see a per-identity name/profile picture, but would potentially see proxy tags. For example, if I were to use the proxy prefix ve: for the identity "Vermillion", my client with support may display

<Vermillion> here is my message!

While their client without support may display

<artemis> ve: here is my message!

Some semantic content is lost, but enough is preserved to be useful. This is how many plural communities proxied, and still do proxy, on platforms that do not have good first-class plurality support such as IRC, so there is social context to make this work.

Clients that support the new method would instead render the proxy-stripped message.

Prefix the display name in the message body, add a new message field that specifies an offset which will skip the display name.

I'm imagining this like you have

{ "body": "<Vermillion> hello here is my message"
, "some_field_name": length("<Vermillion> ")
}

Or just duplicating the message as in the previous method, if we don't care about the data amplification as much.

and then old clients would display:

<artemis> <Vermillion> hello here is my message

This is very similar to the previous method, but the primary difference is including the full display name instead of proxy tags. This results in a similar display behavior as to discord/IRC bridges that prefix the author of a message in front of the bridged message instead of puppeting. This may be preferable, because some systems do not use proxy tags and use other methods that clients could potentially implement (such as switch-tracking, proxy-latching, etc.). Basically this just feels a bit more client-agnostic to me, but may result in more visual noise in chat logs for older clients if someone uses a long display name.

Send an old-style avatar/display name change event before each message, that newer clients will know to filter out

Somehow I like this the least, but it is an option we could do.

Ok that's all I have the energy to contribute right now

But please expand on this as seems fit, contradict me, ask for clarification, suggest sub-issues, etc.

Let's get the ball rolling on discussing plurality support. This is basically an infodump of everything I've thought about it as it relates to matrix, and we can go from there. ## Background There are many situations where someone might want to use different names and profile pictures in different social contexts. This is _already supported_ to a degree in the protocol with per-room avatar/name events, though the UX leaves something to be desired if you want to change in multiple rooms. However, there are also situations where we want a single account to be able to use multiple display names/profile pictures within a single conversation, in a way that is semantically important to the conversation. The two most well-known cases are plurality and role-playing. This has to date not been terribly well supported in Matrix, and it is one of the common things stopping plural systems from being willing to leave Discord. We wrote a thread some time ago [on mastodon](https://tenforward.social/@artemis/110535177666370157) about various ways to hack around this, under the assumption that we could not get support for it into the protocol. But here we are, so let's talk about it. ### Plurality In overly simplistic terms, [plural people](https://morethanone.info/) are in a situation where multiple people co-exist within the same body. Often we use "system" to refer to the entire set of people within the one body. This is not a comprehensive definition of the term, but we'll use this to discuss the issue as it relates to matrix. People within a system often would like to speak as themselves, and that typically means having their own profile picture and name displayed alongside their messages rather than a system-wide picture/name. This caries semantic information to other parties, as the context of the author may imply the tone in which words are meant to be read, what information that author has (people within a system may have disjoint memories), or anything else that arises from the relationship between the reader and the author. At present, the protocol has no good way to support this. It would in theory be possible to send an avatar/name change event prior to messages when one account changes author, but this is a loose association, and does not allow chat history to reflect that author changing their picture/name in the future. It is also worth noting that, on Discord, systems may still make multiple accounts for a variety of reasons. Some of these are things we could avoid the need for multiple accounts for on matrix. For example, one main reason is so that the different accounts have access to different Discord guilds, or different channels within those guilds. In this way, some members of the system can opt-out of participation in certain spaces without an everpresent risk of stumbling into those spaces. ### Role-playing Anyone using a chat program for role-playing (TTRPGs, unstructured role-playing, etc.) also benefits from UX and protocol improvements made to support plurality. This has been proven out by the well-known Discord bot, PluralKit, which has become popular amongst both plural systems and roleplayers simultaneously. ### PluralKit, the current standard for UX. [PluralKit](https://pluralkit.me/) is a Discord bot that hacks in support to allow multiple identities to speak from one **or more** accounts. To send a message with a specific identity attached to it, a user sends the message with a particular prefix and/or suffix ("proxy tags"). That prefix/suffix is optionally stripped from the message, and the remaining message is re-sent with the appropriate picture and name. This is known as "proxying". <details> <summary>Click to view animated GIF demonstrating proxying with PluralKit</summary> ![GIF animation demonstrating PluralKit](https://pluralkit.me/assets/img/demo.d0f1b467.gif) </details> There are some well-known issues with this approach: - Messages have to make its way through Discord, into the channel, and into PluralKit, before they can be reproxied. This can lead to messages flashing in/out of existence causing jarring shifts in the chat timeline, and can lead to message re-ordering - PluralKit uses Webhooks with a custom name/profile picture set per message. There is no way to retroactively change these. - Moderation load is increased, as server admins need to use pluralkit-specific tools to query sender information, etc. - Proxied messages evade per-user blocks. - Editing messages is cumbersome, integration with replies is bespoke, etc. There are also some VERY GOOD things we should attempt to replicate: - Proxy tags provides a very low effort way for multiple people within a system to participate in a conversation in real time, without having to switch between accounts, click through menus, or anything like that. - Multiple Discord accounts can be linked to the same PluralKit system, allowing those accounts to share the same set of proxy tags and identities. - We may not want to support this, but we should consider the context in which is is useful. As mentioned above, one reason to have multiple discord accounts is so that one account can participate in some spaces while another account does not have access. If this functionality was implemented within a single account, then the need to have a system linked to multiple accounts is decreased. - It is extremely simple to export and import the list of identities. - This is particularly important in the case of Matrix to support migration between servers. - If this is supported, then some aspects of linking a system to multiple accounts can be implemented after the fact. ## How Many Any ideal solution should account for some users having hundreds or thousands of identities. *Most* users will only have a handful, somewhere in the 2-30 range, but I know multiple systems that are hovering around 100 or so. PluralKit has a default limit of 1000 identities for an account. I went and asked some communities I'm in and learned that some systems do hit that limit, and ask for that limit to be increased for their account. There's a variety of reasons for that, but the end consequence is the same: UX and protocol decisions should accommodate a large number of identities linked to an account. Friction for creating and modifying identities should be low. A large quantity of identities should not cause performance problems. Any hard-cap on the number of identities an account can have should be generous. Most people will not approach those numbers, but _some_ will. Obviously there are computational limits we need to work within, but we should avoid getting ourselves stuck with inefficient decisions here. ## Protocol Considerations There are a number of ways to implement this, but one of the most promising I'm aware of is to introduce an additional UUID or similar identifier to messages that provides the ability to differentiate between authors. Then, we can also associate display names, pictures, or perhaps other metadata (pronouns?) with authors. This also allows us to avoid the moderation/blocking woes that PluralKit suffers from. ### Client-Server Identity resolution should happen on the client, before messages are sent to the server. Clients can choose to attach a author ID to a message, or omit it, in which case the account's primary name/avatar will be displayed alongside a message. To support proxy tags, it would be beneficial for the homeserver to retain a copy of the proxy tags, that way clients can retrieve the list as necessary. Otherwise, tags would need to be configured per-device and per-client. We should just specify this. Proxy tags, following pluralkit style, are something like ``` enum Tag { Prefix(String), Suffix(String), PrefixAndSuffix(String, String) } ``` Identities are not required to have a tag, it is optional. If we're specifying this we should probably also specify the simple logic for matching a message against a list of tags. Proxy tag resolution MUST happen client-side for them to work with end to end encryption. Clients may implement other methods of proxying and tracking state. They may implement latching, (where-in the last used identity is maintained until a new proxy tag is sent), heck they could even integrate with the PluralKit API if the client author wanted to make them. The server does not need to care. ## A Semblance of Backwards Compatibility Old clients will not support these multiple identifiers, and perhaps some never will. What can we do to have some degree of graceful degradation? There are some things we could do: ### Include proxy tags in the normal message body, add a new message body that contains a proxy-stripped message With this method, old clients would not see a per-identity name/profile picture, but would potentially see proxy tags. For example, if I were to use the proxy prefix `ve:` for the identity "Vermillion", my client with support may display ``` <Vermillion> here is my message! ``` While their client without support may display ``` <artemis> ve: here is my message! ``` Some semantic content is lost, but enough is preserved to be useful. This is how many plural communities proxied, and still do proxy, on platforms that do not have good first-class plurality support such as IRC, so there is social context to make this work. Clients that support the new method would instead render the proxy-stripped message. ### Prefix the display name in the message body, add a new message field that specifies an offset which will skip the display name. I'm imagining this like you have ``` { "body": "<Vermillion> hello here is my message" , "some_field_name": length("<Vermillion> ") } ``` Or just duplicating the message as in the previous method, if we don't care about the data amplification as much. and then old clients would display: ``` <artemis> <Vermillion> hello here is my message ``` This is very similar to the previous method, but the primary difference is including the full display name instead of proxy tags. This results in a similar display behavior as to discord/IRC bridges that prefix the author of a message in front of the bridged message instead of puppeting. This may be preferable, because some systems do not use proxy tags and use other methods that clients could potentially implement (such as switch-tracking, proxy-latching, etc.). Basically this just feels a bit more client-agnostic to me, but may result in more visual noise in chat logs for older clients if someone uses a long display name. ### Send an old-style avatar/display name change event before each message, that newer clients will know to filter out Somehow I like this the least, but it is an option we could do. ## Ok that's all I have the energy to contribute right now But please expand on this as seems fit, contradict me, ask for clarification, suggest sub-issues, etc.

👀 1

sorunome commented

2023-08-01 09:54:22 +02:00

Another method to go about this is something luke he tportable identities idea that has been floating around a lot, so that you can have multiple identities connected to one and the same account

artemis_everfree referenced this issue

2023-08-01 21:58:00 +02:00

Removing profile changes from the event graph #18

zocker.net commented

2023-08-02 10:55:22 +02:00

I’m a fan of this, allowing to have different personas for different contexts (hopefully with a better UX than now). I would propose to allow having encrypted profiles/personas for use in E2EE rooms (only). These should be protected enough so these are only visible to the users I allow them to see. This would e.g. allow one to use their real, full name (maybe with postal address) in the contexts where there are appropriate or needed without exposing the name to the whole web.

zocker.net commented

2023-08-02 11:03:53 +02:00

And for UX reasons: Profiles should be linkable. This should users allow to have one encrypted (maybe even private only) profile with all their data. Then, if they want to expose some of this data to others, they may just a create a “subprofile”, linked to the one full one, where they can select, which data should be linked & maybe customize some. This allows users to have all profiles changed automatically by changing their data once (e.g. when moving).

This should be handled by the own client side only (but be added to the spec for UX & compatibility reasons), so the links aren’t exposed and don’t need to be handled by the servers or participants directly.

However, this may be an extension to the spec later on, and can be ignored for now, if specifying & implementing that takes more time than required.

And for UX reasons: Profiles should be linkable. This should users allow to have one encrypted (maybe even private only) profile with all their data. Then, if they want to expose some of this data to others, they may just a create a “subprofile”, linked to the one full one, where they can select, which data should be linked & maybe customize some. This allows users to have all profiles changed automatically by changing their data once (e.g. when moving). This should be handled by the own client side only (but be added to the spec for UX & compatibility reasons), so the links aren’t exposed and don’t need to be handled by the servers or participants directly. However, this may be an extension to the spec later on, and can be ignored for now, if specifying & implementing that takes more time than required.

ckie referenced this issue

2023-08-02 18:41:01 +02:00

Identity indirection (Extensible profiles as rooms) #19

Rows
Columns

Plurality Support #14