Multi-relay routing by link2xt · Pull Request #7898 · chatmail/core

link2xt · 2026-02-25T21:45:58Z

This change adds information about all relay addresses to the public key and sends messages to all addresses listed in the public key contains the list of relays.

Public key is re-generated on the fly to include Direct Key Signature with Notation Data subpacket. Notation name is relays@chatmail.at and the value is a comma-separated list of addresses. We send all relays, but only use the first 3 now. Last seen "From" address is remembered for contacts and is used in the To header, as a fallback when relay list is not in the public key etc., but is not used as the recipient if it is not included in the notation subpacket as well. This will allow us to send from addresses that we don't want to receive messages to in the future if we decide to do so, e.g. send from a classic email server but receive chat messages only on chatmail relays.

There is a known problem that removing a transport does not restart I/O, so you receive messages on the removed transport and it creates ad hoc group as you don't recognize your own address in the To field. Fixing the problem is not for this PR, we probably want to restart i/o and also recognize that the message is a 1:1 message for us just by the intended recipient fingerprint even if there is an unrecognized address in the To field.

Closes #7865

TODO:

Fix tests/test_multitransport.py::test_download_on_demand. It is currently broken because receiving the same pre-message second time results in trashing the second pre-message and replacing first pre-message, so essentially the message that already existed as a pre-message gets trashed. Then you get problem from Post-message resulted in receive_imf failure #7872 when post-message download is automatically requested, but pre-message got trashed.
Bring back all the commented out signature subpackets or decide to drop them. SEIPDv2 feature is needed, we also want to recognize it and start sending SEIPDv2 if everyone supports it. Preferred algorithms and hashes practically just increase the signature size at the moment.
Have some way to update contact public key (aka OpenPGP certificate). Currently (as of 2.44.0) we never update public keys from Autocrypt header. Simple way to fix this is to accept updates, but only from the signed messages and remember the timestamp, so it is impossible to e.g. forge an update with dropped encryption subkey. But we also want to update from Autocrypt-Gossip, this will need more careful merging, e.g. keeping old certificate but replacing user ID with the one that has highest timestamp on the signature. Currently I just trust the updates from Autocrypt header. EDIT: implemented certificate merging
Merge certificates when vCard is imported as well.
Per-context lock around prefetching (IMAP loop) or around receive_imf. Currently multitransport tests with download on demand fail because two IMAP loops receive the same messages and download them in parallel and this is not handled properly. EDIT: I have been running this PR for some time and got duplicate message once when sending in a chat that has two my profiles. Duplicate messages have the same Message-ID in message info, so the problem is real even if this happens rarely.
Have some limit on the number of relays we take from the key, take no more than two or three relays.
Online Python test that after removing primary relay user can still receive messages. Currently it works already but message is assigned to ad hoc group because the message is not recognized as sent to us. This is supposed to work via "intended recipient fingerprint" (e.g. fix: receive_imf: Look up key contact by intended recipient fingerprint (#7661) #7786) but something does not work. The problem is apparently caused by not stopping IMAP loop when deleting the transport.
Add relay list of the contact to encryption info if it is in the public key (https://support.delta.chat/t/the-current-profile-in-v2-43-0-no-longer-provides-any-way-to-show-all-relays-design-regression/4873)
Only update the signature timestamp when actual changes happen to avoid leaking app start time, save the timestamp somewhere in the config (EDIT: using transport timestamp)

This way we can place whatever we want into the signatures. We put all relay addresses as a notation subpacket in the direct key signature to distribute the relay addresses.

src/pgp.rs

j-g00da · 2026-03-11T11:58:03Z

src/pgp.rs

+    } = new_certificate;
+
+    // Public keys may be serialized differently, e.g. using old and new packet type,
+    // so we compare imprints instead, which are calculated over normalized packets.


If we understand correctly imprints are used here instead of fingerprints, if that's the case,
can you clarify that imprints are compared instead of fingerprints?

Suggested change

// so we compare imprints instead, which are calculated over normalized packets.

// so we compare imprints, which are calculated over normalized packets (instead of fingerprints).

I changed the comment. Imprints are compared instead of the keys, not instead of the fingerprints. Imprint vs fingerprint does not matter much, the only difference is that fingerprints are SHA1 for v4 keys.

j-g00da · 2026-03-11T11:59:31Z

src/pgp.rs

+///
+/// See <https://openpgp.dev/book/adv/certificates.html#merging>
+/// and <https://openpgp.dev/book/adv/certificates.html#certificate-minimization>.
+pub fn merge_openpgp_certificates(


Not sure why we can't just take the newer key? Can you clarify this a bit?

I wrote more in the documentation comment now.

src/pgp.rs

src/mimeparser.rs

j-g00da · 2026-03-11T12:16:03Z

src/key.rs

+        )?;
+        vec![user_id_packet.into_signed(signature)]
+    } else {
+        vec![]


Can you clarify, why we are not including users for V6?

I added a comment describing why V4 keys include User ID.

j-g00da · 2026-03-11T12:23:59Z

src/key.rs

+    };
+
+    let users = if signed_secret_key.version() == KeyVersion::V4 {
+        let addr = context.get_primary_self_addr().await?;


Should we call context.self_public_key.lock().await.take(); when changing the primary self addr?

We already do when ConfiguredAddr is set, send_sync_transports is called and it drops the public key.
I added code in receive_imf to do it when primary transport is synchronized.

deltachat-rpc-client/tests/test_multitransport.py

Hocuri

FTR this PR does not add an "unpublished" flag to a transport. But this can also come in a later PR.

Hocuri · 2026-03-11T08:01:04Z

src/imap.rs

            // probably shutdown
            bail!("IMAP operation attempted while it is torn down");
        }
+        let _imap_lock_guard = context.imap_lock.lock().await;


Didn't we want to take the lock after calling prefetch()?

+1. Maybe also rename it to fetch_msgs_lock

src/key.rs

iequidoo · 2026-03-12T00:52:21Z

src/imap.rs

            // probably shutdown
            bail!("IMAP operation attempted while it is torn down");
        }
+        let _imap_lock_guard = context.imap_lock.lock().await;


+1. Maybe also rename it to fetch_msgs_lock

iequidoo · 2026-03-12T00:52:37Z

deltachat-rpc-client/tests/test_multitransport.py

-        addr = alice_transport["addr"]
-        assert (addr == new_alice_addr) == (addr in msg.get_info())
+    msg_info = msg.get_info()
+    assert new_alice_addr in msg.get_info()


Suggested change

assert new_alice_addr in msg.get_info()

assert new_alice_addr in msg_info

And maybe check for new_alice_addr + "/INBOX"?

iequidoo · 2026-03-12T00:52:43Z

src/contact.rs

+            && let Some(relay_addrs) = addresses_from_public_key(&public_key)
+        {
+            ret += "\n\nRelays:";
+            for relay in &relay_addrs {


.join("\n")?

iequidoo · 2026-03-12T00:52:49Z

src/key.rs

+                   FROM transports
+                   UNION ALL
+                   SELECT remove_timestamp AS timestamp
+                   FROM removed_transports)",


Don't we need one more timestamp for primary address change? The returned pubkey depends on which address is primary

src/pgp.rs

iequidoo · 2026-03-12T00:52:55Z

src/pgp.rs

+    old_certificate: SignedPublicKey,
+    new_certificate: SignedPublicKey,
+) -> Result<SignedPublicKey> {
+    old_certificate


The old certificate can expire so this will fail and we should just return the new one if key imprints are the same i guess

Certificate or the primary key cannot expire by itself, at least in OpenPGP, since V4. Direct key signature (or primary User ID signature if there is no direct key signature) of the primary key or subkey my have "key expiration time" subpacket that tells when the key/subkey expires, we currently ignore it. For encryption subkeys we want to use expiration eventually for forward secrecy, but for this PR i have not looked at encryption subkeys at all. For primary key expiration we will likely have to ignore it, saying that the contact has expired or telling the user that their profile has expired does not make much sense and if we do it now this will break profiles of the users who have in the past imported the keys with 1 year or 5 year expiration time and forgot about it.

Even if we have old certificate with expired primary key and new certificate with renewed primary key, this is just a number from the primary key signature. It does not tell you anything about encryption subkeys. E.g. you may have a contact with an encryption subkey and expired primary key. Someone may gossip you a certificate for this contact with a renewed primary key (newer DKS with extended expiration time) but no encryption subkeys - this does not mean you should now drop all encryption subkeys and accept new certificate without subkeys as a whole without merging.

…icate

link2xt force-pushed the link2xt/regen-user-id branch 3 times, most recently from 722e3e6 to 5be3918 Compare February 25, 2026 22:06

link2xt force-pushed the link2xt/keypair branch from 032442a to 0e45663 Compare February 26, 2026 23:49

link2xt force-pushed the link2xt/regen-user-id branch 3 times, most recently from d810ee5 to e0744da Compare February 27, 2026 01:17

link2xt force-pushed the link2xt/keypair branch from 0e45663 to d5fd33c Compare February 27, 2026 01:17

link2xt mentioned this pull request Feb 27, 2026

Post-message resulted in receive_imf failure #7872

Open

Base automatically changed from link2xt/keypair to main February 28, 2026 16:27

link2xt force-pushed the link2xt/regen-user-id branch 2 times, most recently from 79a7943 to 895db47 Compare March 1, 2026 22:40

link2xt changed the base branch from main to link2xt/qkzzkkylmtmk March 1, 2026 22:42

link2xt force-pushed the link2xt/regen-user-id branch from 895db47 to 48a715b Compare March 2, 2026 03:30

link2xt force-pushed the link2xt/qkzzkkylmtmk branch from a3deffe to 37e02ee Compare March 2, 2026 03:30

Base automatically changed from link2xt/qkzzkkylmtmk to main March 2, 2026 16:39

link2xt force-pushed the link2xt/regen-user-id branch 3 times, most recently from 527561e to fa350fb Compare March 2, 2026 21:20

link2xt self-assigned this Mar 2, 2026

link2xt force-pushed the link2xt/regen-user-id branch 8 times, most recently from 1b13ddd to 5554654 Compare March 3, 2026 02:55

link2xt changed the title ~~WIP: regenerate signed user ID in memory~~ WIP: multi-relay routing Mar 4, 2026

link2xt force-pushed the link2xt/regen-user-id branch from 5554654 to 9260fd5 Compare March 4, 2026 15:18

link2xt added 4 commits March 10, 2026 14:53

fix: update device chats at the end of configuration

f46b967

feat: merge OpenPGP certificates

2496296

WIP: regenerate OpenPGP certificate in memory

b313102

This way we can place whatever we want into the signatures. We put all relay addresses as a notation subpacket in the direct key signature to distribute the relay addresses.

Mutex to prevent fetching from multiple IMAP servers at the same time

8f60ea5

link2xt force-pushed the link2xt/regen-user-id branch from ed3868a to 8f60ea5 Compare March 10, 2026 14:53

link2xt added 2 commits March 10, 2026 15:09

Limit the number of returned relays for the key to 3

a3905cf

test: message can be received after deleting primary transport

13fe754

link2xt force-pushed the link2xt/regen-user-id branch 2 times, most recently from af884f1 to cbb16fa Compare March 10, 2026 20:38

link2xt added 4 commits March 10, 2026 21:21

add relay list to encryption info

669afb1

do not accept invalid addresses from the key

8952bea

reset key when transport is synced

2a70e38

use transport timestamp as signature timestamp

458ba69

link2xt force-pushed the link2xt/regen-user-id branch from cbb16fa to 458ba69 Compare March 10, 2026 21:23

link2xt changed the title ~~WIP: multi-relay routing~~ Multi-relay routing Mar 10, 2026

clippy fixes

3e1d88d

link2xt commented Mar 10, 2026

View reviewed changes

src/pgp.rs Outdated Show resolved Hide resolved

link2xt marked this pull request as ready for review March 10, 2026 21:37

j-g00da reviewed Mar 11, 2026

View reviewed changes

Hocuri reviewed Mar 11, 2026

View reviewed changes

link2xt added 2 commits March 11, 2026 17:19

Describe why User ID is included in V4 keys

73ce906

make secret_key_to_public_key non-async

fdcc6c7

iequidoo reviewed Mar 12, 2026

View reviewed changes

link2xt added 7 commits March 12, 2026 10:03

keep the verificiation

a39ccdb

use import_public_key when importing from attachment

85aa9e4

don't use random in tests

83883dc

document why merge_openpgp_certificates does not just take new certif…

0c2c123

…icate

clarify that imprints are compared instead of the keys

d7c0b30

reference recommendation to include User ID in V4 keys

68f7508

regenerate user ID when primary transport is changed

7d5f593

	// so we compare imprints instead, which are calculated over normalized packets.
	// so we compare imprints, which are calculated over normalized packets (instead of fingerprints).

	assert new_alice_addr in msg.get_info()
	assert new_alice_addr in msg_info

Uh oh!

Conversation

link2xt commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Hocuri left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

link2xt commented Feb 25, 2026 •

edited

Loading