3786 commits

Author	SHA1	Message	Date
Andrew Morgan	3eb92369ca	Fix mypy errors on Twisted 24.11.0 (#17998) ... Fixes various `mypy` errors associated with Twisted `24.11.0`. Hopefully addresses https://github.com/element-hq/synapse/issues/17075, though I've yet to test against `trunk`. Changes should be compatible with our currently pinned Twisted version of `24.7.0`.	2024-12-18 11:49:38 +00:00
Andrew Morgan	f1b0f9a4ef	Bump mypy from 1.11.2 to 1.12.1 and fix new typechecking errors (#17999) ... Supersedes https://github.com/element-hq/synapse/pull/17958. Awkwardly, the changes made to fix the mypy errors in 1.12.1 cause errors in 1.11.2. So you'll need to update your mypy version to 1.12.1 to eliminate typechecking errors during developing.	2024-12-18 11:42:17 +00:00
cynhr	f1ecf46647	Add email.tlsname config option (#17849) ... The existing `email.smtp_host` config option is used for two distinct purposes: it is resolved into the IP address to connect to, and used to (request via SNI and) validate the server's certificate if TLS is enabled. This new option allows specifying a different name for the second purpose. This is especially helpful, if `email.smtp_host` isn't a global FQDN, but something that resolves only locally (e.g. "localhost" to connect through the loopback interface, or some other internally routed name), that one cannot get a valid certificate for. Alternatives would of course be to specify a global FQDN as `email.smtp_host`, or to disable TLS entirely, both of which might be undesirable, depending on the SMTP server configuration.	2024-12-17 18:05:38 -06:00
V02460	57bf44941e	Add macaroon_secret_key_path config option (#17983) ... Another config option on my quest to a `*_path` variant for every secret. This time it’s `macaroon_secret_key_path`. Reading secrets from files has the security advantage of separating the secrets from the config. It also simplifies secrets management in Kubernetes. Also useful to NixOS users.	2024-12-16 18:01:33 -06:00
Shay	8208186e3c	Add some useful endpoints to Admin API (#17948) ... - Fetch the number of invites the provided user has sent after a given timestamp - Fetch the number of rooms the provided user has joined after a given timestamp, regardless if they have left/been banned from the rooms subsequently - Get report IDs of event reports where the provided user was the sender of the reported event	2024-12-16 13:27:34 -06:00
Wilson	eedab12e6d	forward requester id to check username for spam callbacks (#17916)	2024-12-13 14:17:41 +00:00
Travis Ralston	b90ad26ebc	Promote account suspension to stable (#17964) ... MSC: https://github.com/matrix-org/matrix-spec-proposals/pull/3823	2024-12-04 17:56:42 -06:00
Quentin Gliech	23b626f2e6	Support for MSC4190: device management for application services (#17705) ... This is an implementation of MSC4190, which allows appservices to manage their user's devices without /login & /logout. --------- Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2024-12-04 12:04:49 +01:00
manuroe	abf44ad324	MSC4076: Add disable_badge_count to pusher configuration (#17975) ... This PR implements [MSC4076: Let E2EE clients calculate app badge counts themselves (disable_badge_count)](https://github.com/matrix-org/matrix-spec-proposals/pull/4076).	2024-12-03 22:58:43 +00:00
Quentin Gliech	657dd5151e	Merge branch 'master' into develop	2024-12-03 17:44:48 +01:00
Devon Hudson	4b7154c585	Don't allow unsupported content-type ... Co-authored-by: Eric Eastwood <erice@element.io>	2024-12-03 09:53:21 +01:00
Eric Eastwood	4daa533e82	Sliding Sync: Fix state leaking on incremental sync	2024-12-03 09:53:21 +01:00
Eric Eastwood	e5d3bfba30	Sliding Sync: Include invite, ban, kick, targets when $LAZY-loading room members (#17947) ... Part of https://github.com/element-hq/synapse/issues/17929	2024-12-02 10:17:55 -06:00
Eric Eastwood	6a909aade2	Consolidate SSO redirects through /_matrix/client/v3/login/sso/redirect(/{idpId}) (#17972) ... Consolidate SSO redirects through `/_matrix/client/v3/login/sso/redirect(/{idpId})` Spawning from https://github.com/element-hq/sbg/pull/421#discussion_r1859497330 where we have a proxy that intercepts responses to `/_matrix/client/v3/login/sso/redirect(/{idpId})` in order to upgrade them to use OAuth 2.0 Pushed Authorization Requests (PAR). Instead of needing to intercept multiple endpoints that redirect to the authorization endpoint, it seems better to just have Synapse consolidate to a single flow. ### Testing strategy 1. Create a new OAuth application. I'll be using GitHub for example but there are [many options](be65a8ec01/docs/openid.md). Visit https://github.com/settings/developers -> **New OAuth App** - Application name: `Synapse local testing` - Homepage URL: `http://localhost:8008` - Authorization callback URL: `http://localhost:8008/_synapse/client/oidc/callback` 1. Update your Synapse `homeserver.yaml` ```yaml server_name: "my.synapse.server" public_baseurl: http://localhost:8008/ listeners: - port: 8008 bind_addresses: [ #'::1', '127.0.0.1' ] tls: false type: http x_forwarded: true resources: - names: [client, federation, metrics] compress: false # SSO login testing oidc_providers: - idp_id: github idp_name: Github idp_brand: "github" # optional: styling hint for clients discover: false issuer: "https://github.com/" client_id: "xxx" # TO BE FILLED client_secret: "xxx" # TO BE FILLED authorization_endpoint: "https://github.com/login/oauth/authorize" token_endpoint: "https://github.com/login/oauth/access_token" userinfo_endpoint: "https://api.github.com/user" scopes: ["read:user"] user_mapping_provider: config: subject_claim: "id" localpart_template: "{{ user.login }}" display_name_template: "{{ user.name }}" ``` 1. Start Synapse: `poetry run synapse_homeserver --config-path homeserver.yaml` 1. Visit `http://localhost:8008/_synapse/client/pick_idp?redirectUrl=http%3A%2F%2Fexample.com` 1. Choose GitHub 1. Notice that you're redirected to GitHub to sign in (`https://github.com/login/oauth/authorize?...`) Tested locally and works: 1. `http://localhost:8008/_synapse/client/pick_idp?idp=oidc-github&redirectUrl=http%3A//example.com` -> 1. `http://localhost:8008/_matrix/client/v3/login/sso/redirect/oidc-github?redirectUrl=http://example.com` -> 1. `https://github.com/login/oauth/authorize?response_type=code&client_id=xxx&redirect_uri=http%3A%2F%2Flocalhost%3A8008%2F_synapse%2Fclient%2Foidc%2Fcallback&scope=read%3Auser&state=xxx&nonce=xxx`	2024-11-29 11:26:37 -06:00
Richard van der Hoff	d80cd57c54	Fix new scheduled tasks jumping the queue (#17962) ... Currently, when a new scheduled task is added and its scheduled time has already passed, we set it to ACTIVE. This is problematic, because it means it will jump the queue ahead of all other SCHEDULED tasks; furthermore, if the Synapse process gets restarted, it will jump ahead of any ACTIVE tasks which have been started but are taking a while to run. Instead, we leave it set to SCHEDULED, but kick off a call to `_launch_scheduled_tasks`, which will decide if we actually have capacity to start a new task, and start the newly-added task if so.	2024-11-28 18:06:19 +00:00
Erik Johnston	3943d2fde7	Fix up logic for delaying sending read receipts over federation. (#17933) ... For context of why we delay read receipts, see https://github.com/matrix-org/synapse/issues/4730. Element Web often sends read receipts in quick succession, if it reloads the timeline it'll send one for the last message in the old timeline and again for the last message in the new timeline. This caused remote users to see a read receipt for older messages come through quickly, but then the second read receipt taking a while to arrive for the most recent message. There are two things going on in this PR: 1. There was a mismatch between seconds and milliseconds, and so we ended up delaying for far longer than intended. 2. Changing the logic to reuse the `DestinationWakeupQueue` (used for presence) The changes in logic are: - Treat the first receipt and subsequent receipts in a room in the same way - Whitelist certain classes of receipts to never delay being sent, i.e. receipts in small rooms, receipts for events that were sent within the last 60s, and sending receipts to the event sender's server. - The maximum delay a receipt can have before being sent to a server is 30s, and we'll send out receipts to remotes at least at 50Hz (by default) The upshot is that this should make receipts feel more snappy over federation. This new logic should send roughly between 10%–20% of transactions immediately on matrix.org.	2024-11-25 18:12:33 +00:00
Shay	4587decd67	Return suspended status when querying user account (#17952)	2024-11-22 12:37:19 +00:00
Travis Ralston	d0a474d312	Enable authenticated media by default (#17889) ... Co-authored-by: Olivier 'reivilibre <oliverw@matrix.org>	2024-11-20 14:48:22 +00:00
Richard van der Hoff	c5e89f5fae	Create one-off scheduled task to delete old OTKs (#17934) ... To work around the fact that, pre-https://github.com/element-hq/synapse/pull/17903, our database may have old one-time-keys that the clients have long thrown away the private keys for, we want to delete OTKs that look like they came from libolm. To spread the load a bit, without holding up other background database updates, we use a scheduled task to do the work.	2024-11-19 11:20:48 +00:00
Poruri Sai Rahul	c812a79422	Removal: Remove support for experimental msc3886 (#17638)	2024-11-13 14:10:20 +00:00
Benjamin Bouvier	bfb197c596	Fix typo in error message when a media ID isn't known (#17865)	2024-11-12 16:41:14 +00:00
Erik Johnston	cacd4fd7bd	Fix MSC4222 returning full state (#17915) ... There was a bug that meant we would return the full state of the room on incremental syncs when using lazy loaded members and there were no entries in the timeline. This was due to trying to use `state_filter or state_filter.all()` as a short hand for handling `None` case, however `state_filter` implements `__bool__` so if the state filter was empty it would be set to full. c.f. MSC4222 and #17888	2024-11-08 16:41:24 +00:00
Erik Johnston	c7a1d0aa1a	Fix Twisted tests with latest release (#17911) ... c.f. #17906 and #17907	2024-11-07 16:22:09 +00:00
Erik Johnston	d0fc1e904a	Fix cancellation tests with new Twisted. (#17906) ... The latest Twisted release changed how they implemented `__await__` on deferreds, which broke the machinery we used to test cancellation. This PR changes things a bit to instead patch the `__await__` method, which is a stable API. This mostly doesn't change the core logic, except for fixing two bugs: - We previously did not intercept all await points - After cancellation we now need to not only unblock currently blocked await points, but also make sure we don't block any future await points. c.f. https://github.com/twisted/twisted/pull/12226 --------- Co-authored-by: Devon Hudson <devon.dmytro@gmail.com>	2024-11-07 15:26:14 +00:00
Erik Johnston	77eafd47df	Fix other unit tests with latest twisted (#17907) ... There's also https://github.com/element-hq/synapse/pull/17906	2024-11-07 10:11:13 +00:00
Richard van der Hoff	2a321bac35	Issue one time keys in upload order (#17903) ... Currently, one-time-keys are issued in a somewhat random order. (In practice, they are issued according to the lexicographical order of their key IDs.) That can lead to a situation where a client gives up hope of a given OTK ever being used, whilst it is still on the server. Related: https://github.com/element-hq/element-meta/issues/2356	2024-11-06 22:21:06 +00:00
Devon Hudson	eda735e4bb	Remove support for python 3.8 (#17908) ... ### Pull Request Checklist <!-- Please read https://element-hq.github.io/synapse/latest/development/contributing_guide.html before submitting your pull request --> * [X] Pull request is based on the develop branch * [X] Pull request includes a [changelog file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog). The entry should: - Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from `EventStore` to `EventWorkerStore`.". - Use markdown where necessary, mostly for `code blocks`. - End with either a period (.) or an exclamation mark (!). - Start with a capital letter. - Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry. * [X] [Code style](https://element-hq.github.io/synapse/latest/code_style.html) is correct (run the [linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters)) --------- Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2024-11-06 19:36:01 +00:00
Alexander Udovichenko	211c31dbd7	Fix WheelTimer implementation that can expired timeout early (#17850) ... When entries insert in the end of timer queue, then unnecessary entry inserted (with duplicated key). This can lead to some timeouts expired early and consume memory.	2024-11-05 12:08:17 -06:00
Erik Johnston	361bdafb87	Add experimental support for MSC4222 (#17888) ... Basically, if the client sets a special query param on `/sync` v2 instead of responding with `state` at the *start* of the timeline, we instead respond with `state_after` at the *end* of the timeline. We do this by using the `current_state_delta_stream` table, which is actually reliable, rather than messing around with "state at" points on the timeline. c.f. MSC4222	2024-11-05 14:45:57 +00:00
Eric Eastwood	2c9ed5e510	Remove usage of internal header encoding API (#17894) ... ```py from twisted.web.http_headers import Headers Headers()._canonicalNameCaps Headers()._encodeName ``` Introduced in https://github.com/matrix-org/synapse/pull/15913 <- https://github.com/matrix-org/synapse/pull/15773	2024-11-04 12:20:07 -06:00
Eric Eastwood	0932c77539	Sliding Sync: Lazy-loading room members on incremental sync (remember memberships) (#17809) ... Lazy-loading room members on incremental sync and remember which memberships we've sent down the connection before (up-to 100) Fix https://github.com/element-hq/synapse/issues/17804	2024-11-04 10:17:58 -06:00
Travis Ralston	c705beebf7	Support & use stable endpoints for MSC4151 (#17374) ... https://github.com/matrix-org/matrix-spec-proposals/pull/4151 has finished FCP. See https://github.com/element-hq/synapse/issues/17373 for unstable endpoint removal --------- Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>	2024-10-31 09:55:30 +00:00
Lama	3ae80b0de4	Check if user is in room before being able to tag it (#17839) ... Fix #17819	2024-10-30 11:55:23 -05:00
Erik Johnston	83513b75f7	Speed up sliding sync by computing extensions in parallel (#17884) ... The main change here is to add a helper function `gather_optional_coroutines`, which works in a similar way as `yieldable_gather_results` but takes a set of coroutines rather than a function	2024-10-30 10:51:04 +00:00
Shay	58deef5eba	Add admin handler to list of handlers used for background tasks (#17847) ... Fixes #17823 While we're at it, makes a change where the redactions are sent as the admin if the user is not a member of the server (otherwise these fail with a "User must be our own" message).	2024-10-29 13:50:13 -05:00
Andrew Morgan	57efc8c03e	Add media tests for a CMYK JPEG image (#17786)	2024-10-23 18:26:01 +01:00
Eric Eastwood	a5e16a4ab5	Sliding Sync: Reset forgotten status when membership changes (like rejoining a room) (#17835) ... Reset `sliding_sync_membership_snapshots` -> `forgotten` status when membership changes (like rejoining a room). Fix https://github.com/element-hq/synapse/issues/17781 ### What was the problem before? Previously, if someone used `/forget` on one of their rooms, it would update `sliding_sync_membership_snapshots` as expected but when someone rejoined the room (or had any membership change), the upsert didn't overwrite and reset the `forgotten` status so it remained `forgotten` and invisible down the Sliding Sync endpoint.	2024-10-22 11:06:46 +01:00
dependabot[bot]	9512b84a72	Bump mypy from 1.10.1 to 1.11.2 (#17842) ... Bumps [mypy](https://github.com/python/mypy) from 1.10.1 to 1.11.2. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/python/mypy/blob/master/CHANGELOG.md">mypy's changelog</a>.</em></p> <blockquote> <h3>Mypy 1.11.2</h3> <ul> <li>Alternative fix for a union-like literal string (Ivan Levkivskyi, PR <a href="https://redirect.github.com/python/mypy/pull/17639">17639</a>)</li> <li>Unwrap <code>TypedDict</code> item types before storing (Ivan Levkivskyi, PR <a href="https://redirect.github.com/python/mypy/pull/17640">17640</a>)</li> </ul> <h3>Acknowledgements</h3> <p>Thanks to all mypy contributors who contributed to this release:</p> <ul> <li>Alex Waygood</li> <li>Alexander Leopold Shon</li> <li>Ali Hamdan</li> <li>Anders Kaseorg</li> <li>Ben Brown</li> <li>Bénédikt Tran</li> <li>bzoracler</li> <li>Christoph Tyralla</li> <li>Christopher Barber</li> <li>dexterkennedy</li> <li>gilesgc</li> <li>GiorgosPapoutsakis</li> <li>Ivan Levkivskyi</li> <li>Jelle Zijlstra</li> <li>Jukka Lehtosalo</li> <li>Marc Mueller</li> <li>Matthieu Devlin</li> <li>Michael R. Crusoe</li> <li>Nikita Sobolev</li> <li>Seo Sanghyeon</li> <li>Shantanu</li> <li>sobolevn</li> <li>Steven Troxler</li> <li>Tadeu Manoel</li> <li>Tamir Duberstein</li> <li>Tushar Sadhwani</li> <li>urnest</li> <li>Valentin Stanciu</li> </ul> <p>I’d also like to thank my employer, Dropbox, for supporting mypy development.</p> <h2>Mypy 1.10</h2> <p>We’ve just uploaded mypy 1.10 to the Python Package Index (<a href="https://pypi.org/project/mypy/">PyPI</a>). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:</p> <pre><code>python3 -m pip install -U mypy </code></pre> <p>You can read the full documentation for this release on <a href="http://mypy.readthedocs.io">Read the Docs</a>.</p> <h3>Support TypeIs (PEP 742)</h3> <p>Mypy now supports <code>TypeIs</code> (<a href="https://peps.python.org/pep-0742/">PEP 742</a>), which allows</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="789f02c83a"><code>789f02c</code></a> Bump version to 1.11.2</li> <li><a href="917cc75fd6"><code>917cc75</code></a> An alternative fix for a union-like literal string (<a href="https://redirect.github.com/python/mypy/issues/17639">#17639</a>)</li> <li><a href="7d805b364e"><code>7d805b3</code></a> Unwrap TypedDict item types before storing (<a href="https://redirect.github.com/python/mypy/issues/17640">#17640</a>)</li> <li><a href="32675dddfa"><code>32675dd</code></a> Revert &quot;Fix Literal strings containing pipe characters&quot; (<a href="https://redirect.github.com/python/mypy/issues/17638">#17638</a>)</li> <li><a href="778542b93a"><code>778542b</code></a> Revert &quot;Fix <code>RawExpressionType.accept</code> crash with <code>--cache-fine-grained</code>&quot; (<a href="https://redirect.github.com/python/mypy/issues/1">#1</a>...</li> <li><a href="14ab742dec"><code>14ab742</code></a> Bump version to 1.11.2+dev</li> <li><a href="570b90a7a3"><code>570b90a</code></a> Bump version to 1.11</li> <li><a href="b3a102ef31"><code>b3a102e</code></a> Fix <code>RawExpressionType.accept</code> crash with <code>--cache-fine-grained</code> (<a href="https://redirect.github.com/python/mypy/issues/17588">#17588</a>)</li> <li><a href="aec04c7448"><code>aec04c7</code></a> Fix PEP 604 isinstance caching (<a href="https://redirect.github.com/python/mypy/issues/17563">#17563</a>)</li> <li><a href="cb44e4d8f1"><code>cb44e4d</code></a> Fix <code>typing.TypeAliasType</code> being undefined on python &lt; 3.12 (<a href="https://redirect.github.com/python/mypy/issues/17558">#17558</a>)</li> <li>Additional commits viewable in <a href="https://github.com/python/mypy/compare/v1.10.1...v1.11.2">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-10-17 15:05:00 +00:00
Tulir Asokan	11bc9a1b3a	Implement MSC4210: Remove legacy mentions (#17783)	2024-10-14 14:24:28 +01:00
Eric Eastwood	adda2a4613	Sliding Sync: Slight optimization when fetching state for the room (get_events_as_list(...)) (#17718) ... Spawning from @kegsay [pointing out](https://matrix.to/#/!cnVVNLKqgUzNTOFQkz:matrix.org/$ExOO7J8uPUQSyH-9Uxc_QCa8jlXX9uK4VRtkSC0EI3o?via=element.io&via=matrix.org&via=jki.re) that the Sliding Sync endpoint doesn't handle a large room with a lot of state well on initial sync (requesting all state via `required_state: [ ["*","*"] ]`) (it just takes forever). After investigating further, the slow part is just `get_events_as_list(...)` fetching all of the current state ID's out for the room (which can be 100k+ events for rooms with a lot of membership). This is just a slow thing in Synapse in general and the same thing happens in Sync v2 or the `/state` endpoint. --- The only idea I had to improve things was to use `batch_iter` to only try fetching a fixed amount at a time instead of working with large maps, lists, and sets. This doesn't seem to have much effect though. There is already a `batch_iter(event_ids, 200)` in `_fetch_event_rows(...)` for when we actually have to touch the database and that's inside a queue to deduplicate work. I did notice one slight optimization to use `get_events_as_list(...)` directly instead of `get_events(...)`. `get_events(...)` just turns the result from `get_events_as_list(...)` into a dict and since we're just iterating over the events, we don't need the dict/map.	2024-10-14 13:47:35 +01:00
Erik Johnston	d025b5ab50	Correctly changes to required state config in sliding sync (#17785) ... Fixes https://github.com/element-hq/synapse/issues/17698 This handles `required_state` changes by checking if new state has been added to the config, and if so fetching and returning that from the current state. This also takes care to ensure that given a state entry S that is added, removed and then re-added that we do *not* send S down a second time if there have been no changes to S in the current state. This is fine for Rust SDK (as it just remembers all state), but we might decide not to do this behaviour in the MSC. If we decide to always send down S then its easy enough to rip out all the code. --------- Co-authored-by: Eric Eastwood <eric.eastwood@beta.gouv.fr>	2024-10-14 13:31:22 +01:00
Nathan	05576f0b4b	Added display_name_claim in jwt_config which sets the user's display name upon registration (#17708)	2024-10-09 12:21:08 +00:00
Shay	a5986ac229	Improvements to admin redact api (#17792) ... - better validation on user input - fix an early task completion - when checking membership in rooms, check for rooms user has been banned from as well	2024-10-08 14:23:21 +01:00
Andrew Ferrazzutti	006251a5d0	Add missing license header (#17799) ... Co-authored-by: Erik Johnston <erik@matrix.org>	2024-10-08 12:01:44 +01:00
Erik Johnston	422f3ecec1	Sliding sync: omit bump stamp when it is unchanged (#17788) ... This saves some DB lookups in rooms	2024-10-08 11:17:23 +01:00
Andrew Morgan	e8c8924b81	Clarify test_forget_when_not_left docstring (#17628)	2024-10-07 16:34:32 +01:00
V02460	e8e0f0fad7	Add config option redis.password_path (#17717) ... Adds the option to load the Redis password from a file, instead of giving it in the config directly. The code is similar to how it’s done for `registration_shared_secret_path`. I changed the example in the documentation to represent the best practice regarding the handling of secrets. Reading secrets from files has the security advantage of separating the secrets from the config. It also simplifies secrets management in Kubernetes.	2024-10-07 09:46:51 +01:00
Andrew Ferrazzutti	9920417723	Don't say MSC4140 is supported when it's disabled (#17780)	2024-10-04 13:42:34 +01:00
Erik Johnston	81e0f57800	Fix perf when streams don't change often (#17767) ... There is a bug with the `StreamChangeCache` where it would incorrectly return that all entities had changed if asked for entities changed *since* the earliest stream position. Note that for streams we use the inequalities: `$min_stream_id < stream_id <= $max_stream_id`, i.e. when we ask the stream change cache for all things that have changed since `$stream_id` we don't care for events that happened *at* `$stream_id`. Specifically: `_earliest_known_stream_pos` is the position at which we know that we'll have entries for all changes since that point, we can use the cache for any stream IDs that equal `_earliest_known_stream_pos`. `_earliest_known_stream_pos` is set in three places: - On startup we set it either to: - the current maximum stream ID, with not prefilled values; or - the minimum of the latest N values we pulled from the DB - When we evict items from the bottom, we set it to the stream ID of the evicted items. This was changed in https://github.com/matrix-org/synapse/pull/14435, but I think we were overly conservative there. --------- Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2024-09-30 13:52:33 +01:00
Erik Johnston	93889eb2e7	Optimise notifier (#17765) ... The notifier is quite inefficient when it has to wake up many user streams all at once From a silly benchmark this takes the time to notify 1M user streams from ~30s to ~5s	2024-09-30 12:58:13 +01:00