Bump version to 0.11.3

Remove some unused fields
Update mautrix-python
2022-04-17 13:30:38 +03:00 · 2022-04-13 14:43:53 +03:00 · 2022-04-09 20:52:45 +03:00 · 2022-04-09 20:46:25 +03:00 · 2022-04-08 18:06:24 +03:00 · 2022-04-06 19:04:12 +03:00
51 changed files with 2520 additions and 1292 deletions
@@ -17,5 +17,5 @@ max_line_length = 99
 [*.{yaml,yml,py}]
 indent_style = space

-[.gitlab-ci.yml]
+[{.gitlab-ci.yml,.pre-commit-config.yaml,mautrix_telegram/web/provisioning/spec.yaml}]
 indent_size = 2
@@ -13,6 +13,14 @@ jobs:
    - uses: isort/isort-action@master
      with:
        sortPaths: "./mautrix_telegram"
-    - uses: psf/black@21.12b0
+    - uses: psf/black@stable
      with:
        src: "./mautrix_telegram"
+        version: "22.3.0"
+    - name: pre-commit
+      run: |
+        pip install pre-commit
+        pre-commit run -av trailing-whitespace
+        pre-commit run -av end-of-file-fixer
+        pre-commit run -av check-yaml
+        pre-commit run -av check-added-large-files
@@ -0,0 +1,20 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.1.0
+    hooks:
+      - id: trailing-whitespace
+        exclude_types: [markdown]
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-added-large-files
+  - repo: https://github.com/psf/black
+    rev: 22.3.0
+    hooks:
+      - id: black
+        language_version: python3
+        files: ^mautrix_telegram/.*\.pyi?$
+  - repo: https://github.com/PyCQA/isort
+    rev: 5.10.1
+    hooks:
+      - id: isort
+        files: ^mautrix_telegram/.*\.pyi?$
@@ -1,9 +1,80 @@
-# v0.11.1 (2021-01-10)
+# v0.11.3 (2022-04-17)

+**N.B.** This release drops support for old homeservers which don't support the
+new `/v3` API endpoints. Synapse 1.48+, Dendrite 0.6.5+ and Conduit 0.4.0+ are
+supported. Legacy `r0` API support can be temporarily re-enabled with `pip install mautrix==0.16.0`.
+However, this option will not be available in future releases.
+
+### Added
+* Added `list-invite-links` command to list invite links in a chat.
+* Added option to use [MSC2246] async media uploads.
+* Provisioning API for listing contacts and starting private chats.
+
+### Improved
+* Dropped Python 3.7 support.
+* Telegram->Matrix message formatter will now replace `t.me/c/chatid/messageid`
+  style links with a link to the bridged Matrix event (in addition to the
+  previously supported `t.me/username/messageid` links).
+* Updated formatting converter to keep newlines in code blocks as `\n` instead
+  of converting them to `<br/>`.
+* Removed `max_document_size` option. The bridge will now fetch the max size
+  automatically using the media repo config endpoint.
+* Removed redundant `msgtype` field in sticker events sent to Matrix.
+* Disabled file logging in Docker image by default.
+  * If you want to enable it, set the `filename` in the file log handler to a
+    path that is writable, then add `"file"` back to `logging.root.handlers`.
+* Reactions are now marked as read when bridging read receipts from Matrix.
+
+### Fixed
+* Fixed `!tg bridge` throwing error if the parameter is not an integer
+* Fixed `!tg bridge` failing if the command had been previously run with an
+  incorrectly prefixed chat ID (e.g. `!tg bridge -1234567` followed by
+  `!tg bridge -1001234567`).
+* Fixed `bridge_matrix_leave` config option not actually being used correctly.
+* Fixed public channel mentions always bridging into a user mention on Matrix
+  rather than a room mention.
+  * The bridge will now make room mentions if the portal exists and fall back
+    to user mentions otherwise.
+* Fixed newlines being lost in unformatted forwarded messages.
+
+[MSC2246]: https://github.com/matrix-org/matrix-spec-proposals/pull/2246
+
+# v0.11.2 (2022-02-14)
+
+**N.B.** This will be the last release to support Python 3.7. Future versions
+will require Python 3.8 or higher. In general, the mautrix bridges will only
+support the lowest Python version in the latest Debian or Ubuntu LTS.
+
+### Added
+* Added simple fallback message for live location and venue messages from Telegram.
+* Added support for `t.me/+code` style invite links in `!tg join`.
+* Added support for showing channel profile when users send messages as a channel.
+* Added "user joined Telegram" message when Telegram auto-creates a DM chat for
+  a new user.
+
+### Improved
+* Added option for adding a random prefix to relayed user displaynames to help
+  distinguish them on the Telegram side.
+* Improved syncing profile info to room info when using encryption and/or the
+  `private_chat_profile_meta` config option.
+* Removed legacy `community_id` config option.
+
+### Fixed
+* Fixed newlines disappearing when bridging channel messages with signatures.
+* Fixed login throwing an error if a previous login code expired.
+* Fixed bug in v0.11.0 that broke `!tg create`.
+
+# v0.11.1 (2022-01-10)
+
+### Added
 * Added support for message reactions.
 * Added support for spoiler text.
-* Improved support for voice messages.
-* Improved color of blue text from Telegram to be more readable on dark themes.
+
+### Improved
+* Support for voice messages.
+* Changed color of blue text from Telegram to be more readable on dark themes.
+
+### Fixed
 * Fixed syncing contacts throwing an error for new accounts.
 * Fixed migrating pre-v0.11 legacy databases if the database schema had been
  corrupted (e.g. by using 3rd party tools for SQLite -> Postgres migration).
@@ -199,8 +270,8 @@ path.
 * Bridging events of a user whose power level is malformed (i.e. a string
  instead of an integer) now works.

-[MSC2409]: https://github.com/matrix-org/matrix-doc/pull/2409
-[MSC2778]: https://github.com/matrix-org/matrix-doc/pull/2778
+[MSC2409]: https://github.com/matrix-org/matrix-spec-proposals/pull/2409
+[MSC2778]: https://github.com/matrix-org/matrix-spec-proposals/pull/2778

 # v0.8.2 (2020-07-27)

@@ -248,7 +319,7 @@ update (v0.5.8) and a fix to the Docker image.
 * Fixed `sync_direct_chats` option creating non-working portals.
 * Fixed video thumbnailing sometimes leaving behind downloaded videos in `/tmp`.

-[MSC2346]: https://github.com/matrix-org/matrix-doc/pull/2346
+[MSC2346]: https://github.com/matrix-org/matrix-spec-proposals/pull/2346

 ## rc1 (2020-04-25)

@@ -0,0 +1,3 @@
+pre-commit>=2.10.1,<3
+isort>=5.10.1,<6
+black>=22.3,<23
@@ -2,7 +2,13 @@

 # Define functions.
 function fixperms {
-	chown -R $UID:$GID /data /opt/mautrix-telegram
+	chown -R $UID:$GID /data
+
+	# /opt/mautrix-telegram is read-only, so disable file logging if it's pointing there.
+	if [[ "$(yq e '.logging.handlers.file.filename' /data/config.yaml)" == "./mautrix-telegram.log" ]]; then
+		yq -I4 e -i 'del(.logging.root.handlers[] | select(. == "file"))' /data/config.yaml
+		yq -I4 e -i 'del(.logging.handlers.file)' /data/config.yaml
+	fi
 }

 cd /opt/mautrix-telegram
@@ -18,7 +24,10 @@ if [ ! -f /data/config.yaml ]; then
 fi

 if [ ! -f /data/registration.yaml ]; then
-	python3 -m mautrix_telegram -g -c /data/config.yaml -r /data/registration.yaml
+	python3 -m mautrix_telegram -g -c /data/config.yaml -r /data/registration.yaml || exit $?
+	echo "Didn't find a registration file."
+	echo "Generated one for you."
+	echo "See https://docs.mau.fi/bridges/general/registering-appservices.html on how to use it."
 	fixperms
 	exit
 fi
@@ -1,2 +1,2 @@
-__version__ = "0.11.1"
+__version__ = "0.11.3"
 __author__ = "Tulir Asokan <tulir@maunium.net>"
@@ -86,6 +86,7 @@ class TelegramBridge(Bridge):
        Portal.init_cls(self)
        self.add_startup_actions(Puppet.init_cls(self))
        self.add_startup_actions(User.init_cls(self))
+        self.add_startup_actions(Portal.restart_scheduled_disappearing())
        if self.bot:
            self.add_startup_actions(self.bot.start())
        if self.config["bridge.resend_bridge_info"]:
@@ -1,5 +1,5 @@
 # mautrix-telegram - A Matrix-Telegram puppeting bridge
-# Copyright (C) 2021 Tulir Asokan
+# Copyright (C) 2022 Tulir Asokan
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as published by
@@ -15,7 +15,7 @@
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 from __future__ import annotations

-from typing import TYPE_CHECKING, Any, Type, Union
+from typing import TYPE_CHECKING, Any, Union
 from abc import ABC, abstractmethod
 import asyncio
 import logging
@@ -34,6 +34,7 @@ from telethon.tl.types import (
    Chat,
    MessageActionChannelMigrateFrom,
    MessageEmpty,
+    PeerChannel,
    PeerChat,
    PeerUser,
    TypeUpdate,
@@ -147,7 +148,7 @@ class AbstractUser(ABC):
        return self.client and self.client.is_connected()

    @property
-    def _proxy_settings(self) -> tuple[Type[Connection], tuple[Any, ...] | None]:
+    def _proxy_settings(self) -> tuple[type[Connection], tuple[Any, ...] | None]:
        proxy_type = self.config["telegram.proxy.type"].lower()
        connection = ConnectionTcpFull
        connection_data = (
@@ -385,7 +386,7 @@ class AbstractUser(ABC):
        if not message:
            return

-        puppet = await pu.Puppet.get_by_tgid(TelegramID(update.peer.user_id))
+        puppet = await pu.Puppet.get_by_peer(update.peer)
        await puppet.intent.mark_read(portal.mxid, message.mxid)

    async def update_own_read_receipt(
@@ -444,10 +445,7 @@ class AbstractUser(ABC):
            return

        if isinstance(update, (UpdateChannelUserTyping, UpdateChatUserTyping)):
-            # Can typing notifications come from non-user peers?
-            if not update.from_id.user_id:
-                return
-            sender = await pu.Puppet.get_by_tgid(TelegramID(update.from_id.user_id))
+            sender = await pu.Puppet.get_by_peer(update.from_id)

        if not sender or not portal or not portal.mxid:
            return
@@ -456,8 +454,8 @@ class AbstractUser(ABC):

    async def _handle_entity_updates(self, entities: dict[int, User | Chat | Channel]) -> None:
        try:
-            users = (entity for entity in entities.values() if isinstance(entity, User))
-            puppets = ((await pu.Puppet.get_by_tgid(TelegramID(user.id)), user) for user in users)
+            users = (entity for entity in entities.values() if isinstance(entity, (User, Channel)))
+            puppets = ((await pu.Puppet.get_by_peer(user), user) for user in users)
            await asyncio.gather(
                *[puppet.try_update_info(self, info) async for puppet, info in puppets if puppet]
            )
@@ -471,9 +469,11 @@ class AbstractUser(ABC):
            puppet.username = update.username
            if await puppet.update_displayname(self, update):
                await puppet.save()
+                await puppet.update_portals_meta()
        elif isinstance(update, UpdateUserPhoto):
            if await puppet.update_avatar(self, update.photo):
                await puppet.save()
+                await puppet.update_portals_meta()
        else:
            self.log.warning(f"Unexpected other user info update: {type(update)}")

@@ -515,8 +515,8 @@ class AbstractUser(ABC):
            portal = await po.Portal.get_by_entity(update.peer_id, tg_receiver=self.tgid)
            if update.out:
                sender = await pu.Puppet.get_by_tgid(self.tgid)
-            elif isinstance(update.from_id, PeerUser):
-                sender = await pu.Puppet.get_by_tgid(TelegramID(update.from_id.user_id))
+            elif isinstance(update.from_id, (PeerUser, PeerChannel)):
+                sender = await pu.Puppet.get_by_peer(update.from_id)
            else:
                sender = None
        else:
@@ -225,7 +225,11 @@ class Bot(AbstractUser):
        elif isinstance(message.to_id, PeerChat):
            return reply(str(-message.to_id.chat_id))
        elif isinstance(message.to_id, PeerUser):
-            return reply(f"Your user ID is {message.to_id.user_id}.")
+            return reply(
+                f"Your user ID is {message.to_id.user_id}.\n\n"
+                f"If you're trying to bridge a group chat to Matrix, you must run the command in "
+                f"the group, not here. **The ID above will not work** with `!tg bridge`."
+            )
        else:
            return reply("Failed to find chat ID.")

@@ -137,9 +137,9 @@ class CommandHandler(BaseCommandHandler):

    async def get_permission_error(self, evt: CommandEvent) -> str | None:
        if self.needs_puppeting and not evt.sender.puppet_whitelisted:
-            return "This command requires puppeting privileges."
+            return "That command is limited to users with puppeting privileges."
        elif self.needs_matrix_puppeting and not evt.sender.matrix_puppet_whitelisted:
-            return "This command requires Matrix puppeting privileges."
+            return "That command is limited to users with full puppeting privileges."
        return await super().get_permission_error(evt)

    def has_permission(self, key: HelpCacheKey) -> bool:
@@ -81,5 +81,5 @@ async def enter_matrix_token(evt: CommandEvent) -> EventID:
    except InvalidAccessToken:
        return await evt.reply("Failed to verify access token.")
    return await evt.reply(
-        "Replaced your Telegram account's Matrix puppet with {puppet.custom_mxid}."
+        f"Replaced your Telegram account's Matrix puppet with {puppet.custom_mxid}."
    )
@@ -59,17 +59,22 @@ async def bridge(evt: CommandEvent) -> EventID:

    # The /id bot command provides the prefixed ID, so we assume
    tgid_str = evt.args[0]
-    if tgid_str.startswith("-100"):
-        tgid = TelegramID(int(tgid_str[4:]))
-        peer_type = "channel"
-    elif tgid_str.startswith("-"):
-        tgid = TelegramID(-int(tgid_str))
-        peer_type = "chat"
-    else:
+    tgid = None
+    try:
+        if tgid_str.startswith("-100"):
+            tgid = TelegramID(int(tgid_str[4:]))
+            peer_type = "channel"
+        elif tgid_str.startswith("-"):
+            tgid = TelegramID(-int(tgid_str))
+            peer_type = "chat"
+    except ValueError:
+        # Invalid integer
+        pass
+    if not tgid:
        return await evt.reply(
            "That doesn't seem like a prefixed Telegram chat ID.\n\n"
-            "If you did not get the ID using the `/id` bot command, please "
-            "prefix channel IDs with `-100` and normal group IDs with `-`.\n\n"
+            "If you did not get the ID using the `/id` bot command, please prefix"
+            "channel/supergroup IDs with `-100` and non-super group IDs with `-`.\n\n"
            "Bridging private chats to existing rooms is not allowed."
        )

@@ -80,7 +85,7 @@ async def bridge(evt: CommandEvent) -> EventID:
            "If you're the bridge admin, try "
            "`$cmdprefix+sp filter whitelist <Telegram chat ID>` first."
        )
-    if portal.mxid:
+    elif portal.mxid:
        has_portal_message = (
            "That Telegram chat already has a portal at "
            f"[{portal.alias or portal.mxid}](https://matrix.to/#/{portal.mxid}). "
@@ -96,7 +101,7 @@ async def bridge(evt: CommandEvent) -> EventID:
            "mxid": portal.mxid,
            "bridge_to_mxid": room_id,
            "tgid": portal.tgid,
-            "peer_type": portal.peer_type,
+            "peer_type": peer_type,
            "force_use_bot": force_use_bot,
        }
        return await evt.reply(
@@ -112,7 +117,7 @@ async def bridge(evt: CommandEvent) -> EventID:
        "action": "Room bridging",
        "bridge_to_mxid": room_id,
        "tgid": portal.tgid,
-        "peer_type": portal.peer_type,
+        "peer_type": peer_type,
        "force_use_bot": force_use_bot,
    }
    return await evt.reply(
@@ -163,6 +168,18 @@ async def confirm_bridge(evt: CommandEvent) -> EventID | None:
    is_logged_in = await evt.sender.is_logged_in() and not status["force_use_bot"]

    if "mxid" in status:
+        if portal.peer_type != status["peer_type"]:
+            evt.log.warning(
+                "Portal %d in database has mismatching peer type %s (expected %s),"
+                " trusting database as a room already existed",
+                portal.tgid,
+                portal.peer_type,
+                status["peer_type"],
+            )
+            await evt.reply(
+                "Mismatching peer type in command and portal table, "
+                "trusting portal as room already existed"
+            )
        ok, coro = await cleanup_old_portal_while_bridging(evt, portal)
        if not ok:
            return None
@@ -181,6 +198,19 @@ async def confirm_bridge(evt: CommandEvent) -> EventID | None:
            "Please use `$cmdprefix+sp continue` to confirm the bridging or "
            "`$cmdprefix+sp cancel` to cancel."
        )
+    elif portal.peer_type != status["peer_type"]:
+        evt.log.warning(
+            "Portal %d in database has mismatching peer type %s (expected %s),"
+            " trusting new peer type as there's no existing room",
+            portal.tgid,
+            portal.peer_type,
+            status["peer_type"],
+        )
+        await evt.reply(
+            "Mismatching peer type in command and portal table, "
+            "trusting you as portal room doesn't exist"
+        )
+        portal.peer_type = status["peer_type"]

    evt.sender.command_status = None
    async with portal._room_create_lock:
@@ -221,7 +251,7 @@ async def _locked_confirm_bridge(
    await portal.save()
    await portal.update_bridge_info()

-    asyncio.create_task(portal.update_matrix_room(user, entity, direct=False, levels=levels))
+    asyncio.create_task(portal.update_matrix_room(user, entity, levels=levels))

    await warn_missing_power(levels, evt)

@@ -65,7 +65,7 @@ async def create(evt: CommandEvent) -> EventID:
        about=about,
        encrypted=encrypted,
    )
-    invites, errors = await portal.get_telegram_users_in_matrix_room(evt.sender)
+    invites, errors = await portal.get_telegram_users_in_matrix_room(evt.sender, pre_create=True)
    if len(errors) > 0:
        error_list = "\n".join(f"* [{mxid}](https://matrix.to/#/{mxid})" for mxid in errors)
        await evt.reply(
@@ -25,12 +25,22 @@ from telethon.errors import (
    UsernameNotModifiedError,
    UsernameOccupiedError,
 )
+from telethon.helpers import add_surrogate
 from telethon.tl.functions.channels import GetFullChannelRequest
-from telethon.tl.functions.messages import GetFullChatRequest
+from telethon.tl.functions.messages import GetExportedChatInvitesRequest, GetFullChatRequest
+from telethon.tl.types import (
+    ChatInviteExported,
+    InputMessageEntityMentionName,
+    InputUserSelf,
+    MessageEntityMention,
+    TypeInputPeer,
+    TypeInputUser,
+)
+from telethon.tl.types.messages import ExportedChatInvites

 from mautrix.types import EventID

-from ... import portal as po
+from ... import formatter as fmt, portal as po, puppet as pu
 from .. import SECTION_MISC, SECTION_PORTAL_MANAGEMENT, CommandEvent, command_handler
 from .util import user_has_power_level

@@ -101,30 +111,37 @@ async def get_id(evt: CommandEvent) -> EventID:


 invite_link_usage = (
-    "**Usage:** `$cmdprefix+sp invite-link [--uses=<amount>] [--expire=<delta>]`"
+    "**Usage:** `$cmdprefix+sp invite-link "
+    "[--uses=<amount>] [--expire=<delta>] [--request-needed] -- [title]`"
    "\n\n"
    "* `--uses`: the number of times the invite link can be used."
    "            Defaults to unlimited.\n"
    "* `--expire`: the duration after which the link will expire."
-    "              A number suffixed with d(ay), h(our), m(inute) or s(econd)"
+    "              A number suffixed with d(ay), h(our), m(inute) or s(econd)\n"
+    "* `--request-needed`: should the link require admins to approve joins?\n"
+    "* `title`: a description of the link (only shown to admins)."
 )


 def _parse_flag(args: list[str]) -> tuple[str, str]:
    arg = args.pop(0).lower()
+    if arg == "--":
+        return "", ""
+    value = ""
    if arg.startswith("--"):
-        value_start = arg.index("=")
-        if value_start:
+        value_start = arg.find("=")
+        if value_start > 0:
            flag = arg[2:value_start]
            value = arg[value_start + 1 :]
        else:
            flag = arg[2:]
-            value = args.pop(0).lower()
+            if arg not in ("request", "request-needed"):
+                value = args.pop(0).lower()
    elif arg.startswith("-"):
        flag = arg[1]
        if len(arg) > 3 and arg[2] == "=":
            value = arg[3:]
-        else:
+        elif arg != "r":
            value = args.pop(0).lower()
    else:
        raise ValueError("invalid flag")
@@ -159,18 +176,24 @@ def _parse_delta(value: str) -> timedelta | None:
@command_handler(
    help_section=SECTION_PORTAL_MANAGEMENT,
    help_text="Get a Telegram invite link to the current chat.",
-    help_args="[--uses=<amount>] [--expire=<time delta, e.g. 1d>]",
+    help_args="[--uses=<amount>] [--expire=<time delta, e.g. 1d>] [--request-needed] -- [title]",
 )
 async def invite_link(evt: CommandEvent) -> EventID:
+    if not evt.is_portal:
+        return await evt.reply("This is not a portal room.")
+
    # TODO once we switch to Python 3.9 minimum, use argparse with exit_on_error=False
    uses = None
    expire = None
+    request_needed = False
    while evt.args:
        try:
            flag, value = _parse_flag(evt.args)
        except (ValueError, IndexError):
            return await evt.reply(invite_link_usage)
-        if flag in ("uses", "u"):
+        if not flag:
+            break
+        elif flag in ("uses", "u"):
            try:
                uses = int(value)
            except ValueError:
@@ -180,23 +203,90 @@ async def invite_link(evt: CommandEvent) -> EventID:
            if not expire_delta:
                await evt.reply("Invalid format for expiry time delta")
            expire = datetime.now() + expire_delta
+        elif flag in ("request", "request-needed", "r"):
+            request_needed = True
+    title = " ".join(evt.args)

-    portal = await po.Portal.get_by_mxid(evt.room_id)
-    if not portal:
-        return await evt.reply("This is not a portal room.")
-
-    if portal.peer_type == "user":
+    if evt.portal.peer_type == "user":
        return await evt.reply("You can't invite users to private chats.")

    try:
-        link = await portal.get_invite_link(evt.sender, uses=uses, expire=expire)
-        return await evt.reply(f"Invite link to {portal.title}: {link}")
+        link = await evt.portal.get_invite_link(
+            evt.sender, uses=uses, expire=expire, request_needed=request_needed, title=title
+        )
+        return await evt.reply(f"Invite link to {evt.portal.title}: {link}")
    except ValueError as e:
        return await evt.reply(e.args[0])
    except ChatAdminRequiredError:
        return await evt.reply("You don't have the permission to create an invite link.")


+async def _format_invite_link(link: ChatInviteExported) -> str:
+    desc = f"* {link.link}"
+    if link.title:
+        desc += f" - {link.title}"
+    if link.expire_date:
+        desc += f"  \n  Expires at {link.expire_date.isoformat()}"
+    if link.usage_limit:
+        desc += f"  \n  Used {link.usage or 0} out of {link.usage_limit} times"
+    elif link.usage:
+        desc += f"  \n  Used {link.usage} times"
+    else:
+        desc += "  \n  Never used"
+    if link.request_needed:
+        desc += "  \n  Join requests enabled - using link requires admin approval"
+    return desc
+
+
+async def _hacky_find_mention(evt: CommandEvent) -> TypeInputUser | TypeInputPeer | None:
+    if len(evt.args) == 0:
+        return None
+    text, entities = await fmt.matrix_to_telegram(
+        evt.sender.client, text=evt.content.body, html=evt.content.formatted_body
+    )
+    for entity in entities:
+        if isinstance(entity, MessageEntityMention):
+            admin_username = add_surrogate(text)[entity.offset + 1 : entity.offset + entity.length]
+            return await evt.sender.client.get_input_entity(admin_username)
+        elif isinstance(entity, InputMessageEntityMentionName):
+            return entity.user_id
+    return None
+
+
+@command_handler(
+    help_section=SECTION_PORTAL_MANAGEMENT,
+    help_text="List existing Telegram invite links to the current chat.",
+    help_args="[creator]",
+)
+async def list_invite_links(evt: CommandEvent) -> EventID:
+    admin_id = InputUserSelf()
+    try:
+        admin_id = await _hacky_find_mention(evt) or InputUserSelf()
+    except Exception:
+        pass
+    resp: ExportedChatInvites = await evt.sender.client(
+        GetExportedChatInvitesRequest(
+            peer=await evt.portal.get_input_entity(evt.sender),
+            admin_id=admin_id,
+            limit=100,
+        )
+    )
+    if resp.count == 0:
+        if isinstance(admin_id, InputUserSelf):
+            return await evt.reply("You haven't created any invite links to the current chat")
+        else:
+            return await evt.reply("That user hasn't created any invite links to the current chat")
+    formatted_links = "\n".join([await _format_invite_link(link) for link in resp.invites])
+    if isinstance(admin_id, InputUserSelf):
+        await evt.reply(f"Your links to this chat:\n\n{formatted_links}")
+    else:
+        puppet = await pu.Puppet.get_by_peer(admin_id)
+        await evt.reply(
+            f"[{puppet.displayname}](https://matrix.to/#/{puppet.mxid})'s links to this chat:\n\n"
+            f"{formatted_links}"
+        )
+
+
@command_handler(
    help_section=SECTION_PORTAL_MANAGEMENT,
    help_text="Upgrade a normal Telegram group to a supergroup.",
@@ -37,6 +37,7 @@ from telethon.errors import (
 )
 from telethon.tl.types import User

+from mautrix.client import Client
 from mautrix.types import (
    EventID,
    ImageInfo,
@@ -230,9 +231,23 @@ async def login_qr(evt: CommandEvent) -> EventID:
 )
 async def login(evt: CommandEvent) -> EventID:
    override_sender = False
-    if len(evt.args) > 0 and evt.sender.is_admin:
-        evt.sender = await u.User.get_and_start_by_mxid(UserID(evt.args[0]))
+    if len(evt.args) > 0 and evt.sender.is_admin and evt.args[0]:
+        override_user_id = UserID(evt.args[0])
+        try:
+            Client.parse_user_id(override_user_id)
+        except ValueError:
+            return await evt.reply(
+                f"**Usage:** `$cmdprefix+sp login [override user ID]`\n\n"
+                f"{override_user_id!r} is not a valid Matrix user ID"
+            )
+        orig_user_id = evt.sender.mxid
+        evt.sender = await u.User.get_and_start_by_mxid(override_user_id)
        override_sender = True
+        if orig_user_id != evt.sender:
+            await evt.reply(
+                f"Admin override: logging in as {evt.sender.mxid} instead of {orig_user_id}"
+            )
+
    if await evt.sender.is_logged_in():
        return await evt.reply(f"You are already logged in as {evt.sender.human_tg_id}.")

@@ -26,6 +26,7 @@ from telethon.errors import (
    EmoticonInvalidError,
    InviteHashExpiredError,
    InviteHashInvalidError,
+    InviteRequestSentError,
    OptionsTooMuchError,
    TakeoutInitDelayError,
    UserAlreadyParticipantError,
@@ -171,6 +172,8 @@ async def _join(
            return (await evt.sender.client(ImportChatInviteRequest(identifier))), None
        except UserAlreadyParticipantError:
            return None, await evt.reply("You are already in that chat.")
+        except InviteRequestSentError:
+            return None, await evt.reply("Invite request sent successfully.")
    else:
        channel = await evt.sender.client.get_entity(identifier)
        if not channel:
@@ -208,6 +211,9 @@ async def join(evt: CommandEvent) -> EventID | None:
    link_type = data["type"]
    if link_type:
        link_type = link_type.lower()
+    elif identifier.startswith("+"):
+        link_type = "joinchat"
+        identifier = identifier[1:]
    updates, _ = await _join(evt, identifier, link_type)
    if not updates:
        return None
@@ -84,6 +84,10 @@ class Config(BaseBridgeConfig):

        copy("appservice.provisioning.enabled")
        copy("appservice.provisioning.prefix")
+        if base["appservice.provisioning.prefix"].endswith("/v1"):
+            base["appservice.provisioning.prefix"] = base["appservice.provisioning.prefix"][
+                : -len("/v1")
+            ]
        copy("appservice.provisioning.shared_secret")
        if base["appservice.provisioning.shared_secret"] == "generate":
            base["appservice.provisioning.shared_secret"] = self._new_token()
@@ -95,8 +99,6 @@ class Config(BaseBridgeConfig):
        if "pool_pre_ping" in base["appservice.database_opts"]:
            del base["appservice.database_opts.pool_pre_ping"]

-        copy("appservice.community_id")
-
        copy("metrics.enabled")
        copy("metrics.listen_port")

@@ -138,11 +140,13 @@ class Config(BaseBridgeConfig):
        copy("bridge.invite_link_resolve")
        copy("bridge.inline_images")
        copy("bridge.image_as_file_size")
-        copy("bridge.max_document_size")
+        copy("bridge.image_as_file_pixels")
        copy("bridge.parallel_file_transfer")
        copy("bridge.federate_rooms")
        copy("bridge.animated_sticker.target")
-        copy("bridge.animated_sticker.args")
+        copy("bridge.animated_sticker.args.width")
+        copy("bridge.animated_sticker.args.height")
+        copy("bridge.animated_sticker.args.fps")
        copy("bridge.encryption.allow")
        copy("bridge.encryption.default")
        copy("bridge.encryption.database")
@@ -159,6 +163,7 @@ class Config(BaseBridgeConfig):
        copy("bridge.tag_only_on_create")
        copy("bridge.bridge_matrix_leave")
        copy("bridge.kick_on_logout")
+        copy("bridge.always_read_joined_telegram_notice")
        copy("bridge.backfill.invite_own_puppet")
        copy("bridge.backfill.takeout_limit")
        copy("bridge.backfill.initial_limit")
@@ -180,6 +185,7 @@ class Config(BaseBridgeConfig):
            del self["bridge.message_formats"]
        copy_dict("bridge.message_formats", override_existing_map=False)
        copy("bridge.emote_format")
+        copy("bridge.relay_user_distinguishers")

        copy("bridge.state_event_formats.join")
        copy("bridge.state_event_formats.leave")
@@ -16,6 +16,7 @@
 from mautrix.util.async_db import Database

 from .bot_chat import BotChat
+from .disappearing_message import DisappearingMessage
 from .message import Message
 from .portal import Portal
 from .puppet import Puppet
@@ -27,7 +28,17 @@ from .user import User


 def init(db: Database) -> None:
-    for table in (Portal, Message, Reaction, User, Puppet, TelegramFile, BotChat, PgSession):
+    for table in (
+        Portal,
+        Message,
+        Reaction,
+        User,
+        Puppet,
+        TelegramFile,
+        BotChat,
+        PgSession,
+        DisappearingMessage,
+    ):
        table.db = db


@@ -42,4 +53,5 @@ __all__ = [
    "TelegramFile",
    "BotChat",
    "PgSession",
+    "DisappearingMessage",
 ]
@@ -0,0 +1,78 @@
+# mautrix-telegram - A Matrix-Telegram puppeting bridge
+# Copyright (C) 2021 Sumner Evans
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, ClassVar
+
+import asyncpg
+
+from mautrix.bridge import AbstractDisappearingMessage
+from mautrix.types import EventID, RoomID
+from mautrix.util.async_db import Database
+
+fake_db = Database.create("") if TYPE_CHECKING else None
+
+
+class DisappearingMessage(AbstractDisappearingMessage):
+    db: ClassVar[Database] = fake_db
+
+    async def insert(self) -> None:
+        q = """
+            INSERT INTO disappearing_message (room_id, event_id, expiration_seconds, expiration_ts)
+            VALUES ($1, $2, $3, $4)
+        """
+        await self.db.execute(
+            q, self.room_id, self.event_id, self.expiration_seconds, self.expiration_ts
+        )
+
+    async def update(self) -> None:
+        q = "UPDATE disappearing_message SET expiration_ts=$3 WHERE room_id=$1 AND event_id=$2"
+        await self.db.execute(q, self.room_id, self.event_id, self.expiration_ts)
+
+    async def delete(self) -> None:
+        q = "DELETE from disappearing_message WHERE room_id=$1 AND event_id=$2"
+        await self.db.execute(q, self.room_id, self.event_id)
+
+    @classmethod
+    def _from_row(cls, row: asyncpg.Record) -> DisappearingMessage:
+        return cls(**row)
+
+    @classmethod
+    async def get(cls, room_id: RoomID, event_id: EventID) -> DisappearingMessage | None:
+        q = """
+            SELECT room_id, event_id, expiration_seconds, expiration_ts FROM disappearing_message
+            WHERE room_id=$1 AND mxid=$2
+        """
+        try:
+            return cls._from_row(await cls.db.fetchrow(q, room_id, event_id))
+        except Exception:
+            return None
+
+    @classmethod
+    async def get_all_scheduled(cls) -> list[DisappearingMessage]:
+        q = """
+            SELECT room_id, event_id, expiration_seconds, expiration_ts FROM disappearing_message
+            WHERE expiration_ts IS NOT NULL
+        """
+        return [cls._from_row(r) for r in await cls.db.fetch(q)]
+
+    @classmethod
+    async def get_unscheduled_for_room(cls, room_id: RoomID) -> list[DisappearingMessage]:
+        q = """
+            SELECT room_id, event_id, expiration_seconds, expiration_ts FROM disappearing_message
+            WHERE room_id = $1 AND expiration_ts IS NULL
+        """
+        return [cls._from_row(r) for r in await cls.db.fetch(q, room_id)]
@@ -21,7 +21,7 @@ from asyncpg import Record
 from attr import dataclass

 from mautrix.types import EventID, RoomID
-from mautrix.util.async_db import Database
+from mautrix.util.async_db import Database, Scheme

 from ..types import TelegramID

@@ -76,7 +76,7 @@ class Message:
    async def get_first_by_tgids(
        cls, tgids: list[TelegramID], tg_space: TelegramID
    ) -> list[Message]:
-        if cls.db.scheme == "postgres":
+        if cls.db.scheme in (Scheme.POSTGRES, Scheme.COCKROACH):
            q = (
                f"SELECT {cls.columns} FROM message"
                " WHERE tgid=ANY($1) AND tg_space=$2 AND edit_index=0"
@@ -123,7 +123,7 @@ class Message:
    async def get_by_mxids(
        cls, mxids: list[EventID], mx_room: RoomID, tg_space: TelegramID
    ) -> list[Message]:
-        if cls.db.scheme == "postgres":
+        if cls.db.scheme in (Scheme.POSTGRES, Scheme.COCKROACH):
            q = (
                f"SELECT {cls.columns} FROM message"
                " WHERE mxid=ANY($1) AND mx_room=$2 AND tg_space=$3"
@@ -54,6 +54,8 @@ class Portal:
    title: str | None
    about: str | None
    photo_id: str | None
+    name_set: bool
+    avatar_set: bool

    local_config: dict[str, Any] = attr.ib(factory=lambda: {})

@@ -67,7 +69,8 @@ class Portal:

    columns: ClassVar[str] = (
        "tgid, tg_receiver, peer_type, megagroup, mxid, avatar_url, encrypted, sponsored_event_id,"
-        "sponsored_event_ts, sponsored_msg_random_id, username, title, about, photo_id, config"
+        "sponsored_event_ts, sponsored_msg_random_id, username, title, about, photo_id, "
+        "name_set, avatar_set, config"
    )

    @classmethod
@@ -86,10 +89,15 @@ class Portal:
        return cls._from_row(await cls.db.fetchrow(q, username.lower()))

    @classmethod
-    async def find_private_chats(cls, tg_receiver: TelegramID) -> list[Portal]:
+    async def find_private_chats_of(cls, tg_receiver: TelegramID) -> list[Portal]:
        q = f"SELECT {cls.columns} FROM portal WHERE tg_receiver=$1 AND peer_type='user'"
        return [cls._from_row(row) for row in await cls.db.fetch(q, tg_receiver)]

+    @classmethod
+    async def find_private_chats_with(cls, tgid: TelegramID) -> list[Portal]:
+        q = f"SELECT {cls.columns} FROM portal WHERE tgid=$1 AND peer_type='user'"
+        return [cls._from_row(row) for row in await cls.db.fetch(q, tgid)]
+
    @classmethod
    async def all(cls) -> list[Portal]:
        rows = await cls.db.fetch(f"SELECT {cls.columns} FROM portal")
@@ -111,17 +119,20 @@ class Portal:
            self.title,
            self.about,
            self.photo_id,
+            self.name_set,
+            self.avatar_set,
            self.megagroup,
            json.dumps(self.local_config) if self.local_config else None,
        )

    async def save(self) -> None:
-        q = (
-            "UPDATE portal SET mxid=$4, avatar_url=$5, encrypted=$6, sponsored_event_id=$7,"
-            "                  sponsored_event_ts=$8, sponsored_msg_random_id=$9, username=$10,"
-            "                  title=$11, about=$12, photo_id=$13, megagroup=$14, config=$15 "
-            "WHERE tgid=$1 AND tg_receiver=$2 AND (peer_type=$3 OR true)"
-        )
+        q = """
+        UPDATE portal
+        SET mxid=$4, avatar_url=$5, encrypted=$6, sponsored_event_id=$7, sponsored_event_ts=$8,
+            sponsored_msg_random_id=$9, username=$10, title=$11, about=$12, photo_id=$13,
+            name_set=$14, avatar_set=$15, megagroup=$16, config=$17
+        WHERE tgid=$1 AND tg_receiver=$2 AND (peer_type=$3 OR true)
+        """
        await self.db.execute(q, *self._values)

    async def update_id(self, id: TelegramID, peer_type: str) -> None:
@@ -135,12 +146,13 @@ class Portal:
        self.peer_type = peer_type

    async def insert(self) -> None:
-        q = (
-            "INSERT INTO portal (tgid, tg_receiver, peer_type, mxid, avatar_url, encrypted,"
-            "                    sponsored_event_id, sponsored_event_ts, sponsored_msg_random_id,"
-            "                    username, title, about, photo_id, megagroup, config) "
-            "VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15)"
-        )
+        q = """
+        INSERT INTO portal (
+            tgid, tg_receiver, peer_type, mxid, avatar_url, encrypted,
+            sponsored_event_id, sponsored_event_ts, sponsored_msg_random_id,
+            username, title, about, photo_id, name_set, avatar_set, megagroup, config
+        ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17)
+        """
        await self.db.execute(q, *self._values)

    async def delete(self) -> None:
@@ -1,5 +1,5 @@
 # mautrix-telegram - A Matrix-Telegram puppeting bridge
-# Copyright (C) 2021 Tulir Asokan
+# Copyright (C) 2022 Tulir Asokan
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as published by
@@ -21,7 +21,7 @@ from asyncpg import Record
 from attr import dataclass
 from yarl import URL

-from mautrix.types import SyncToken, UserID
+from mautrix.types import ContentURI, SyncToken, UserID
 from mautrix.util.async_db import Database

 from ..types import TelegramID
@@ -43,8 +43,13 @@ class Puppet:
    displayname_quality: int
    disable_updates: bool
    username: str | None
+    phone: str | None
    photo_id: str | None
+    avatar_url: ContentURI | None
+    name_set: bool
+    avatar_set: bool
    is_bot: bool | None
+    is_channel: bool

    custom_mxid: UserID | None
    access_token: str | None
@@ -61,8 +66,8 @@ class Puppet:

    columns: ClassVar[str] = (
        "id, is_registered, displayname, displayname_source, displayname_contact, "
-        "displayname_quality, disable_updates, username, photo_id, is_bot, "
-        "custom_mxid, access_token, next_batch, base_url"
+        "displayname_quality, disable_updates, username, phone, photo_id, avatar_url, "
+        "name_set, avatar_set, is_bot, is_channel, custom_mxid, access_token, next_batch, base_url"
    )

    @classmethod
@@ -101,8 +106,13 @@ class Puppet:
            self.displayname_quality,
            self.disable_updates,
            self.username,
+            self.phone,
            self.photo_id,
+            self.avatar_url,
+            self.name_set,
+            self.avatar_set,
            self.is_bot,
+            self.is_channel,
            self.custom_mxid,
            self.access_token,
            self.next_batch,
@@ -110,21 +120,23 @@ class Puppet:
        )

    async def save(self) -> None:
-        q = (
-            "UPDATE puppet "
-            "SET is_registered=$2, displayname=$3, displayname_source=$4, displayname_contact=$5,"
-            "    displayname_quality=$6, disable_updates=$7, username=$8, photo_id=$9, is_bot=$10,"
-            "    custom_mxid=$11, access_token=$12, next_batch=$13, base_url=$14 "
-            "WHERE id=$1"
-        )
+        q = """
+        UPDATE puppet
+        SET is_registered=$2, displayname=$3, displayname_source=$4, displayname_contact=$5,
+            displayname_quality=$6, disable_updates=$7, username=$8, phone=$9, photo_id=$10,
+            avatar_url=$11, name_set=$12, avatar_set=$13, is_bot=$14, is_channel=$15,
+            custom_mxid=$16, access_token=$17, next_batch=$18, base_url=$19
+        WHERE id=$1
+        """
        await self.db.execute(q, *self._values)

    async def insert(self) -> None:
-        q = (
-            "INSERT INTO puppet ("
-            "    id, is_registered, displayname, displayname_source, displayname_contact,"
-            "    displayname_quality, disable_updates, username, photo_id, is_bot,"
-            "    custom_mxid, access_token, next_batch, base_url"
-            ") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)"
-        )
+        q = """
+        INSERT INTO puppet (
+            id, is_registered, displayname, displayname_source, displayname_contact,
+            displayname_quality, disable_updates, username, phone, photo_id, avatar_url, name_set,
+            avatar_set, is_bot, is_channel, custom_mxid, access_token, next_batch, base_url
+        ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,
+                  $19)
+        """
        await self.db.execute(q, *self._values)
@@ -24,7 +24,7 @@ from telethon.crypto import AuthKey
 from telethon.sessions import MemorySession
 from telethon.tl.types import PeerChannel, PeerChat, PeerUser, updates

-from mautrix.util.async_db import Database
+from mautrix.util.async_db import Database, Scheme

 fake_db = Database.create("") if TYPE_CHECKING else None

@@ -153,7 +153,7 @@ class PgSession(MemorySession):
        ] = self._entities_to_rows(tlo)
        if not rows:
            return
-        if self.db.scheme == "postgres":
+        if self.db.scheme == Scheme.POSTGRES:
            q = (
                "INSERT INTO telethon_entities (session_id, id, hash, username, phone, name) "
                "VALUES ($1, unnest($2::bigint[]), unnest($3::bigint[]), "
@@ -201,7 +201,7 @@ class PgSession(MemorySession):
            utils.get_peer_id(PeerChat(key)),
            utils.get_peer_id(PeerChannel(key)),
        )
-        if self.db.scheme == "postgres":
+        if self.db.scheme in (Scheme.POSTGRES, Scheme.COCKROACH):
            return await self._select_entity("id=ANY($1)", ids)
        else:
            return await self._select_entity(f"id IN ($1, $2, $3)", *ids)
@@ -2,4 +2,12 @@ from mautrix.util.async_db import UpgradeTable

 upgrade_table = UpgradeTable()

-from . import v01_initial_revision, v02_sponsored_events, v03_reactions
+from . import (
+    v01_initial_revision,
+    v02_sponsored_events,
+    v03_reactions,
+    v04_disappearing_messages,
+    v05_channel_ghosts,
+    v06_puppet_avatar_url,
+    v07_puppet_phone_number,
+)
@@ -0,0 +1,200 @@
+# mautrix-telegram - A Matrix-Telegram puppeting bridge
+# Copyright (C) 2022 Tulir Asokan
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from mautrix.util.async_db import Connection
+
+
+async def create_v7_tables(conn: Connection) -> int:
+    await conn.execute(
+        """CREATE TABLE "user" (
+            mxid TEXT   PRIMARY KEY,
+            tgid BIGINT UNIQUE,
+            tg_username    TEXT,
+            tg_phone       TEXT,
+            is_bot         BOOLEAN NOT NULL DEFAULT false,
+            saved_contacts INTEGER NOT NULL DEFAULT 0
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE portal (
+            tgid        BIGINT,
+            tg_receiver BIGINT,
+            peer_type   TEXT NOT NULL,
+            mxid        TEXT UNIQUE,
+            avatar_url  TEXT,
+            encrypted   BOOLEAN NOT NULL DEFAULT false,
+            username    TEXT,
+            title       TEXT,
+            about       TEXT,
+            photo_id    TEXT,
+            name_set    BOOLEAN NOT NULL DEFAULT false,
+            avatar_set  BOOLEAN NOT NULL DEFAULT false,
+            megagroup   BOOLEAN,
+            config      jsonb,
+
+            sponsored_event_id     TEXT,
+            sponsored_event_ts     BIGINT,
+            sponsored_msg_random_id bytea,
+
+            PRIMARY KEY (tgid, tg_receiver)
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE message (
+            mxid         TEXT   NOT NULL,
+            mx_room      TEXT   NOT NULL,
+            tgid         BIGINT,
+            tg_space     BIGINT,
+            edit_index   INTEGER,
+            redacted     BOOLEAN NOT NULL DEFAULT false,
+            content_hash bytea,
+            PRIMARY KEY (tgid, tg_space, edit_index),
+            UNIQUE (mxid, mx_room, tg_space)
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE reaction (
+            mxid      TEXT NOT NULL,
+            mx_room   TEXT NOT NULL,
+            msg_mxid  TEXT NOT NULL,
+            tg_sender BIGINT,
+            reaction  TEXT NOT NULL,
+
+            PRIMARY KEY (msg_mxid, mx_room, tg_sender),
+            UNIQUE (mxid, mx_room)
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE disappearing_message (
+            room_id             TEXT,
+            event_id            TEXT,
+            expiration_seconds  BIGINT,
+            expiration_ts       BIGINT,
+
+            PRIMARY KEY (room_id, event_id)
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE puppet (
+            id BIGINT PRIMARY KEY,
+
+            is_registered BOOLEAN NOT NULL DEFAULT false,
+
+            displayname         TEXT,
+            displayname_source  BIGINT,
+            displayname_contact BOOLEAN NOT NULL DEFAULT true,
+            displayname_quality INTEGER NOT NULL DEFAULT 0,
+            disable_updates     BOOLEAN NOT NULL DEFAULT false,
+            username            TEXT,
+            phone               TEXT,
+            photo_id            TEXT,
+            avatar_url          TEXT,
+            name_set            BOOLEAN NOT NULL DEFAULT false,
+            avatar_set          BOOLEAN NOT NULL DEFAULT false,
+            is_bot              BOOLEAN,
+            is_channel          BOOLEAN NOT NULL DEFAULT false,
+
+            access_token TEXT,
+            custom_mxid  TEXT,
+            next_batch   TEXT,
+            base_url     TEXT
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE telegram_file (
+            id              TEXT PRIMARY KEY,
+            mxc             TEXT NOT NULL,
+            mime_type       TEXT,
+            was_converted   BOOLEAN NOT NULL DEFAULT false,
+            timestamp       BIGINT  NOT NULL DEFAULT 0,
+            size            BIGINT,
+            width           INTEGER,
+            height          INTEGER,
+            thumbnail       TEXT,
+            decryption_info jsonb,
+            FOREIGN KEY (thumbnail) REFERENCES telegram_file(id)
+                ON UPDATE CASCADE ON DELETE SET NULL
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE bot_chat (
+            id   BIGINT PRIMARY KEY,
+            type TEXT NOT NULL
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE user_portal (
+            "user"          BIGINT,
+            portal          BIGINT,
+            portal_receiver BIGINT,
+            PRIMARY KEY ("user", portal, portal_receiver),
+            FOREIGN KEY ("user") REFERENCES "user"(tgid) ON DELETE CASCADE ON UPDATE CASCADE,
+            FOREIGN KEY (portal, portal_receiver) REFERENCES portal(tgid, tg_receiver)
+                 ON DELETE CASCADE ON UPDATE CASCADE
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE contact (
+            "user"  BIGINT,
+            contact BIGINT,
+            PRIMARY KEY ("user", contact),
+            FOREIGN KEY ("user")  REFERENCES "user"(tgid) ON DELETE CASCADE ON UPDATE CASCADE,
+            FOREIGN KEY (contact) REFERENCES puppet(id)   ON DELETE CASCADE ON UPDATE CASCADE
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE telethon_sessions (
+            session_id     TEXT PRIMARY KEY,
+            dc_id          INTEGER,
+            server_address TEXT,
+            port           INTEGER,
+            auth_key       bytea
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE telethon_entities (
+            session_id TEXT,
+            id         BIGINT,
+            hash       BIGINT NOT NULL,
+            username   TEXT,
+            phone      TEXT,
+            name       TEXT,
+            PRIMARY KEY (session_id, id)
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE telethon_sent_files (
+            session_id TEXT,
+            md5_digest bytea,
+            file_size  INTEGER,
+            type       INTEGER,
+            id         BIGINT,
+            hash       BIGINT,
+            PRIMARY KEY (session_id, md5_digest, file_size, type)
+        )"""
+    )
+    await conn.execute(
+        """CREATE TABLE telethon_update_state (
+            session_id   TEXT,
+            entity_id    BIGINT,
+            pts          BIGINT,
+            qts          BIGINT,
+            date         BIGINT,
+            seq          BIGINT,
+            unread_count INTEGER,
+            PRIMARY KEY (session_id, entity_id)
+        )"""
+    )
+    return 7
@@ -15,29 +15,38 @@
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 from __future__ import annotations

-from asyncpg import Connection
+from mautrix.util.async_db import Connection, Scheme

 from . import upgrade_table
+from .v00_latest_revision import create_v7_tables

 legacy_version_query = "SELECT version_num FROM alembic_version"
 last_legacy_version = "bfc0a39bfe02"


 def table_exists(scheme: str, name: str) -> str:
-    if scheme == "sqlite":
+    if scheme == Scheme.SQLITE:
        return f"SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE type='table' AND name='{name}')"
-    elif scheme == "postgres":
-        return f"SELECT EXISTS(SELECT FROM information_schema.tables WHERE table_name='{name}')"
+    elif scheme in (Scheme.POSTGRES, Scheme.COCKROACH):
+        return f"SELECT EXISTS(SELECT 1 FROM information_schema.tables WHERE table_name='{name}')"
    raise RuntimeError("unsupported database scheme")


-@upgrade_table.register(description="Initial asyncpg revision")
-async def upgrade_v1(conn: Connection, scheme: str) -> None:
+async def first_upgrade_target(conn: Connection, scheme: str) -> int:
+    is_legacy = await conn.fetchval(table_exists(scheme, "alembic_version"))
+    # If it's a legacy db, the upgrade process will go to v1 and run each migration up to v7.
+    # If it's a new db, we'll create the v7 tables directly (see the create_v7_tables call).
+    return 1 if is_legacy else 7
+
+
+@upgrade_table.register(description="Initial asyncpg revision", upgrades_to=first_upgrade_target)
+async def upgrade_v1(conn: Connection, scheme: str) -> int:
    is_legacy = await conn.fetchval(table_exists(scheme, "alembic_version"))
    if is_legacy:
        await migrate_legacy_to_v1(conn, scheme)
+        return 1
    else:
-        await create_v1_tables(conn)
+        return await create_v7_tables(conn)


 async def drop_constraints(conn: Connection, table: str, contype: str) -> None:
@@ -178,151 +187,3 @@ async def varchar_to_text(conn: Connection) -> None:
    for table, columns in columns_to_adjust.items():
        for column in columns:
            await conn.execute(f'ALTER TABLE "{table}" ALTER COLUMN {column} TYPE TEXT')
-
-
-async def create_v1_tables(conn: Connection) -> None:
-    await conn.execute(
-        """CREATE TABLE "user" (
-            mxid TEXT   PRIMARY KEY,
-            tgid BIGINT UNIQUE,
-            tg_username    TEXT,
-            tg_phone       TEXT,
-            is_bot         BOOLEAN NOT NULL DEFAULT false,
-            saved_contacts INTEGER NOT NULL DEFAULT 0
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE portal (
-            tgid        BIGINT,
-            tg_receiver BIGINT,
-            peer_type   TEXT NOT NULL,
-            mxid        TEXT UNIQUE,
-            avatar_url  TEXT,
-            encrypted   BOOLEAN NOT NULL DEFAULT false,
-            username    TEXT,
-            title       TEXT,
-            about       TEXT,
-            photo_id    TEXT,
-            megagroup   BOOLEAN,
-            config      jsonb,
-            PRIMARY KEY (tgid, tg_receiver)
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE message (
-            mxid       TEXT,
-            mx_room    TEXT,
-            tgid       BIGINT NOT NULL,
-            tg_space   BIGINT NOT NULL,
-            edit_index INTEGER NOT NULL,
-            redacted   BOOLEAN NOT NULL DEFAULT false,
-            PRIMARY KEY (tgid, tg_space, edit_index),
-            UNIQUE (mxid, mx_room, tg_space)
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE puppet (
-            id BIGINT PRIMARY KEY,
-
-            is_registered BOOLEAN NOT NULL DEFAULT false,
-
-            displayname         TEXT,
-            displayname_source  BIGINT,
-            displayname_contact BOOLEAN NOT NULL DEFAULT true,
-            displayname_quality INTEGER NOT NULL DEFAULT 0,
-            disable_updates     BOOLEAN NOT NULL DEFAULT false,
-            username            TEXT,
-            photo_id            TEXT,
-            is_bot              BOOLEAN,
-
-            access_token TEXT,
-            custom_mxid  TEXT,
-            next_batch   TEXT,
-            base_url     TEXT
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE telegram_file (
-            id              TEXT PRIMARY KEY,
-            mxc             TEXT NOT NULL,
-            mime_type       TEXT,
-            was_converted   BOOLEAN NOT NULL DEFAULT false,
-            timestamp       BIGINT  NOT NULL DEFAULT 0,
-            size            BIGINT,
-            width           INTEGER,
-            height          INTEGER,
-            thumbnail       TEXT,
-            decryption_info jsonb,
-            FOREIGN KEY (thumbnail) REFERENCES telegram_file(id)
-                ON UPDATE CASCADE ON DELETE SET NULL
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE bot_chat (
-            id   BIGINT PRIMARY KEY,
-            type TEXT NOT NULL
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE user_portal (
-            "user"          BIGINT,
-            portal          BIGINT,
-            portal_receiver BIGINT,
-            PRIMARY KEY ("user", portal, portal_receiver),
-            FOREIGN KEY ("user") REFERENCES "user"(tgid) ON DELETE CASCADE ON UPDATE CASCADE,
-            FOREIGN KEY (portal, portal_receiver) REFERENCES portal(tgid, tg_receiver)
-                 ON DELETE CASCADE ON UPDATE CASCADE
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE contact (
-            "user"  BIGINT,
-            contact BIGINT,
-            PRIMARY KEY ("user", contact),
-            FOREIGN KEY ("user")  REFERENCES "user"(tgid) ON DELETE CASCADE ON UPDATE CASCADE,
-            FOREIGN KEY (contact) REFERENCES puppet(id)   ON DELETE CASCADE ON UPDATE CASCADE
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE telethon_sessions (
-            session_id     TEXT PRIMARY KEY,
-            dc_id          INTEGER,
-            server_address TEXT,
-            port           INTEGER,
-            auth_key       bytea
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE telethon_entities (
-            session_id TEXT,
-            id         BIGINT,
-            hash       BIGINT NOT NULL,
-            username   TEXT,
-            phone      TEXT,
-            name       TEXT,
-            PRIMARY KEY (session_id, id)
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE telethon_sent_files (
-            session_id TEXT,
-            md5_digest bytea,
-            file_size  INTEGER,
-            type       INTEGER,
-            id         BIGINT,
-            hash       BIGINT,
-            PRIMARY KEY (session_id, md5_digest, file_size, type)
-        )"""
-    )
-    await conn.execute(
-        """CREATE TABLE telethon_update_state (
-            session_id   TEXT,
-            entity_id    BIGINT,
-            pts          BIGINT,
-            qts          BIGINT,
-            date         BIGINT,
-            seq          BIGINT,
-            unread_count INTEGER,
-            PRIMARY KEY (session_id, entity_id)
-        )"""
-    )
@@ -13,7 +13,7 @@
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
-from asyncpg import Connection
+from mautrix.util.async_db import Connection

 from . import upgrade_table

@@ -13,7 +13,7 @@
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
-from asyncpg import Connection
+from mautrix.util.async_db import Connection

 from . import upgrade_table

@@ -0,0 +1,32 @@
+# mautrix-telegram - A Matrix-Telegram puppeting bridge
+# Copyright (C) 2022 Tulir Asokan
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from mautrix.util.async_db import Connection
+
+from . import upgrade_table
+
+
+@upgrade_table.register(description="Add support for disappearing messages")
+async def upgrade_v4(conn: Connection) -> None:
+    await conn.execute(
+        """CREATE TABLE disappearing_message (
+            room_id             TEXT,
+            event_id            TEXT,
+            expiration_seconds  BIGINT,
+            expiration_ts       BIGINT,
+
+            PRIMARY KEY (room_id, event_id)
+        )"""
+    )
@@ -0,0 +1,25 @@
+# mautrix-telegram - A Matrix-Telegram puppeting bridge
+# Copyright (C) 2022 Tulir Asokan
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from mautrix.util.async_db import Connection, Scheme
+
+from . import upgrade_table
+
+
+@upgrade_table.register(description="Add separate ghost users for channel senders")
+async def upgrade_v5(conn: Connection, scheme: str) -> None:
+    await conn.execute("ALTER TABLE puppet ADD COLUMN is_channel BOOLEAN NOT NULL DEFAULT false")
+    if scheme == Scheme.POSTGRES:
+        await conn.execute("ALTER TABLE puppet ALTER COLUMN is_channel DROP DEFAULT")
@@ -0,0 +1,31 @@
+# mautrix-telegram - A Matrix-Telegram puppeting bridge
+# Copyright (C) 2022 Tulir Asokan
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from mautrix.util.async_db import Connection
+
+from . import upgrade_table
+
+
+@upgrade_table.register(description="Store avatar mxc URI in puppet table")
+async def upgrade_v6(conn: Connection) -> None:
+    await conn.execute("ALTER TABLE puppet ADD COLUMN avatar_url TEXT")
+    await conn.execute("ALTER TABLE puppet ADD COLUMN name_set BOOLEAN NOT NULL DEFAULT false")
+    await conn.execute("ALTER TABLE puppet ADD COLUMN avatar_set BOOLEAN NOT NULL DEFAULT false")
+    await conn.execute("UPDATE puppet SET name_set=true WHERE displayname<>''")
+    await conn.execute("UPDATE puppet SET avatar_set=true WHERE photo_id<>''")
+    await conn.execute("ALTER TABLE portal ADD COLUMN name_set BOOLEAN NOT NULL DEFAULT false")
+    await conn.execute("ALTER TABLE portal ADD COLUMN avatar_set BOOLEAN NOT NULL DEFAULT false")
+    await conn.execute("UPDATE portal SET name_set=true WHERE title<>'' AND mxid<>''")
+    await conn.execute("UPDATE portal SET avatar_set=true WHERE photo_id<>'' AND mxid<>''")
@@ -0,0 +1,23 @@
+# mautrix-telegram - A Matrix-Telegram puppeting bridge
+# Copyright (C) 2022 Tulir Asokan
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+from mautrix.util.async_db import Connection
+
+from . import upgrade_table
+
+
+@upgrade_table.register(description="Store phone number in puppet table")
+async def upgrade_v7(conn: Connection) -> None:
+    await conn.execute("ALTER TABLE puppet ADD COLUMN phone TEXT")
@@ -21,7 +21,7 @@ from asyncpg import Record
 from attr import dataclass

 from mautrix.types import UserID
-from mautrix.util.async_db import Database
+from mautrix.util.async_db import Database, Scheme

 from ..types import TelegramID

@@ -104,7 +104,7 @@ class User:
        records = [(self.tgid, puppet_id) for puppet_id in puppets]
        async with self.db.acquire() as conn, conn.transaction():
            await conn.execute('DELETE FROM contact WHERE "user"=$1', self.tgid)
-            if self.db.scheme == "postgres":
+            if self.db.scheme == Scheme.POSTGRES:
                await conn.copy_records_to_table("contact", records=records, columns=columns)
            else:
                q = 'INSERT INTO contact ("user", contact) VALUES ($1, $2)'
@@ -120,7 +120,7 @@ class User:
        records = [(self.tgid, tgid, tg_receiver) for tgid, tg_receiver in portals]
        async with self.db.acquire() as conn, conn.transaction():
            await conn.execute('DELETE FROM user_portal WHERE "user"=$1', self.tgid)
-            if self.db.scheme == "postgres":
+            if self.db.scheme == Scheme.POSTGRES:
                await conn.copy_records_to_table("user_portal", records=records, columns=columns)
            else:
                q = 'INSERT INTO user_portal ("user", portal, portal_receiver) VALUES ($1, $2, $3)'
@@ -16,6 +16,9 @@ homeserver:
    status_endpoint: null
    # Endpoint for reporting per-message status.
    message_send_checkpoint_endpoint: null
+    # Whether asynchronous uploads via MSC2246 should be enabled for media.
+    # Requires a media repo that supports MSC2246.
+    async_media: false

 # Application service host/registration related details
 # Changing these values requires regeneration of the registration.
@@ -64,7 +67,7 @@ appservice:
        # Whether or not the provisioning API should be enabled.
        enabled: true
        # The prefix to use in the provisioning API endpoints.
-        prefix: /_matrix/provision/v1
+        prefix: /_matrix/provision
        # The shared secret to authorize users of the API.
        # Set to "generate" to generate and save a new token.
        shared_secret: generate
@@ -78,12 +81,6 @@ appservice:
    bot_displayname: Telegram bridge bot
    bot_avatar: mxc://maunium.net/tJCRmUyJDsgRNgqhOgoiHWbX

-    # Community ID for bridged users (changes registration file) and rooms.
-    # Must be created manually.
-    #
-    # Example: "+telegram:example.com". Set to false to disable.
-    community_id: false
-
    # Whether or not to receive ephemeral events via appservice transactions.
    # Requires MSC2409 support (i.e. Synapse 1.22+).
    # You should disable bridge -> sync_with_custom_puppets when this is enabled.
@@ -214,8 +211,8 @@ bridge:
    inline_images: false
    # Maximum size of image in megabytes before sending to Telegram as a document.
    image_as_file_size: 10
-    # Maximum size of Telegram documents in megabytes to bridge.
-    max_document_size: 100
+    # Maximum number of pixels in an image before sending to Telegram as a document. Defaults to 1280x1280 = 1638400.
+    image_as_file_pixels: 1638400
    # Enable experimental parallel file transfer, which makes uploads/downloads much faster by
    # streaming from/to Matrix and using many connections for Telegram.
    # Note that generating HQ thumbnails for videos is not possible with streamed transfers.
@@ -285,6 +282,8 @@ bridge:
    bridge_matrix_leave: true
    # Should the user be kicked out of all portals when logging out of the bridge?
    kick_on_logout: true
+    # Should the "* user joined Telegram" notice always be marked as read automatically?
+    always_read_joined_telegram_notice: true
    # Settings for backfilling messages from Telegram.
    backfill:
        # Whether or not the Telegram ghosts of logged in Matrix users should be
@@ -328,9 +327,12 @@ bridge:
        # List of user IDs for whom the previous flag is flipped.
        # e.g. if bridge_notices.default is false, notices from other users will not be bridged, but
        #      notices from users listed here will be bridged.
-        exceptions:
-        - "@importantbot:example.com"
+        exceptions: []

+    # An array of possible values for the $distinguisher variable in message formats.
+    # Each user gets one of the values here, based on a hash of their user ID.
+    # If the array is empty, the $distinguisher variable will also be empty.
+    relay_user_distinguishers: ["🟦", "🟣", "🟩", "⭕️", "🔶", "⬛️", "🔵", "🟢"]
    # The formats to use when sending messages to Telegram via the relay bot.
    # Text msgtypes (m.text, m.notice and m.emote) support HTML, media msgtypes don't.
    #
@@ -338,16 +340,17 @@ bridge:
    #   $sender_displayname - The display name of the sender (e.g. Example User)
    #   $sender_username    - The username (Matrix ID localpart) of the sender (e.g. exampleuser)
    #   $sender_mxid        - The Matrix ID of the sender (e.g. @exampleuser:example.com)
+    #   $distinguisher      - A random string from the options in the relay_user_distinguishers array.
    #   $message            - The message content
    message_formats:
-        m.text: "<b>$sender_displayname</b>: $message"
-        m.notice: "<b>$sender_displayname</b>: $message"
-        m.emote: "* <b>$sender_displayname</b> $message"
-        m.file: "<b>$sender_displayname</b> sent a file: $message"
-        m.image: "<b>$sender_displayname</b> sent an image: $message"
-        m.audio: "<b>$sender_displayname</b> sent an audio file: $message"
-        m.video: "<b>$sender_displayname</b> sent a video: $message"
-        m.location: "<b>$sender_displayname</b> sent a location: $message"
+        m.text: "$distinguisher <b>$sender_displayname</b>: $message"
+        m.notice: "$distinguisher <b>$sender_displayname</b>: $message"
+        m.emote: "* $distinguisher <b>$sender_displayname</b> $message"
+        m.file: "$distinguisher <b>$sender_displayname</b> sent a file: $message"
+        m.image: "$distinguisher <b>$sender_displayname</b> sent an image: $message"
+        m.audio: "$distinguisher <b>$sender_displayname</b> sent an audio file: $message"
+        m.video: "$distinguisher <b>$sender_displayname</b> sent a video: $message"
+        m.location: "$distinguisher <b>$sender_displayname</b> sent a location: $message"
    # Telegram doesn't have built-in emotes, this field specifies how m.emote's from authenticated
    # users are sent to telegram. All fields in message_formats are supported. Additionally, the
    # Telegram user info is available in the following variables:
@@ -363,9 +366,9 @@ bridge:
    #
    # Set format to an empty string to disable the messages for that event.
    state_event_formats:
-        join: "<b>$displayname</b> joined the room."
-        leave: "<b>$displayname</b> left the room."
-        name_change: "<b>$prev_displayname</b> changed their name to <b>$displayname</b>"
+        join: "$distinguisher <b>$displayname</b> joined the room."
+        leave: "$distinguisher <b>$displayname</b> left the room."
+        name_change: "$distinguisher <b>$prev_displayname</b> changed their name to $distinguisher <b>$displayname</b>"

    # Filter rooms that can/can't be bridged. Can also be managed using the `filter` and
    # `filter-mode` management commands.
@@ -20,8 +20,7 @@ import logging
 from telethon import TelegramClient

 from mautrix.types import RoomID, UserID
-from mautrix.util.formatter import MatrixParser as BaseMatrixParser, RecursionContext
-from mautrix.util.formatter.html_reader_htmlparser import HTMLNode, read_html
+from mautrix.util.formatter import HTMLNode, MatrixParser as BaseMatrixParser, RecursionContext
 from mautrix.util.logging import TraceLogger

 from ... import portal as po, puppet as pu, user as u
@@ -33,7 +32,6 @@ log: TraceLogger = logging.getLogger("mau.fmt.mx")
 class MatrixParser(BaseMatrixParser[TelegramMessage]):
    e = TelegramEntityType
    fs = TelegramMessage
-    read_html = staticmethod(read_html)
    client: TelegramClient

    def __init__(self, client: TelegramClient) -> None:
@@ -20,7 +20,7 @@ import logging
 import re

 from telethon.errors import RPCError
-from telethon.helpers import add_surrogate, del_surrogate
+from telethon.helpers import add_surrogate, del_surrogate, within_surrogate
 from telethon.tl.custom import Message
 from telethon.tl.types import (
    MessageEntityBlockquote,
@@ -52,9 +52,9 @@ from mautrix.appservice import IntentAPI
 from mautrix.types import (
    EventType,
    Format,
+    InReplyTo,
    MessageType,
    RelatesTo,
-    RelationType,
    TextMessageEventContent,
 )

@@ -74,16 +74,13 @@ async def telegram_reply_to_matrix(evt: Message, source: au.AbstractUser) -> Rel
        )
        msg = await DBMessage.get_one_by_tgid(TelegramID(evt.reply_to.reply_to_msg_id), space)
        if msg:
-            return RelatesTo(rel_type=RelationType.REPLY, event_id=msg.mxid)
+            return RelatesTo(in_reply_to=InReplyTo(event_id=msg.mxid))
    return None


 async def _add_forward_header(
    source: au.AbstractUser, content: TextMessageEventContent, fwd_from: MessageFwdHeader
 ) -> None:
-    if not content.formatted_body or content.format != Format.HTML:
-        content.format = Format.HTML
-        content.formatted_body = escape(content.body)
    fwd_from_html, fwd_from_text = None, None
    if isinstance(fwd_from.from_id, PeerUser):
        user = await u.User.get_by_tgid(TelegramID(fwd_from.from_id.user_id))
@@ -94,9 +91,7 @@ async def _add_forward_header(
            )

        if not fwd_from_text:
-            puppet = await pu.Puppet.get_by_tgid(
-                TelegramID(fwd_from.from_id.user_id), create=False
-            )
+            puppet = await pu.Puppet.get_by_peer(fwd_from.from_id, create=False)
            if puppet and puppet.displayname:
                fwd_from_text = puppet.displayname or puppet.mxid
                fwd_from_html = (
@@ -141,6 +136,7 @@ async def _add_forward_header(
        fwd_from_text = "unknown source"
        fwd_from_html = f"unknown source"

+    content.ensure_has_html()
    content.body = "\n".join([f"> {line}" for line in content.body.split("\n")])
    content.body = f"Forwarded from {fwd_from_text}:\n{content.body}"
    content.formatted_body = (
@@ -162,8 +158,6 @@ async def _add_reply_header(
    if not msg:
        return

-    content.relates_to = RelatesTo(rel_type=RelationType.REPLY, event_id=msg.mxid)
-
    try:
        event = await main_intent.get_event(msg.mx_room, msg.mxid)
        if event.type == EventType.ROOM_ENCRYPTED and source.bridge.matrix.e2ee:
@@ -174,6 +168,7 @@ async def _add_reply_header(
        content.set_reply(event, displayname=puppet.displayname if puppet else event.sender)
    except Exception:
        log.exception("Failed to get event to add reply fallback")
+        content.set_reply(msg.mxid)


 async def telegram_to_matrix(
@@ -195,18 +190,13 @@ async def telegram_to_matrix(
    if entities:
        content.format = Format.HTML
        html = await _telegram_entities_to_matrix_catch(add_surrogate(content.body), entities)
-        content.formatted_body = del_surrogate(html).replace("\n", "<br/>")
-
-    def force_html():
-        if not content.formatted_body:
-            content.format = Format.HTML
-            content.formatted_body = escape(content.body)
+        content.formatted_body = del_surrogate(html)

    if require_html:
-        force_html()
+        content.ensure_has_html()

    if prefix_html:
-        force_html()
+        content.ensure_has_html()
        content.formatted_body = prefix_html + content.formatted_body
    if prefix_text:
        content.body = prefix_text + content.body
@@ -218,7 +208,7 @@ async def telegram_to_matrix(
        await _add_reply_header(source, content, evt, main_intent)

    if isinstance(evt, Message) and evt.post and evt.post_author:
-        force_html()
+        content.ensure_has_html()
        content.body += f"\n- {evt.post_author}"
        content.formatted_body += f"<br/><i>- <u>{evt.post_author}</u></i>"

@@ -236,30 +226,51 @@ async def _telegram_entities_to_matrix_catch(text: str, entities: list[TypeMessa


 async def _telegram_entities_to_matrix(
-    text: str, entities: list[TypeMessageEntity], offset: int = 0, length: int = None
+    text: str,
+    entities: list[TypeMessageEntity],
+    offset: int = 0,
+    length: int = None,
+    in_codeblock: bool = False,
 ) -> str:
+    def text_to_html(
+        val: str, _in_codeblock: bool = in_codeblock, escape_html: bool = True
+    ) -> str:
+        if escape_html:
+            val = escape(val)
+        if not _in_codeblock:
+            val = val.replace("\n", "<br/>")
+        return val
+
    if not entities:
-        return escape(text)
+        return text_to_html(text)
    if length is None:
        length = len(text)
    html = []
    last_offset = 0
    for i, entity in enumerate(entities):
-        if entity.offset > offset + length:
+        if entity.offset >= offset + length:
            break
        relative_offset = entity.offset - offset
        if relative_offset > last_offset:
-            html.append(escape(text[last_offset:relative_offset]))
+            html.append(text_to_html(text[last_offset:relative_offset]))
        elif relative_offset < last_offset:
            continue

+        while within_surrogate(text, relative_offset, length=length):
+            relative_offset += 1
+        while within_surrogate(text, relative_offset + entity.length, length=length):
+            entity.length += 1
+
        skip_entity = False
+        is_code_entity = isinstance(entity, (MessageEntityCode, MessageEntityPre))
        entity_text = await _telegram_entities_to_matrix(
            text=text[relative_offset : relative_offset + entity.length],
            entities=entities[i + 1 :],
            offset=entity.offset,
            length=entity.length,
+            in_codeblock=is_code_entity,
        )
+        entity_text = text_to_html(entity_text, is_code_entity, escape_html=False)
        entity_type = type(entity)

        if entity_type == MessageEntityBold:
@@ -287,7 +298,7 @@ async def _telegram_entities_to_matrix(
        elif entity_type == MessageEntityEmail:
            html.append(f"<a href='mailto:{entity_text}'>{entity_text}</a>")
        elif entity_type in (MessageEntityTextUrl, MessageEntityUrl):
-            skip_entity = await _parse_url(
+            await _parse_url(
                html, entity_text, entity.url if entity_type == MessageEntityTextUrl else None
            )
        elif entity_type in (
@@ -302,7 +313,7 @@ async def _telegram_entities_to_matrix(
        else:
            skip_entity = True
        last_offset = relative_offset + (0 if skip_entity else entity.length)
-    html.append(escape(text[last_offset:]))
+    html.append(text_to_html(text[last_offset:]))

    return "".join(html)

@@ -318,12 +329,24 @@ def _parse_pre(html: list[str], entity_text: str, language: str) -> bool:
 async def _parse_mention(html: list[str], entity_text: str) -> bool:
    username = entity_text[1:]

+    mxid = None
+    portal = None
+    # This is a bit complicated because public channels have both Puppet and Portal instances.
+    # Basically the currently intended output is:
+    # User/bot mention (bridge user)          -> real user mention
+    # User/bot mention (normal Telegram user) -> ghost user mention
+    # Public channel with existing portal     -> room mention
+    # Public channel without portal           -> ghost user mention
+    # Other chat                              -> room mention
    user = await u.User.find_by_username(username) or await pu.Puppet.find_by_username(username)
    if user:
+        if isinstance(user, pu.Puppet) and user.is_channel:
+            portal = await po.Portal.get_by_tgid(user.tgid)
        mxid = user.mxid
    else:
        portal = await po.Portal.find_by_username(username)
-        mxid = portal.alias or portal.mxid if portal else None
+    if portal and (portal.mxid or not user):
+        mxid = portal.alias or portal.mxid

    if mxid:
        html.append(f"<a href='https://matrix.to/#/{mxid}'>{entity_text}</a>")
@@ -347,11 +370,15 @@ async def _parse_name_mention(html: list[str], entity_text: str, user_id: Telegr


 message_link_regex = re.compile(
-    r"https?://t(?:elegram)?\.(?:me|dog)/([A-Za-z][A-Za-z0-9_]{3,}[A-Za-z0-9])/([0-9]{1,50})"
+    r"https?://t(?:elegram)?\.(?:me|dog)"
+    # /username or /c/id
+    r"/([A-Za-z][A-Za-z0-9_]{3,31}[A-Za-z0-9]|[Cc]/[0-9]{1,20})"
+    # /messageid
+    r"/([0-9]{1,20})"
 )


-async def _parse_url(html: list[str], entity_text: str, url: str) -> bool:
+async def _parse_url(html: list[str], entity_text: str, url: str):
    url = escape(url) if url else entity_text
    if not url.startswith(("https://", "http://", "ftp://", "magnet://")):
        url = "http://" + url
@@ -361,11 +388,13 @@ async def _parse_url(html: list[str], entity_text: str, url: str) -> bool:
        group, msgid_str = message_link_match.groups()
        msgid = int(msgid_str)

-        portal = await po.Portal.find_by_username(group)
+        if group.lower().startswith("c/"):
+            portal = await po.Portal.get_by_tgid(TelegramID(int(group[2:])))
+        else:
+            portal = await po.Portal.find_by_username(group)
        if portal:
            message = await DBMessage.get_one_by_tgid(TelegramID(msgid), portal.tgid)
            if message:
                url = f"https://matrix.to/#/{portal.mxid}/{message.mxid}"

    html.append(f"<a href='{url}'>{entity_text}</a>")
-    return False
@@ -1,5 +1,5 @@
 # mautrix-telegram - A Matrix-Telegram puppeting bridge
-# Copyright (C) 2021 Tulir Asokan
+# Copyright (C) 2022 Tulir Asokan
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as published by
@@ -61,90 +61,22 @@ class MatrixHandler(BaseMatrixHandler):

        self._previously_typing = {}

-    async def handle_puppet_invite(
-        self, room_id: RoomID, puppet: pu.Puppet, inviter: u.User, event_id: EventID
+    async def handle_puppet_group_invite(
+        self,
+        room_id: RoomID,
+        puppet: pu.Puppet,
+        invited_by: u.User,
+        evt: StateEvent,
+        members: list[UserID],
    ) -> None:
-        intent = puppet.default_mxid_intent
-        self.log.debug(f"{inviter.mxid} invited puppet for {puppet.tgid} to {room_id}")
-        if not await inviter.is_logged_in():
-            await intent.error_and_leave(
-                room_id, text="Please log in before inviting Telegram puppets."
-            )
-            return
-        portal = await po.Portal.get_by_mxid(room_id)
-        if portal:
-            if portal.peer_type == "user":
-                await intent.error_and_leave(
-                    room_id, text="You can not invite additional users to private chats."
-                )
-                return
-            await portal.invite_telegram(inviter, puppet)
-            await intent.join_room(room_id)
-            return
-        try:
-            members = await intent.get_room_members(room_id)
-        except MatrixError:
-            self.log.exception(f"Failed to get members after joining {room_id} as {intent.mxid}")
-            return
        if self.az.bot_mxid not in members:
-            if len(members) > 2:
-                await intent.error_and_leave(
-                    room_id,
-                    text=None,
-                    html=(
-                        f"Please invite "
-                        f"<a href='https://matrix.to/#/{self.az.bot_mxid}'>the bridge bot</a> "
-                        f"first if you want to create a Telegram chat."
-                    ),
-                )
-                return
-
-            await intent.join_room(room_id)
-            portal = await po.Portal.get_by_tgid(
-                puppet.tgid, tg_receiver=inviter.tgid, peer_type="user"
+            await puppet.default_mxid_intent.leave_room(
+                room_id, reason="This ghost does not join multi-user rooms without the bridge bot."
            )
-            if portal.mxid:
-                try:
-                    await portal.invite_to_matrix(inviter.mxid)
-                    await intent.send_notice(
-                        room_id,
-                        text=f"You already have a private chat with me: {portal.mxid}",
-                        html=(
-                            "You already have a private chat with me: "
-                            f"<a href='https://matrix.to/#/{portal.mxid}'>Link to room</a>"
-                        ),
-                    )
-                    await intent.leave_room(room_id)
-                    return
-                except MatrixError:
-                    pass
-            portal.mxid = room_id
-            e2be_ok = await portal.check_dm_encryption()
-            await portal.save()
-            await inviter.register_portal(portal)
-            if e2be_ok is True:
-                evt_type, content = await self.e2ee.encrypt(
-                    room_id,
-                    EventType.ROOM_MESSAGE,
-                    TextMessageEventContent(
-                        msgtype=MessageType.NOTICE,
-                        body=(
-                            "Portal to private chat created and end-to-bridge encryption enabled."
-                        ),
-                    ),
-                )
-                await intent.send_message_event(room_id, evt_type, content)
-            else:
-                message = "Portal to private chat created."
-                if e2be_ok is False:
-                    message += "\n\nWarning: Failed to enable end-to-bridge encryption"
-                await intent.send_notice(room_id, message)
-            await portal.update_bridge_info()
        else:
-            await intent.join_room(room_id)
-            await intent.send_notice(
+            await puppet.default_mxid_intent.send_notice(
                room_id,
-                "This puppet will remain inactive until a Telegram chat is created for this room.",
+                "This ghost will remain inactive until a Telegram chat is created for this room.",
            )

    async def handle_invite(
@@ -155,9 +87,13 @@ class MatrixHandler(BaseMatrixHandler):
            return
        await user.ensure_started()
        portal = await po.Portal.get_by_mxid(room_id)
-        if user and await user.has_full_access(allow_bot=True):
-            if portal and portal.allow_bridging:
-                await portal.invite_telegram(inviter, user)
+        if (
+            user
+            and portal
+            and await user.has_full_access(allow_bot=True)
+            and portal.allow_bridging
+        ):
+            await portal.handle_matrix_invite(inviter, user)

    async def handle_join(self, room_id: RoomID, user_id: UserID, event_id: EventID) -> None:
        user = await u.User.get_and_start_by_mxid(user_id)
@@ -62,7 +62,6 @@ media_content_table = {


 class PortalDedup:
-    pre_db_check: bool = False
    cache_queue_length: int = 256

    _dedup: deque[bytes | int]
@@ -1,5 +1,5 @@
 # mautrix-telegram - A Matrix-Telegram puppeting bridge
-# Copyright (C) 2021 Tulir Asokan
+# Copyright (C) 2022 Tulir Asokan
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as published by
@@ -20,10 +20,19 @@ from difflib import SequenceMatcher
 import unicodedata

 from telethon.tl.types import (
+    Channel,
+    ChatPhoto,
+    ChatPhotoEmpty,
    InputPeerPhotoFileLocation,
+    InputPeerUser,
+    PeerChannel,
+    PeerChat,
    PeerUser,
+    TypeChatPhoto,
    TypeInputPeer,
    TypeInputUser,
+    TypePeer,
+    TypeUserProfilePhoto,
    UpdateUserName,
    User,
    UserProfilePhoto,
@@ -33,7 +42,6 @@ from yarl import URL

 from mautrix.appservice import IntentAPI
 from mautrix.bridge import BasePuppet, async_getter_lock
-from mautrix.errors import MatrixError
 from mautrix.types import ContentURI, RoomID, SyncToken, UserID
 from mautrix.util.simple_template import SimpleTemplate

@@ -65,8 +73,13 @@ class Puppet(DBPuppet, BasePuppet):
        displayname_quality: int = 0,
        disable_updates: bool = False,
        username: str | None = None,
+        phone: str | None = None,
        photo_id: str | None = None,
+        avatar_url: ContentURI | None = None,
+        name_set: bool = False,
+        avatar_set: bool = False,
        is_bot: bool = False,
+        is_channel: bool = False,
        custom_mxid: UserID | None = None,
        access_token: str | None = None,
        next_batch: SyncToken | None = None,
@@ -81,8 +94,13 @@ class Puppet(DBPuppet, BasePuppet):
            displayname_quality=displayname_quality,
            disable_updates=disable_updates,
            username=username,
+            phone=phone,
            photo_id=photo_id,
+            avatar_url=avatar_url,
+            name_set=name_set,
+            avatar_set=avatar_set,
            is_bot=is_bot,
+            is_channel=is_channel,
            custom_mxid=custom_mxid,
            access_token=access_token,
            next_batch=next_batch,
@@ -109,7 +127,19 @@ class Puppet(DBPuppet, BasePuppet):

    @property
    def peer(self) -> PeerUser:
-        return PeerUser(user_id=self.tgid)
+        return (
+            PeerChannel(channel_id=self.tgid) if self.is_channel else PeerUser(user_id=self.tgid)
+        )
+
+    @property
+    def contact_info(self) -> dict:
+        return {
+            "name": self.displayname,
+            "username": self.username,
+            "phone": f"+{self.phone.lstrip('+')}" if self.phone else None,
+            "is_bot": self.is_bot,
+            "avatar_url": self.avatar_url,
+        }

    @property
    def plain_displayname(self) -> str:
@@ -185,9 +215,12 @@ class Puppet(DBPuppet, BasePuppet):
        return name

    @classmethod
-    def get_displayname(cls, info: User, enable_format: bool = True) -> tuple[str, int]:
-        fn = cls._filter_name(info.first_name)
-        ln = cls._filter_name(info.last_name)
+    def get_displayname(cls, info: User | Channel, enable_format: bool = True) -> tuple[str, int]:
+        if isinstance(info, Channel):
+            fn, ln = cls._filter_name(info.title), ""
+        else:
+            fn = cls._filter_name(info.first_name)
+            ln = cls._filter_name(info.last_name)
        data = {
            "phone number": info.phone if hasattr(info, "phone") else None,
            "username": info.username,
@@ -214,18 +247,28 @@ class Puppet(DBPuppet, BasePuppet):

        return (cls.displayname_template.format_full(name) if enable_format else name), quality

-    async def try_update_info(self, source: au.AbstractUser, info: User) -> None:
+    async def try_update_info(self, source: au.AbstractUser, info: User | Channel) -> None:
        try:
            await self.update_info(source, info)
        except Exception:
            source.log.exception(f"Failed to update info of {self.tgid}")

-    async def update_info(self, source: au.AbstractUser, info: User) -> None:
-        changed = False
+    async def update_info(self, source: au.AbstractUser, info: User | Channel) -> None:
+        is_bot = False if isinstance(info, Channel) else info.bot
+        is_channel = isinstance(info, Channel)
+        changed = is_bot != self.is_bot or is_channel != self.is_channel
+
+        self.is_bot = is_bot
+        self.is_channel = is_channel
+
        if self.username != info.username:
            self.username = info.username
            changed = True

+        if getattr(info, "phone", None) and self.phone != info.phone:
+            self.phone = info.phone
+            changed = True
+
        if not self.disable_updates:
            try:
                changed = await self.update_displayname(source, info) or changed
@@ -233,32 +276,46 @@ class Puppet(DBPuppet, BasePuppet):
            except Exception:
                self.log.exception(f"Failed to update info from source {source.tgid}")

-        self.is_bot = info.bot
-
        if changed:
+            await self.update_portals_meta()
            await self.save()

+    async def update_portals_meta(self) -> None:
+        if not p.Portal.private_chat_portal_meta and not self.mx.e2ee:
+            return
+        async for portal in p.Portal.find_private_chats_with(self.tgid):
+            await portal.update_info_from_puppet(self)
+
    async def update_displayname(
-        self, source: au.AbstractUser, info: User | UpdateUserName
+        self, source: au.AbstractUser, info: User | Channel | UpdateUserName
    ) -> bool:
        if self.disable_updates:
            return False
-        if source.is_relaybot or source.is_bot:
-            allow_because = "user is bot"
+        if (
+            self.displayname
+            and self.displayname.startswith("Deleted user ")
+            and not getattr(info, "deleted", False)
+        ):
+            allow_because = "target user was previously deleted"
+            self.displayname_quality = 0
+        elif source.is_relaybot or source.is_bot:
+            allow_because = "source user is a bot"
        elif self.displayname_source == source.tgid:
-            allow_because = "user is the primary source"
+            allow_because = "source user is the primary source"
+        elif isinstance(info, Channel):
+            allow_because = "target user is a channel"
        elif not isinstance(info, UpdateUserName) and not info.contact:
-            allow_because = "user is not a contact"
+            allow_because = "target user is not a contact"
        elif not self.displayname_source:
            allow_because = "no primary source set"
        elif not self.displayname:
-            allow_because = "user has no name"
+            allow_because = "target user has no name"
        else:
            return False

        if isinstance(info, UpdateUserName):
-            info = await source.client.get_entity(PeerUser(self.tgid))
-        if not info.contact:
+            info = await source.client.get_entity(self.peer)
+        if isinstance(info, Channel) or not info.contact:
            self.displayname_contact = False
        elif not self.displayname_contact:
            if not self.displayname:
@@ -267,7 +324,9 @@ class Puppet(DBPuppet, BasePuppet):
                return False

        displayname, quality = self.get_displayname(info)
-        if displayname != self.displayname and quality >= self.displayname_quality:
+        needs_reset = displayname != self.displayname or not self.name_set
+        is_high_quality = quality >= self.displayname_quality
+        if needs_reset and is_high_quality:
            allow_because = f"{allow_because} and quality {quality} >= {self.displayname_quality}"
            self.log.debug(
                f"Updating displayname of {self.id} (src: {source.tgid}, allowed "
@@ -281,11 +340,10 @@ class Puppet(DBPuppet, BasePuppet):
                await self.default_mxid_intent.set_displayname(
                    displayname[: self.config["bridge.displayname_max_length"]]
                )
-            except MatrixError:
-                self.log.exception("Failed to set displayname")
-                self.displayname = ""
-                self.displayname_source = None
-                self.displayname_quality = 0
+                self.name_set = True
+            except Exception as e:
+                self.log.warning(f"Failed to set displayname: {e}")
+                self.name_set = False
            return True
        elif source.is_relaybot or self.displayname_source is None:
            self.displayname_source = source.tgid
@@ -293,42 +351,44 @@ class Puppet(DBPuppet, BasePuppet):
        return False

    async def update_avatar(
-        self, source: au.AbstractUser, photo: UserProfilePhoto | UserProfilePhotoEmpty
+        self, source: au.AbstractUser, photo: TypeUserProfilePhoto | TypeChatPhoto
    ) -> bool:
        if self.disable_updates:
            return False

-        if photo is None or isinstance(photo, UserProfilePhotoEmpty):
+        if photo is None or isinstance(photo, (UserProfilePhotoEmpty, ChatPhotoEmpty)):
            photo_id = ""
-        elif isinstance(photo, UserProfilePhoto):
+        elif isinstance(photo, (UserProfilePhoto, ChatPhoto)):
            photo_id = str(photo.photo_id)
        else:
            self.log.warning(f"Unknown user profile photo type: {type(photo)}")
            return False
        if not photo_id and not self.config["bridge.allow_avatar_remove"]:
            return False
-        if self.photo_id != photo_id:
+        if self.photo_id != photo_id or not self.avatar_set:
            if not photo_id:
                self.photo_id = ""
-                try:
-                    await self.default_mxid_intent.set_avatar_url(ContentURI(""))
-                except MatrixError:
-                    self.log.exception("Failed to set avatar")
-                    self.photo_id = ""
-                return True
-
-            loc = InputPeerPhotoFileLocation(
-                peer=await self.get_input_entity(source), photo_id=photo.photo_id, big=True
-            )
-            file = await util.transfer_file_to_matrix(source.client, self.default_mxid_intent, loc)
-            if file:
+                self.avatar_url = None
+            elif self.photo_id != photo_id or not self.avatar_url:
+                file = await util.transfer_file_to_matrix(
+                    client=source.client,
+                    intent=self.default_mxid_intent,
+                    location=InputPeerPhotoFileLocation(
+                        peer=await self.get_input_entity(source), photo_id=photo.photo_id, big=True
+                    ),
+                    async_upload=self.config["homeserver.async_media"],
+                )
+                if not file:
+                    return False
                self.photo_id = photo_id
-                try:
-                    await self.default_mxid_intent.set_avatar_url(file.mxc)
-                except MatrixError:
-                    self.log.exception("Failed to set avatar")
-                    self.photo_id = ""
-                return True
+                self.avatar_url = file.mxc
+            try:
+                await self.default_mxid_intent.set_avatar_url(self.avatar_url or "")
+                self.avatar_set = True
+            except Exception as e:
+                self.log.warning(f"Failed to set avatar: {e}")
+                self.avatar_set = False
+            return True
        return False

    async def default_puppet_should_leave_room(self, room_id: RoomID) -> bool:
@@ -345,7 +405,9 @@ class Puppet(DBPuppet, BasePuppet):

    @classmethod
    @async_getter_lock
-    async def get_by_tgid(cls, tgid: TelegramID, *, create: bool = True) -> Puppet | None:
+    async def get_by_tgid(
+        cls, tgid: TelegramID, *, create: bool = True, is_channel: bool = False
+    ) -> Puppet | None:
        if tgid is None:
            return None

@@ -360,13 +422,37 @@ class Puppet(DBPuppet, BasePuppet):
            return puppet

        if create:
-            puppet = cls(tgid)
+            puppet = cls(tgid, is_channel=is_channel)
            await puppet.insert()
            puppet._add_to_cache()
            return puppet

        return None

+    @staticmethod
+    def get_id_from_peer(peer: TypePeer | User | Channel) -> TelegramID:
+        if isinstance(peer, (PeerUser, InputPeerUser)):
+            return TelegramID(peer.user_id)
+        elif isinstance(peer, PeerChannel):
+            return TelegramID(peer.channel_id)
+        elif isinstance(peer, PeerChat):
+            return TelegramID(peer.chat_id)
+        elif isinstance(peer, (User, Channel)):
+            return TelegramID(peer.id)
+        raise TypeError(f"invalid type {type(peer).__name__!r} in _id_from_peer()")
+
+    @classmethod
+    async def get_by_peer(
+        cls, peer: TypePeer | User | Channel, *, create: bool = True
+    ) -> Puppet | None:
+        if isinstance(peer, PeerChat):
+            return None
+        return await cls.get_by_tgid(
+            cls.get_id_from_peer(peer),
+            create=create,
+            is_channel=isinstance(peer, (PeerChannel, Channel)),
+        )
+
    @classmethod
    def get_by_mxid(cls, mxid: UserID, create: bool = True) -> Awaitable[Puppet | None]:
        return cls.get_by_tgid(cls.get_id_from_mxid(mxid), create=create)
@@ -40,7 +40,7 @@ class MautrixTelegramClient(TelegramClient):
        mime_type: str = None,
        attributes: List[TypeDocumentAttribute] = None,
        file_name: str = None,
-        max_image_size: float = 10 * 1000 ** 2,
+        max_image_size: float = 10 * 1000**2,
    ) -> Union[InputMediaUploadedDocument, InputMediaUploadedPhoto]:
        file_handle = await super().upload_file(file, file_name=file_name)

@@ -195,7 +195,8 @@ class User(DBUser, AbstractUser, BaseUser):
            if self.tgid:
                await self.push_bridge_state(BridgeStateEvent.UNKNOWN_ERROR, message=str(e))
        except UnauthorizedError as e:
-            self.log.error(f"Authorization error in start(): {type(e)}: {e}")
+            if delete_unless_authenticated or self.tgid:
+                self.log.error(f"Authorization error in start(): {type(e)}: {e}")
            if self.tgid:
                await self.push_bridge_state(
                    BridgeStateEvent.BAD_CREDENTIALS,
@@ -240,7 +241,7 @@ class User(DBUser, AbstractUser, BaseUser):
                )
            else:
                await self.push_bridge_state(
-                    BridgeStateEvent.UNKNOWN_ERROR, ttl=240, error="tg-not-connected"
+                    BridgeStateEvent.TRANSIENT_DISCONNECT, ttl=240, error="tg-not-connected"
                )

    async def fill_bridge_state(self, state: BridgeState) -> None:
@@ -268,6 +269,13 @@ class User(DBUser, AbstractUser, BaseUser):
            return None
        return await pu.Puppet.get_by_tgid(self.tgid)

+    async def get_portal_with(self, puppet: pu.Puppet, create: bool = True) -> po.Portal | None:
+        if not self.tgid:
+            return None
+        return await po.Portal.get_by_tgid(
+            puppet.tgid, tg_receiver=self.tgid, peer_type="user" if create else None
+        )
+
    async def stop(self) -> None:
        if self._track_connection_task:
            self._track_connection_task.cancel()
@@ -371,7 +379,7 @@ class User(DBUser, AbstractUser, BaseUser):
        if not self.config["bridge.kick_on_logout"]:
            return
        portals = await self.get_cached_portals()
-        for _, portal in portals.values():
+        for portal in portals.values():
            if not portal or portal.deleted or not portal.mxid or portal.has_bot:
                continue
            if portal.peer_type == "user":
@@ -448,7 +456,7 @@ class User(DBUser, AbstractUser, BaseUser):
    async def get_direct_chats(self) -> dict[UserID, list[RoomID]]:
        return {
            pu.Puppet.get_mxid_from_id(portal.tgid): [portal.mxid]
-            async for portal in po.Portal.find_private_chats(self.tgid)
+            async for portal in po.Portal.find_private_chats_of(self.tgid)
            if portal.mxid
        }

@@ -461,17 +469,22 @@ class User(DBUser, AbstractUser, BaseUser):
        if active and tag_info is None:
            tag_info = RoomTagInfo(order=0.5)
            tag_info[DOUBLE_PUPPET_SOURCE_KEY] = self.bridge.name
+            self.log.debug("Adding tag {tag} to {portal.mxid}/{portal.tgid}")
            await puppet.intent.set_room_tag(portal.mxid, tag, tag_info)
        elif (
            not active and tag_info and tag_info.get(DOUBLE_PUPPET_SOURCE_KEY) == self.bridge.name
        ):
+            self.log.debug("Removing tag {tag} from {portal.mxid}/{portal.tgid}")
            await puppet.intent.remove_room_tag(portal.mxid, tag)

-    async def _mute_room(cls, puppet: pu.Puppet, portal: po.Portal, mute_until: datetime) -> None:
-        if not cls.config["bridge.mute_bridging"] or not portal or not portal.mxid:
+    async def _mute_room(self, puppet: pu.Puppet, portal: po.Portal, mute_until: datetime) -> None:
+        if not self.config["bridge.mute_bridging"] or not portal or not portal.mxid:
            return
        now = datetime.utcnow().replace(tzinfo=timezone.utc)
        if mute_until is not None and mute_until > now:
+            self.log.debug(
+                f"Muting {portal.mxid}/{portal.tgid} (muted until {mute_until} on Telegram)"
+            )
            await puppet.intent.set_push_rule(
                PushRuleScope.GLOBAL,
                PushRuleKind.ROOM,
@@ -483,6 +496,7 @@ class User(DBUser, AbstractUser, BaseUser):
                await puppet.intent.remove_push_rule(
                    PushRuleScope.GLOBAL, PushRuleKind.ROOM, portal.mxid
                )
+                self.log.debug(f"Unmuted {portal.mxid}/{portal.tgid}")
            except MNotFound:
                pass

@@ -645,20 +659,28 @@ class User(DBUser, AbstractUser, BaseUser):
            acc = (acc * 20261 + contact) & 0xFFFFFFFF
        return acc & 0x7FFFFFFF

-    async def sync_contacts(self) -> None:
+    async def sync_contacts(self, get_info: bool = False) -> dict[TelegramID, dict]:
        existing_contacts = await self.get_contacts()
        contact_hash = self._hash_contacts(self.saved_contacts, existing_contacts)
        response = await self.client(GetContactsRequest(hash=contact_hash))
        if isinstance(response, ContactsNotModified):
-            return
+            if get_info:
+                return {
+                    tgid: (await pu.Puppet.get_by_tgid(tgid)).contact_info
+                    for tgid in existing_contacts
+                }
+            return {}
        self.log.debug(f"Updating contacts of {self.name}...")
        if self.saved_contacts != response.saved_count:
            self.saved_contacts = response.saved_count
            await self.save()
+        contacts = {}
        for user in response.users:
-            puppet = await pu.Puppet.get_by_tgid(user.id)
+            puppet: pu.Puppet = await pu.Puppet.get_by_tgid(user.id)
            await puppet.update_info(self, user)
-        await self.set_contacts(user.id for user in response.users)
+            contacts[user.id] = puppet.contact_info
+        await self.set_contacts(contacts.keys())
+        return contacts

    # endregion
    # region Class instance lookup
@@ -144,6 +144,7 @@ async def transfer_thumbnail_to_matrix(
    custom_data: bytes | None = None,
    width: int | None = None,
    height: int | None = None,
+    async_upload: bool = False,
 ) -> DBTelegramFile | None:
    if not Image or not VideoFileClip:
        return None
@@ -178,7 +179,7 @@ async def transfer_thumbnail_to_matrix(
    if encrypt:
        file, decryption_info = encrypt_attachment(file)
        upload_mime_type = "application/octet-stream"
-    content_uri = await intent.upload_media(file, upload_mime_type)
+    content_uri = await intent.upload_media(file, upload_mime_type, async_upload=async_upload)
    if decryption_info:
        decryption_info.url = content_uri

@@ -220,6 +221,7 @@ async def transfer_file_to_matrix(
    filename: str | None = None,
    encrypt: bool = False,
    parallel_id: int | None = None,
+    async_upload: bool = False,
 ) -> DBTelegramFile | None:
    location_id = _location_to_id(location)
    if not location_id:
@@ -246,6 +248,7 @@ async def transfer_file_to_matrix(
            filename,
            encrypt,
            parallel_id,
+            async_upload=async_upload,
        )


@@ -260,6 +263,7 @@ async def _unlocked_transfer_file_to_matrix(
    filename: str | None,
    encrypt: bool,
    parallel_id: int | None,
+    async_upload: bool = False,
 ) -> DBTelegramFile | None:
    db_file = await DBTelegramFile.get(loc_id)
    if db_file:
@@ -305,7 +309,7 @@ async def _unlocked_transfer_file_to_matrix(
        if encrypt and encrypt_attachment:
            file, decryption_info = encrypt_attachment(file)
            upload_mime_type = "application/octet-stream"
-        content_uri = await intent.upload_media(file, upload_mime_type)
+        content_uri = await intent.upload_media(file, upload_mime_type, async_upload=async_upload)
        if decryption_info:
            decryption_info.url = content_uri

@@ -325,7 +329,13 @@ async def _unlocked_transfer_file_to_matrix(
            thumbnail = thumbnail.location
        try:
            db_file.thumbnail = await transfer_thumbnail_to_matrix(
-                client, intent, thumbnail, video=file, mime_type=mime_type, encrypt=encrypt
+                client,
+                intent,
+                thumbnail,
+                video=file,
+                mime_type=mime_type,
+                encrypt=encrypt,
+                async_upload=async_upload,
            )
        except FileIdInvalidError:
            log.warning(f"Failed to transfer thumbnail for {thumbnail!s}", exc_info=True)
@@ -340,6 +350,7 @@ async def _unlocked_transfer_file_to_matrix(
            mime_type=converted_anim.thumbnail_mime,
            width=converted_anim.width,
            height=converted_anim.height,
+            async_upload=async_upload,
        )

    try:
@@ -126,8 +126,10 @@ class AuthAPI(abc.ABC):
                mxid=user.mxid,
                state="code",
                status=200,
-                message="Code requested successfully. Check your SMS "
-                "or Telegram client and enter the code below.",
+                message=(
+                    "Code requested successfully. Check your SMS "
+                    "or Telegram client and enter the code below."
+                ),
            )
        except PhoneNumberInvalidError:
            return self.get_login_response(
@@ -167,8 +169,10 @@ class AuthAPI(abc.ABC):
                state="request",
                status=429,
                errcode="phone_number_flood",
-                error="Your phone number has been temporarily blocked for flooding. "
-                "The ban is usually applied for around a day.",
+                error=(
+                    "Your phone number has been temporarily blocked for flooding. "
+                    "The ban is usually applied for around a day."
+                ),
            )
        except FloodWaitError as e:
            return self.get_login_response(
@@ -176,8 +180,10 @@ class AuthAPI(abc.ABC):
                state="request",
                status=429,
                errcode="flood_wait",
-                error="Your phone number has been temporarily blocked for flooding. "
-                f"Please wait for {format_duration(e.seconds)} before trying again.",
+                error=(
+                    "Your phone number has been temporarily blocked for flooding. "
+                    f"Please wait for {format_duration(e.seconds)} before trying again."
+                ),
            )
        except Exception:
            self.log.exception("Error requesting phone code")
@@ -237,6 +243,14 @@ class AuthAPI(abc.ABC):
    async def post_login_code(
        self, user: User, code: int, password_in_data: bool
    ) -> web.Response | None:
+        if not code:
+            return self.get_login_response(
+                mxid=user.mxid,
+                state="code",
+                status=400,
+                errcode="phone_code_missing",
+                error="You must provide the code from your phone.",
+            )
        try:
            user_info = await user.client.sign_in(code=code)
            await self.postprocess_login(user, user_info)
@@ -21,7 +21,7 @@ import json
 import logging

 from aiohttp import web
-from telethon.tl.types import ChannelForbidden, ChatForbidden, TypeChat
+from telethon.tl.types import ChannelForbidden, ChatForbidden, TypeChat, User as TLUser
 from telethon.utils import get_peer_id, resolve_id

 from mautrix.appservice import AppService
@@ -53,18 +53,20 @@ class ProvisioningAPI(AuthAPI):

        self.app = web.Application(loop=bridge.loop, middlewares=[self.error_middleware])

-        portal_prefix = "/portal/{mxid:![^/]+}"
+        portal_prefix = "/v1/portal/{mxid:![^/]+}"
        self.app.router.add_route("GET", f"{portal_prefix}", self.get_portal_by_mxid)
-        self.app.router.add_route("GET", "/portal/{tgid:-[0-9]+}", self.get_portal_by_tgid)
+        self.app.router.add_route("GET", "/v1/portal/{tgid:-[0-9]+}", self.get_portal_by_tgid)
        self.app.router.add_route(
            "POST", portal_prefix + "/connect/{chat_id:-[0-9]+}", self.connect_chat
        )
        self.app.router.add_route("POST", f"{portal_prefix}/create", self.create_chat)
        self.app.router.add_route("POST", f"{portal_prefix}/disconnect", self.disconnect_chat)

-        user_prefix = "/user/{mxid:@[^:]*:[^/]+}"
+        user_prefix = "/v1/user/{mxid:@[^:]*:[^/]+}"
        self.app.router.add_route("GET", f"{user_prefix}", self.get_user_info)
        self.app.router.add_route("GET", f"{user_prefix}/chats", self.get_chats)
+        self.app.router.add_route("GET", f"{user_prefix}/contacts", self.get_contacts)
+        self.app.router.add_route("POST", f"{user_prefix}/pm/{{identifier}}", self.start_dm)

        self.app.router.add_route("POST", f"{user_prefix}/logout", self.logout)
        self.app.router.add_route("POST", f"{user_prefix}/login/bot_token", self.send_bot_token)
@@ -212,7 +214,7 @@ class ProvisioningAPI(AuthAPI):
            portal.photo_id = ""
            await portal.save()

-        asyncio.create_task(portal.update_matrix_room(user, entity, direct=False, levels=levels))
+        asyncio.create_task(portal.update_matrix_room(user, entity, levels=levels))

        return web.Response(status=202, body="{}")

@@ -393,6 +395,62 @@ class ProvisioningAPI(AuthAPI):
                ]
            )

+    async def get_contacts(self, request: web.Request) -> web.Response:
+        data, user, err = await self.get_user_request_info(request, expect_logged_in=True)
+        if err is not None:
+            return err
+        return web.json_response(data=await user.sync_contacts())
+
+    async def start_dm(self, request: web.Request) -> web.Response:
+        data, user, err = await self.get_user_request_info(request, expect_logged_in=True)
+        if err is not None:
+            return err
+        try:
+            identifier: str | int = request.match_info["identifier"]
+            if isinstance(identifier, str) and identifier.isdecimal():
+                identifier = int(identifier)
+            target = await user.client.get_entity(identifier)
+        except ValueError:
+            return web.json_response(
+                {
+                    "error": "Invalid user identifier or user not found.",
+                    "errcode": "M_NOT_FOUND",
+                },
+                status=404,
+            )
+
+        if not target:
+            return web.json_response(
+                {
+                    "error": "User not found.",
+                    "errcode": "M_NOT_FOUND",
+                },
+                status=404,
+            )
+        elif not isinstance(target, TLUser):
+            return web.json_response(
+                {
+                    "error": "Identifier is not a user.",
+                },
+                status=400,
+            )
+        portal = await Portal.get_by_entity(target, tg_receiver=user.tgid)
+        puppet = await portal.get_dm_puppet()
+        if portal.mxid:
+            just_created = False
+        else:
+            await portal.create_matrix_room(user, target, [user.mxid])
+            just_created = True
+        return web.json_response(
+            {
+                "room_id": portal.mxid,
+                "just_created": just_created,
+                "id": portal.tgid,
+                "contact_info": puppet.contact_info,
+            },
+            status=201 if just_created else 200,
+        )
+
    async def send_bot_token(self, request: web.Request) -> web.Response:
        data, user, err = await self.get_user_request_info(request)
        if err is not None:
@@ -574,7 +632,7 @@ class ProvisioningAPI(AuthAPI):
        data = None
        if want_data and (request.method == "POST" or request.method == "PUT"):
            data = await self.get_data(request)
-            if not data:
+            if data is None:
                return (
                    None,
                    None,
@@ -18,7 +18,7 @@ moviepy>=1,<2
 phonenumbers>=8,<9

 #/metrics
-prometheus_client>=0.6,<0.13
+prometheus_client>=0.6,<0.15

 #/e2be
 python-olm>=3,<4
@@ -4,9 +4,9 @@ force_to_top = "typing"
 from_first = true
 combine_as_imports = true
 known_first_party = "mautrix"
+known_third_party = "telethon"
 line_length = 99

 [tool.black]
 line-length = 99
 target-version = ["py38"]
-required-version = "21.12b0"
@@ -3,10 +3,10 @@ python-magic>=0.4,<0.5
 commonmark>=0.8,<0.10
 aiohttp>=3,<4
 yarl>=1,<2
-mautrix>=0.14.3,<0.15
+mautrix>=0.16.0,<0.17
 #telethon>=1.24,<1.25
-# Fork to make session storage async and update to layer 137
-tulir-telethon==1.25.0a3
+# Fork to make session storage async and update to layer 138
+tulir-telethon==1.25.0a7
 asyncpg>=0.20,<0.26
 mako>=1,<2
 setuptools
@@ -51,7 +51,7 @@ setuptools.setup(

    install_requires=install_requires,
    extras_require=extras_require,
-    python_requires="~=3.7",
+    python_requires="~=3.8",

    classifiers=[
        "Development Status :: 4 - Beta",
Author	SHA1	Message	Date
Tulir Asokan	21c6a7d87f	Bump version to 0.11.3	2022-04-17 13:30:38 +03:00
Tulir Asokan	7c2a569235	Remove some unused fields	2022-04-13 14:43:53 +03:00
Tulir Asokan	1f5b91cbec	Update mautrix-python	2022-04-09 20:52:45 +03:00
Tulir Asokan	937f37eff0	Don't print generated registration message if config is invalid	2022-04-09 20:46:25 +03:00
Tulir Asokan	4f9f74204a	Update dependencies	2022-04-08 18:06:24 +03:00
Tulir Asokan	ed6735f10f	Fix creating new database	2022-04-06 19:04:12 +03:00
Tulir Asokan	5acd3cf007	Move API version number to endpoint definition	2022-04-06 14:33:03 +03:00
Tulir Asokan	279b997bd3	Add contacts and PM endpoints to OpenAPI spec	2022-04-06 14:29:50 +03:00
Tulir Asokan	4eb6095822	Update provisioning API spec to OpenAPI 3.1.0	2022-04-06 14:06:10 +03:00
Tulir Asokan	da5b8556f2	Add phone number field for puppets	2022-04-06 12:49:01 +03:00
Tulir Asokan	261f99ac82	Add provisioning API for listing contacts and starting DMs	2022-04-06 12:40:55 +03:00
Tulir Asokan	61f3c39cc2	Mark reactions as read when reading from Matrix	2022-04-01 15:56:16 +03:00
Tulir Asokan	39ab1d0c22	Fix another bug	2022-03-31 01:58:40 +03:00
Tulir Asokan	8abb9c3884	Fix bugs in Telegram entity parser	2022-03-31 01:53:51 +03:00
Tulir Asokan	58f8ee2ee2	Add config option to mark joined Telegram notices as read automatically	2022-03-30 11:58:40 +03:00
Tulir Asokan	474bcc9544	Update and unpin black	2022-03-28 22:29:22 +03:00
Tulir Asokan	a3f4e25101	Fix some bugs and add command to list invite links	2022-03-28 15:49:08 +03:00
Tulir Asokan	8befb664b6	Handle accepted into group action messages	2022-03-28 15:06:35 +03:00
Tulir Asokan	819dd1bcff	Allow generating invite links that need join approval	2022-03-28 15:03:22 +03:00
Tulir Asokan	2b8b853fec	Add proper message when requesting to join via invite link	2022-03-28 15:03:05 +03:00
Tulir Asokan	c536c4a265	Update changelog	2022-03-27 23:39:46 +03:00
Tulir Asokan	f13acfe825	Clarify that supergroups are channels in !tg bridge	2022-03-27 23:39:46 +03:00
Sumner Evans	8e763ba067	Merge pull request #775 from mautrix/sumner/bri-2582 async media: add ability to upload media asynchronously	2022-03-27 12:31:39 -06:00
Sumner Evans	8d7cfd8e46	parallel transfer: disable async_upload	2022-03-27 12:26:44 -06:00
Sumner Evans	601058d61c	async media: add ability to upload media asynchronously Co-authored-by: Tulir Asokan <tulir@maunium.net>	2022-03-27 12:26:44 -06:00
Tulir Asokan	f8596ef368	Use new ensure_has_html method instead of duplicating code	2022-03-23 19:51:01 +02:00
Tulir Asokan	7f0494d52d	Merge remote-tracking branch 'origin/sumner/bri-2496'	2022-03-22 16:29:48 +02:00
Sumner Evans	828478514b	Merge pull request #772 from mautrix/fix-kick-from-portals user: fix bug in kick_from_portals	2022-03-22 08:00:02 -06:00
Tulir Asokan	146f5437d1	Drop Python 3.7 support	2022-03-22 13:44:52 +02:00
Tulir Asokan	c28760f2a8	Adjust permission error messages	2022-03-22 13:44:52 +02:00
Tulir Asokan	04f30f6f29	Update mautrix-python	2022-03-22 13:44:52 +02:00
Tulir Asokan	caa1d3565b	Update changelog	2022-03-22 13:44:52 +02:00
Sumner Evans	1a7a020bb2	backfill: set timestamp on backfilled reactions to message timestamp	2022-03-22 00:48:12 -06:00
Sumner Evans	077ab2bb38	user: fix bug in kick_from_portals	2022-03-22 00:46:32 -06:00
Sumner Evans	6f491bf7d1	Merge pull request #771 from ProkopRandacek/master Add missing f in front of the f-string	2022-03-21 10:51:51 -06:00
Prokop Randacek	9b80c21d0a	add missing F	2022-03-21 10:11:45 +01:00
Tulir Asokan	e9dc76a860	Fix public channel mentions always using user instead of portal mxid	2022-03-15 16:32:21 +02:00
Tulir Asokan	9e73324a20	Fix bridge_matrix_leave config option	2022-03-14 12:00:14 +02:00
Tulir Asokan	7df93485d8	Remove extra parameter in call	2022-03-11 12:02:02 +02:00
Tulir Asokan	9018cea5ae	Update changelog	2022-03-07 18:52:15 +02:00
Tulir Asokan	32e023231d	Catch invalid integers passed to !tg create	2022-03-05 20:16:04 +02:00
Tulir Asokan	4766d14359	Move DM creation code to mautrix-python	2022-03-04 16:12:02 +02:00
Tulir Asokan	526b99ec04	Disable file logging in Docker by default To enable it, use a custom path that points at a writable volume	2022-03-04 10:57:08 +02:00
Nick Mills-Barrett	da132438bd	Only change the data directory ownership on Docker start	2022-03-03 18:17:39 +02:00
Tulir Asokan	54176ba2db	Fix self parameter name in _mute_room. Fixes #764	2022-03-02 14:33:09 +02:00
Tulir Asokan	1eca3c2ffd	Check peer_type in database when manually bridging portal	2022-03-02 14:33:06 +02:00
Tulir Asokan	98142f28cd	Improve logging of backfill count	2022-02-28 12:36:43 +02:00
Tulir Asokan	2cf7fc7059	Improve backfilling to fetch less redundant messages	2022-02-28 12:26:24 +02:00
Tulir Asokan	a34a18c6cc	Deduplicate user joined telegram messages	2022-02-28 11:59:44 +02:00
Tulir Asokan	fa738fbadf	Fix condition	2022-02-26 17:20:22 +02:00
Tulir Asokan	9ea0516166	Log when tagging and muting rooms	2022-02-25 19:35:05 +02:00
Tulir Asokan	b760aadb01	Add custom flag for force sending images as document	2022-02-25 12:38:01 +02:00
Tulir Asokan	24162e14ac	Remove msgtype in stickers	2022-02-23 14:36:53 +02:00
Tulir Asokan	9ea495324d	Don't try to set room state in non-existent portals	2022-02-23 12:46:16 +02:00
Tulir Asokan	437e86a15b	Keep newlines as-is in code blocks	2022-02-23 12:44:56 +02:00
Tulir Asokan	d9e0b75e9b	Update mautrix-python again	2022-02-22 13:53:43 +02:00
Tulir Asokan	9606518ba7	Update mautrix-python again	2022-02-22 12:40:16 +02:00
Tulir Asokan	e2774b830f	Update mautrix-python version	2022-02-22 11:58:27 +02:00
Tulir Asokan	951d82ad27	Remove max_document_size option and use media repo config directly	2022-02-20 13:47:40 +02:00
Tulir Asokan	4a55cf589c	Add initial db upgrade that jumps to latest version	2022-02-19 00:19:49 +02:00
Tulir Asokan	b07d80d876	Add support for converting t.me/c/<id>/<msgid> links	2022-02-18 17:22:26 +02:00
Tulir Asokan	ff995b2149	Bump version to 0.11.2	2022-02-14 18:19:03 +02:00
Tulir Asokan	2fb08d59c7	Return error if user tries to send empty login code to API	2022-02-09 12:05:16 +02:00
Sumner Evans	7950c5aa61	Merge pull request #754 from mautrix/sumner/bri-1893 link previews: support from Telegram -> Beeper	2022-02-08 11:37:23 -07:00
Sumner Evans	bf65824429	link previews: support from Telegram -> Beeper Co-authored-by: Tulir Asokan <tulir@maunium.net>	2022-02-08 11:35:38 -07:00
Tulir Asokan	4013f822de	Remove community_id config option	2022-02-06 17:38:15 +02:00
Tulir Asokan	b27519fd88	Add proper error message for syntax errors in !tg login. Fixes #755	2022-02-05 12:27:02 +02:00
Tulir Asokan	22f97756f7	Update CHANGELOG.md	2022-02-03 19:26:11 +02:00
Tulir Asokan	da3f4af171	Fix newlines in unformatted channel posts	2022-02-03 17:43:35 +02:00
Tulir Asokan	a55d9ae36a	Improve profile info syncing	2022-02-01 20:51:55 +02:00
Tulir Asokan	ecf3a12bd4	Mark user joined Telegram notice as read if it's backfilled	2022-02-01 17:33:53 +02:00
Tulir Asokan	e7248e2418	Fix timestamp of photo has expired messages in backfill	2022-02-01 16:48:51 +02:00
Tulir Asokan	fba118f0d9	Send joined telegram message instead of leaving portal empty	2022-02-01 16:44:31 +02:00
Tulir Asokan	100394d161	Add support for relay user distinguishers. Fixes #750	2022-02-01 16:05:56 +02:00
Tulir Asokan	a9908781be	Add basic support for MSC3488 location descriptions	2022-02-01 15:25:24 +02:00
Tulir Asokan	0f050edcd9	Add proper support for receiving messages sent as a channel. Fixes #740	2022-02-01 15:20:05 +02:00
Tulir Asokan	2182dfc86b	Update to Telegram API layer 138	2022-02-01 13:35:27 +02:00
Tulir Asokan	99fa7a57d2	Add config option to set maximum image pixels before sending as document Fixes #552	2022-01-31 15:57:00 +02:00
Tulir Asokan	6bf3d10e29	Improve handling of disappearing photos and files Fixes #508	2022-01-31 15:49:39 +02:00
Tulir Asokan	ebd2a38e56	Update black and fix version in CI	2022-01-30 12:29:05 +02:00
Tulir Asokan	03b094e4d4	Update mautrix-python	2022-01-30 12:04:21 +02:00
Tulir Asokan	21b509e5a0	Copy animated sticker args explicitly to remove unsupported args	2022-01-29 18:15:54 +02:00
Tulir Asokan	2732a85f9e	Update dependencies	2022-01-26 13:41:20 +02:00
Tulir Asokan	033141e435	Add warning for users who don't know what they're doing	2022-01-22 16:31:43 +02:00
Sumner Evans	251458a1d7	Merge pull request #745 from mautrix/pre-commit-config pre-commit: add configuration	2022-01-21 14:13:44 -07:00
Sumner Evans	7c4f406ac6	ci: add pre-commit-hooks to lint process	2022-01-21 11:15:52 -07:00
Sumner Evans	984c52afc9	dev-requirements: add pre-commit, isort, black	2022-01-21 11:15:21 -07:00
Sumner Evans	f664d4ad90	pre-commit: add configuration	2022-01-21 10:07:12 -07:00
Sumner Evans	8f61be76f9	Merge pull request #738 from mautrix/sumner/bri-1583-telegram-has-disconnected-i-woke-up-to bridge state: use TRANSIENT_DISCONNECT if connection drops and is expected to come back soon	2022-01-13 07:44:34 -07:00
Tulir Asokan	8003b9aa1c	Fix bug in !tg create. Fixes #736	2022-01-12 21:52:25 +02:00
Sumner Evans	a0fd98b9e2	bridge state: use TRANSIENT_DISCONNECT if connection drops and is expected to come back soon	2022-01-12 08:59:09 -07:00
Scott Weber	feac31e841	Very basic support for live location	2022-01-11 13:36:15 +02:00
Tulir Asokan	dd83d6278c	Add support for t.me/+code invite links	2022-01-10 23:23:16 +02:00