iartamonov/mautrix-telegram
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge branch 'master' into allow-portals-without-power

Browse Source
This commit is contained in:
Tulir Asokan
2018-03-11 10:24:15 +02:00
parent 8cce7a7c3a b06e7932f0
commit ad9cd27185
39 changed files with 1144 additions and 1356 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LICENSE
+67 -80
View File
@@ -1,23 +1,21 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
@@ -26,44 +24,34 @@ them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
@@ -72,7 +60,7 @@ modification follow.
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
@@ -549,35 +537,45 @@ to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
@@ -635,40 +633,29 @@ the "copyright" line and a pointer to where the full notice is found.
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
GNU Affero General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
README.md
+1 -2
View File
@@ -8,5 +8,4 @@ A Matrix-Telegram hybrid puppeting/relaybot bridge.
## Discussion
Matrix room: [`#telegram:maunium.net`](https://matrix.to/#/#telegram:maunium.net)
A Telegram chat bridged to the Matrix room will be created once the bridge supports using a bot
for unauthenticated users.
A Telegram chat bridged to the Matrix room might be created at some point.
ROADMAP.md
+1 -18
View File
@@ -45,7 +45,7 @@
* [x] Video messages
* [x] Documents
* [x] Message deletions
* [ ] Message edits (not yet supported in Matrix)
* [x] Message edits
* [x] Avatars
* [x] Presence
* [x] Typing notifications
@@ -74,23 +74,6 @@
* [x] Private chat creation by inviting Matrix puppet of Telegram user to new room
* [x] Option to use bot to relay messages for unauthenticated Matrix users
* [ ] Option to use own Matrix account for messages sent from other Telegram clients
* [Commands](https://github.com/tulir/mautrix-telegram/wiki/Management-commands)
* [x] Logging in and out (`login` + code entering)
* [x] Logging out
* [ ] Registering (`register`)
* [x] Searching for users (`search`)
* [x] Starting private chats (`pm`)
* [x] Joining chats with invite links (`join`)
* [x] Creating a Telegram chat for an existing Matrix room (`create`)
* [x] Upgrading the chat of a portal room into a supergroup (`upgrade`)
* [x] Change username of supergroup/channel (`group-name`)
* [x] Getting the Telegram invite link to a Matrix room (`invite-link`)
* [ ] Bridging existing Matrix rooms to existing Telegram chats (`bridge`)
* [ ] Unbridging Matrix rooms from Telegram chats (`unbridge`)
* Bridge administration
* [x] Clean up and forget a portal room (`delete-portal`)
* [x] Find and clean up old portal rooms (`clean-rooms`)
* [ ] Setting Matrix-only power levels (`powerlevel`)
† Information not automatically sent from source, i.e. implementation may not be possible
‡ Maybe, i.e. this feature may or may not be implemented at some point
alembic/versions/cfc972368e50_add_metadata_to_telegramfile.py
+35
View File
@@ -0,0 +1,35 @@
"""Add metadata to TelegramFile
Revision ID: cfc972368e50
Revises: 501dad2868bc
Create Date: 2018-03-09 16:07:01.236712
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = 'cfc972368e50'
down_revision = '501dad2868bc'
branch_labels = None
depends_on = None
def upgrade():
with op.batch_alter_table("telegram_file") as batch_op:
batch_op.add_column(sa.Column('size', sa.Integer(), nullable=True))
batch_op.add_column(sa.Column('width', sa.Integer(), nullable=True))
batch_op.add_column(sa.Column('height', sa.Integer(), nullable=True))
batch_op.add_column(sa.Column('thumbnail', sa.String(), nullable=True))
batch_op.create_foreign_key(constraint_name="fk_file_thumbnail",
referent_table="telegram_file",
local_cols=['thumbnail'],
remote_cols=['id'])
def downgrade():
with op.batch_alter_table("telegram_file") as batch_op:
batch_op.drop_column('size')
batch_op.drop_column('width')
batch_op.drop_column('height')
batch_op.drop_column('thumbnail')
example-config.yaml
+22 -9
View File
@@ -2,6 +2,7 @@
homeserver:
address: https://matrix.org
domain: matrix.org
verify_ssl: true
# Application service host/registration related details
# Changing these values requires regeneration of the registration.
@@ -70,15 +71,11 @@ bridge:
- username
- phone number
# Whether or not to use native Matrix replies. At the time of writing, only riot-web supports
# replies and the format of them is subject to change.
native_replies: true
# If native replies are disabled, should the custom replies contain a link to the message being
# replied to?
link_in_reply: false
# Show message editing as a reply to the original message.
# If this is false, message edits are not shown at all, as Matrix does not support editing yet.
edits_as_replies: false
# Highlight changed/added parts in edits. Requires lxml.
highlight_edits: false
# Whether or not Matrix bot messages (type m.notice) should be bridged.
bridge_notices: false
# The maximum number of simultaneous Telegram deletions to handle.
@@ -87,8 +84,13 @@ bridge:
# Allow logging in within Matrix. If false, the only way to log in is using the out-of-Matrix
# login website (see appservice.public config section)
allow_matrix_login: true
# Whether or not to allow creating portals from Telegram.
authless_relaybot_portals: true
# Use inline images instead of m.image to make rich captions possible.
# N.B. Inline images are not supported on all clients (e.g. Riot iOS).
inline_images: false
# Whether or not to bridge plaintext highlights.
# Only enable this if your displayname_template has some static part that the bridge can use to
# reliably identify what is a plaintext highlight.
plaintext_highlights: false
# The prefix for commands. Only required in non-management rooms.
command_prefix: "!tg"
@@ -108,6 +110,17 @@ bridge:
"public.example.com": "full"
"@admin:example.com": "admin"
# Options related to the message relay Telegram bot.
relaybot:
# Whether or not to allow creating portals from Telegram.
authless_portals: true
# Whether or not to allow Telegram group admins to use the bot commands.
whitelist_group_admins: true
# List of usernames/user IDs who are also allowed to use the bot commands.
whitelist:
- myusername
- 12345678
# Telegram config
telegram:
# Get your own API keys at https://my.telegram.org/apps
@@ -118,4 +131,4 @@ telegram:
# The version of the config. The bridge will read this and automatically update the config if
# the schema has changed. For the latest version, check the example config.
version: 1
version: 2
mautrix_appservice/__init__.py
-5
View File
@@ -1,5 +0,0 @@
from .appservice import AppService
from .errors import MatrixError, MatrixRequestError, IntentError
__version__ = "0.1.0"
__author__ = "Tulir Asokan <tulir@maunium.net>"
mautrix_appservice/appservice.py
-179
View File
@@ -1,179 +0,0 @@
# -*- coding: future_fstrings -*-
# matrix-appservice-python - A Matrix Application Service framework written in Python.
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Partly based on github.com/Cadair/python-appservice-framework (MIT license)
from contextlib import contextmanager
from aiohttp import web
import aiohttp
import asyncio
import logging
from .intent_api import HTTPAPI
from .state_store import StateStore
class AppService:
def __init__(self, server, domain, as_token, hs_token, bot_localpart, loop=None, log=None,
query_user=None, query_alias=None):
self.server = server
self.domain = domain
self.as_token = as_token
self.hs_token = hs_token
self.bot_mxid = f"@{bot_localpart}:{domain}"
self.state_store = StateStore(autosave_file="mx-state.json")
self.state_store.load("mx-state.json")
self.transactions = []
self._http_session = None
self._intent = None
self.loop = loop or asyncio.get_event_loop()
self.log = (logging.getLogger(log) if isinstance(log, str)
else log or logging.getLogger("mautrix_appservice"))
async def default_query_handler(_):
return None
self.query_user = query_user or default_query_handler
self.query_alias = query_alias or default_query_handler
self.event_handlers = []
self.app = web.Application(loop=self.loop)
self.app.router.add_route("PUT", "/transactions/{transaction_id}",
self._http_handle_transaction)
self.app.router.add_route("GET", "/rooms/{alias}", self._http_query_alias)
self.app.router.add_route("GET", "/users/{user_id}", self._http_query_user)
self.matrix_event_handler(self.update_state_store)
@property
def http_session(self):
if self._http_session is None:
raise AttributeError("the http_session attribute can only be used "
"from within the `AppService.run` context manager")
else:
return self._http_session
@property
def intent(self):
if self._intent is None:
raise AttributeError("the intent attribute can only be used from "
"within the `AppService.run` context manager")
else:
return self._intent
@contextmanager
def run(self, host="127.0.0.1", port=8080):
self._http_session = aiohttp.ClientSession(loop=self.loop)
self._intent = HTTPAPI(base_url=self.server, domain=self.domain, bot_mxid=self.bot_mxid,
token=self.as_token, log=self.log, state_store=self.state_store,
client_session=self._http_session).bot_intent()
yield self.loop.create_server(self.app.make_handler(), host, port)
self._intent = None
self._http_session.close()
self._http_session = None
def _check_token(self, request):
try:
token = request.rel_url.query["access_token"]
except KeyError:
return False
if token != self.hs_token:
return False
return True
async def _http_query_user(self, request):
if not self._check_token(request):
return web.Response(status=401)
user_id = request.match_info["userId"]
try:
response = await self.query_user(user_id)
except Exception:
self.log.exception("Exception in user query handler")
return web.Response(status=500)
if not response:
return web.Response(status=404)
return web.json_response(response)
async def _http_query_alias(self, request):
if not self._check_token(request):
return web.Response(status=401)
alias = request.match_info["alias"]
try:
response = await self.query_alias(alias)
except Exception:
self.log.exception("Exception in alias query handler")
return web.Response(status=500)
if not response:
return web.Response(status=404)
return web.json_response(response)
async def _http_handle_transaction(self, request):
if not self._check_token(request):
return web.Response(status=401)
transaction_id = request.match_info["transaction_id"]
if transaction_id in self.transactions:
return web.Response(status=200)
json = await request.json()
try:
events = json["events"]
except KeyError:
return web.Response(status=400)
for event in events:
self.handle_matrix_event(event)
self.transactions.append(transaction_id)
return web.json_response({})
async def update_state_store(self, event):
event_type = event["type"]
if event_type == "m.room.power_levels":
self.state_store.set_power_levels(event["room_id"], event["content"])
elif event_type == "m.room.member":
self.state_store.set_membership(event["room_id"], event["state_key"],
event["content"]["membership"])
def handle_matrix_event(self, event):
async def try_handle(handler):
try:
await handler(event)
except Exception:
self.log.exception("Exception in Matrix event handler")
for handler in self.event_handlers:
asyncio.ensure_future(try_handle(handler), loop=self.loop)
def matrix_event_handler(self, func):
self.event_handlers.append(func)
return func
mautrix_appservice/errors.py
-38
View File
@@ -1,38 +0,0 @@
# -*- coding: future_fstrings -*-
# mautrix-telegram - A Matrix-Telegram puppeting bridge
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
class MatrixError(Exception):
"""A generic Matrix error. Specific errors will subclass this."""
pass
class IntentError(MatrixError):
def __init__(self, message, source):
super().__init__(message)
self.source = source
class MatrixRequestError(MatrixError):
""" The home server returned an error response. """
def __init__(self, code=0, text="", errcode=None, message=None):
super().__init__(f"{code}: {text}")
self.code = code
self.text = text
self.errcode = errcode
self.message = message
mautrix_appservice/intent_api.py
-587
View File
@@ -1,587 +0,0 @@
# -*- coding: future_fstrings -*-
# mautrix-telegram - A Matrix-Telegram puppeting bridge
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from urllib.parse import quote
from time import time
from json.decoder import JSONDecodeError
from aiohttp.client_exceptions import ContentTypeError
import re
import json
import magic
import asyncio
from .errors import MatrixError, MatrixRequestError, IntentError
class HTTPAPI:
def __init__(self, base_url, domain=None, bot_mxid=None, token=None, identity=None, log=None,
state_store=None, client_session=None, child=False):
self.base_url = base_url
self.token = token
self.identity = identity
self.validate_cert = True
self.session = client_session
self.domain = domain
self.bot_mxid = bot_mxid
self._bot_intent = None
self.state_store = state_store
if child:
self.log = log
else:
self.intent_log = log.getChild("intent")
self.log = log.getChild("api")
self.txn_id = 0
self.children = {}
def user(self, user):
try:
return self.children[user]
except KeyError:
child = ChildHTTPAPI(user, self)
self.children[user] = child
return child
def bot_intent(self):
if self._bot_intent:
return self._bot_intent
return IntentAPI(self.bot_mxid, self, state_store=self.state_store, log=self.intent_log)
def intent(self, user):
return IntentAPI(user, self.user(user), self.bot_intent(), self.state_store,
self.intent_log)
async def _send(self, method, endpoint, content, query_params, headers):
while True:
query_params["access_token"] = self.token
request = self.session.request(method, endpoint, params=query_params,
data=content, headers=headers)
async with request as response:
if response.status < 200 or response.status >= 300:
errcode = message = None
try:
response_data = await response.json()
errcode = response_data["errcode"]
message = response_data["error"]
except (JSONDecodeError, ContentTypeError, KeyError):
pass
raise MatrixRequestError(code=response.status, text=await response.text(),
errcode=errcode, message=message)
if response.status == 429:
await asyncio.sleep(response.json()["retry_after_ms"] / 1000)
else:
return await response.json()
def _log_request(self, method, path, content, query_params):
log_content = content if not isinstance(content, bytes) else f"<{len(content)} bytes>"
log_content = log_content or "(No content)"
query_identity = query_params["user_id"] if "user_id" in query_params else "No identity"
self.log.debug("%s %s %s as user %s", method, path, log_content, query_identity)
def request(self, method, path, content=None, query_params=None, headers=None,
api_path="/_matrix/client/r0"):
content = content or {}
query_params = query_params or {}
headers = headers or {}
method = method.upper()
if method not in ["GET", "PUT", "DELETE", "POST"]:
raise MatrixError("Unsupported HTTP method: %s" % method)
if "Content-Type" not in headers:
headers["Content-Type"] = "application/json"
if headers["Content-Type"] == "application/json":
content = json.dumps(content)
if self.identity:
query_params["user_id"] = self.identity
self._log_request(method, path, content, query_params)
endpoint = self.base_url + api_path + path
return self._send(method, endpoint, content, query_params, headers or {})
def get_download_url(self, mxcurl):
if mxcurl.startswith('mxc://'):
return f"{self.base_url}/_matrix/media/r0/download/{mxcurl[6:]}"
else:
raise ValueError("MXC URL did not begin with 'mxc://'")
async def get_display_name(self, user_id):
content = await self.request("GET", f"/profile/{user_id}/displayname")
return content.get('displayname', None)
async def get_avatar_url(self, user_id):
content = await self.request("GET", f"/profile/{user_id}/avatar_url")
return content.get('avatar_url', None)
async def get_room_id(self, room_alias):
content = await self.request("GET", f"/directory/room/{quote(room_alias)}")
return content.get("room_id", None)
def set_typing(self, room_id, is_typing=True, timeout=5000, user=None):
content = {
"typing": is_typing
}
if is_typing:
content["timeout"] = timeout
user = user or self.identity
return self.request("PUT", f"/rooms/{room_id}/typing/{user}", content)
class ChildHTTPAPI(HTTPAPI):
def __init__(self, user, parent):
super().__init__(parent.base_url, parent.domain, parent.bot_mxid, parent.token, user,
parent.log, parent.state_store, parent.session, child=True)
self.parent = parent
@property
def txn_id(self):
return self.parent.txn_id
@txn_id.setter
def txn_id(self, value):
self.parent.txn_id = value
class IntentAPI:
mxid_regex = re.compile("@(.+):(.+)")
def __init__(self, mxid, client, bot=None, state_store=None, log=None):
self.client = client
self.bot = bot
self.mxid = mxid
self.log = log
results = self.mxid_regex.match(mxid)
if not results:
raise ValueError("invalid MXID")
self.localpart = results.group(1)
self.state_store = state_store
def user(self, user):
if not self.bot:
return self.client.intent(user)
else:
self.log.warning("Called IntentAPI#user() of child intent object.")
return self.bot.client.intent(user)
# region User actions
async def get_joined_rooms(self):
await self.ensure_registered()
response = await self.client.request("GET", "/joined_rooms")
return response["joined_rooms"]
async def set_display_name(self, name):
await self.ensure_registered()
content = {"displayname": name}
return await self.client.request("PUT", f"/profile/{self.mxid}/displayname", content)
async def set_presence(self, status="online", ignore_cache=False):
await self.ensure_registered()
if not ignore_cache and self.state_store.has_presence(self.mxid, status):
return
content = {
"presence": status
}
resp = await self.client.request("PUT", f"/presence/{self.mxid}/status", content)
self.state_store.set_presence(self.mxid, status)
return resp
async def set_avatar(self, url):
await self.ensure_registered()
content = {"avatar_url": url}
return await self.client.request("PUT", f"/profile/{self.mxid}/avatar_url", content)
async def upload_file(self, data, mime_type=None):
await self.ensure_registered()
mime_type = mime_type or magic.from_buffer(data, mime=True)
return await self.client.request("POST", "", content=data,
headers={"Content-Type": mime_type},
api_path="/_matrix/media/r0/upload")
async def download_file(self, url):
await self.ensure_registered()
url = self.client.get_download_url(url)
async with self.client.session.get(url) as response:
return await response.read()
# endregion
# region Room actions
async def create_room(self, alias=None, is_public=False, name=None, topic=None,
is_direct=False, invitees=None, initial_state=None,
guests_can_join=False):
await self.ensure_registered()
content = {
"visibility": "private",
"is_direct": is_direct,
"preset": "public_chat" if is_public else "private_chat",
"guests_can_join": guests_can_join,
}
if alias:
content["room_alias_name"] = alias
if invitees:
content["invite"] = invitees
if name:
content["name"] = name
if topic:
content["topic"] = topic
if initial_state:
content["initial_state"] = initial_state
return await self.client.request("POST", "/createRoom", content)
def _invite_direct(self, room_id, user_id):
content = {"user_id": user_id}
return self.client.request("POST", "/rooms/" + room_id + "/invite", content)
async def invite(self, room_id, user_id, check_cache=False):
await self.ensure_joined(room_id)
try:
ok_states = {"invite", "join"}
do_invite = (not check_cache
or self.state_store.get_membership(room_id, user_id) not in ok_states)
if do_invite:
response = await self._invite_direct(room_id, user_id)
self.state_store.invited(room_id, user_id)
return response
except MatrixRequestError as e:
if e.errcode != "M_FORBIDDEN":
raise IntentError(f"Failed to invite {user_id} to {room_id}", e)
if "is already in the room" in e.message:
self.state_store.joined(room_id, user_id)
def set_room_avatar(self, room_id, avatar_url, info=None):
content = {
"url": avatar_url,
}
if info:
content["info"] = info
return self.send_state_event(room_id, "m.room.avatar", content)
async def add_room_alias(self, room_id, localpart):
await self.ensure_registered()
content = {"room_id": room_id}
alias = f"#{localpart}:{self.client.domain}"
return await self.client.request("PUT", f"/directory/room/{quote(alias)}", content)
async def remove_room_alias(self, localpart):
await self.ensure_registered()
alias = f"#{localpart}:{self.client.domain}"
return await self.client.request("DELETE", f"/directory/room/{quote(alias)}")
def set_room_name(self, room_id, name):
body = {"name": name}
return self.send_state_event(room_id, "m.room.name", body)
async def get_power_levels(self, room_id, ignore_cache=False):
await self.ensure_joined(room_id)
if not ignore_cache:
try:
return self.state_store.get_power_levels(room_id)
except KeyError:
pass
levels = await self.client.request("GET",
f"/rooms/{quote(room_id)}/state/m.room.power_levels")
self.state_store.set_power_levels(room_id, levels)
return levels
async def set_power_levels(self, room_id, content):
if "events" not in content:
content["events"] = {}
response = await self.send_state_event(room_id, "m.room.power_levels", content)
self.state_store.set_power_levels(room_id, content)
return response
async def get_pinned_messages(self, room_id):
await self.ensure_joined(room_id)
response = await self.client.request("GET", f"/rooms/{room_id}/state/m.room.pinned_events")
return response["content"]["pinned"]
def set_pinned_messages(self, room_id, events):
return self.send_state_event(room_id, "m.room.pinned_events", {
"pinned": events
})
async def pin_message(self, room_id, event_id):
events = await self.get_pinned_messages(room_id)
if event_id not in events:
events.append(event_id)
await self.set_pinned_messages(room_id, events)
async def unpin_message(self, room_id, event_id):
events = await self.get_pinned_messages(room_id)
if event_id in events:
events.remove(event_id)
await self.set_pinned_messages(room_id, events)
async def set_join_rule(self, room_id, join_rule):
if join_rule not in ("public", "knock", "invite", "private"):
raise ValueError(f"Invalid join rule \"{join_rule}\"")
await self.send_state_event(room_id, "m.room.join_rules", {
"join_rule": join_rule,
})
async def get_event(self, room_id, event_id):
await self.ensure_joined(room_id)
return await self.client.request("GET", f"/rooms/{room_id}/event/{event_id}")
async def set_typing(self, room_id, is_typing=True, timeout=5000, ignore_cache=False):
await self.ensure_joined(room_id)
if not ignore_cache and is_typing == self.state_store.is_typing(room_id, self.mxid):
return
content = {
"typing": is_typing
}
if is_typing:
content["timeout"] = timeout
resp = await self.client.request("PUT", f"/rooms/{room_id}/typing/{self.mxid}", content)
self.state_store.set_typing(room_id, self.mxid, is_typing, timeout)
return resp
async def mark_read(self, room_id, event_id):
await self.ensure_joined(room_id)
return await self.client.request("POST", f"/rooms/{room_id}/receipt/m.read/{event_id}",
content={})
def send_notice(self, room_id, text, html=None, relates_to=None):
return self.send_text(room_id, text, html, "m.notice", relates_to)
def send_emote(self, room_id, text, html=None, relates_to=None):
return self.send_text(room_id, text, html, "m.emote", relates_to)
def send_image(self, room_id, url, info=None, text=None, relates_to=None):
return self.send_file(room_id, url, info or {}, text, "m.image", relates_to)
def send_file(self, room_id, url, info=None, text=None, file_type="m.file", relates_to=None):
return self.send_message(room_id, {
"msgtype": file_type,
"url": url,
"body": text or "Uploaded file",
"info": info or {},
"m.relates_to": relates_to or None,
})
def send_text(self, room_id, text, html=None, msgtype="m.text", relates_to=None):
if html:
if not text:
text = html
return self.send_message(room_id, {
"body": text,
"msgtype": msgtype,
"format": "org.matrix.custom.html",
"formatted_body": html or text,
"m.relates_to": relates_to or None,
})
else:
return self.send_message(room_id, {
"body": text,
"msgtype": msgtype,
"m.relates_to": relates_to or None,
})
def send_message(self, room_id, body):
return self.send_event(room_id, "m.room.message", body)
async def error_and_leave(self, room_id, text, html=None):
await self.ensure_joined(room_id)
await self.send_notice(room_id, text, html=html)
await self.leave_room(room_id)
def kick(self, room_id, user_id, message):
return self.set_membership(room_id, user_id, "leave", message)
def get_membership(self, room_id, user_id):
return self.get_state_event(room_id, "m.room.member", state_key=user_id)
def set_membership(self, room_id, user_id, membership, reason="", profile=None):
body = {
"membership": membership,
"reason": reason
}
profile = profile or {}
if "displayname" in profile:
body["displayname"] = profile["displayname"]
if "avatar_url" in profile:
body["avatar_url"] = profile["avatar_url"]
return self.send_state_event(room_id, "m.room.member", body, state_key=user_id)
def redact(self, room_id, event_id, reason=None, txn_id=None):
txn_id = txn_id or str(self.client.txn_id) + str(int(time() * 1000))
self.client.txn_id += 1
content = {}
if reason:
content["reason"] = reason
return self.client.request("PUT",
f"/rooms/{quote(room_id)}/redact/{quote(event_id)}/{txn_id}",
content)
@staticmethod
def _get_event_url(room_id, event_type, txn_id):
if not room_id:
raise ValueError("Room ID not given")
elif not event_type:
raise ValueError("Event type not given")
elif not txn_id:
raise ValueError("Transaction ID not given")
return f"/rooms/{quote(room_id)}/send/{quote(event_type)}/{quote(txn_id)}"
async def send_event(self, room_id, event_type, content, txn_id=None):
if not room_id:
raise ValueError("Room ID not given")
elif not event_type:
raise ValueError("Event type not given")
await self.ensure_joined(room_id)
await self._ensure_has_power_level_for(room_id, event_type)
txn_id = txn_id or str(self.client.txn_id) + str(int(time() * 1000))
self.client.txn_id += 1
url = self._get_event_url(room_id, event_type, txn_id)
return await self.client.request("PUT", url, content)
@staticmethod
def _get_state_url(room_id, event_type, state_key=""):
if not room_id:
raise ValueError("Room ID not given")
elif not event_type:
raise ValueError("Event type not given")
url = f"/rooms/{quote(room_id)}/state/{quote(event_type)}"
if state_key:
url += f"/{quote(state_key)}"
return url
async def send_state_event(self, room_id, event_type, content, state_key=""):
if not room_id:
raise ValueError("Room ID not given")
elif not event_type:
raise ValueError("Event type not given")
await self.ensure_joined(room_id)
await self._ensure_has_power_level_for(room_id, event_type, is_state_event=True)
url = self._get_state_url(room_id, event_type, state_key)
return await self.client.request("PUT", url, content)
async def get_state_event(self, room_id, event_type, state_key=""):
if not room_id:
raise ValueError("Room ID not given")
elif not event_type:
raise ValueError("Event type not given")
await self.ensure_joined(room_id)
url = self._get_state_url(room_id, event_type, state_key)
return await self.client.request("GET", url)
def join_room(self, room_id):
if not room_id:
raise ValueError("Room ID not given")
return self.ensure_joined(room_id, ignore_cache=True)
def _join_room_direct(self, room):
if not room:
raise ValueError("Room ID not given")
return self.client.request("POST", f"/join/{quote(room)}")
def leave_room(self, room_id):
if not room_id:
raise ValueError("Room ID not given")
try:
self.state_store.left(room_id, self.mxid)
return self.client.request("POST", f"/rooms/{quote(room_id)}/leave")
except MatrixRequestError as e:
if "not in room" not in e.message:
raise
def get_room_memberships(self, room_id):
if not room_id:
raise ValueError("Room ID not given")
return self.client.request("GET", f"/rooms/{quote(room_id)}/members")
async def get_room_members(self, room_id, allowed_memberships=("join",)):
memberships = await self.get_room_memberships(room_id)
return [membership["state_key"] for membership in memberships["chunk"] if
membership["content"]["membership"] in allowed_memberships]
async def get_room_state(self, room_id):
await self.ensure_joined(room_id)
state = await self.client.request("GET", f"/rooms/{quote(room_id)}/state")
# TODO update values based on state?
return state
# endregion
# region Ensure functions
async def ensure_joined(self, room_id, ignore_cache=False):
if not room_id:
raise ValueError("Room ID not given")
if not ignore_cache and self.state_store.is_joined(room_id, self.mxid):
return
await self.ensure_registered()
try:
await self._join_room_direct(room_id)
self.state_store.joined(room_id, self.mxid)
except MatrixRequestError as e:
if e.errcode != "M_FORBIDDEN" or not self.bot:
raise IntentError(f"Failed to join room {room_id} as {self.mxid}", e)
try:
await self.bot.invite(room_id, self.mxid)
await self._join_room_direct(room_id)
self.state_store.joined(room_id, self.mxid)
except MatrixRequestError as e2:
raise IntentError(f"Failed to join room {room_id} as {self.mxid}", e2)
def _register(self):
content = {"username": self.localpart}
query_params = {"kind": "user"}
return self.client.request("POST", "/register", content, query_params)
async def ensure_registered(self):
if self.state_store.is_registered(self.mxid):
return
try:
await self._register()
except MatrixRequestError as e:
if e.errcode != "M_USER_IN_USE":
self.log.exception(f"Failed to register {self.mxid}!")
# raise IntentError(f"Failed to register {self.mxid}", e)
return
self.state_store.registered(self.mxid)
async def _ensure_has_power_level_for(self, room_id, event_type, is_state_event=False):
if not room_id:
raise ValueError("Room ID not given")
elif not event_type:
raise ValueError("Event type not given")
if not self.state_store.has_power_levels(room_id):
await self.get_power_levels(room_id)
if self.state_store.has_power_level(room_id, self.mxid, event_type,
is_state_event=is_state_event):
return
elif not self.bot:
self.log.warning(
f"Power level of {self.mxid} is not enough for {event_type} in {room_id}")
# raise IntentError(f"Power level of {self.mxid} is not enough"
# f"for {event_type} in {room_id}")
return
# TODO implement
# endregion
mautrix_appservice/state_store.py
-155
View File
@@ -1,155 +0,0 @@
# -*- coding: future_fstrings -*-
# matrix-appservice-python - A Matrix Application Service framework written in Python.
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import json
import time
class StateStore:
def __init__(self, autosave_file=None):
self.autosave_file = autosave_file
# Persistent storage
self.registrations = set()
self.memberships = {}
self.power_levels = {}
# Non-persistent storage
self.presence = {}
self.typing = {}
def save(self, file):
if isinstance(file, str):
output = open(file, "w")
else:
output = file
json.dump({
"registrations": list(self.registrations),
"memberships": self.memberships,
"power_levels": self.power_levels,
}, output)
if isinstance(file, str):
output.close()
def load(self, file):
if isinstance(file, str):
try:
input_source = open(file, "r")
except FileNotFoundError:
return
else:
input_source = file
data = json.load(input_source)
if "registrations" in data:
self.registrations = set(data["registrations"])
if "memberships" in data:
self.memberships = data["memberships"]
if "power_levels" in data:
self.power_levels = data["power_levels"]
if isinstance(file, str):
input_source.close()
def _autosave(self):
if self.autosave_file:
self.save(self.autosave_file)
def set_presence(self, user, presence):
self.presence[user] = presence
def has_presence(self, user, presence):
try:
return self.presence[user] == presence
except KeyError:
return False
def set_typing(self, room_id, user, is_typing, timeout=0):
if is_typing:
ts = int(round(time.time() * 1000))
self.typing[(room_id, user)] = ts + timeout
else:
del self.typing[(room_id, user)]
def is_typing(self, room_id, user):
ts = int(round(time.time() * 1000))
try:
return self.typing[(room_id, user)] > ts
except KeyError:
return False
def is_registered(self, user):
return user in self.registrations
def registered(self, user):
self.registrations.add(user)
self._autosave()
def get_membership(self, room, user):
return self.memberships.get(room, {}).get(user, "left")
def is_joined(self, room, user):
return self.get_membership(room, user) == "join"
def set_membership(self, room, user, membership):
if room not in self.memberships:
self.memberships[room] = {}
self.memberships[room][user] = membership
self._autosave()
def joined(self, room, user):
return self.set_membership(room, user, "join")
def invited(self, room, user):
return self.set_membership(room, user, "invite")
def left(self, room, user):
return self.set_membership(room, user, "left")
def has_power_levels(self, room):
return room in self.power_levels
def get_power_levels(self, room):
return self.power_levels[room]
def has_power_level(self, room, user, event, is_state_event=False, default=None):
room_levels = self.power_levels.get(room, {})
default_required = default or (room_levels.get("state_default", 50) if is_state_event
else room_levels.get("events_default", 0))
required = room_levels.get("events", {}).get(event, default_required)
has = room_levels.get("users", {}).get(user, room_levels.get("users_default", 0))
return has >= required
def set_power_level(self, room, user, level):
if room not in self.power_levels:
self.power_levels[room] = {
"users": {},
"events": {},
}
elif "users" not in self.power_levels[room]:
self.power_levels[room]["users"] = {}
self.power_levels[room]["users"][user] = level
self._autosave()
def set_power_levels(self, room, content):
if "events" not in content:
content["events"] = {}
if "users" not in content:
content["users"] = {}
self.power_levels[room] = content
self._autosave()
mautrix_telegram/__main__.py
+8 -5
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
import argparse
import sys
import logging
@@ -35,6 +35,7 @@ from .user import init as init_user, User
from .bot import init as init_bot
from .portal import init as init_portal
from .puppet import init as init_puppet
from .formatter import init as init_formatter
from .public import PublicBridgeWebsite
from .context import Context
@@ -85,7 +86,8 @@ loop = asyncio.get_event_loop()
appserv = AppService(config["homeserver.address"], config["homeserver.domain"],
config["appservice.as_token"], config["appservice.hs_token"],
config["appservice.bot_username"], log="mau.as", loop=loop)
config["appservice.bot_username"], log="mau.as", loop=loop,
verify_ssl=config["homeserver.verify_ssl"])
context = Context(appserv, db_session, config, loop, None, None, telethon_session_container)
@@ -98,6 +100,7 @@ with appserv.run(config["appservice.hostname"], config["appservice.port"]) as st
init_abstract_user(context)
context.bot = init_bot(context)
context.mx = MatrixHandler(context)
init_formatter(context)
init_portal(context)
init_puppet(context)
startup_actions = init_user(context) + [start, context.mx.init_as_bot()]
mautrix_telegram/abstract_user.py
+5 -5
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
import platform
import os
@@ -136,7 +136,7 @@ class AbstractUser:
async def update_pinned_messages(self, update):
portal = po.Portal.get_by_tgid(update.channel_id)
if portal and portal.mxid:
await portal.update_telegram_pin(self, update.id)
await portal.receive_telegram_pin_id(update.id)
async def update_participants(self, update):
portal = po.Portal.get_by_tgid(update.participants.chat_id)
mautrix_telegram/bot.py
+69 -21
View File
@@ -3,23 +3,24 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from typing import Awaitable, Callable
import logging
import re
from telethon_aio.tl.types import *
from telethon_aio.tl.functions.messages import GetChatsRequest
from telethon_aio.tl.functions.channels import GetChannelsRequest
from telethon_aio.tl.functions.messages import GetChatsRequest, GetFullChatRequest
from telethon_aio.tl.functions.channels import GetChannelsRequest, GetParticipantRequest
from telethon_aio.errors import ChannelInvalidError, ChannelPrivateError
from .abstract_user import AbstractUser
@@ -29,16 +30,33 @@ from . import puppet as pu, portal as po, user as u
config = None
ReplyFunc = Callable[[str], Awaitable[Message]]
class Bot(AbstractUser):
log = logging.getLogger("mau.bot")
mxid_regex = re.compile("@.+:.+")
def __init__(self, token):
def __init__(self, token: str):
super().__init__()
self.token = token
self.whitelisted = True
self.username = None
self.chats = {chat.id: chat.type for chat in BotChat.query.all()}
self.tg_whitelist = []
self.whitelist_group_admins = config["bridge.relaybot.whitelist_group_admins"] or False
async def init_permissions(self):
whitelist = config["bridge.relaybot.whitelist"] or []
for id in whitelist:
if isinstance(id, str):
entity = await self.client.get_input_entity(id)
if isinstance(entity, InputUser):
id = entity.user_id
else:
id = None
if isinstance(id, int):
self.tg_whitelist.append(id)
async def start(self):
await super().start()
@@ -48,6 +66,7 @@ class Bot(AbstractUser):
return self
async def post_login(self):
await self.init_permissions()
info = await self.client.get_me()
self.tgid = info.id
self.username = info.username
@@ -68,19 +87,19 @@ class Bot(AbstractUser):
except (ChannelPrivateError, ChannelInvalidError):
self.remove_chat(id.channel_id)
def register_portal(self, portal):
def register_portal(self, portal: po.Portal):
self.add_chat(portal.tgid, portal.peer_type)
def unregister_portal(self, portal):
def unregister_portal(self, portal: po.Portal):
self.remove_chat(portal.tgid)
def add_chat(self, id, type):
def add_chat(self, id: int, type: str):
if id not in self.chats:
self.chats[id] = type
self.db.add(BotChat(id=id, type=type))
self.db.commit()
def remove_chat(self, id):
def remove_chat(self, id: int):
try:
del self.chats[id]
except KeyError:
@@ -88,8 +107,34 @@ class Bot(AbstractUser):
self.db.delete(BotChat.query.get(id))
self.db.commit()
async def handle_command_portal(self, portal, reply):
if not config["bridge.authless_relaybot_portals"]:
async def _can_use_commands(self, chat, tgid):
if tgid in self.tg_whitelist:
return True
user = u.User.get_by_tgid(tgid)
if user and user.is_admin:
self.tg_whitelist.append(user.tgid)
return True
if self.whitelist_group_admins:
if isinstance(chat, PeerChannel):
p = await self.client(GetParticipantRequest(chat, tgid))
return isinstance(p, (ChannelParticipantCreator, ChannelParticipantAdmin))
elif isinstance(chat, PeerChat):
chat = await self.client(GetFullChatRequest(chat.chat_id))
participants = chat.full_chat.participants.participants
for p in participants:
if p.user_id == tgid:
return isinstance(p, (ChatParticipantCreator, ChatParticipantAdmin))
async def check_can_use_commands(self, event: Message, reply: ReplyFunc):
if not await self._can_use_commands(event.to_id, event.from_id):
await reply("You do not have the permission to use that command.")
return False
return True
async def handle_command_portal(self, portal: po.Portal, reply: ReplyFunc):
if not config["bridge.relaybot.authless_portals"]:
return await reply("This bridge doesn't allow portal creation from Telegram.")
await portal.create_matrix_room(self)
@@ -101,7 +146,7 @@ class Bot(AbstractUser):
return await reply(
"Portal is not public. Use `/invite <mxid>` to get an invite.")
async def handle_command_invite(self, portal, reply, mxid):
async def handle_command_invite(self, portal: po.Portal, reply: ReplyFunc, mxid: str):
if len(mxid) == 0:
return await reply("Usage: `/invite <mxid>`")
elif not portal.mxid:
@@ -120,14 +165,14 @@ class Bot(AbstractUser):
await portal.main_intent.invite(portal.mxid, user.mxid)
return await reply(f"Invited `{user.mxid}` to the portal.")
def handle_command_id(self, message, reply):
def handle_command_id(self, message: Message, reply: ReplyFunc):
# Provide the prefixed ID to the user so that the user wouldn't need to specify whether the
# chat is a normal group or a supergroup/channel when using the ID.
if isinstance(message.to_id, PeerChannel):
return reply(f"-100{message.to_id.channel_id}")
return reply(str(-message.to_id.chat_id))
def match_command(self, text, command):
def match_command(self, text: str, command: str) -> bool:
text = text.lower()
command = f"/{command.lower()}"
command_targeted = f"{command}@{self.username.lower()}"
@@ -142,7 +187,7 @@ class Bot(AbstractUser):
return False
async def handle_command(self, message):
async def handle_command(self, message: Message):
def reply(reply_text):
return self.client.send_message(message.to_id, reply_text, markdown=True,
reply_to=message.id)
@@ -155,15 +200,19 @@ class Bot(AbstractUser):
portal = po.Portal.get_by_entity(message.to_id)
if self.match_command(text, "portal"):
if not await self.check_can_use_commands(message, reply):
return
await self.handle_command_portal(portal, reply)
elif self.match_command(text, "invite"):
if not await self.check_can_use_commands(message, reply):
return
try:
mxid = text[text.index(" ") + 1:]
except ValueError:
mxid = ""
await self.handle_command_invite(portal, reply, mxid=mxid)
def handle_service_message(self, message):
def handle_service_message(self, message: MessageService):
to_id = message.to_id
if isinstance(to_id, PeerChannel):
to_id = to_id.channel_id
@@ -183,7 +232,6 @@ class Bot(AbstractUser):
async def update(self, update):
if not isinstance(update, (UpdateNewMessage, UpdateNewChannelMessage)):
return
if isinstance(update.message, MessageService):
return self.handle_service_message(update.message)
@@ -193,11 +241,11 @@ class Bot(AbstractUser):
if is_command:
return await self.handle_command(update.message)
def is_in_chat(self, peer_id):
def is_in_chat(self, peer_id) -> bool:
return peer_id in self.chats
@property
def name(self):
def name(self) -> str:
return "bot"
mautrix_telegram/commands/auth.py
+71 -25
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
import asyncio
from telethon_aio.errors import *
@@ -51,6 +51,50 @@ def register(evt):
return evt.reply("Not yet implemented.")
@command_handler(needs_auth=False, management_only=True)
async def register(evt):
if evt.sender.logged_in:
return await evt.reply("You are already logged in.")
elif len(evt.args) < 1:
return await evt.reply("**Usage:** `$cmdprefix+sp register <phone> <full name>`")
phone_number = evt.args[0]
full_name = evt.args[1:].split(" ", 1)
if len(full_name) == 1:
full_name.append("")
await request_code(evt, phone_number, {
"next": enter_code_register,
"action": "Register",
"full_name": full_name,
})
async def enter_code_register(evt):
if len(evt.args) == 0:
return await evt.reply("**Usage:** `$cmdprefix+sp <code>`")
try:
await evt.sender.ensure_started(even_if_no_session=True)
first_name, last_name = evt.sender.command_status["full_name"]
user = await evt.sender.client.sign_up(evt.args[0], first_name, last_name)
asyncio.ensure_future(evt.sender.post_login(user), loop=evt.loop)
evt.sender.command_status = None
return await evt.reply(f"Successfully registered to Telegram.")
except PhoneNumberOccupiedError:
return await evt.reply("That phone number has already been registered. "
"You can log in with `$cmdprefix+sp login`.")
except FirstNameInvalidError:
return await evt.reply("Invalid name. Please set a Matrix displayname before registering.")
except PhoneCodeExpiredError:
return await evt.reply(
"Phone code expired. Try again with `$cmdprefix+sp register <phone>`.")
except PhoneCodeInvalidError:
return await evt.reply("Invalid phone code.")
except Exception:
evt.log.exception("Error sending phone code")
return await evt.reply("Unhandled exception while sending code. "
"Check console for more details.")
@command_handler(needs_auth=False, management_only=True)
async def login(evt):
if evt.sender.logged_in:
@@ -80,22 +124,12 @@ async def login(evt):
return await evt.reply("This bridge instance has been configured to not allow logging in.")
@command_handler(needs_auth=False)
async def enter_phone(evt):
if len(evt.args) == 0:
return await evt.reply("**Usage:** `$cmdprefix+sp enter-phone <phone>`")
elif not evt.config.get("bridge.allow_matrix_login", True):
return await evt.reply("This bridge instance does not allow in-Matrix login. "
"Please use `$cmdprefix+sp login` to get login instructions")
phone_number = evt.args[0]
async def request_code(evt, phone_number, next_status):
ok = False
try:
await evt.sender.ensure_started(even_if_no_session=True)
await evt.sender.client.sign_in(phone_number)
evt.sender.command_status = {
"next": enter_code,
"action": "Login",
}
ok = True
return await evt.reply(f"Login code sent to {phone_number}. Please send the code here.")
except PhoneNumberAppSignupForbiddenError:
return await evt.reply(
@@ -109,17 +143,31 @@ async def enter_phone(evt):
"Your phone number has been temporarily blocked for flooding. "
f"Please wait for {format_duration(e.seconds)} before trying again.")
except PhoneNumberBannedError:
return await evt.reply("Your phone number has been banned from Telegram.")
return await evt.reply("Your phone number has been banned from Telegram.")
except PhoneNumberUnoccupiedError:
return await evt.reply("That phone number has not been registered. "
"Please register with `$cmdprefix+sp register <phone>`.")
return await evt.reply("That phone number has not been registered. "
"Please register with `$cmdprefix+sp register <phone>`.")
except Exception:
evt.log.exception("Error requesting phone code")
return await evt.reply("Unhandled exception while requesting code. "
"Check console for more details.")
finally:
if evt.sender.command_status["next"] == enter_phone:
evt.sender.command_status = None
evt.sender.command_status = next_status if ok else None
@command_handler(needs_auth=False)
async def enter_phone(evt):
if len(evt.args) == 0:
return await evt.reply("**Usage:** `$cmdprefix+sp enter-phone <phone>`")
elif not evt.config.get("bridge.allow_matrix_login", True):
return await evt.reply("This bridge instance does not allow in-Matrix login. "
"Please use `$cmdprefix+sp login` to get login instructions")
phone_number = evt.args[0]
await request_code(evt, phone_number, {
"next": enter_code,
"action": "Login",
})
@command_handler(needs_auth=False)
@@ -136,8 +184,7 @@ async def enter_code(evt):
evt.sender.command_status = None
return await evt.reply(f"Successfully logged in as @{user.username}")
except PhoneCodeExpiredError:
return await evt.reply(
"Phone code expired. Try again with `$cmdprefix+sp login <phone>`.")
return await evt.reply("Phone code expired. Try again with `$cmdprefix+sp login`.")
except PhoneCodeInvalidError:
return await evt.reply("Invalid phone code.")
except SessionPasswordNeededError:
@@ -160,7 +207,6 @@ async def enter_password(evt):
elif not evt.config.get("bridge.allow_matrix_login", True):
return await evt.reply("This bridge instance does not allow in-Matrix login. "
"Please use `$cmdprefix+sp login` to get login instructions")
try:
await evt.sender.ensure_started(even_if_no_session=True)
user = await evt.sender.client.sign_in(password=evt.args[0])
mautrix_telegram/commands/clean_rooms.py
+4 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from mautrix_appservice import MatrixRequestError
from . import command_handler
mautrix_telegram/commands/handler.py
+6 -5
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
import markdown
import logging
@@ -81,12 +81,13 @@ class CommandHandler:
async def handle(self, room, sender, command, args, is_management, is_portal):
evt = CommandEvent(self, room, sender, command, args,
is_management, is_portal)
orig_command = command
command = command.lower()
try:
command = command_handlers[command]
except KeyError:
if sender.command_status and "next" in sender.command_status:
args.insert(0, command)
args.insert(0, orig_command)
evt.command = ""
command = sender.command_status["next"]
else:
mautrix_telegram/commands/meta.py
+10 -6
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from . import command_handler
@@ -57,7 +57,8 @@ def help(evt):
#### Miscellaneous things
**search** [_-r|--remote_] <_query_> - Search your contacts or the Telegram servers for users.
**sync** [`chats`|`contacts`|`me`] - Synchronize your chat portals, contacts and/or own info.
**ping-bot** - Get info of the message relay Telegram bot.
**ping-bot** - Get info of the message relay Telegram bot.
**set-pl** <_level_> [_mxid_] - Set a temporary power level without affecting Telegram.
#### Initiating chats
**pm** <_identifier_> - Open a private chat with the given Telegram user. The identifier is either
@@ -72,7 +73,10 @@ def help(evt):
**delete-portal** - Remove all users from the current portal room and forget the portal.
Only works for group chats; to delete a private chat portal, simply
leave the room.
**unbridge** - Remove puppets from the current portal room and forget the portal.
**unbridge** - Remove puppets from the current portal room and forget the portal.
**bridge** [_id_] - Bridge the current Matrix room to the Telegram chat with the given
ID. The ID must be the prefixed version that you get with the `/id`
command of the Telegram-side bot.
**group-name** <_name_|`-`> - Change the username of a supergroup/channel. To disable, use a dash
(`-`) as the name.
**clean-rooms** - Clean up unused portal/management rooms.
mautrix_telegram/commands/portal.py
+22 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
import asyncio
from telethon_aio.errors import *
@@ -23,6 +23,24 @@ from .. import portal as po
from . import command_handler, CommandEvent
@command_handler(needs_admin=True, needs_auth=False, name="set-pl")
async def set_power_level(evt: CommandEvent):
try:
level = int(evt.args[0])
except KeyError:
return await evt.reply("**Usage:** `$cmdprefix+sp set-power <level> [mxid]`")
except ValueError:
return await evt.reply("The level must be an integer.")
levels = await evt.az.intent.get_power_levels(evt.room_id)
mxid = evt.args[1] if len(evt.args) > 1 else evt.sender.mxid
levels["users"][mxid] = level
try:
await evt.az.intent.set_power_levels(evt.room_id, levels)
except MatrixRequestError:
evt.log.exception("Failed to set power level.")
return await evt.reply("Failed to set power level.")
@command_handler()
async def invite_link(evt: CommandEvent):
portal = po.Portal.get_by_mxid(evt.room_id)
mautrix_telegram/commands/telegram.py
+4 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from telethon_aio.errors import *
from telethon_aio.tl.types import User as TLUser
from telethon_aio.tl.functions.messages import ImportChatInviteRequest, CheckChatInviteRequest
mautrix_telegram/config.py
+107 -14
View File
@@ -3,19 +3,21 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from ruamel.yaml import YAML
from ruamel.yaml.comments import CommentedMap
from ruamel.yaml.tokens import CommentToken
from ruamel.yaml.error import CommentMark
import random
import string
@@ -42,6 +44,9 @@ class DictWithRecursion:
def __getitem__(self, key):
return self.get(key, None)
def __contains__(self, key):
return self[key] is not None
def _recursive_set(self, data, key, value):
if '.' in key:
key, next_key = key.split('.', 1)
@@ -71,6 +76,7 @@ class DictWithRecursion:
return
try:
del data[key]
del data.ca.items[key]
except KeyError:
pass
@@ -80,6 +86,7 @@ class DictWithRecursion:
return
try:
del self._data[key]
del self._data.ca.items[key]
except KeyError:
pass
@@ -93,10 +100,19 @@ class DictWithRecursion:
except ValueError:
path = None
entry = self[path] if path else self._data
c = self._data.ca.items.setdefault(key, [None, [], None, None])
c = entry.ca.items.setdefault(key, [None, [], None, None])
c[1] = []
entry.yaml_set_comment_before_after_key(key=key, before=message, indent=indent)
def comment_newline(self, key):
try:
path, key = key.rsplit(".", 1)
except ValueError:
path = None
entry = self[path] if path else self._data
c = entry.ca.items.setdefault(key, [None, [], None, None])
c[2] = CommentToken("\n\n", CommentMark(0), None)
class Config(DictWithRecursion):
def __init__(self, path, registration_path):
@@ -131,10 +147,16 @@ class Config(DictWithRecursion):
del self["bridge.whitelist"]
del self["bridge.admins"]
self["bridge.authless_relaybot_portals"] = self.get("bridge.authless_relaybot_portals",
True)
self.comment("bridge.authless_relaybot_portals",
"Whether or not to allow creating portals from Telegram.")
if "bridge.authless_relaybot_portals" not in self:
self["bridge.authless_relaybot_portals"] = True
self.comment("bridge.authless_relaybot_portals",
"Whether or not to allow creating portals from Telegram.")
if "bridge.max_telegram_delete" not in self:
self["bridge.max_telegram_delete"] = 10
self.comment("bridge.max_telegram_delete",
"The maximum number of simultaneous Telegram deletions to handle.\n"
"A large number of simultaneous redactions could put strain on your "
"homeserver.")
self.comment("bridge.permissions", "\n".join((
"",
@@ -156,13 +178,84 @@ class Config(DictWithRecursion):
"\nThe version of the config. The bridge will read this and automatically "
"update the config if\nthe schema has changed. For the latest version, "
"check the example config.")
return self["version"]
def update_1_2(self):
del self["bridge.link_in_reply"]
del self["bridge.native_replies"]
if "bridge.bridge_notices" not in self:
self["bridge.bridge_notices"] = False
self.comment("bridge.bridge_notices",
"Whether or not Matrix bot messages (type m.notice) should be bridged.")
if "bridge.allow_matrix_login" not in self:
self["bridge.allow_matrix_login"] = True
self.comment("bridge.allow_matrix_login",
"Allow logging in within Matrix. If false, the only way to log in is "
"using the out-of-Matrix login website (see appservice.public config "
"section)")
if "bridge.inline_images" not in self:
self["bridge.inline_images"] = False
self.comment("bridge.inline_images",
"Use inline images instead of m.image to make rich captions possible.\n"
"N.B. Inline images are not supported on all clients (e.g. Riot iOS).")
if "appservice.public" not in self:
self["appservice.public.enabled"] = False
self["appservice.public.prefix"] = "/public"
self["appservice.public.external"] = "https://example.com/public"
self.comment("appservice.public",
"Public part of web server for out-of-Matrix interaction with the "
"bridge.\nUsed for things like login if the user wants to make sure the "
"2FA password isn't stored in the HS database.")
self.comment("appservice.public.enabled",
"Whether or not the public-facing endpoints should be enabled.")
self.comment("appservice.public.prefix",
"The prefix to use in the public-facing endpoints.")
self.comment("appservice.public.external",
"The base URL where the public-facing endpoints are available. The "
"prefix is not added\nimplicitly.")
if "homeserver.verify_ssl" not in self:
self["homeserver.verify_ssl"] = True
self["version"] = 2
return self["version"]
def update_2_3(self):
if "bridge.plaintext_highlights" not in self:
self["bridge.plaintext_highlights"] = False
self.comment("bridge.plaintext_highlights",
"Whether or not to bridge plaintext highlights.\n"
"Only enable this if your displayname_template has some static part that "
"the bridge can use to\nreliably identify what is a plaintext highlight.")
if "bridge.highlight_edits" not in self:
self["bridge.highlight_edits"] = False
self.comment("bridge.highlight_edits",
"Highlight changed/added parts in edits. Requires lxml.")
if "bridge.relaybot" not in self:
self["bridge.relaybot.authless_portals"] = bool(
self["bridge.authless_relaybot_portals"]) or True
del self["bridge.authless_relaybot_portals"]
self["bridge.relaybot.whitelist_group_admins"] = True
self["bridge.relaybot.whitelist"] = []
self.comment("bridge.relaybot", "Options related to the message relay Telegram bot.")
self.comment("bridge.relaybot.authless_portals",
"Whether or not to allow creating portals from Telegram.")
self.comment("bridge.relaybot.whitelist_group_admins",
"Whether or not to allow Telegram group admins to use the bot commands.")
self.comment("bridge.relaybot.whitelist",
"List of usernames/user IDs who are also allowed to use the bot commands.")
self["version"] = 3
return self["version"]
def check_updates(self):
if self.get("version", 0) == 0:
self.update_0_1()
else:
return
self.save()
version = self.get("version", 0)
new_version = version
if version < 1:
new_version = self.update_0_1()
if version < 2:
new_version = self.update_1_2()
if version < 3:
new_version = self.update_2_3()
if new_version != version:
self.save()
def _get_permissions(self, key):
level = self["bridge.permissions"].get(key, "")
mautrix_telegram/context.py
+4 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
class Context:
mautrix_telegram/db.py
+11 -6
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from sqlalchemy import (Column, UniqueConstraint, ForeignKey, ForeignKeyConstraint, Integer,
BigInteger, String, Boolean)
from sqlalchemy.orm import relationship
@@ -81,8 +81,8 @@ class Contact(Base):
query = None
__tablename__ = "contact"
user = Column("user", Integer, ForeignKey("user.tgid"), primary_key=True)
contact = Column("contact", Integer, ForeignKey("puppet.id"), primary_key=True)
user = Column(Integer, ForeignKey("user.tgid"), primary_key=True)
contact = Column(Integer, ForeignKey("puppet.id"), primary_key=True)
class Puppet(Base):
@@ -112,6 +112,11 @@ class TelegramFile(Base):
mime_type = Column(String)
was_converted = Column(Boolean)
timestamp = Column(BigInteger)
size = Column(Integer, nullable=True)
width = Column(Integer, nullable=True)
height = Column(Integer, nullable=True)
thumbnail_id = Column("thumbnail", String, ForeignKey("telegram_file.id"), nullable=True)
thumbnail = relationship("TelegramFile", uselist=False)
def init(db_session):
mautrix_telegram/formatter/__init__.py
+9 -2
View File
@@ -1,2 +1,9 @@
from .from_matrix import matrix_reply_to_telegram, matrix_to_telegram, matrix_text_to_telegram
from .from_telegram import telegram_reply_to_matrix, telegram_to_matrix
from .from_matrix import (matrix_reply_to_telegram, matrix_to_telegram, matrix_text_to_telegram,
init_mx)
from .from_telegram import (telegram_reply_to_matrix, telegram_to_matrix, init_tg)
from ..context import Context
def init(context: Context):
init_mx(context)
init_tg(context)
mautrix_telegram/formatter/from_matrix.py
+130 -31
View File
@@ -3,31 +3,48 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from html import unescape
from html.parser import HTMLParser
from collections import deque
from typing import Optional, List, Tuple, Type, Callable, Dict, Union
import math
import re
import logging
from telethon_aio.tl.types import *
from telethon_aio.tl.types import (MessageEntityMention,
InputMessageEntityMentionName, MessageEntityEmail,
MessageEntityUrl, MessageEntityTextUrl, MessageEntityBold,
MessageEntityItalic, MessageEntityCode, MessageEntityPre,
MessageEntityBotCommand, MessageEntityHashtag,
MessageEntityMentionName, InputUser)
try:
from telethon_aio.tl.types import TypeMessageEntity
except ImportError:
TypeMessageEntity = Union[
MessageEntityMention, MessageEntityHashtag, MessageEntityBotCommand, MessageEntityUrl,
MessageEntityEmail, MessageEntityBold, MessageEntityItalic, MessageEntityCode,
MessageEntityPre, MessageEntityTextUrl, MessageEntityMentionName]
from ..context import Context
from .. import user as u, puppet as pu, portal as po
from ..db import Message as DBMessage
from .util import add_surrogates, remove_surrogates
from .util import (add_surrogates, remove_surrogates, trim_reply_fallback_html,
trim_reply_fallback_text, html_to_unicode)
log = logging.getLogger("mau.fmt.mx")
should_bridge_plaintext_highlights = False
class MatrixParser(HTMLParser):
@@ -35,7 +52,7 @@ class MatrixParser(HTMLParser):
room_regex = re.compile("https://matrix.to/#/(#.+:.+)")
block_tags = ("br", "p", "pre", "blockquote",
"ol", "ul", "li",
"h1", "h2", "h3", "h4", "h5", "h6"
"h1", "h2", "h3", "h4", "h5", "h6",
"div", "hr", "table")
def __init__(self):
@@ -49,21 +66,20 @@ class MatrixParser(HTMLParser):
self._line_is_new = True
self._list_entry_is_new = False
def _parse_url(self, url, args):
def _parse_url(self, url: str, args: Dict[str, str]
) -> Tuple[Optional[Type[TypeMessageEntity]], Optional[str]]:
mention = self.mention_regex.match(url)
if mention:
mxid = mention.group(1)
user = (pu.Puppet.get_by_mxid(mxid, create=False)
user = (pu.Puppet.get_by_mxid(mxid)
or u.User.get_by_mxid(mxid, create=False))
if not user:
return None, None
if user.username:
entity_type = MessageEntityMention
url = f"@{user.username}"
return MessageEntityMention, f"@{user.username}"
else:
entity_type = MessageEntityMentionName
args["user_id"] = user.tgid
return entity_type, url
args["user_id"] = InputUser(user.tgid, 0)
return InputMessageEntityMentionName, user.displayname or None
room = self.room_regex.match(url)
if room:
@@ -80,7 +96,7 @@ class MatrixParser(HTMLParser):
args["url"] = url
return MessageEntityTextUrl, None
def handle_starttag(self, tag, attrs):
def handle_starttag(self, tag: str, attrs: List[Tuple[str, str]]):
self._open_tags.appendleft(tag)
self._open_tags_meta.appendleft(0)
@@ -127,7 +143,7 @@ class MatrixParser(HTMLParser):
self._building_entities[tag] = entity_type(offset=offset, length=0, **args)
@property
def _list_indent(self):
def _list_indent(self) -> int:
indent = 0
first_skipped = False
for index, tag in enumerate(self._open_tags):
@@ -143,24 +159,41 @@ class MatrixParser(HTMLParser):
indent += 3
return indent
def _newline(self, allow_multi=False):
if self._line_is_new or allow_multi:
def _newline(self, allow_multi: bool = False):
if self._line_is_new and not allow_multi:
return
self.text += "\n"
self._line_is_new = True
for entity in self._building_entities.values():
entity.length += 1
def handle_data(self, text):
text = unescape(text)
def _handle_special_previous_tags(self, text: str) -> str:
if "pre" not in self._open_tags and "code" not in self._open_tags:
text = text.replace("\n", "")
else:
text = text.strip()
previous_tag = self._open_tags[0] if len(self._open_tags) > 0 else ""
extra_offset = 0
if previous_tag == "a":
url = self._open_tags_meta[0]
if url:
text = url
elif previous_tag == "command":
text = f"/{text}"
return text
def _html_to_unicode(self, text: str) -> str:
strikethrough, underline = "del" in self._open_tags, "u" in self._open_tags
if strikethrough and underline:
text = html_to_unicode(text, "\u0336\u0332")
elif strikethrough:
text = html_to_unicode(text, "\u0336")
elif underline:
text = html_to_unicode(text, "\u0332")
return text
def _handle_tags_for_data(self, text: str) -> Tuple[str, int]:
extra_offset = 0
list_entry_handled_once = False
# In order to maintain order of things like blockquotes in lists or lists in blockquotes,
# we can't just have ifs/elses and we need to actually loop through the open tags in order.
@@ -188,52 +221,77 @@ class MatrixParser(HTMLParser):
text = indent + prefix + text
self._list_entry_is_new = False
list_entry_handled_once = True
return text, extra_offset
def _extend_entities_in_construction(self, text: str, extra_offset: int):
for tag, entity in self._building_entities.items():
entity.length += len(text) - extra_offset
entity.offset += extra_offset
def handle_data(self, text: str):
text = unescape(text)
text = self._handle_special_previous_tags(text)
text = self._html_to_unicode(text)
text, extra_offset = self._handle_tags_for_data(text)
self._extend_entities_in_construction(text, extra_offset)
self._line_is_new = False
self.text += text
def handle_endtag(self, tag):
def handle_endtag(self, tag: str):
try:
self._open_tags.popleft()
self._open_tags_meta.popleft()
except IndexError:
pass
if tag in self.block_tags:
self._newline(allow_multi=tag == "br")
entity = self._building_entities.pop(tag, None)
if entity:
self.entities.append(entity)
if tag in self.block_tags:
self._newline(allow_multi=tag == "br")
command_regex = re.compile("(\s|^)!([A-Za-z0-9@]+)")
plain_mention_regex = None
def matrix_text_to_telegram(text):
text = command_regex.sub(r"\1/\2", text)
return text
def plain_mention_to_html(match):
puppet = pu.Puppet.find_by_displayname(match.group(2))
if puppet:
return (f"{match.group(1)}"
f"<a href='https://matrix.to/#/{puppet.mxid}'>"
f"{puppet.displayname}"
"</a>")
return "".join(match.groups())
def matrix_to_telegram(html):
def matrix_to_telegram(html: str) -> Tuple[str, List[TypeMessageEntity]]:
try:
parser = MatrixParser()
html = html.replace("\n", "")
html = command_regex.sub(r"\1<command>\2</command>", html)
if should_bridge_plaintext_highlights:
html = plain_mention_regex.sub(plain_mention_to_html, html)
parser.feed(add_surrogates(html))
return remove_surrogates(parser.text.strip()), parser.entities
except Exception:
log.exception("Failed to convert Matrix format:\nhtml=%s", html)
def matrix_reply_to_telegram(content, tg_space, room_id=None):
def matrix_reply_to_telegram(content: dict, tg_space: int, room_id: Optional[str] = None
) -> Optional[int]:
try:
reply = content["m.relates_to"]["m.in_reply_to"]
room_id = room_id or reply["room_id"]
event_id = reply["event_id"]
try:
if content["format"] == "org.matrix.custom.html":
content["formatted_body"] = trim_reply_fallback_html(content["formatted_body"])
except KeyError:
pass
content["body"] = trim_reply_fallback_text(content["body"])
message = DBMessage.query.filter(DBMessage.mxid == event_id,
DBMessage.tg_space == tg_space,
DBMessage.mx_room == room_id).one_or_none()
@@ -242,3 +300,44 @@ def matrix_reply_to_telegram(content, tg_space, room_id=None):
except KeyError:
pass
return None
def matrix_text_to_telegram(text: str) -> Tuple[str, List[TypeMessageEntity]]:
text = command_regex.sub(r"\1/\2", text)
if should_bridge_plaintext_highlights:
entities, pmr_replacer = plain_mention_to_text()
text = plain_mention_regex.sub(pmr_replacer, text)
else:
entities = []
return text, entities
def plain_mention_to_text() -> Tuple[List[TypeMessageEntity], Callable[[str], str]]:
entities = []
def replacer(match):
puppet = pu.Puppet.find_by_displayname(match.group(2))
if puppet:
offset = match.start()
length = match.end() - offset
if puppet.username:
entity = MessageEntityMention(offset, length)
text = f"@{puppet.username}"
else:
entity = InputMessageEntityMentionName(offset, length,
user_id=InputUser(puppet.tgid, 0))
text = puppet.displayname
entities.append(entity)
return text
return "".join(match.groups())
return entities, replacer
def init_mx(context: Context):
global plain_mention_regex, should_bridge_plaintext_highlights
config = context.config
dn_template = config.get("bridge.displayname_template", "{displayname} (Telegram)")
dn_template = re.escape(dn_template).replace(re.escape("{displayname}"), "[^>]+")
plain_mention_regex = re.compile(f"(\s|^)({dn_template})")
should_bridge_plaintext_highlights = config["bridge.plaintext_highlights"] or False
mautrix_telegram/formatter/from_telegram.py
+115 -48
View File
@@ -3,31 +3,55 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from html import escape
import logging
from typing import Optional, List, Tuple, Union
try:
from lxml.html.diff import htmldiff
except ImportError:
htmldiff = None
import logging
import re
from telethon_aio.tl.types import (MessageEntityMention, MessageEntityMentionName,
MessageEntityEmail, MessageEntityUrl, MessageEntityTextUrl,
MessageEntityBold, MessageEntityItalic, MessageEntityCode,
MessageEntityPre, MessageEntityBotCommand, Message, PeerChannel,
MessageEntityHashtag)
try:
from telethon_aio.tl.types import TypeMessageEntity
except ImportError:
TypeMessageEntity = Union[
MessageEntityMention, MessageEntityHashtag, MessageEntityBotCommand, MessageEntityUrl,
MessageEntityEmail, MessageEntityBold, MessageEntityItalic, MessageEntityCode,
MessageEntityPre, MessageEntityTextUrl, MessageEntityMentionName]
from telethon_aio.tl.types import *
from mautrix_appservice import MatrixRequestError
from mautrix_appservice.intent_api import IntentAPI
from .. import user as u, puppet as pu, portal as po
from ..context import Context
from ..db import Message as DBMessage
from .util import add_surrogates, remove_surrogates
from .util import (add_surrogates, remove_surrogates, trim_reply_fallback_html,
trim_reply_fallback_text, unicode_to_html)
log = logging.getLogger("mau.fmt.tg")
should_highlight_edits = False
def telegram_reply_to_matrix(evt, source):
def telegram_reply_to_matrix(evt: Message, source: u.User) -> dict:
if evt.reply_to_msg_id:
space = (evt.to_id.channel_id
if isinstance(evt, Message) and isinstance(evt.to_id, PeerChannel)
@@ -43,7 +67,8 @@ def telegram_reply_to_matrix(evt, source):
return {}
async def _add_forward_header(source, text, html, fwd_from_id):
async def _add_forward_header(source, text: str, html: Optional[str],
fwd_from_id: Optional[int]) -> Tuple[str, str]:
if not html:
html = escape(text)
user = u.User.get_by_tgid(fwd_from_id)
@@ -67,8 +92,23 @@ async def _add_forward_header(source, text, html, fwd_from_id):
return text, html
async def _add_reply_header(source, text, html, evt, relates_to,
native_replies, message_link_in_reply, main_intent, reply_text):
def highlight_edits(new_html: str, old_html: str) -> str:
# Don't include `Edit:` text in diff.
if old_html.startswith("<u>Edit:</u> "):
old_html = old_html[len("<u>Edit:</u> "):]
# Generate diff with lxml
new_html = htmldiff(old_html, new_html)
# Replace <ins> with <u> since Riot doesn't allow <ins>
new_html = new_html.replace("<ins>", "<u>").replace("</ins>", "</u>")
# Remove <del>s since we just want to hide deletions.
new_html = re.sub("<del>.+?</del>", "", new_html)
return new_html
async def _add_reply_header(source: u.User, text: str, html: str, evt: Message, relates_to: dict,
main_intent: IntentAPI, is_edit: bool) -> Tuple[str, str]:
space = (evt.to_id.channel_id
if isinstance(evt, Message) and isinstance(evt.to_id, PeerChannel)
else source.tgid)
@@ -77,52 +117,71 @@ async def _add_reply_header(source, text, html, evt, relates_to,
if not msg:
return text, html
if native_replies:
relates_to["m.in_reply_to"] = {
"event_id": msg.mxid,
"room_id": msg.mx_room,
}
if reply_text == "Edit":
html = f"<u>Edit:</u> {html or escape(text)}"
text = f"Edit: {text}"
return text, html
relates_to["m.in_reply_to"] = {
"event_id": msg.mxid,
"room_id": msg.mx_room,
}
reply_displayname = "unknown user"
try:
event = await main_intent.get_event(msg.mx_room, msg.mxid)
content = event["content"]
body = (content["formatted_body"]
if "formatted_body" in content
else content["body"])
sender = event['sender']
puppet = pu.Puppet.get_by_mxid(sender, create=False)
reply_displayname = puppet.displayname if puppet else sender
reply_to_user = f"<a href='https://matrix.to/#/{sender}'>{reply_displayname}</a>"
reply_to_msg = (("<a href='https://matrix.to/#/"
f"{msg.mx_room}/{msg.mxid}'>{reply_text}</a>")
if message_link_in_reply else "Reply")
quote = f"{reply_to_msg} to {reply_to_user}<blockquote>{body}</blockquote>"
r_sender = event["sender"]
r_text_body = trim_reply_fallback_text(content["body"])
r_html_body = trim_reply_fallback_html(content["formatted_body"]
if "formatted_body" in content
else escape(content["body"]))
puppet = pu.Puppet.get_by_mxid(r_sender, create=False)
r_displayname = puppet.displayname if puppet else r_sender
r_sender_link = f"<a href='https://matrix.to/#/{r_sender}'>{r_displayname}</a>"
if is_edit and should_highlight_edits:
html = highlight_edits(html or escape(text), r_html_body)
except (ValueError, KeyError, MatrixRequestError):
quote = f"{reply_text} to unknown user <em>(Failed to fetch message)</em>:<br/>"
if not html:
html = escape(text)
html = quote + html
text = f"{reply_text} to {reply_displayname}:\n{text}"
return text, html
r_sender_link = "unknown user"
# r_sender = "unknown user"
r_text_body = "Failed to fetch message"
r_html_body = "<em>Failed to fetch message</em>"
if is_edit:
html = f"<u>Edit:</u> {html or escape(text)}"
text = f"Edit: {text}"
r_keyword = "In reply to" if not is_edit else "Edit to"
r_msg_link = f"<a href='https://matrix.to/#/{msg.mx_room}/{msg.mxid}'>{r_keyword}</a>"
html = (f"<blockquote data-mx-reply>{r_msg_link} {r_sender_link}\n{r_html_body}</blockquote>"
+ (html or escape(text)))
lines = r_text_body.strip().split("\n")
text_with_quote = f"> <{r_displayname}> {lines.pop(0)}"
for line in lines:
if line:
text_with_quote += f"\n> {line}"
text_with_quote += "\n\n"
text_with_quote += text
return text_with_quote, html
async def telegram_to_matrix(evt, source, native_replies=False, message_link_in_reply=False,
main_intent=None, reply_text="Reply"):
async def telegram_to_matrix(evt: Message, source: u.User, main_intent: Optional[IntentAPI] = None,
is_edit: bool = False, prefix_text: Optional[str] = None,
prefix_html: Optional[str] = None) -> Tuple[str, str, dict]:
text = add_surrogates(evt.message)
html = _telegram_entities_to_matrix_catch(text, evt.entities) if evt.entities else None
relates_to = {}
if prefix_html:
html = prefix_html + (html or escape(text))
if prefix_text:
text = prefix_text + text
if evt.fwd_from:
text, html = await _add_forward_header(source, text, html, evt.fwd_from.from_id)
if evt.reply_to_msg_id:
text, html = await _add_reply_header(source, text, html, evt, relates_to, native_replies,
message_link_in_reply, main_intent, reply_text)
text, html = await _add_reply_header(source, text, html, evt, relates_to, main_intent,
is_edit)
if isinstance(evt, Message) and evt.post and evt.post_author:
if not html:
@@ -130,13 +189,16 @@ async def telegram_to_matrix(evt, source, native_replies=False, message_link_in_
text += f"\n- {evt.post_author}"
html += f"<br/><i>- <u>{evt.post_author}</u></i>"
html = unicode_to_html(text, html, "\u0336", "del")
html = unicode_to_html(text, html, "\u0332", "u")
if html:
html = html.replace("\n", "<br/>")
return remove_surrogates(text), remove_surrogates(html), relates_to
def _telegram_entities_to_matrix_catch(text, entities):
def _telegram_entities_to_matrix_catch(text: str, entities: List[TypeMessageEntity]) -> str:
try:
return _telegram_entities_to_matrix(text, entities)
except Exception:
@@ -146,7 +208,7 @@ def _telegram_entities_to_matrix_catch(text, entities):
text, entities)
def _telegram_entities_to_matrix(text, entities):
def _telegram_entities_to_matrix(text: str, entities: List[TypeMessageEntity]) -> str:
if not entities:
return text
html = []
@@ -190,7 +252,7 @@ def _telegram_entities_to_matrix(text, entities):
return "".join(html)
def _parse_pre(html, entity_text, language):
def _parse_pre(html: List[str], entity_text: str, language: str) -> bool:
if language:
html.append("<pre>"
f"<code class='language-{language}'>{entity_text}</code>"
@@ -200,7 +262,7 @@ def _parse_pre(html, entity_text, language):
return False
def _parse_mention(html, entity_text):
def _parse_mention(html: List[str], entity_text: str) -> bool:
username = entity_text[1:]
user = u.User.find_by_username(username) or pu.Puppet.find_by_username(username)
@@ -217,7 +279,7 @@ def _parse_mention(html, entity_text):
return False
def _parse_name_mention(html, entity_text, user_id):
def _parse_name_mention(html: List[str], entity_text: str, user_id: int) -> bool:
user = u.User.get_by_tgid(user_id)
if user:
mxid = user.mxid
@@ -231,9 +293,14 @@ def _parse_name_mention(html, entity_text, user_id):
return False
def _parse_url(html, entity_text, url):
def _parse_url(html: List[str], entity_text: str, url: str) -> bool:
url = escape(url) if url else entity_text
if not url.startswith(("https://", "http://", "ftp://", "magnet://")):
url = "http://" + url
html.append(f"<a href='{url}'>{entity_text}</a>")
return False
def init_tg(context: Context):
global should_highlight_edits
should_highlight_edits = htmldiff and context.config["bridge.highlight_edits"]
mautrix_telegram/formatter/util.py
+72 -4
View File
@@ -1,16 +1,84 @@
# Unicode surrogate handling
# From https://github.com/LonamiWebs/Telethon/blob/master/telethon/extensions/markdown.py
# -*- coding: future_fstrings -*-
# mautrix-telegram - A Matrix-Telegram puppeting bridge
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from html import escape
from typing import Optional
import struct
import re
def add_surrogates(text):
# add_surrogates and remove_surrogates are unicode surrogate utility functions from Telethon.
# Licensed under the MIT license.
# https://github.com/LonamiWebs/Telethon/blob/master/telethon/extensions/markdown.py
def add_surrogates(text: Optional[str]) -> Optional[str]:
if text is None:
return None
return "".join("".join(chr(y) for y in struct.unpack("<HH", x.encode("utf-16-le")))
if (0x10000 <= ord(x) <= 0x10FFFF) else x for x in text)
def remove_surrogates(text):
def remove_surrogates(text: Optional[str]) -> Optional[str]:
if text is None:
return None
return text.encode("utf-16", "surrogatepass").decode("utf-16")
def trim_reply_fallback_text(text: str) -> str:
if not text.startswith("> ") or "\n" not in text:
return text
lines = text.split("\n")
while len(lines) > 0 and lines[0].startswith("> "):
lines.pop(0)
return "\n".join(lines)
html_reply_fallback_regex = re.compile(r"^<blockquote data-mx-reply>[\s\S]+?</blockquote>")
def trim_reply_fallback_html(html: str) -> str:
return html_reply_fallback_regex.sub("", html)
def unicode_to_html(text: str, html: str, ctrl: str, tag: str) -> str:
if ctrl not in text:
return html
if not html:
html = escape(text)
tag_start = f"<{tag}>"
tag_end = f"</{tag}>"
characters = html.split(ctrl)
html = ""
in_tag = False
for char in characters:
if not in_tag:
if len(char) > 1:
html += char[0:-1]
char = char[-1]
html += tag_start
in_tag = True
html += char
else:
if len(char) > 1:
html += tag_end
in_tag = False
html += char
if in_tag:
html += tag_end
return html
def html_to_unicode(text: str, ctrl: str) -> str:
return ctrl.join(text) + ctrl
mautrix_telegram/matrix.py
+23 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
import logging
from mautrix_appservice import MatrixRequestError
@@ -222,6 +222,18 @@ class MatrixHandler:
return
await handler(sender, content[content_key])
async def handle_room_pin(self, room, sender, new_events, old_events):
portal = Portal.get_by_mxid(room)
sender = await User.get_by_mxid(sender).ensure_started()
if sender.has_full_access and portal:
events = new_events - old_events
if len(events) > 0:
# New event pinned, set that as pinned in Telegram.
await portal.handle_matrix_pin(sender, events.pop())
elif len(new_events) == 0:
# All pinned events removed, remove pinned event in Telegram.
await portal.handle_matrix_pin(sender, None)
def filter_matrix_event(self, event):
return (event["sender"] == self.az.bot_mxid
or Puppet.get_id_from_mxid(event["sender"]) is not None)
@@ -250,3 +262,10 @@ class MatrixHandler:
evt["prev_content"])
elif type in ("m.room.name", "m.room.avatar", "m.room.topic"):
await self.handle_room_meta(type, evt["room_id"], evt["sender"], evt["content"])
elif type == "m.room.pinned_events":
new_events = set(evt["content"]["pinned"])
try:
old_events = set(evt["unsigned"]["prev_content"]["pinned"])
except KeyError:
old_events = set()
await self.handle_room_pin(evt["room_id"], evt["sender"], new_events, old_events)
mautrix_telegram/portal.py
+147 -48
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from collections import deque
from datetime import datetime
import asyncio
@@ -66,6 +66,8 @@ class Portal:
self._main_intent = None
self._room_create_lock = asyncio.Lock()
self._temp_pinned_message_id = None
self._temp_pinned_message_sender = None
self._dedup = deque()
self._dedup_mxid = {}
@@ -516,12 +518,10 @@ class Portal:
try:
current_extension = body[body.rindex("."):]
if mimetypes.types_map[current_extension] == mime:
file_name = body
else:
file_name = f"matrix_upload{mimetypes.guess_extension(mime)}"
return body
except (ValueError, KeyError):
file_name = f"matrix_upload{mimetypes.guess_extension(mime)}"
return file_name, None if file_name == body else body
pass
return f"matrix_upload{mimetypes.guess_extension(mime)}"
async def leave_matrix(self, user, source, event_id):
if not user.logged_in:
@@ -570,25 +570,54 @@ class Portal:
@staticmethod
def _preprocess_matrix_message(sender, message):
if message["msgtype"] == "m.emote":
msgtype = message["msgtype"]
if msgtype == "m.emote":
if "formatted_body" in message:
message["formatted_body"] = f"* {sender.displayname} {message['formatted_body']}"
message["body"] = f"* {sender.displayname} {message['body']}"
message["msgtype"] = "m.text"
elif not sender.logged_in:
if "formatted_body" in message:
message["formatted_body"] = (f"&lt;{sender.displayname}&gt; "
f"{message['formatted_body']}")
message["body"] = f"<{sender.displayname}> {message['body']}"
return type
html = message["formatted_body"] if "formatted_body" in message else None
text = message["body"]
if msgtype == "m.text":
if html:
html = f"&lt;{sender.displayname}&gt; {html}"
text = f"<{sender.displayname}> {text}"
else:
msgtype = msgtype[len("m."):]
prefix = {
"file": "a ",
"image": "an ",
"audio": "",
"video": "a ",
"location": "a ",
}.get(msgtype, "")
if html:
html = f"{sender.displayname} sent {prefix}{msgtype}: {html}"
text = ": " + text if text else ""
text = f"{sender.displayname} sent {prefix}{msgtype}{text}"
if html:
message["formatted_body"] = html
message["body"] = text
def _handle_matrix_text(self, client, message, reply_to):
if "format" in message and message["format"] == "org.matrix.custom.html":
message, entities = formatter.matrix_to_telegram(message["formatted_body"])
return client.send_message(self.peer, message, entities=entities, reply_to=reply_to)
else:
message = formatter.matrix_text_to_telegram(message["body"])
return client.send_message(self.peer, message, reply_to=reply_to)
async def _matrix_event_to_entities(self, client, event):
try:
if event.get("format", None) == "org.matrix.custom.html":
message, entities = formatter.matrix_to_telegram(event["formatted_body"])
# TODO remove this crap
for entity in entities:
if isinstance(entity, InputMessageEntityMentionName):
entity.user_id = await client.get_input_entity(entity.user_id.user_id)
else:
message, entities = formatter.matrix_text_to_telegram(event["body"])
except KeyError:
message, entities = None, None
return message, entities
async def _handle_matrix_text(self, client, message, reply_to):
message, entities = await self._matrix_event_to_entities(client, message)
return await client.send_message(self.peer, message, entities=entities, reply_to=reply_to)
async def _handle_matrix_file(self, client, message, reply_to):
file = await self.main_intent.download_file(message["url"])
@@ -596,32 +625,53 @@ class Portal:
info = message["info"]
mime = info["mimetype"]
file_name, caption = self._get_file_meta(message["body"], mime)
file_name = self._get_file_meta(message["mxtg_filename"], mime)
attributes = [DocumentAttributeFilename(file_name=file_name)]
if "w" in info and "h" in info:
attributes.append(DocumentAttributeImageSize(w=info["w"], h=info["h"]))
caption = message["body"] if message["body"] != file_name else None
return await client.send_file(self.peer, file, mime, caption=caption,
attributes=attributes, file_name=file_name,
reply_to=reply_to)
async def _handle_matrix_location(self, client, message, reply_to):
try:
lat, long = message["geo_uri"][len("geo:"):].split(",")
lat, long = float(lat), float(long)
except (KeyError, ValueError):
self.log.exception("Failed to parse location")
return None
message, entities = await self._matrix_event_to_entities(client, message)
media = MessageMediaGeo(geo=GeoPoint(lat, long))
return await client.send_media(self.peer, media, reply_to=reply_to, caption=message,
entities=entities)
async def handle_matrix_message(self, sender, message, event_id):
client = sender.client if sender.logged_in else self.bot.client
space = (self.tgid if self.peer_type == "channel" # Channels have their own ID space
else (sender.tgid if sender.logged_in else self.bot.tgid))
reply_to = formatter.matrix_reply_to_telegram(message, space, room_id=self.mxid)
message["mxtg_filename"] = message["body"]
self._preprocess_matrix_message(sender, message)
type = message["msgtype"]
if type == "m.text" or (self.bridge_notices and type == "m.notice"):
response = await self._handle_matrix_text(client, message, reply_to)
elif type == "m.location":
response = await self._handle_matrix_location(client, message, reply_to)
elif type in ("m.image", "m.file", "m.audio", "m.video"):
response = await self._handle_matrix_file(client, message, reply_to)
else:
self.log.debug("Unhandled Matrix event: %s", message)
response = None
if not response:
return
self.log.debug("Handled Matrix message: %s", response)
self.is_duplicate(response, (event_id, space))
self.db.add(DBMessage(
tgid=response.id,
@@ -630,6 +680,20 @@ class Portal:
mxid=event_id))
self.db.commit()
async def handle_matrix_pin(self, sender, pinned_message):
if self.peer_type != "channel":
return
try:
if not pinned_message:
await sender.client(UpdatePinnedMessageRequest(channel=self.peer, id=0))
else:
message = DBMessage.query.filter(DBMessage.mxid == pinned_message,
DBMessage.tg_space == self.tgid,
DBMessage.mx_room == self.mxid).one_or_none()
await sender.client(UpdatePinnedMessageRequest(channel=self.peer, id=message.tgid))
except ChatNotModifiedError:
pass
async def handle_matrix_deletion(self, deleter, event_id):
space = self.tgid if self.peer_type == "channel" else deleter.tgid
message = DBMessage.query.filter(DBMessage.mxid == event_id,
@@ -650,7 +714,7 @@ class Portal:
edit_messages=moderator, delete_messages=moderator,
ban_users=moderator, invite_users=moderator,
invite_link=moderator, pin_messages=moderator,
add_admins=admin, manage_call=moderator)
add_admins=admin)
await sender.client(
EditAdminRequest(channel=await self.get_input_entity(sender),
user_id=user_id, admin_rights=rights))
@@ -823,12 +887,19 @@ class Portal:
if self.mxid:
await user.intent.set_typing(self.mxid, is_typing=True)
async def handle_telegram_photo(self, source, intent, evt, relates_to=None):
async def handle_telegram_photo(self, source: u.User, intent, evt: Message, relates_to=None):
largest_size = self._get_largest_photo_size(evt.media.photo)
file = await util.transfer_file_to_matrix(self.db, source.client, intent,
largest_size.location)
if not file:
return None
if config["bridge.inline_images"] and (evt.message or evt.fwd_from or evt.reply_to_msg_id):
text, html, relates_to = await formatter.telegram_to_matrix(
evt, source, self.main_intent,
prefix_html=f"<img src='{file.mxc}' alt='Inline Telegram photo'/><br/>",
prefix_text="Inline image: ")
await intent.set_typing(self.mxid, is_typing=False)
return await intent.send_text(self.mxid, text, html=html, relates_to=relates_to)
info = {
"h": largest_size.h,
"w": largest_size.w,
@@ -842,25 +913,40 @@ class Portal:
return await intent.send_image(self.mxid, file.mxc, info=info, text=name,
relates_to=relates_to)
async def handle_telegram_document(self, source, intent, evt, relates_to=None):
async def handle_telegram_document(self, source, intent, evt: Message, relates_to=None):
document = evt.media.document
file = await util.transfer_file_to_matrix(self.db, source.client, intent, document)
file = await util.transfer_file_to_matrix(self.db, source.client, intent, document,
document.thumb)
if not file:
return None
name = evt.message
width, height = file.width, file.height
for attr in document.attributes:
if not name and isinstance(attr, DocumentAttributeFilename):
name = attr.file_name
if isinstance(attr, DocumentAttributeFilename):
name = name or attr.file_name
if not file.was_converted:
(mime_from_name, _) = mimetypes.guess_type(name)
file.mime_type = mime_from_name or file.mime_type
elif isinstance(attr, DocumentAttributeSticker):
name = f"Sticker for {attr.alt}"
elif isinstance(attr, DocumentAttributeVideo) and (not width or not height):
width, height = attr.w, attr.h
mime_type = document.mime_type or file.mime_type
info = {
"size": document.size,
"size": file.size,
"mimetype": mime_type,
}
if file.thumbnail:
info["thumbnail_url"] = file.thumbnail.mxc
info["thumbnail_info"] = {
"mimetype": file.thumbnail.mime_type,
"h": file.thumbnail.height or document.thumb.h,
"w": file.thumbnail.width or document.thumb.w,
"size": file.thumbnail.size,
}
if height and width:
info["h"] = height
info["w"] = width
type = "m.file"
if mime_type.startswith("video/"):
type = "m.video"
@@ -900,11 +986,7 @@ class Portal:
async def handle_telegram_text(self, source, intent, evt):
self.log.debug(f"Sending {evt.message} to {self.mxid} by {intent.mxid}")
text, html, relates_to = await formatter.telegram_to_matrix(
evt, source,
config["bridge.native_replies"],
config["bridge.link_in_reply"],
self.main_intent)
text, html, relates_to = await formatter.telegram_to_matrix(evt, source, self.main_intent)
await intent.set_typing(self.mxid, is_typing=False)
return await intent.send_text(self.mxid, text, html=html, relates_to=relates_to)
@@ -928,11 +1010,8 @@ class Portal:
return
evt.reply_to_msg_id = evt.id
text, html, relates_to = await formatter.telegram_to_matrix(
evt, source,
config["bridge.native_replies"],
config["bridge.link_in_reply"],
self.main_intent, reply_text="Edit")
text, html, relates_to = await formatter.telegram_to_matrix(evt, source, self.main_intent,
is_edit=True)
intent = sender.intent if sender else self.main_intent
await intent.set_typing(self.mxid, is_typing=False)
response = await intent.send_text(self.mxid, text, html=html, relates_to=relates_to)
@@ -966,7 +1045,9 @@ class Portal:
DBMessage(tgid=evt.id, mx_room=self.mxid, mxid=mxid, tg_space=tg_space))
self.db.commit()
return
media = evt.media if hasattr(evt, "media") else None
allowed_media = (MessageMediaPhoto, MessageMediaDocument, MessageMediaGeo)
media = evt.media if hasattr(evt, "media") and isinstance(evt.media,
allowed_media) else None
intent = sender.intent if sender else self.main_intent
if not media and evt.message:
response = await self.handle_telegram_text(source, intent, evt)
@@ -988,7 +1069,7 @@ class Portal:
if not response:
return
self.log.debug("Handled Telegram message: %s", evt)
mxid = response["event_id"]
DBMessage.query \
.filter(DBMessage.mx_room == self.mxid,
@@ -1013,7 +1094,6 @@ class Portal:
or self.is_duplicate_action(update))
if should_ignore:
return
# TODO figure out how to see changes to about text / channel username
if isinstance(action, MessageActionChatEditTitle):
await self.update_title(action.title, save=True)
@@ -1031,6 +1111,8 @@ class Portal:
self.peer_type = "channel"
self.migrate_and_save(action.channel_id)
await sender.intent.send_emote(self.mxid, "upgraded this group to a supergroup.")
elif isinstance(action, MessageActionPinMessage):
await self.receive_telegram_pin_sender(sender)
else:
self.log.debug("Unhandled Telegram action in %s: %s", self.title, action)
@@ -1045,13 +1127,30 @@ class Portal:
levels["users"][puppet.mxid] = 50
await self.main_intent.set_power_levels(self.mxid, levels)
async def update_telegram_pin(self, source, id):
space = self.tgid if self.peer_type == "channel" else source.tgid
message = DBMessage.query.get((id, space))
async def receive_telegram_pin_sender(self, sender):
self._temp_pinned_message_sender = sender
if self._temp_pinned_message_id:
await self.update_telegram_pin()
async def update_telegram_pin(self):
intent = (self._temp_pinned_message_sender.intent
if self._temp_pinned_message_sender else self.main_intent)
id = self._temp_pinned_message_id
self._temp_pinned_message_id = None
self._temp_pinned_message_sender = None
message = DBMessage.query.get((id, self.tgid))
if message:
await self.main_intent.set_pinned_messages(self.mxid, [message.mxid])
await intent.set_pinned_messages(self.mxid, [message.mxid])
else:
await self.main_intent.set_pinned_messages(self.mxid, [])
await intent.set_pinned_messages(self.mxid, [])
async def receive_telegram_pin_id(self, id):
if id == 0:
return await self.update_telegram_pin()
self._temp_pinned_message_id = id
if self._temp_pinned_message_sender:
await self.update_telegram_pin()
@staticmethod
def _get_level_from_participant(participant, _):
mautrix_telegram/public/__init__.py
+10 -7
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from aiohttp import web
from mako.template import Template
import asyncio
@@ -46,7 +46,9 @@ class PublicBridgeWebsite:
user = (User.get_by_mxid(request.rel_url.query["mxid"], create=False)
if "mxid" in request.rel_url.query else None)
if not user:
return self.render_login(mxid=request.rel_url.query["mxid"], state="request")
return self.render_login(
mxid=request.rel_url.query["mxid"] if "mxid" in request.rel_url.query else None,
state="request")
elif not user.whitelisted:
return self.render_login(mxid=user.mxid, error="You are not whitelisted.", status=403)
await user.ensure_started()
@@ -144,7 +146,7 @@ class PublicBridgeWebsite:
if "mxid" not in data:
return self.render_login(error="Please enter your Matrix ID.", status=400)
user = await User.get_by_mxid(data["mxid"]).ensure_started()
user = await User.get_by_mxid(data["mxid"]).ensure_started(even_if_no_session=True)
if not user.whitelisted:
return self.render_login(mxid=user.mxid, error="You are not whitelisted.", status=403)
elif user.logged_in:
@@ -153,7 +155,8 @@ class PublicBridgeWebsite:
if "phone" in data:
return await self.post_login_phone(user, data["phone"])
elif "code" in data:
resp = await self.post_login_code(user, data["code"], password_in_data="password" in data)
resp = await self.post_login_code(user, data["code"],
password_in_data="password" in data)
if resp or "password" not in data:
return resp
elif "password" not in data:
mautrix_telegram/public/login.css
+17
View File
@@ -1,3 +1,20 @@
/*
* mautrix-telegram - A Matrix-Telegram puppeting bridge
* Copyright (C) 2018 Tulir Asokan
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
form > div {
display: none;
}
mautrix_telegram/public/login.html.mako
+17
View File
@@ -1,3 +1,20 @@
<!--
mautrix-telegram - A Matrix-Telegram puppeting bridge
Copyright (C) 2018 Tulir Asokan
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
-->
<!DOCTYPE html>
<html>
<head>
mautrix_telegram/puppet.py
+19 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from difflib import SequenceMatcher
import re
import logging
@@ -191,6 +191,21 @@ class Puppet:
return None
@classmethod
def find_by_displayname(cls, displayname):
if not displayname:
return None
for _, puppet in cls.cache.items():
if puppet.displayname and puppet.displayname == displayname:
return puppet
puppet = DBPuppet.query.filter(DBPuppet.displayname == displayname).one_or_none()
if puppet:
return cls.from_db(puppet)
return None
def init(context):
global config
mautrix_telegram/tgclient.py
+9 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from io import BytesIO
from telethon_aio import TelegramClient
@@ -73,6 +73,11 @@ class MautrixTelegramClient(TelegramClient):
reply_to_msg_id=reply_to)
return self._get_response_message(request, await self(request))
async def send_media(self, entity, media, caption=None, entities=None, reply_to=None):
request = SendMediaRequest(entity, media, message=caption or "", entities=entities or [],
reply_to_msg_id=reply_to)
return self._get_response_message(request, await self(request))
async def download_file_bytes(self, location):
if isinstance(location, Document):
location = InputDocumentFileLocation(location.id, location.access_hash,
mautrix_telegram/user.py
+4 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
import logging
import asyncio
import re
mautrix_telegram/util/file_transfer.py
+105 -15
View File
@@ -3,27 +3,39 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
from io import BytesIO
import time
import logging
import magic
from PIL import Image
from sqlalchemy.exc import IntegrityError
from sqlalchemy.exc import IntegrityError, InvalidRequestError
from sqlalchemy.orm.exc import FlushError
try:
from PIL import Image
except ImportError:
Image = None
try:
from moviepy.editor import VideoFileClip
import random
import string
import os
import mimetypes
except ImportError:
VideoFileClip = random = string = os = mimetypes = None
from telethon_aio.tl.types import (Document, FileLocation, InputFileLocation,
InputDocumentFileLocation)
InputDocumentFileLocation, PhotoSize, PhotoCachedSize)
from telethon_aio.errors import LocationInvalidError
from ..db import TelegramFile as DBTelegramFile
@@ -32,24 +44,89 @@ log = logging.getLogger("mau.util")
def _convert_webp(file, to="png"):
if not Image:
return "image/webp", file
try:
image = Image.open(BytesIO(file)).convert("RGBA")
new_file = BytesIO()
image.save(new_file, to)
return f"image/{to}", new_file.getvalue()
w, h = image.size
return f"image/{to}", new_file.getvalue(), w, h
except Exception:
log.exception(f"Failed to convert webp to {to}")
return "image/webp", file
async def transfer_file_to_matrix(db, client, intent, location):
def _temp_file_name(ext):
return ("/tmp/mxtg-video-"
+ "".join(random.choice(string.ascii_uppercase + string.digits) for _ in range(10))
+ ext)
def _read_video_thumbnail(data, video_ext="mp4", frame_ext="png", max_size=(1024, 720)):
# We don't have any way to read the video from memory, so save it to disk.
temp_file = _temp_file_name(video_ext)
with open(temp_file, "wb") as file:
file.write(data)
# Read temp file and get frame
clip = VideoFileClip(temp_file)
frame = clip.get_frame(0)
# Convert to png and save to BytesIO
image = Image.fromarray(frame).convert("RGBA")
thumbnail_file = BytesIO()
if max_size:
image.thumbnail(max_size, Image.ANTIALIAS)
image.save(thumbnail_file, frame_ext)
os.remove(temp_file)
w, h = image.size
return thumbnail_file.getvalue(), w, h
def _location_to_id(location):
if isinstance(location, (Document, InputDocumentFileLocation)):
id = f"{location.id}-{location.version}"
return f"{location.id}-{location.version}"
elif isinstance(location, (FileLocation, InputFileLocation)):
id = f"{location.volume_id}-{location.local_id}"
return f"{location.volume_id}-{location.local_id}"
else:
return None
async def transfer_thumbnail_to_matrix(client, intent, thumbnail_loc, video, mime):
if not Image or not VideoFileClip:
return None
id = _location_to_id(thumbnail_loc)
if not id:
return None
video_ext = mimetypes.guess_extension(mime)
if VideoFileClip and video_ext:
try:
file, width, height = _read_video_thumbnail(video, video_ext, frame_ext="png")
except OSError:
return None
mime_type = "image/png"
else:
file = await client.download_file_bytes(thumbnail_loc)
width, height = None, None
mime_type = magic.from_buffer(file, mime=True)
uploaded = await intent.upload_file(file, mime_type)
return DBTelegramFile(id=id, mxc=uploaded["content_uri"], mime_type=mime_type,
was_converted=False, timestamp=int(time.time()), size=len(file),
width=width, height=height)
async def transfer_file_to_matrix(db, client, intent, location, thumbnail=None):
id = _location_to_id(location)
if not id:
return None
db_file = DBTelegramFile.query.get(id)
if db_file:
return db_file
@@ -58,24 +135,37 @@ async def transfer_file_to_matrix(db, client, intent, location):
file = await client.download_file_bytes(location)
except LocationInvalidError:
return None
width, height = None, None
mime_type = magic.from_buffer(file, mime=True)
image_converted = False
if mime_type == "image/webp":
mime_type, file = _convert_webp(file, to="png")
mime_type, file, width, height = _convert_webp(file, to="png")
thumbnail = None
image_converted = True
uploaded = await intent.upload_file(file, mime_type)
db_file = DBTelegramFile(id=id, mxc=uploaded["content_uri"],
mime_type=mime_type, was_converted=image_converted,
timestamp=int(time.time()))
timestamp=int(time.time()), size=len(file),
width=width, height=height)
if thumbnail and (mime_type.startswith("video/") or mime_type == "image/gif"):
if isinstance(thumbnail, (PhotoSize, PhotoCachedSize)):
thumbnail = thumbnail.location
db_file.thumbnail = await transfer_thumbnail_to_matrix(client, intent, thumbnail, file,
mime_type)
try:
db.add(db_file)
db.commit()
except IntegrityError:
except FlushError as e:
log.exception(f"{e.__class__.__name__} while saving transferred file data. "
"This was probably caused by two simultaneous transfers of the same file, "
"and should not cause any problems.")
except (IntegrityError, InvalidRequestError) as e:
db.rollback()
log.exception("Integrity error while saving transferred file data. "
log.exception(f"{e.__class__.__name__} while saving transferred file data. "
"This was probably caused by two simultaneous transfers of the same file, "
"and should not cause any problems.")
mautrix_telegram/util/format_duration.py
+4 -4
View File
@@ -3,17 +3,17 @@
# Copyright (C) 2018 Tulir Asokan
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
def format_duration(seconds):
requirements/base.txt
+1 -2
View File
@@ -1,9 +1,8 @@
aiohttp
mautrix-appservice
ruamel.yaml
python-magic
SQLAlchemy
alembic
Markdown
Pillow
future-fstrings
cryptg
requirements/optional.txt
+4
View File
@@ -0,0 +1,4 @@
lxml
cryptg
Pillow
moviepy
setup.py
+11 -3
View File
@@ -3,6 +3,14 @@ import sys
import glob
import mautrix_telegram
extras = {
"highlight_edits": ["lxml>=4.1.1,<5"],
"fast_crypto": ["cryptg>=0.1,<0.2"],
"webp_convert": ["Pillow>=5.0.0,<6"],
"hq_thumbnails": ["moviepy>=0.2,<0.3"],
}
extras["all"] = [deps[0] for deps in extras.values()]
setuptools.setup(
name="mautrix-telegram",
version=mautrix_telegram.__version__,
@@ -18,22 +26,22 @@ setuptools.setup(
install_requires=[
"aiohttp>=3.0.1,<4",
"mautrix-telegram>=0.1,<0.2",
"SQLAlchemy>=1.2.3,<2",
"alembic>=0.9.8,<0.10",
"Markdown>=2.6.11,<3",
"ruamel.yaml>=0.15.35,<0.16",
"Pillow>=5.0.0,<6",
"future-fstrings>=0.4.2",
"python-magic>=0.4.15,<0.5",
"cryptg>=0.1,<0.2",
"telethon-aio>=0.18,<0.19" if sys.version_info >= (3, 6) else "telethon-aio-git",
],
dependency_links=[
"https://github.com/tulir/telethon-asyncio/tarball/9b389cfb4b6d3876e9661c23507f17e96897e4b0#egg=telethon-aio-git-0.18.0+1"
],
extras_require=extras,
classifiers=[
"Development Status :: 4 Beta",
"Development Status :: 4 - Beta",
"License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
"Topic :: Communications :: Chat",
"Programming Language :: Python",