In March 2024, the Australian internet safety regulator eSafety began an investigation into moderation practices at Telegram. The editors of @tginfo have reviewed the published document, in which the messenger discloses to the regulator some details about how the systems that block illegal content work.
- Moderation in private chats
- Moderation in secret chats
- Moderation in private channels and groups
- Automatic detection of illegal content
- Technical structure of automatic systems
- Processing user reports
- Cooperation with organizations
- Criteria for permanent bans
- Strike system
- Composition of the moderation team
- Response time to reports
- Success rate of ban appeals
- Statistics for automatic detection of illegal content
- Other information about moderation
- About the FAQ update in September 2024
- Other comments from the regulator
- Other facts
Moderation in private chats
- Telegram does not scan private chats and does not detect new or known illegal content in them.
- The messenger can perform an automatic check or forward messages to a moderator only if the recipient submits a report.
Moderation in secret chats
- Messages from secret chats are not forwarded to moderators, even if one of the parties submits a report1.
- Instead of message text, for reports from secret chats Telegram uses «alternative signals» that cannot be disclosed to the public.
Moderation in private channels and groups
- Most private channels and groups are invisible to moderators unless one of the participants submits a report on the community.
- Telegram’s rules prohibit the promotion of violence and the distribution of illegal pornographic content in publicly viewable channels, groups, bots, etc. Despite this, the messenger does not tolerate illegal content in any part of the service.
- Telegram representatives clarify that public in the rules refers to any actively moderated parts of the application.
- The absence of a username does not necessarily make a channel private: For moderators, a channel or group is considered public if it becomes available to the general public through an invitation link. For example, if a link to a private group is posted in a publicly accessible part of Telegram or another social network, then such a private community is considered publicly accessible for content moderation purposes (because moderators can follow the link and view messages inside).
- Photos and videos in all private groups and channels are scanned by the system for automatic detection of known illegal content.
Automatic detection of illegal content
- Telegram can automatically detect illegal content in photos, videos, and text, but it does it only in select cases.
- The messenger does not automatically scan personal correspondence unless one of the participants submits a report.
- 65% of automatically detected potentially extremist content is sent for manual review by moderators.
- When likely new material with child abuse is detected, it is either immediately processed automatically or sent for review by a moderator «depending on the degree of confidence with which the relevant model is able to issue a judgment, combined with other factors».
The messenger can automatically detect both known illegal content and new, potentially illegal content. Different rules apply to them:
Known content (extremism and child abuse)
- Telegram automatically scans for known photos and videos in all parts of the app except private chats. Even private groups and channels are scanned.
- By default, Telegram does not scan the text of messages.
- If a report is submitted on a message or chat, Telegram will scan the text for matches with known illegal materials.
New potentially illegal content (extremism and child abuse)
- Text, photos, and videos in reported items, public groups and chats, stories, avatars, descriptions, and usernames are automatically scanned.
- Private messages, private groups and channels are not scanned to detect new illegal content, except when a report is submitted on them.
- Removed materials are added to the database of known prohibited content.
Technical structure of automatic systems
- Mainly, Telegram uses only its own signature databases for automatic detection of illegal content. The database is collected only from materials identified by the messenger’s moderators manually.
- Telegram also regularly reviews Europol’s signature databases for integration into its systems.
- Models and tools for detecting extremist texts work well enough in most languages.
- Telegram systems track when, why, and how often a user is removed by group administrators, and in some cases may take appropriate action. In addition, these metrics are considered by Telegram’s AI models both for prioritizing reports and for automatic decision-making.
- The messenger uses some signals to identify new groups that are similar to previously blocked groups.
- Link blacklists are not used. Telegram stated that «focusing efforts on machine learning-based classification tends to yield better results compared to static link blacklists».
- Telegram stated that it conducts «extensive monitoring» of media sources and also reviews links sent by «unregistered users and trusted organizations» to Telegram via email.
Among the models used, Telegram named:
- fine-tuned self-supervised multilingual transformer-based language models;
- fine-tuned vision transformer models;
- multilingual transformer-based end-to-end ASR (Automatic Speech Recognition) systems;
- multimodal transformer-based models aligned on image-text datasets;
- multilingual transformer-based large language models; and
- custom data clustering algorithms.
According to Telegram, «several» of these models had been deployed by the end of the reporting period (February 29, 2024), and since then the company has significantly expanded the use of AI and machine learning technologies.
Processing user reports
- The messenger noted that user reports are processed by Telegram’s «tools», moderators, and, if necessary, artificial intelligence/machine learning models.
- 75% of user reports on extremist content are sent for manual review by moderators.
Cooperation with organizations
Telegram regularly interacts with the following organizations:
- UK Home Office
- Etidal
- EU Internet Forum
- Europol
- Ofcom
- UNSC Counter-Terrorism Committee Executive Directorate
Telegram also stated that, as of October 2024, it was «in the process of joining the Internet Watch Foundation’s safety programs, including, among other things, access to URL lists containing links to known CSAM websites».
Criteria for permanent bans
- Disseminating material that calls for violence in the form of text, image, recordings, footage or otherwise. Telegram specified this means material «like concrete and specified threats of physical harm».
- Disseminating material that is gruesome or shockingly graphic. Telegram gave such examples as «graphic details of
torture, accident photos» or material that «glorifies or promotes violent or terrorist ideologies». - Soliciting funds for terrorist organizations or causes.
- Owning or being an administrator of a Community involved in the above activities.
Strike system
If a community or account belonging to a «journalist» or «researcher» publishes extremist content for the purpose of disseminating «legitimate scientific research, historical materials, or news», Telegram may:
- grant an exception,
- or issue up to two warnings before terminating the community or account.
Telegram stated the decision on enforcement depends on the «severity, purpose and relevance of the posted content under applicable law».
Composition of the moderation team
Telegram provided the following data to the regulator2:
| Employee Category | Number of Employees |
|---|---|
| Trust and Safety Engineers | 5 |
| Content Moderators Employed by Telegram | 0 |
| Content Moderators Contracted with Telegram | 150 |
| Telegram Trust and Safety Staff (excluding engineers and content moderators) | 4 |
eSafety asked Telegram if it had a rapid response team:
| Job Title | Number of Employees | Number of Contractors |
|---|---|---|
| Trust and Safety Workers | 3 | 13 |
- All Telegram moderators are contractors, not employees.
- Telegram moderators cover 60 languages.
- eSafety notes that among major languages, Telegram lacks moderators for Vietnamese.
Response time to reports
| Service | Average Response Time |
|---|---|
| Private Chats (extremism) | 18 hours |
| Channels and Chats (extremism) | 15 hours |
| Private Chats (child abuse) | 11 hours |
| Channels and Chats (child abuse) | 10 hours |
| Report from a trusted organization | 1 hour |
Success rate of ban appeals
- 3% success rate for appeals of automatic bans related to the distribution of extremist content.
- 2% success rate for appeals of bans based on user reports of the distribution of extremist content.
- 8% success rate for appeals of automatic bans related to the distribution of content with child abuse.
- 8% success rate for appeals of bans based on user reports of the distribution of content with child abuse.
Statistics for automatic detection of illegal content
Extremist content:
| Feature | Detected Automatically | User Reports |
|---|---|---|
| Private Chats | N/A | 100% |
| Secret Chats | N/A | 100% |
| Groups (public) | 67% | 33% |
| Groups (private) | 82% | 18% |
| Channels (public) | 69% | 31% |
| Channels (private) | 79% | 21% |
| Stories | 60% | 40% |
Child Abuse:
| Feature | Detected Automatically | User Reports |
|---|---|---|
| Private Chats | N/A | 100% |
| Secret Chats | N/A | 100% |
| Groups (public) | 71% | 29% |
| Groups (private) | 85% | 15% |
| Channels (public) | 74% | 26% |
| Channels (private) | 80% | 20% |
| Stories | 65% | 35% |
Other information about moderation
- After deleting a chat or channel, moderators manually check the most popular search queries that users used to find that community. This is done to potentially exclude these queries from Telegram’s public search and thus limit the spread and availability of similar content in the future.
- If the group administrators themselves do not violate Telegram’s rules (for example, if the group was temporarily affected by the actions of attackers), Telegram moderators may, at their discretion, temporarily close it, giving the administrators the opportunity to eliminate the violations.
- However, if a community is used to distribute prohibited content, regardless of whether it is by administrators or participants, such community, its administrators, and users who have violated Telegram’s terms of use may be permanently banned.
- When Telegram identifies a user distributing extremist materials, the messenger checks «further reports linked to this user, as well as to any Communities which the user owns or administrates». Telegram indicated that any communities found to be distributing such materials would also be removed.
- Subscribers of channels or members of groups who «do not engage in administration, direct distribution, or promotion of prohibited content, even if they are members of communities where such content may be present, will not be automatically banned». This approach, according to Telegram, was adopted in order not to cause accidental harm to law enforcement officers, journalists, activists, and other users who may be members of such groups for objective reasons.
- Given the severity of child abuse, any violation related to it usually results in the permanent removal of related accounts and communities. Owners of such groups and channels may also be removed.
Additionally: About the FAQ update in September 2024
In September 2024, Telegram updated a number of documents on the official website regarding moderation and disclosure of data to law enforcement agencies. The editors of @tginfo then noted that there were no significant changes, and the new text merely shifted the emphasis.
The regulator drew attention to the text that was present in the FAQ at the time: «All Telegram chats and group chats are private amongst their participants. We do not process any requests related to them». The messenger reported that this part of the document was outdated and «was the result of statements about Telegram’s stance on copyright infringement having been mistakenly copied to a section dealing with Telegram’s stance on illegal content».
Using archive.org, eSafety examined older versions of the documents and noticed a discrepancy with Telegram’s statement: the fragment existed on the page even before any mention of copyright infringement in Telegram’s FAQs.
It is important to mention that at the end of September 2024, the messenger made additional changes to the documents, which expanded the list of conditions under which the messenger would disclose users’ IP addresses and phone numbers to the authorities.
Other comments from the regulator
- eSafety believes that if report tools are only available in cases where the offender is not in the user’s contacts, this may prevent Telegram from adequately identifying offenders and preventing their further harmful actions on the platform, even if the user has already blocked them.
- eSafety emphasizes that Telegram’s use of only their own signature datasets increases the risk of missing illegal content. As a result, such content may continue to be distributed, even though other online services have already detected it and added digital fingerprints to large shared databases.
- eSafety notes that NCMEC and IWF have stated that prior to the arrest of Telegram’s CEO on August 27, 2024, any attempts to contact Telegram were ignored.
Other facts
- Telegram reported that users from Australia make up 0.2% of the messenger’s monthly active audience.
- Telegram shares the personal data of banned users only with law enforcement agencies.
- The company assesses between 1 and 5% of all reports daily, randomly distributing them among moderators to determine potential error rates. Telegram stated, «Moderators with suboptimal error rates, or involved in systematic, gross, or material errors are replaced».
- Telegram cites the technical impossibility of disclosing these messages. The regulator notes that a technical solution is possible; for example, WhatsApp allows users to attach messages from end-to-end encrypted chats directly to a report to moderators without resorting to breaking the encryption of the chats. ↩︎
- Telegram stated that these figures reflect the number of employees who «may from time to time be involved in making decisions regarding content or reports from Australia and do not reflect or approximate the total number of employees involved in global content moderation, as well as trust and safety, employed by Telegram». ↩︎