Telegram Message Formatting: Entities and UTF-16 Lengths

Formatting sits beside the text, not inside it

Bold, italics, links and other formatting in a Telegram message are not stored as ** or tags inside the text. The text is plain characters, and the formatting is attached separately as a list of entities. Each entity has three parts: a type, a starting position (offset) and a length (length).

text:     Hello and welcome
entities: [{type: bold, offset: 0, length: 5}]
          -> only "Hello" is bold

Things the server can recognise on its own, such as URLs, hashtags, @usernames and /commands, get entities automatically. Entities can overlap (bold italic, for example), but some types, such as code blocks, cannot contain other formatting.

Positions are counted in UTF-16 units

This is the most common mistake. offset and length are not character counts; they are counted in UTF-16 code units.

If you compute positions in a language that counts characters (code points), such as Python's len(), every formatting range after an emoji shifts by one or two places. Use the length of the string encoded as UTF-16 (bytes divided by two), or the native length in a language that already counts in UTF-16, such as JavaScript.

def u16(s):  # length in UTF-16 units
    return len(s.encode("utf-16-le")) // 2

Where Markdown and HTML get converted

• User-account API: the app (client) converts what the user typed in Markdown or HTML into entities, and sends the server plain text plus the entity list.
• Bot API: set parse_mode to MarkdownV2 or HTML and Telegram converts it for you. You can also pass an entity list directly (entities); positions are still in UTF-16 units. See your first Bot API requests.

How the formatting looks to people writing messages is covered in formatting text in Telegram.