Adding an AI Model to a Telegram Bot

speed Streaming into a chat is not streaming into a web page

The visible typing effect people expect is produced by editing a message repeatedly. Every edit is a request, and requests are paced — Step 096's outgoing half. Update on every token and you will be told to wait, at which point the reply stops mid-sentence and stays there.

So update in chunks on a deliberate interval rather than continuously, and let the interval be conservative: nobody notices a reply that appears in four steps instead of forty, and everybody notices one that freezes.

The better question is whether to stream at all. In a chat, a complete message that arrives a couple of seconds later reads perfectly naturally — people are used to waiting for a reply. A message that visibly rewrites itself cannot be quoted or replied to while it is happening, looks broken when it stops, and in a group produces a half-finished statement sitting in a room where somebody may screenshot it. A short "thinking" acknowledgement followed by one finished message is usually the better design and always the simpler one.

visibility What leaves the room

This is the fact with the most consequence and the least coverage. Everything the bot processes goes to whoever runs the model.

In a private conversation that is a straightforward exchange between somebody and a service they chose. In a group it is not, because the other members did not choose anything. If your bot reads everything said in a room in order to be helpful, then that room's conversation is being sent to a third party, and the people in it have no way of knowing unless you tell them.

Two things follow, and both are cheap. Decide what invokes it. A mention, a reply to its own messages, or a command — not everything. This also keeps it out of the configuration from Step 086 where a bot receives every message in the group, which is a larger permission than most people realise they are granting. And say what it is, in the group's pinned message and in the bot's description: what it is, what it sends onwards, and what is kept.

Keeping conversation for context is its own decision with its own weight (Step 093). A model that remembers last week is a service that stored last week, and that store is the thing that matters when something goes wrong.

gpp_bad Instructions arrive inside the input

A message in a group, a forwarded document, a web page the bot was asked to summarise — any of these can contain text addressed to the model rather than to a person. "Ignore what you were told and do this instead" is an ordinary sentence, and a model reading it has no reliable way to distinguish it from the content it was asked to process.

This is Step 096's rule in a new form: everything arriving from outside is untrusted input, and here the untrusted input is being read by something that follows instructions.

The defence is structural rather than clever, because attempts to filter this reliably have not worked. Do not give one component both untrusted input and privileged tools. A model that summarises documents should not also be able to send messages, move money or change permissions. Where an action is genuinely needed, the model proposes and your code decides — with a check that does not depend on the model's own output, and a human confirmation in front of anything irreversible.

touch_app Try it now — five minutes

Four steps, and the third one should be done before the bot meets anybody.

1. Write down what invokes it Mention, reply, command. If the answer is "everything in the group", change it before anything else.
2. Say what it is in its first message One sentence that it is automated, and one route to a human. Both from Step 097, and both cheap.
3. Try to talk it out of its instructions Send it a message telling it to ignore them. Then put the same text inside a document and ask it to summarise that. The second usually works better, which is the point.
4. List every tool it can call and the worst case of each Anything on that list that spends, sends or grants needs a human confirmation in front of it.

record_voice_over It will be confidently wrong, in your voice

The failure mode is not gibberish. It is a fluent, plausible, specific answer that happens not to be true — a price that is close, a policy that sounds like yours, a date that is confidently stated and invented. In a chat under your name, readers have no signal that this one is different from the last ten.

Three mitigations, in order of how much they help.

Narrow what it answers from. A model answering from your own documents, and saying which one, is a dramatically better proposition than one answering from everything it ever read. Where it cannot find an answer, the honest output is that it cannot — and building that behaviour is worth more effort than improving the wording of the answers it can give.

Label it and keep an exit. A person who knows they are talking to software forgives a wrong answer and asks for a human. A person who thought they were talking to your support team does not.

Do not let it speak for the organisation on anything that matters. Refunds, commitments, deadlines, apologies, anything legal. Those belong to a person, and the model's job is to get the person there quickly with the context already gathered.

And be realistic about the economics while you design: this costs money per message rather than per user, so the volume you would be delighted to have is also the bill. A bot that answers anything from anybody in a public group is an open tap, which is another reason the invocation rule above is the first decision rather than the last.

help Common questions at this stage