#1542

import-telegram: import a Telegram Desktop JSON export, the way import-messenger imports a Messenger export

OpenJoiopened 3d ago

Summary

Telegram is the one chat network I use that msgvault can only reach through Beeper Desktop (add-beeper / sync-beeper). I do not want to run Beeper. Telegram Desktop exports the account's own history locally as JSON with media, and msgvault already imports an export of that shape for Facebook Messenger (import-messenger). I am asking for the same for Telegram: an import-telegram command that reads a Telegram Desktop export.

Context

v0.21.0, macOS arm64, SQLite archive of about 3.9M messages: mail from 20 accounts, WhatsApp, iMessage, 22 Slack workspaces, meeting transcripts. Everything is local; nothing in the archive leaves the machine. The Beeper route would add a hosted bridge account in the middle of the one network that still has no path in, so I would rather import the official export.

The source format

Telegram Desktop → Settings → Advanced → Export Telegram data, with "Machine-readable JSON" selected, writes an export root with result.json and a chats/chat_NN/ folder per chat (photos, video_files, voice_messages, files, stickers, …). The schema is documented at https://core.telegram.org/import-export. The parts an importer needs:

  • personal_information carries the exporting user's user_id, names and username, so the "me" identity can be read from the export instead of a flag.
  • chats.list[] (and left_chats.list[]): each chat has id, name, type (personal_chat, bot_chat, private_group, private_supergroup, public_supergroup, private_channel, public_channel, saved_messages, replies) and messages[].
  • each message has id, type (message or service), date and date_unixtime, from and from_id (user123456, channel123456, chat123456), text (a string, or an array of strings and entity objects) plus text_entities, reply_to_message_id, edited, forwarded_from, and media as a relative path in photo or file with media_type and mime_type.
  • a single-chat export writes the same result.json with one chat object at the top level.

Telegram user ids are stable integers, unlike the Messenger export, which only has names. Telegram also exports a date range, so an incremental re-export is possible by hand until a live source exists.

Asks

  1. msgvault import-telegram <export-dir>, modelled on import-messenger: a telegram message type, one conversation per chat keyed on the chat id, the sender identity from from_id, the owner from personal_information.user_id (with a --me override), text_entities flattened to plain text, service messages dropped or kept as system messages, media copied from the relative paths, and --limit / resume checkpoints like the other importers.
  2. Idempotent re-import keyed on chat id + message id, so a later export of the same account (full or date-ranged) only adds what is new, the way import-slackdump behaves next to sync-slack.
  3. Participant identity namespaced the way the Beeper importer namespaces its provider ids (provider_ids.go), so a Telegram user seen through this importer and the same user seen through a Beeper bridge resolve to one participant rather than two, and a bare numeric id can never collide with another network's id.

A live source (add-telegram / sync-telegram through a user session, like add-matrix) would be the follow-on; the export importer is the part that gets the history in, and I would use it on the day it ships. I can test against a real export of my own account, and I can share a redacted result.json excerpt for fixtures if that helps.

Activity

0

No activity yet.