Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chat partner was not found in recipient-table #244

Closed
Aerilius opened this issue Sep 26, 2024 · 6 comments
Closed

Chat partner was not found in recipient-table #244

Aerilius opened this issue Sep 26, 2024 · 6 comments

Comments

@Aerilius
Copy link

Aerilius commented Sep 26, 2024

I try to "convert" a desktop backup into a phone backup so that I can import it into the Android app.

However, signalbackup-tools requires the phone backup file as a "scaffold" to operate on. So I reinstalled and re-registered the Android app, exported a backup file of the empty app (1.7MB). When running:

./signalbackup-tools signal-empty.backup [passkey] --importfromdesktop --output signal-from-desktop.backup

it succeeds, but the new backup file is still empty and small (2.1MB instead of > 600MB)

The log contains these warnings:

Failed to find recipient for uuid
 *** Starting log: 2024-09-26 15:34:17 *** 
signalbackup-tools (./signalbackup-tools) source version 20240925.172224

BACKUPFILE VERSION: 1
BACKUPFILE SIZE: 1673143
COUNTER: 1258050808
Reading backup file: 100.0%... done!
Database version: 247
[Error]: Decryption appears to have failed (padding bytes have unexpected value)
Trying to match conversation (1/24) (type: private)
           Failed to find matching thread for conversation, creating. (id: 8e1adc3c-1f96-4a7c-81f8-b0c17b1bb68f, thread_id: 2)
 - Importing 12 messages into thread._id 2
Trying to match conversation (2/24) (type: group)
[Warning]: Failed to find recipient for uuid: __signal_group__v2__!e1cdxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: __signal_group__v2__!e1cd770ed1b94324aa583d812c00f58301a5bf2dbd0e3b74a2315a8da142134a)
Trying to match conversation (3/24) (type: private)
[Warning]: Failed to find recipient for uuid: f8a5xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: f8a546a5-7dee-422f-8468-c4dda0c4dd18)
Trying to match conversation (4/24) (type: private)
[Warning]: Failed to find recipient for uuid: 2155xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 2155fdc8-7876-4d58-b708-b53f06646656)
Trying to match conversation (5/24) (type: private)
[Warning]: Failed to find recipient for uuid: 8ac5xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 8ac54d1a-fee5-42d5-8c20-127bc430f084)
Trying to match conversation (6/24) (type: private)
[Warning]: Failed to find recipient for uuid: dc61xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: dc613dea-bdab-4248-8718-c11e4310a684)
Trying to match conversation (7/24) (type: private)
[Warning]: Failed to find recipient for uuid: e90exxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: e90efa40-d685-4474-87b0-22c7a6a7afb0)
Trying to match conversation (8/24) (type: private)
[Warning]: Failed to find recipient for uuid: cc23xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: cc232f2c-44cb-4077-831b-3f0380321c29)
Trying to match conversation (9/24) (type: private)
[Warning]: Failed to find recipient for uuid: 83b5xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 83b54970-71c5-4be3-8562-25bc61774051)
Trying to match conversation (10/24) (type: group)
[Warning]: Failed to find recipient for uuid: __signal_group__v2__!4070xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: __signal_group__v2__!4070f997b88d719ce43c0a00a7857aa6765f37cfe1ea84d85b4a489784db01ea)
Trying to match conversation (11/24) (type: private)
[Warning]: Failed to find recipient for uuid: 3c76xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 3c76e6cd-b69f-4bbf-9d32-0fefd2110658)
Trying to match conversation (12/24) (type: private)
[Warning]: Failed to find recipient for uuid: 70dcxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 70dce216-0207-4397-aebb-2d70721fb335)
Trying to match conversation (13/24) (type: private)
[Warning]: Failed to find recipient for uuid: ca56xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: ca563a6f-5af0-419c-a070-2a753464a85d)
Trying to match conversation (14/24) (type: private)
[Warning]: Failed to find recipient for uuid: 1e5bxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 1e5b2428-3afe-49f6-9e03-674d451567fe)
Trying to match conversation (15/24) (type: private)
[Warning]: Failed to find recipient for uuid: b695xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: b695ffa9-4c24-4361-b341-44f3e6e84441)
Trying to match conversation (16/24) (type: private)
[Warning]: Failed to find recipient for uuid: a2c7xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: a2c77cdc-06a9-4258-a1aa-c914b772ab29)
Trying to match conversation (17/24) (type: private)
[Warning]: Failed to find recipient for uuid: 1e6cxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 1e6c7da2-ab53-4d38-9eda-8016158df2cb)
Trying to match conversation (18/24) (type: private)
[Warning]: Failed to find recipient for uuid: c8f1xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: c8f13066-146f-4c9f-88ed-a69341fa340f)
Trying to match conversation (19/24) (type: group)
[Warning]: Failed to find recipient for uuid: __signal_group__v2__!a1e5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: __signal_group__v2__!a1e5241ce512c05744131767c7388496554cefccbe8ed9942abbfe63906ef6d4)
Trying to match conversation (20/24) (type: group)
[Warning]: Failed to find recipient for uuid: __signal_group__v2__!5302xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: __signal_group__v2__!5302d9dec55305f0772146b7c384ae24781f265855ec9c3e729796515d70fcd4)
Trying to match conversation (21/24) (type: private)
[Warning]: Failed to find recipient for uuid: 8842xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 8842d2ca-59f6-4c44-a29e-ec6331e43a8c)
Trying to match conversation (22/24) (type: private)
[Warning]: Failed to find recipient for uuid: 7deaxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 7dea6bff-8aa8-4188-bc00-7dcf3c833815)
Trying to match conversation (23/24) (type: group)
[Warning]: Failed to find recipient for uuid: __signal_group__v2__!99d5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: __signal_group__v2__!99d5bab2993ef6e0a659291162e6be9e4a2569d112a3ad72ffa955821fdea4a5)
Trying to match conversation (24/24) (type: private)
[Warning]: Failed to find recipient for uuid: 6c98xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[Warning]: Chat partner was not found in recipient-table. Skipping. (id: 6c98008c-925a-401f-aee1-13b93db92856)
reorderMmsSmsIds...ok
updateThreadsEntries
  Dealing with thread id: 2, 1
Checking foreign key constraints... ok
Checking database integrity (full)... ok

Exporting backup to 'signal-from-desktop.backup'
Writing HeaderFrame...
Writing DatabaseVersionFrame...
Writing SqlStatementFrame(s)...
  Dealing with table 'attachment'... 6/6 entries...done
  Dealing with table 'avatar_picker'... 0/0 entries...done
  Dealing with table 'recipient'... 136/136 entries...done
  Dealing with table 'thread'... 2/2 entries...done
  Dealing with table 'message'... 17/17 entries...done
  Dealing with table 'call'... 0/0 entries...done
  Dealing with table 'call_link'... 0/0 entries...done
  Dealing with table 'cds'... 134/134 entries...done
  Dealing with table 'chat_colors'... 0/0 entries...done
  Dealing with table 'distribution_list'... 1/1 entries...done
  Dealing with table 'distribution_list_member'... 0/0 entries...done
  Dealing with table 'donation_receipt'... 0/0 entries...done
  Dealing with table 'drafts'... 0/0 entries...done
  Dealing with table 'emoji_search'... 0/0 entries...done
  Dealing with table 'groups'... 0/0 entries...done
  Dealing with table 'group_membership'... 0/0 entries...done
  Dealing with table 'group_receipts'... 0/0 entries...done
  Dealing with table 'identities'... 2/2 entries...done
  Dealing with table 'in_app_payment'... 0/0 entries...done
  Dealing with table 'in_app_payment_subscriber'... 0/0 entries...done
  Dealing with table 'kyber_prekey'... 202/202 entries...done
  Dealing with table 'mention'... 0/0 entries...done
  Dealing with table 'msl_payload'... 0/0 entries...done
  Dealing with table 'msl_message'... 0/0 entries...done
  Dealing with table 'msl_recipient'... 0/0 entries...done
  Dealing with table 'name_collision'... 0/0 entries...done
  Dealing with table 'name_collision_membership'... 0/0 entries...done
  Dealing with table 'notification_profile'... 0/0 entries...done
  Dealing with table 'notification_profile_allowed_members'... 0/0 entries...done
  Dealing with table 'notification_profile_schedule'... 0/0 entries...done
  Dealing with table 'payments'... 0/0 entries...done
  Dealing with table 'pending_pni_signature_message'... 0/0 entries...done
  Dealing with table 'pending_retry_receipts'... 0/0 entries...done
  Dealing with table 'reaction'... 0/0 entries...done
  Dealing with table 'remapped_recipients'... 0/0 entries...done
  Dealing with table 'remapped_threads'... 0/0 entries...done
  Dealing with table 'remote_megaphone'... 0/0 entries...done
  Dealing with table 'sender_key_shared'... 0/0 entries...done
  Dealing with table 'sender_keys'... 0/0 entries...done
  Dealing with table 'sticker'... 0/0 entries...done
  Dealing with table 'storage_key'... 0/0 entries...done
  Dealing with table 'story_sends'... 0/0 entries...done
Writing SharedPrefFrame(s)...
Writing KeyValueFrame(s)...
Writing Avatars...
Writing EndFrame...
Done! Wrote 2112588 bytes.

Why can the recipients not be added from the desktop backup?

Afterall, Signal desktop shows recipient names and phone numbers, so it should certainly contains not only anonymous UUIDs to do the match. I tried the same with:

  • an empty Android Signal backup after giving permission for contacts (assuming the table must be populated with contacts, but maybe the app assigns new UUIDs that won't match). --listrecipients also shows that all recipients are there.
  • an empty Android Signal backup without giving permission for contacts (assuming signalbackup-tools would mismatch if the recipient exists, but add it if not existing)
@Aerilius
Copy link
Author

Interestingly, signalbackup-tools was able to merge the up-to-date desktop backup into a very old Android Signal backup file. So I now also have my data back in a very old backup format (database version 106, Signal 5.17.1).

I'm a bit hopeless I can ever get my data back into the app (and thus into future backups).

  • Latest Signal Android fails to import old backups
  • Old Signal Android cannot do phone number verification anymore, thus I cannot import my data in the old format and update the app.
  • signalbackup-tools cannot migrate between different database versions.
    I wish there was an exchange format that is more robust than constantly changing databases schemas.

@Aerilius
Copy link
Author

Aerilius commented Sep 26, 2024

if (recipientid_for_thread == -1)
{
Logger::warning("Chat partner was not found in recipient-table. Skipping. (id: ",

I see there is a getRecipientIdFromPhoneMapped, but getRecipientIdFromUuidMapped was used instead.

@bepaald
Copy link
Owner

bepaald commented Sep 26, 2024

Why can the recipients not be added from the desktop backup?

This is a known and expected situation, there is more to adding contacts than just inserting an entry into the recipient table and setting the UUID, there need to be valid keys exchanged and such. I believe the required data to do this is not present in the Desktop client (and if it is I don't know how to do it). It may even be necessary to communicate with Signal's servers to create valid contacts.

This limitation is also mentioned in the README.

Afterall, Signal desktop shows recipient names and phone numbers, so it should certainly contains not only anonymous UUIDs to do the match.

Signal Desktop certainly has the UUIDs, the problem (probably) is they are not in your Android backup.

an empty Android Signal backup after giving permission for contacts (assuming the table must be populated with contacts, but maybe the app assigns new UUIDs that won't match). --listrecipients also shows that all recipients are there.

This may be interesting. Generally, in cases like this, I tell people to make sure the contacts exist in the backup by starting some chats with them. But, if --listrecipients already shows them all, that used to mean it should work.

However, things have actually changed a little recently, since phone numbers are no longer required (they are for registration, but not for communicating with a new contact), some contacts are split in two: 1 from the system phone book and 1 'true' registered Signal contact (that may not have a phone number associated). I wonder if that's what's happening here. My guess would be that the UUID's for these contacts are not set. You could check this by running:

signalbackup-tools [input] [passphrase] --runprettysqlquery "SELECT COALESCE(profile_joined_name,system_joined_name),aci,e164 FROM recipient"

You could compare this to the output when running on the old v106 backup (due to renaming of columns, change the query to "SELECT COALESCE(profile_joined_name,system_display_name),uuid,phone FROM recipient" on the older backup).

My theory is that the UUID's ('aci') on the new database are empty, because as far as Signal knows these contacts are not actual Signal contacts.

I see there is a getRecipientIdFromPhoneMapped, but getRecipientIdFromUuidMapped was used instead.

Yes this is intentional. Getting the recipient by phone number is only done when no UUID is found. This is for dealing with old Signal Desktop data, valid contacts without a UUID can not exist in current databases.

I'm a bit hopeless I can ever get my data back into the app (and thus into future backups).

We should be able to get this working. It's unfortunate your only other backup is so old.

I think the best chance of fixing this right now, is what I tell everyone: do not use a completely empty backup as the base, but try to actually start a thread with as many of your contacts as possible, to make sure they are present in the Android database.

If that is not possible for some reason, I might be able to write some migration code for the old backup (there is already code that migrates from version 132 upwards, maybe I can extend that down a bit), or write a custom function to copy the contacts, with their keys, from the old database into the new one (not sure about that though, it's certainly never been done).

Let me know if you can get the contacts in your Android backup, and if that fixes things. Then we'll see from there.

Thanks!

@Aerilius
Copy link
Author

Aerilius commented Sep 26, 2024

Thank you for your detailed reply! In the meantime, I figured out on my own that neither UUID nor phone number could be matched, since the recipient table is empty (I didn't know that it was from the input backup, not the desktop backup). So basically, it does a left join. Also, as expected, createmissingcontacts will make Signal crash (although it shows up with all conversations for a moment).

I hoped to fix it in one go before continuing messaging, but as it seems I have to leave it as it is.

  • When I have again enough recipients, I can try exporting an Android backup and importing into it the messages from the desktop backup (will take some time)
  • or I wait until Signal will release backup import&export for all clients (including desktop).

@bepaald
Copy link
Owner

bepaald commented Sep 27, 2024

Thank you for your detailed reply! In the meantime, I figured out on my own that neither UUID nor phone number could be matched, since the recipient table is empty (I didn't know that it was from the input backup, not the desktop backup). So basically, it does a left join. Also, as expected, createmissingcontacts will make Signal crash (although it shows up with all conversations for a moment).

The identifiers need to be present in both databases to be able to match between them. Once recipients are matched, messages/attachments can be transferred fro the Desktop database into the Android backup, also creating threads if not present.

I'm slightly confused about the empty recipient table, as you mentioned --listrecipients shows all recipients?

I hoped to fix it in one go before continuing messaging, but as it seems I have to leave it as it is.

Yes, if exchanging some messages to get the recipients in your Android database is not an option than you'd indeed have to wait. I was thinking about some other ways to get the contacts in your Android installation (like sharing them in note-to-self from Desktop), but I can't come up with anything that works.

  • When I have again enough recipients, I can try exporting an Android backup and importing into it the messages from the desktop backup (will take some time)
  • or I wait until Signal will release backup import&export for all clients (including desktop).

I also think these are your options, unless you want me to attempt to write custom code to migrate the old v106 database (or data from it) to a newer version. But if the above are acceptable for you they would probably be safer (less experimental).

Let me know if you have any more questions or if I can do anything else for you. Otherwise this issue can probably be closed.

Thanks!

@bepaald bepaald closed this as completed Oct 7, 2024
@bepaald
Copy link
Owner

bepaald commented Oct 15, 2024

Just wanted to quickly let you know, I have added a (highly experimental) option to import valid contacts from the Signal Desktop database.

Obviously you don't have to test it, you can just wait for the contacts to show up, if you prefer, but I thought I'd at least let you know. Some more info here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants