open failed in ios 15.2

[jiangzhiguo1992]

After ios upgrade to 15.2.1, encrypted database open failed.

sqflite: ^2.0.2
sqflite_sqlcipher: ^2.1.0

2022-01-24 17:26:59.672175+0800 Runner[2338:552005] Unknown error calling sqlite3_step (26: file is not a database) rs
2022-01-24 17:26:59.672959+0800 Runner[2338:552005] Unknown error calling sqlite3_step (26: file is not a database) eu
2022-01-24 17:26:59.673004+0800 Runner[2338:552005] DB Query: BEGIN EXCLUSIVE
2022-01-24 17:26:59.673042+0800 Runner[2338:552005] Unknown error finalizing or resetting statement (26: file is not a database)
2022-01-24 17:26:59.673064+0800 Runner[2338:552005] DB Query: BEGIN EXCLUSIVE
2022-01-24 17:26:59.673328+0800 Runner[2338:552009] flutter: 17:26:59：error DatabaseException(Error Domain=FMDatabase Code=26 "file is not a database" UserInfo={NSLocalizedDescription=file is not a database}) sql 'BEGIN EXCLUSIVE' args [] during open, closing...
2022-01-24 17:26:59.673465+0800 Runner[2338:552005] Error calling sqlite3_step (1: cannot rollback - no transaction is active) SQLITE_ERROR
2022-01-24 17:26:59.673507+0800 Runner[2338:552005] DB Query: ROLLBACK
2022-01-24 17:26:59.673540+0800 Runner[2338:552005] Unknown error finalizing or resetting statement (1: cannot rollback - no transaction is active)
2022-01-24 17:26:59.673568+0800 Runner[2338:552005] DB Query: ROLLBACK

[jiangzhiguo1992]

Has anyone encountered this problem?

[davidmartos96]

Can you run the following to check if SQLCipher is being loaded properly?

final db = await openDatabase(":memory:");
final rows = await db.rawQuery("PRAGMA cipher_version");
print(rows[0]);

It should say something like "SQLCipher Community version 4.x"
If it does not, could you share your pubspec file?

[davidmartos96]

And another thing I've just noticed. You shouldn't need both sqflite and sqflite_sqlcipher in the pubspec. This library already exposes the same functions and API from sqflite

[jiangzhiguo1992]

PRAGMA return null and my problem is the same as described in this link https://developer.apple.com/forums/thread/697001

[jiangzhiguo1992]

this is my pubspec.yaml

environment:
sdk: ">=2.12.0 <3.0.0"

dependencies:
flutter:
sdk: flutter
flutter_localizations:
sdk: flutter

image_picker: ^0.8.4
local_auth: ^1.1.7
package_info_plus: ^1.0.6
device_info_plus: ^2.1.0
share_plus: ^2.1.4
connectivity_plus: ^2.2.0
path_provider: ^2.0.3
shared_preferences: ^2.0.7
url_launcher: ^6.0.2

flutter_bloc: ^7.2.0
equatable: ^2.0.3
synchronized: ^3.0.0
dio: ^4.0.0
get_it: ^7.2.0
uuid: ^3.0.4
crypto: ^3.0.1
mime_type: ^1.0.0
permission_handler: ^8.1.6
vibration: ^1.7.3
flutter_local_notifications: ^8.2.0
background_fetch: ^1.0.1

logger: ^1.1.0
sentry_flutter: ^6.0.1

intl: ^0.17.0
flutter_cupertino_localizations: ^1.0.1

flutter_secure_storage: ^4.2.1
sqflite: ^2.0.0+4
sqflite_sqlcipher: ^2.0.0

flutter_screenutil: ^5.0.0+2
animations: ^2.0.1
bot_toast: ^4.0.1
flutter_spinkit: ^5.1.0
flutter_markdown: ^0.6.6
flutter_slidable: ^0.6.0

flutter_svg: ^0.22.0
font_awesome_flutter: ^9.1.0
cupertino_icons: ^1.0.3

path: ^1.8.0
file_picker: ^4.0.3

photo_view: ^0.12.0
flutter_image_compress: ^1.0.0
image_cropper: ^1.4.1

flutter_sound: ^8.3.10

qr_code_scanner: ^0.5.2
qr_flutter: ^4.0.0

web3dart: ^2.3.1

nkn_sdk_flutter:
path: ./plugins/nkn-sdk-flutter

dev_dependencies:
flutter_test:
sdk: flutter

flutter:

uses-material-design: true

assets:
- assets/
- assets/contact/
- assets/icons/
- assets/chat/
- assets/wallet/
- assets/splash/

flutter_intl:
enabled: true

[davidmartos96]

There is another thing you can try. There was one time when SQLCipher was not being loaded for me because one native dependency on iOS was linking explicitly with SQLite.
What fixed it for me was the following:
Put -framework SQLCipher in "Other Linker Flags" in your project's settings on XCode.

Unfortunately I don't know what else could be happening. An app I have with this package works correctly on iOS 15.2

[jiangzhiguo1992]

I just checked my "Other Linker Flags" and it always has "-framework SQLCipher" in it.
In fact my app also works on ios15.2. But if I install the app in ios14.x version and upgrade to ios15.2, the database will fail to open.
You can try to install the demo on ios14.x, then upgrade to 15.2 and see if it will appear.

[davidmartos96]

That sounds really odd. I don't see how an iOS upgrade can affect the SQLCipher library in the app binary.
Out of curiosity, how are you handling downgrading and upgrading the device to different iOS versions. Can you do it with the simulator?

[jiangzhiguo1992]

I haven't tried the emulator, but all users in my app who upgraded to ios15.2 had this problem.

[jiangzhiguo1992]

It suddenly occurred to me that the emulator cannot be upgraded.

[jiangzhiguo1992]

I have the same problem after upgrading to iOS 15.2 on my own phone.

[davidmartos96]

Can you try to reproduce it in the example app of this repository? That way we could tell if it's related to this or is some other issue.

[jiangzhiguo1992]

I haven't been able to reproduce the issue yet because I don't have the right equipment around.
But I deleted the code in my app, only the database open operation is left, and the error still exists.
And after I uninstalled and reinstalled the app, there is no such problem.

[davidmartos96]

You should try without any other dependency in the pubspec. I have a feeling that it could be a conflict with some dependency.
If you don't get the error with just sqflite_sqlcipher, then start adding the rest of dependencies until it starts failing

[navjyot11singh]

Not able to access database when updating in android 15.2 and above. Same issue

[davidmartos96]

@davidmartos96 Flutter application

But in terms of updating, what are you changing in the application? Are you simply running the app in the new iOS and it fails?

[navjyot11singh]

updating the new version of flutter app from App Store

[davidmartos96]

@navjyot11singh @jiangzhiguo1992 Can you try the following when opening the database? It's possible that previously SQLCipher wasn't being linked properly and the database was using normal SQLite (no encryption), so now providing the password won't open the database.

late final Database db;
try {
  db = await openDatabase(file, password: '1234');
} catch(e) {
  db = await openDatabase(file, password: ''); 
}

[jiangzhiguo1992]

I've tried await openDatabase(file, password: '') and await openDatabase(file) but unfortunately this still doesn't work.

[davidmartos96]

@jiangzhiguo1992 Are you able to reproduce the issue with the iOS Simulator? If so, please create a repository with a basic app that reproduces the error and with the steps to make it appear. As of right now I'm not able to debug the issue.

[jiangzhiguo1992]

I've found a place at the moment and don't know if it helps with my problem.
When setting a password, setKey returns false.

[queue inDatabase:^(FMDatabase *database) {
         if (password == nil) {
             [database setKey:@""];
         } else {
             [database setKey:password];
         }
     }];

And int rc = sqlite3_key(_db, [keyData bytes], (int)[keyData length]); the return value is SQLITE_MISUSE(21), not SQLITE_OK(0).

[davidmartos96]

@jiangzhiguo1992 That sounds relevant. Do you have a repository I can try?

[davidmartos96]

@jiangzhiguo1992 Did you include the dependency_overrides from the README in the pubspec.yaml?
If you are not getting any output in the PRAGMA cipher_version , then it's definitely a problem with the linking of the SQLCipher library.

Also, remember that you don't need to include sqflite in your pubspec. With just sqflite_sqlcipher should suffice.

dependency_overrides:
  sqflite:
    git:
      url: https://www.github.com/davidmartos96/sqflite_sqlcipher.git
      path: sqflite
      ref: fmdb_override

[jiangzhiguo1992]

After I set dependency_overrides, the result is still the same.
The replacement result of PRAGMA cipher_version is {cipher_version: 4.4.2 community}.

[davidmartos96]

@jiangzhiguo1992 So with the dependency_overrides you get the PRAGMA correctly, and without it you don't get any output right? At the start of the thread you mentioned the PRAGMA wasn't working.

[jiangzhiguo1992]

Yes, I found that adding dependency_overrides does make a difference. But my database still can't be opened, no matter if the password is entered correctly or empty.

[jiangzhiguo1992]

I also found that if I add dependency_overrides, then int rc = sqlite3_key(_db, [keyData bytes], (int)[keyData length]); returns SQLITE_OK (0), but the strange thing is that the database still cannot be opened.

[davidmartos96]

@jiangzhiguo1992 that looks ok now.
Could you try the whole process of using iOS 14 and 15? Now it's very hard to tell the state of the database stored in the device.
I imagine the database is created from within the device, and not downloaded or imported from somewhere else, right?
If cipher_version works, the open failed is probably not wrong, and it must be failing because of the format in which the database is right now.

[jiangzhiguo1992]

I have now obtained the database file in question and sent it to your e-mail.

[Heech]

@davidmartos96 @jiangzhiguo1992
Hi were you ever able to discover the problem/solution to this problem? 🙇

I'm also seeing different behavior on iOS 14. Even though we are not using the required dependency_overrides the DB seem to be encrypted correctly on iOS 14. But unencrypted on iOS 15 and above devices. (having a dependency on cached_network_image)

But trying to update app with using dependency_overrides the DB can no longer be read on iOS 14 device.

[davidmartos96]

Could you try the following configuration in XCode?

https://pub.dev/packages/sqlcipher_flutter_libs#incompatibilities-with-sqlite3-on-ios-and-macos

[Heech]

Thank you for the suggestion. Unfortunately seem I'm running into the iOS 14 issue commented in the link below.
https://discuss.zetetic.net/t/db-file-is-encrypted-without-using-sqlcipher/5420/2

Some versions of iOS shipped an undocumented encryption feature in the system SQLite library. We originally identified this behavior in iOS 13 and 14. The encryption function does not use SQLCipher, but it does respond to PRAGMA key. As far as we can tell, this undocumented encryption feature was removed sometime after iOS 15.0, likely because it was not intended for application use.

Coming back to SQLCipher, unfortunately if the database is not encrypted using SQLCipher, the library doesn’t know how to open it and can’t convert or re-encrypt it. Unfortunately, you may need to remove the database and recreate from scratch in that case.

My app is most likely using another undocumented encryption function only on iOS 14.
This causing the DB to be unreadable after updating to the correct implementation of sqflite_sqlcipher
or just by updating the app above iOS 14. 🙇

From what I can tell there seem to be no way to handle this case without recreating the DB from scratch..