fix: DB sync, upload reliability, parallel uploads, and sticker bug

.gitignore

@@ -17,3 +17,18 @@ local.properties
 !fastlane/
 !fastlane/**/*.txt
 /builderrors.md
+
+# Server (local development only)
+__pycache__/
+*.pyc
+.env
+
+# Server sensitive files
+server/.env
+server/photo_cache/
+server/thumbnails/
+server/__pycache__/
+server/**/__pycache__/
+server/.venv/
+server/*.db
+.kotlin/

PLAN_V2.md

@@ -0,0 +1,121 @@
+# CloudGallery Plan V2 — Telegram-Native Architecture (Final)
+
+## Core Principle
+**Everything goes through Telegram. No HTTP server connection from the app. Ever.**
+
+The server is a background processor that:
+1. Downloads photos from Telegram
+2. Runs face detection, EXIF extraction, location geocoding
+3. Pushes the processed metadata DB back to Telegram as a file
+4. App downloads and merges that metadata DB
+
+## Architecture
+
+```
+┌─────────────────────────────────────────┐
+│          Telegram Channel               │
+│                                         │
+│  Photos (documents with thumbnails)     │
+│  Metadata DB (SQLite file, pinned msg)  │
+│                                         │
+└──────────────┬──────────────────────────┘
+               │
+    ┌──────────┴──────────┐
+    │                     │
+    ▼                     ▼
+┌─────────┐         ┌──────────┐
+│  Phone  │         │  Server  │
+│  (App)  │         │ (Docker) │
+│         │         │          │
+│ - Grid  │         │ - Face   │
+│ - View  │         │ - EXIF   │
+│ - Sync  │         │ - Places │
+│ - Search│         │ - Albums │
+└─────────┘         └──────────┘
+```
+
+## How Metadata Sync Works
+
+### Server Side (runs at home, background)
+1. Server processes all photos: face detection, EXIF, GPS, clustering
+2. Generates a `metadata.db` SQLite file containing:
+   - `people` table: person_id, name, face_thumbnail_file_id, photo_ids
+   - `places` table: place_id, name, city, country, lat, lon, photo_ids  
+   - `photo_metadata` table: remote_id, date_taken, camera, width, height, person_ids, place_id
+   - `albums` table: album_id, name, photo_ids (auto-generated: favorites, trips, etc.)
+3. Uploads `metadata.db` to the Telegram channel as a document
+4. Pins the message (or uses a known caption like `#cloudgallery_metadata_v1`)
+5. Re-runs periodically (daily, or when new photos are uploaded)
+
+### App Side
+1. On app launch: check for pinned message or search for `#cloudgallery_metadata_v1`
+2. Compare message_id with last synced message_id (stored in SharedPreferences)
+3. If new metadata available: download the SQLite file
+4. Import into local Room DB: merge people, places, photo_metadata
+5. Now the app has all AI features locally — People, Places, Search by date/location
+
+### Benefits
+- ✅ Works from anywhere (mobile data, any WiFi)
+- ✅ No HTTP connection needed
+- ✅ No port forwarding, no LAN dependency
+- ✅ Metadata is always up-to-date (server pushes new DB when ready)
+- ✅ App is fully offline-capable after first sync
+- ✅ Server can be turned off after processing — data persists in Telegram
+
+## What's Already Built
+
+### App (all deployed ✅)
+| Feature | Status |
+|---------|--------|
+| Local thumbnail DB (zero-network scroll) | ✅ |
+| ThumbnailSyncWorker (10 concurrent, newest-first) | ✅ |
+| Amazon Photos grid UX (date groups, 2dp gaps) | ✅ |
+| Full-screen preview (zoom, download, share, delete) | ✅ |
+| Server settings UI (HTTP — will be replaced) | ✅ |
+| Cleartext HTTP allowed | ✅ |
+| Aggressive paging (200 per page, auto-load all) | ✅ |
+
+### Server (Docker, port 8100)
+| Phase | Feature | Status |
+|-------|---------|--------|
+| 1 | EXIF + Timeline | ✅ |
+| 2 | Face Detection | ✅ |
+| 3 | Places + Geocoding | ✅ |
+| 4 | Search + Albums | ✅ |
+| 5 | Web Dashboard | ✅ |
+| 6 | Thumbnail Worker | ✅ (generating) |
+
+## Remaining Phases
+
+### Phase H: Telegram Metadata Push (Server Side)
+1. Server generates `metadata.db` from processed data
+2. Uploads to Telegram channel with caption `#cloudgallery_metadata_v1`
+3. Pins the message
+4. Cron job: re-generate and re-upload daily
+
+### Phase I: Telegram Metadata Pull (App Side)  
+1. App searches channel for `#cloudgallery_metadata_v1` message
+2. Downloads the SQLite file
+3. Imports into local Room DB (People, Places, Photo metadata)
+4. UI: People tab, Places tab, enhanced Search
+5. Shows "Last synced: 2 hours ago" with manual refresh button
+
+### Phase J: People & Places UI
+1. People grid: circular face thumbnails with name
+2. Tap person → see all their photos
+3. Places list: location cards with photo count
+4. Tap place → see photos from that location
+5. Search: by person name, place, date, camera
+
+### Phase K: Auto-Albums & Smart Features
+1. Server generates auto-albums: trips, events, favorites
+2. "On This Day" memories
+3. Duplicate detection surfaced in app
+4. Storage stats and cleanup tools
+
+## Performance Targets
+- Grid: 60fps, 9,500+ photos, zero network during scroll
+- Thumbnail sync: ~6 min for 9,500 photos (10 concurrent)
+- Metadata sync: <5 seconds (download one SQLite file)
+- Full photo: <3s download from Telegram CDN
+- Works on: WiFi, mobile data, airplane mode (after initial sync)

app/src/main/AndroidManifest.xml

@@ -13,7 +13,7 @@
     <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"
         tools:ignore="ScopedStorage" />
 
-    <application
+    <application android:usesCleartextTraffic="true"
         android:name=".App"
             android:enableOnBackInvokedCallback="true"
             android:windowIsTranslucent="false"

app/src/main/java/com/akslabs/cloudgallery/App.kt

@@ -15,6 +15,7 @@ import coil.memory.MemoryCache
 import android.graphics.Bitmap
 import coil.Coil
 import com.akslabs.cloudgallery.utils.NotificationHelper
+import com.akslabs.cloudgallery.workers.ThumbnailSyncWorker
 import java.io.File
 
 
@@ -49,6 +50,9 @@ class App : Application() {
             .build()
         Coil.setImageLoader(imageLoader)
         NotificationHelper.createNotificationChannels(this)
+
+        // Start thumbnail sync worker to pre-download all cloud thumbnails
+        ThumbnailSyncWorker.enqueue(this)
     }
 
     }

app/src/main/java/com/akslabs/cloudgallery/api/BotApi.kt

@@ -10,9 +10,6 @@ import com.github.kotlintelegrambot.dispatcher.command
 import com.github.kotlintelegrambot.entities.ChatId
 import com.github.kotlintelegrambot.entities.Message
 import com.github.kotlintelegrambot.entities.TelegramFile
-import com.github.kotlintelegrambot.entities.Update
-import com.github.kotlintelegrambot.entities.files.Document
-import com.github.kotlintelegrambot.entities.files.PhotoSize
 import com.github.kotlintelegrambot.network.Response
 import java.io.File
 import kotlinx.coroutines.Dispatchers
@@ -88,7 +85,7 @@ object BotApi {
                 chatId = ChatId.fromId(channelId),
                 document = TelegramFile.ByFile(file),
                 caption = caption,
-                disableContentTypeDetection = false
+                disableContentTypeDetection = true
             )
         }
     }
@@ -99,6 +96,25 @@ object BotApi {
         }
     }
 
+    /**
+     * Resolves a Telegram file_id to a direct download URL.
+     * Returns a URL like: https://api.telegram.org/file/bot{TOKEN}/{file_path}
+     * These URLs are valid for ~1 hour.
+     */
+    suspend fun getFileUrl(fileId: String): String? {
+        return withContext(Dispatchers.IO) {
+            try {
+                val result = bot.getFile(fileId)
+                val filePath = result.first?.body()?.result?.filePath ?: return@withContext null
+                val token = Preferences.getEncryptedString(Preferences.botToken, "")
+                "https://api.telegram.org/file/bot$token/$filePath"
+            } catch (e: Exception) {
+                Log.e(TAG, "getFileUrl failed for $fileId", e)
+                null
+            }
+        }
+    }
+
     suspend fun deleteMessage(chatId: Long, messageId: Long): Boolean {
         return withContext(Dispatchers.IO) {
             try {
@@ -112,89 +128,33 @@ object BotApi {
     }
 
     /**
-     * Scan Telegram channel/chat for all media files (documents and photos)
-     * Returns a list of discovered media files with their metadata
+     * Scan for cloud photos by verifying known uploads.
+     *
+     * NOTE: The Telegram Bot API does NOT support reading channel message history.
+     * `getUpdates()` only returns unprocessed incoming updates (24h window) and is
+     * fundamentally wrong for channel scanning.  Instead we rely on upload-time
+     * tracking: every successful upload already inserts into `remote_photos`.
+     *
+     * This method now returns an empty success result. Historical discovery from
+     * the channel is not possible via Bot API — use the daily DB backup/restore
+     * flow for the "reinstall" scenario.
      */
     suspend fun scanChannelForMedia(
         channelId: Long,
         limit: Int = 100,
         offsetMessageId: Long? = null
     ): ChannelScanResult {
         return withContext(Dispatchers.IO) {
-            try {
-                Log.d(TAG, "=== SCANNING CHANNEL FOR MEDIA ===")
-                Log.d(TAG, "Channel ID: $channelId, Limit: $limit, Offset: $offsetMessageId")
-
-                val updates = bot.getUpdates(
-                    offset = offsetMessageId,
-                    limit = limit,
-                    timeout = 30
-                )
-
-                val mediaFiles = mutableListOf<DiscoveredMediaFile>()
-                var lastMessageId: Long? = null
-
-                if (updates.isSuccess) {
-                    val updateList = updates.get()
-                    Log.i(TAG, "Received ${updateList.size} updates from Telegram")
-
-                    updateList.forEach { update ->
-                        update.message?.let { message ->
-                            // Only process messages from the target channel
-                            if (message.chat.id == channelId) {
-                                lastMessageId = message.messageId.toLong()
-
-                                // Check for document attachments
-                                message.document?.let { document ->
-                                    val mediaFile = DiscoveredMediaFile(
-                                        fileId = document.fileId,
-                                        fileName = document.fileName,
-                                        fileSize = document.fileSize?.toLong(),
-                                        mimeType = document.mimeType,
-                                        uploadDate = message.date * 1000L, // Convert to milliseconds
-                                        messageId = message.messageId.toInt(),
-                                        mediaType = MediaType.DOCUMENT
-                                    )
-                                    mediaFiles.add(mediaFile)
-                                    Log.d(TAG, "Found document: ${document.fileName} (${document.fileId})")
-                                }
-
-                                // Check for photo attachments
-                                message.photo?.let { photos ->
-                                    // Get the largest photo size
-                                    val largestPhoto = photos.maxByOrNull { it.fileSize ?: 0 }
-                                    largestPhoto?.let { photo ->
-                                        val mediaFile = DiscoveredMediaFile(
-                                            fileId = photo.fileId,
-                                            fileName = "photo_${message.messageId}.jpg",
-                                            fileSize = photo.fileSize?.toLong(),
-                                            mimeType = "image/jpeg",
-                                            uploadDate = message.date * 1000L,
-                                            messageId = message.messageId.toInt(),
-                                            mediaType = MediaType.PHOTO
-                                        )
-                                        mediaFiles.add(mediaFile)
-                                        Log.d(TAG, "Found photo: ${photo.fileId}")
-                                    }
-                                }
-                            }
-                        }
-                    }
-
-                    Log.i(TAG, "Scan complete: Found ${mediaFiles.size} media files")
-                    ChannelScanResult.Success(
-                        mediaFiles = mediaFiles,
-                        hasMore = updateList.size == limit,
-                        nextOffset = lastMessageId?.plus(1)
-                    )
-                } else {
-                    Log.e(TAG, "Failed to get updates from Telegram")
-                    ChannelScanResult.Error("Failed to fetch updates from Telegram")
-                }
-            } catch (e: Exception) {
-                Log.e(TAG, "Exception during channel scan", e)
-                ChannelScanResult.Error("Exception during channel scan: ${e.message}")
-            }
+            Log.i(TAG, "scanChannelForMedia called — Bot API cannot read channel history.")
+            Log.i(TAG, "Cloud photo inventory is maintained at upload time. Returning empty result.")
+
+            // Return empty success — the remote_photos table is the source of truth,
+            // populated during each upload in sendFileApi().
+            ChannelScanResult.Success(
+                mediaFiles = emptyList(),
+                hasMore = false,
+                nextOffset = null
+            )
         }
     }
 }