From 5f1d0633caa6fa41ca5e2bacae77d2a227d3c162 Mon Sep 17 00:00:00 2001 From: QwQ-dev Date: Sat, 11 Jan 2025 21:16:15 +0800 Subject: [PATCH] docs: README. --- cache/README.md | 242 +++++++++++++++++++---------------------------- player/README.md | 195 +++++++++++++++++++++++++------------- 2 files changed, 227 insertions(+), 210 deletions(-) diff --git a/cache/README.md b/cache/README.md index d2cb7d1..c138a62 100644 --- a/cache/README.md +++ b/cache/README.md @@ -1,180 +1,134 @@ -# 🎮 Player Module   ![Version](https://img.shields.io/badge/version-1.0-blue) ![License](https://img.shields.io/badge/license-MIT-green) +# 🚀 Cache Module -> A **high-performance** and **scalable** player data management system. It leverages **L1 (Caffeine)** and **L2 (Redis)** caching, **MongoDB** persistence, and **Redis Streams** for near “enterprise-level” performance in distributed Minecraft or similar server environments. +> A powerful multi-level caching solution supporting Caffeine and Redis, with functional programming support and automatic lock handling. ---- +## ✨ Features -## 📚 Table of Contents +- 🔄 Support for Caffeine's `Cache` and `AsyncCache` +- 📦 In-memory Redis database implementation +- 🔒 Automatic lock handling mechanism +- 🎯 Functional programming support +- 📚 Multi-level cache architecture +- ⚡ High-performance operations +- 🛡️ Thread-safe implementation + +## 📋 Table of Contents + +- [Installation](#-installation) +- [Usage](#-usage) + - [Caffeine Cache](#-caffeine-cache) + - [Redis Cache](#-redis-cache) + - [Custom Cache](#-custom-cache) + - [Multi-Level Cache](#-multi-level-cache) +- [Contributing](#-contributing) +- [License](#-license) -- [Introduction](#introduction) -- [Automation](#automation) -- [Key Components](#key-components) -- [Data Flow](#data-flow) -- [Performance Highlights](#performance-highlights) -- [Why Near Enterprise-Level?](#why-near-enterprise-level) -- [Summary](#summary) -- [License](#license) +## 📥 Installation ---- +Add the following dependency to your project: -## Introduction +```kotlin +dependencies { + compileOnly(files("libs/cache-1.0-SNAPSHOT-sources.jar")) +} +``` -The **Player Module** is a core part of a distributed system aimed at managing player data with **high throughput** and **low latency**. By integrating: -- **L1 Cache (Caffeine):** In-memory, ultra-fast access for online players. -- **L2 Cache (Redis):** Central cache for multi-server environments, synchronized via Redis Streams. -- **MongoDB:** Document-based persistence for flexible schema and long-term data storage. +## 💻 Usage -This architecture ensures that both frequent reads/writes (online players) and less common operations (offline queries, global sync) are handled efficiently. +### 🔵 Caffeine Cache ---- +Caffeine provides high-performance caching capabilities with excellent hit rates. -## Automation +```java +// Create Caffeine cache service +CacheServiceInterface, String> caffeineCache = + CacheServiceFactory.createCaffeineCache(); -- **L1 ↔ L2 Automatic Sync** - - Whenever a player’s data is updated in L1, tasks like `L1ToL2PlayerDataSyncTask` push changes to L2. - - Periodically or on-demand, Redis Streams also trigger updates across different servers. +// Basic get operation +String value = caffeineCache.get( + cache -> cache.getIfPresent(1), // Get cache value + () -> "defaultValue", // Cache miss handler + (cache, queryValue) -> cache.put(1, queryValue), // Cache storage + true // Enable caching +); -- **Automated Persistence** - - The module uses timed tasks (e.g., `PlayerDataPersistenceTask`) to write L2 (Redis) data into MongoDB, ensuring eventual consistency without manual intervention. +// Thread-safe operation with lock +String valueWithLock = caffeineCache.get( + cache -> new ReentrantLock(), // Lock acquisition + cache -> cache.getIfPresent(1), // Cache retrieval + () -> "defaultValue", // Cache miss handler + (cache, queryValue) -> cache.put(1, queryValue), // Cache storage + true, // Enable caching + LockSettings.of(1, 1, TimeUnit.MINUTES) // Lock configuration +); +``` -- **Annotation-Driven Registration** - - Classes annotated with `@RStreamAccepterRegister` (like `L1ToL2PlayerDataSyncByNameRStreamAccepter`) automatically handle Redis Streams. - - Type adapters annotated with `@TypeAdapterRegister` are auto-registered for custom serialization/deserialization (see `PairTypeAdapter`). - ---- +### 🔴 Redis Cache -## Key Components +Redis cache service typically serves as a second-level cache for distributed systems. -### LegacyPlayerDataService +```java +// Redis configuration +Config config = new Config(); +config.useSingleServer().setAddress("redis://127.0.0.1:6379"); -A flexible “service object” encapsulating: -- **Redis Connection** (L2 Cache + Streams) -- **MongoDB Connection** (Persistence) -- **Caffeine Cache** (L1 Cache) +// Create Redis cache service +RedisCacheServiceInterface redisCache = + CacheServiceFactory.createRedisCache(config); -By creating multiple `LegacyPlayerDataService` instances, you can manage different database or Redis clusters independently. -> Internally, it uses multi-level caching: L1 for ultra-fast memory-based access, and L2 for cross-server data sharing. +// Type-safe value retrieval +int intValue = redisCache.getWithType( + cache -> cache.getBucket("key").get(), + () -> 1, + (cache, queryValue) -> cache.getBucket("key").set(queryValue), + true +); +``` -### Caching Tiers +### 🔧 Custom Cache -1. **L1 (Caffeine)** - - **Fast Memory Access** - - Typically holds data for **online** or **recently active** players. - - Minimizes network overhead and database queries. +Implement your own cache using Java's `ConcurrentHashMap` or other solutions. -2. **L2 (Redis)** - - **Distributed, Centralized View** - - Suited for multi-server setups. - - Uses **Redis Streams** for broadcasting player data changes (e.g., join/quit, data update) across nodes. +```java +CacheServiceInterface, String> customCache = + CacheServiceFactory.createCustomCache(new ConcurrentHashMap<>()); -### MongoDB +// Access the underlying cache implementation +Map cache = customCache.getCache(); +``` -- **Document-based Storage** -- Efficient reads/writes for dynamic player data fields. -- Durable final persistence layer if caches miss. +### 📚 Multi-Level Cache ---- +The `FlexibleMultiLevelCacheService` provides sophisticated management of multiple cache levels. -## Data Flow - -Below is a simplified, high-level view of how player data moves between caches and the database: - -1. **Player Data Retrieval** - 1. **L1 Cache Check** - - If the player is **online** and data is in L1, return immediately (fastest path). - - If **not found**, proceed to L2. - 2. **L2 Cache (Redis) Check** - - If found, populate L1 for future quick lookups and return to caller. - - If still **not found**, query MongoDB. - 3. **Database (MongoDB) Fetch** - - On success, load into L1. - - If the player has never been seen before, create a new record and store it in L1. - -2. **Player Logout & Data Sync** - 1. **Gather All `LegacyPlayerDataService` Instances** - - Each has its own L1 + L2 caches. - 2. **Compare & Sync L1 → L2** - - Any unsaved changes in L1 are written to L2. - - Allows later scheduled tasks to persist them into MongoDB. - 3. **Remove Player Data from L1** - - Frees up memory, avoids stale data if the player stays offline for a long time. - -3. **Data Synchronization & Persistence** - 1. **L1 → L2**: - - `L1ToL2PlayerDataSyncTask` iterates all L1 entries, pushing to Redis if differences are found. - - Runs periodically or on certain triggers (like logout). - 2. **L2 → MongoDB**: - - `PlayerDataPersistenceTask` captures all data in Redis, writes it to MongoDB in a batched or scheduled manner. - - Ensures final, durable storage of changes. - -4. **Player Data Update (Cross-Server)** - 1. **Publish to Redis Streams** - - Using methods like `pubRStreamTask`, an update command is added to the stream (`RStreamTask`). - - Example: `PlayerDataUpdateByNameRStreamAccepter` or `PlayerDataUpdateByUuidRStreamAccepter` consumes the stream and applies updates. - 2. **Other Servers Receive & Update** - - If the player is online on another server, that server’s L1 cache is updated in real-time. - - This ensures data consistency across the network. +```java +// Create primary cache (memory) +CacheServiceInterface, String> caffeineCache = + CacheServiceFactory.createCaffeineCache(); ---- +// Initialize multi-level cache service +FlexibleMultiLevelCacheService flexibleMultiLevelCacheService = + CacheServiceFactory.createFlexibleMultiLevelCacheService(Set.of( + TieredCacheLevel.of(1, caffeineCache.getCache()) + )); +``` -## Performance Highlights +## 🤝 Contributing -- **High Throughput** - - **Caffeine L1** eliminates network round-trips for frequent reads/writes. - - **Redis L2** allows horizontal scaling with distributed caching. +Contributions are welcome! Feel free to: -- **Reduced DB Load** - - Most lookups are fulfilled by L1/L2 caches, hitting MongoDB only on cache misses or scheduled persistence. - - Frees the database from constant read pressure. +- 🐛 Report bugs +- 💡 Suggest new features +- 📝 Improve documentation +- 🔧 Submit pull requests -- **Asynchronous Task Handling** - - Redis Streams + `RStreamAccepterInterface` classes (e.g., `L1ToL2PlayerDataSyncByUuidRStreamAccepter`) process updates asynchronously, preventing blocking or contention. - - Spreads out load across multiple servers. +## 📄 License -- **Document-Oriented Flexibility** - - Using MongoDB for final storage avoids rigid schema definitions and supports easy extension of player attributes. +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. --- -## Why Near Enterprise-Level? - -1. **Multilayer Caching**: - - A pattern seen in large-scale enterprise systems: local (memory) + distributed caching for maximum throughput and minimal latency. - -2. **Distributed Messaging**: - - **Redis Streams** act like a message queue/bus, commonly used in microservices or high-scale systems for cross-node sync and event broadcasting. - -3. **Concurrency Control**: - - Uses distributed locks (Redisson) and concurrency settings (`LockSettings`) to ensure safe writes under high concurrency. -4. **Annotation-Driven Architecture**: - - Similar to Spring Boot or other enterprise frameworks, you declare components (`@RStreamAccepterRegister`, `@TypeAdapterRegister`) and let reflection + scanning wire them up. - -5. **Scalable & Modular**: - - Multiple `LegacyPlayerDataService` can be spun up for different clusters or shards, each safely isolated. - - Add or remove services as your player base grows. - -While labeled for “Minecraft servers,” the structure and design choices (multi-tier cache, streaming, concurrency, flexible data layer) align well with many enterprise-level backend systems. - ---- - -## Summary - -Bringing together **Caffeine** (L1 Cache), **Redis** (L2 Cache + Streams), and **MongoDB** (persistence), this module delivers: -- **Ultrafast read/writes** for active players (Caffeine). -- **Cross-instance data consistency** (Redis). -- **Reliable final storage** (MongoDB). -- **Seamless distribution** of updates (Redis Streams). -- **Robust task scheduling** for synchronization and persistence. - -By instantiating a dedicated `LegacyPlayerDataService`, you can manage: -- **Player session data** in memory, -- **Multi-server sync** via Redis, -- **Long-term data** in MongoDB, - all while keeping overhead low and performance high. This design can scale from single-server scenarios to large distributed networks, ensuring minimal data conflicts and fast responses—essentially approaching an enterprise-grade architecture within the realm of Minecraft (or similar) server applications. - ---- +Made with ❤️ by [LegacyLands Team](https://github.com/LegacyLands) -## License -This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. \ No newline at end of file diff --git a/player/README.md b/player/README.md index be86906..725f8c7 100644 --- a/player/README.md +++ b/player/README.md @@ -1,117 +1,180 @@ # 🎮 Player Module   ![Version](https://img.shields.io/badge/version-1.0-blue) ![License](https://img.shields.io/badge/license-MIT-green) -> A high-performance distributed player data management system utilizing multi-tier caching for optimal data consistency and access efficiency. +> A **high-performance** and **scalable** player data management system. It leverages **L1 (Caffeine)** and **L2 (Redis)** caching, **MongoDB** persistence, and **Redis Streams** for near “enterprise-level” performance in distributed Minecraft or similar server environments. --- ## 📚 Table of Contents -- [Introduction](#-introduction) -- [Automation](#-automation) -- [Key Components](#-key-components) -- [Data Processes](#-data-processes) -- [Performance Optimizations](#️-performance-optimizations) -- [Summary](#-summary) -- [License](#-license) +- [Introduction](#introduction) +- [Automation](#automation) +- [Key Components](#key-components) +- [Data Flow](#data-flow) +- [Performance Highlights](#performance-highlights) +- [Why Near Enterprise-Level?](#why-near-enterprise-level) +- [Summary](#summary) +- [License](#license) --- -## 📝 Introduction +## Introduction -This module is designed for processing player data within a distributed environment, employing L1 (Caffeine) and L2 (Redis) caches to ensure data consistency and efficient access. +The **Player Module** is a core part of a distributed system aimed at managing player data with **high throughput** and **low latency**. By integrating: +- **L1 Cache (Caffeine):** In-memory, ultra-fast access for online players. +- **L2 Cache (Redis):** Central cache for multi-server environments, synchronized via Redis Streams. +- **MongoDB:** Document-based persistence for flexible schema and long-term data storage. + +This architecture ensures that both frequent reads/writes (online players) and less common operations (offline queries, global sync) are handled efficiently. --- -## 🤖 Automation +## Automation + +- **L1 ↔ L2 Automatic Sync** + - Whenever a player’s data is updated in L1, tasks like `L1ToL2PlayerDataSyncTask` push changes to L2. + - Periodically or on-demand, Redis Streams also trigger updates across different servers. -Automates the management and synchronization of player data across multiple layers, including caches and databases, ensuring seamless data flow and integrity. +- **Automated Persistence** + - The module uses timed tasks (e.g., `PlayerDataPersistenceTask`) to write L2 (Redis) data into MongoDB, ensuring eventual consistency without manual intervention. + +- **Annotation-Driven Registration** + - Classes annotated with `@RStreamAccepterRegister` (like `L1ToL2PlayerDataSyncByNameRStreamAccepter`) automatically handle Redis Streams. + - Type adapters annotated with `@TypeAdapterRegister` are auto-registered for custom serialization/deserialization (see `PairTypeAdapter`). --- -## 🔑 Key Components +## Key Components -### 🏗️ LegacyPlayerDataService +### LegacyPlayerDataService -- **Description**: A composite object that can be instantiated indefinitely, each instance having its own dedicated connections: - - **🔗 Redis Connection**: For L2 caching layer. - - **💾 MongoDB Connection**: For persistent storage. - - **🔄 Redis Streams Connection**: For inter-server communication. +A flexible “service object” encapsulating: +- **Redis Connection** (L2 Cache + Streams) +- **MongoDB Connection** (Persistence) +- **Caffeine Cache** (L1 Cache) -### 🗄️ Caching Layers +By creating multiple `LegacyPlayerDataService` instances, you can manage different database or Redis clusters independently. +> Internally, it uses multi-level caching: L1 for ultra-fast memory-based access, and L2 for cross-server data sharing. -- **L1 Cache (Caffeine)**: In-memory cache designed for online player data, offering ultra-fast data access. -- **L2 Cache (Redis)**: Secondary caching layer meant for persistent storage of player data, providing a centralized view. +### Caching Tiers ---- +1. **L1 (Caffeine)** + - **Fast Memory Access** + - Typically holds data for **online** or **recently active** players. + - Minimizes network overhead and database queries. -## 🔄 Data Processes +2. **L2 (Redis)** + - **Distributed, Centralized View** + - Suited for multi-server setups. + - Uses **Redis Streams** for broadcasting player data changes (e.g., join/quit, data update) across nodes. -### 📊 Player Data Query Process +### MongoDB -1. **Check L1 Cache**: - - **Note**: Optimized for online players; not queried for offline data. +- **Document-based Storage** +- Efficient reads/writes for dynamic player data fields. +- Durable final persistence layer if caches miss. -2. **Query L2 Cache (Redis)**: - - **Cache Hit**: Retrieve data, store in L1, return to requester. - - **Cache Miss**: Query MongoDB. +--- -3. **Query Database (MongoDB)**: - - **Success**: Retrieve, store in L1, return data. - - **No Data Found**: Create new record, store in L1. +## Data Flow + +Below is a simplified, high-level view of how player data moves between caches and the database: + +1. **Player Data Retrieval** + 1. **L1 Cache Check** + - If the player is **online** and data is in L1, return immediately (fastest path). + - If **not found**, proceed to L2. + 2. **L2 Cache (Redis) Check** + - If found, populate L1 for future quick lookups and return to caller. + - If still **not found**, query MongoDB. + 3. **Database (MongoDB) Fetch** + - On success, load into L1. + - If the player has never been seen before, create a new record and store it in L1. + +2. **Player Logout & Data Sync** + 1. **Gather All `LegacyPlayerDataService` Instances** + - Each has its own L1 + L2 caches. + 2. **Compare & Sync L1 → L2** + - Any unsaved changes in L1 are written to L2. + - Allows later scheduled tasks to persist them into MongoDB. + 3. **Remove Player Data from L1** + - Frees up memory, avoids stale data if the player stays offline for a long time. + +3. **Data Synchronization & Persistence** + 1. **L1 → L2**: + - `L1ToL2PlayerDataSyncTask` iterates all L1 entries, pushing to Redis if differences are found. + - Runs periodically or on certain triggers (like logout). + 2. **L2 → MongoDB**: + - `PlayerDataPersistenceTask` captures all data in Redis, writes it to MongoDB in a batched or scheduled manner. + - Ensures final, durable storage of changes. + +4. **Player Data Update (Cross-Server)** + 1. **Publish to Redis Streams** + - Using methods like `pubRStreamTask`, an update command is added to the stream (`RStreamTask`). + - Example: `PlayerDataUpdateByNameRStreamAccepter` or `PlayerDataUpdateByUuidRStreamAccepter` consumes the stream and applies updates. + 2. **Other Servers Receive & Update** + - If the player is online on another server, that server’s L1 cache is updated in real-time. + - This ensures data consistency across the network. -### 🚪 Player Logout Process +--- -1. **Retrieve All Instances**: Gather all instances of `LegacyPlayerDataService`. +## Performance Highlights -2. **Compare L1 and L2 Cache**: - - **Consistent**: No changes needed. - - **Inconsistent**: Update L2 with L1 data. +- **High Throughput** + - **Caffeine L1** eliminates network round-trips for frequent reads/writes. + - **Redis L2** allows horizontal scaling with distributed caching. -3. **Remove Player Data**: Delete from L1 to maintain integrity. +- **Reduced DB Load** + - Most lookups are fulfilled by L1/L2 caches, hitting MongoDB only on cache misses or scheduled persistence. + - Frees the database from constant read pressure. -### 🔄 Data Synchronization +- **Asynchronous Task Handling** + - Redis Streams + `RStreamAccepterInterface` classes (e.g., `L1ToL2PlayerDataSyncByUuidRStreamAccepter`) process updates asynchronously, preventing blocking or contention. + - Spreads out load across multiple servers. -1. **Synchronize L1 to L2**: Transfer all data from L1 to L2. -2. **Persist L2 Data to Database**: Save L2 data into MongoDB. +- **Document-Oriented Flexibility** + - Using MongoDB for final storage avoids rigid schema definitions and supports easy extension of player attributes. -### ✏️ Player Data Modification Process +--- -1. **Publish Update Tasks Using Redis Stream**: - - Ensure distributed synchronization. - - Update L2 cache directly. - - **L2 Cache Miss**: Update database. +## Why Near Enterprise-Level? -2. **Process on Each Server**: - - **Online Player**: Sync updated data from L2 to L1. +1. **Multilayer Caching**: + - A pattern seen in large-scale enterprise systems: local (memory) + distributed caching for maximum throughput and minimal latency. -### 📥 Player Data Acquisition Process +2. **Distributed Messaging**: + - **Redis Streams** act like a message queue/bus, commonly used in microservices or high-scale systems for cross-node sync and event broadcasting. -1. **L1 L2 Data Synchronization**: Use Redis Stream for synchronization. -2. **Search**: Directly in L2; if miss, search database. +3. **Concurrency Control**: + - Uses distributed locks (Redisson) and concurrency settings (`LockSettings`) to ensure safe writes under high concurrency. ---- +4. **Annotation-Driven Architecture**: + - Similar to Spring Boot or other enterprise frameworks, you declare components (`@RStreamAccepterRegister`, `@TypeAdapterRegister`) and let reflection + scanning wire them up. -## ⚙️ Performance Optimizations +5. **Scalable & Modular**: + - Multiple `LegacyPlayerDataService` can be spun up for different clusters or shards, each safely isolated. + - Add or remove services as your player base grows. -- **L1 Cache**: Indefinite retention for active players, rapid access. -- **L2 Cache**: Long-term storage with expiration policies. -- **Database**: Accessed mainly on cache misses, reducing load. +While labeled for “Minecraft servers,” the structure and design choices (multi-tier cache, streaming, concurrency, flexible data layer) align well with many enterprise-level backend systems. --- -## 📜 Summary +## Summary -This architecture utilizes a multi-tier caching strategy for efficient player data management: -- **L1 Cache (Caffeine)**: Speedy access for online players. -- **L2 Cache (Redis)**: Centralized caching of persistent data. -- **Database (MongoDB)**: Main storage for cache misses. -- **Redis Streams**: Real-time server synchronization. -- **LegacyPlayerDataService**: Dedicated connections for Redis, MongoDB, and Redis Streams. +Bringing together **Caffeine** (L1 Cache), **Redis** (L2 Cache + Streams), and **MongoDB** (persistence), this module delivers: +- **Ultrafast read/writes** for active players (Caffeine). +- **Cross-instance data consistency** (Redis). +- **Reliable final storage** (MongoDB). +- **Seamless distribution** of updates (Redis Streams). +- **Robust task scheduling** for synchronization and persistence. -In conclusion, this design maximizes data retrieval efficiency, ensures consistency across distributed servers, and optimizes overall performance, delivering a seamless experience for players. +By instantiating a dedicated `LegacyPlayerDataService`, you can manage: +- **Player session data** in memory, +- **Multi-server sync** via Redis, +- **Long-term data** in MongoDB, + all while keeping overhead low and performance high. This design can scale from single-server scenarios to large distributed networks, ensuring minimal data conflicts and fast responses—essentially approaching an enterprise-grade architecture within the realm of Minecraft (or similar) server applications. --- -## 📄 License +## License -This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. \ No newline at end of file