Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/edge' into edge
Browse files Browse the repository at this point in the history
  • Loading branch information
math-GH committed May 7, 2024
2 parents e5bf420 + fd7157e commit 99916e7
Show file tree
Hide file tree
Showing 20 changed files with 448 additions and 131 deletions.
77 changes: 62 additions & 15 deletions Docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Example for Linux Debian / Ubuntu:

```sh
# Install default Docker Compose and automatically the corresponding version of Docker
apt install docker-compose
apt install docker-compose-v2
```

## Quick run
Expand Down Expand Up @@ -194,6 +194,8 @@ docker run -d --restart unless-stopped --log-opt max-size=10m \

In the FreshRSS setup, you will then specify the name of the container (`freshrss-db`) as the host for the database.

See also the section [Docker Compose with PostgreSQL](#docker-compose-with-postgresql) below.

### [MySQL](https://hub.docker.com/_/mysql/) or [MariaDB](https://hub.docker.com/_/mariadb)

```sh
Expand Down Expand Up @@ -285,13 +287,13 @@ See [`docker-compose.yml`](./freshrss/docker-compose.yml)
```sh
cd ./FreshRSS/Docker/freshrss/
# Update
docker-compose pull
docker compose pull
# Run
docker-compose -f docker-compose.yml -f docker-compose-local.yml up -d --remove-orphans
docker compose -f docker-compose.yml -f docker-compose-local.yml up -d --remove-orphans
# Logs
docker-compose logs -f --timestamps
docker compose logs -f --timestamps
# Stop
docker-compose down --remove-orphans
docker compose down --remove-orphans
```

Detailed (partial) example of Docker Compose for FreshRSS:
Expand Down Expand Up @@ -378,13 +380,15 @@ See [`docker-compose-db.yml`](./freshrss/docker-compose-db.yml)
```sh
cd ./FreshRSS/Docker/freshrss/
# Update
docker-compose -f docker-compose.yml -f docker-compose-db.yml pull
docker compose -f docker-compose.yml -f docker-compose-db.yml pull
# Run
docker-compose -f docker-compose.yml -f docker-compose-db.yml -f docker-compose-local.yml up -d --remove-orphans
docker compose -f docker-compose.yml -f docker-compose-db.yml -f docker-compose-local.yml up -d --remove-orphans
# Logs
docker-compose -f docker-compose.yml -f docker-compose-db.yml logs -f --timestamps
docker compose -f docker-compose.yml -f docker-compose-db.yml logs -f --timestamps
```

See also the section [Migrate database](#migrate-database) below to upgrade to a major PostgreSQL version with Docker Compose.

### Docker Compose for development

Use the local (git) FreshRSS source code instead of the one inside the Docker container,
Expand All @@ -396,11 +400,11 @@ See [`docker-compose-development.yml`](./freshrss/docker-compose-development.yml
cd ./FreshRSS/Docker/freshrss/
# Update
git pull --ff-only --prune
docker-compose pull
docker compose pull
# Run
docker-compose -f docker-compose-development.yml -f docker-compose.yml -f docker-compose-local.yml up --remove-orphans
docker compose -f docker-compose-development.yml -f docker-compose.yml -f docker-compose-local.yml up --remove-orphans
# Stop with [Control]+[C] and purge
docker-compose down --remove-orphans --volumes
docker compose down --remove-orphans --volumes
```

> ℹ️ You can combine it with `-f docker-compose-db.yml` to spin a PostgreSQL database.
Expand Down Expand Up @@ -446,13 +450,13 @@ See [`docker-compose-proxy.yml`](./freshrss/docker-compose-proxy.yml)
```sh
cd ./FreshRSS/Docker/freshrss/
# Update
docker-compose -f docker-compose.yml -f docker-compose-proxy.yml pull
docker compose -f docker-compose.yml -f docker-compose-proxy.yml pull
# Run
docker-compose -f docker-compose.yml -f docker-compose-proxy.yml up -d --remove-orphans
docker compose -f docker-compose.yml -f docker-compose-proxy.yml up -d --remove-orphans
# Logs
docker-compose -f docker-compose.yml -f docker-compose-proxy.yml logs -f --timestamps
docker compose -f docker-compose.yml -f docker-compose-proxy.yml logs -f --timestamps
# Stop
docker-compose -f docker-compose.yml -f docker-compose-proxy.yml down --remove-orphans
docker compose -f docker-compose.yml -f docker-compose-proxy.yml down --remove-orphans
```

> ℹ️ You can combine it with `-f docker-compose-db.yml` to spin a PostgreSQL database.
Expand Down Expand Up @@ -650,3 +654,46 @@ docker run -d --restart unless-stopped --log-opt max-size=10m \
--name freshrss_cron freshrss/freshrss:alpine \
crond -f -d 6
```

## Migrate database

Our [CLI](../cli/README.md) offers commands to back-up and migrate user databases,
with `cli/db-backup.php` and `cli/db-restore.php` in particular.

Here is an example (assuming our [Docker Compose example](#docker-compose-with-postgresql))
intended for migrating to a newer major version of PostgreSQL,
but which can also be used to migrate between other databases (e.g. MySQL to PostgreSQL).

```sh
# Stop FreshRSS container (Web server + cron) during maintenance
docker compose down freshrss

# Optional additional pre-upgrade back-up using PostgreSQL own mechanism
docker compose -f docker-compose-db.yml \
exec freshrss-db pg_dump -U freshrss freshrss | gzip -9 > freshrss-postgres-backup.sql.gz
# ------↑ Name of your PostgreSQL Docker container
# -----------------------------↑ Name of your PostgreSQL user for FreshRSS
# --------------------------------------↑ Name of your PostgreSQL database for FreshRSS

# Back-up all users’ respective tables to SQLite files
docker compose -f docker-compose.yml -f docker-compose-db.yml \
run --rm freshrss cli/db-backup.php

# Remove old database (PostgreSQL) container and its data volume
docker compose -f docker-compose-db.yml \
down --volumes freshrss-db

# Edit your Compose file to use new database (e.g. newest postgres:xx)
nano docker-compose-db.yml

# Start new database (PostgreSQL) container and its new empty data volume
docker compose -f docker-compose.yml -f docker-compose-db.yml \
up -d freshrss-db

# Restore all users’ respective tables from SQLite files
docker compose -f docker-compose.yml -f docker-compose-db.yml \
run --rm freshrss cli/db-restore.php --delete-backup

# Restart a new FreshRSS container after maintenance
docker compose -f docker-compose.yml -f docker-compose-db.yml up -d freshrss
```
2 changes: 1 addition & 1 deletion Docker/freshrss/docker-compose-proxy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ volumes:
services:

traefik:
image: traefik:2.11
image: traefik:3.0
container_name: traefik
restart: unless-stopped
logging:
Expand Down
9 changes: 5 additions & 4 deletions app/Controllers/feedController.php
Original file line number Diff line number Diff line change
Expand Up @@ -455,8 +455,8 @@ public static function actualizeFeeds(?int $feed_id = null, ?string $feed_url =
continue; //When PubSubHubbub is used, do not pull refresh so often
}

if ($feed->mute()) {
continue; //Feed refresh is disabled
if ($feed->mute() && $feed_id === null) {
continue; // If the feed is disabled, only allow refresh if manually requested for that specific feed
}
$mtime = $feed->cacheModifiedTime() ?: 0;
$ttl = $feed->ttl();
Expand Down Expand Up @@ -1121,7 +1121,7 @@ public function contentSelectorPreviewAction(): void {
$feed_id = Minz_Request::paramInt('id');
$content_selector = Minz_Request::paramString('selector');

if (!$content_selector) {
if ($content_selector === '') {
$this->view->fatalError = _t('feedback.sub.feed.selector_preview.selector_empty');
return;
}
Expand All @@ -1143,11 +1143,12 @@ public function contentSelectorPreviewAction(): void {

//Get feed.
$feed = $entry->feed();

if ($feed === null) {
$this->view->fatalError = _t('feedback.sub.feed.selector_preview.no_feed');
return;
}
$feed->_pathEntries($content_selector);
$feed->_attribute('path_entries_filter', Minz_Request::paramString('selector_filter', true));

//Fetch & select content.
try {
Expand Down
22 changes: 19 additions & 3 deletions app/Models/DatabaseDAO.php
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,18 @@ public function testConnection(): string {
}
}

public function exits(): bool {
$sql = 'SELECT * FROM `_entry` LIMIT 1';
$stm = $this->pdo->query($sql);
if ($stm !== false) {
$res = $stm->fetchAll(PDO::FETCH_COLUMN, 0);
if ($res !== false) {
return true;
}
}
return false;
}

public function tablesAreCorrect(): bool {
$res = $this->fetchAssoc('SHOW TABLES');
if ($res == null) {
Expand Down Expand Up @@ -242,6 +254,7 @@ public function dbCopy(string $filename, int $mode, bool $clearFirst = false): b
}
$error = '';

$databaseDAO = FreshRSS_Factory::createDatabaseDAO();
$userDAO = FreshRSS_Factory::createUserDao();
$catDAO = FreshRSS_Factory::createCategoryDao();
$feedDAO = FreshRSS_Factory::createFeedDao();
Expand All @@ -259,15 +272,18 @@ public function dbCopy(string $filename, int $mode, bool $clearFirst = false): b
$error = 'Error: SQLite import file is not readable: ' . $filename;
} elseif ($clearFirst) {
$userDAO->deleteUser();
$userDAO = FreshRSS_Factory::createUserDao();
if ($this->pdo->dbType() === 'sqlite') {
//We cannot just delete the .sqlite file otherwise PDO gets buggy.
//SQLite is the only one with database-level optimization, instead of at table level.
$this->optimize();
}
} else {
$nbEntries = $entryDAO->countUnreadRead();
if (!empty($nbEntries['all'])) {
$error = 'Error: Destination database already contains some entries!';
if ($databaseDAO->exits()) {
$nbEntries = $entryDAO->countUnreadRead();
if (isset($nbEntries['all']) && $nbEntries['all'] > 0) {
$error = 'Error: Destination database already contains some entries!';
}
}
}
break;
Expand Down
9 changes: 6 additions & 3 deletions app/Models/Entry.php
Original file line number Diff line number Diff line change
Expand Up @@ -748,11 +748,14 @@ public function getContentByParsing(string $url = '', int $maxRedirs = 3): strin
}

$content = '';
$nodes = $xpath->query((new Gt\CssXPath\Translator($feed->pathEntries()))->asXPath());
$cssSelector = htmlspecialchars_decode($feed->pathEntries(), ENT_QUOTES);
$cssSelector = trim($cssSelector, ', ');
$nodes = $xpath->query((new Gt\CssXPath\Translator($cssSelector))->asXPath());
if ($nodes != false) {
$path_entries_filter = $feed->attributeString('path_entries_filter');
$path_entries_filter = $feed->attributeString('path_entries_filter') ?? '';
$path_entries_filter = trim($path_entries_filter, ', ');
foreach ($nodes as $node) {
if ($path_entries_filter != null) {
if ($path_entries_filter !== '') {
$filterednodes = $xpath->query((new Gt\CssXPath\Translator($path_entries_filter))->asXPath(), $node) ?: [];
foreach ($filterednodes as $filterednode) {
if ($filterednode->parentNode === null) {
Expand Down
22 changes: 13 additions & 9 deletions app/Models/UserQuery.php
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ public function __construct(array $query, array $categories, array $labels) {
$this->labels = $labels;
if (isset($query['get'])) {
$this->parseGet($query['get']);
} else {
$this->get_type = 'all';
}
if (isset($query['name'])) {
$this->name = trim($query['name']);
Expand Down Expand Up @@ -107,7 +109,9 @@ public function toArray(): array {
*/
private function parseGet(string $get): void {
$this->get = $get;
if (preg_match('/(?P<type>[acfistT])(_(?P<id>\d+))?/', $get, $matches)) {
if ($this->get === '') {
$this->get_type = 'all';
} elseif (preg_match('/(?P<type>[acfistT])(_(?P<id>\d+))?/', $get, $matches)) {
$id = intval($matches['id'] ?? '0');
switch ($matches['type']) {
case 'a':
Expand Down Expand Up @@ -155,22 +159,22 @@ public function isDeprecated(): bool {

/**
* Check if the user query has parameters.
* If the type is 'all', it is considered equal to no parameters
*/
public function hasParameters(): bool {
if ($this->get_type === 'all') {
return false;
}
if ($this->hasSearch()) {
if ($this->get_type !== 'all') {
return true;
}
if ($this->state) {
if ($this->hasSearch()) {
return true;
}
if ($this->order) {
if (!in_array($this->state, [
0,
FreshRSS_Entry::STATE_READ | FreshRSS_Entry::STATE_NOT_READ,
FreshRSS_Entry::STATE_READ | FreshRSS_Entry::STATE_NOT_READ | FreshRSS_Entry::STATE_FAVORITE | FreshRSS_Entry::STATE_NOT_FAVORITE
], true)) {
return true;
}
if ($this->get) {
if ($this->order !== '' && $this->order !== FreshRSS_Context::userConf()->sort_order) {
return true;
}
return false;
Expand Down
2 changes: 1 addition & 1 deletion app/Services/ExportService.php
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ public function zip(array $files): array {
$zip_filename = 'freshrss_' . $this->username . '_' . $day . '_export.zip';

// From https://stackoverflow.com/questions/1061710/php-zip-files-on-the-fly
$zip_file = tempnam('/tmp', 'zip');
$zip_file = tempnam(TMP_PATH, 'zip');
if ($zip_file == false) {
return [$zip_filename, false];
}
Expand Down
2 changes: 1 addition & 1 deletion app/layout/aside_feed.phtml
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@
?><div class="dropdown no-mobile">
<div class="dropdown-target"></div><a class="dropdown-toggle" data-fweb="<?= $feed->website() ?>"><?= _i('configure') ?></a><?php /* feed_config_template */ ?>
</div><?php
if (FreshRSS_Context::userConf()->show_favicons) { ?><img class="favicon test" src="<?= $feed->favicon() ?>" alt="✇" loading="lazy" /><?php }
if (FreshRSS_Context::userConf()->show_favicons) { ?><img class="favicon" src="<?= $feed->favicon() ?>" alt="✇" loading="lazy" /><?php }
endif;
?><a class="item-title" data-unread="<?= format_number($feed->nbNotRead()) ?>" href="<?=
_url('index', $actual_view, 'get', 'f_' . $feed->id()) . $state_filter_manual ?>"><?= $feed->name() ?></a></li>
Expand Down
8 changes: 8 additions & 0 deletions cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,14 @@ cd /usr/share/FreshRSS
```sh
cd /usr/share/FreshRSS

./cli/db-backup.php
# Back-up all users respective database to `data/users/*/backup.sqlite`

./cli/db-restore.php --delete-backup --force-overwrite
# Restore all users respective database from `data/users/*/backup.sqlite`
# --delete-backup: delete `data/users/*/backup.sqlite` after successful import
# --force-overwrite: will clear the users respective database before import

./cli/db-optimize.php --user username
# Optimize database (reduces the size) for a given user (perform `OPTIMIZE TABLE` in MySQL, `VACUUM` in SQLite)
```
Expand Down
20 changes: 20 additions & 0 deletions cli/db-backup.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env php
<?php
declare(strict_types=1);
require(__DIR__ . '/_cli.php');

performRequirementCheck(FreshRSS_Context::systemConf()->db['type'] ?? '');
$ok = true;

foreach (listUsers() as $username) {
$username = cliInitUser($username);
$filename = DATA_PATH . '/users/' . $username . '/backup.sqlite';
@unlink($filename);

echo 'FreshRSS backup database to SQLite for user “', $username, "”…\n";

$databaseDAO = FreshRSS_Factory::createDatabaseDAO($username);
$ok &= $databaseDAO->dbCopy($filename, FreshRSS_DatabaseDAO::SQLITE_EXPORT);
}

done((bool)$ok);
Loading

0 comments on commit 99916e7

Please sign in to comment.