where/how to run recurring tasks that today are cronjobs?

[evgeni]

SSIA

[ehelms]

In theory they can be run as cron jobs still but executing within the container context instead of as a script. We should compile a list of the cron jobs.

[ekohl]

I'd like to convert things to within the application if possible and systemd timers otherwise. The benefit of the former is that it'll ease the path to HA deployments. https://wiki.archlinux.org/title/Systemd/Timers#As_a_cron_replacement has a nice overview of reasons for the latter (and also downsides).

[archanaserver]

I've started exploring this, and for the prototype i added a quadlet-based systemd timer for the reports:daily rake task, running inside the foreman container. this is def not giving true HA behaviour yet in this case, but for now, though the systemd timer route feels like a good middle ground to validate the concept and handle simple maintenance tasks (db:sessions:clear, reports:expire, etc.). #252

so thinking ahead i came across couple of direction we could explore longer-term solution:

• one(as @ekohl mentioned) is moving recurring tasks into the foreman application layer itself, i find out we can do that with sidekiq-cron. that would make it more HA-friendly and easily visible in the logs, though it's require code changes in core foreman, so i'm not sure about that right now.
• thinking more in-terms of HA-safe way, another could be having a dedicated lightweight job runner container(sort of like a foreman-core service) which basically going to loops and executes rake tasks periodically.

also while reviewing foreman.cron.d, i noticed not all cron jobs need to be replicated in foremanctl. for example, ldap:refresh_usergroups feels more like something that should eventually live inside the app itself rather than as an external container execution way.

[rriemann]

A few ideas that come to my mind:

1. extra points if whatever the machanism is, it can be efficiently translated to kubernetes. Running an own container just for some jobs every day is maybe overkill. See also this discussion: use Kubernetes CronJob instead of sidekar with cronjob daemon nextcloud/helm#683
2. generally, Nextcloud also has regular jobs and one could learn from their approach: https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/background_jobs_configuration.html

This looks like this in docker compose:

services:
  server: &server-template
    image: docker.io/library/nextcloud:$NEXTCLOUD_VERSION
    restart: always

  cron:
    <<: *server-template
    user: 33:33
    ports: []
    command: /cron.sh
    healthcheck:
      disable: true

Mastodon is in a similar situation:

services:
  mastodon:
    container_name: "mastodon-web"
    image: ghcr.io/mastodon/mastodon:$MASTODON_VERSION
    restart: always
    env_file: $ENVIRONMENT_FILE
    environment:
      LOCAL_DOMAIN: $MASTODON_DOMAIN
    command: bundle exec puma -C config/puma.rb
  sidekiq:
    container_name: "mastodon-sidekiq"
    # build: .
    image: ghcr.io/mastodon/mastodon:$MASTODON_VERSION
    restart: always
    env_file: $ENVIRONMENT_FILE
    environment:
      LOCAL_DOMAIN: $MASTODON_DOMAIN
    command: bundle exec sidekiq

[vkrizan]

In the context of a system, systemd timers might be utilized.

[ekohl]

We already have recurring logic in Foreman. A challenge is that it's within foreman-tasks but I think we can find a solution to that.

[archanaserver]

We already have recurring logic in Foreman. A challenge is that it's within foreman-tasks but I think we can find a solution to that.

@ekohl while exploring this, I looked into the existing recurring scheduling logic in foreman-tasks. Foreman already supports recurring Actions through ForemanTasks::RecurringLogic, but most of the maintenance rake tasks we run via cron (db:sessions:clear, reports:expire, etc.) I see it don’t yet have equivalent Actions.
I believe for now, this quadlet + systemd timer approach provides a lightweight and container-friendly way to schedule these tasks, while keeping the door open for future integration with Foreman’s internal recurring system for HA environments.